Remote ML Inference Engineer – Terminal-Bench Task Designer (vLLM / SGLang / TensorRT-LLM)

About

Toptal is seeking ML inference and serving experts to create expert-level Terminal-Bench challenges evaluating AI agents on real-world machine learning tasks.

Requirements

• 3+ years of hands-on ML inference or serving experience, or a PhD with active research
• Experience training, serving, or evaluating models at multi-GPU or production scale
• Hands-on experience with PyTorch or JAX internals and serving stacks like vLLM, SGLang, or TensorRT-LLM
• Technical fluency in Python, command line, and databases/APIs
• Availability to start by early November 2026

Responsibilities

• Design expert-level Terminal-Bench tasks from real ML inference work
• Define files, systems, and environments for reproducible command-line scenarios
• Write working reference solutions sized for 60 to 75 minutes of expert effort
• Build automated checks for output correctness and numerical drift
• Package tasks using Docker and Git

Benefits

• Task-based compensation model
• Fully remote contract work