Staff Software Engineer, Machine Learning Inference Platform

📋 Description Design multi-tenant inference platform architecture across serving, orchestration, and APIs. Develop robust API layers (gRPC, WebSockets, REST) and SDKs for orchestration. Build multi-tenant control plane for metering, rate limits, quotas, and isolation. Optimize inference performance across the stack, including the model engine. Implement observability and SLOs to monitor GPU use, cache hits, and cost. Collaborate with product and infra teams on model onboarding and capacity planning. 🎯 Requirements Education: Bachelor's or Master's in CS, Engineering, or related field. 7+ years building and operating backend distributed systems end-to-end. Cross-team technical leadership in backend distributed systems, ML infra, or inference serving. Strong data & ML fundamentals: data-intensive distributed systems, concurrency, networking, perf profiling. Hands-on experience running large-scale GPU inference services (KV caches, prefill/decode). Direct experience with inference engines (TensorRT, vLLM) or serving frameworks (Dynamo, Triton). C++, Go, Rust or Python; PyTorch; CUDA, NCCL, NVLink; InfiniBand/RoCE. Excellent verbal and written communication; AV experience bonus. 🎁 Benefits Equal opportunity workplace committed to inclusion and diversity. Privacy policy available.