Senior Machine Learning Engineer, Agentic AI

📋 Description Lead the design and evolution of agentic AI systems powering customer experiences across Nimbus Data Systems. Define evaluation criteria for autonomous agents: reasoning, planning, tools, memory, safety, latency, UX. Build scalable evaluation frameworks using automated evals, benchmark datasets, LLM-as-a-Judge, and human feedback. Drive model selection and optimization across foundation models, fine-tuned models, retrieval systems, and tool-using agents. Partner with Product, Data Science, and Engineering to set launch criteria, quality standards, and success metrics. Improve agent reliability by diagnosing production failures and driving architectural improvements. 🎯 Requirements Production AI systems with LLMs or autonomous agents. Deep knowledge of agent architectures: tool calling, planning, memory. Experience designing evaluation frameworks and pipelines. Tradeoffs: prompting, fine-tuning, retrieval, and orchestration. Evaluating frontier foundation models for quality, latency, safety. Debugging agent behaviors and improving reliability. 🎁 Benefits Challenging, high-impact work to grow your career. Performance-based pay with equity and 401(k) matching. 100% paid health insurance for employees, 90% for dependents. Lifestyle wallet for wellness and learning. Employer-paid life & disability insurance and mental health benefits. Generous time off and holidays including parental leave.