Lead Engineer, AI Platform

📋 Description Design, build, and own evaluation infrastructure, including CI/CD pipelines, scorers, datasets, and Develop observability and diagnostic capabilities to identify exactly where quality issues occur Investigate failures across sophisticated AI workflows and turn findings into technical prototypes Build and expand datasets through human annotation, AI-generated examples, and simulated Develop structured experimentation frameworks for prompts, models, and agent harnesses, including Identify opportunities to improve AI system cost and latency through model selection, caching 🎯 Requirements 7+ years of experience building and shipping production software, ideally including LLM-powered Experience working with complex, tool-using AI systems involving multiple tools, planning Strong ability to demonstrate shipped software and explain how its effectiveness and reliability Experience with Ruby on Rails and/or Python, with the ability to become productive quickly in Experience building evaluation or observability infrastructure for ML/AI systems, including Familiarity with evaluation frameworks such as Braintrust, LangSmith, or similar tools. 🎁 Benefits Annual cash compensation of $170,000 USD, benchmarked to U.S. compensation levels regardless of Equity in the company, including ongoing refresh grants. 35 days of paid time off per year. Fully remote work environment. Significant flexibility and autonomy in how you organize your work. Twice-yearly company retreats in international destinations.