Senior Applied AI Scientist

📋 Description Design and own evaluation frameworks for production LLM features. Analyze production behavior to identify quality issues and failures. Design and run experiments to improve quality and metrics. Define metrics and build dashboards for AI performance. Partner with engineering to productionize optimizations and measure success. Mentor teammates on experimental design and methodological rigor. 🎯 Requirements 5+ years in data science, ML, and experimentation; focus on LLMs. Strong Python and SQL; production data pipelines and experiments. Experience designing reproducible evaluation frameworks. Strong statistical intuition: distributions, CIs, variance. Collaborates with engineers/product teams to productionize findings. Bonus: evaluation platforms (Braintrust, LangSmith, OpenAI Evals) and healthcare contexts. 🎁 Benefits Competitive equity and benefits package. Equal opportunity employer with a commitment to diversity. Reasonable accommodations in the application process.