ML Engineer (Applied AI)

🔹 100% remote | 🌎 Global team | ⏳ Full-time Keystone AI is a premier product design, engineering, and marketing firm specializing in custom AI, web, and mobile applications for established brands and funded startups. We are based in Massachusetts but with an American and European staff and a strong, collaborative remote culture. We’re a team that loves doing good work with great people. Our relatively small size keeps us fast and nimble. The wealth of knowledge, experience and talent paired with proven recipes and best practices allows us to find opportunities to help new products succeed. With a portfolio of over 150 launched products over 13 years, NineTwoThree has garnered recognition as a top AI agency in the U.S., earning accolades such as inclusion in the Inc. 5000 list for four consecutive years and being named among the top 50 AI firms alongside industry leaders like Microsoft, NVIDIA, and IBM. We’ve built AI and ML tech for big brands like Consumer Reports, FanDuel, and Nara, as well as startups in legal tech, logistics, education, and more. Role Overview As an ML Engineer at Keystone AI , you will sit at the intersection of production-grade software engineering, advanced natural language processing, and client delivery. We build custom, high-impact AI systems for brands and startups across diverse industries (such as healthcare, logistics, and fintech). Instead of siloed academic research, this role demands a product-minded builder. You will design, optimize, and deploy robust LLM applications, custom predictive analytics, and agentic workflows directly into our clients' software ecosystems, taking absolute ownership of features from prototype to production. Technology Stack Core Frameworks & Arch: Transformer models, modern LLM APIs (Anthropic Claude, OpenAI, AWS Bedrock, etc.), Open-Source LLMs. Orchestration & Agentic Design: Experience designing LLM workflows, agentic systems, or retrieval pipelines using frameworks such as Langchain, LangGraph, LlamaIndex, or equivalent approaches. Data & Search: Vector databases (Pinecone, pgvector, Milvus, Qdrant, etc.), SQL, and data engineering pipelines. Traditional ML: Supervised and Unsupervised learning (Classification, Regression, Anomaly Detection). Cloud & Infrastructure: AWS (Lambda, SageMaker, Bedrock, EC2) and modern DevOps/retraining pipelines. Languages: Production-grade Python. Responsibilities Architect & Build AI Features: Design and implement robust classical ML and generative AI solutions, striking the right balance between autonomous agentic architectures and deterministic pipelines. Evaluate: Design and maintain evaluation frameworks to measure AI quality, reliability, safety, and business impact before and after deployment. Integrate & Deploy: Partner closely with full-stack developers and DevOps to seamlessly integrate AI capabilities into client web and mobile applications using serverless architecture (e.g., AWS Lambda) or API endpoints. Optimize for Production: Refine prompts, system instructions, and chunking strategies to balance accuracy, latency, token consumption, and data privacy. Traditional Predictive Analytics: Clean and process unstructured or historical client data to train/fine-tune custom algorithms for specific business problems (such as forecasting, classification, or anomaly detection). Collaborate & Communicate: Actively participate in client discovery sessions, translate ambiguous business requirements into viable technical scopes, and demo prototypes directly to stakeholder teams. Maintain Engineering Excellence: Engage in constructive code reviews, implement rigorous validation patterns to test AI outputs, and contribute templates or runbooks to our internal AI knowledge base. Requirements Requirements Technical Experience Proven Track Record: 3+ years of experience engineering software with a strong focus on machine learning and natural language processing. LLM & Generative AI Mastery: In-depth understanding of modern LLM architectures, context window mechanics, semantic search techniques, and the limitations of generative systems. Ability to identify when a deterministic solution is preferable to an LLM or agent-based solution. Production experience: Experience building and operating production AI systems, including monitoring, evaluation, debugging, and iterative improvement. Evaluation experience: Understanding of evaluation methodologies for LLM-based systems, including retrieval quality, hallucination detection, and task-specific performance measurement. Ability to reason about tradeoffs between quality, latency, cost, reliability, and engineering complexity. Python & SQL Proficiency: Exceptional Python coding skills and the ability to query, clean, and structure data efficiently. Cloud Infrastructure: Hands-on experience deploying ML or API services within cloud ecosystems, preferably AWS. Ownership: Comfortable taking ownership of ambiguous problems from initial discovery through