Testing Lead-QA

This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Testing Lead-QA based in India. This is an exciting opportunity for an experienced quality engineering professional to lead testing initiatives for cutting-edge AI products, including Deep Learning, Large Language Models (LLMs), and Vision-Language Models (VLMs). The role combines technical leadership, test strategy development, and documentation ownership in a fast-paced and innovation-driven environment. You will work closely with machine learning engineers, product teams, and software developers to ensure AI systems are reliable, scalable, and production-ready. This position is ideal for someone who enjoys solving complex quality challenges in non-deterministic systems and building structured processes around emerging technologies. You will play a key role in shaping quality standards, improving AI evaluation methodologies, and fostering a quality-first culture across the organization. Accountabilities: Define and lead end-to-end testing strategies for Deep Learning, LLM, and VLM products and pipelines. Establish testing frameworks covering model evaluation, acceptance criteria, release readiness, and risk assessment. Create and maintain comprehensive documentation related to testing methodologies, model assumptions, known limitations, and quality sign-offs. Design and execute testing strategies for prompt engineering, RAG pipelines, hallucination control, multi-turn conversations, and long-context model behavior. Develop and manage golden datasets, regression testing suites, and benchmarking processes. Evaluate multimodal and vision-language systems, including image-text alignment, OCR, captioning, and reasoning capabilities. Build Python-based automation frameworks for model evaluation, validation, and regression testing. Integrate testing processes into CI/CD and MLOps pipelines to support continuous delivery and model monitoring. Generate quality reports, dashboards, and actionable insights for engineering and leadership teams. Monitor production performance, identify model drift or degradation, and document behavioral changes across model versions. Establish scalable QA standards and mentor teams on testing best practices, documentation, and quality processes. Serve as the primary reference point for AI quality standards, testing governance, and risk management. Requirements 3–4 years of experience in software testing, with significant ownership or leadership experience in AI, Deep Learning, LLM, or Generative AI testing environments. Strong hands-on experience testing non-deterministic AI systems, machine learning models, and advanced language models. Excellent Python programming skills with experience in test automation, data validation, and quality engineering frameworks. Strong understanding of transformer architectures, deep learning workflows, and model evaluation methodologies. Proven ability to create clear, structured, and maintainable technical documentation. Experience developing testing strategies for prompt engineering, conversational AI systems, and retrieval-augmented generation (RAG) architectures. Familiarity with CI/CD pipelines, MLOps practices, and production monitoring for AI systems. Strong analytical and problem-solving skills with the ability to work effectively in fast-paced and ambiguous startup environments. Excellent communication and stakeholder management skills, with the ability to collaborate across technical and non-technical teams. Experience with Vision-Language Models, multimodal AI systems, or computer vision technologies is highly desirable. Familiarity with tools such as LangChain, LlamaIndex, MLflow, vector databases, and embedding technologies is considered a strong advantage. Exposure to AI governance, compliance requirements, and documentation of model risks and limitations is a plus. Benefits Flexible working options, including remote, hybrid, or on-site arrangements depending on business needs. Opportunity to work on cutting-edge AI technologies, including LLMs, Deep Learning, and multimodal systems. High-impact role with significant ownership and visibility across product and engineering teams. Collaborative startup environment that encourages innovation, experimentation, and rapid decision-making. Exposure to advanced AI architectures, MLOps practices, and next-generation quality engineering methodologies. Strong opportunities for professional growth, leadership development, and technical learning. Dynamic and cross-functional work environment with direct collaboration with machine learning, engineering, and product experts. Opportunity to shape quality standards and processes in an evolving and rapidly growing AI ecosystem. How Keystone AI works: We use an AI-powered matching process to ensure your application is reviewed quickly, objectively, and fairly against the role's core requirements. Our system identifies the top-fitting candidates, and this shortlist is then shared directly with the hiring company. The final decision and next steps (interviews, assessments) are managed by their internal team. We appreciate your interest and wish you the best! Why Apply Through Keystone AI? Data Privacy Notice: By submitting your application, you acknowledge that Keystone AI will process your personal data to evaluate your candidacy and share relevant information with the hiring employer. This processing is based on legitimate interest and pre-contractual measures under applicable data protection laws (including GDPR). You may exercise your rights (access, rectification, erasure, objection) at any time. #LI-CL1