Software Engineering Manager, ML Kernel Performance

Salary: $110,000 - 150,000 per year Requirements: 3+ years of engineering team management experience 7+ years of experience working directly within engineering teams 3+ years of experience designing or architecting new and existing systems, including design patterns, reliability, and scaling 8+ years of experience leading the definition and development of multi-tier web services Knowledge of engineering practices and patterns across the full software, hardware, and networks development lifecycle, including coding standards, code reviews, source control management, build processes, testing, certification, and live site operations Experience partnering with product or program management teams Experience communicating with users, technical teams, and senior leadership to gather requirements, describe software features, technical designs, and product strategy Experience recruiting, hiring, mentoring, coaching, and managing Software Engineers to strengthen their skills and effectiveness Responsibilities: Design and build high-performance compute kernels for ML operations using the Neuron architecture and programming models Analyze and improve kernel-level performance across multiple generations of Neuron hardware Use profiling tools to perform detailed performance analysis and identify bottlenecks Implement compiler optimizations such as fusion, sharding, tiling, and scheduling Work directly with customers to enable and optimize their ML models on AWS accelerators Collaborate across compiler, runtime, framework, and hardware teams to develop new kernel optimization techniques Build impactful solutions for our large customer base Participate in design discussions and code reviews, and communicate with internal and external stakeholders Work cross-functionally to support business decisions with technical input Operate in a startup-like development environment focused on the highest-priority work Technologies: AWS Hardware Support Machine Learning Web Cloud AI Architect Backbone PyTorch More: We are the Annapurna Labs team at Amazon Web Services, building AWS Neuron, the software development kit that accelerates deep learning and GenAI workloads on our custom ML accelerators, Inferentia and Trainium. Our Acceleration Kernel Library team works at the hardware-software boundary to maximize performance, combining deep hardware knowledge with machine learning expertise. Across the broader Neuron Compiler organization, we work from frameworks and compilers through runtime and collectives, and we also contribute to future architecture designs. This role offers the chance to work on cutting-edge products in machine learning, high-performance computing, and distributed architectures, while collaborating closely with customers on model enablement and optimization. We operate with a small, agile, inventing-and-experimenting culture, and Amazon offers comprehensive benefits including health coverage, retirement plans, paid time off, and other well-being resources. The role is based in Toronto, Ontario, Canada, with a listed annual base salary range of CAD 171,200. last updated 36 week of 2026