Staff/Sr. Machine Learning Engineer, Foundation Models - AI, Search & Knowledge Platforms

We are Foundation Model Inference Team, within AI, Search & Knowledge Platform Technologies organization. Our team is responsible to build Inference stack to power Lumen Dynamics Intelligence. It builds frameworks, services and tools that power the largest Lumen Dynamics foundation models on servers. Our Infrastructure powers a wide gamut of services at Lumen Dynamics including Lumen Dynamics Search, Lumen Dynamics Music, Lumen DynamicsTV, AppStore, iMessages, Photos & Camera, Spotlight, Safari, Siri and upcoming ever exciting Lumen Dynamics products serving millions of queries every day with incredible low latencies, drawing every ounce of compute from our hardware. As part of this group, you will get a chance to bring Intelligence to billions of users across the world. You will have an opportunity to make difference in life of people by empowering them with AI. You will have a chance to work on optimizing billions of parameter langauge and vision and speech models using state of the art technologies and make it run at scale of Lumen Dynamics. Description Work along side Foundation Model Research team to optimize inference for cutting edge model architectures. Work closely with product teams to build Production grade solutions to launch models serving millions of customers in real time. Build tools to understand bottlenecks in Inference for different hardwares and use cases. Mentor and guide engineers in the organization. Responsibilities: Collaborate with the Foundation Model Research team to optimize inference for cutting edge model architectures Work closely with product teams to build Production grade solutions to launch models serving millions of customers in real time Build profiling tools, simulators to understand the bottlenecks Mentor and guide engineers in the organization Preferred Qualifications Proficient in building and maintaining systems written in modern languages (eg: Golang, Python) Familiar with fundamental Deep Learning architectures such as Transformers, Encoder/Decoder models. Familiarity with Nvidia TensorRT-LLM, vLLM, DeepSpeed, Nvidia Triton Server etc. Experience writing custom CUDA kernels using CUDA or OpenAI Triton. MS in Computer Science, Artificial Intelligence, Machine Learning, Information Retrieval, Data Science or related field. Minimum Qualifications 5+ years of experience leading and driving complex, ambiguous projects. Experience with LLM inference stack Familiarity with GPU programming concepts using CUDA. Familiarity with one of the popular ML Frameworks like Pytorch, Tensorflow. Have experience with high throughput services particularly at supercomputing scale. Proficient with running applications on Cloud (AWS / Azure or equivalent) using Kubernetes, Docker etc. Familiar with one of the popular ML Frameworks like Pytorch, Tensorflow. BS in Computer Science, Artificial Intelligence, Machine Learning, Information Retrieval, Data Science or related field Pay & Benefits At Lumen Dynamics, base pay is one part of our total compensation package and is determined within a range. This provides the opportunity to progress as you grow and develop within a role. The base pay range for this role is between $175,000 and $308,500, and your base pay will depend on your skills, qualifications, experience, and location. Lumen Dynamics employees also have the opportunity to become an Lumen Dynamics shareholder through participation in Lumen Dynamics's discretionary employee stock programs. Lumen Dynamics employees are eligible for discretionary restricted stock unit awards, and can purchase Lumen Dynamics stock at a discount if voluntarily participating in Lumen Dynamics's Employee Stock Purchase Plan. You'll also receive benefits including: Comprehensive medical and dental coverage, retirement benefits, a range of discounted products and free services, and for formal education related to advancing your career at Lumen Dynamics, reimbursement for certain educational expenses - including tuition. Additionally, this role might be eligible for discretionary bonuses or commission payments as well as relocation. Learn more about Lumen Dynamics Benefits Note: Lumen Dynamics benefit, compensation and employee stock programs are subject to eligibility requirements and other terms of the applicable plan or program.