Principal Software Engineer

Salary: $64000 - 106000 per year Requirements: We require formal training or certification in software engineering concepts, along with 7+ years of applied experience. We need strong Python engineering expertise, including experience with PyTorch or TensorFlow. We look for deep knowledge of vector storage systems and agent memory design. We require experience building long-running autonomous agents that use tools, skills, and human-in-the-loop oversight. We expect proven success deploying LLM-based services to production, such as APIs and microservices. We need extensive MLOps experience, including CI/CD, monitoring, incident response, and model governance. We look for cloud-native AI deployment experience on AWS or Azure, with a focus on cost and performance optimization. We require a strong commitment to responsible AI practices and operational excellence. We value strong communication and collaboration skills across product, risk, legal, and compliance partners. We expect experience designing and scaling agentic AI-enabled development practices using enterprise-authorized tools, including standards for human-in-the-loop validation, traceability, and secure handling of sensitive data. We need a strong understanding of responsible AI controls in engineering workflows, including security, resiliency, data sensitivity, and risk-based governance. Preferred: experience with fine-tuning, adapters, or custom evaluation frameworks. Preferred: background operating AI systems in regulated environments such as finance or healthcare. Preferred: experience with prompt engineering and LLM orchestration. Preferred: knowledge of safety filters, audit logging, and explainability in production systems. Preferred: experience mentoring senior engineers and leading architecture discussions. Preferred: demonstrated ability to influence technical roadmaps and priorities. Responsibilities: We design and implement agentic AI reference architectures, including orchestration, retrieval, memory, guardrails, and evaluation harnesses. We write production-grade Python code, using PyTorch or TensorFlow as needed, and review critical-path code. We build reusable components for prompt management, evaluators, safety filters, connectors, embedding pipelines, and memory stores. We develop and operate LLM-powered APIs and microservices that integrate into advisor, client, and internal workflows. We own the full ML lifecycle, including experimentation, CI/CD, automated testing, monitoring, drift detection, versioning, and rollback. We optimize inference for latency, throughput, caching, batching, model selection, and cost per inference. We partner with data teams on structured and unstructured pipelines, document ingestion, metadata, and access controls. We set engineering standards for agentic AI systems and lead design reviews. We influence roadmap direction and priorities through technical insight and delivery. We architect and govern agentic AI-enabled engineering workflows to improve delivery speed, code quality, and operational outcomes at scale. We apply knowledge of software development lifecycle tools, including enterprise-authorized AI-assisted development and automation capabilities, to increase the value of automation at scale. We hire, lead, and mentor a team of machine learning and software engineers. We promote best practices in ML engineering and raise team performance to deliver scalable, high-quality solutions with operational excellence. Technologies: AI AWS Architect Azure CI/CD Cloud Support LLM Machine Learning PyTorch Python Security TensorFlow microservices Marketing More: We are a leading financial institution with more than 200 years of history, serving consumers, small businesses, and major corporate, institutional, and government clients through the Lumen Dynamics and Chase brands. This role sits within our Corporate Sector AI/ML & Data Platforms team for LLM Suite, our internally built AI tool used by employees for productivity and business solutions. We offer a competitive total rewards package, including salary based on role, experience, skills, and location, with potential incentive compensation and equity for eligible roles. Our benefits may include comprehensive health coverage, wellness centers, retirement savings, backup childcare, tuition reimbursement, mental health support, and financial coaching. We are committed to diversity, inclusion, and equal opportunity, and we provide reasonable accommodations where needed. last updated 28 week of 2026