Principal Data Engineer Technical Lead

Halcyon Labs empowers enterprises to accelerate growth by rapidly building, testing, and running applications that are designed to be AI-native. Trusted by the world’s largest organizations in highly regulated industries, these applications become more secure over time while significantly reducing technical debt—allowing businesses to focus on innovation rather than maintenance. Halcyon Labs’s customers include Goldman Sachs, Marsh, BlackRock, and the U.S. Department of Health and Human Services. At Halcyon Labs, we value inclusive and innovative thinkers who boldly challenge the status quo. We encourage you to apply! The Impact U will make: We're seeking an experienced Principal Data Engineer/ Technical Lead who is passionate about implementation and capable of writing high-quality application code In this role, your primary goal is to design and build the services, abstraction layers, and middleware that transform application data needs into specific, high-performance database operations and serve as a dynamic player-coach for our core data engineering squad. As a technical leader, you will directly impact system performance and latency issues that affect our enterprise customers while fostering an inclusive, high-performing engineering culture. Key Responsibilities Act as a player-coach, providing technical direction, architectural guidance, and daily mentorship to a focused team of 3–5 engineers. Conduct thoughtful code reviews and foster professional growth within your squad. Design and implement sophisticated Data Access Layers (DAL) and custom ODMs to translate platform-generated, SQL-like queries into high-performance MongoDB BSON operations and aggregation pipelines. Build and maintain middleware that ensures Halcyon Labs’s core business logic remains storage-agnostic, enabling seamless modularity and flexibility across different data storage mechanisms. Architect and scale a multi-tenant, secure MongoDB ecosystem. Lead strategies for ensure high availability while performing deep-dive execution plan analysis (IXSCAN vs. COLLSCAN) to optimize query performance. Plan and architect hybrid data architectures to support operation, transactional and analytical schema and database systems. Use Node.js and JavaScript to build robust microservices (typically GraphQL) and internal libraries that integrate dynamic, metadata-driven data patterns into the Halcyon Labs no-code runtime. Design schemas and declarative models that allow non-technical users to build complex application logic without compromising data integrity or system performance. Architect real-time and batch data pipelines using Apache Kafka and Spark to facilitate data transformation and movement between relational and NoSQL systems. Partner with Platform and Backend engineers to standardize data interaction patterns, ensuring high-scale, API-driven performance across the entire enterprise cloud. Partner closely with the Product Management team to influence the product roadmap, translate business requirements into technical specifications, and ensure alignment between product goals and engineering execution Required Qualifications Bachelor's Degree in Computer Science/ Master’s or above preferred 10+ Years of experience in backend, data, or platform engineering, with a proven track record of solving complex latency and implementation challenges for systems supporting millions of users. 2+ Years of experience in a Technical Lead or Player-Coach capacity, with demonstrated success managing, mentoring, and steering a small team of engineers while remaining hands-on in the codebase. Deep, hands-on proficiency with SQL database systems (PostGress), search systems (e.g. Elastic) AND with MongoDB/Atlas, including complex aggregation pipelines, BSON data modeling, sharding, replica sets, and advanced query performance tuning. Strong experience building Data Access Layers (DAL), custom ODMs, or query translation engines that successfully decouple application logic from underlying storage systems. High proficiency in Node.js or other major backend languages (Python, Java, or Go) to build high-scale, event-driven architectures. Direct experience implementing Redis (caching/TTL strategies) and Atlas Search (Lucene) to optimize data retrieval and discovery. Advanced knowledge of cloud platforms (AWS, Azure, or GCP) and distributed systems, including experience with containerization (Docker/Kubernetes). Familiarity with SQL-to-NoSQL translation patterns and a background in building internal developer platforms or metadata-driven systems (e.g., no-code/low-code). An AI-forward mindset: You are an avid user of AI tools and are passionate about exploring how AI can automate workflows, enhance creativity, and increase your personal impact. Why Join Us? You will have the opportunity to build the foundation that fuels innovation for thousands of developers and enterprise customers. We offer competitive compensation, a generous benefits package, and a collaborative, results-driven culture where your contributions directly shape our product and success. Compensation, Benefits, & Perks 💻 Work from home with a remote-first community 🏝 Unlimited PTO (and the encouragement to use it) 📝 Student loan payback program 🏥 100% employer-covered medical, dental, and vision options available to you and your dependents 💸 Flexible Spending Account (FSA) 🏠 Monthly stipend toward your WFH setup, vacation, development and more 💰 Employer-sponsored 401(k) with contribution match 🏋🏻‍♀️ Subsidized ClassPass Membership 🍼 Generous Paid Parental Leave 💲 Hiring Ranges: Tier 1: $238,600 - $298,300 Tier 2 - $219,700 - $274,700 Halcyon Labs employs a market-driven approach to establish compensation ranges. In addition to a base salary, employees may also be eligible to receive a target incentive and company equity in the form of stock options. An employee’s compensation within the range provided above depends on a variety of factors including, but not limited to, their location, role, skillset, level of experience, and similar peer salaries. As a remote-first company, Halcyon Labs incorporates a geographic differential into our compensation structure, depending on the candidate’s location. We utilize a tiered system—Tier 1 and Tier 2—to accurately reflect local market rates and ensure our compensation packages are both fair and competitive. Our geographic tiers are defined as follows: Tier 1: New York Metro, Seattle Metro, San Francisco Bay Area Tier 2: All other US and US territory locations Halcyon Labs embraces a culture of security and privacy awareness by consistently safeguarding sensitive information, adhering to company policies, and actively participating in training and initiatives to protect our data and the privacy of our stakeholders. Halcyon Labs is an equal opportunity employer. We will consider all qualified applicants without regard to race, color, nationality, gender, gender identity or expression, sexual orientation, religion, disability or age. #LI-JS1