Senior Data Engineer

Salary: $111000 - 215000 per year Requirements: 7+ years of experience in data engineering or a closely related discipline. Strong command of Python, SQL, Spark, and distributed data processing. Practical experience with Databricks, Delta Lake, and Lakehouse-based architectures. Solid grasp of ETL/ELT design, data modeling, and data quality best practices. Track record of building scalable, production-ready data pipelines. Experience working alongside analytics, machine learning, and product teams. Excellent communication skills with the ability to translate data needs and technical choices clearly. Responsibilities: Design, build, and support scalable ETL/ELT pipelines using Spark, Python, SQL, and Databricks. Create dependable ingestion frameworks for both batch and streaming data sources. Make sure pipelines meet service-level expectations, quality standards, and production reliability requirements. Develop data models across raw, curated, and semantic layers using Delta Lake. Build dimensional models, star schemas, and domain-specific datasets for analytics and machine learning. Define and uphold standards for schema structure, metadata, and lineage. Put in place data validation, anomaly detection, documentation, and SLA monitoring across pipelines. Create automated testing, monitoring, and alerting for freshness, completeness, and accuracy. Partner with platform teams to strengthen observability and operational tooling. Work with analysts to understand business KPIs and deliver curated datasets. Collaborate with ML engineers and data scientists to create reusable feature pipelines. Coordinate with data platform engineers to improve compute, governance, and orchestration. Tune Spark jobs, SQL queries, cluster settings, and storage patterns for better performance and lower cost. Improve reliability, reduce technical debt, and simplify complex workflows. Apply best practices for role-based access control, privacy, and PII handling through Unity Catalog. Ensure compliance with governance frameworks and documentation standards. Stay current on modern data engineering patterns, Lakehouse architecture, orchestration, and related best practices. Evaluate new technologies that improve reliability, scalability, and developer productivity. Technologies: Databricks ETL Support Machine Learning Python SQL Spark Unity GameDev AI RBAC Security More: We are CareScout, a wholly owned subsidiary of Lumen Dynamics Financial, Inc., and we are building a new experience for care seekers and their families by bringing together long-term care options, non-healthcare resources, education, and human support in one place. Our mission is to simplify and dignify the aging experience, and our team is curious, collaborative, and deeply committed to belonging. This Senior Data Engineer role is fully remote for applicants based in approved Eastern Time locations, and it is not eligible for employment visa sponsorship. We offer a competitive compensation range, bonus eligibility, and a strong employee benefits package that includes healthcare coverage, retirement savings options, paid time off, family leave, tuition support, wellness resources, and caregiver and mental health support. last updated 26 week of 2026