Senior Data Engineer - Databricks

📋 Description Own the Databricks pipelines for Sugar Predict, ensuring reliability and scalable delivery. Migrate legacy ETL/ELT pipelines to Databricks; build automation to minimize manual steps. Design and build high-performance Databricks pipelines for ERP/CRM data across Azure and AWS. Enforce data security across Databricks, including RBAC and secrets management. Own the Delta Lake architecture: schema design, partitioning, data quality, incremental processing. Improve pipeline cost efficiency and throughput through optimization. 🎯 Requirements 4+ years of data engineering experience. 2+ years on Databricks or Apache Spark (Azure/AWS). Proficiency in PySpark, SQL, and Python with production-grade pipelines under SLA. Strong Delta Lake expertise: ACID, schema design, incremental/streaming. PostgreSQL experience in production data pipelines. Data governance, security, and multi-tenant patterns experience. Ability to collaborate across data science, product, and infra teams. Cost governance and cross-cloud data access experience preferred. 🎁 Benefits Excellent healthcare for you and family. 401(k) match. Unlimited Paid Time Off. Paid Parental Leave. Online Legal Services (Rocket Lawyer). Corporate Benefit Program with travel/entertainment discounts.