Data Engineer

📋 Description Creating and supporting batch, incremental, and real-time ETL pipelines Standardizing ingestion, validation, and cleaning processes across clients Automating data validation to increase data quality Managing and evolving schemas in all parts of our pipeline Monitoring, tuning, and optimizing Spark jobs Becoming a domain-expert in connected vehicle data 🎯 Requirements 3+ years as a data engineer Experience as a tech lead or mentor Proficiency in Python/Scala/Java/C++ and SQL 3+ years of experience with Spark or equivalent technologies 2+ years of experience with a workflow scheduler (Airflow, Prefect, Argo, etc) 2+ years of experience with distributed file-systems (HDFS, S3, etc)