Data Engineer

Job Description We are seeking an experienced Data Engineer to design, build and maintain scalable data solutions on our Databricks platform. The successful candidate will be responsible for developing robust ETL/ELT processes, building high-performance data pipelines using PySpark, and integrating data from a variety of source systems into our Lakehouse environment. This role will play a key part in enabling data-driven decision making by delivering reliable, high-quality and governed data assets. The ideal candidate will have strong experience with Databricks, modern cloud-based data platforms, Data Lakehouse architecture, Medallion Architecture principles and enterprise-scale data engineering best practices. Key Accountabilities Design, develop and maintain scalable data pipelines within the Databricks platform. Build and optimise ETL/ELT processes using PySpark and Spark SQL. Implement and support Data Lakehouse and Medallion Architecture frameworks. Ingest, transform and integrate data from multiple internal and external source systems. Ensure data quality, accuracy, consistency and governance across data platforms. Monitor, troubleshoot and optimise data processing workloads and platform performance. Support the delivery of analytical, reporting and operational data solutions. Collaborate with the Product Owner, Business Analysts and business stakeholders to understand and deliver data requirements. Contribute to the development of data engineering standards, best practices and reusable frameworks. Support deployment, release management and CI/CD processes for data solutions. Qualifications Bachelor’s degree in Computer Science, Information Technology, Data Science or a related discipline. Microsoft Certified: Azure Data Engineer Associate (preferred). Databricks Certified Data Engineer Associate or Professional (preferred). Azure Fundamentals or Azure Data Fundamentals certifications (desirable). Relevant cloud, data engineering or analytics certifications are advantageous. Experience Minimum of 4+ years of hands-on experience specifically within Databricks and PySpark environments. Broad background in Data Warehousing (10+ years preferred). Strong hands-on experience building ETL/ELT pipelines using PySpark and Spark SQL. Company Description MSX has been a trusted partner to leading vehicle manufacturers, their retailers, and mobility organizations globally for more than 30 years. Our unwavering commitment is to help our clients transform their businesses and effectively manage operations in the areas of: Sales Performance; Repair Optimization and Compliance; Parts and Accessories Sales Performance; and Consumer Engagement. With our global teams, industry expertise, and the power of technology, we design and deliver tailored, sustainable, and innovative solutions and services that help our clients optimize their operations and captivate their customers. Additional Information With over 5,000 employees based in more than 80 countries across the globe, our teams provide industry leading expertise that spans: Consumer Engagement Parts, Accessories & Service Performance Actionable Insights Repair Optimization & Compliance Learning Solutions Distribution & Sales Performance Our proven track record means that we now partner with almost every car manufacturer on the market. The MSX Purpose To empower Movers and Makers to thrive in our ever-changing world The MSX Mission To harness our expertise in mobility, the creativity of our global teams, and the power of technology, to craft tailored, sustainable and innovative solutions. The MSX Vision To be the clients’ first choice, recognized for our operational excellence and commitment to driving change and innovation in the mobility industry. MSX is an equal opportunities employer and encourages applications from suitably qualified and eligible candidates regardless of sex, race, disability, neurodiversity or other personal characteristics and backgrounds, age, sexual orientation, gender reassignment, religion or belief, or marital and parental status. As users of the Disability Confident scheme, we interview all disabled applicants who meet the minimum criteria for the vacancy.