Azure Databricks Data Engineer

We are seeking an experienced Azure Databricks Data Engineer with Scala to join our growing data engineering team. In this role, you will design, build, and optimise modern data platforms and analytics solutions on Microsoft Azure. You will be responsible for developing scalable, secure, and high-performing data pipelines while following engineering best practices and supporting business-critical HR and enterprise data initiatives.This is an excellent opportunity for a hands-on engineer who is passionate about cloud technologies, automation, data engineering, and delivering high-quality solutions in an Agile environment.Key ResponsibilitiesDesign, develop, and enhance scalable data platforms, digital products, and technology solutions using Azure services.Build, maintain, and optimise end-to-end data pipelines using Azure Databricks, Azure Data Factory (ADF), and other Azure data services with Scala.Translate business requirements into robust, scalable, and maintainable technical solutions.Apply software engineering best practices, including code reviews, automated testing, CI/CD, and infrastructure automation.Ensure solutions meet security, reliability, performance, and compliance standards while adhering to enterprise architecture principles.Develop observability and monitoring capabilities to proactively identify production issues and support incident resolution.Collaborate with architects, product owners, business analysts, and cross-functional engineering teams to deliver high-quality data solutions.Continuously evaluate emerging technologies and recommend improvements that enhance scalability, maintainability, and operational efficiency.Support Agile delivery by contributing to sprint planning, estimation, development, testing, and production releases.Required Skills & Experience3-8 years with of hands-on experience in Data Engineering, with extensive experience designing and implementing enterprise data solutions on Microsoft Azure.Strong expertise in:Azure DatabricksAzure Data Factory (ADF)Azure Data Lake Storage Gen2Azure SQL Database / PostgreSQLAzure Analytics ServicesAzure Identity (Service Principals, System Assigned Managed Identity, User Assigned Managed Identity)Proven experience designing large-scale data integration, data lake, data warehouse, and analytics solutions handling high-volume, multi-format data.Strong programming skills in: ScalaSQLExperience with scripting using Linux Shell or PowerShell.Previous ETL development experience using tools such as Informatica, SSIS, Talend, or similar.Experience with Apache Kafka or streaming data technologies is desirable.Hands-on experience working with Azure cloud services in an Agile SDLC environment.Experience with Docker and version control systems such as Git, GitHub, or GitLab.Experience with workflow orchestration tools such as:Apache Airflow (preferred)Control-MAutoSysStrong understanding of Git workflows, branching strategies, pull requests, and CI/CD practices.Knowledge of DevOps and Infrastructure as Code tools including GitLab CI/CD, YAML, Terraform, ARM Templates, Puppet, Azure Functions, and Logic Apps is advantageous.Experience working within Agile frameworks including Scrum, Kanban, or XP.Bachelor's degree in Computer Science, Information Technology, Engineering, or a related discipline (or equivalent professional experience).Excellent communication and stakeholder management skills with strong written and spoken English.Preferred ExperienceExperience building enterprise HR data platforms and analytics solutions.Exposure to Data Mesh architecture and modern data engineering principles.Experience implementing cloud-native, scalable, and highly available Azure solutions.Familiarity with test-driven development (TDD) and engineering best practices.What We're Looking ForWe're looking for a proactive, solution-oriented Azure Databricks Data Engineer who enjoys solving complex data challenges and building scalable cloud solutions. The ideal candidate is passionate about engineering excellence, automation, and continuous improvement, and can work independently while collaborating effectively with cross-functional teams to deliver high-quality, business-critical data solutions.