Azure Databricks Data Engineering Lead

Job Description 7+ years of experience in Databricks, Microsoft Azure, PySpark and Python. Lead large-scale, complex, cross-functional projects to build the technical roadmap for the WFM Data Services platform. Lead and review design artifacts. Build and own the automation and monitoring frameworks that showcase reliable, accurate, easy-to-understand metrics and operational KPIs to stakeholders for data pipeline quality. Execute proof of concept on new technology and tools to pick the best tools and solutions. Support business objectives by collaborating with business partners to identify opportunities and drive resolution. Communicate status and issues to Senior Starbucks leadership and stakeholders. Direct project team and cross-functional teams on all technical aspects of the projects. Lead with engineering team to build and support real-time, highly available data, data pipeline and technology capabilities. Translate strategic requirements into business requirements to ensure solutions meet business needs. Define and implement data retention policies and procedures. Define and implement data governance policies and procedures. Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability. Enable team to pursue insights and applied breakthroughs, while also driving the solutions to Starbucks scale. Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of structured and unstructured data sources using big data technologies. Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency, and other key business performance metrics. Work with stakeholders including the Executive, Product, Data and Design teams to assist with data-related technical issues and support their data infrastructure needs. Perform root cause analysis to identify permanent resolutions to software or business process issues. MUST HAVE: 7+ years of experience in Databricks, Microsoft Azure, PySpark and Python. Originally posted on Himalayas