Senior Data Engineer / Tech Lead (Spark)

Important: Addepto may publish job opportunities on external job boards, but all applications are collected and processed through our official recruitment system.Please make sure that any further recruitment communication you receive comes from an official Addepto recruitment channel. If you receive a job offer, interview invitation, or recruitment email from an unofficial or unverified source, please be cautious, as it may not come from Addepto.Addepto is a leading AI consulting (https://addepto.com/ai-consulting/) and data engineering (https://addepto.com/data-engineering-services/) company that builds scalable, ROI-focused AI solutions for some of the world's largest enterprises and pioneering startups, including Rolls Royce, Continental, Porsche, ABB, and WGU. With an exclusive focus on Artificial Intelligence and Big Data, Addepto helps organizations unlock the full potential of their data through systems designed for measurable business impact and long-term growth.The company's work extends beyond client engagements. Drawing from real-world challenges and insights, Addepto has developed its own product - ContextClue - and actively contributes open-source solutions to the AI community. This commitment to transforming practical experience into scalable innovation has earned Addepto recognition by Forbes as one of the top 10 AI consulting companies worldwide.As part of KMS Technology, a US-based global technology group, Addepto combines deep AI specialization with enterprise-scale delivery capabilities—enabling the partnership to move clients from AI experimentation to production impact, securely and at scale.As a Senior Data Engineer (Spark), you will join a complex data engineering project focused on building and developing scalable data pipelines and modern data processing solutions.The role combines approximately 70% hands-on data engineering with 30% technical leadership. Alongside designing and developing data solutions, you will take technical ownership of selected areas, contribute to architectural decisions, and support the team in delivering reliable and scalable solutions.🎁 Discover our perks & benefits:Work in a supportive team of passionate enthusiasts of AI & Big Data.Engage with top-tier global enterprises and cutting-edge startups on international projects.Enjoy flexible work arrangements, allowing you to work remotely or from modern offices and coworking spaces. Accelerate your professional growth through career paths, knowledge-sharing initiatives, language classes, and sponsored training or conferences, including a partnership with Databricks and Anthropic, which offers industry-leading training materials and certifications.Participate in team-building events and utilize the integration budget.Celebrate work anniversaries, birthdays, and milestones.Access medical and sports packages, eye care, and well-being support services, including psychotherapy and coaching.Get full work equipment for optimal productivity, including a laptop and other necessary devices.With our backing, you can boost your personal brand by speaking at conferences, writing for our blog, or participating in meetups.Experience a smooth onboarding with a dedicated buddy, and start your journey in our friendly, supportive, and autonomous culture.🚀 Your main responsibilities:Design, develop, and maintain scalable data pipelines using Python, SQL, Spark, and AirflowBuild and improve data processing solutions based on Apache Iceberg and modern lakehouse architecturesWork with technologies such as PyArrow, PyIceberg, PostgreSQL, and TrinoTake technical ownership of selected areas and contribute to technical and architectural decisionsCombine hands-on development with technical leadership and knowledge sharing within the teamBuild and maintain containerized workloads using Docker and KubernetesEnsure data pipelines are scalable, reliable, maintainable, and production-readyCollaborate with engineers and other stakeholders to understand business and technical requirements and translate them into effective solutionsContribute to engineering best practices, including code quality, testing, CI/CD, and automationMonitor, troubleshoot, and continuously improve existing data pipelines and platform componentsUse modern development practices, including AI-assisted development, to improve engineering efficiency.🎯 What you'll need to succeed in this role:At least 4 years of commercial experience in Data Engineering or Big Data projectsStrong programming skills in PythonVery good knowledge of SQLStrong hands-on experience with Apache Spark and distributed data processingPractical experience with workflow orchestration tools, preferably Apache AirflowExperience with Apache Iceberg and/or modern lakehouse architecturesExperience working with Docker and KubernetesGood understanding of data modelling, data pipelines, and data processing architecturesExperience working with relational databases and/or query engines such as PostgreSQL or TrinoGeneral understanding of cloud environments and cloud-based data solutionsAbility to take ownership of technical solutions and contribute to architectural and technical decisionsExperience supporting other engineers through technical guidance, knowledge sharing, or mentoringAbility to work independently and take ownership of project deliverablesStrong communication and collaboration skillsFluent English (at least C1 level).Bachelor’s degree in technical or mathematical studies.➕ Nice to have:Experience with Kafka or other streaming technologiesFamiliarity with modern data modelling, transformation, and ingestion tools (e.g. dbt, dlt, Airbyte)Experience with CI/CD and DevOps tools (e.g. GitHub Actions, ArgoCD)Familiarity with monitoring and visualization tools (e.g. Grafana, Power BI)Experience with Databricks and cloud-based data environmentsAre you interested in Addepto and would like to join us?Get in touch! We are looking forward to receiving your application. Would you like to know more about us?Visit our website (career page) and social media (Facebook, LinkedIn).