Spark Developer

Salary: $135,200 - 145,600 per year Requirements: 5+ years of hands-on Apache Spark development experience. Strong knowledge of Spark SQL, Structured Streaming, DataFrames/Datasets, and performance tuning and optimization. Experience integrating Apache Spark with Apache Kafka. Strong Java and Python development skills. Experience designing and supporting large-scale distributed systems. Solid understanding of software engineering principles, object-oriented design, and testing practices. Experience with relational and distributed data platforms. Preferred: Databricks SQL experience. Preferred: AWS experience with services such as S3, EMR, Glue, Lambda, ECS, or EKS. Preferred: Experience supporting large-scale cloud migration initiatives. Preferred: Experience with lakehouse architectures and modern data platform patterns. Preferred: Experience working with multi-terabyte or petabyte-scale datasets. Preferred: Financial services or regulated-industry experience. Standout experience includes delivering large-scale cloud migrations involving hundreds of terabytes or petabytes of data; building or modernizing enterprise data platforms with Spark, Kafka, Databricks, and AWS; and designing scalable, resilient, cost-efficient distributed systems. Strong software engineering fundamentals and expertise in Java and/or Python. Ability to influence architecture decisions and collaborate across multiple engineering teams. Ability to develop iterative solutions to easy dynamic programming problems with O(n) runtime and O(1) space. Responsibilities: Design, develop, and optimize Spark applications for large-scale data processing. Build streaming and batch data solutions using Apache Kafka and Apache Spark. Develop reusable frameworks and components for enterprise data engineering initiatives. Support migration from on-premises data platforms to Databricks and AWS. Work with architects and platform engineers to implement cloud-native data solutions. Improve Spark jobs for performance, scalability, reliability, and operational efficiency. Participate in system design, code reviews, testing, and production support. Implement CI/CD, monitoring, and observability practices for distributed data applications. Troubleshoot and resolve complex issues across Spark, Kafka, Databricks, cloud environments, and other enterprise messaging systems. Technologies: AWS Lambda CI/CD Cloud Databricks Support Java Kafka Python SQL Spark DevOps More: We are hiring a Senior Apache Spark Developer to help build and expand our next-generation data processing platform. The role supports scaling data processing, cloud adoption, and strategic migration initiatives involving Databricks and AWS, with high-volume, mission-critical real-time and batch workloads. This is an 18-month W-2 contract based in Jersey City, with a hybrid schedule of three onsite days and two remote days per week. The rate is $65–$70 per hour. last updated 41 week of 2026