Databricks Platform Engineer

Position:Databricks Platform Engineer Location:Bloomington,IL ( 90% offsite, 10% onsite) Duration:Long term Job Description: ** Do not submit profiles limited to traditional ETL tools, basic SQL development, or AWS Glue without meaningful hands-on Databricks and Spark experience. Databricks Platform Engineer: Must-Have Skills 6+ years of cloud platform or data infrastructure engineering experience, including 2+ years of production Databricks experience. Strong hands-on experience designing and administering Databricks workspaces on AWS. Deep AWS infrastructure experience with VPC, IAM roles and policies, cross-account access, SCPs, S3, KMS, Secrets Manager, and CloudTrail. Strong Terraform experience, specifically with the Databricks Terraform provider. Hands-on expertise with Unity Catalog administration, including metastore configuration, catalog and schema ownership, storage credentials, external locations, privilege management, and RBAC. Experience implementing column-level masking, row-level security, dynamic views, lineage, and audit logging. Strong knowledge of private networking, including AWS PrivateLink, VPC peering, transit gateways, security groups, NACLs, and no-public-IP configurations. Hands-on experience defining Databricks cluster policies, instance pools, job clusters, autoscaling, Photon, runtime standards, and cost controls. Experience with Databricks Workflows, system tables, CLI, SDK, Repos, and Git integration. Strong CI/CD experience using GitHub Actions, Azure DevOps, or equivalent for Databricks deployments. Experience implementing platform monitoring, cost attribution, and observability using Databricks system tables, Datadog, Grafana, or equivalent tools. Ability to create platform architecture documentation, operational runbooks, onboarding standards, and production support procedures. Profiles should demonstrate all the following: Production Databricks platform administration on AWS Unity Catalog implementation and administration Terraform-based Databricks provisioning AWS networking and cross-account IAM experience Databricks security, governance, cluster policies, and cost controls CI/CD automation for Databricks platform resources Thanks, Ganesh R Jupiter Consulting Services MWBE Certified Company 16024 Manchester Road, Suite 262, St. Louis, MO 63011. Email Jupiter Consulting Services, LLC is an Equal Opportunity Employer (EOE).