Senior Data Engineer

This is a remote position. At S5, we are on a mission to make Advanced Data Science consumable to Business Users. S5 is a SaaS software provider of AI/ML-powered products which transform planning and decision making capabilities across the digital supply chain for the Retail and CPG industries. To support our aggressive growth, we are actively seeking talent (at multiple levels) to join our team. The Role: Meridian Softworks is seeking an experienced Sr. Data Engineer, capable of creating and optimizing a scalable, resilient, and event-driven data pipeline architecture through our cloud-based solutions. The right person will have a proven ability to innovate, work across a variety of APIs and databases, and maintain a comprehensive catalog of all data streams, data mapping, and data integration solutions. The right fit will embrace the challenge and opportunity of an informal, fast-paced startup environment with a flat organizational structure. What You’ll Do: ​ Develop and automate large scale, high-performance data processing systems (batch and/or streaming) Translate business needs into data models: implement data strategies, build data flows and develop conceptual data models. Implement deployment and infrastructure automation strategies. Build data support for our experimentation efforts, solving problems from statistical test automation to building real-time M/L pipelines Automate manual processes, optimize data delivery, and re-design infrastructure for greater scalability. Promote data modeling standardization, define, and drive adoption of the standards. Build analytics and reporting tools that utilize the data pipeline to provide results and business value to make data-driven decisions. Maintain and improve upon our existing ETL pipeline Perform routine maintenance tasks and upgrades of our server architecture Work with stakeholders to assist with data-related technical issues and support data infrastructure needs. Apply best practices for testing and deployment in an agile environment. Contribute to shared Data Engineering tooling & standards to improve engineering productivity across the company Requirements What We’re Looking For: Expert-level understanding of relational databases (columnar and row-based) Expertise in SQL is a must. Prior experience working with Tier-1 technology consulting firms is a huge plus Proven hands-on experience building complex ETLs in a business environment with large-scale, complex datasets Scripting experience in python and bash shell required Fully proficient in pipeline building and job automations Proficiency with Git Experience building RESTful APIs Experience with error handling and data validation Experience working in GCP or other cloud platforms Bachelors in Computer Science, Engineering or similar Must be eligible to work in the US. Benefits What’s in it for You? At S5, you will get an opportunity to truly innovate while working with some of the smartest brains in the industry to solve existing and emerging business problems. Additionally, you will play a key role in defining the next generation work culture and work ethic to solve problems that others thought - simply impossible. There’s never been a more exciting time to join our team! Job Type: Full-time Originally posted on Himalayas