Principal Engineer (Big Data Engineer)

About the job Accelerate your career with PradeepIT PradeepIT is one of the largest, globally recognized IT Consulting firm to connect India's deeply vetted talent team to global customer. Were headquartered in Bengaluru, Silicon Valley of India. PradeepITs customers include SAP Lab, Bosch, Rolls-Royce, Daikin, Daimler and J&J and hundreds of other Fortune 500 companies and fast-growing startups. With continuous hard work and working remotely by choice, PradeepIT is certified as a Great Place to Work! Trusted by leading brands and fortune 500 companies from around the world, we have achieved: 6+ Years of Experience 580+ Open source technology Consultant 120+ SAP Consultant 40+ Salesforce Consultant 60+ Adobe Consultant 100+ Mobility Consultant 890+ Clients in APAC, EMEA & USA Our Beliefs PradeepIT believes in connecting people across the globe and provide them an opportunity work on remotely. Being a people-first organization, PradeepIT constantly strives for individuals who won't just keep up, but break new ground, work with cutting edge technology and ramp-up their skills with course created by our Vertical Heads, Senior Architect for freely with help of PradeepIT Academy. Principal Engineer (Big Data Engineer) Experience: 12 15 years Requirements: Should have a strong technical background with Big data, Spark, Python (PySpark) / Scala/Java, AWS/Azure/GCP ; Experience working with MS SQL and NoSQL databases (Hbase, Cassandra). Abilities to build Data Pipelines (from ETL to consumption) using Apache Stack Tools, GCP, AWS and Azure. Design and develop complex mappings, performance tuning of mappings, Process Flows Hands-on experience in System designing, Event sourcing, and message broker architecture Requirement gathering and analysis Define, understand, and analyze non-functional requirements for the project Lead the technical team Should be able to compare technologies to find the best fit for project requirements Should be able to troubleshoot complex or unusual bugs Responsibilities: Writing and reviewing great quality code/support Understanding the client's business use cases and technical requirements and being able to convert them into a technical design that elegantly meets the requirements. Mapping decisions with requirements and being able to translate the same to developers. Identifying different solutions and being able to narrow down the best option that meets the client's requirements. Defining guidelines and benchmarks for NFR considerations during project implementation Writing and reviewing design documents explaining the overall architecture, framework, and high-level design of the application for the developers Review architecture and design on various aspects like extensibility, scalability, security, design patterns, user experience, NFRs, etc., and ensure that all relevant best practices are followed. Developing and designing the overall solution for defined functional and non-functional requirements; and defining technologies, patterns, and frameworks to materialize it Understanding and relating technology integration scenarios and applying these learnings in projects Resolving issues that are raised during code/review, through exhaustive systematic analysis of the root cause, and being able to justify the decision taken. Originally posted on Himalayas