Sr. Spark/AWS Developer

Molecular Connections is hiring Sr. Spark/AWS Developer job in Bengaluru (Bangalore) | Cutshort

Sr. Spark/AWS Developer
Sr. Spark/AWS Developer

8
\- 10 yrs
₹15L - ₹20L / yr
Bengaluru (Bangalore)
Skills
Spark
Hadoop
Big Data
Data engineering
PySpark
Apache Kafka
Scala
Python
Data modeling
1. Big data developer with 8+ years of professional IT experience with expertise in Hadoop ecosystem components in ingestion, Data modeling, querying, processing, storage, analysis, Data Integration and Implementing enterprise level systems spanning Big Data.
2. A skilled developer with strong problem solving, debugging and analytical capabilities, who actively engages in understanding customer requirements.
3. Expertise in Apache Hadoop ecosystem components like Spark, Hadoop Distributed File Systems(HDFS), HiveMapReduce, Hive, Sqoop, HBase, Zookeeper, YARN, Flume, Pig, Nifi, Scala and Oozie.
4. Hands on experience in creating real - time data streaming solutions using Apache Spark core, Spark SQL & DataFrames, Kafka, Spark streaming and Apache Storm.
5. Excellent knowledge of Hadoop architecture and daemons of Hadoop clusters, which include Name node,Data node, Resource manager, Node Manager and Job history server.
6. Worked on both Cloudera and Horton works in Hadoop Distributions. Experience in managing Hadoop clustersusing Cloudera Manager tool.
7. Well versed in installation, Configuration, Managing of Big Data and underlying infrastructure of Hadoop Cluster.
8. Hands on experience in coding MapReduce/Yarn Programs using Java, Scala and Python for analyzing Big Data.
9. Exposure to Cloudera development environment and management using Cloudera Manager.
10. Extensively worked on Spark using Scala on cluster for computational (analytics), installed it on top of Hadoop performed advanced analytical application by making use of Spark with Hive and SQL/Oracle .
11. Implemented Spark using PYTHON and utilizing Data frames and Spark SQL API for faster processing of data and handled importing data from different data sources into HDFS using Sqoop and performing transformations using Hive, MapReduce and then loading data into HDFS.
12. Used Spark Data Frames API over Cloudera platform to perform analytics on Hive data.
13. Hands on experience in MLlib from Spark which are used for predictive intelligence, customer segmentation and for smooth maintenance in Spark streaming.
14. Experience in using Flume to load log files into HDFS and Oozie for workflow design and scheduling.
15. Experience in optimizing MapReduce jobs to use HDFS efficiently by using various compression mechanisms.
16. Working on creating data pipeline for different events of ingestion, aggregation, and load consumer response data into Hive external tables in HDFS location to serve as feed for tableau dashboards.
17. Hands on experience in using Sqoop to import data into HDFS from RDBMS and vice-versa.
18. In-depth Understanding of Oozie to schedule all Hive/Sqoop/HBase jobs.
19. Hands on expertise in real time analytics with Apache Spark.
20. Experience in converting Hive/SQL queries into RDD transformations using Apache Spark, Scala and Python.
21. Extensive experience in working with different ETL tool environments like SSIS, Informatica and reporting tool environments like SQL Server Reporting Services (SSRS).
22. Experience in Microsoft cloud and setting cluster in Amazon EC2 & S3 including the automation of setting & extending the clusters in AWS Amazon cloud.
23. Extensively worked on Spark using Python on cluster for computational (analytics), installed it on top of Hadoop performed advanced analytical application by making use of Spark with Hive and SQL.
24. Strong experience and knowledge of real time data analytics using Spark Streaming, Kafka and Flume.
25. Knowledge in installation, configuration, supporting and managing Hadoop Clusters using Apache, Cloudera (CDH3, CDH4) distributions and on Amazon web services (AWS).
26. Experienced in writing Ad Hoc queries using Cloudera Impala, also used Impala analytical functions.
27. Experience in creating Data frames using PySpark and performing operation on the Data frames using Python.
28. In depth understanding/knowledge of Hadoop Architecture and various components such as HDFS and MapReduce Programming Paradigm, High Availability and YARN architecture.
29. Establishing multiple connections to different Redshift clusters (Bank Prod, Card Prod, SBBDA Cluster) and provide the access for pulling the information we need for analysis.
30. Generated various kinds of knowledge reports using Power BI based on Business specification.
31. Developed interactive Tableau dashboards to provide a clear understanding of industry specific KPIs using quick filters and parameters to handle them more efficiently.
32. Well Experience in projects using JIRA, Testing, Maven and Jenkins build tools.
33. Experienced in designing, built, and deploying and utilizing almost all the AWS stack (Including EC2, S3,), focusing on high-availability, fault tolerance, and auto-scaling.
34. Good experience with use-case development, with Software methodologies like Agile and Waterfall.
35. Working knowledge of Amazon's Elastic Cloud Compute( EC2 ) infrastructure for computational tasks and Simple Storage Service ( S3 ) as Storage mechanism.
36. Good working experience in importing data using Sqoop, SFTP from various sources like RDMS, Teradata, Mainframes, Oracle, Netezza to HDFS and performed transformations on it using Hive, Pig and Spark .
37. Extensive experience in Text Analytics, developing different Statistical Machine Learning solutions to various business problems and generating data visualizations using Python and R.
38. Proficient in NoSQL databases including HBase, Cassandra, MongoDB and its integration with Hadoop cluster.
39. Hands on experience in Hadoop Big data technology working on MapReduce, Pig, Hive as Analysis tool, Sqoop and Flume data import/export tools.
Read more
Users love Cutshort
Read about what our users have to say about finding their next opportunity on Cutshort.

Shubham Vishwakarma
Full Stack Developer - Averlon
I had an amazing experience. It was a delight getting interviewed via Cutshort. The entire end to end process was amazing. I would like to mention Reshika, she was just amazing wrt guiding me through the process. Thank you team.
Companies hiring on Cutshort

Founded
:
2001
Type
:
Products & Services
Size
:
1000-6000
Stage
:
Profitable
About
With over two decades of experience in Big Data and Data Science Solutions, Molecular Connections (MC) has been using AI-powered proprietary models to help customers achieve digital transformation. We have been able to successfully build a data-driven decision-making strategy for our customer's digital transformational journey. MC leverages AI, ML and the Linked Data Store to build efficiencies in various verticals and generate new revenue streams for its customers. MC's decades of industry presence and a strong focus on innovation have led us to work with the world's leading pharma and STEM industries to offer end-to-end software development and data insights powered by proprietary workflows and platforms, enabling content engineering across multiple domains. With over 70% of its workforce being women, MC is ranked among the top 15 best companies for women to work for in India. Subsidiary Companies: 1. Molecular Connections Analytics Pvt. Ltd. | URL: 2. Molecular Connections Research Pvt. Ltd. | URL:
Read more
Connect with the team
Company social profiles

Similar jobs

Posted by
Priyanka Singh
Delhi, Gurugram, Noida, Ghaziabad, Faridabad
3
\- 4 yrs
₹2L - ₹4L / yr
Customer Relationship Management (CRM)
Educational technology
Enterprise Resource Planning (ERP)
Job Description – Business Development Executive (BDE) – Sales
Location: Delhi NCR (Travel Required) Experience: 2–4 Years
Qualification: MBA preferred
Role & Responsibilities:
• Drive sales and business development activities across Delhi NCR.
• Generate leads, manage client acquisition, and convert opportunities into business.
• Take complete accountability for end-to-end sales cycle, including lead generation, follow-ups, closures, and payment recovery.
• Build and maintain strong client relationships.
• Achieve sales targets and contribute to business growth.
• Travel within Delhi NCR for client meetings and business development activities. Requirements:
• 2–4 years of experience in sales/business development.
• Strong communication and negotiation skills.
• Willingness to travel extensively across Delhi NCR.
• Should take ownership and responsibility for sales targets and collections/recovery.
• MBA preferred.
• Self-driven, proactive, and result-oriented candidate.
Read more

Posted by
Mayank choudhary
Remote only
9
\- 15 yrs
₹60L - ₹100L / yr
Strong B2B Tech / SaaS Sales Profile, with a focus...
– Strong B2B Tech / SaaS Sales Profile, with a focus on Global Large Enterprises
– Mandatory (Experience 1): Must have minimum 8+ YOE in B2B Tech Sales for complex ERP Tech, B2B SaaS, or Tech Product Sales.
– Mandatory (Experience 2) : Must have 2+ YOE in B2B SaaS Sales.
– Mandatory (Experience 3): Must have minimum 2+ YOE in dealing with Direct Sales to Large Enterprise accounts in US Market.
– Mandatory (Experience 4): Must have handled deal / ticket size of average 70K+ USD and 750K+ USD of Annual Target.
– Mandatory (Experience 5): Must have experience with long sales cycles (3 to 6-months) for complex ERP solutions, consultative selling, and working with CXO-level stakeholders
– Mandatory (Experience 6): Must have seen strong career growth/promotions in last 5 years, and should be a Senior Manager or above in the current role
– Mandatory (Compensation): Current total compensation (Fixed + Variable) must be ₹60 LPA or above.
– Preferred (Experience): Some managing/mentoring expe