Data Engineer (Hybrid, BGC)

Overall Duties and Responsibilities Does (The tasks / responsibilities that the role performs to address requirements in Key Result Areas). 1. Data Warehousing Design Design data warehouse and operational data marts following the Client standards. Prepare data model, data dictionary, and mapping documents 2. Data Pipeline design, development, testing, and support Design and develop data pipelines that will ingest the data from the respective source systems into the data warehouse Support and optimize existing data pipelines to ensure that the downstream SLAs are met Ensures the data engineering standards and practices are followed like code version history, readability, functionality, extendibility, test runs, deployments, and post deployment support Manage data storage, computing unit requirements, and data security within the data warehouse 3. Comply with Client's agile ways of working Responsible in attending agile ceremonies as part of the Agile Tribe set-up and delivering the different demands of the Tribe related to data ingestions to the data warehouse platform Gather data requirements coming from Tribe demands 4. Maintain high data quality in the data warehouse Ensures data quality within the data warehouse tables by implementing data validation rules and data quality checks within the data pipeline 5. Incident Management Support Supports L2 data operations team during production incidents by troubleshooting data pipelines and understanding data behaviors Supports stakeholders of the data warehouse platform by solving data related issues and conducting data investigations Displays (The knowledge, skills and behaviors indicating how tasks / responsibilities will be performed). Good understanding of data warehousing concepts and data modeling Proficient in SQL and optimizing queries Good communication and problem solving skills Delivers (The specific outputs / tangible results produced by the role; resources responsible for) Data Delivery Documentations – Data Dictionary, Data Mapping Documents, Data Models Optimized data pipelines Root cause analysis for production related issues and incidents Qualifications (Minimum requirements in terms of experience and background, skills and competencies for sourcing purposes) Graduate of any IT related courses like BS Computer Science, Information Technology or the likes 4-6 years of experience in IT project delivery or data development delivery particularly in Data Warehousing or Business Intelligence Proficient in ETL development related to Data Warehousing, Business Intelligence, or other Data Delivery projects. Proficient in optimizing data pipelines in order to meet SLA requirements and save cost Proficient in SQL, Python, R, Java, Scala Proficient in Agile delivery methodologies like Scrum or Kanban Proficient in data modeling, data integrations, data quality, and data architecture is a plus Competent in Big Data, Hadoop, EMR, Hive, and cloud environments is a plus Competent in using Alibaba Cloud products particularly Dataworks, Maxcompute, OSS, Hologres Competent in Kafka, Confluent, Real-time processing, Data Streaming is a plus Proficient in using AWS Cloud products particularly S3, Glue, Athena, Data Sync, Lake Formation, Airflow, Secrets Manager, IAM, KMS, Cloudwatch Competent in using Terraform, Gitlab