Software Engineer Ii - Data Engineer - Data & Ml Engineering
Bristol Myers Squibb UK
Hyderabad, India
On-site
Databricks expertise
Real-time data ingestion pipelines in aws
Cloudformation expertise
The Data Engineer will be responsible for developing and maintaining ETL/ELT pipelines for ingesting data from various sources into our data warehouse
Job Summary
The Data Engineer will be responsible for developing and maintaining ETL/ELT pipelines for ingesting data from various sources into our data warehouse.
Collaborate with data architects, data analysts and data scientists to understand their data needs and ensure that the data infrastructure supports their requirements.
Stay up-to-date with emerging trends and technologies in data engineering and analytic.
Matching Summary
The Data Engineer will be responsible for developing and maintaining ETL/ELT pipelines for ingesting data from various sources into our data warehouse.
Skills & Requirements
Must-have
Databricks expertise
Real-time data ingestion pipelines in AWS
CloudFormation expertise
GitHub Workflows development and integration with AWS
AWS Glue services and ecosystem
API development using AWS services
Nice-to-have
End-to-end ownership mindset
Agile/Product based environment experience
Fast-paced environment with minimal oversight
Experience with Redshift, Athena, Lakeformation
Experience with Cloudera Data Platform
Experience with Tableau
Key Requirements
3-5 years of hands-on experience in data capabilities and solutions
5+ years of experience in data engineering or software development
Experience with data lakehouses, master/reference data management, data quality and analytics/AI ML
Expertise with boto3 APIs for Lambda, S3, Glue, Crawlers, Data Zone (preferred)
Experience developing and using Custom Resources in CFTs