3+ years designing scalable distributed data pipelines
Extensive experience with python and sql
Must have databricks platform and pyspark experience
You will be responsible for the design and implementation of scalable data solutions providing enterprise-scale data transformation across a broad range of projects
Job Summary
You will be responsible for the design and implementation of scalable data solutions providing enterprise-scale data transformation across a broad range of projects.
The role focuses on delivering solutions that utilize large-scale data ingestion, processing, storage, streaming, and batch analytics using Databricks.
WPP offers a culture of creativity, belonging, and continuous learning with opportunities to work at an unparalleled scale in the industry.
Matching Summary
You will be responsible for the design and implementation of scalable data solutions providing enterprise-scale data transformation across a broad range of projects.
Skills & Requirements
Must-have
3+ years designing scalable distributed data pipelines
Extensive experience with Python and SQL
Must have Databricks platform and PySpark experience
Experience with Microsoft Azure Data Factory and ADLS gen2
Nice-to-have
Exposure to AI and ML technologies
Experience with Kafka and Delta Lake
Familiarity with Pandas library
Knowledge of CI/CD practices using Azure DevOps
Key Requirements
Minimum 3 years of experience in data engineering
Fluency in English required
Demonstrable understanding of Agile practices and testing