You will design and develop robust data pipelines using DeltaLake, Spark Structured Streaming, and Unity Catalog on the Azure platform
Job Summary
You will design and develop robust data pipelines using DeltaLake, Spark Structured Streaming, and Unity Catalog on the Azure platform.
The role offers a competitive benefits package including discretionary bonuses, health insurance, and 5 weeks of annual leave with flexibility options.
You will partner with clients to translate data needs into strategic technical solutions while championing clean code and software engineering best practices.
Matching Summary
You will design and develop robust data pipelines using DeltaLake, Spark Structured Streaming, and Unity Catalog on the Azure platform.
Skills & Requirements
Must-have
Databricks platform experience
Python PySpark SQL skills
DeltaLake Spark Structured Streaming
Azure Event Hubs Kafka knowledge
CI/CD pipeline development
Nice-to-have
Scala or Java development experience
Cloudera Hadoop HIVE ecosystem familiarity
GDPR compliance expertise
Collaborative mindset for innovation
Quick adaptation to new technologies
Key Requirements
Proven hands-on experience with Databricks orchestration
Strong distributed data systems knowledge
Full lifecycle data pipeline development experience