Senior Azure Data Engineer (databricks)

Capco

London, United Kingdom
On-site
Databricks platform and orchestration
Python, pyspark, and sql
Deltalake, spark structured streaming, unity catalog
Design and develop robust pipelines using DeltaLake, Spark Structured Streaming, and Unity Catalog

Job Summary

  • Design and develop robust pipelines using DeltaLake, Spark Structured Streaming, and Unity Catalog.
  • Apply DevOps principles to develop CI/CD pipelines using Azure DevOps, Jenkins, or GitHub Actions.
  • Deliver high-impact technology solutions for Tier 1 financial institutions.

Matching Summary

Design and develop robust pipelines using DeltaLake, Spark Structured Streaming, and Unity Catalog.

Skills & Requirements

Must-have

  • Databricks platform and orchestration
  • Python, PySpark, and SQL
  • DeltaLake, Spark Structured Streaming, Unity Catalog
  • real-time event-driven solutions
  • CI/CD pipelines
  • data lakehouse architecture

Nice-to-have

  • Scala or Java development
  • Cloudera, Hadoop, HIVE, Spark
  • data privacy regulations
  • adapt new technologies quickly
  • collaborative mindset and innovation

Key Requirements

  • Proven hands-on experience with Databricks
  • Strong skills in Python, PySpark, and SQL
  • Expertise in full lifecycle data pipelines
  • Experience with data lakehouse architecture
  • Working knowledge of DevOps tools and CI/CD

Work Rights

Not specified

Tailored Resume

Cover Letter