Principal Azure Data Engineer (databricks)

Capco

London, United Kingdom
On-site
Databricks platform
Azure ecosystem
Deltalake, spark structured streaming, unity catalog
Lead the architecture of data platforms driving the future of financial services

Job Summary

  • Lead the architecture of data platforms driving the future of financial services.
  • Architect secure and scalable pipelines using DeltaLake, Spark Structured Streaming, and Unity Catalog.
  • Deliver high-impact technology solutions for Tier 1 financial institutions.

Matching Summary

Lead the architecture of data platforms driving the future of financial services.

Skills & Requirements

Must-have

  • Databricks platform
  • Azure ecosystem
  • DeltaLake, Spark Structured Streaming, Unity Catalog
  • Python, PySpark
  • CI/CD pipeline development
  • data lakehouse principles

Nice-to-have

  • client-facing and commercial skills
  • coaching and mentoring
  • Scala or Java development
  • PII and sensitive data handling

Key Requirements

  • Proven experience with Databricks platform
  • Expertise in Python, PySpark
  • Extensive CI/CD pipeline development experience
  • In-depth understanding of data lakehouse principles
  • Experience building production-grade data pipelines

Work Rights

Not specified

Tailored Resume

Cover Letter