Principal Azure Data Engineer (databricks)

Capco

London, United Kingdom
On-site
Databricks platform
Azure ecosystem
Python, pyspark
Lead the architecture of data platforms driving the future of financial services

Job Summary

  • Lead the architecture of data platforms driving the future of financial services.
  • Architect secure and scalable pipelines using DeltaLake, Spark Structured Streaming, and Unity Catalog.
  • Deliver high-impact technology solutions for Tier 1 financial institutions.

Matching Summary

Lead the architecture of data platforms driving the future of financial services.

Skills & Requirements

Must-have

  • Databricks platform
  • Azure ecosystem
  • Python, PySpark
  • Spark Structured Streaming
  • DeltaLake, Unity Catalog
  • CI/CD pipeline development
  • Azure DevOps, Jenkins, GitHub Actions

Nice-to-have

  • client-facing and commercial skills
  • coaching and mentoring engineering teams
  • Scala or Java development
  • PII and sensitive data handling
  • GDPR regulatory frameworks

Key Requirements

  • Proven experience with Databricks platform
  • Expertise in Python, PySpark
  • Extensive background in CI/CD pipeline development
  • In-depth understanding of data lakehouse principles
  • Experience building robust, production-grade data pipelines

Work Rights

Not specified

Tailored Resume

Cover Letter