Senior Data Engineer (Azure Databricks)

UARROW PTE. LTD.

Singapore
Not specified (assumed office-based unless stated otherwise).
Azure databricks expertise
Apache spark optimization
Delta lake implementation
UARROW PTE. LTD. is seeking a Senior Data Engineer specializing in Azure and Databricks to design and implement large-scale data lakehouse solutions. The role focuses on building and optimizing ETL/ELT pipelines, ensuring data quality, and enabling advanced analytics through collaboration with cross-functional teams

Job Summary

  • The role involves designing and implementing large-scale Azure and Databricks Data Lakehouse solutions to support enterprise data needs.
  • Candidates will build and optimize ETL/ELT pipelines for both batch and real-time streaming workloads using advanced tools like Apache Spark and Kafka.
  • The position requires establishing best practices, mentoring junior engineers, and ensuring robust data governance through Unity Catalog and least-privilege principles.

Matching Summary

Match Score: 85

UARROW PTE. LTD. is seeking a Senior Data Engineer specializing in Azure and Databricks to design and implement large-scale data lakehouse solutions. The role focuses on building and optimizing ETL/ELT pipelines, ensuring data quality, and enabling advanced analytics through collaboration with cross-functional teams.

Skills & Requirements

Must-have

  • Azure Databricks expertise
  • Apache Spark optimization
  • Delta Lake implementation
  • Unity Catalog security
  • PySpark library development
  • Kafka streaming integration
  • Azure DevOps CI/CD

Nice-to-have

  • Great Expectations validation
  • MLflow integration experience
  • Legacy Hive migration skills
  • Cross-functional collaboration
  • Mentorship capabilities

Key Requirements

  • Senior level experience in data engineering
  • Proficiency in Azure cloud services
  • Strong background in PySpark and SQL

Work Rights

Not specified

Tailored Resume

Cover Letter