Lead Data Engineer

Karbon

Melbourne, Australia
On-site
7+ years data engineering experience
Databricks architecture expertise
Dbt for data modeling
Karbon is seeking a Lead Data Engineer to architect a new unified data platform on Databricks that powers their AI tools and insights

Job Summary

  • Karbon is seeking a Lead Data Engineer to architect a new unified data platform on Databricks that powers their AI tools and insights.
  • The role involves designing scalable Medallion Architecture using dbt and managing resilient data pipelines for both batch and real-time processing.
  • Engineers are expected to balance delivery speed with high-quality standards while leveraging AI tools to enhance productivity and innovation.

Matching Summary

Karbon is seeking a Lead Data Engineer to architect a new unified data platform on Databricks that powers their AI tools and insights.

Skills & Requirements

Must-have

  • 7+ years data engineering experience
  • Databricks architecture expertise
  • dbt for data modeling
  • Python and SQL proficiency
  • Batch and streaming pipeline development
  • Azure cloud ecosystem experience
  • Terraform infrastructure as code

Nice-to-have

  • Snowflake replication experience
  • MLOps and MLflow implementation
  • Feature store management
  • API endpoint development
  • AI tool integration enthusiasm
  • Cross-functional team mentorship

Key Requirements

  • 7+ years relevant work experience
  • Extensive Databricks experience
  • Strong Python and SQL skills
  • DevOps and CI/CD experience
  • Experience scaling multi-cloud pipelines

Work Rights

Not specified

Tailored Resume

Cover Letter