Data Engineer – Databricks

Air Products & Chemicals Inc

Pune, India
Hybrid
Databricks platform experience
Apache spark pyspark development
Aws cloud infrastructure
Air Products & Chemicals Inc is seeking a Senior Data Engineer with expertise in Databricks, AWS, and Apache Spark to build and scale cloud-native data platforms at their Pune location. The role involves collaborating on data engineering initiatives and migrating enterprise data workloads to Databricks, emphasizing a strong engineering culture and commitment to safety and diversity

Job Summary

  • The role involves migrating enterprise data workloads from AWS-based platforms to Databricks as part of a broader data modernization journey.
  • Candidates will build scalable production-grade data pipelines utilizing the Medallion architecture and Structured Streaming.
  • The position offers ownership of the Databricks Lakehouse architecture with opportunities to collaborate closely with AI/ML teams.

Matching Summary

Match Score: 85

Air Products & Chemicals Inc is seeking a Senior Data Engineer with expertise in Databricks, AWS, and Apache Spark to build and scale cloud-native data platforms at their Pune location. The role involves collaborating on data engineering initiatives and migrating enterprise data workloads to Databricks, emphasizing a strong engineering culture and commitment to safety and diversity.

Skills & Requirements

Must-have

  • Databricks platform experience
  • Apache Spark PySpark development
  • AWS cloud infrastructure
  • Delta Lake architecture
  • Terraform Infrastructure as Code
  • Batch and streaming pipelines

Nice-to-have

  • Unity Catalog governance
  • CI/CD pipeline familiarity
  • dbt or Airflow experience
  • Databricks certification preferred
  • Mentoring junior engineers

Key Requirements

  • 4-8 years Data Engineering experience
  • Strong hands-on Databricks on AWS
  • Experience with Terraform IAC
  • Production-grade pipeline development

Work Rights

Not specified

Tailored Resume

Cover Letter