Build, secure, and automate the enterprise Data Platform on AWS, going beyond basic pipeline creation to design and maintain underlying infrastructure and CI/CD frameworks
Job Summary
Build, secure, and automate the enterprise Data Platform on AWS, going beyond basic pipeline creation to design and maintain underlying infrastructure and CI/CD frameworks.
Implement and manage cloud infrastructure, Unity Catalog, security controls, and CI/CD pipelines using Terraform, GitLab CI, and Databricks Asset Bundles.
Ensure platform stability through observability, serve as an escalation point for incidents, and share knowledge by documenting best practices and conducting workshops.
Matching Summary
Build, secure, and automate the enterprise Data Platform on AWS, going beyond basic pipeline creation to design and maintain underlying infrastructure and CI/CD frameworks.
Skills & Requirements
Must-have
Databricks Lakehouse architecture
Unity Catalog implementation
Terraform for AWS infrastructure
GitLab CI/CD pipelines
Databricks Asset Bundles
Python and Bash scripting
Nice-to-have
Mentoring and collaboration skills
Upskilling data engineers
CloudWatch integration
Key Requirements
5+ years industry experience
3+ years with Databricks
Bachelor's in computer science or related field
Databricks Certified Data Engineer Professional (preferred)