The role involves designing and developing scalable data pipelines using Azure Data Factory and Databricks to ingest and transform data across multiple domains including US Healthcare and Insurance
Job Summary
The role involves designing and developing scalable data pipelines using Azure Data Factory and Databricks to ingest and transform data across multiple domains including US Healthcare and Insurance.
Candidates must leverage AI-assisted tools like Databricks Assistant, Genie, and Copilot to enhance solution delivery efficiency and client outcomes.
The team serves as a technical hub for the organization, requiring strong collaboration with stakeholders to deliver high-quality software solutions in an Agile environment.
Matching Summary
The role involves designing and developing scalable data pipelines using Azure Data Factory and Databricks to ingest and transform data across multiple domains including US Healthcare and Insurance.
Skills & Requirements
Must-have
Azure Data Factory (ADF)
Azure Databricks
Python and PySpark programming
Complex SQL query optimization
Data pipeline architecture
Unity Catalog proficiency
Nice-to-have
Machine Learning model experience
AI agent integration skills
HPCC and ECL language knowledge
Talend tool familiarity
Positive attitude and learning desire
Key Requirements
10+ years of total professional experience
6+ years of relevant Analytics Engineering experience