Build and maintain scalable AI data pipelines using Databricks, Spark, and cloud-native tools (e.g., Azure, AWS)
Job Summary
Build and maintain scalable AI data pipelines using Databricks, Spark, and cloud-native tools (e.g., Azure, AWS).
Develop and evaluate ML models using classical and deep learning techniques, including GenAI, LLMs, SLMs, and Retrieval-Augmented Generation (RAG).
Collaborate with product, legal, and policy teams to ensure regulatory compliance and create reusable templates, frameworks, and documentation to accelerate ML development across teams.
Matching Summary
Build and maintain scalable AI data pipelines using Databricks, Spark, and cloud-native tools (e.g., Azure, AWS).
Salary
Base: $190,200 -- $345,650 annually; Bonus/Equity: Not specified; Benefits: Not specified
Skills & Requirements
Must-have
Databricks, Spark, cloud-native tools
backend services and platform components
classical and deep learning techniques
GenAI, LLMs, SLMs, RAG
synthetic data generation, differential privacy
CI/CD, version control, code reviews
Python and ML frameworks
Nice-to-have
document intelligence, OCR, NLP
vector databases, modern NLP techniques
contributions to open-source ML tools
culture of technical excellence and inclusion
Key Requirements
Graduate degree (MS or Ph.D.)
10+ years of experience in ML engineering or platform roles
Experience deploying ML models in production
Familiarity with cloud platforms (Azure preferred)