Manage and maintain workspaces, clusters, pools, and jobs in Databricks or other big data systems, monitoring performance, usage, and costs to optimize cluster configurations
Job Summary
Manage and maintain workspaces, clusters, pools, and jobs in Databricks or other big data systems, monitoring performance, usage, and costs to optimize cluster configurations.
Configure role-based access control (RBAC) and audit logging, ensuring compliance with data governance and security standards.
Provide support to users like data scientists, engineers, and analysts, troubleshooting job failures, performance bottlenecks, and integration issues.
Matching Summary
Manage and maintain workspaces, clusters, pools, and jobs in Databricks or other big data systems, monitoring performance, usage, and costs to optimize cluster configurations.
Skills & Requirements
Must-have
Manage cloud data platforms
Strong knowledge of Apache Spark
Big data or related technologies
CI/CD pipelines for notebooks
Automate cluster lifecycle management
Support data scientists, engineers, analysts
Integrate Databricks with enterprise systems
Nice-to-have
Passion for enabling data teams
Proactive approach to management
Collaborative and creative culture
Commitment to sustainability
Key Requirements
Proven experience administering big data platforms
Experience with cloud platforms (Azure preferred)
Familiarity with Unity Catalog, Delta Lake
Proficiency in Python, SQL, and shell scripting
Experience with CI/CD tools and infrastructure-as-code