The Data Lake team is responsible for data ingestion from internal source systems in batch/real-time modes, curation and governance of the data assets created in the platform
Job Summary
The Data Lake team is responsible for data ingestion from internal source systems in batch/real-time modes, curation and governance of the data assets created in the platform.
You will be an expert contributor and part of the Rating Organization’s Data Services Product Engineering Team with a unique opportunity to build and evolve S&P Ratings next gen data and analytics platform.
Our benefits include Health & Wellness, Flexible Downtime, Continuous Learning, Invest in Your Future, and Family Friendly Perks.
Matching Summary
The Data Lake team is responsible for data ingestion from internal source systems in batch/real-time modes, curation and governance of the data assets created in the platform.
Skills & Requirements
Must-have
Design & Build data pipelines
Data Lake solution on AWS
Databricks, Airflow, DuckDB
Data governance principles
Data quality checks
Data lineage implementation
Scala, Python and related libraries
Nice-to-have
Continuous learner
Emerging trends in data lake
Financial services industry experience
Knowledge sharing and collaboration
User community experience enhancement
Key Requirements
5+ years of experience building solutions in big data technologies
3+ years of experience in data transformation and analysis
Strong foundational understanding on data models and PL/SQL
Experience in continuous delivery through CI/CD pipelines
BE, MCA or MS degree in Computer Science or Information Technology