The role involves designing and developing custom high-throughput data pipelines using Databricks to process large volumes of data in near real-time
Job Summary
The role involves designing and developing custom high-throughput data pipelines using Databricks to process large volumes of data in near real-time.
Candidates will be responsible for the full software development lifecycle including coding, testing, deployment, and documentation of scalable ETL solutions.
State Street offers a comprehensive benefits program including retirement savings with company match, insurance coverage, and paid time off.
Matching Summary
The role involves designing and developing custom high-throughput data pipelines using Databricks to process large volumes of data in near real-time.
Salary
Base: $90,000 - $157,500 Annual; Bonus/Equity: Eligible for annual performance-based awards; Benefits: Comprehensive package including 401K match, insurance, and PTO
Skills & Requirements
Must-have
Hands-on Databricks SQL development
Scala and PySpark programming expertise
Azure cloud-based infrastructure experience
End-to-end data pipeline design
DevOps pipeline troubleshooting skills
Microservices and event-driven architecture
Nice-to-have
Strong critical thinking and problem-solving
Experience in financial industry domain
Agile development methodology background
Cross-geographical stakeholder collaboration
Positive attitude and self-motivation
Key Requirements
Bachelor's degree in Computer Science or IT related field