Sr. Databricks Data Engineer, Onsite Avp

State Street Corporation

Toronto, Canada
$91,000 - $140,500 annual py
On-site
Databricks sql & scala/pyspark
High volume data processing
Near real-time data processing
Design & Develop custom high throughput and configurable data pipelines using DataBricks

Job Summary

  • Design & Develop custom high throughput and configurable data pipelines using DataBricks.
  • Design, build and test end to end data ingestion pipelines(streaming, events, and batch), data integration, data curation.
  • Optimize and tune Databricks job performance, including query optimization, resource management, and scaling strategies.

Matching Summary

Design & Develop custom high throughput and configurable data pipelines using DataBricks.

Salary

$91,000 - $140,500 Annual

Skills & Requirements

Must-have

  • Databricks SQL & Scala/PySpark
  • High volume data processing
  • Near real-time data processing
  • End-to-end data ingestion pipelines
  • Scalable ETL pipelines on cloud

Nice-to-have

  • Micro Services - API/Event Driven Architecture
  • Collaboration and influence
  • Agile development methodology
  • Financial industry experience

Key Requirements

  • 7+ years of overall IT experience
  • 5+ years of Data Engineering experience
  • 3+ years of Databricks hands-on experience
  • 3+ years of cloud-based development (Azure/AWS)

Work Rights

Not specified

Tailored Resume

Cover Letter