Sr. Databricks Data Engineer, Onsite Avp

State Street UK

Toronto, Canada
$91,000 - $140,500 annual; not specified; not spec...
On-site
Databricks sql & scala/pyspark
Data lakehouse architecture
Etl pipelines on cloud
Design & Develop custom high throughput and configurable data pipelines using DataBricks

Job Summary

  • Design & Develop custom high throughput and configurable data pipelines using DataBricks.
  • Design, build and test end to end data ingestion pipelines (streaming, events, and batch), data integration, data curation.
  • Optimize and tune Databricks job performance, including query optimization, resource management, and scaling strategies.

Matching Summary

Design & Develop custom high throughput and configurable data pipelines using DataBricks.

Salary

$91,000 - $140,500 Annual; Not specified; Not specified

Skills & Requirements

Must-have

  • Databricks SQL & Scala/PySpark
  • Data Lakehouse architecture
  • ETL pipelines on cloud
  • DevOps pipelines and Azure services
  • Micro Services - API/Event Driven Architecture

Nice-to-have

  • Financial industry experience
  • Agile development methodology
  • Cross-geographical collaboration

Key Requirements

  • 7+ years of overall IT experience
  • 5+ years of Data Engineering experience
  • 3+ years of Databricks hands-on experience
  • 3+ years of cloud-based development experience (Azure/AWS)
  • Bachelor’s Degree in computer or IT related subject

Work Rights

Not specified

Tailored Resume

Cover Letter