Design and maintain robust and efficient data pipeline architecture, constructing large, complex data sets to fulfill business requirements
Job Summary
Design and maintain robust and efficient data pipeline architecture, constructing large, complex data sets to fulfill business requirements.
Manage and administer a wide range of AWS services, focusing on ingestion, transformation, and secure storage in cloud-based systems.
Collaborate effectively with internal and external stakeholders, managing expectations and expertly handling all inquiries, while acting as an effective team player and mentoring junior staff.
Matching Summary
Design and maintain robust and efficient data pipeline architecture, constructing large, complex data sets to fulfill business requirements.
Salary
Base: 100,000/year - 120,000/year CAD; Bonus/Equity: Not specified; Benefits: Not specified
Skills & Requirements
Must-have
Big Data technologies (Spark, Hive, Presto)
AWS services (S3, EC2, EMR)
Python, Scala, Java, Shell scripting
Data warehousing and data modeling
Performance tuning for Spark and Hive
Agile methodologies
Nice-to-have
Equity Markets business knowledge
Terraform, Ansible experience
Kafka experience
Machine learning technology stack design
Strategic thinking and multitasking
Key Requirements
4+ years in Big Data technologies
5-8 years overall IT experience
University degree, College diploma or relevant job experience