Build and maintain systems that collect, store, process, and analyse data, such as data pipelines, data warehouses and data lakes to ensure that all data is accurate, accessible, and secure
Job Summary
Build and maintain systems that collect, store, process, and analyse data, such as data pipelines, data warehouses and data lakes to ensure that all data is accurate, accessible, and secure.
Design and implement scalable and efficient data transformation/storage solutions using Snowflake and develop processing and analysis algorithms fit for the intended data complexity and volumes.
Join as a Data Engineer responsible for supporting the successful delivery of Location Strategy projects to plan, budget, agreed quality and governance standards.
Matching Summary
Build and maintain systems that collect, store, process, and analyse data, such as data pipelines, data warehouses and data lakes to ensure that all data is accurate, accessible, and secure.
Skills & Requirements
Must-have
pyspark, Dataframes, RDD, SparkSQL
AWS Cloud development
AWS Glue, S3, Lambda, Lake formation, Athena
Snowflake data solutions
DBT with Snowflake
Advanced SQL and PL SQL
Enterprise data warehouse implementation
Nice-to-have
Data governance and lineage tools
Orchestration tools like Airflow
Ab Initio ETL tool knowledge
Stakeholder engagement and requirement elicitation
Infrastructure understanding and solutioning
Data Marts and Data Warehousing concepts
Analytical and interpersonal skills
Key Requirements
Hands on experience with pyspark
Hands on Experience in developing applications on AWS Cloud
Strong hold on AWS Data Analytics Technology Stack
Design and implement solutions using Snowflake
Experience in Data ingestion to Snowflake
Experience in using DBT with snowflake
Experience in Writing advanced SQL and PL SQL programs
Worked on at least two major project implementations