Ifc - Cloud Data Engineer

Barclays

Pune, India
Aws databricks, s3, ec2, lambda, rds, glue, athena
Data pipelines and step functions
Distributed computing architecture
Build and maintain systems that collect, store, process, and analyse data, such as data pipelines, data warehouses and data lakes to ensure that all data is accurate, accessible, and secure

Job Summary

  • Build and maintain systems that collect, store, process, and analyse data, such as data pipelines, data warehouses and data lakes to ensure that all data is accurate, accessible, and secure.
  • Design and implementation of data warehoused and data lakes that manage the appropriate data volumes and velocity and adhere to the required security measures.
  • Spearhead the evolution of our infrastructure and deployment pipelines, driving innovation and operational excellence by harnessing cutting-edge technology to build and manage robust, scalable and secure infrastructure.

Matching Summary

Build and maintain systems that collect, store, process, and analyse data, such as data pipelines, data warehouses and data lakes to ensure that all data is accurate, accessible, and secure.

Skills & Requirements

Must-have

  • AWS Databricks, S3, EC2, Lambda, RDS, Glue, Athena
  • Data pipelines and Step Functions
  • Distributed computing architecture
  • Core Hadoop components (HDFS, Spark, Yarn, HIVE, Impala)
  • Expertise in SQL, advanced SQL skills
  • Analytical services knowledge

Nice-to-have

  • Shell scripting and programming (Scala/Python)
  • ETL implementation on Hadoop
  • Technical Design for ETL/Hadoop/AWS components
  • Continuous improvement
  • Risk and controls
  • Business acumen
  • Strategic thinking

Key Requirements

  • Experience with AWS services
  • Experience with distributed computing architecture
  • Experience with Hadoop components
  • Advanced SQL skills
  • Experience with Scala/Python

Work Rights

Not specified

Tailored Resume

Cover Letter