Lead Aws Software Data Engineer

London Stock Exchange Group

Not specified; not specified; healthcare, retireme...
Python code with type hints and tests
Apache spark for large-scale processing
Aws tools glue emr serverless lambda s3
The role involves designing and developing scalable, testable data pipelines using Python and Apache Spark within a modern AWS environment

Job Summary

  • The role involves designing and developing scalable, testable data pipelines using Python and Apache Spark within a modern AWS environment.
  • Candidates will collaborate closely with product and business teams to translate requirements into reliable, high-quality data solutions for FTSE Russell.
  • LSEG offers a culture of diversity and innovation where employees can grow their skills through continuous learning and meaningful career opportunities.

Matching Summary

The role involves designing and developing scalable, testable data pipelines using Python and Apache Spark within a modern AWS environment.

Salary

Not specified; Not specified; Healthcare, retirement planning, paid volunteering days and wellbeing initiatives

Skills & Requirements

Must-have

  • Python code with type hints and tests
  • Apache Spark for large-scale processing
  • AWS tools Glue EMR Serverless Lambda S3
  • Relational databases Aurora PostgreSQL
  • CI/CD version control modular design

Nice-to-have

  • Experience with Apache Iceberg table formats
  • Familiarity with Great Expectations or Deequ
  • Curiosity about financial markets and indices
  • Exposure to Databricks platform
  • Collaboration in Agile teams

Key Requirements

  • Strong experience in relational databases
  • Understanding of batch processing and schema evolution
  • Ability to work in Agile teams

Work Rights

Not specified

Tailored Resume

Cover Letter