Apache Spark-Data Engineer

EXCELTECH COMPUTERS PTE. LTD.

Singapore
Not specified
Apache spark for large-scale processing
Sql query development and optimization
Aws s3 cloud storage implementation
EXCELTECH COMPUTERS PTE. LTD. is seeking an Apache Spark Data Engineer to design and manage scalable data pipelines, leveraging AWS and tools like SQL, Collibra, and Apache Airflow. The ideal candidate should have strong experience with data processing, cloud solutions, and Agile methodologies

Job Summary

  • The role involves designing and managing scalable data pipelines using Apache Spark for large-scale data processing.
  • Candidates will implement cloud-based storage solutions on AWS and ensure data quality using Collibra.
  • The position requires orchestrating automated data workflows with Apache Airflow within an Agile delivery team.

Matching Summary

Match Score: 85

EXCELTECH COMPUTERS PTE. LTD. is seeking an Apache Spark Data Engineer to design and manage scalable data pipelines, leveraging AWS and tools like SQL, Collibra, and Apache Airflow. The ideal candidate should have strong experience with data processing, cloud solutions, and Agile methodologies.

Skills & Requirements

Must-have

  • Apache Spark for large-scale processing
  • SQL query development and optimization
  • AWS S3 cloud storage implementation
  • Collibra data governance management
  • Presto interactive data querying
  • Apache Airflow workflow orchestration

Nice-to-have

  • Agile delivery team experience
  • Iterative improvement mindset
  • Stakeholder requirement management

Key Requirements

  • Experience with Apache Spark and SQL
  • Proficiency in AWS services and S3
  • Knowledge of Collibra for data governance
  • Familiarity with Presto and Airflow
  • Ability to work in Agile teams

Work Rights

Not specified

Tailored Resume

Cover Letter