Data Engineer (PySpark)

PERSOL SINGAPORE PTE. LTD.

Singapore, Singapore
Pyspark or scala expertise
Talend to spark migration experience
Aws or azure cloud services
The role involves analyzing existing Talend ETL jobs and designing equivalent Spark-based pipelines for migration and optimization

Job Summary

  • The role involves analyzing existing Talend ETL jobs and designing equivalent Spark-based pipelines for migration and optimization.
  • Candidates will collaborate with data architects to define scalable data lake and lakehouse solutions using cloud platforms.
  • The position requires hands-on experience across all phases of the project lifecycle with involvement in at least four project cycles.

Matching Summary

Match Score: 85

The role involves analyzing existing Talend ETL jobs and designing equivalent Spark-based pipelines for migration and optimization.

Skills & Requirements

Must-have

  • PySpark or Scala expertise
  • Talend to Spark migration experience
  • AWS or Azure cloud services
  • Databricks platform implementation
  • Informatica IDMC development
  • Airflow orchestration tools

Nice-to-have

  • Lakehouse architecture design
  • Systems impact analysis skills
  • BI project collaboration
  • Technical design leadership

Key Requirements

  • Minimum 5 years data engineering experience
  • Degree in Computer Science or related field
  • Implementation in at least 4 project cycles

Work Rights

Not specified

Tailored Resume

Cover Letter