Data Engineer

FINALTO ASIA PTE. LTD.

Singapore
Pyspark and sql proficiency
Databricks lakehouse architecture
Apache spark structured streaming
The role involves building and optimizing scalable data pipelines using PySpark and SQL for batch and streaming data

Job Summary

  • The role involves building and optimizing scalable data pipelines using PySpark and SQL for batch and streaming data.
  • Candidates will design ETL/ELT processes and implement data solutions within the Databricks Lakehouse and Delta Lake ecosystem.
  • The position requires managing infrastructure with Terraform while ensuring data quality, governance, and reliability for analytics teams.

Matching Summary

Match Score: 85

The role involves building and optimizing scalable data pipelines using PySpark and SQL for batch and streaming data.

Skills & Requirements

Must-have

  • PySpark and SQL proficiency
  • Databricks Lakehouse architecture
  • Apache Spark Structured Streaming
  • Infrastructure as Code Terraform
  • Kafka and CDC data ingestion

Nice-to-have

  • Strong communication skills
  • Agile development environment experience
  • CI/CD and modular development
  • Scala programming language knowledge
  • High level of ownership

Key Requirements

  • Demonstrated experience in data engineering
  • Hands-on experience with Apache Spark
  • Experience with cloud platforms AWS Azure or GCP
  • Familiarity with CI/CD practices
  • Proven ability to build scalable data solutions

Work Rights

Not specified

Tailored Resume

Cover Letter