Senior Data Engineer

Skillfield

Melbourne, Australia
Hybrid
Apache spark development
Scala programming
Hadoop ecosystem experience
Skillfield is a specialist technology consultancy operating at the intersection of cyber, data, and AI, partnering with enterprise clients to deliver scalable solutions

Job Summary

  • Skillfield is a specialist technology consultancy operating at the intersection of cyber, data, and AI, partnering with enterprise clients to deliver scalable solutions.
  • The Senior Data Engineer will design and maintain high-performance data pipelines across cloud and on-prem environments using technologies like Apache Spark, Hadoop, and ClickHouse.
  • Employees enjoy a collaborative, outcome-driven consulting environment with flexibility, support, and a focus on sustainable delivery.

Matching Summary

Skillfield is a specialist technology consultancy operating at the intersection of cyber, data, and AI, partnering with enterprise clients to deliver scalable solutions.

Skills & Requirements

Must-have

  • Apache Spark development
  • Scala programming
  • Hadoop ecosystem experience
  • ETL and ELT pipeline design
  • Linux environment operation
  • Distributed systems optimisation

Nice-to-have

  • Kafka or streaming platform experience
  • Databricks or EMR exposure
  • Orchestration tools like Airflow
  • Familiarity with data security and governance
  • Git-based workflows and CI/CD pipelines

Key Requirements

  • Hands-on Apache Spark experience
  • Strong Scala development skills
  • Experience with HDFS, Hive, and HBase
  • Knowledge of distributed systems
  • Experience in cloud, hybrid, or on-prem platforms

Work Rights

Not specified

Tailored Resume

Cover Letter