Staff Software Engineer (data Platform)

Arcana.io Inc

Bangalore, India
On-site
Real-time data ingestion and processing
Large-scale enterprise data solutions
Production data pipelines with pyspark
Develop and implement real-time data ingestion and processing systems

Job Summary

  • Develop and implement real-time data ingestion and processing systems.
  • Design, build, and operationalize large-scale enterprise data solutions and applications.
  • Create and manage production data pipelines, from ingestion to consumption, within a big data architecture using PySpark.

Matching Summary

Develop and implement real-time data ingestion and processing systems.

Skills & Requirements

Must-have

  • real-time data ingestion and processing
  • large-scale enterprise data solutions
  • production data pipelines with PySpark
  • Python, Airflow, SQL, cloud platforms
  • big data architecture
  • ETL processes, data integration

Nice-to-have

  • computer science fundamentals
  • data structures and algorithms
  • system design expertise
  • compute and I/O-intensive applications

Key Requirements

  • 8+ years of professional experience
  • Bachelor's Degree in Computer Science or similar
  • Proficiency in SQL and Data Warehouse Concepts
  • Experience with SQL or NoSQL databases
  • Knowledge of AWS technologies and big data tools

Work Rights

Not specified

Tailored Resume

Cover Letter