Data Engineer (gen Ai)

Interactive Brokers Group, Inc.

New York, NY, United States
Base: $150,000 to $200,000 py; bonus/equity: annua...
On-site
Aws cloud services (s3, glue, athena, emr, lambda)
Python for data processing and etl
Pyspark on emr for large-scale processing
Design, build, and maintain scalable data pipelines, data lake platforms, and analytics solutions that support enterprise-wide AI initiatives

Job Summary

  • Design, build, and maintain scalable data pipelines, data lake platforms, and analytics solutions that support enterprise-wide AI initiatives.
  • Develop Python-based applications for data ingestion, processing, and integration supporting Gen AI RAG workflows and knowledge base systems.
  • This is a Hybrid role (3 days in office / 2 days remote) with competitive salary, bonus, stock grant, and comprehensive benefits.

Matching Summary

Design, build, and maintain scalable data pipelines, data lake platforms, and analytics solutions that support enterprise-wide AI initiatives.

Salary

Base: $150,000 to $200,000 per year; Bonus/Equity: annual performance-based bonus and stock grant; Benefits: health and wellness benefits, 401(k) with match, paid time off, parental leave, education reimbursement

Skills & Requirements

Must-have

  • AWS cloud services (S3, Glue, Athena, EMR, Lambda)
  • Python for data processing and ETL
  • PySpark on EMR for large-scale processing
  • SQL for data analysis and transformation
  • Kafka for streaming data pipelines
  • Data lake architectures and Iceberg tables
  • CI/CD practices with Git and Docker

Nice-to-have

  • AI/ML technologies
  • Emerging data engineering trends
  • Software engineering best practices

Key Requirements

  • 6+ years of data engineering experience
  • Experience with modern data stack technologies
  • Experience building and maintaining ETL/ELT pipelines at scale
  • Understanding of data modeling and data warehousing concepts

Work Rights

Not specified

Tailored Resume

Cover Letter