Open source components (hadoop, spark, flink, kafka, hbase)
Participate in the overall architecture design of the big data platform, covering the full link of data collection, storage, computing, analysis, and visualization, ensuring system high availability, high performance, and scalability
Job Summary
Participate in the overall architecture design of the big data platform, covering the full link of data collection, storage, computing, analysis, and visualization, ensuring system high availability, high performance, and scalability.
Evaluate and introduce cutting-edge technologies (such as AI large models, cloud-native technologies) to optimize data processing efficiency and resource utilization.
Lead medium to large data platform projects (such as data warehouses, data lakes, real-time analysis systems), from participation in requirement analysis to landing and delivery.
Matching Summary
Participate in the overall architecture design of the big data platform, covering the full link of data collection, storage, computing, analysis, and visualization, ensuring system high availability, high performance, and scalability.
Skills & Requirements
Must-have
Big data platform architecture
Cloud platform technologies (Aliyun/AWS)
Open source components (Hadoop, Spark, Flink, Kafka, HBase)
Distributed system design
Real-time computing engines
Performance tuning and troubleshooting
Data governance and compliance
Nice-to-have
AI large models
Cloud-native technologies
Data-driven decision support
Innovative solutions
Collaborative working culture
Ethical business practices
Key Requirements
8+ years of big data experience
3+ years of big data architecture design experience