1.
Cloud & Infrastructure
- AWS services : Must be proficient in building scalable data pipelines and managing cloud-native ETL workflows.
- Snowflake: Moderate understanding of Snowflake architecture.
- CICD - Terraform or CloudFormation, Jenkins ,Bitbucket : For infrastructure-as-code and deployment automation.
2.
Programming & Scripting
- Python & PySpark: Ability to write efficient scripts for data transformation, and pipeline orchestration, knowledge of Spark or any distributing frameworks .
- SQL: Advanced querying, optimization, and data modelling.
3.
ETL & Data Modelling
- Familiarity with event-driven architectures, API -based data sources ,data quality validation, archival strategies, and incremental loading techniques.