Experience - 8+ years
Role - Data Engineer(Tech Lead / Lead Consultant)
Location - Bangalore, Hyderabad, Chennai
Responsibilities:
Key Responsibility Areas:
- Design, build, and manage scalable and reliable data pipelines using Python or PySpark and Azure services.
- Lead a team of data engineers;
mentor junior team members and enforce best practices in data engineering. - Collaborate with data scientists, analysts, and business stakeholders to understand data requirements.
- Develop and maintain data models, ETL/ELT workflows, and data orchestration pipelines.
- Optimize performance of data solutions, ensuring efficient data access, transformation, and storage.
- Oversee the implementation and maintenance of Azure Data Factory pipelines for automated data movement.
- Ensure proper data governance, security, and compliance across all data platforms.
- Participate in architectural design and decision-making for enterprise-scale data platforms.
Key Skills & Qualifications:
Must-Have Skills:
- Strong proficiency in Python or PySpark for data engineering and ETL development.
- Advanced SQL skills for data transformation and query optimization.
- Deep experience with:
- Azure Data Lake Storage (Gen2)
- Azure Data Factory (ADF)
- Azure Databricks
- Experience with performance tuning and optimizing big data solutions.
- Strong understanding of data warehousing concepts, data modelling, and data governance.