Azure Data Engineer - SQL/PySpark

<p><p><b>Description :</b><br/><br/>Exp - 6-10 Years<br/><br/>Location - Chennai ,Hyderabad , Bangalore<br/><br/>Notice : Immediate to 30 days only<br/><br/><b>Skill :</b> Azure Data Engineer<br/><br/><b>Must have :</b> Azure Databricks, Pysparks, SQL, Devops(CI/CD), Azure services(ADF, ADLS, Event Hub)<br/><br/><b>Good to have :</b> Data modelling, Gen AI, Agentic AI, Streaming Analytics<br/><br/><b>Key Responsibilities :</b></p><p><p><b><br/></b></p><br/>- Design and implement scalable, metadata-driven frameworks for data ingestion, quality, and transformation across both batch and streaming datasets.<br/><br/></p><p><br/>- Develop and optimize end-to-end data pipelines to process structured and unstructured data, enabling the creation of analytical data products.<br/><br/></p><p><br/>- Build robust exception handling, logging, and monitoring mechanisms for better observability and operational support.<br/><br/></p><p><br/>- Take ownership of complex modules and lead the development of critical data workflows and components.<br/><br/></p><p><br/>- Provide guidance to data engineers and peers on best practices.<br/><br/></p><p><br/>- Collaborate with cross-functional teams including business consultants, data architects, scientists, and application developers to deliver impactful analytics solutions.<br/><br/><b>EXPERTISE AND QUALIFICATIONS :</b></p><p><p><b><br/></b></p><br/>- 5+ years of overall technical experience, with a minimum of 2 years of hands-on experience with Microsoft Azure and Databricks.<br/><br/></p><p><br/>- Proven experience delivering at least one end-to-end Data Lakehouse solution on Azure Databricks using the Medallion Architecture.<br/><br/></p><p><br/>- Strong working knowledge of the Databricks ecosystem, including: PySpark, Notebooks, Structured Streaming, Unity Catalog, Delta Live Tables, Workflows, and SQL Warehouse.<br/><br/></p><p><br/>- Advanced programming, unit testing, and debugging skills in Python and SQL.<br/><br/></p><p><br/>- Hands-on experience with Azure-native services such as: Azure Data Factory, ADLS Gen2, Azure SQL Database, and Event Hub.<br/><br/></p><p><br/>- Solid understanding of data modeling techniques, including both Dimensional and Third Normal Form (3NF) models.<br/><br/></p><p><br/>- Exposure to developing LLM/Generative AI-powered applications.<br/><br/></p><p><br/>- Must have excellent understanding of CI/CD workflows using Azure DevOps.<br/><br/></p><p><br/>- Bonus: Knowledge of Azure infrastructure, including provisioning, networking, security, and governance.<br/><br/></p><br/></p> (ref:hirist.tech)