Azure Data Engineer - Python/SQL/ETL

<p><p><b>About the Opportunity : </b></p><br/><p>We are looking for an experienced Azure Data Engineer to design and implement high-performance data integration and transformation pipelines within the Microsoft Azure ecosystem.</p><br/><p>This role requires strong hands-on expertise in Azure Data Factory (ADF), Databricks (PySpark), and SQL, with a solid understanding of big data processing and schema-on-read principles.</p><br/><p>The ideal candidate will work closely with analytics and BI teams to build robust ETL/ELT pipelines, optimize data flow, and ensure scalability, accuracy, and security across enterprise data environments.</p><br/><p><b>Key Responsibilities : </b></p><br/><p>- Design, build, and manage ETL/ELT data pipelines using Azure Data Factory (ADF) and Azure Databricks (PySpark).</p><br/><p>- Develop scalable data ingestion, transformation, and processing solutions from multiple sources into Azure Data Lake or Synapse.</p><br/><p>- Implement data validation, monitoring, and automation processes to ensure consistency and reliability.</p><br/><p>- Collaborate with architects and data modelers to support data warehouse and data lake design.</p><br/><p>- Optimize data pipelines for performance, scalability, and cost efficiency in Azure environments.</p><br/><p>- Write complex SQL queries for data profiling, quality checks, and analytics enablement.</p><br/><p>- Integrate Azure Synapse Analytics for advanced data orchestration and warehousing use cases.</p><br/><p>- Work with reporting tools such as Power BI or Tableau to enable visualization and business reporting.</p><br/><p>- Ensure adherence to data governance, security, and compliance standards within the Azure cloud ecosystem.</p><br/><p><b>Required Skills and Qualifications : </b></p><br/><p>- Minimum of 5 years of experience in data engineering, with at least 3+ years in Azure-based solutions.</p><br/><p>- Strong proficiency in SQL, including complex joins, stored procedures, and performance tuning.</p><br/><p>- Hands-on expertise with : </p><br/><p> - Azure Data Factory (ADF) for ETL/ELT workflows.</p><br/><p> - Azure Databricks using PySpark for distributed data processing.</p><br/><p> - Azure Data Lake Storage (ADLS Gen2) for scalable storage and schema-on-read architecture.</p><br/><p>- Understanding of big data principles, data pipelines, and schema-on-read methodologies.</p><br/><p>- Experience developing and automating data transformations, monitoring, and CI/CD workflows.</p><br/><p>- Strong scripting skills in Python (PySpark) for data manipulation and automation.</p><br/><p>- Working knowledge of Power BI or Tableau for dashboarding and reporting.</p><br/><p>- Good understanding of data governance, access control, and cloud security best practices.</p><br/><p>- Excellent problem-solving, communication, and collaboration skills.</p><br/><p><b>Preferred Skills : </b></p><br/><p>- Experience with Azure Synapse Analytics or Azure SQL Data Warehouse.</p><br/><p>- Knowledge of data orchestration and workflow tools (Airflow, Prefect).</p><br/><p>- Exposure to Power BI DAX and data modeling for analytics.</p><br/><p>- Familiarity with CI/CD pipelines using Azure DevOps or GitHub Actions.</p><br/><p>- Certifications such as Microsoft Certified : Azure Data Engineer Associate are a plus</p><br/></p> (ref:hirist.tech)