Senior Data Engineer - Azure Databricks

<p><p><b>Description :</b><br/><br/>About the Role :</p><p><br/>We are seeking an experienced and highly skilled Senior Data Engineer Azure Databricks to join our data engineering team.<br/><br/></p><p>In this role, you will be responsible for designing, building, and maintaining scalable data pipelines and solutions on Azure using Databricks, with a strong focus on performance, reliability, and data quality.<br/>You will work closely with data architects, analysts, and business stakeholders to support data-driven decision-making and digital transformation initiatives.<br/><br/><b>Key Responsibilities : </b></p><p><br/></p><p>- Design and develop scalable ETL/ELT data pipelines using Azure Databricks, Spark, and Azure Data Factory.<br/><br/></p><p>- Work with structured and unstructured data, implementing data ingestion, transformation, cleansing, and enrichment logic.<br/><br/></p><p>- Develop and optimize big data processing jobs using PySpark, Scala, or SQL in Databricks.<br/><br/></p><p>- Collaborate with cross-functional teams to understand business requirements and translate them into efficient data solutions.<br/><br/></p><p>- Ensure data quality, governance, and security by implementing best practices for data validation, monitoring, and access control.<br/><br/></p><p>- Participate in the design of modern data architecture including data lakes, lakehouses, and data warehouses.<br/><br/></p><p>- Optimize and tune data pipelines for performance and cost-efficiency.<br/><br/></p><p>- Deploy data solutions using CI/CD pipelines, manage version control, and support DevOps practices.<br/><br/></p><p>- Mentor junior engineers and support best practices across the data engineering team.<br/><br/></p><p>- Stay up to date with the latest Azure services and Databricks features to recommend enhancements and innovations.<br/><br/><b>Required Qualifications :</b></p><p><p><b><br/></b></p>- Bachelors or Masters degree in Computer Science, Engineering, or a related field.<br/><br/></p><p>- 5+ years of experience in data engineering or big data development roles.<br/><br/></p><p>- Strong hands-on experience with Azure Databricks and Apache Spark.<br/><br/></p><p>- Proficiency in PySpark, Scala, or SQL for data processing and transformation.<br/><br/></p><p>- Solid experience with Azure Data Factory, Azure Data Lake, Blob Storage, and Azure Synapse Analytics.<br/><br/></p><p>- Experience with Delta Lake and Lakehouse architecture.<br/><br/></p><p>- Understanding of data modeling, partitioning strategies, and schema design for big data solutions.<br/><br/></p><p>- Experience with CI/CD, version control (e.g., Git), and deployment automation in data environments.<br/><br/></p><p>- Familiarity with data security, RBAC, encryption, and compliance practices in Azure.<br/><br/></p><p>- Strong problem-solving, communication, and collaboration skills</p><br/></p> (ref:hirist.tech)