Azure Databricks/PySpark Developer - ETL

<p><p><b>Key Responsibilities :</b><br/><br/></p><p>- Design, develop, and maintain scalable data pipelines and ETL/ELT processes using PySpark, SQL, and Python.<br/><br/></p><p>- Build and manage data workflows on Azure Data Lake, Azure Data Factory (ADF), and Databricks.<br/><br/></p><p>- Collaborate with data scientists, analysts, and other stakeholders to understand data needs and ensure data quality and availability.<br/><br/></p><p>- Implement data modeling, transformation, and integration solutions from various structured and unstructured data sources.<br/><br/></p><p>- Design and implement CI/CD pipelines for data engineering projects using industry best practices.<br/><br/></p><p>- Monitor and optimize performance of data systems, ensuring high availability and reliability.<br/><br/></p><p>- Document data workflows, schemas, and architectures to support knowledge sharing and maintenance.<br/><br/></p><p>- Actively participate in code reviews and contribute to a culture of continuous improvement.<br/><br/></p><p>- Communicate effectively with technical and non-technical stakeholders, translating business requirements into technical Qualifications :</b></p><p><br/></p>- 6+ years of hands-on experience in Data Engineering.<br/><br/></p><p>- Proficiency in SQL, Python, and PySpark for data processing and transformation.<br/><br/></p><p>- Experience working with Azure Data Lake Storage, Azure Data Factory, and Databricks.<br/><br/></p><p>- Solid understanding of data warehousing concepts, data modeling, and data integration.<br/><br/></p><p>- Experience with CI/CD tools and deployment practices (e.g., Azure DevOps, Git, Jenkins).<br/><br/></p><p>- Strong analytical, problem-solving, and debugging skills.<br/><br/></p><p>- Excellent verbal and written communication skills.<br/><br/></p><p>- Strong team player with the ability to work independently when needed</p><br/></p> (ref:hirist.tech)