Coducer Technologies - Senior Data Engineer - Azure/PySpark

<p>Job Summary : </p><p><br/></p><p>We are seeking a highly skilled Senior Data Engineer with 3-5 years of professional experience, specializing in the <b>Azure Databricks platform.</b> The ideal candidate will have a strong track record of designing, building, and optimizing scalable data pipelines.

You will play a crucial role in enhancing our data infrastructure to support critical business intelligence and advanced analytics.<br/><br/>Key Responsibilities :<br/><br/>- Design, develop, and maintain robust data pipelines on Azure Databricks using PySpark.<br/><br/>- Implement and manage ETL/ELT processes for both batch and streaming data.<br/><br/>- Work with data scientists and analysts to understand their data requirements and deliver reliable data solutions.<br/><br/>- Optimize Databricks clusters and jobs for performance and cost efficiency.<br/><br/>- Ensure data quality, integrity, and security across all data pipelines.<br/><br/>Required Skills & Qualifications : <br/><br/>- Bachelor's degree in Computer Science, Engineering, or a related field.<br/><br/>- <b>3-5 years of demonstrable experience as a Data Engineer with a strong focus on Azure Databricks mandatory .</b><br/><br/>- Proficient in PySpark and the Spark ecosystem.<br/><br/>- Experience with Delta Lake for building data lakes and managing ACID transactions is a must.<br/><br/>- Strong command of SQL and solid understanding of data warehousing concepts.<br/><br/>- Hands-on experience with other Azure data services, including Azure Data Lake Storage (ADLS) and Azure Synapse Analytics.<br/><br/>- Excellent problem-solving and communication skills.</p> (ref:hirist.tech)