Lead Data Engineer - Azure Databricks

<p><p><b>Posting title :</b> Lead Data Engineer (Databricks)<br/><br/><b>Experience :</b> 7+ Years<br/><br/><b>Location :</b> Pune/ Ahmedabad<br/><br/><b>Work mode :</b> Hybrid<br/><br/><b>Primary skills :</b> Databricks, Data Modelling/ Data Lakehouse, Gen AI, SQL, Python/Scala, Azure Data Factory (ADF)/AWS Glue, Docker, Team Lead<br/><br/><b>Qualification :</b> Any Technical Degree</p><p><br/></p><p><b>Roles & Responsibilities :</b><br/><br/>- Lead the design, development, and optimization of data solutions using Databricks, ensuring they are scalable, efficient, and secure.<br/><br/>- Collaborate with cross-functional teams to gather and analyse data requirements, translating them into robust data architectures and solutions.<br/><br/>- Develop and maintain ETL pipelines, leveraging Databricks and integrating with Azure Data Factory as needed.<br/><br/>- Implement machine learning models and advanced analytics solutions, incorporating Generative AI to drive innovation.<br/><br/>- Ensure data quality, governance, and security practices are adhered to, maintaining the integrity and reliability of data solutions.<br/><br/>- Provide technical leadership and mentorship to junior engineers, fostering an environment of learning and growth.<br/><br/>- Stay updated on the latest trends and advancements in data engineering, Databricks, Generative AI, and Azure Data Factory to continually enhance team Qualifications :</b><br/><br/>- Bachelors or masters degree in computer science, Information Technology, or a related field.<br/><br/>- 7+ to 10 years of experience in data engineering, with a focus on Databricks.<br/><br/>- Proven expertise in building and optimizing data solutions using Databricks and integrating with Azure Data Factory/AWS Glue.<br/><br/>- Proficiency in SQL and programming languages such as Python or Scala.<br/><br/>- Strong understanding of data modelling, ETL processes, and Data Warehousing/Data Lakehouse concepts.<br/><br/>- Familiarity with cloud platforms, particularly Azure, and containerization technologies such as Docker.<br/><br/>- Excellent analytical, problem-solving, and communication skills.<br/><br/>- Demonstrated leadership ability with experience mentoring and guiding junior team members.</p><p><br/></p><p><b>Preferred Skills :</b><br/><br/>- Experience with Generative AI technologies and their applications.<br/><br/>- Familiarity with other cloud platforms, such as AWS or GCP.<br/><br/>- Knowledge of data governance frameworks and tools.</p><br/></p> (ref:hirist.tech)