Know ATS Score
CV/Résumé Score
  • Expertini Resume Scoring: Our Semantic Matching Algorithm evaluates your CV/Résumé before you apply for this job role: Data Engineering Lead SQL/Python.
India Jobs Expertini

Urgent! Data Engineering Lead - SQL/Python Job Opening In Bengaluru – Now Hiring Andor Tech(India) Pvt Ltd

Data Engineering Lead SQL/Python



Job description

<p><b>Role Overview :</b><br/><br/>As a Data Engineering Technical Lead, you will design and deliver robust data platforms using Databricks (Delta Live Tables, Structured Streaming, PySpark), Python, and SQL on Microsoft Azure.

You will work with to build, optimise data pipelines for optimal performance and cost, ensure governance and quality, and collaborate with cross-functional teams to unlock strategic insights from trading data.

</p><p><br/></p><p>You will be a key contributor to Shell's digital transformation, enabling smarter trading decisions and supporting the energy transition through data-driven innovation.<br/><br/><b>Key Responsibilities :</b><br/><br/><b>Technical Leadership :</b><br/><br/>- Architect, Design and implement scalable Azure Data Lakehouse solutions using ADF, ADLS, Synapse, Stream Analytics, and Databricks.<br/><br/>- Lead the development of real-time data pipelines using Databricks DLT and Structured Streaming for high-volume data ingestion and transformation.<br/><br/>- Optimise pipeline performance and cloud resource usage to ensure cost-effective operations.<br/><br/>- Leverage Unity Catalog for metadata management, data lineage, and governance.<br/><br/><b>Domain Expertise :</b><br/><br/>- Integrate and manage data from ETRM systems to support Crude & Products trading analytics.<br/><br/>- Apply functional knowledge of commodity trading (Oil & Gas) to align data solutions with business needs.<br/><br/>- Collaborate with trading desks, analysts, and IT teams to deliver actionable insights.<br/><br/><b>Governance & Quality :</b><br/><br/>- Establish and enforce data governance frameworks to ensure compliance, security, and data quality.<br/><br/>- Monitor pipeline performance and proactively resolve production issues.<br/><br/><b>Stakeholder Engagement :</b><br/><br/>- Communicate effectively with business and technical stakeholders, translating complex technical concepts into clear business goal/value.<br/><br/>- Lead executive-level discussions and presentations to align data initiatives with strategic priorities.<br/><br/><b>Team & Capability Building :</b><br/><br/>- Mentor and lead a high-performing team of data engineers and analysts.<br/><br/>- Drive continuous learning and capability development across Azure, Databricks, and streaming technologies.<br/><br/><b>Required Skills & Experience Expertise :</b></p><p><br/>- Deep experience with Databricks DLT, Structured Streaming, PySpark, Python, and SQL.</p><br/>- Strong knowledge of Azure data services (ADF, ADLS, Synapse, Stream Analytics).<br/><br/>- Proven ability to optimise Databricks for high-volume, real-time data processing.<br/><br/>- Experience with ETRM platforms and trading data workflows.<br/><br/></p><p><b>Leadership & Communication :<br/></b><br/>- Strong stakeholder management and executive communication skills.<br/><br/>- Demonstrated ownership, accountability, and passion for delivering impactful solutions.<br/><br/>- Ability to lead cross-functional teams and drive strategic data initiatives.<br/><br/><b>Education & Certification :</b></p><p><p><b><br/></b></p>- Bachelor's or Master's degree in IT or related discipline.<br/><br/>- 12+ years of experience in data engineering and analytics.<br/><br/>- Certifications in Azure Data Engineering and Databricks (preferred).<br/><br/><b>Nice to Have :</b><br/><br/>- Experience with Kafka, Hadoop, Airflow, and PowerBI/QlikSense.<br/><br/>- Familiarity with AI/ML, NLP, and GenAI concepts.<br/><br/>- Exposure to Agile/Kanban methodologies and CI/CD practices.</p> (ref:hirist.tech)


Required Skill Profession

Computer Occupations



Your Complete Job Search Toolkit

✨ Smart • Intelligent • Private • Secure

Start Using Our Tools

Join thousands of professionals who've advanced their careers with our platform

Rate or Report This Job
If you feel this job is inaccurate or spam kindly report to us using below form.
Please Note: This is NOT a job application form.


    Unlock Your Data Engineering Potential: Insight & Career Growth Guide