Glencore Senior Data Platform Engineer CI/CD Pipeline

Job description

<p><p><b>Senior Data Platform Engineer</b><br/><br/> Experience 10+ Years.<br/><br/><b>THE ROLE :</b><br/><br/>- As a Senior Data Platform Engineer, you will lead the design and governance of scalable, secure, and compliant data architectures on Microsoft Azure.<br/><br/>- You will work across oil trading, refining, and retail fuel operations to align cloud data strategies with business goals, regulatory requirements, and industry best practices.<br/><br/>- You will play a hands-on technical role within the data engineering team, interfacing with the CloudOps team in London to design and implement cost effective solutions across our suite of Azure products adhering to best practice principles.<br/><br/>- Experience in DevOps is highly desirable as you will be responsible for designing CI/CD pipelines using Github Actions and containerization technologies.<br/><br/>- In addition, the role will involve exploring new technologies and navigating implementation within Glencores private Azure tenant, according to the guidelines enforced at Group level.<br/><br/>- With these in mind, you will play an advisory role across the Assets, assisting with design and implementation, and on-premises to cloud migrations as required.<br/><br/>- This role sits within the Oil Assets IT team, reporting to the Data Engineering Team Lead.<br/><br/><b>PROFILE :</b><br/><br/>- Experienced IT professional with over 10 years of experience, including 24 years leading cloud-native data architecture initiatives.<br/><br/>- Proven track record in designing and delivering scalable, end-to-end data platform solutions across hybrid and multi-cloud environments.<br/><br/>- Adept at driving best practices in data engineering, modernizing legacy systems, and advising stakeholders on strategic data platform decisions.<br/><br/>- Strong focus on performance, reliability, and governance across enterprise-grade data ecosystems.<br/><br/>- Skilled in setting up foundational components, enforcing engineering standards, and enabling team-wide productivity through automation, CI/CD, and reusable frameworks.<br/><br/>- Plays a key leadership role in a growing data engineering team, working alongside senior and junior data engineers.<br/><br/><b>RESPONSIBILITIES :</b><br/><br/><b>Platform Engineering & Architecture :</b><br/><br/>- Design and implement scalable, secure, and cost-effective data platform solutions using Azure Synapse Analytics, Azure Data Lake, and related services.<br/><br/>- Translate business and regulatory requirements into cloud-native data architectures.<br/><br/>- Contribute to the architectural roadmap, including the adoption and integration of Microsoft Fabric for unified data experiences.<br/><br/>- Define and enforce platform standards, patterns, and reusable components to accelerate delivery across the team.<br/><br/><b>- CI/CD, Automation & Containerization</b><br/><br/>- Lead the implementation of CI/CD pipelines for data engineering workflows using Azure DevOps and Git.<br/><br/>- Automate infrastructure provisioning and deployment using Infrastructure as Code (e.g., Bicep, Terraform).<br/><br/>- Develop and maintain containerized data services using Docker, and orchestrate deployments using Kubernetes (AKS).<br/><br/>- Establish automated testing, validation, and monitoring practices to ensure data quality and platform reliability.<br/><br/><b>Team Enablement & Collaboration :</b><br/><br/>- Collaborate closely with the other Senior Engineer to co-lead technical direction and ensure consistency across projects.<br/><br/>- Mentor and support data engineers and developers, providing guidance on best practices, code reviews, and technical upskilling.<br/><br/>- Work cross-functionally with Data Architects, Analysts, and Product Owners to translate business needs into robust data solutions.<br/><br/><b>Governance, Security & Operations :</b><br/><br/>- Implement data governance and lineage practices using Microsoft Purview and Fabric-native tools.<br/><br/>- Ensure compliance with security and privacy standards through RBAC, encryption, and secure data handling.<br/><br/>- Monitor platform performance and proactively address operational issues and bottlenecks.<br/><br/><b>Innovation & Continuous Improvement :</b><br/><br/>- Stay current with Microsofts evolving data ecosystem, including Fabric, and evaluate new tools and services for team adoption.<br/><br/>- Drive continuous improvement in engineering workflows, platform scalability, and developer experience.<br/><br/>- Maintain understanding of the latest trends and advancements in AI and machine learning, and advise on incorporation of these innovations across the department, alongside governance.<br/><br/><b>COMPETENCIES :</b><br/><br/>- Bachelors degree in Computer Science, Information Technology, or equivalent experience.<br/><br/>- Excellent understanding of core data modelling concepts and techniques.<br/><br/>- Ability to manage stakeholders and develop effective relationships across different cultures.<br/><br/> Cloud & Platform : </p><p><br/></p><p>- Azure Synapse Analytics, Microsoft Fabric (emerging), Azure Data Lake Gen2, Azure SQL, Azure DatabricksReal-Time & IoT: Azure Event Hubs, Azure Stream Analytics, Azure IoT Hub.<br/><br/> Data Engineering : </p><p><br/></p><p>- Azure Data Factory, Synapse Pipelines, Spark, Delta Lake, T-SQL, Python.<br/><br/> Governance and Security : </p><p><br/></p><p>- Microsoft Purview, Key Vault, RBAC, Data Masking, Encryption.<br/><br/> Monitoring and Observability : </p><p><br/></p><p>- Azure Monitor, Log Analytics, Application Insights.<br/><br/> Governance and Security : </p><p><br/></p><p>- Microsoft Purview, Key Vault, RBAC, Data Masking, Encryption.<br/><br/> Monitoring and Observability : </p><p><br/></p><p>- Azure Monitor, Log Analytics, Application Insights.<br/><br/> DevOps and Automation : </p><p><br/></p><p>- Azure DevOps, Git/GitHub, CI/CD, Bicep, Terraform, YAML pipelines.<br/><br/> Containerization and Orchestration : </p><p><br/></p><p>- Docker, Kubernetes (AKS), Helm.<br/><br/><b>DESIRABLE :</b><br/><br/>- Exposure to DataOps practices and awareness of advanced Data Architecture patterns.<br/><br/>- Certifications: Azure Solutions Architect, Azure Data Engineer Associate, Azure DevOps Engineer Expert.<br/><br/>- Visualisation & BI: Power BI, Azure Analysis Services.<br/><br/>- Machine Learning: Azure ML.<br/></p><br/></p> (ref:hirist.tech)

Required Skill Profession

Computer Occupations