Job description
Key Responsibilities
• Administer and optimize Azure Databricks, including clusters, jobs, and user access.
• Manage Snowflake environments (warehouses, roles/permissions, performance tuning, cost optimization).
• Design, implement, and monitor Azure Data Factory (ADF) pipelines for batch and streaming workloads.
• Enable data movement and replication across global environments.
• Orchestrate pipelines using ADF, Apache Airflow, and automation frameworks.
• Automate processes and reduce manual effort with Python, PySpark, SQL, and PowerShell.
• Work with Azure Data Lake Storage (ADLS), Function Apps, and Key Vault for secure and efficient data operations.
• Ensure compliance with security, governance, and cost management best practices.
• Collaborate with Data Engineering, DNA, and Data Science teams to ensure high availability and reliable data delivery.
• Troubleshoot platform issues, implement fixes, and contribute to runbooks and documentation.
• Partner with architects and engineers to continuously improve platform scalability, reliability, and efficiency.
Required Skills and Experience
• 5+ years of experience in data platform engineering/administration.
• Strong expertise in Azure Databricks, Snowflake, and Azure Data Factory.
• Experience on Azure Synapse, Event Hubs, or other Azure data services.
• Hands-on experience with data pipelines, orchestration, and automation.
• Proficiency in SQL, Python, PySpark, and scripting (PowerShell/Bash).
• Experience with Apache Airflow for scheduling and orchestration.
• Strong understanding of Azure services including ADLS, Function Apps, Key Vault, Networking, and Monitoring.
• Familiarity with CI/CD pipelines and Infrastructure as Code (Terraform/ARM).
• Solid knowledge of data governance, security, and cost optimization practices
Soft Skills
• Strong problem-solving and analytical skills.
• Ability to work independently and in a global, collaborative team environment.
• Effective communication skills for technical and non-technical stakeholders.
• Proactive and adaptable mindset with focus on continuous improvement.
Nice to Have
• Experience in DevOps practices for data engineering.
• Knowledge of data governance and compliance frameworks (e.g., GDPR, PCI).
• Familiarity with agile methodologies (JIRA, Confluence).
• Microsoft Azure certifications (e.g., DP-203, AZ-204, AZ-104, AZ-305).
Required Skill Profession
Computer Occupations