Data Consultant Databricks Data Engineer

Job description

This job is with Kyndryl, an inclusive employer and a member of myGwork – the largest global platform for the LGBTQ+ business community.

Please do not contact the recruiter directly.

Who We Are
At Kyndryl, we design, build, manage and modernize the mission-critical technology systems that the world depends on every day.

So why work at Kyndryl?

We are always moving forward - always pushing ourselves to go further in our efforts to build a more equitable, inclusive world for our employees, our customers and our communities.

The Role
At Kyndryl, we design, build, manage and modernize the mission-critical technology systems that the world depends on every day.

So why work at Kyndryl?

We are always moving forward - always pushing ourselves to go further in our efforts to build a more equitable, inclusive world for our employees, our customers and our communities.
As a Data Engineer, you will leverage your expertise in Databricks, big data platforms, and modern data engineering practices to develop scalable data solutions for our clients.

Candidates with healthcare experience, particularly with EPIC systems, are strongly encouraged to apply.

This includes creating data pipelines, integrating data from various sources, and implementing data security and privacy measures.

The Data Engineer will also be responsible for monitoring and troubleshooting data flows and optimizing data storage and processing for performance and cost efficiency.
Responsibilities:
Develop data ingestion, data processing and analytical pipelines for big data, relational databases and data warehouse solutions

Design and implement data pipelines and ETL/ELT processes using Databricks, Apache Spark, and related tools.

Collaborate with business stakeholders, analysts, and data scientists to deliver accessible, high-quality data solutions.

Provide guidance on cloud migration strategies and data architecture patterns such as Lakehouse and Data Mesh

Provide pros/cons, and migration considerations for private and public cloud architectures

Provide technical expertise in troubleshooting, debugging, and resolving complex data and system issues.

Create and maintain technical documentation, including system diagrams, deployment procedures, and troubleshooting guides

Experience working with Data Governance, Data security and Data Privacy (Unity Catalogue or Purview)

Your Future at Kyndryl
Every position at Kyndryl offers a way forward to grow your career.

We have opportunities that you won't find anywhere else, including hands-on experience, learning opportunities, and the chance to certify in all four major platforms.

Whether you want to broaden your knowledge base or narrow your scope and specialize in a specific sector, you can find your opportunity here.

Who You Are
You're good at what you do and possess the required experience to prove it.

However, equally as important - you have a growth mindset; keen to drive your own personal and professional development.

You are customer-focused - someone who prioritizes customer success in their work.

And finally, you're open and borderless - naturally inclusive in how you work with others.
Required Technical and Professional Experience:
3+ years of consulting or client service delivery experience on Azure

Graduate/Postgraduate in computer science, computer engineering, or equivalent with minimum of 8 years of experience in the IT industry.

3+ years of experience in developing data ingestion, data processing and analytical pipelines for big data, relational databases such as SQL server and data warehouse solutions such as Azure Synapse

Extensive hands-on experience implementing data ingestion, ETL and data processing.

Hands-on experience in and Big Data technologies such as Java, Python, SQL, ADLS/Blob, PySpark and Spark SQL, Databricks, HD Insight and live streaming technologies such as EventHub.

Experience with cloud-based database technologies (Azure PAAS DB, AWS RDS and NoSQL).

Cloud migration methodologies and processes including tools like Azure Data Factory, Data Migration Service, etc.

Experience with monitoring and diagnostic tools (SQL Profiler, Extended Events, etc).

Expertise in data mining, data storage and Extract-Transform-Load (ETL) processes.

Experience with relational databases and expertise in writing and optimizing T-SQL queries and stored procedures.

Experience in using Big Data File Formats and compression techniques.

Experience in Developer tools such as Azure DevOps, Visual Studio Team Server, Git, Jenkins, etc.

Experience with private and public cloud architectures, pros/cons, and migration considerations.

Excellent problem-solving, analytical, and critical thinking skills.

Ability to manage multiple projects simultaneously, while maintaining a high level of attention to detail.

Communication Skills: Must be able to communicate with both technical and nontechnical.

Able to derive technical requirements with the stakeholders.

Preferred Technical And Professional Experience
Cloud platform certification, e.g., Microsoft Certified: (DP-700) Azure Data Engineer Associate, AWS Certified Data Analytics - Specialty, Elastic Certified Engineer, Google Cloud Professional Data Engineer

Professional certification, e.g., Open Certified Technical Specialist with Data Engineering Specialization.

Experience working with EPIC healthcare systems (e.g., Clarity, Caboodle).

Databricks certifications (e.g., Databricks Certified Data Engineer Associate or Professional).

Knowledge of GenAI tools, Microsoft Fabric, or Microsoft Copilot.

Familiarity with healthcare data standards and compliance (e.g., HIPAA, GDPR).

Experience with DevSecOps and CI/CD deployments

Experience in NoSQL databases design

Knowledge on , Gen AI fundamentals and industry supporting use cases.

Hands-on experience with Delta Lake and Delta Tables within the Databricks environment for building scalable and reliable data pipelines.

Being You
Diversity is a whole lot more than what we look like or where we come from, it's how we think and who we are.

We welcome people of all cultures, backgrounds, and experiences.

But we're not doing it single-handily: Our Kyndryl Inclusion Networks are only one of many ways we create a workplace where all Kyndryls can find and provide support and advice.

This dedication to welcoming everyone into our company means that Kyndryl gives you - and everyone next to you - the ability to bring your whole self to work, individually and collectively, and support the activation of our equitable culture.

That's the Kyndryl Way.

What You Can Expect
With state-of-the-art resources and Fortune 100 clients, every day is an opportunity to innovate, build new capabilities, new relationships, new processes, and new value.

Kyndryl cares about your well-being and prides itself on offering benefits that give you choice, reflect the diversity of our employees and support you and your family through the moments that matter - wherever you are in your life journey.

Our employee

learning programs

give you access to the best learning in the industry to receive certifications,

including Microsoft,

Google, Amazon, Skillsoft, and many more.

Through our company-wide volunteering and giving platform, you can donate, start fundraisers, volunteer, and search over 2 million non-profit organizations.

At Kyndryl, we invest heavily in you, we want you to succeed so that together, we will all succeed.

Get Referred!
If you know someone that works at Kyndryl, when asked 'How Did You Hear About Us' during the application process, select 'Employee Referral' and enter your contact's Kyndryl email address.

Required Skill Profession

Computer Occupations