Know ATS Score
CV/Résumé Score
  • Expertini Resume Scoring: Our Semantic Matching Algorithm evaluates your CV/Résumé before you apply for this job role: Lead Data Pipeline Engineer.
India Jobs Expertini

Urgent! Lead Data Pipeline Engineer Job Opening In Hyderabad – Now Hiring Straive

Lead Data Pipeline Engineer



Job description

Job Summary

We're looking for a Senior Data Engineer with 5-8 years of experience to build and maintain scalable, production-grade data pipelines.

The ideal candidate is a strong software engineer with hands-on experience in Spark (3.X), Scala, SQL, and Python.

You'll be responsible for designing and implementing ETL/ELT solutions, collaborating with teams to deliver data products, and mentoring junior engineers.


Key Responsibilities

  • Design and build data pipelines using Apache Spark 3.X, Scala, Python, and SQL.
  • Tune Spark jobs for performance and cost.
  • Ensure code quality with unit tests, CI/CD, and code reviews.
  • Collaborate with platform and DevOps teams to deploy and monitor pipelines.
  • Troubleshoot production issues and perform root-cause analysis.


Qualifications

  • 5-8 years of experience as a Data Engineer.
  • Strong hands-on experience with Apache Spark 3.X.
  • Proficient in Scala, Python, and advanced SQL.
  • Experience with streaming frameworks (e.G., Spark Structured Streaming, Kafka).
  • Knowledge of orchestration tools (e.G., Apache Airflow).
  • Experience with cloud platforms like AWS Glue, EMR, Athena, Redshift, and S3 is a plus.
  • Solid understanding of software engineering fundamentals (Git, CI/CD).


Required Skill Profession

Computer Occupations



Your Complete Job Search Toolkit

✨ Smart • Intelligent • Private • Secure

Start Using Our Tools

Join thousands of professionals who've advanced their careers with our platform

Rate or Report This Job
If you feel this job is inaccurate or spam kindly report to us using below form.
Please Note: This is NOT a job application form.


    Unlock Your Lead Data Potential: Insight & Career Growth Guide