Straive - Senior Data Engineer - ETL/Python

<p><p><b>Description :</b></p><br/><p><b>About the job :</b></p><br/><p><b>Job Summary :</b></p><br/><p>We're looking for a Senior Data Engineer with 5-8 years of experience to build and maintain scalable, production-grade data pipelines.</p><br/><p>The ideal candidate is a strong software engineer with hands-on experience in Spark (3.x), Scala, SQL, and Python.</p><br/><p>You'll be responsible for designing and implementing ETL/ELT solutions, collaborating with teams to deliver data products, and mentoring junior engineers.</p><br/><p><b>Key Responsibilities :</b></p><br/><p>- Design and build data pipelines using Apache Spark 3.x, Scala, Python, and SQL.</p><br/><p>- Tune Spark jobs for performance and cost.</p><br/><p>- Ensure code quality with unit tests, CI/CD, and code reviews.</p><br/><p>- Collaborate with platform and DevOps teams to deploy and monitor pipelines.</p><br/><p>- Troubleshoot production issues and perform root-cause analysis.</p><br/><p><b>Qualifications :</b></p><br/><p>- 5 to 8 years of experience as a Data Engineer.</p><br/><p>- Strong hands-on experience with Apache Spark 3.x.</p><br/><p>- Proficient in Scala, Python, and advanced SQL.</p><br/><p>- Experience with streaming frameworks (e.g., Spark Structured Streaming, Kafka).</p><br/><p>- Knowledge of orchestration tools (e.g., Apache Airflow).</p><br/><p>- Experience with cloud platforms like AWS Glue, EMR, Athena, Redshift, and S3 is a plus.</p><br/><p>- Solid understanding of software engineering fundamentals (Git, CI/CD)</p><br/></p> (ref:hirist.tech)