Principal Engineer - Scala/ETL

<p><p><b>Description :</b></p><br/><p><b>Responsibilities :</b></p><br/><p>- Lead technical initiatives and guide the team to develop innovative software solutions that address complex challenges.</p><br/><p>- Build scalable, efficient, and high-performance pipelines and workflows for processing large volumes of batch and real-time data.</p><br/><p>- Maintain and enhance existing software systems to ensure performance and reliability.</p><br/><p>- Recommend and implement technology upgrades to drive continuous improvement.</p><br/><p>- Support real-time streams, ETL pipelines, data warehouses, and reporting services.</p><br/><p>- Design and develop data frameworks, applications, and microservices that seamlessly integrate with other services.</p><br/><p>- Utilize Big Data tools such as Kafka, AWS S3 Data Lake, EMR, and Spark to ingest, store, transform, and query data.</p><br/><p>- Adhere to coding best practices, including unit testing, design/code reviews, and comprehensive documentation.</p><br/><p>- Conduct thorough code reviews to maintain quality, mentor junior team members, and promote continuous learning.</p><br/><p>- Perform performance analyses and capacity planning for each release.</p><br/><p>- Work effectively as part of an Agile team, contributing to process improvements and innovative solutions.</p><br/><p>- Implement and promote security protocols and data governance standards across development projects.</p><br/><p>- Proactively introduce new approaches to overcome software challenges throughout the product lifecycle.</p><br/><p><b>Requirements :</b></p><br/><p>- Strong software design skills with a deep understanding of design patterns and performance optimization.</p><br/><p>- Expertise in writing high-quality, well-structured Scala code with an emphasis on functional programming and test-driven development.</p><br/><p>- Ability to produce clear, concise, and organized documentation.</p><br/><p>- Knowledge of Amazon cloud computing services (Aurora MySQL, DynamoDB, EMR, Lambda, Step Functions, and S3).</p><br/><p>- Excellent communication skills and the ability to collaborate effectively with team members of varying technical backgrounds.</p><br/><p>- Proficiency in conducting detailed code reviews focused on improving code quality and mentoring developers.</p><br/><p>- Familiarity with software engineering and project management tools.</p><br/><p>- Commitment to following security protocols and best practices in data governance.</p><br/><p>- Capability to construct KPIs and use metrics for continuous process improvement.</p><br/><p><b>Minimum qualifications :</b></p><br/><p>- 15+ years of experience designing and developing enterprise-level software solutions.</p><br/><p>- 5+ years of experience with large volume data processing and Big Data tools such as Apache Spark, Scala, Hadoop, and Snowflake.</p><br/><p>- 5+ years of experience developing Scala/Java applications and microservices using Spring Boot.</p><br/><p>- 5+ years of experience working with SQL and relational databases.</p><br/><p>- 2+ years of experience working within Agile/Scrum environments.</p><br/></p> (ref:hirist.tech)