Senior Data Engineer - Scala/Python

<p><p><b>Job Description TYPE : </b> Full-Time, Permanent</p><br/><b>LOCATION : </b> Remote (Pan India)<br/><br/><b>SHIFT TIMINGS : </b> 2 : 00 pm-11 : 00 pm IST<br/><br/><b>Budget</b>- As per company standards<br/><br/><b>REPORTING : </b> This position will report to our CEO or any other Lead as assigned by Management.<br/><br/>The Senior Data Engineer will be responsible for building and extending our data pipeline architecture, as well as optimizing data flow and collection for cross-functional teams.

The ideal candidate is an experienced data pipeline builder and data wrangler who enjoys working with big data and building systems from the ground up.<br/><br/>You will collaborate with our software engineers, database architects, data analysts, and data scientists to ensure our data delivery architecture is consistent throughout the platform.

You must be self-directed and comfortable supporting the data needs of multiple teams, systems and products.

The right candidate will be excited by the prospect of optimizing or even re-designing our companys data architecture to support our next generation of products and data initiatives.<br/><br/><b>What Youll Be Doing : </b><br/><br/>- Design and build parts of our data pipeline architecture for extraction, transformation, and loading of data from a wide variety of data sources using the latest Big Data technologies.<br/><br/>- Identify, design, and implement internal process improvements : automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.<br/><br/>- Work with stakeholders including the Executive, Product, Data and Design teams to assist with data-related technical issues and support their data infrastructure needs.<br/><br/>- Work with machine learning, data, and analytics experts to drive innovation, accuracy and greater functionality in our data system.<br/><br/><b>Qualifications : </b><br/><br/>- Bachelor's degree in Engineering, Computer Science, or relevant field.<br/><br/>- 10+ years of relevant and recent experience in a Data Engineer role.<br/><br/>- 5+ years recent experience with Apache Spark and solid understanding of the fundamentals.<br/><br/>- Deep understanding of Big Data concepts and distributed systems.<br/><br/>- Strong coding skills with Scala, Python, Java and/or other languages and the ability to quickly switch between them with ease.<br/><br/>- Advanced working SQL knowledge and experience working with a variety of relational databases such as Postgres and/or MySQL.<br/><br/>- Cloud Experience with Data Bricks<br/><br/>- Experience working with data stored in many formats including Delta Tables, Parquet, CSV and JSON.<br/><br/>- Comfortable working in a Linux shell environment and writing scripts as needed.<br/><br/>- Comfortable working in an Agile environment<br/><br/>- Machine Learning knowledge is a plus</p><br/></p> (ref:hirist.tech)