Big Data Engineer - Spark/Scala

<p><p><b>Role & responsibilities : </b><br/><br/>- Evaluate domain, financial and technical feasibility of solution ideas with help of all key stakeholders<br/><br/></p><p>- Design, develop, and maintain highly scalable data processing applications<br/><br/></p><p>- Write efficient, reusable and well documented code<br/><br/></p><p>- Deliver big data projects using Spark, Scala , Python, SQL<br/><br/></p><p>- Maintain and tune existing Spark applications to the fullest.<br/><br/></p><p>- Find opportunities for optimizing existing spark applications.<br/><br/></p><p>- Work closely with QA, Operations and various teams to deliver error free software on time<br/><br/></p><p>- Actively lead / participate daily agile / scrum meetings<br/><br/></p><p>- Take responsibility for Apache Spark development and implementation<br/><br/></p><p>- Translate complex technical and functional requirements into detailed designs<br/><br/></p><p>- Investigate alternatives for data storing and processing to ensure implementation of the most streamlined solutions<br/><br/></p><p>- Serve as a mentor for junior staff members by conducting technical training sessions and reviewing project outputs<br/><br/><b>Qualifications : </b><br/><br/>- Engineering graduates in computer science backgrounds preferred with 8+ years of software development experience with Hadoop framework components(HDFS, Spark, Spark, Scala, PySpark)<br/><br/></p><p>- Excellent at verbal, written and presentation skills<br/><br/></p><p>- Ability to present and defend a solution with technical facts & business proficiency<br/><br/></p><p>- Understanding of data-warehousing and data-modeling techniques<br/><br/></p><p>- Strong data engineering skills<br/><br/></p><p>- Knowledge of Core Java, Linux, SQL, and any scripting language<br/><br/></p><p>- At least 6+ years of experience using Python / Scala, Spark, SQL<br/><br/></p><p>- Knowledge of shell scripting is a plus<br/><br/></p><p>- Knowledge of Core and Advance Java is a plus.<br/><br/></p><p>- Experience in developing and tuning spark applications<br/><br/></p><p>- Excellent understanding of spark architecture, data frames and tuning spark<br/><br/></p><p>- Strong knowledge of database concepts, systems architecture, and data structures is a must Process oriented with strong analytical and problem solving skills<br/><br/></p><p>- Experience in writing Python standalone applications dealing with PySpark API<br/><br/></p><p>- Knowledge of DELTA.<br/><br/></p><p>- IO package is a plus<br/></p><br/></p> (ref:hirist.tech)