96631 Data Engineer (PySpark+Python)

Job description

<table class="Table" style="width:1448.35pt; border-collapse:collapse; border:undefined" width="1931"> <tbody> <tr style="height:16.0pt"> <td nowrap="nowrap" style="width:1448.35pt; padding:0cm 5.4pt 0cm 5.4pt; height:16.0pt" width="1931"> <p style="text-indent:12.05pt"><b>Experience:</b></p> </td> </tr> <tr style="height:16.0pt"> <td nowrap="nowrap" style="width:1448.35pt; padding:0cm 5.4pt 0cm 5.4pt; height:16.0pt" width="1931"> <p style="text-indent:24.0pt">5 10 years of hands-on experience in the data analytics space as a data engineer.

Familiar with ETL, DQ, DM, and reject and recycling concepts.</p> </td> </tr> <tr style="height:16.0pt"> <td nowrap="nowrap" style="width:1448.35pt; padding:0cm 5.4pt 0cm 5.4pt; height:16.0pt" width="1931"> <p style="text-indent:24.0pt">A significant portion of this experience should involve building data analytics solutions in a big data environment using Hadoop clusters or cloud environment.</p> </td> </tr> <tr style="height:16.0pt"> <td nowrap="nowrap" style="width:1448.35pt; padding:0cm 5.4pt 0cm 5.4pt; height:16.0pt" width="1931"> <p style="text-indent:12.05pt"><b>Technical Skills:</b></p> </td> </tr> <tr style="height:16.0pt"> <td nowrap="nowrap" style="width:1448.35pt; padding:0cm 5.4pt 0cm 5.4pt; height:16.0pt" width="1931"> <p style="text-indent:24.0pt">Extensive experience in building data pipelines using Spark, particularly PySpark.</p> </td> </tr> <tr style="height:16.0pt"> <td nowrap="nowrap" style="width:1448.35pt; padding:0cm 5.4pt 0cm 5.4pt; height:16.0pt" width="1931"> <p style="text-indent:150.0pt"> Candidates must have a minimum of 3 years of hands-on experience in coding with PySpark applications using RRDs, DataFrames & datasets and NOT Spark SQLs.</p> </td> </tr> <tr style="height:16.0pt"> <td nowrap="nowrap" style="width:1448.35pt; padding:0cm 5.4pt 0cm 5.4pt; height:16.0pt" width="1931"> <p style="text-indent:150.0pt"> Candide should has developed numerous spark application for various use cases of processing large volumes of data, used performance tuning, works extensively on complex transformation skills using group, window,</p> </td> </tr> <tr style="height:16.0pt"> <td nowrap="nowrap" style="width:1448.35pt; padding:0cm 5.4pt 0cm 5.4pt; height:16.0pt" width="1931"> <p style="text-indent:150.0pt"> Candidate who has participated in PySpark code hackathon</p> </td> </tr> <tr style="height:16.0pt"> <td nowrap="nowrap" style="width:1448.35pt; padding:0cm 5.4pt 0cm 5.4pt; height:16.0pt" width="1931"> <p style="text-indent:150.0pt"> Please apply only if you are confident in writing decent PySpark code during the interview.</p> </td> </tr> <tr style="height:16.0pt"> <td nowrap="nowrap" style="width:1448.35pt; padding:0cm 5.4pt 0cm 5.4pt; height:16.0pt" width="1931"> <p style="text-indent:24.0pt">Strong proficiency in Python.</p> </td> </tr> <tr style="height:16.0pt"> <td nowrap="nowrap" style="width:1448.35pt; padding:0cm 5.4pt 0cm 5.4pt; height:16.0pt" width="1931"> <p style="text-indent:150.0pt"> <i>Write clean, efficient, and reusable Python code</i></p> </td> </tr> <tr style="height:16.0pt"> <td nowrap="nowrap" style="width:1448.35pt; padding:0cm 5.4pt 0cm 5.4pt; height:16.0pt" width="1931"> <p style="text-indent:150.0pt"> <i>Identify, troubleshoot, and fix bugs in programs to ensure code qualit</i></p> </td> </tr> <tr style="height:16.0pt"> <td nowrap="nowrap" style="width:1448.35pt; padding:0cm 5.4pt 0cm 5.4pt; height:16.0pt" width="1931"> <p style="text-indent:150.0pt"> <i>Creating scripts and tools to automate tasks and processes </i></p> </td> </tr> <tr style="height:16.0pt"> <td nowrap="nowrap" style="width:1448.35pt; padding:0cm 5.4pt 0cm 5.4pt; height:16.0pt" width="1931"> <p style="text-indent:150.0pt"> <i>Note:</i> This role requires advanced Python coding skills, not just basic knowledge or simple coding experience.

Candidates will be required to demonstrate their Python skills during the interview.</p> </td> </tr> <tr style="height:16.0pt"> <td nowrap="nowrap" style="width:1448.35pt; padding:0cm 5.4pt 0cm 5.4pt; height:16.0pt" width="1931"> <p style="text-indent:24.0pt">Familiarity or exposure to tools such as Airflow, Databricks, and Azure is a plus, but not mandatory.

The primary focus is on Spark, PySpark, and data engineering.</p> </td> </tr> <tr style="height:16.0pt"> <td nowrap="nowrap" style="width:1448.35pt; padding:0cm 5.4pt 0cm 5.4pt; height:16.0pt" width="1931"> <p style="text-indent:12.05pt"><b>Additional Skills:</b></p> </td> </tr> <tr style="height:16.0pt"> <td nowrap="nowrap" style="width:1448.35pt; padding:0cm 5.4pt 0cm 5.4pt; height:16.0pt" width="1931"> <p style="text-indent:24.0pt">Strong problem-solving and analytical abilities.</p> </td> </tr> <tr style="height:16.0pt"> <td nowrap="nowrap" style="width:1448.35pt; padding:0cm 5.4pt 0cm 5.4pt; height:16.0pt" width="1931"> <p style="text-indent:24.0pt">Ability to comprehend business requirements and translate them into technical solutions.</p> </td> </tr> <tr style="height:16.0pt"> <td nowrap="nowrap" style="width:1448.35pt; padding:0cm 5.4pt 0cm 5.4pt; height:16.0pt" width="1931"> <p style="text-indent:24.0pt">Good communication and collaboration skills to work effectively with team members.</p> </td> </tr> <tr style="height:16.0pt"> <td nowrap="nowrap" style="width:1448.35pt; padding:0cm 5.4pt 0cm 5.4pt; height:16.0pt" width="1931"> <p style="text-indent:24.0pt">Familiarity with the software development lifecycle, including CI/CD pipelines.</p> </td> </tr> <tr style="height:16.0pt"> <td nowrap="nowrap" style="width:1448.35pt; padding:0cm 5.4pt 0cm 5.4pt; height:16.0pt" width="1931"> <p style="text-indent:24.0pt">Experience working in an Agile environment.</p> </td> </tr> </tbody> </table>

Required Skill Profession

Computer Occupations