ArabyAds - Data Engineer - Python Programming

<p><p><b>Responsibilities :</b></p><p><p><b><br/></b></p>- Design, Develop, Maintain and own data engineering projects built from requirements by different stakeholders, including but not limited to product teams, data scientists, data analysts and BI.<br/><br/></p><p>- Fulfill any requirement related to data pipelines, data warehouses and databases required by the application.<br/><br/></p><p>- Create ingestion pipelines to collect data from different sources and push to targets.<br/><br/></p><p>- Design and Create SQL, tables, procedures, functions etc.<br/><br/></p><p>- Operate and Maintain existing Hadoop Cluster.<br/><br/></p><p>- Write Spark Tasks/Applications mainly in Python/Java<br/><br/></p><p>- Evaluate, test and update data applications according to changing requirements.<br/><br/></p><p>- Monitor and report data systems health and security.<br/><br/></p><p>- Support internal teams to use our systems, troubleshoot & identify issues, and resolve them.<br/><br/></p><p>- Follow best coding practices and participate in the code review process.<br/><br/></p><p>- Use efficiently the development environment, compiling tools, version control and bug </p><p>tracking systems for all Competencies :</b></p><p><b><br/></b></p><p><b>Must Have :</b><br/><br/></p><p>- 3 - 5years of experience in Data Engineering.<br/><br/></p><p>- Good Knowledge/Experience in Python/Java, C# and .NET core.<br/><br/></p><p>- Good Knowledge/Experience Hadoop Stack, Oozie, Spark, Kafka<br/><br/></p><p>- Hands-on in writing SQL queries, Stored Procedures and Database code<br/><br/></p><p>- Good Knowledge with concepts, design and mechanics of traditional databases, data </p><p>warehouses and data lakes<br/><br/></p><p>- Good Knowledge with ETL, ELT, Transformation, Processing, Computation etc.<br/><br/></p><p>- Experience with relational databases such as MySQL, PostgreSQL, MariaDB.<br/><br/></p><p>- Good Knowledge/Experience with Git, CI/CD, and writing unit tests.<br/><br/></p><p>- Good Knowledge/Experience working on an agile development project.<br/><br/></p><p>- Good Knowledge/Experience with Jira and the Atlassian tool suite.<br/><br/></p><p>- Good Knowledge/Experience in Linux Shell to Have :</b></p><p><b><br/></b></p>- Experience with NoSQL databases such as Aerospike, DynamoDB, Redis, Elasticsearch, Cassandra, MongoDB<br/><br/></p><p>- Good knowledge of Snowflake databases, schemas, tables, procedures, functions, views, stages, pipes etc.<br/><br/></p><p>- Knowledge on one of the ETL tools<br/><br/></p><p>- Experience with performance and scalability issues.</p><p><b><br/></b></p><p><b>Non Technical Competencies :</b></p><p><p><b><br/></b></p>- Good English and communication skills.<br/><br/></p><p>- Positive and 'can do' attitude.<br/><br/></p><p>- Able to work independently as well as in a team.<br/><br/></p><p>- Able to take up challenges and come up with innovative solutions.<br/><br/></p><p>- We highly appreciate self-starters, initiators and contributors.</p><br/></p> (ref:hirist.tech)