Proven experience in SQL, Spark, Hadoop ecosystem<br/><br/></p><p>- Have worked on multiple TBs of data volume from ingestion to consumption.<br/><br/></p><p>- Work with business stakeholders to identify and document high impact business problems and potential solutions.<br/><br/></p><p>- Good understanding of Data Lake/Lakehouse architecture and experience/exposure to Hadoop (cloudera, hortonworks) and/or AWS.<br/><br/></p><p>- Work on end-to-end data lifecycle from Data Ingestion, Data Transformation and Data Consumption layer.
Versed with API and its usability.<br/><br/></p><p>- Suitable candidate will also be proficient in Spark, Spark Streaming, hive, SQLs.<br/><br/></p><p>- Suitable candidate will also demonstrate experience with big data infrastructure inclusive of MapReduce, Hive, HDFS, YARN, HBase, Oozie, etc.<br/><br/></p><p>- The candidate will additionally demonstrate substantial experience and a deep knowledge of relational databases.<br/><br/></p><p>- Good skills in technical debugging of the code in case of issues.
Also, working with git for code versioning<br/><br/></p><p>- Creating Technical Design Documentation of the projects/pipelines</p><br/></p> (ref:hirist.tech)