Job Description
<p>Job Description :<br/><br/>Job Title : Lead GCP Data Engineer<br/><br/>Skills required : GCP DE Experience, Big query, SQL, Cloud compressor/Python, Cloud functions, Dataproc+pyspark, Python injection, Dataflow+PUB/SUB<br/><br/>Notice period : Only Immediate to 15 days joiners<br/><br/>Job Requirement :<br/><br/>- Have Implemented and Architected solutions on Google Cloud Platform using the components of GCP<br/><br/>- Experience with Apache Beam/Google Dataflow/Apache Spark in creating end to end data pipelines.<br/><br/>- Experience in some of the following : Python, Hadoop, Spark, SQL, Big Query, Big Table Cloud Storage, Datastore, Spanner, Cloud SQL, Machine Learning.<br/><br/>- Experience programming in Java, Python, etc.<br/><br/>- Expertise in at least two of these technologies : Relational Databases, Analytical Databases, NoSQL databases.<br/><br/>- Certified in Google Professional Data Engineer/ Solution Architect is a major Advantage.<br/><br/>Skills Required :<br/><br/>- 8+ Years of experience in IT or professional services experience in IT delivery or large-scale IT analytics projects<br/><br/>- Candidates must have expertise knowledge of Google Cloud Platform; the other cloud platforms are nice to have.<br/><br/>- Expert knowledge in SQL development.<br/><br/>- Expertise in building data integration and preparation tools using cloud technologies (like Snaplogic, Google Dataflow, Cloud Dataprep, Python, etc).<br/><br/>- Experience with Apache Beam/Google Dataflow/Apache Spark in creating end to end data pipelines.<br/><br/>- Experience in some of the following : Python, Hadoop, Spark, SQL, Big Query, Big Table Cloud Storage, Datastore, Spanner, Cloud SQL, Machine Learning.<br/><br/>- Experience programming in Java, Python, etc.<br/><br/>- Identify downstream implications of data loads/migration (e.g., data quality, regulatory, etc.)<br/><br/>- Implement data pipelines to automate the ingestion, transformation, and augmentation of data sources, and provide best practices for pipeline operations.<br/><br/>- Capability to work in a rapidly changing business environment and to enable simplified user access to massive data by building scalable data solutions<br/><br/>- Advanced SQL writing and experience in data mining (SQL, ETL, data warehouse, etc.) and using databases in a business environment with complex datasets</p> (ref:hirist.tech)