<p>Job Summary :<br/><br/>We are immediately hiring a skilled and motivated GCP Cloud Data Engineer for one of our esteemed clients.
The ideal candidate should have hands-on experience with Python, Apache Airflow, Google BigQuery, SQL, Apache Spark, and Pub/Sub.
You will be responsible for designing, developing, and maintaining robust data pipelines and cloud-based data solutions on Google Cloud Platform (GCP).<br/><br/>Key Responsibilities :<br/><br/>- Design, build, and optimize scalable and efficient data pipelines using Python and Airflow.<br/><br/>- Develop and maintain ETL/ELT workflows to process data across various sources and sinks.<br/><br/>- Work extensively with Google BigQuery for data transformation and analysis.<br/><br/>- Integrate real-time and batch data processing solutions using Pub/Sub and Spark.<br/><br/>- Write complex and optimized SQL queries for large-scale datasets.<br/><br/>- Collaborate with data scientists, analysts, and platform teams to deliver end-to-end data solutions.<br/><br/>- Ensure data quality, performance, and governance across pipelines.<br/><br/>- Monitor, troubleshoot, and optimize data workflows in production environments.<br/><br/>Required Technical Skills :<br/><br/>Programming : </p><p><br/></p><p>- Proficient in Python for data engineering tasks and workflow orchestration.<br/><br/>Orchestration Tools : </p><p><br/></p><p>- Experience with Apache Airflow.<br/><br/>Cloud Platform : </p><p><br/></p><p>Strong experience with Google Cloud Platform (GCP) services, particularly :<br/><br/>- BigQuery<br/><br/>- Pub/Sub<br/><br/>Data Processing : </p><p><br/></p><p>- Hands-on experience with Apache Spark.<br/><br/>Database Skills : </p><p><br/></p><p>- Advanced SQL skills for querying and managing large datasets.<br/><br/>Preferred Qualifications :<br/><br/>- GCP Data Engineer certification is a plus.<br/><br/>- Experience with CI/CD pipelines and DevOps practices in data engineering.<br/><br/>- Familiarity with other GCP services like Cloud Storage, Dataflow, or Dataproc.<br/><br/>- Strong problem-solving skills and ability to work in a fast-paced, agile environment.</p> (ref:hirist.tech)