<p><p><b>Job Summary :</b></p><p><br/>We are seeking an experienced Senior Python Developer with deep expertise in Google Cloud Platform (GCP), PySpark, and Apache Airflow.
The ideal candidate will be responsible for designing, developing, and maintaining scalable data pipelines, orchestrating complex workflows using Airflow, and optimizing data processing jobs in a distributed environment.<br/><br/><b>Key Responsibilities :</b></p><p><p><b><br/></b></p>- Design and develop robust, scalable, and efficient data pipelines using Python and PySpark.<br/><br/>- Create, schedule, and monitor DAGs in Apache Airflow for data workflow orchestration.<br/><br/>- Work with GCP services (such as BigQuery, Cloud Composer, GCS, Pub/Sub, Dataflow, etc.) to build end-to-end data solutions.<br/><br/>- Implement data transformation, cleansing, aggregation, and validation logic in pipelines.<br/><br/>- Collaborate with Data Engineers, Analysts, and other stakeholders to understand business requirements and translate them into technical solutions.<br/><br/>- Optimize performance and cost-efficiency of data pipelines on GCP.<br/><br/>- Monitor production workflows and troubleshoot issues to ensure data accuracy and availability.<br/><br/>- Follow coding best practices and maintain high standards for data quality, security, and compliance.<br/><br/>- Participate in code reviews, testing, and deployment processes in CI/CD environments.<br/><br/><b>Required Skills :</b></p><p><p><b><br/></b></p><p>- 7+ years of professional experience in software development with a strong focus on Python.</p><br/>- 3+ years hands-on experience with Apache Airflow for orchestration and automation.<br/><br/>- 3+ years of experience working with PySpark and large-scale data processing.<br/><br/>- Strong hands-on experience with Google Cloud Platform (GCP), especially services like :<br/><ul><li>BigQuery</li><li>Cloud Composer (Airflow)</li><li>Cloud Storage (GCS)</li><li>Pub/Sub</li><li>Dataflow (optional)</li></ul>- Solid understanding of data modeling, ETL/ELT pipelines, and batch/stream processing.<br/><br/>- Proficient in SQL and working with large datasets.<br/><br/>- Experience with version control (Git) and CI/CD pipelines.<br/><br/>- Familiarity with containerization (Docker) and Kubernetes is a plus.<br/><br/><b>Preferred Qualifications :</b></p><p><b><br/></b>- GCP Certification (e.g., Professional Data Engineer or Cloud Developer) is a plus.<br/><br/>- Experience with Terraform or other infrastructure-as-code tools.<br/><br/>- Exposure to other workflow tools (e.g., Luigi, Prefect) is a plus.<br/><br/>- Strong problem-solving and analytical skills.<br/><br/>- Excellent communication and collaboration skills.</p><br/></p> (ref:hirist.tech)