Estuate - GCP Data Engineer - Big Data

<p>Job Title : Data Engineer / Integration Engineer<br/><br/>Job Summary : <br/><br/>We are seeking a highly skilled Data Engineer / Integration Engineer to join our team.

The ideal candidate will have expertise in Python, workflow orchestration, cloud platforms (GCP/Google BigQuery), big data frameworks (Apache Spark or similar), API integration, and Oracle EBS.

The role involves designing, developing, and maintaining scalable data pipelines, integrating various systems, and ensuring data quality and consistency across platforms.

Knowledge of Ascend.io is a plus.<br/><br/>Key Responsibilities : <br/><br/>- Design, build, and maintain scalable data pipelines and workflows.<br/><br/>- Develop and optimize ETL/ELT processes using Python and workflow automation tools.<br/><br/>- Implement and manage data integration between various systems, including APIs and Oracle EBS.<br/><br/>- Work with Google Cloud Platform (GCP) or Google BigQuery (GBQ) for data storage, processing, and analytics.<br/><br/>- Utilize Apache Spark or similar big data frameworks for efficient data processing.<br/><br/>- Develop robust API integrations for seamless data exchange between applications.<br/><br/>- Ensure data accuracy, consistency, and security across all systems.<br/><br/>- Monitor and troubleshoot data pipelines, identifying and resolving performance issues.<br/><br/>- Collaborate with data analysts, engineers, and business teams to align data solutions with business goals.<br/><br/>- Document data workflows, processes, and best practices for future reference.<br/><br/>Required Skills & Qualifications : <br/><br/>- Strong proficiency in Python for data engineering and workflow automation.<br/><br/>- Experience with workflow orchestration tools (e.g., Apache Airflow, Prefect, or similar).<br/><br/>- Hands-on experience with Google Cloud Platform (GCP) or Google BigQuery (GBQ).<br/><br/>- Expertise in big data processing frameworks, such as Apache Spark.<br/><br/>- Experience with API integrations (REST, SOAP, GraphQL) and handling structured/unstructured data.<br/><br/>- Strong problem-solving skills and ability to optimize data pipelines for performance.<br/><br/>- Experience working in an agile environment with CI/CD processes.<br/><br/>- Strong communication and collaboration skills.<br/><br/>Preferred Skills & Nice-to-Have : <br/><br/>- Experience with Ascend.io platform for data pipeline automation.<br/><br/>- Knowledge of SQL and NoSQL databases.<br/><br/>- Familiarity with Docker and Kubernetes for containerized workloads.<br/><br/>- Exposure to machine learning workflows is a plus.<br/><br/>Why Join Us?<br/><br/>- Opportunity to work on cutting-edge data engineering projects.<br/><br/>- Collaborative and dynamic work environment.<br/><br/>- Competitive compensation and benefits.<br/><br/>- Professional growth opportunities with exposure to the latest technologies.<br/></p> (ref:hirist.tech)