Bits In Glass Data Engineer Python/ETL

Job description

<p><p><b>Title : </b> Senior Data Engineer / ETL Engineer.<br/><br/><b>Location : </b> Pune / Hyderabad / Mohali.<br/><br/><b>Job Summary : </b><br/><br/> As a Senior Data Engineer / ETL Engineer, you will be instrumental in designing, developing, and optimizing data processing systems that support our organization's data initiatives.<br/><br/> Your expertise in Bigdata and Google Cloud Platform (GCP) will be essential in building robust ETL pipelines that transform raw data into actionable insights.<br/><br/> You will collaborate with cross-functional teams to ensure that data is accurate, accessible, and valuable for : </b><br/><br/><b>Design and Develop ETL Processes : </b><br/><br/> - Lead the design and implementation of ETL processes using all kinds of batch/streaming tools to extract, transform, and load data from various sources into GCP.<br/><br/> - Collaborate with stakeholders to gather requirements and ensure that ETL solutions meet business needs.<br/><br/><b>Data Pipeline Optimization : </b><br/><br/> - Optimize data pipelines for performance, scalability, and reliability, ensuring efficient data processing workflows.<br/><br/> - Monitor and troubleshoot ETL processes, proactively addressing issues and bottlenecks.<br/><br/><b>Data Integration and Management : </b><br/><br/> - Integrate data from diverse sources, including databases, APIs, and flat files, ensuring data quality and consistency.<br/><br/> - Manage and maintain data storage solutions in GCP (e.g., BigQuery, Cloud Storage) to support analytics and reporting.<br/><br/><b>GCP Dataflow Development : </b><br/><br/> - Write Apache Beam based Dataflow Job for data extraction, transformation, and analysis, ensuring optimal performance and accuracy.<br/><br/> - Collaborate with data analysts and data scientists to prepare data for analysis and reporting.<br/><br/><b>Automation and Monitoring : </b><br/><br/> - Implement automation for ETL workflows using tools like Apache Airflow or Cloud Composer, enhancing efficiency and reducing manual intervention.<br/><br/> - Set up monitoring and alerting mechanisms to ensure the health of data pipelines and compliance with SLAs.<br/><br/><b>Data Governance and Security : </b><br/><br/> - Apply best practices for data governance, ensuring compliance with industry regulations (e.g., GDPR, HIPAA) and internal policies.<br/><br/> - Collaborate with security teams to implement data protection measures and address and Knowledge Sharing : </b><br/><br/> - Document ETL processes, data models, and architecture to facilitate knowledge sharing and onboarding of new team members.<br/><br/> - Conduct training sessions and workshops to share expertise and promote best practices within the team.<br/><br/><b>Qualifications : </b><br/><br/><b>Education : </b><br/><br/> Bachelors degree in Computer Science, Information Systems, or a related field.<br/><br/><b>Experience : </b><br/><br/> - Minimum of 5 years of industry experience in data engineering or ETL development, with a strong focus on Data Stage and GCP.<br/><br/> - Proven experience in designing and managing ETL solutions, including data modeling, data warehousing, and SQL development.<br/><br/><b>Technical Skills : </b><br/><br/> - Strong knowledge of GCP services (e.g., BigQuery, Dataflow, Cloud Storage, Pub/Sub) and their application in data engineering.<br/><br/> - Experience of cloud-based solutions, especially in GCP, cloud certified candidate is preferred.<br/><br/> - Experience and knowledge of Bigdata data processing in batch mode and streaming mode, proficient in Bigdata eco systems, e.g. Hadoop, HBase, Hive, MapReduce, Kafka, Flink, Spark, etc.<br/><br/> - Familiarity with Java & Python for data manipulation on Cloud/Bigdata platform.<br/><br/><b>Analytical Skills : </b><br/><br/> - Strong problem-solving skills with a keen attention to detail.<br/><br/> - Ability to analyze complex data sets and derive meaningful insights.<br/><br/><b>Company Description : </b> Bits in Glass India.<br/><br/><b>Industry Leader : </b><br/><br/> - Bits in Glass(BIG) has been in business for more than 20 years.<br/><br/> - In 2021 Bits in Glass joined hands with Crochet Technologies, forming a larger organization under the Bits In Glass brand to better serve customers across the globe.<br/><br/> - Offices across three locations in India : Pune, Hyderabad & Chandigarh.<br/><br/> - Specialized Pega partner since 2017, delivering Pega solutions with deep industry expertise and experience.<br/><br/> - Proudly ranked among the top 30 Pega partners, Bits In Glass has been one of the very few sponsors of the annual PegaWorld event.<br/><br/> - Elite Appian partner since 2008, delivering Appian solutions with deep industry expertise and experience.<br/><br/> - Operating in the United States, Canada, United Kingdom, and India.<br/><br/> - Dedicated global Pega CoE to support our customers and internal dev teams.<br/><br/><b>Employee Benefits : </b><br/><br/> - Career Growth : Opportunities for career advancement and professional development.<br/><br/> - Challenging Projects : Work on innovative, cutting-edge projects that make a global impact.<br/><br/> - Global Exposure : Collaborate with international teams and clients to broaden your professional network.<br/><br/> - Flexible Work Arrangements : Support for work-life balance through flexible working conditions.<br/><br/> - Comprehensive Benefits : Competitive compensation packages and comprehensive benefits including health insurance, and paid time off.<br/><br/> - Learning OpportunitiesGreat opportunity to upskill yourself and work on new technologies like AI-enabled Pega solutions, Data engineering, Integration, cloud migration etc.<br/><br/><b>Company Culture : </b><br/><br/> - Collaborative Environment : Emphasizes teamwork, innovation, and knowledge sharing.<br/><br/> - Inclusive Workplace : Values diversity and fosters an inclusive environment where all ideas are respected.<br/><br/> - Continuous Learning : Encourages professional development through ongoing learning opportunities and certifications.<br/><br/><b>Core Values : </b><br/><br/> - Integrity : Commitment to ethical practices and transparency in all business dealings.<br/><br/> - Excellence : Strive for the highest standards in everything we do.<br/><br/> - Client-Centric Approach : Focus on delivering the best solutions tailored to client needs.</p><br/></p> (ref:hirist.tech)

Required Skill Profession

Computer Occupations