Know ATS Score
CV/Résumé Score
  • Expertini Resume Scoring: Our Semantic Matching Algorithm evaluates your CV/Résumé before you apply for this job role: IN_Senior Associate _Pyspark Data Engineer__Data and Analytics_ Advisory _Bangalore.
India Jobs Expertini

Urgent! IN_Senior Associate _Pyspark Data Engineer__Data and Analytics_ Advisory _Bangalore Job Opening In Kalhalli – Now Hiring PwC

IN_Senior Associate _Pyspark Data Engineer__Data and Analytics_ Advisory _Bangalore



Job description

Description

& Summary: A career within….

A career within Data and Analytics services will provide you with the opportunity to help organisations uncover enterprise insights and drive business results using smarter data analytics.

We focus on a collection of organisational technology capabilities, including business intelligence, data management, and data assurance that help our clients drive innovation, growth, and change within their organisations in order to keep up with the changing nature of customers and technology.

We make impactful decisions by mixing mind and machine to leverage data, understand and navigate risk, and help our clients gain a competitive edge.

Responsibilities:

· Manage the development of end-to-end data ingestion, transformation, and ETL workflows using Hadoop, PySpark, and related big data technologies.

· Design and implementation of distributed computing frameworks and real-time data processing solutions using Spark and PySpark.

· Ensure the effective management and operation of Hadoop clusters (HDFS, YARN, Hive, HBase) and related big data tools, ensuring high availability, security, and performance.

· Develop and maintain data pipelines to process large volumes of structured and unstructured data, ensuring data consistency, integrity, and efficiency.

· Provide technical guidance, mentorship, and career development for junior engineers and team members in the Hadoop and PySpark ecosystem.

· Design and implementation of data models and data storage strategies, including integration with cloud platforms such as AWS, Google Cloud, or Azure.

· Optimize performance of data pipelines and queries, troubleshooting any issues related to data processing and performance bottlenecks.

· Ensure adherence to best practices in code quality, data governance, and security standards.

· Communicate complex technical challenges and solutions effectively to both technical and non-technical stakeholders.

· Drive innovation by staying up-to-date with the latest big data technologies and incorporating relevant advancements into the platform.

· 3 years of experience in Big Data Engineering, with a strong background in Hadoop and PySpark (at least 2-3 years in a leadership or managerial role).

· Proven experience in designing and implementing data pipelines using PySpark for large-scale data processing.

· Strong understanding and hands-on experience with the Hadoop ecosystem, including HDFS, MapReduce, Hive, HBase, and YARN.

· Proficiency in Python and PySpark, including working with distributed computing frameworks and data processing workflows.

· Expertise in working with ETL processes, data lakes, data warehousing, and cloud platforms (AWS, GCP, or Azure).

Mandatory skill sets:

Pyspark, Python, Hadoop, Sql

Preferred skill sets:

Pyspark, Python, Hadoop, Sql

Years of experience required:

3-6

Education qualification:

B.Tech / M.Tech / MBA / MCA

Education

Degrees/Field of Study required: Master of Engineering, Master of Business Administration, Bachelor of EngineeringDegrees/Field of Study preferred:

Certifications

Required Skills

Microsoft Azure, PySpark

Optional Skills

Accepting Feedback, Accepting Feedback, Active Listening, Agile Scalability, Amazon Web Services (AWS), Analytical Thinking, Apache Airflow, Apache Hadoop, Azure Data Factory, Communication, Creativity, Data Anonymization, Data Architecture, Database Administration, Database Management System (DBMS), Database Optimization, Database Security Best Practices, Databricks Unified Data Analytics Platform, Data Engineering, Data Engineering Platforms, Data Infrastructure, Data Integration, Data Lake, Data Modeling, Data Pipeline {+ 27 more}

Desired Languages

Travel Requirements

Not Specified

Available for Work Visa Sponsorship?

No

Government Clearance Required?

No

Job Posting End Date


Required Skill Profession

Other General



Your Complete Job Search Toolkit

✨ Smart • Intelligent • Private • Secure

Start Using Our Tools

Join thousands of professionals who've advanced their careers with our platform

Rate or Report This Job
If you feel this job is inaccurate or spam kindly report to us using below form.
Please Note: This is NOT a job application form.


    Unlock Your IN_Senior Associate Potential: Insight & Career Growth Guide