Description
& Summary: A career within….Responsibilities: ● Design, develop, and optimize data pipelines and ETL
processes using PySpark or Scala to extract,
transform, and load large volumes of structured and
unstructured data from diverse sources.
● Implement data ingestion, processing, and storage
solutions on Azure cloud platform, leveraging
services such as Azure Databricks, Azure Data Lake
Storage, and Azure Synapse Analytics.
● Develop and maintain data models, schemas, and
metadata to support efficient data access, query
performance, and analytics requirements.
● Monitor pipeline performance, troubleshoot issues,
and optimize data processing workflows for
scalability, reliability, and cost-effectiveness.
● Implement data security and compliance measures to
protect sensitive information and ensure regulatory
compliance.
Requirement
● Proven experience as a Data Engineer, with expertise
in building and optimizing data pipelines using
PySpark, Scala, and Apache Spark.
● Hands-on experience with cloud platforms,
particularly Azure, and proficiency in Azure services
such as Azure Databricks, Azure Data Lake Storage,
Azure Synapse Analytics, and Azure SQL Database.
● Strong programming skills in Python and Scala, with
experience in software development, version control,
and CI/CD practices.
● Familiarity with data warehousing concepts,
dimensional modeling, and relational databases (e.g.,
SQL Server, PostgreSQL, MySQL).
● Experience with big data technologies and
frameworks (e.g., Hadoop, Hive, HBase) is a plus.
Mandatory skill sets: Spark, Pyspark, Azure
Preferred skill sets: Spark, Pyspark, Azure
Years of experience required: 4 - 8
Education qualification: B.Tech / M.Tech / MBA / MCA
Education
Degrees/Field of Study required: Master of Business Administration, Bachelor of Engineering, Master of EngineeringDegrees/Field of Study preferred:Certifications
Required Skills
PySpark, Python (Programming Language)Optional Skills
Accepting Feedback, Accepting Feedback, Active Listening, Analytical Reasoning, Analytical Thinking, Application Software, Business Data Analytics, Business Management, Business Technology, Business Transformation, Communication, Creativity, Documentation Development, Embracing Change, Emotional Regulation, Empathy, Implementation Research, Implementation Support, Implementing Technology, Inclusion, Intellectual Curiosity, Learning Agility, Optimism, Performance Assessment, Performance Management Software {+ 16 more}Desired Languages
Travel Requirements
Not SpecifiedAvailable for Work Visa Sponsorship?
NoGovernment Clearance Required?
NoJob Posting End Date