Job Description: Senior Data Engineer
Role Overview:
We are seeking an experienced Senior Data Engineer to design, develop, and optimize our data pipeline architecture. This role involves creating scalable and efficient systems to process and manage large datasets across the organization. As a Senior Data Engineer, you will collaborate closely with software engineers, database architects, data analysts, and data scientists to ensure the consistency and efficiency of our data infrastructure.
Key Responsibilities:
Build and enhance data pipeline architecture for data extraction, transformation, and loading (ETL) processes.
Automate manual processes and optimize data delivery while redesigning infrastructure for scalability and efficiency.
Partner with cross-functional teams, including executive, product, data, and design teams, to address data-related technical issues and infrastructure needs.
Collaborate with machine learning and analytics teams to drive innovation and improve the functionality of the data system.
Qualifications:
Bachelor's degree in Engineering, Computer Science, or a related field.
10+ years of experience in a Data Engineer role.
Proficiency in Apache Spark with a solid understanding of Big Data concepts and distributed systems.
Strong programming skills in Scala, Python, Java, or similar languages, with the flexibility to adapt quickly.
Advanced knowledge of SQL and experience with relational databases like Postgres or MySQL.
Familiarity with data formats such as Delta Tables, Parquet, CSV, and JSON.
Experience with cloud platforms, specifically DataBricks.
Comfortable with Linux shell environments and Agile development practices.
Strong written and verbal communication skills in English.
Knowledge of machine learning is a plus.
Key Competencies:
Ability to work independently and deliver stable, efficient, and reliable solutions.
Excellent problem-solving skills and adaptability to dynamic environments.
A proactive approach to improving existing processes and systems.
This is an exciting opportunity to redefine data architecture and contribute to the
development of cutting-edge products and initiatives. Candidates with a passion for
innovation and deep technical expertise are encouraged to apply.
Shift- 2 PM to 11 PM IST