Senior Data Engineer

Responsibilities:
Design and Development:
Architect, design, and develop robust, scalable, and efficient data pipelines.
Design and manage platform solutions to support data engineering needs to ensure seamless integration and performance.
Write clean, efficient, and maintainable code.
Leadership and Collaboration:
Lead and mentor a team of Data engineers, providing technical guidance and fostering a collaborative environment.
Collaborate with cross-functional teams including data scientists, analysts, and business stakeholders to understand data requirements and deliver solutions that meet business needs.
Drive the adoption of best practices in software development lifecycle (SDLC), including code reviews, testing, and continuous integration/continuous deployment (CI/CD).
Data Management and Optimization:
Ensure data quality, integrity, and security across all data pipelines.
Optimize data processing workflows for performance and cost-efficiency.
Develop and maintain comprehensive documentation for data pipelines and related processes.
Innovation and Continuous Improvement:
Stay current with emerging technologies and industry trends in big data and cloud computing.
Propose and implement innovative solutions to improve data processing and analytics capabilities.
Continuously evaluate and improve existing data infrastructure and processes.
Qualifications:
Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field.
8+ years of experience in software engineering with a focus on data engineering and building data platform
Strong programming experience using Python or Java.
Proven experience with Big data technologies like Apache Spark, Amazon EMR, Apache Iceberg, Amazon Redshift, etc or Similar technologies
Proven experience in RDBMS(Postgres, MySql, etc) and NoSQL(MongoDB, DynamoDB, etc) database
Proficient in AWS cloud services (e.g., Lambda, S3, Athena, Glue) or comparable cloud technologies.
Demonstrated leadership experience, with a track record of leading and mentoring engineering teams.
In-depth understanding of SDLC best practices, including Agile methodologies, code reviews, and CI/CD.
Experience working in Event driven and Serverless Architecture
Experience with platform solutions and containerization technologies (e.g., Docker, Kubernetes).
Excellent problem-solving skills and the ability to work in a fast-paced, dynamic environment.
Strong communication skills, both written and verbal.