Big data developer

Job Title: Senior Big Data Engineer (Python)Experience: 5+ YearsLocations: Chennai, Bangalore, GurugramEmployment Type: Full-TimeNotice Period: Immediate to 30 Days PreferredJob SummaryWe are seeking an experienced Big Data Engineer with strong expertise in Python to design, build, and manage large-scale data pipelines and analytics solutions.

The ideal candidate will have hands-on experience working with Big Data technologies and cloud platforms, and a passion for writing efficient, scalable, and maintainable code.Key ResponsibilitiesDesign, develop, and maintain scalable big data pipelines using Python and other data processing tools.Work with distributed data processing frameworks like Spark, Hadoop, Hive, or similar.Implement ETL processes for structured and unstructured data from various sources.Collaborate with data scientists, analysts, and other engineering teams to understand data needs.Optimize data workflows and ensure data quality, integrity, and security.Deploy solutions on cloud platforms (AWS/GCP/Azure) and automate data workflows.Monitor data pipelines, troubleshoot issues, and ensure high availability and performance.Required Skills & QualificationsBachelor’s or Master’s degree in Computer Science, Engineering, or a related field.Minimum 5 years of hands-on experience in Big Data engineering roles.Strong proficiency in Python for data processing and scripting.Experience with Apache Spark, Hadoop ecosystem (Hive, HDFS, HBase), Kafka.Solid understanding of data warehousing concepts and data modeling.Hands-on experience with SQL and No SQL databases.Familiarity with cloud data platforms like AWS (EMR, S3, Glue), Azure, or Google Cloud.Good understanding of CI/CD, containerization (Docker, Kubernetes), and version control (Git).Preferred Skills (Nice to Have)Experience with Airflow or other workflow orchestration tools.Knowledge of real-time data processing (e.g., Spark Streaming, Flink).Exposure to data governance, data lineage, and catalog tools.Why Join Us?Opportunity to work on cutting-edge Big Data solutions with global clients.Collaborative and innovation-driven work culture.Competitive compensation and career growth opportunities.Flexible work environment with hybrid (based on role).