Know ATS Score
CV/Résumé Score
  • Expertini Resume Scoring: Our Semantic Matching Algorithm evaluates your CV/Résumé before you apply for this job role: Data Engineer Big Data Technologies.
India Jobs Expertini

Urgent! Data Engineer - Big Data Technologies Job Opening In Chennai – Now Hiring Zyoin Group

Data Engineer Big Data Technologies



Job description

<p><p>The core responsibilities for the job include the following :</p><br/><p><b>Data Pipeline Development :</b></p><p><br/></p><p>- Design, develop, and maintain data pipelines to ingest, process, and transform data from various sources into usable formats.</p><p><br/></p><p>- Implement data integration solutions that connect disparate data systems, including databases, APIs, and third-party data sources.</p><br/><p><b>Data Storage and Warehousing :</b></p><p><br/></p><p>- Create and manage data storage solutions, such as data lakes, data warehouses, and NoSQL databases.</p><p><br/></p><p>- Optimize data storage for performance, scalability, and cost-efficiency.</p><br/><p><b>Data Quality and Governance :</b></p><p><br/></p><p>- Establish data quality standards and implement data validation and cleansing processes.</p><p><br/></p><p>- Collaborate with data analysts and data scientists to ensure data consistency and accuracy.</p><br/><p><b>ETL (Extract, Transform, Load) :</b></p><p><br/></p><p>- Develop ETL processes to transform raw data into a structured and usable format.</p><p><br/></p><p>- Monitor and troubleshoot ETL jobs to ensure data flows smoothly.</p><br/><p><b>Data Security and Compliance :</b></p><p><br/></p><p>- Implement data security measures and access controls to protect sensitive data.</p><p><br/></p><p>- Ensure compliance with data privacy regulations and industry standards (e.

g., GDPR, HIPAA).</p><br/><p><b>Performance Tuning :</b></p><p><br/></p><p>- Optimize data pipelines and queries for improved performance and efficiency.</p><p><br/></p><p>- Identify and resolve bottlenecks in data processing.</p><br/><p><b>Data Documentation :</b></p><p><br/></p><p>- Maintain comprehensive documentation for data pipelines, schemas, and data dictionaries.</p><p><br/></p><p>- Create and update data lineage and metadata documentation.</p><br/><p><b>Scalability and Reliability :</b></p><p><br/></p><p>- Design data infrastructure to scale with growing data volumes and business requirements.</p><p><br/></p><p>- Implement data recovery and backup strategies to ensure data availability and resilience.</p><br/><p><b>Collaboration :</b></p><p><br/></p><p>- Collaborate with cross-functional teams, including data scientists, analysts, and business stakeholders, to understand data requirements and deliver data solutions.</p><p><br/></p><p><b>Required Technical Skills :</b></p><p> </p><p><br/></p><p>- Programming Languages : Proficiency in Python is essential.

Experience with SQL for database querying and manipulation is also required.

Knowledge of Java or Scala is a plus.</p><p><br/></p><p>- Big Data Technologies : Hands-on experience with big data frameworks such as Apache Spark, Hadoop, or Flink.</p><p><br/></p><p>- Cloud Platforms : Strong experience with at least one major cloud provider's data services (AWS, GCP, or Azure).

This includes services like AWS S3, Redshift, Glue; GCP BigQuery, Dataflow, Cloud Storage; or Azure Synapse, Data Factory, Blob Storage.</p><p><br/></p><p>- Databases and Warehousing : Deep expertise in relational databases (PostgreSQL, MySQL) and NoSQL databases (MongoDB, Cassandra).

Experience with modern data warehouses (Snowflake, Redshift, BigQuery) is highly preferred.</p><p><br/></p><p>- ETL/ELT Tools : Experience with ETL/ELT tools like Apache Airflow for workflow orchestration, or other similar tools.</p><p><br/></p><p>- Data Governance and Security : Understanding of data governance principles and experience with implementing security measures like encryption, access controls, and data masking.</p><p><br/></p><p>- Version Control : Proficiency with Git for version control.</p><p><br/></p><p>- Containerization : Experience with Docker and Kubernetes is a plus.</p><p></p><br/></p> (ref:hirist.tech)


Required Skill Profession

Computer Occupations



Your Complete Job Search Toolkit

✨ Smart • Intelligent • Private • Secure

Start Using Our Tools

Join thousands of professionals who've advanced their careers with our platform

Rate or Report This Job
If you feel this job is inaccurate or spam kindly report to us using below form.
Please Note: This is NOT a job application form.


    Unlock Your Data Engineer Potential: Insight & Career Growth Guide