Data Engineer - PySpark/SQL

<p><p><b>Key Responsibilities :</b><br/><br/></p><p>- Design, build, and optimize data pipelines leveraging Apache Spark, PySpark, and SQL.<br/><br/></p><p>- Develop scalable data integration, transformation, and modeling solutions for analytics and reporting.<br/><br/></p><p>- Implement and manage CI/CD pipelines, release management, and production deployments.<br/><br/></p><p>- Collaborate with data science teams to enable advanced analytics workloads using Python and PySpark.<br/><br/></p><p>- Orchestrate workflows using Apache Airflow or equivalent platforms (SSIS, etc.<br/><br/></p><p>- Support data ingestion from multiple structured/unstructured sources and develop custom connectors where needed.<br/><br/></p><p>- Monitor, troubleshoot, and optimize production data pipelines for performance and reliability.<br/><br/></p><p>- Work with Kubernetes/Docker for containerized data services and deployments.<br/><br/></p><p>- Ensure data quality, security, and compliance across all Technical Skills :</b></p><p><br/></p>- Apache Spark (Strong expertise Mandatory)<br/><br/></p><p>- PySpark (Strong expertise Mandatory)<br/><br/></p><p>- Python (Data Science focused)<br/><br/></p><p>- SQL / MySQL (Advanced, with query optimization)<br/><br/></p><p>- Linux, Git, CI/CD, Release Management, Production Deployment<br/><br/></p><p>- Kubernetes / Docker (Good hands-on experience)<br/><br/></p><p>- Java (Good to have working Required :</b></p><p><br/></p>- 6 to 8 years hands-on in core skills : Spark, PySpark, Python, SQL, Data Engineering.<br/><br/></p><p>- 5+ years overall experience in data engineering, ELT development, and data modeling.<br/><br/></p><p>- Proven experience with workflow orchestration, data ingestion, and performance optimization</p><br/></p> (ref:hirist.tech)