Python Data Engineer

<p><p><b>Role Overview :</b><br/><br/>We are looking for an experienced Python Data Engineer with strong expertise in PySpark, distributed data engineering, and LLM integration.<br/><br/>The role involves building scalable data pipelines, AI-powered workflows, and enabling data-driven automation using platforms such as Databricks, AWS EMR, and LangChain.<br/><br/>As an SE3 engineer, you will primarily be responsible for hands-on development and delivery, while also contributing to solution design and collaborating with cross-functional teams.</p><p><b><br/></b></p><p><b>Key Responsibilities :</b></p><p><br/>- Develop and optimize ETL pipelines using PySpark, SparkSQL, and distributed frameworks.<br/><br/>- Work with LangChain to integrate LLM-based solutions into data workflows (Agents, Toolkits, Vector Stores).<br/><br/>- Implement data transformations, lineage, and governance controls in data platforms.<br/><br/>- Support ML teams in deploying embeddings, retrieval-augmented generation (RAG), and NLP pipelines.<br/><br/>- Build workflows on Databricks and AWS EMR, ensuring cost and performance efficiency.<br/><br/>- Apply best practices in coding, testing, CI/CD, and Skills :</b></p><p><br/>- Strong proficiency in Python for data engineering.<br/><br/>- Hands-on experience in PySpark, SparkSQL, and ETL design.<br/><br/>- Working knowledge of LangChain, OpenAI APIs, or Hugging Face.<br/><br/>- Experience with Databricks, AWS EMR, or similar cloud data platforms.<br/><br/>- Good understanding of SQL, data modeling, and distributed data Skills :</b></p><p><br/>- Familiarity with Google ADK Prompt Engineering.<br/><br/>- Experience with vector databases like FAISS, Pinecone, or Chroma.<br/><br/>- Exposure to MLflow, Unity Catalog, or SageMaker.<br/><br/>- Interest in LLM-powered applications and generative AI workflows</p><br/></p> (ref:hirist.tech)