<p><p><b>Position : </b> ML Engineer<br/><br/><b>Experience : </b> 4-7 Years<br/><br/><b>Location : </b> Bangalore, India</p><p><br/></p><p><b>Job Summary :</b></p><p><br/></p><p>We are seeking a highly skilled ML Engineer with 4-7 years of experience in building production systems that handle significant scale.
The ideal candidate will have a deep, hands-on understanding of asynchronous and event-driven architectures and a proven track record of scaling AI/ML inference in production.
This role requires a professional who can design and implement resilient, low-latency systems that manage AI workloads, integrate multiple models, and ensure a seamless user experience.
You will be responsible for owning the end-to-end performance of critical, revenue-generating AI conversation Responsibilities :</b></p><p><b><br/></b></p><p><b>System Design & Implementation : </b> Design and implement robust, asynchronous multi-agent orchestration systems.
Build resilient inference pipelines that can gracefully degrade under heavy load.</p><p><br/><b>Latency & Performance Optimization : </b> Own the end-to-end latency from a user's message to an AI response.
Implement intelligent request routing and load balancing to optimize AI workloads.
Optimize credit data retrieval and caching strategies to enhance system speed and efficiency.</p><br/><b>Resilience & Reliability : </b> Design and implement circuit breakers and fallback strategies for AI model failures.
Migrate critical AI conversation flows from monolithic architectures to dedicated microservices to improve resilience and scalability.<br/><br/><b>Real-time Communication & Observability : </b> Implement WebSocket/streaming infrastructure for real-time chat and other communication needs.
Build comprehensive observability systems to monitor and analyze AI system performance.<br/><br/><b>Technical Leadership : </b> Debug production issues under high AI inference load.
Make technical decisions that directly affect revenue-generating conversations and customer subscription retention.</p><p><br/></p><p><b>Required Skills & Qualifications :</b></p><p><p><b><br/></b></p><b>Core Experience : </b><br/><br/></p><p>- 4+ years of experience building production systems that handle over 10k concurrent users.<br/><br/></p><p>- Proven, hands-on experience scaling ML/AI inference in production.<br/><br/><b>Mandatory Technical Skills : </b><br/><br/></p><p>- Proven experience with async/event-driven architectures, not just traditional REST APIs.<br/><br/></p><p>- Deep understanding of caching strategies using technologies like Redis, in-memory caches, or CDNs.<br/><br/></p><p>- Experience with message queues and real-time communication protocols.<br/><br/></p><p>- Proven experience building systems that integrate multiple LLM/AI models in production.<br/><br/></p><p>- Knowledge of AI model serving frameworks like TensorFlow Serving or Triton.<br/><br/><b>Professional Attributes : </b><br/><br/></p><p>- Strong problem-solving skills and experience debugging complex production issues under high load.<br/><br/></p><p>- A deep understanding of conversation state management and context handling.<br/><br/></p><p>- A mindset of ownership and a clear focus on technical decisions that drive business Skills :</b></p><p><br/></p><p>- Exposure to cutting-edge AI infrastructure challenges.</p><p><br/></p>- Direct experience with optimization techniques like batching, caching, and model </p><p>quantization.<br/><br/></p><p>- Prior experience with AI-powered conversational platforms.</p><br/></p> (ref:hirist.tech)