Data Engineer Lead Data Engineer

Job description

About Us:Paytm is India’s largest digital payments and financial services platform, leading the mobile QR revolution.

We power millions of businesses and individuals, and we’re building scalable, resilient systems to serve half a billion Indians and beyond.Here at Paytm, technology isn't just about keeping up, it's about building the future.

We believe the next generation of engineers must not only scale systems to billions but also leverage AI and GPT tools to accelerate innovationAbout the Role:We're looking for a Lead AI engineer with 6-10 years of experience, who can build production-grade agentic AI systems.

You'll be working at the intersection of cutting-edge AI research and scalable engineering, creating autonomous agents that can reason, plan, and execute complex tasks reliably at scaleWhat We're Looking For:Agentic AI & LLM EngineeringYou should have hands-on experience with:1) Multi-agent systems: Building agents that coordinate, communicate, and work together on complex workflows.2) Agent orchestration: Designing systems where AI agents can plan multi-step tasks, use tools, and make autonomous decisions.3) LLMOps Experience: End-to-End LLM Lifecycle Management - hands-on experience managing the complete LLM workflow from prompt engineering and dataset curation through model fine-tuning, evaluation, and deployment.

This includes versioning prompts, managing training datasets, orchestrating distributed training jobs, and implementing automated model validation pipelines.

Production LLM Infrastructure - experience building and maintaining production LLM serving infrastructure including model registries, A/B testing frameworks for comparing model versions, automated rollback mechanisms, and monitoring systems that track model performance, latency, and cost metrics in real-time.4) AI Observability: Experience implementing comprehensive monitoring and tracing for AI systems, including prompt tracking, model output analysis, cost monitoring, and agent decision-making visibility across complex workflows.5) Evaluation frameworks: Creating comprehensive testing for agent performance, safety, and goal achievement.6) LLM inference optimization: Scaling model serving with techniques like batching, caching, and efficient frameworks (vLLM, TensorRT-LLM)Systems EngineeringStrong backend development skills including:1) Python expertise: FastAPI, Django, or Flask for building robust APIs that handle agent workflows2) Distributed systems: Microservices, event-driven architectures, and message queues (Kafka, RabbitMQ) for agent coordination3) Database strategy: Vector databases, traditional SQL/NoSQL, and caching layers optimized for agent state management4) Web-scale design: Systems handling millions of requests with proper load balancing and fault toleranceDevOps (Non-negotiable)1) Kubernetes: Working knowledge required - deployments, services, cluster management2) Containerization: Docker with production optimization and security best practices3) CI/CD: Automated testing and deployment pipelines4) Infrastructure as Code: Terraform, Helm charts5) Monitoring: Prometheus, Grafana for tracking complex agent behaviorsPrograming Language : Java , PythonWhat You'll BuildYou'll architect the infrastructure that powers our autonomous AI systems:1) Agent Orchestration Platform: Multi-agent coordination systems that handle complex, long-running workflows with proper state management and failure recovery.2) Evaluation Infrastructure: Comprehensive frameworks that assess agent performance across goal achievement, efficiency, safety, and decision-making quality.3) Production AI Services: High-throughput systems serving millions of users with intelligent resource management and robust fallback mechanisms.4) Training Systems: Scalable pipelines for SFT and DPO that continuously improve agent capabilities based on real-world performance and human feedback.Ideal Profile:1) You've spent serious time in production environments building AI systems that actually work.

You understand the unique challenges of agentic AI - managing state across long conversations, handling partial failures in multi-step processes, and ensuring agents stay aligned with their intended goals.2) You've dealt with the reality that the hardest problems aren't always algorithmic.

Sometimes it's about making an agent retry gracefully when an API call fails, or designing an observability layer that catches when an agent starts behaving unexpectedly, or building systems that can scale from handling dozens of agent interactions to millions.3) You're excited about the potential of AI agents but pragmatic about the engineering work required to make them reliable in production.Preferred Qualifications: Bachelor's/Master's Degree in Computer Science or equivalent

Required Skill Profession

Computer Occupations

Job Details

Company: Paytm
Location: Bengaluru, Karnataka, India
Job Type: Full-time Employment
Salary: Undisclosed
Posted: 2025-10-15
Status: (Active)
Download This Job Details in PDF
Why Work at Paytm
Paytm
Company Info
Other Openings
Industry: Computer Occupations

Related Jobs

Data Engineer- Lead Data Engineer

Paytm

Bengaluru, Karnataka, India