Job DescriptionFunctional Tester – Agentic AI Solution (Banking & Wealth Management)
Location: Noida
Experience Level: 2–4 years
Reports To: QA Lead / Product Manager
We are looking for an experienced Functional Tester to validate the business workflows, compliance logic, and customer-facing features of an Agentic AI solution built for the banking and wealth management domain.
This solution leverages LLMs, retrieval-augmented generation (RAG), and domain ontologies to provide intelligent data retrieval, client interaction, and regulatory compliance automation.
Key Responsibilities
Functional Testing
- Understand functional and non-functional requirements for Agentic AI features (e.g., document Q&A, investment summarization, client onboarding, compliance tracking)
- Validate end-to-end user journeys across banking and wealth advisory use cases
- Test conversational workflows and agentic logic flows (multi-step LLM tasks)
- Verify outputs from LLMs for relevance, consistency, and context-awareness
- Ensure correct role-based access across advisors, relationship managers, and compliance officers
Test Planning & Execution
- Prepare detailed test plans, test cases, and traceability matrices from EPICs, features, and user stories
- Execute manual testing for user interfaces, APIs, and chat/agent interactions
- Perform regression testing across AI pipelines after prompt or model updates
- Log, track, and retest defects using tools like Azure DevOps, Jira, or TestRail
AI Model & RAG Testing
- Validate grounding of LLM responses from vector databases (e.g., Azure AI Search, Pinecone)
- Check performance of RAG pipelines including hallucination mitigation, context window boundaries, and fallback logic
- Test interaction of agent orchestration flows (LangChain/Autogen/semantic kernel)
- Verify prompt-based test cases for accuracy and coverage
Integration & Data Testing
- Validate PDF/statement extraction logic from custodians (e.g., Morgan Stanley, Pershing)
- Check structured data output (JSON, tables) from Azure Form Recognizer or OpenAI pipelines
- Ensure data integrity and lineage from ingestion → processing → user response
Must-Have Skills & Tools
Category
| Tools/Tech
|
Test Management
| Azure / AWA DevOps, Jira, TestRail, Azure Board
|
Test Types
| Functional, UI, Integration, AI behavior
|
Domain Knowledge
| Wealth Mgmt workflows, KYC, portfolio views, regulatory rules
|
AI Concepts
| LLM output validation, RAG testing, prompt engineering basics
|
Data
| JSON validation, SQL for test data setup
|
Tools (Optional)
| Postman, Python (for test automation scripts), Excel-based test mapping
|
Requirements
Preferred Qualifications
- Bachelor’s degree in Engineering, Finance, or Computer Science
- Experience testing AI/ML solutions or conversational AI products
- Familiarity with Azure AI stack, OpenAI, or AWS Bedrock
- Exposure to compliance-heavy domains like banking, wealth, or insurance
- Ability to work with cross-functional Agile teams (product, engineering, AI/ML, compliance)
Nice-to-Have Skills
- Experience writing prompt-based test cases for LLMs
- Familiarity with agentic workflows (e.g., task chaining, autonomy testing)
- Understanding of financial statement formats (portfolio summaries, trade confirms)
- Basic scripting to validate data output (e.g.,Python, bash)
RequirementsPreferred Qualifications Bachelor’s degree in Engineering, Finance, or Computer Science Experience testing AI/ML solutions or conversational AI products Familiarity with Azure AI stack, OpenAI, or AWS Bedrock Exposure to compliance-heavy domains like banking, wealth, or insurance Ability to work with cross-functional Agile teams (product, engineering, AI/ML, compliance) Nice-to-Have Skills Experience writing prompt-based test cases for LLMs Familiarity with agentic workflows (e.g., task chaining, autonomy testing) Understanding of financial statement formats (portfolio summaries, trade confirms) Basic scripting to validate data output (e.g.,Python, bash)