Job Description
<p><p><b>Company Overview :</b></p><p><br/></p><p>At Codvo, software and people transformations go hand-in-hand.<br/><br/>We are a global empathy-led technology services company.<br/><br/>Product innovation and mature software engineering are part of our core DNA.<br/><br/>Respect, Fairness, Growth, Agility, and Inclusiveness are the core values that we aspire to live by each day.<br/><br/>We continue to expand our digital strategy, design, architecture, and product management capabilities to offer expertise, outside-the-box thinking, and measurable results.<br/><br/><b>About The Role :</b><br/><br/>We are looking for an experienced QA Lead - AI Systems to lead the validation and quality assurance of AI-powered products, including LLM/RAG systems, Software as a Medical Device (SaMD), and other AI-driven digital solutions.<br/><br/>This is a key leadership role responsible for establishing end-to-end QA frameworks, validation strategies, and compliance processes for AI/ML models across development and deployment pipelines.<br/><br/>The ideal candidate will bridge data science, software engineering, and regulatory QA to ensure AI systems meet performance, safety, and compliance standards.<br/><br/><b>Key Responsibilities :</b><br/><br/>- Own and drive QA strategy for AI systems across the model, data, and product lifecycle.<br/><br/>- Lead AI model validation efforts, covering LLM/RAG testing, bias analysis, and performance evaluation.<br/><br/>- Define and implement AI Evaluation Metrics (accuracy, fairness, drift, explainability) aligned with business and regulatory expectations.<br/><br/>- Establish frameworks for Explainability Testing (SHAP, LIME, XAI) and ensure interpretability of AI outcomes.<br/><br/>- Collaborate with Data Science and MLOps teams to validate models within cloud environments (Azure ML, AWS Sagemaker, Vertex AI).<br/><br/>- Drive verification and validation (V&V) for AI models and applications under FDA and SaMD compliance frameworks.<br/><br/>- Ensure test traceability, documentation, and audit readiness in line with ISO 13485, IEC 62304, and ISO 14971.<br/><br/>- Develop Python-based automation for AI testing, data validation, and model evaluation pipelines.<br/><br/>- Provide technical leadership to QA teams, ensuring alignment with AI/ML development best practices.<br/><br/>- Collaborate with cross-functional teams (Product, Data, Regulatory, Engineering) to identify risks, gaps, and opportunities for test optimization.<br/><br/>- Present validation findings, risks, and recommendations to senior stakeholders and regulatory reviewers.<br/><br/><b>Required Qualifications :</b><br/><br/>- Bachelors or Masters degree in Computer Science, Data Engineering, Biomedical Engineering, or related field.<br/><br/>- 8- 12 years of total QA experience, with 2- 3 years directly in AI/ML or GenAI QA.<br/><br/>- Proven experience in AI Model Validation, LLM/RAG Testing, and AI Evaluation Metrics.<br/><br/>- Strong knowledge of MLOps concepts and cloud platforms (Azure ML, AWS Sagemaker, Vertex AI).<br/><br/>- Understanding of FDA AI/ML Compliance, SaMD testing, and regulated software QA.<br/><br/>- Hands-on expertise in Python automation and testing AI/ML pipelines.<br/><br/>- Excellent documentation and communication skills with ability to produce traceable validation artifacts.</p><br/></p> (ref:hirist.tech)