JobCubby JobCubby
Sign up
Dailoqa

Senior QA Engineer – Agentic AI Testing & Evaluation

Dailoqa Noida, Uttar Pradesh, India

Business Consulting and Services · 201-500 employees

Yesterday
qa Senior (5-10 yrs) Full-time India
Create a free account to apply — email only, no card. You can also save this posting or score it against your profile with AI.

Applying here? Try the free cover letter tool — paste this posting and your résumé, no account needed.

About the role

The Senior QA Engineer will own the quality, testing, and evaluation strategy for the Broccoli Platform, focusing on AI agents and multi-agent workflows. They will collaborate with cross-functional teams to ensure the delivery of reliable, secure, and scalable AI solutions.

What they look for

Quality Assurance Test Automation Python Java JavaScript TypeScript Selenium Playwright Cypress API Testing CI/CD Azure DevOps Large Language Models Agentic AI Performance Testing Security Testing

Requirements

Candidates must have 7-10 years of experience in quality engineering with strong proficiency in test automation frameworks and cloud platforms. Deep expertise in LLMs, AI evaluation, and performance/security testing is essential for this role.

Full description

About the Role

Dailoqa is looking for an experienced Senior QA Engineer – Agentic AI Testing & Evaluation to own the quality, testing, and evaluation strategy for our Broccoli Platform. This role will be responsible for establishing robust testing frameworks for AI agents, multi-agent workflows, orchestration layers, tools, and enterprise-grade AI applications.

The ideal candidate will have deep expertise in test automation, quality engineering, AI/ML validation, security testing, and performance engineering, with a strong understanding of Large Language Models (LLMs), Agentic AI systems, and modern DevOps practices.

You will work closely with Capability Leads, AI Architects, Product Teams, and Platform Engineers to ensure that Broccoli delivers reliable, secure, scalable, and high-performing AI solutions.

Requirements

Required Skills & Qualifications

Technical Expertise

  • 7–10

years of experience in Quality Assurance, Test Automation, or Quality Engineering.

  • Strong

experience with automation frameworks using Python, Java, or JavaScript/TypeScript.

  • Hands-on

experience with:

  • Selenium,

Playwright, Cypress, or Robot Framework

  • API

testing tools such as Postman, Rest Assured, or Karate

  • CI/CD

platforms including Azure DevOps, GitHub Actions, Jenkins, or GitLab

  • Experience

with cloud platforms such as Azure, AWS, or Google Cloud.

Agentic AI & LLM Testing

  • Understanding

of:

  • Large

Language Models (LLMs)

  • Retrieval-Augmented

Generation (RAG)

  • AI

Agents and Agent Orchestration Frameworks

  • Prompt

Engineering and Evaluation

  • Experience

with AI evaluation platforms and frameworks is preferred.

  • Knowledge

of hallucination detection, safety validation, guardrails, and model benchmarking.

Performance & Security Testing

  • Experience

with performance testing tools such as JMeter, k6, or Locust.

  • Knowledge

of application security testing and threat modeling.

  • Familiarity

with AI security testing, adversarial testing, and governance controls.

Soft Skills

  • Strong

analytical and problem-solving skills.

  • Excellent

stakeholder management and communication abilities.

  • Experience

working in cross-functional Agile teams.

  • Ability

to lead quality initiatives in fast-paced, innovation-driven environments.

Preferred Qualifications

  • Experience

testing AI-powered enterprise applications.

  • Exposure

to Microsoft Azure AI, OpenAI, LangChain, LangGraph, CrewAI, Semantic Kernel, or similar agent frameworks.

  • Certifications

such as:

  • ISTQB

Advanced Test Analyst/Test Manager

  • Certified

Agile Tester

  • Azure

AI Engineer Associate

  • Azure

DevOps Engineer Expert

  • Experience

working with BFSI, enterprise transformation, or regulated industry environments.

Similar roles