Arthur J. Gallagher and Co.

Senior QA Engineer (AI Testing)

Arthur J. Gallagher and Co. Colombo, Western Province, Sri Lanka

Insurance · 10,001+ employees

2 h ago
qa Senior (5-10 yrs) Full-time Sri Lanka
Create a free account to apply — email only, no card. You can also save this posting or score it against your profile with AI.

About the role

The Senior QA Engineer will develop and maintain comprehensive test strategies for AI-enabled solutions, including evaluating accuracy, robustness, and safety. They will also collaborate with stakeholders to define acceptance criteria and monitor AI quality in production environments.

What they look for

AI Testing Generative AI Machine Learning Intelligent Automation Prompt Engineering Retrieval-Augmented Generation Test Strategy Development Automated Regression Testing CI/CD Pipelines Azure DevOps Postman JMeter Performance Testing Security Testing Accessibility Testing Agile Methodologies

Requirements

Candidates should have proven experience in software testing with specific expertise in AI risks, retrieval-augmented generation, and prompt engineering. Familiarity with CI/CD platforms, secure software engineering practices, and Agile methodologies is required.

Full description

Introduction

At Gallagher, we help clients face risk with confidence because we believe that when businesses are protected, they’re free to grow, lead, and innovate. You’ll be backed by our digital ecosystem: a client-centric suite of consulting tools making it easier for you to meet your clients where they want to be met. Advanced data and analytics providing a comprehensive overview of the risk landscape is at your fingertips. Here, you’re not just improving clients' risk profiles, you’re building trust. You’ll find a culture grounded in teamwork, guided by integrity, and fueled by a shared commitment to do the right thing. We value curiosity, celebrate new ideas, and empower you to take ownership of your career while making a meaningful impact for the businesses we serve. If you’re ready to bring your unique perspective to a place where your work truly matters; think of Gallagher.

Overview

The Senior QA Engineer (AI Testing) ensures the quality and reliability of AI-enabled solutions across their lifecycle, including testing, evaluation, and production monitoring. The role involves collaboration with stakeholders to define acceptance criteria, assess risks, and implement testing strategies for AI solutions such as generative AI, machine learning, and intelligent automation. Based in Colombo, Sri Lanka, this position reports to the QA Practice Lead.

How you'll make an impact

  • Develop and maintain comprehensive test strategies for AI-enabled solutions.
  • Evaluate AI behaviour for accuracy, robustness, fairness, and safety.
  • Test AI-specific risks, including hallucination, data leakage, and model drift.
  • Build automated regression suites and integrate them into CI/CD pipelines.
  • Collaborate with stakeholders to clarify requirements and communicate quality risks.
  • Monitor AI quality in production and contribute to continuous improvement.
  • Stay updated on industry trends and introduce advanced testing tools and techniques.

About you

  • Proven experience in software testing, including AI-specific testing.
  • Expertise in AI risks, retrieval-augmented generation, and prompt engineering.
  • Familiarity with secure software engineering practices and Agile methodologies.
  • Experience with performance, accessibility, and security testing.
  • Knowledge of CI platforms (e.g., Azure DevOps) and tools like Postman and JMeter.
  • ISTQB certification and training in AI or machine learning are advantageous.

This role is ideal for candidates passionate about advancing AI testing practices in a collaborative environment.

#LI-Hybrid

Similar roles