RADcube - A NLogix Company

Validation Engineer, AI & Data

RADcube - A NLogix Company Indianapolis, Indiana, United States

IT System Custom Software Development · 201-500 employees

6 d ago
Mid (2-5 yrs) Full-time United States
Create a free account to apply — email only, no card. You can also save this posting or score it against your profile with AI.

About the role

Design and implement testing strategies for AI models, business context repositories, and ontology mappings to ensure response accuracy. Develop automated evaluation frameworks, regression tests, and reporting dashboards to validate AI performance against ground truth data.

What they look for

Data Quality Engineering SQL Python Automated Testing PyTest ETL Data Validation AI Evaluation LLM Evaluation Ground Truth Testing Regression Testing Test Automation Ontology Mapping Context Injection Data Analysis

Requirements

Requires technical expertise in data quality engineering, SQL, Python, and automated testing frameworks like PyTest. Candidates must have experience with ETL processes, AI/LLM evaluation, and data validation methodologies.

Full description

Validation Engineer, AI & Data

Location: Indianapolis, IN

Summary: Validate the accuracy, reliability, completeness, and consistency of business context and AI generated responses against approved business requirements and ground truth answers.

Responsibilities:

  • Design and implement the testing strategy for the Domain Context Model, Business Context Repository, ontology mappings, and AI response evaluation.
  • Develop representative ground truth question and answer test sets with defined expected results and tolerances, validated by data owners.
  • Build and maintain automated evaluation tests for AI responses generated through the client’s Context Injection toolkit.
  • Evaluate AI responses with and without domain context to establish baseline accuracy and measure improvements.
  • Validate numeric answers, SQL and query results, business logic, aggregations, join relationships, and narrative responses.
  • Identify and classify failures, including hallucinations, stale information, incorrect schemas, faulty joins, missing business context, and persona mismatches.
  • Develop repeatable evaluation scripts, scoring rubrics, regression tests, and accuracy reporting dashboards or scorecards.
  • Verify context repository data completeness, classification consistency, access control behavior, and changes introduced through automated context updates.
  • Document test evidence, defects, remediation results, AI readiness assessments, and validation outcomes.
  • Support release readiness, weekly demonstrations, and handover of reusable evaluation frameworks and documentation.

Requirements

Required Technical Expertise: Data quality engineering, SQL, Python, automated testing, PyTest or equivalent frameworks, ETL and data validation, AI and LLM evaluation, ground truth testing, regression testing, and test automation.