JobCubby JobCubby
Sign up
Sofico

Senior Test Automation Engineer - Data & AI Team (m/f/d)

Sofico Ghent, East Flanders, Belgium

Software Development · 501-1,000 employees

5 h ago
Senior (5-10 yrs) Full-time Belgium
Log in to apply, save this posting, or score it against your profile with AI.

About the role

You will design and maintain a robust test automation strategy for the Data & AI platform, covering unit, integration, and end-to-end testing. Additionally, you will develop evaluation approaches for AI-powered agentic behaviors and collaborate across engineering teams to ensure platform-wide quality.

What they look for

Test automation Java Python API testing CI/CD Microservices Distributed systems LLM evaluation Agentic systems Playwright Kubernetes Contract testing Regression testing End-to-end testing Cloud-native technologies Workflow orchestration

Requirements

The ideal candidate has strong experience in test automation frameworks, CI/CD integration, and backend service testing. Proficiency in Java or Python and experience with AI evaluation or distributed systems are highly valued.

Benefits

Competitive salary package Meal allowance Private health insurance Hybrid working Flexible hours Structured onboarding program Team activities

Full description

At SOFICO, we're building the future of mobility through Miles, our market-leading platform for leasing, fleet, and mobility operations worldwide. As an international team, we combine innovation, collaboration, and ownership to deliver real impact for our customers, in a culture of continuous learning, mutual support, and shared success.

From our office in Ghent, we are scaling our Data & AI organization, the team behind SOFICO AI Cloud and the Miles Enterprise Agentic Platform (AutoPilot & AutoRoute). Our platform combines AI, cloud-native technologies, and enterprise software, leveraging Java/Quarkus services, Python/LangGraph agents, Azure AI Foundry, Temporal workflows, and Langfuse observability.

As a Senior Test Automation Engineer, you will drive the quality and reliability of this platform by building and owning a robust test automation strategy across the entire AI ecosystem. Working closely with engineers across AI Runtime, Cloud Ops, API & Saga, Knowledge, Governance, and Data teams, you will ensure that critical business flows, integrations, and AI-powered capabilities are validated through scalable, automated testing.

What you’ll do...

Deterministic testing (classic, pass/fail quality gates)

  • Own and evolve the test automation strategy for the Data & AI platform — unit, integration, contract, regression, and end-to-end — designing, building, and maintaining automated frameworks and suites integrated into CI/CD pipelines.
  • Validate end-to-end business processes within sandboxed Miles environments, ensuring correctness, reliability, and safe failure handling.
  • Develop API and contract testing approaches for distributed services and integrations.

Non-deterministic testing (AI and agentic behavior)

• Design and own evaluation approaches for AI-powered and agentic capabilities where outcomes are probabilistic rather than pass/fail: LLM-as-a-judge setups, baseline/benchmark datasets, and process/quality metrics for agent behavior.

• Establish baselines and regression tracking for model and agent output quality over time, this alongside the deterministic CI gates.

Platform-wide

  • Collaborate across engineering domains to ensure platform-wide quality and release confidence.
  • Drive continuous improvement in test automation practices and tooling, and act as a mentor and quality advocate, helping teams adopt effective automated testing approaches.

What you bring...

We're looking for a strong overall profile, without expecting you will tick every box.

Please indicate in your application your current experience level (e.g. none / familiar / proficient / expert) for each of the areas below. This helps us understand your profile and have a more focused conversation.

  • Test automation engineering: unit/integration/e2e frameworks, CI/CD integration
  • Testing APIs, backend services, microservices, and distributed systems
  • Programming in Java and/or Python, including typed schemas, mocking, and test-data setup/teardown
  • Designing scalable automation frameworks from the ground up, and coaching others in their adoption
  • Non-deterministic / AI evaluation testing: LLM-as-a-judge, benchmarking, quality metrics for agentic systems (plus)
  • Workflow orchestration platforms, cloud-native applications, and LLM/agentic systems (plus)
  • Tooling such as Playwright, Kubernetes, and modern DevOps practices (advantage)

An analytical mindset, attention to detail, and a genuine drive to improve software quality matter as much as any single line above.

What's in it for you…

  • Competitive salary package.
  • Meal allowance provided per working day.
  • Fully covered private health insurance, so you can focus on what matters most.
  • Hybrid working and flexible hours, giving you the freedom to shape your workday.
  • A modern co-working space offering a comfortable working environment.
  • A structured onboarding program from day one, with a dedicated coach to help you get up to speed and grow in your role.
  • Regular opportunities to connect beyond day-to-day work — casual drinks, team activities, and company-wide events.

Ready to build the secure and deterministic capabilities behind the next generation of mobility software? Join SOFICO and let's shape what's next together!