Research Engineer - Data Eval

About us:

Calder is solving the data bottleneck for embodied AI and world models. Today's AI models lack the real-world, 4D data needed to reason on spatial problems and operate in physical environments. We capture, process, and deliver high-fidelity human-generated spatial data at scale, providing the substrate that trains the next generation of generalizable models for manipulation, navigation, and interaction.

About the role:

Calder collects, curates, and quality-evaluates real-world and egocentric capture data used to train world models and robotic foundation models. Data quality is our core differentiator, and this role owns the measurement loop that proves it — from raw capture all the way to downstream model impact.

The work focuses on two forms of data evaluation:

• Running ablations: training, fine-tuning, and benchmarking open-source world models and robotic policies to quantify how our collection, curation, and labeling choices actually affect downstream performance, and how we compare to public baselines.

• Model-based validation of the raw data we collect to flag low-quality, corrupted, or out-of-distribution captures: catching defects like motion blur, exposure problems, dropped frames, sensor desync, calibration drift, and SLAM or stereo-depth failures to make sure problems surface early rather than after labeling.

You will also:

• Help set the direction for Calder's evaluation research agenda, identifying the open problems that matter most for embodied and physical AI, and running disciplined experimental programs to chase them down.

• Invent benchmarks, metrics, and methodologies that probe how frontier robotic and world models sense, reason, and act.

• Run model evaluation end to end, from offline studies and A/B or regression testing to live, in-the-wild deployment, spanning diverse sensor stacks, long-tail edge cases, and real-world tasks across manipulation, navigation, and interaction.

• Stand up the pipelines, dashboards, and tooling that make evaluation fast, reproducible, and largely automated for both internal teams and external partners.

• Hunt for the corner cases, failure modes, and safety, privacy, and risk issues and feed what you learn back to the research and robotics teams to make the next models stronger.

• Provide leadership and mentorship across cross-functional efforts, while leaning into the pace, while building out our evaluation team.

About you:

You move fast and take ownership, turning recent papers into prototypes and tools that run at scale on real sensor and trajectory data. You lead technically, mentor naturally, and partner well across research, robotics, engineering, and product. You communicate clearly, document thoroughly, and thrive in the ambiguity and pace of an early-stage company.

You should be comfortable with the ML experimentation stack (PyTorch, training/eval infrastructure, open-source model ecosystems), rigorous about experimental design, and skeptical by instinct. Experience with robotics, imitation learning, world models, computer vision models, or large-scale data pipelines is a strong plus.

• A history of shipping high-impact research in fast-moving settings, backed by first-author papers at venues such as NeurIPS, ICML, ICLR, CVPR, CoRL, RSS, ICRA, or IROS.

• Sharp mathematical foundations and real depth in at least one area near the heart of physical AI evaluation, whether that's reward modeling for embodied agents (including reward hacking and specification gaming), calibration and uncertainty quantification for perception and policy models, simulation-to-real validity, or the human side of AI through active learning, annotation quality, and preference elicitation on spatial data.

• Hands-on experience with modern ML systems across both research and production, with meaningful exposure to evaluating physical AI such as robotic manipulation, navigation, world models, vision-language-action models, or multimodal egocentric perception.

• Fluent in Python and strong on the engineering side, comfortable building scalable data pipelines, evaluation harnesses across sim and real-world deployments, and the metrics that make embodied model quality legible.

• PhD in Computer Science, Machine Learning, Robotics, Computer Vision, or a closely related discipline, ideally with a thesis or research focus that touches evaluation-adjacent topics like reward modeling, preference learning, human feedback, calibration, or automated assessment for embodied or perception systems.

Calder is an equal opportunity employer. All applicants will be considered for employment without attention to race, color, religion, sex, sexual orientation, gender identity, national origin, veteran or disability status. Learn more here.

By applying for this position, you consent to the collection, storage, and processing of your personal data by Calder for recruitment purposes. We will handle your information securely and in accordance with local privacy policies. Your data may be shared with authorized third-party service providers aiding in the hiring process.