Research Engineer, Benchmarking - Member of Technical Staff
Callosum · London
Don’t apply blind. See how your CV matches Engineer first — free, in 30 seconds.
You will leave NewLuxJob. We do not receive or handle applications.
- Company
- Callosum
- Location
- London
- Posted
- August 20, 2026
About this job
About the Role Choosing between algorithmic strategies for multi-step LLM work is a measurement problem, and most teams solve it badly: comparisons run case by case, by whoever needs them that week, on whatever task is closest to hand. That doesn't scale, and it doesn't hold up to outside scrutiny - from a customer, or from a reviewer. Callosum needs one benchmarking system: reproducible, contamination-controlled, and trusted enough to be the evidence that decides which approach ships. This role owns that system. You will build a harness that measures task success, quality, and robustness across motifs, agent topologies, and decomposition strategies, grounded in execution - real commits, real traces, sandboxed grading - rather than self-reported or model-graded scores.…
This is a short summary.
Want to know if you're a fit? Check your CV against this role — free, in 30 seconds.
Engineer salaries in Germany
Based on 331 postings that disclose pay.
Estimated from advertised pay across our sources — indicative gross annual figures, not an offer.
Similar jobs
See if your CV fits this job
Paste your CV for an instant match score against this role — and get a tailored cover letter in one click.
- Instant match score for this role
- Tailored cover letter in one click
- Free — no credit card