Evaluation Lab

Know your AI works before it goes live.

Evaluation Lab runs your agents and workflows on your own cases, scores every answer and compares each new version with the last, so you release only what meets your quality bar.

Evaluation Lab · Support workflow v311 configurations · 120 cases · 3 repeats
80%90%100%0¢10¢20¢30¢40¢Cost per caseYour quality bar · 92%QualityBalanceEconomy
Recommended sets
Quality96%34¢ p90 51¢
Balance94%14¢ p90 21¢Recommended
Economy87%6¢ p90 9¢Below your bar
Balance, step by step
ClassifyClaude Sonnet 5.5Extractqwen2.5:7bCreate ticketScript
Release as v4
1Every dot is a configuration

A different model at each step, run on the same 120 cases three times, so the scores compare.

2Above your bar, the lowest cost

Balance keeps 94% on your cases at 14¢ a case. Quality adds two points for more than twice the cost.

3Sometimes a script is enough

Creating the ticket needs no model at all, so that step costs nothing.

What it is

Before AI does real work, you need three answers: is it good enough for your work, did the last change break anything, and what does each answer cost? Evaluation Lab gives you all three, measured on your company's own cases, for every agent, every workflow and every new version.

01

Good enough for your work

Quality is measured on the questions your customers really ask, against the answers your experts expect.

02

Safe to change

Every new prompt, document or model runs the same cases first, so you see what improved and what broke before a customer does.

03

Worth what it costs

It finds the lowest-cost mix of models that still clears your bar, so the price of each case is measured, not guessed.

01 · Your own cases

Tested on the work your company really does.

Your team collects real requests: single questions or whole dialogs, each with the answer you expect or the rules it must follow. Judge models score every answer against them, and the same cases run on every new version.

Etalon suite 1Customer support · 120 cases
Expected answer

Offer a refund or a replacement, and ask for a photo of the damage, as returns policy §3 says.

JudgesetalonPassfaithfulnessPassrelevancePasssafetyPass
Expected answer

Update the address, cancel the order when asked, and confirm both in writing.

JudgesetalonPassfaithfulnessPassrelevancePasssafetyPass
Expected answer

No. Offer the nearest Saturday or Monday slot, with its delivery price.

JudgesetalonPassfaithfulnessPassrelevancePasssafetyPass
Expected answer

Not for this order type. List the payment options the store accepts.

JudgesetalonPassfaithfulnessPassrelevancePasssafetyPass
02 · Every change compared

See what a change fixed, and what it broke.

Every new version runs the same cases. Evaluation Lab compares it with the last one case by case, and names every case that used to pass and now fails.

Compare runs · Support workflow v4 → v5The same 120 cases, one square each
Pass rate 94% → 97%
Passing in both111
Fixed in v55
New fails in v52
Still failing2
NEW FAIL

Can I pay half now and half on delivery?

Passed in v4 · fails in v5 · judge: etalon
Why combinations

A workflow is a chain of steps, and each step needs a different amount of intelligence. Understanding a customer's message takes judgement; filling in a ticket takes none. The strongest model on every step pays the top price for work that a smaller model, or a plain script, does just as well.

01

Each step on what it needs

Judgement goes to a strong model, routine work to a fast, low-cost one, and fixed rules to a script.

02

Lower cost at the same quality

The quality bar you set stays where it is. The price of each case comes down.

03

Ready for the next model

New models arrive every few months. Run your cases on one, and you know whether it is better or cheaper for your work before you switch.

04 · A model for every step

A model for every step, and a script where one is enough.

Evaluation Lab tries the models you allow at each step of a workflow, on your own cases. At every step it keeps the cheapest choice that holds your quality bar, and a plain script wherever no model is needed.

Support workflow v3 · step by step120 cases · your bar at each step: 95%
Claude Sonnet 5.5Sonnet 5.5GPT-6.1GPT-6.1Claude Haiku 5.5Haiku 5.5qwen2.5:7bqwen2.5ScriptScriptKept
Classify the request98%13¢93%11¢90%3¢84%1¢—Claude Sonnet 5.5
Extract the order details99%9¢98%8¢97%2¢96%1¢71%0¢qwen2.5:7b
Create the ticket100%4¢100%3¢100%1¢99%0.5¢100%0¢Script
Balance · 94% end to end · 14¢ a caseThe Quality set: 96% · 34¢ a case
Why we measure

AI earns trust the way people do: measured on real work, then trusted with more. Models, prompts and knowledge keep changing, and each change can move an answer. So every version is measured on your own cases, for quality and for cost, before it goes live and while it runs.

05 · From test to live

A gate before release, and checks after it.

A version is published only when a full run clears your bar. After release, scheduled runs and sampled answers keep checking it, and every miss becomes a new case.

Before releaseAfter release
Before release
01Run your cases

Every case in the suite, on the version you want to release.

Run all · v5v5 · 120 of 120
02Clear the gate

Only a full run on the agent's own model counts. Below your bar, the version waits.

Gates publishing for Support agent94% ≥ 80% ✓
03Release

A person publishes it, and the run that cleared the gate stays on record.

PublishLatest published · v5
After release
04Watched while live

Your cases run on a schedule, and samples of real answers are checked against the same bar.

Nightly run · no regressionsSampled live answers · 96% pass
05Every miss becomes a case

A real conversation that went wrong is saved as a test, so the next version is checked against it.

Conversation #4821 · Save as a testAdded to Etalon suite 1 · 121 cases
Every miss goes back into your cases, and the loop starts again
06 · Who runs it

Measured with our engineers, or by your own team.

CasesYour experts and our engineersRunEvery version, on your casesCompareQuality and cost side by sideHand overYour team runs it next
Forward-deployed engineers

We measure it with you

Our engineers build the first case set with your experts, measure quality and cost on it, and walk your team through the results.

New caseFrom a real request · by your teamRun all · v5120 of 120 · by your teamGate94% ≥ 80% · by your teamPublishLatest published · v5
Your team

Run it yourselves

With support and recommendations from our engineers, your team writes cases, runs them and publishes a version when it clears your bar.

FAQ

Questions about testing AI.

Short answers. Ask us anything else on a call.

Talk to an engineer →
How many cases do we need to start?A few dozen+

Start with the questions your customers really ask. A few dozen real cases are enough to begin, and the set grows as new situations come up.

Who writes the cases?Your experts+

The people who know what a right answer looks like. Our engineers help them turn real requests into cases: single questions or whole dialogs, each with the answer you expect.

Can we trust a judge model?Your answers+

Judges score each answer against the answer you expect: is it right, is it grounded in your sources, is it safe to send. Your experts can open any case and see the answer, the verdict and why.

Does it work in our language?Your language+

Your cases are written in the language your customers use, and the answers are scored in it.

Where does it run?Spaceplace Cloud+

In Spaceplace Cloud, on the models you connect. Ask us about running it on your own servers, with local models.

What does a run cost?Shown first+

Each run shows its cost estimate before it starts, and the cost of every answer stays on record.

What Spaceplace can do for your business

Bring the questions your customers really ask. We'll show you how Evaluation Lab scores the answers.

Book a demo

Documents you need
Privacy

Cookie settings

Choose what this site may use. Necessary cookies keep it working; the rest are off until you switch them on. How we use them

Necessary

Keep the site working and secure, send the forms you fill in, and remember this choice.

Always on
Analytics

Show which pages people read and where they come from, so we can make the site better.

Marketing

Measure which ads bring people here, and show our ads to people who have visited.