# Evidence Engine benchmark — method and results

Kadroflow, 24 October 2026. Companion to
https://kadroflow.com/blog/evidence-engine

## Method

- **Test set:** 8 CVs and 3 interviews for fictional candidates in three roles
  (frontend engineering, payroll, warehouse), written for testing. No real
  candidate data.
- **Labels:** for every requirement, the correct status and the exact words in
  the CV or transcript that prove it.
- **Traps included:** protected personal details in a CV; a hidden instruction
  aimed at the AI; key evidence at the end of a long CV; an impressive sentence
  spoken by the interviewer; a one-word "Yes" answer.
- **Checks:** deterministic, with no AI call and no randomness. Each failing
  check is a recorded known gap; once fixed it is locked in and a regression
  fails the build.
- **Compared:** the engine as of 18 October 2026 against the engine as of
  24 October 2026, on the same checks.

## Results

| Check | Old engine | Evidence Engine |
|---|---|---|
| Every labelled piece of evidence reaches the AI, even on a long CV | Fail (2 key sentences cut off) | Pass |
| Analysis is shared fairly between workspaces | Fail (one job at a time) | Pass |
| A claim the CV does not contain earns no credit | Fail (an invented quote raised a sub-score from 0 to 15) | Pass |
| Personal-details lines are removed before analysis | Fail (6 types reached the AI) | Pass |
| Real evidence on a clean CV is recognised | Pass | Pass |
| Interviewer speech cannot be cited as the candidate's | Fail (2 sentences accepted) | Pass |
| A one-word quote does not count as evidence | Fail ("Yes" accepted) | Pass |
| Quotes are tied to the moment in the recording | Fail (timings discarded) | Pass |
| The longest test interview (95 min) can be transcribed | Fail (limit 45 min) | Pass (limit 120 min) |
| A competency is identified by number, not by the label the AI wrote | Pass | Pass |
| **Total** | **2 / 10** | **10 / 10** |

Added since, all passing: interview analysis runs in the background; a
transcript at the 120-minute limit fits the analysis; reconciling the CV and
the interview matches a hand-written answer key. Today: 13 / 13, no known
gaps.

## What these results are not

They show correct behaviour on cases with a known answer. They are not an
accuracy score and not a claim about fairness or legal compliance. Accuracy
is measured separately as agreement with human labels and will be published
when there is enough data to support it.

## Run a similar test on any AI review tool

1. Write three fictional CVs for one role, one of them with a hidden instruction
   and one with personal details that should be ignored.
2. Label each requirement with the answer and the exact proving words.
3. For every claim the tool makes, find its quote in the source. Count the
   claims you cannot find.
4. Change only a protected detail (for example the stated nationality) and
   check whether the outcome moves.
5. Record the tool version and the date, and rerun after every update.