Human-controlled AI in recruitment: a practical review checklist

Start with the decision, not the score

An AI summary can help a recruiter find relevant information. It cannot tell the team which trade-offs it should accept for a particular role. Before evaluating a tool, write down the decision it supports: checking a required qualification, preparing interview questions or finding evidence for a debrief. These are different tasks and should not share an unexplained score.

Use a fictional candidate for your first evaluation. Give the system a short CV with one clear qualification, one ambiguous statement and one missing requirement. Keep your expected interpretation beside the test. A useful demonstration shows how uncertainty is handled, rather than whether the output sounds confident. This is a proposed test method, not a claim that a vendor has passed it.

Five checks to run during a product demonstration

Ask the person demonstrating the tool to follow an assessment all the way from its source to a recorded decision. A polished summary is only the beginning of that workflow.

1. Trace a claim: open the source passage supporting it. Check that the passage actually establishes the job-related requirement.
2. Inspect missing evidence: an unstated qualification should remain unknown, not silently become a negative fact about the candidate.
3. Find the decision boundary: identify the human action required before a consequential stage change or candidate message.
4. Correct a mistake: change an inaccurate interpretation and check whether the next reviewer can understand the correction.
5. Identify ownership: locate the person responsible for the next step and the evidence they will review.

Example: an ambiguous project description

Suppose a fictional CV says, ‘Supported the launch of a new service.’ The role requires independently leading a launch. That sentence establishes involvement, but not independent ownership. A defensible review note would separate the source, the interpretation and the unanswered question: ‘What part did you own, which decisions did you make, and who approved the launch?’

Do not turn that uncertainty into either an automatic pass or a rejection. Ask for the missing evidence and record the answer. If another candidate provided clearer documentation, that difference should be visible without pretending that the first candidate lacks the underlying ability.

Run a small pilot before expanding use

Choose one role, agree on the assessment criteria and have a reviewer examine every output in the pilot. Count unsupported claims, missed evidence and corrections that fail to reach the next reviewer. Keep the source material fixed when comparing versions. Changing both the input and the model makes it difficult to explain why an output changed.

Measure reviewer time as well as accuracy of the extracted evidence. A summary that takes longer to verify than reading the source has not demonstrated an operational benefit. Record the sample size, role and time period alongside any result. A pilot is not a general proof of fairness or legal compliance.

Keep claims narrower than the evidence

Avoid labels such as ‘bias-free’, ‘fully compliant’ or ‘guaranteed best hire’. A clear review process is valuable, but it does not establish those claims. Ask how candidate information is processed and who can access it; use the current vendor documentation and your organisation’s review process before introducing a new use.

Use the checklist below as a meeting record. Mark each check demonstrated, partly demonstrated or not demonstrated, then attach the example that supports your conclusion. That produces something your team can revisit when a feature changes.