Why two analysts reach two different verdicts — and how to make every assessment reproducible

Ask a compliance team whether they assess similar cases consistently and most will say yes. Hand two experienced analysts the same file and you may still get two different verdicts. Analyst A might flag a complex corporate structure as 'High Risk' due to the jurisdictions involved, while Analyst B looks at the exact same entity, focuses on the provided documentation, and clears it as 'Medium Risk'.

That gap between the consistency teams assume and the consistency they can prove is where audit findings, hard-to-defend escalations, and slow budget conversations begin. The good news: it is a structural problem, and structural problems can be fixed.

At a glance

• When two analysts reach different conclusions on similar cases, the cause is usually structural, because there’s no shared standard to align to
• A compliance decision is only as strong as your ability to reproduce it: same inputs, same documented steps, same verdict
• Reproducibility, not effort, is what an auditor actually tests
• A configurable risk model and guided assessment workflows make consistency a property of the process, not of whoever opened the file
• The payoff arrives at audit: decisions defensible straight from the record, instead of reconstructed under deadline

Two similar files, two verdicts

It rarely looks like a problem at first. Two analysts, both experienced, work two comparable cases. Each reaches a defensible-looking conclusion. Only the conclusions don't match and nobody notices until an auditor lines the files up and calls the inconsistency a quality defect.

The divergence hides because each case looks fine in isolation:

• Each analyst weighs the same risk factors slightly differently, guided by their own experience
• Cases take different paths depending on who picks them up, because no shared workflow enforces the same steps
• The reasoning behind a decision lives in someone's head or a scattered note, not in the record
• Without operational numbers, nobody sees the drift until it surfaces as a finding

This is the reality behind a pain most Heads of Compliance recognise: you cannot guarantee that similar cases are assessed the same way, because every analyst carries their own yardstick.

Inconsistency is a design gap, not a competence gap

When similar cases get different treatment, the reflex is to question the analysts. It is the wrong place to look. Skilled people applying personal standards will always diverge, because there is no shared standard for them to converge on. Fix the individuals and the problem returns with the next hire. Fix the structure and it closes for everyone.

That reframe matters for retention as much as for quality. Blaming an analyst for an inconsistency the process invited is the fastest way to lose the analysts you most want to keep. The question is not "who got it wrong" but "what would have made both of them reach the same call."

What makes an assessment reproducible

Reproducibility is a plain idea: the same case, measured against the same standard, produces the same verdict — regardless of who does the work or when. If the same analyst might reach a different conclusion six months from now, the assessment was never standardised. It was improvised well.

Reproducibility rests on three things:

• A documented standard the decision is measured against, so the yardstick is shared rather than personal
• A record of the steps and evidence behind each verdict, captured as the work happens rather than reconstructed afterward — the same discipline behind treating primary-source evidence as the basis of a defensible file
• Visible exceptions, where any departure from the standard carries a documented reason

Miss any one and the decision stops being reproducible, which, as the next section shows, is exactly what an audit probes.

How consistency becomes a property of the process

Consistency you have to ask for is fragile. Consistency built into how the work flows holds up. The shift is from relying on individual judgment to encoding the institution's judgment once and applying it to every case.

Three capabilities carry it:

• A configurable risk model encodes the institution's risk logic, the factors, their weightings, the policy, so every case is measured against the same rules instead of each analyst's memory of them. It is the same principle behind moving to a digitised, automated risk policy.
• Guided assessment workflows route every case through the same steps in the same order, so two analysts follow one path rather than two
• Documentation as a by-product: because each step records who assessed what, on which evidence, the audit trail builds itself

Set against the old way, the contrast is stark:

Judgment does not disappear. Analysts still decide the genuinely ambiguous cases. What changes is that they decide on a shared basis, and new team members ramp faster because the standard is written down instead of absorbed over years. Providing this shared basis is exactly what modern compliance automation is built to do. Rather than hoping for consistency, platforms like Sinpex embed it directly into your daily operations.

How to tell whether your assessments are actually reproducible

Before assuming consistency, test it. Walk your team through a short, honest self-check:

• Could two analysts, working the same file independently, be expected to reach the same verdict?
• Is the reasoning behind a decision recorded as it is made, or reconstructed when someone asks?
• When a risk level is overridden, is the justification captured automatically?
• Could you show an auditor, from the record alone, that similar cases were assessed the same way?
• Do you have live visibility into where cases stall and how work is distributed — or only a gut sense?

If the honest answer to any of these is "it depends who you ask," the gap is structural, and it is fixable. Teams facing shorter AML review cycles feel this soonest, because there is no slack left to absorb rework.

Reproducibility is what an audit rewards

The payoff is not philosophical; it shows up the next time an auditor arrives. Decisions that are reproducible from the record turn a long examination into a short conversation. There is no scramble to explain why a case was cleared, because the reasoning is already there.

When a compliance team is supported by a system natively designed for reproducibility, the metrics speak for themselves. Evidence produced organically by the workflow, rather than assembled under deadline, is what stands behind the outcomes our clients achieve with Sinpex — audit trails that hold up under review, with typically a 5–10x reduction in manual effort per case.

Frequently asked questions

What does it mean for a compliance decision to be reproducible?

It means the same case, measured against the same documented standard, yields the same verdict no matter who works it or when. That requires a shared risk model, a record of the steps and evidence, and no undocumented personal judgment.

Isn't standardisation just removing analyst discretion?

No. Analysts still decide ambiguous cases. Standardisation gives them a shared starting point and requires that departures from it are documented, so judgment stays, but it becomes visible and comparable rather than private.

How can we measure whether our assessments are consistent?

Sample similar cases handled by different analysts and compare both the verdicts and the documented reasoning behind them. Persistent divergence points to a missing shared standard, not to weak analysts.

Why do auditors care about reproducibility specifically?

Because an auditor tests whether a decision can be justified from the record, not how experienced the analyst was. A verdict that cannot be reproduced suggests the outcome depended on the person rather than the policy — the inconsistency audits flag.

Where should a team start?

Write the risk model down, route every case through the same workflow, and make documentation a by-product of the work rather than a separate task. Measure consistency before and after, so the improvement is provable.

See it in practice

Want to see how a configurable risk model and guided assessment workflows make consistent, reproducible decisions the default? Book a demo with Sinpex, and we'll walk through it with your team.