The question that arrives in month nine.

The question arrives in month nine, from somebody whose job is to ask it, and it is always narrower than teams prepare for.

Not “how does the model work”. Not “is it fair”. Something like: which applications were declined at pre-screening in the first fortnight of March, and on what basis?

That is a reasonable question. It is also, for most deployments, unanswerable without eleven days of work.

Of sixty-one deployments followed from demonstration to production or abandonment, seventeen treated the evidence requirement as a design input and thirteen reached production. Fifteen left it to a review stage and none did. Full method and limits in Report 01.

Why the reconstruction is expensive

The information usually exists. That is what makes this feel like a small problem, right up until somebody tries.

In a typical case the answer lives across three systems. The application record in one place, the model’s inputs in another, the output and its confidence in a log written by an engineer for debugging. The model version that produced it has since been superseded twice, and which version ran on which date is inferable from deployment records if those were kept properly.

Assembling that is doable. In the case we watched most closely it took eleven days of two people’s time and produced an answer the audit function accepted with reservations.

An audit function exists to be unconvinced. Handing it a reconstruction is handing it exactly the thing it is trained to doubt.

The reservations matter more than the eleven days. A reconstruction is an argument that the history was probably this. It is not a record that it was.

What the artefact actually is

Across the deployments that treated evidence as a design input, the artefact was consistent and much simpler than teams expect before they build one.

A per-decision record, written at the moment of the decision, carrying five things:

• The inputs the system saw — what was actually passed, at the point it was passed
• The version that produced the output — written at decision time, not inferred later
• The output itself — including confidence or score where one exists
• Who accepted it, and when — the field most often missing, and the one asked about first
• What happened next — accepted, declined, escalated, or timed out

Nine of thirteen wrote this to a database table. Four wrote to an append-only log. None reconstructed it from application logs.

That is a day or two of engineering, specified at the start. Retrofitting the same thing afterwards was estimated at three to six weeks where it was attempted, and in four of five cases it was never done — not because it was impossible, but because the project had spent the goodwill it would have needed.

Why it gets deferred

Three reasons, all reasonable in the moment.

The requirement is not known yet. Legal or risk have not been engaged, and engaging them early feels like inviting scrutiny before there is anything to scrutinise. So it goes on the list as “confirm with Risk”, and in one case we followed that line stayed there for nine months.

It looks like a compliance step. Evidence sounds like something that happens at the end, alongside sign-off and documentation. Framed that way it sequences last, and by then the system’s shape is fixed.

The pilot does not need it. True, and it is the trap. A pilot with twelve cases can have its history reconstructed by hand in an afternoon. That success gets read as evidence that reconstruction is feasible at scale, which it is not.

Design input versus compliance step

The distinction is not only about timing. It is about what form the requirement takes.

A design input is a specific question with the artefact that answers it: “Risk will ask which applications were declined in a given period and why, so each decision writes a record carrying inputs, version, output, acceptor and outcome.”

A compliance step is a category: “We need to be able to explain decisions.”

The first can be built. The second is a heading under which nothing gets built, and it is what appears in the records of the deployments that failed here.

Getting the first requires one conversation with whoever will ask the question, before the build. In the cases that shipped, it happened in the first three weeks and took under an hour.

The question to ask this week

Pick a decision your system made last month. Ask what would be needed to explain it to somebody who was not there — the inputs, the version, the acceptor, the outcome.

If the answer involves opening more than one system, or inferring the version from deployment history, you have a reconstruction rather than a record. Fixable now at a cost of days. Fixable in month nine at a cost of weeks, if there is anyone left willing to fund it.

This is also what an agent needs, for the same reason — covered in what an agent needs.

Run the governance assessment · Read Report 01