The Sanity Check Was an Accident

I read a post on r/analytics about a company that had been using invented numbers for 3 months. The part I keep coming back to is how they found out: somebody asked the analyst to double-check one figure.

There was no control. Nobody noticed a strange confidence score or found a broken source. A person just asked a question.

I keep trying to turn that into something repeatable. The obvious fixes don’t get me very far. Citations still have to be opened. A scheduled audit tests the cases somebody thought to include. Confidence scores are not much help when the system is confidently wrong. Spot-checking works until the 20th check finds nothing and everyone has other work to do.

The best I have so far is to make checking a claim cheap. Put the source next to it. Show when the source was observed. Keep the path from the sentence to the evidence short enough that somebody can verify it without starting a research project. Then record how often people actually check and what they find.

But that still leaves an owner problem. The report saves time for everyone who reads it. Verification spends the time of the person doing the checking, usually to confirm that nothing is wrong. I don’t know many organizations that are good at protecting that kind of work.

So I don’t think the answer is simply to keep a human in the loop. Which human? Looking at what? How often? With enough time and authority to stop the process when the evidence doesn’t hold?

In this case, the safety system was a person who happened to ask. I don’t think we have a reliable replacement for that accident yet.