Ben Hollebon

Sent is not delivered

Systems report on the thing they can see, which is usually their own action. A green tick means the message left, not that it arrived or that anyone read it.

Red and black GR post office letter container

The systems I build report on what they can see, which is usually their own actions. They know they sent the email. They have no idea whether anyone read it. So the report says sent, and sent gets filed as though it meant delivered.

The gap isn’t evenly spread. A failure that looks like a failure gets fixed the same morning, because someone sees a red light and goes looking. One that reports success can run for months, because nothing in the system is asking the question that would expose it.

I have a job that reads two school calendar feeds and emails me what’s coming up and what has changed since the last time it looked. One week it reported 115 new events and nothing changed. Of those 115, 108 were events it had already told me about, unchanged and word for word.

One of the two feeds issues a fresh identifier for every event each time you fetch it. The comparison was keyed on those identifiers, so the two copies had nothing in common and the whole feed looked new. It could never have got that right. The other feed’s identifiers are stable, and its half of every report was correct, which helped the fault hide.

The check that matters is the one nobody automates: does the person at the far end now know something they didn’t before?

The calendar job had been running wrong for weeks. Another job, which couldn’t read its source, said “nothing new this week” and reported success. The rule I now keep in my notes is that “I couldn’t read it” and “I read it and nothing changed” must never share an exit path.

Two more from my own systems

There was a status check that confirmed every service on a machine was up, and it ran only at boot. The machine had been up for nine days when the thing the check existed to catch went wrong. The check wasn’t running then and had nothing to say afterwards. Anything that deteriorates while a system is running is invisible to it, and a surprising amount of failure happens that way.

The third one sits on a wall at home and shows the household’s week. Nobody reloads a display on a wall and nobody opens its console, so it can sit there being wrong for as long as you let it. One morning it loaded an old cached copy of a script it relies on, failed to parse it, then rendered the page perfectly and never updated again. There was no error and no blank panel, just a tidy picture of a day that was already over.

Reports about people

The same thing happens with people in the loop, and it’s harder to catch because nothing throws an error. “The policy was circulated” is a true statement about an action. It’s also compatible with a distribution list nobody has curated in two years. The message could land in a folder that gets opened once a quarter, and most of the people it was written for might never see it. “The training is published” says something about a file and nothing about whether anyone has done it. Consultation reports rarely say who replied, or whether the people most affected were among them.

The person writing the report isn’t being dishonest. They do what the machine does and report on the part of the work they have evidence for, which is the sending. They’ve no way to measure the receiving, so the true and available number goes in the box. The meeting then reads a number about activity as a number about effect. I’ve written those reports and read them into minutes.

The cost shows up later. The people at the far end know perfectly well whether something reached them. When a report says a thing landed and they know it didn’t, they take it as evidence that the reporting is decorative. After that every green tick gets discounted, including the true ones, and you can no longer tell someone that something is fine and be believed.

Where receipt can be measured, measure it: opens, completions, attendance at the thing itself and replies from people you weren’t expecting to hear from. Where it can’t, put a person in the gap and ask one or two of the people who were meant to receive it what they got. A conversation works better than a survey for this, and it takes less time than writing the report.

Above all, make “I don’t know” something people are allowed to report. If the only two options are fine and broken, everything ambiguous gets rounded to fine, and ambiguous cases are exactly where this failure hides. A weekly summary that says “I couldn’t reach the source this week” tells you more than one that says nothing happened. The second reads better and never gets queried.

Checking the far end falls between everyone’s jobs, and the report is usually due before anyone has gone to look. So the fix has to go in the diary. Before the next report goes in, ring two of the people it says you reached.

Ben

Photograph by Kutan Ural.