Last week I was in the code of a product I built, fixing one specific thing.
Not a crash. I was fixing a place where the software could not tell the difference between "no" and "I could not check".
Here is what that means away from the keyboard. Someone schedules a visit. The software is then supposed to create a matching job card in the field service system on the other side. If that connection was never finished being set up, or the credentials for it are gone, no job card gets created. That much is unavoidable and it is nobody's fault.
The question is what the software does next. What mine did was hand back a result that looked exactly like "this customer's plan does not include visits, so there was never a job card to make". An ordinary, legitimate, nothing-to-see-here answer. Meanwhile the record inside my own product said the visit was scheduled, which was completely true. The evidence that it had worked was sitting right there, correct, and about half the story.
Then I found the part I was not looking for.
The same vagueness ran in the other direction. One of the two places that called this code had a single explanatory sentence written for it, and that sentence named the field service connection as the problem. So a customer on a plan that genuinely has no visits, behaving exactly as designed, would be reported to the owner as a broken connection.
The same weakness hid a real fault and invented a fake one. A vague failure signal does not simply lose information. It makes things up, and then you spend a morning fixing something that was never broken.
Once I could see the shape, I found three more without going hunting, just from being in the neighbourhood. Three more of the same thing is what a habit looks like from the inside. The one I went in for is fixed and shipped. The others are on my list.
Why my code and not yours.
I can read mine. You cannot read the automation somebody set up for you, and you certainly cannot read the inside of the software you pay for every month. What you get instead is the part that looks right. The invoice went out. The job appeared on the board. The confirmation says scheduled. All of that can be perfectly true and still be half of what was supposed to happen.
A person who cannot do their job tells you. They call, they complain, the work piles up somewhere you can see it. Software that cannot do its job hands back something tidy and goes back to sleep. That is the trade nobody mentions when you automate a thing, and it is worth making anyway. You just have to make it with your eyes open.
One more thing worth knowing, and I have been on the wrong end of this one myself. I have shipped something where the version I tested and the version that actually ran were not the same piece of machinery. When an automation gets demonstrated, it usually gets triggered by hand. Somebody clicks the button, the thing happens, everyone in the room agrees it works. But the version a person clicks and the version that runs on its own at two in the morning are two different things, and the second one is what you are buying. Watching the first one work tells you very little about the second.
So, two questions. Both are free, and you can ask them of a vendor, of whoever built the thing, or of yourself.
The first: what does it do when it cannot reach the other side? Not what happens when it fails. Failure is the easy case, because a thing that crashes tells you it crashed. Ask what it does when it cannot even get far enough to ask the question. If the answer is that it carries on and returns something reasonable, you have just been shown exactly where your next unpleasant surprise is coming from.
The second: can it send me a number? Once a day or once a week, to a person, at a predictable time. Fourteen job cards created. Six invoices sent. Nothing it could not attempt.
That last number is the one that matters. A count of what worked tells you nothing about what was never attempted. Both belong in the same message. On the morning something goes quiet, that second number is the only one that moves.
You will not read them, and that is fine. The value is the expectation. A message that arrives every morning at nine is a thing whose absence you notice. A job card that never got created has no arrival time to miss.
Most of a proper audit is exactly this. Which of your automations can answer that first question, and which ones nobody has opened since the week they were switched on: https://www.chanautomation.com/ai-assessment
So here is the one to sit with this week. Pick the automation you would be most annoyed to lose. The one that sends the thing, or moves the thing, or tells the customer the thing.
If it stopped tonight, what would tell you, and how long would that take?
If the honest answer is that a customer would eventually complain, then you do not have an automation. You have a dependency nobody is watching.
Kevin Chan
The Ops Shortcut by ChanAutomation
https://www.chanautomation.com/ai-assessment