Supervision · Method
When monitoring lies
the probe said offline, the SIP server answered
Our public status page showed a service as unavailable while it was running. An investigation into a false outage, and into what a status page should actually display.
The facts
The monitoring card on our Telephony page showed "probe unreachable" while the call server was answering normally. A visitor, a client, could conclude that telephony was down. That is the worst possible defect for a status page: it does not fail in private, it fails in front of people.
Three causes, combined
- The target tested was not the service: the probe was querying the web interface of the call server, not its SIP signalling.
- The cache was cold: ten targets checked in sequence, up to a second and a half each, before the first response.
- The browser-side timeout was too short: the display gave up before the check finished and wrote "unreachable".
Three flaws that, taken separately, are harmless. Together, they produce a credible false outage.
What we fixed
The probe now checks the signalling and the certificate, the response is cached for sixty seconds, and the display waits for the check to complete before concluding. Above all, the page says what it measures and what it does not: an HTTPS check does not prove that a call goes through.
The rules we apply now
- Test what the client uses, not what is easy to test.
- Publish the time of the last check, so the reader knows whether the information is fresh.
- Distinguish "down" from "I don't know": they are two different messages, and the second has no reason to look like the first.
Why it matters beyond the page
The same reasoning applies to internal monitoring. An alert that cries wolf teaches teams to stop reading it, and monitoring that no one reads protects no one. An honest probe says three things: what it measures, when it measured it, and what it ignores.
The matching service: live service status