Blog · Monitoring
Monitoring what you cannot see
logs, metrics and probes
A card that only shows "online" or "offline" lies half the time. Here is how we measure, and what remains out of reach.
Four states, not two
A card that only shows "online" and "offline" lies half the time. We use four:
| State | Meaning |
|---|---|
| Online | The service responds correctly |
| Degraded | It responds, but with a server error |
| Offline | No response |
| Filtered | Reachable only from a restricted network, intentional state, not an outage |
The fourth state is the one that matters most: showing "offline" on a service intentionally closed to the Internet is announcing an outage that does not exist.
What we probe, and how
Three families of probes, run server-side and cached for one minute: an HTTPS request with certificate validation, a hand-built binary DNS query for name servers, and a simple port opening for services without a web interface. A web page cannot do any of this from a browser: the server measures, the browser displays.
From logs to metrics
Devices and servers send their logs to a central collector, which makes them queryable and extracts time series: volume by source, errors by type, availability by service. These series feed dashboards where you can see a drift before it becomes an incident.
What monitoring cannot replace
- A user who does not dare complain a slow service stays "online".
- The last mile the probe sees the server, not the client's WiFi.
- The root cause knowing a service is down does not say why.
Monitoring does not do the work for us. It makes sure we know first, and that is already a lot.
A similar project? We answer technical questions in a meeting, with the same figures as in these articles.
The matching service: hosting, backup and the local data center.