All notes
OPERATIONS

What a useful health check actually proves

A running process, an open port, and a successful request answer different questions.

A green status light is useful only when its meaning is clear. A process can be running while its dependency is unavailable. A port can accept connections while every request fails. A website can return a page while the task a visitor came to perform remains broken.

Use a small ladder of checks

Start with process health: has the service exited or entered a restart loop? Add a listener check: is the intended address and port available? Then test the application with a normal request and a specific expected result.

For HTTPS, include name resolution and certificate validation. A request that disables certificate verification removes one of the most important parts of the test. Use the same hostname that real clients use.

Test from outside

A local request is excellent for diagnosing an application, but it bypasses routing, firewall rules, and sometimes the public TLS endpoint. Pair it with a low-frequency external check from another network. Avoid interpreting a single failure as a definitive diagnosis.

Choose actionable alerts

Alert on a sustained failure, unexpected restarts, low disk space, or an approaching certificate expiry. Put the service name, observation time, and failing layer in the message. A useful alert should suggest the first thing to inspect without exposing secrets.

Keep monitoring independent enough to survive the failure it is meant to report. A dashboard hosted on the same machine as the only monitored service cannot report a complete loss of that machine.

Measure recovery too

Record when service returns and how long the interruption lasted. After a change, verify both successful requests and expected rejection of invalid requests. Reliability includes refusing the wrong work as well as completing the right work.