A test suite is only worth the confidence it earns. Celilo’s end-to-end tests run against a faithful simulation of the real internet — public DNS, a recursive resolver, firewalls, NAT, and a real ACME certificate flow — precisely so they catch the bugs that only appear under real-world constraints. The Honest Tests effort is the ongoing work to make sure that suite can’t quietly pass a system that would break in production.
Why a suite can lie
The dangerous failure isn’t a red test — it’s a green one that shouldn’t be. A test that checks a service from the management box may see it working, while a plain device on the network can’t reach it at all. A test that sleeps for a few seconds may hide a race that bites a real deployment. These are the failures that erode trust, because they let a broken change ship wearing a passing badge.
What “honest” means
The goal is a suite that asserts from the same vantage a real user has — the least-privileged one — waits on the actual condition it cares about instead of a timer, and diagnoses itself when something times out rather than guessing. No shortcuts that make a test pass by weakening the simulation. When these tests are green, the system is genuinely working.