The nightly roll call.
Every night, we reinstall every tool from scratch on a brand-new machine, with test accounts and nobody at the wheel. Each tool gets five checks:
| check | what we test | why it matters |
|---|---|---|
| installs | We download it from the company that makes it and check it hasn't been tampered with. | If this fails, nobody can set the tool up. |
| logs in | It works with just a key: no browser window, no "click here to log in". | Your AI can't click a button on a website, so a login screen stops it cold. |
| readable | It can answer in a format an AI reads reliably (JSON), not just text meant for people. | Without it, your AI has to guess what the tool meant. |
| no prompts | It never stops to ask "are you sure?" and wait. | Nobody's there to answer, so your AI would sit stuck. |
| help | Its built-in help works. | That's how your AI looks up the right command instead of making one up. |
A tool that didn't run shows as a dash, never as a pass. When a new version fails, it gets its own page saying what broke and which version to use instead. See last night →