Skip to content

How we run things

The conventions behind the platform. They are worth writing down because they are the reason a service can be added, moved, or recovered without a project around it.

Everything is a file in a repository

There is no hand-built infrastructure. Every service is a container defined in a compose file, and every host follows the same layout: one directory per project, one file per service, a shared environment file, and a single top-level file that does nothing but list which services are active.

Two consequences that matter more than they sound:

  • Disabling a service is commenting out one line, not deleting anything. A service that was switched off a year ago can be switched back on, with its configuration intact.
  • The file on the host is the authority. Copies kept elsewhere drift, and drifted copies are worse than no copies, because they are trusted. Where a definition and its documentation can disagree, the documentation is treated as the thing that is wrong.

Services declare what they need

A container carries its own routing, dashboard, and management metadata as labels. Nothing external holds a registry that has to be kept in step, which removes an entire class of "the route exists but points at the wrong thing" problem.

The one thing this does not remove is the need to keep an address consistent in the two places it appears — the network definition and the routing metadata. That is a real trap, and it is documented internally as such.

Authentication is a default, not a decision

Every route picks a named middleware chain, and that choice is where authentication is decided. The chain that requires single sign-on is the default; the chain that does not is used only where a service is genuinely meant to be public or has trusted authentication of its own.

Making it a named, visible choice rather than a checkbox means "is this open to the internet?" is answerable by reading one line, which is the property that matters when there are a few dozen services.

Machine clients get a separate door

Applications that talk to each other cannot complete an interactive sign-in. Rather than weaken the human path, machine-to-machine traffic is matched by a distinct rule and routed separately, so the browser path keeps full single sign-on and automated clients are still explicitly authenticated.

Observability answers four separate questions

The tooling is chosen so each question has one obvious place to look:

Question Answered by
What did this service log? Centralised log aggregation, with live tailing for quick checks
Is the host healthy? Host and container metrics, agents reporting outbound to a central hub
Is it up, seen from outside? External uptime monitoring, checking through the proxy the way a user would
What is running, and where? A dashboard built from the same service metadata, plus container management

The external check earns its place: it is the only one that notices a broken certificate, an expired route, or an identity-provider outage, because it is the only one that exercises the whole path.

Backups assume the worst about storage

Snapshots are taken nightly from every host into a single deduplicated repository, and the repository is mirrored offsite. Databases are dumped properly before the snapshot runs rather than being captured as live files, because a copy of a running database is not a backup of it.

The part worth copying elsewhere is the staleness alerting. Every backup source is age-checked on every run, and anything that has not reported within its own threshold raises an alert through channels that interrupt a human. This exists because a backup schedule once stopped silently and went unnoticed for weeks — nothing had failed loudly, it had simply stopped happening.

The principle generalises: a backup system that only reports failures does not tell you when it has stopped running at all.

Documentation is deployed, not attached

Documentation is Markdown in a repository, rendered by a static site generator that watches its source directory and rebuilds on change. Publishing is a file copy.

There are two sites. This one is public and carries project documentation. A second, behind authentication, carries infrastructure documentation. The split exists because those two audiences want different things, and because a document describing infrastructure should not be one misconfiguration away from being public.

That decision is worth stating plainly, because the tempting alternative — one site, with sensitive pages left off it by convention — fails quietly the first time someone forgets.