Running SecretHive in production

What an installation needs before it holds anything real, in the order we closed it ourselves.

The image

One image for linux/amd64 and linux/arm64. It runs as a non-root user, works under an arbitrary user id in group 0, has no shell and no package manager, and writes only under /data.

The two secret files

Both in a password manager before anything else, and both readable only by the container's user on the host. Use the bootstrap token to set things up and for recovery; day to day, people sign in and scripts use keys.

The token does not have to stay. Once a super admin signs in with a password of their own choosing or through a sign-in provider, take admin.token away and restart: the server runs without it and no token opens anything (Admin → Tokens says "not in use"). Without the file and without such an administrator the server refuses to start, so there is never an installation nobody can get into. To get back in later, put a token file in place and restart. While it is there, the token is held to the same pace as a key.

Keep the master key and the backups apart. A backup opens with the master key; whoever holds both holds everything. If a host backup tool snapshots the folder with the database and its backups, exclude the folder with the master key from it, and only after the key is in the password manager. We had this wrong for a day and answered it by replacing the master key.

Settings

Environment variables; __ separates levels.

Behind a reverse proxy

The server speaks HTTPS itself, so the proxy re-encrypts to it. Have the proxy trust exactly the server's certificate for that hop rather than switching verification off; the certificate the server makes for itself is stable across restarts, upgrades and a master key rotation, as long as Tls__HostNames is set. It is valid for 825 days and is made anew in its last 30, or when the names change: the proxy then needs the new one.

Do not put the proxy's own single sign-on in front of it: the API is used by hosts and jobs that carry a key or a token, not a browser session. SecretHive does its own sign-in.

Set Proxy__Trusted to the address the proxy's requests arrive from. That is what makes the audit trail and the key and vault network limits see the visitor instead of the proxy. Check it on a vault's Allowed networks card, which shows the address your own request comes from. (The pace of a key is counted per key, not per address.)

Proxy__ClientAddressHeader hands the decision to that header. With a CDN in front of the proxy, the last hop in X-Forwarded-For is the CDN, and the CDN names the visitor in a header of its own. When SecretHive is told to read it, the address in that header is what the audit trail records and what the allowed networks of a vault or a key are checked against. Whoever can reach the proxy around the CDN, or whose own copy of the header the proxy passes on, chooses that address. Measured: a request from the trusted proxy carrying X-Real-IP: 10.9.9.9 read a vault limited to 10.9.9.9. Set the header only when the proxy overwrites it on every request and nothing reaches the proxy but the CDN; otherwise leave it off and limit networks on the proxy's own addresses.

Database

SQLite in /data is what it is designed around. SQL Server works the same; the storage and restore tests run against both, every build starts the image against both, and a backup made on one restores into the other.

Backup and restore

The server backs itself up: every 24 hours it writes one file with every row, verifies it by reading it back, and keeps the last 14. A backup can also be made on Admin → Backups or with POST /api/v1/backups.

• A backup holds what the database holds: values stay encrypted as they are, and the master key is not in it.
• It is sealed with a key derived from the master key, so a file that was changed, or made for another installation, is refused.
• It is independent of the database product.

Restore is the server's own command, into an empty database only:

docker run --rm -v ./secrets:/run/secrets:ro -v ./data:/data -v ./backup:/backup:ro <image> restore /backup/<file>.shbak

It checks the seal first, writes everything in one transaction, and exits 2 with nothing written when it refuses. Then start the server normally. This runs with the shipped image on every build, including a restore from SQLite into SQL Server.

Copy the backups off the host with whatever backs up the host. What is lost with the host is then at most what changed since the last backup, and the next two sections are about noticing and narrowing that.

Health, and an alert on the backup's age

GET /health on port 8080 needs no credential and holds no secret:

{"database":true,"masterKey":"09e8d4394f664cdc","backup":{"lastVerifiedUtc":"2026-10-10T20:25:44Z","ageHours":0.4},"securityEvents24h":5}

Alert on backup.ageHours over a day and a bit, and on the number being absent: a server that is down, a host that is off and a monitoring agent that died all look like "no data", and that is exactly when nobody is told otherwise. Have the alert evaluated somewhere other than the host itself.

masterKey is the id of the key in use, not the key. securityEvents24h is worth a glance when it moves.

The audit trail off the host

Every audit row is also one JSON line on the container's log, category SecretHive.Audit, written with the row (rows a restore brings back are history and are not logged again). Ship that log to wherever your logs go, or point OTEL_EXPORTER_OTLP_ENDPOINT at a collector. Then the rows written since the last backup survive the host. The server writes several ordinary lines per request; shipping only the audit category and warnings keeps the volume small. A line holds names and never a value.

Replacing the master key

Every version of every secret has a key of its own, and only that key is encrypted under the master key. A rotation rewrites one small record per version and reads no value.

1. Make the new key, and put it in the password manager before the next step.
2. Start the server with the new key as SecretHive__MasterKeyFile and the old one named in SecretHive__RetiredMasterKeyFiles. At that start everything moves across: the versions, a sign-in provider's client secret, the certificate the server made for itself, the session keys. The audit trail gets keyring.rotated with both key ids.
3. Make a backup. Backups from before the rotation open only with the old key.
4. Take the old key out of the configuration and restart. Keep it for as long as you keep backups made under it.

People stay signed in and the server keeps its certificate. A server started with a key the database is not under does not start, changes no secret, and says which key it expects. There is no page for this, on purpose: an administrator's browser session should not be able to re-key the vault.

What every answer carries

X-Content-Type-Options: nosniff, X-Frame-Options: DENY and Referrer-Policy: no-referrer on every answer, and Strict-Transport-Security for a year over HTTPS. The session cookie is HttpOnly, strictly same-site and, with TLS on, only ever sent over HTTPS. A page that is open stops working when its account is disabled or changed.

From 0.27, a Content-Security-Policy as well: scripts, styles, images, fonts and connections from the server itself only, no inline script, nothing may frame a page (frame-ancestors 'none'), and a form may post only to the server and to the enabled sign-in providers. A provider added on the providers page is in the policy within 30 seconds. If your proxy sets its own headers, let this one through: a proxy that rewrites X-Frame-Options to ALLOW-FROM leaves framing open in today's browsers, which ignore that value; frame-ancestors still holds.

Upgrading

Replace the image. The database is brought up to date at start. Copy the database file first; a backup from just before is the way back.

What the server contacts

Nothing on its own behalf: no telemetry, no update check, no licence call. See What to open in a firewall.