Tahinli/forge
public0 watching
★ 0
| 1 | # Backups, phase 1 (contract between workers) |
| 2 | |
| 3 | The background is in `docs/architecture.md` §8.2–8.4. Phase 1 makes complete, verifiable backups **locally**: a directory that an external tool (restic, rclone, or borg with append-only keys) ships off-site. Shipping it, standby replication and push mirrors come later. |
| 4 | |
| 5 | The global rules apply: |
| 6 | - No unwrap in non-test code. |
| 7 | - Tests use `tempfile::TempDir`. |
| 8 | - Never put cargo target dirs or big files in /tmp. |
| 9 | |
| 10 | ## Backup set layout |
| 11 | ``` |
| 12 | <out>/forge-backup-<UTC yyyymmddThhmmssZ>/ |
| 13 | manifest.json # format version, forge version, created_at, instance base_url, files [{path, bytes, sha256}], repos [{id, owner, name, refs_sha256, bundle | null (empty repo)}] |
| 14 | db/forge.db # consistent snapshot |
| 15 | repos/<repo id>.bundle # `git bundle create --all`, then `git bundle verify` before the manifest is written |
| 16 | keys/forge.key # only with --include-key; otherwise the manifest records "key_included": false |
| 17 | ``` |
| 18 | - `refs_sha256` is the sha256 of sorted `git for-each-ref --format='%(objectname) %(refname)'` (the reconcile checksum from architecture §8.2). It's computed from the bundle-time ref list, so a restore can compare against it. |
| 19 | - The directory is written as `.partial-…` and renamed atomically when complete. A crash never leaves something that looks like a finished backup. |
| 20 | - Files are 0600 and directories 0700. |
| 21 | |
| 22 | ## DB snapshot |
| 23 | - Use `VACUUM INTO '<path>'` if Turso 0.8.x supports it; the worker must verify this with a test. |
| 24 | - **Fallback:** hold the store's connection mutex so there are no concurrent writers, run `PRAGMA wal_checkpoint(TRUNCATE)` (or the Turso equivalent), copy the DB file (and the WAL, if any is left), then release. |
| 25 | - Verify the result by opening it read-only with a fresh connection and running `PRAGMA integrity_check` (or `quick_check`), with the migrations version check. |
| 26 | |
| 27 | ## Commands (`forge-web admin …`) |
| 28 | - `backup --out DIR [--include-key] [--repos-only | --db-only]`: prints the path and a summary, and exits non-zero on any failure. |
| 29 | - `backup-verify PATH`: recomputes every sha256, runs `git bundle verify` on each bundle (in a scratch repo under PATH's parent, **not /tmp**), opens the DB snapshot read-only and runs `integrity_check`, and checks the manifest's repo list against the DB snapshot's `repos` table. |
| 30 | - `restore --from PATH --data-dir NEW_DIR`: refuses a non-empty NEW_DIR. Copies the DB, then for each repo runs `git clone --mirror <bundle>` into `repo_path(NEW_DIR, id)` (or `init --bare` + `fetch` from the bundle with `+refs/*:refs/*`) and removes the remote. Then writes the managed gitconfig and hooks the same way boot does, compares `refs_sha256` per repo, and writes `forge.key` if the set has one (otherwise warns loudly that the key must be restored by hand). |
| 31 | - `fsck [--full]`: `git fsck` (`--full` if asked) on every repo plus the DB `integrity_check`. Reports per repo, and exits non-zero on any problem. |
| 32 | |
| 33 | ## Scheduler (config `[backup]`) |
| 34 | ```toml |
| 35 | [backup] |
| 36 | dir = "" # empty = disabled |
| 37 | interval_hours = 24 # full backup set |
| 38 | keep = 7 # keep the newest N complete sets in dir (delete older complete sets only; never touch other files) |
| 39 | healthcheck_url = "" # optional; GET <url>/start, then <url> on success or <url>/fail with the error (healthchecks.io style) |
| 40 | fsck_interval_days = 7 |
| 41 | ``` |
| 42 | - A tokio task started at boot runs the same code as the CLI. Healthcheck pings use a plain reqwest client with a 10s timeout; the URL is admin-configured and trusted. |
| 43 | - `GET /admin/backups` (instance admins): the last 20 runs (time, duration, size, ok or error), backup and fsck statuses, and a "Run backup now" POST with CSRF. Runs are recorded in a table `backup_runs(id, kind 'backup'|'fsck', started_at, finished_at, ok, path, bytes, error)`, migration `0014_backup_runs.sql`. |
| 44 | |
| 45 | ## Docs |
| 46 | `docs/backup.md` covers: |
| 47 | - what is in a set and what isn't (the key); |
| 48 | - how to ship it off-site with restic append-only or rclone, plus a sample systemd timer; |
| 49 | - how to restore, step by step; |
| 50 | - a quarterly restore drill; |
| 51 | - the fencing reminder from architecture §8.5. |