forge
ExploreLog in

Tahinli/forge

public
0 watching
★ 0
32.8 KiB

Forge Architecture

Status: milestones 1 and 2 are implemented. Later milestones are outlined here and not built yet. "Forge" is a placeholder name. Renaming is a find-and-replace.

Sources: the M1 plan (~/.claude/plans/purring-prancing-wirth.md), the interface contract (docs/m1-interfaces.md), and the research notes (docs/research.md, sections 1-16). Items marked (unverified) in the research are not confirmed.

1. Goals and principles

  • Independence. Forge is a standalone app. It has no dependency on other projects, no shared code, and no hosted forge in the loop. The owner was banned from GitHub and lost five years of work, so nothing critical may depend on a third party's account decisions.
  • Data safety first. Metadata and repositories both have to survive disk loss, a compromised server, and a provider account termination. Section 8 covers this.
  • Single writer. One primary instance accepts writes. Replicas and backups are copies. Failover is manual and fenced (section 8.5). There is no consensus layer.
  • The DB holds metadata. Users, sessions, keys, tokens, repos, and later issues, PRs, labels, and milestones live in Turso. Issues and PRs are not stored in git.
  • Git is the source of truth for code. Objects and refs live in bare repositories on disk. The DB never holds file contents or ref values that git can answer.
  • Latest dependencies. Every crate is added with cargo add at its latest release and recorded in [workspace.dependencies]. Versions are not copied from other projects.
  • Security by default. Deny unless allowed. One authorization function. No shell. Private repos return 404. Secrets are hashed or sealed at rest. Section 9 has the checklist.

2. Stack and why

Layer Choice Why
Web and HTTP Topcoat 0.10 on hyper Server-rendered UI and git smart-HTTP in the same app. Topcoat routes accept streaming bodies with no size limit and no default request timeout, which git needs.
Git smart-HTTP Routes in Topcoat, backed by the git binary No axum. The git child process streams request and response bodies directly.
Metadata DB Turso Embedded SQLite-compatible store. Holds all metadata. WAL with synchronous=FULL (section 8.3).
Git serving git binary (upload-pack, receive-pack, merges, blame, verify, maintenance) Git's own implementation is the only trustworthy server side.
Git reads gix in-process Object, tree, and blob reads, rev-walk, diffs, rename detection, archives (research §7).
SSH russh 0.64.x Built-in SSH server for git over SSH. No OpenSSH dependency.
Hooks forge hook subcommand over a unix socket Hook policy lives in the app, not in files users can edit.

Not used: git2 (adds a C dependency, research §7). Dumb HTTP (disabled, research §1).

3. Crates and responsibilities

Cargo workspace at the repo root. Edition 2024, resolver 3.

project_git/
  Cargo.toml  .gitignore  config/forge.toml  deploy/forge.service  style/main.scss
  docs/{research.md, architecture.md, m1-interfaces.md}
  crates/
    forge-core/   domain types, store, migrations, config, secret key, authz, names, passwords, tokens, sessions
    forge-git/    safe git command builder, process spawn and streaming, pkt-line, repo init, managed gitconfig, version check
    forge-ssh/    russh server: pubkey auth, exec allow-list and parse, GIT_PROTOCOL env, piping, timeouts
    forge-web/    crate, builds the `forge` binary: Topcoat UI, auth, smart-HTTP routes, internal hook API, `hook` subcommand, admin CLI

Module ownership (from the contract):

  • forge-core
    • error.rs, time.rs (now_stamp(), stamp_after()), model.rs (domain types)
    • config.rs (Config::load), secret.rs (load_or_create_key, seal, open)
    • names.rs (normalize_name, validate_name, RESERVED), password.rs (argon2id)
    • token.rs (generate_token, hash_token, constant_time_eq), session.rs
    • authz.rs (Action, Denied, authorize)
    • store/mod.rs (Store), store/migrate.rs (migration runner)
    • migrations/0001_init.sql
  • forge-git
    • command.rs (GitCommand), version.rs, pktline.rs, repo.rs (repo_path, init_bare, list_branches)
    • setup.rs (write_managed_gitconfig, write_hook_shims), service.rs (GitService, service_command)
  • forge-ssh
    • backend.rs (GitBackend trait), parse.rs (parse_exec), server.rs (run, host key load or create, russh Handler)
    • The crate does not know about the DB. It calls GitBackend for auth and for preparing the git process.
  • forge-web
    • main.rs (arg dispatch: serve default, hook <name>, admin create-user), boot.rs, state.rs
    • auth.rs (session cookie, current_user(), CSRF check, Basic-auth parse, token auth)
    • git_http.rs, internal.rs (hook API on unix socket), hook_cmd.rs, ssh_backend.rs (impl GitBackend), admin_cli.rs
    • views/mod.rs, pages/{setup, login, dashboard, repo, new_repo, settings_keys, settings_tokens, settings_sessions, admin_users}.rs

Domain types (forge-core::model): Id = String (ULID), Visibility {Public, Private}, Role {Read, Triage, Write, Maintain, Admin}, plus User, Repo, SshKey, AccessToken, Session.

Store API (Store: async, Clone, wraps the Turso connection). open() sets WAL, synchronous=FULL, foreign_keys=ON, and a busy timeout, then runs migrations. It exposes user, session, SSH key, token, repo, collaborator, and push-event methods (the full list is in the contract).

4. Data model

4.1 Milestone 1 tables (0001_init.sql)

Conventions: ids are ULID text. Timestamps are UTC RFC 3339 with 9 fractional digits and a Z suffix. Booleans are integers 0/1. Foreign keys are enforced.

schema_migrations

  • version (PK). Records applied migrations.

users

  • id TEXT PK (ULID)
  • username TEXT NOT NULL (display form)
  • username_folded TEXT NOT NULL UNIQUE (NFKC + case-fold)
  • email TEXT NULL
  • password_hash TEXT NOT NULL (argon2id)
  • is_admin INTEGER NOT NULL DEFAULT 0
  • created_at TEXT NOT NULL

sessions

  • id TEXT PK
  • user_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE
  • token_hash TEXT NOT NULL UNIQUE (sha256 of the random session token; the raw token is only in the cookie)
  • csrf_token TEXT NOT NULL
  • created_at, expires_at, last_seen TEXT NOT NULL
  • user_agent TEXT NULL
  • Queries return only unexpired rows.

ssh_keys

  • id TEXT PK
  • user_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE
  • title TEXT NOT NULL
  • algorithm TEXT NOT NULL
  • key_blob BLOB NOT NULL UNIQUE (raw SSH wire-format public key; exact-match lookup)
  • fingerprint TEXT NOT NULL
  • created_at TEXT NOT NULL, last_used_at TEXT NULL

access_tokens

  • id TEXT PK
  • user_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE
  • name TEXT NOT NULL
  • token_hash TEXT NOT NULL UNIQUE (hex sha256; the token is shown once)
  • scopes TEXT NOT NULL (serialized list)
  • expires_at TEXT NULL (default expiry is 30 days, research §10)
  • created_at TEXT NOT NULL, last_used_at TEXT NULL

repos

  • id TEXT PK (ULID; determines the on-disk path)
  • owner_id TEXT NOT NULL REFERENCES users(id)
  • name TEXT NOT NULL (display form)
  • name_folded TEXT NOT NULL, with UNIQUE(owner_id, name_folded)
  • description TEXT NOT NULL DEFAULT ''
  • visibility TEXT NOT NULL CHECK (public or private)
  • default_branch TEXT NOT NULL
  • archived INTEGER NOT NULL DEFAULT 0
  • created_at TEXT NOT NULL

repo_redirects

  • Old owner and name mapped to a repo id, so old web and git URLs keep working after a rename. Exact columns are set in 0001_init.sql; the contract does not fix them.

collaborators

  • repo_id TEXT NOT NULL REFERENCES repos(id) ON DELETE CASCADE
  • user_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE
  • role TEXT NOT NULL CHECK (read, triage, write, maintain, admin)
  • PK (repo_id, user_id)

push_events

  • id TEXT PK
  • repo_id, actor_id TEXT NOT NULL (foreign keys)
  • ref_name, old_oid, new_oid TEXT NOT NULL
  • protocol TEXT NOT NULL CHECK (http or ssh)
  • created_at TEXT NOT NULL

Secrets (forge.key is not in the DB). Values sealed with XChaCha20-Poly1305 are stored as v1: plus base64(nonce || ciphertext). M1 has no sealed column yet. Mirror credentials and TOTP secrets are the first candidates (M5 and later).

4.2 Later tables (outline)

Table Milestone Purpose
issues M3 Number per repo, title, body, state, author, timestamps
comments M3 Issue and PR comments, polymorphic by parent, edit history
labels M3 Per-repo labels, plus an issue_labels join
milestones M3 Per-repo milestones, due date, state
pulls M4 PR number, base and head refs, head oid, merge state, outdated flag
reviews M4 Reviewer, state (approve, request changes, comment), commit oid
review_comments M4 Commit oid, path, line, side, diff hunk, outdated flag (kept, never deleted)
webhooks M5 Target URL, events, encrypted secret (sealed)
deliveries M5 Delivery id, attempt, status, request and response, retry state
mirrors M5 Pull and push mirrors, interval, backoff, sealed credentials
commit_statuses M6 Commit oid, context, state, target URL, creator
replication_state M5 Per-repo, per-replica last ref checksum, last success and failure

5. Request flows

5.1 Web page with session and CSRF

  1. The browser sends a request with cookie __Host-forge_session (Secure, HttpOnly, SameSite=Lax, Path=/).
  2. auth::current_user() hashes the cookie value and calls session_by_token_hash. No row, or an expired row, means anonymous.
  3. For a valid session, touch_session updates last_seen. The user is loaded with user_by_id.
  4. The page handler builds its view. Repo pages call authorize(actor, repo, role, Action::Read) first (section 6).
  5. A POST form includes the session's csrf_token as a hidden field. The handler compares it in constant time. A missing or wrong token is rejected before any side effect.
  6. On success, the handler performs the write through Store and redirects (PRG).
  7. First run: while user_count() == 0, /setup creates the first admin. After that the route is closed.

5.2 Smart-HTTP fetch and push

  1. Client sends GET /{owner}/{repo}/info/refs?service=git-upload-pack (or git-receive-pack). A trailing .git on the repo segment is stripped.
  2. Auth: for upload-pack on a public repo, anonymous is allowed. Otherwise the request needs Authorization: Basic. The username is ignored. The password is a forge_ token. The token is hashed and checked with token_by_hash. Failure gives 401 with WWW-Authenticate: Basic.
  3. authorize() runs for Read (fetch) or Write (push). A private repo the caller cannot see gives 404.
  4. The handler spawns git <svc> --stateless-rpc --advertise-refs through service_command, with GIT_PROTOCOL set from the Git-Protocol header. It writes the pkt-line header # service=git-<svc>, then 0000, then streams git's stdout. Response has Cache-Control: no-cache.
  5. Client sends POST /{owner}/{repo}/git-upload-pack or /git-receive-pack. If Content-Encoding: gzip, the body is inflated first.
  6. Request body streams to git's stdin and git's stdout streams back. The route has no timeout and no body-size middleware. Nothing buffers a whole push.
  7. For push, receive-pack runs the pre-receive and post-receive hooks (section 5.4). Objects land in quarantine until pre-receive accepts them.
  8. Protocol v2 clients go straight to POST after the GET.

5.3 SSH fetch and push

  1. Client connects to ssh_listen (default 0.0.0.0:2222). russh runs the handshake under auth_timeout and inactivity_timeout.
  2. auth_publickey: GitBackend::authenticate_key(algorithm, key_blob) does an exact byte match on ssh_keys.key_blob. ssh-rsa with SHA-1 is disabled. On success, last_used_at is touched.
  3. Client sends an env request. Only GIT_PROTOCOL is accepted. Other names are ignored or refused.
  4. Client sends exec. parse_exec accepts only git-upload-pack or git-receive-pack with a single-quoted path. Quotes and newlines inside the path are rejected. No shell is used.
  5. GitBackend::prepare(user_id, service, repo_path, git_protocol) resolves the repo, runs authorize(), and builds the git Command through forge-git. On Err(msg), the message goes to stderr and the exit status is 1.
  6. The child's stdin and stdout are piped to the SSH channel. For receive-pack, FORGE_* env vars are set (section 5.4).
  7. When git exits, the server sends exit_status and closes the channel.

5.4 Pre-receive and post-receive hooks over the unix socket

Setup at boot: write_hook_shims() writes <data_dir>/hooks/pre-receive and post-receive (mode 0755) as shims that run exec <forge> hook <name>. core.hooksPath in the managed gitconfig points there. Users cannot write hooks.

Push:

  1. The forge process spawns receive-pack with FORGE_INTERNAL_SOCKET, FORGE_INTERNAL_TOKEN (random per boot), FORGE_ACTOR_ID, FORGE_REPO_ID, and FORGE_PROTOCOL (http or ssh).
  2. Git runs pre-receive with <old> <new> <ref> lines on stdin. Git has placed the new objects in quarantine (GIT_QUARANTINE_PATH, GIT_OBJECT_DIRECTORY, GIT_ALTERNATE_OBJECT_DIRECTORIES). They are invisible to other processes until the hook accepts.
  3. The hook pre-receive client reads stdin and POSTs JSON to /hook/pre-receive on <data_dir>/internal.sock. Body: {repo_id, actor_id, protocol, updates:[{old,new,ref}], quarantine_env:{...}}. Header: X-Forge-Internal-Token.
  4. The internal router checks the token in constant time. It then applies policy:
    • reject pushes to archived repos
    • reject deleting the default branch
    • reject force-pushes to the default branch, using rev-list --max-count=1 <old> ^<new> with the quarantine_env vars set on the command. Any output means a force push.
    • reject ref names with a leading - or containing ..
  5. The response is {allow, message}. The hook prints message to stderr and exits 0 if allowed, 1 if not. A non-zero exit rejects the whole push.
  6. On accept, git moves the objects out of quarantine and updates refs. Git updates refs in one transaction.
  7. Post-receive: hook post-receive POSTs the same updates to /hook/post-receive. The handler calls record_push_event for each ref. Replication (M5), webhooks (M5), PR ref updates (M4), and cache invalidation (M2) attach here. Post-receive cannot reject.
  8. Hook failures are fail-closed for pre-receive. If the socket is unreachable, the push is rejected.

5.5 User-content domain for raw files (M2)

Raw and user-uploaded files are served from user_content_base_url (for example raw.<domain>), a separate origin from the app.

  1. Request arrives on the user-content host. The path is parsed as owner, repo, ref, and file path.
  2. Repo lookup, then authorize(actor, repo, role, Action::Read). Anonymous is allowed for public repos. A private repo the caller cannot see gives 404.
  3. Every path component is checked for symlinks before reading, not only the last one (research §4).
  4. The blob is read through gix and streamed. HTML and SVG are served with Content-Disposition: attachment.
  5. Every response carries X-Content-Type-Options: nosniff and a strict CSP.
  6. Cookies from the app origin are never sent to this origin. Session auth is not accepted on this host.

5.6 Response compression

Topcoat's builder-wide compression is off on the public router, so it cannot compress git responses by mistake. crates/forge-web/src/compression.rs (ResponseCompression) runs instead. It gzips or brotlis a response only when all of these hold:

  • the content type is text (HTML, CSS, plain text, JSON, SVG, JavaScript);
  • the body size is known and between 32 bytes and 4 MiB (streamed bodies are never compressed);
  • there is no Content-Encoding or Content-Range header;
  • the request path is not a git smart-HTTP, LFS, raw, archive or download route (/info/refs, /git-upload-pack, /git-receive-pack, /info/lfs, and raw, archive, download segments).

Content types outside the list (application/x-git-*, application/octet-stream, archives, images) are never compressed. The raw user-content listener has compression off because its ETag and Content-Length must match the stored bytes.

6. Authorization model

One function: forge_core::authz::authorize(actor: Option<&User>, repo: &Repo, collaborator_role: Option<Role>, action: Action) -> Result<(), Denied>.

  • Action: Read, Write, Admin.
  • Denied: NotFound or Forbidden.
  • Callers: web pages, smart-HTTP, SSH prepare, the user-content domain, and hooks. Nothing else decides access.

Rules (in order):

  1. The owner or an instance admin has all actions.
  2. A public repo allows Read to anyone, including anonymous.
  3. A private repo with no role for the actor returns NotFound.
  4. Role Read or Triage allows Read.
  5. Role Write or Maintain allows Read and Write.
  6. Role Admin allows all actions.
  7. The actor can see the repo but lacks the right: Forbidden.
  8. Archived repos deny Write with Forbidden.

Status mapping: NotFound becomes 404 everywhere, including git and SSH. Forbidden becomes 403 on web and after authentication. For git over HTTP, an unauthenticated request that needs auth gets 401 with the Basic challenge. A private repo is never revealed by a 403. Any future go-get-style endpoints follow the same rule.

Roles: Read, Triage, Write, Maintain, Admin (per repo, from collaborators), plus instance admin. Org teams come later (research §10).

7. Git layer

7.1 Command builder (forge-git::GitCommand)

Rules:

  • No shell. Ever. tokio::process::Command only.
  • env_clear(), then PATH=/usr/bin:/bin, HOME=<data_dir>, GIT_CONFIG_NOSYSTEM=1, GIT_CONFIG_GLOBAL=<data_dir>/gitconfig.
  • arg() takes only &'static str flags, which are trusted.
  • git_dir() places --git-dir=<path> before the subcommand.
  • user_revs() appends --end-of-options, then the revisions. It rejects any revision starting with -.
  • paths() appends -- then the paths.
  • env() takes only &'static str keys.
  • User input reaches git only through GitCommand.
  • Repo-local config is not trusted (research §4). Submodules are never recursed.

7.2 Managed gitconfig (<data_dir>/gitconfig)

Written at boot by write_managed_gitconfig(). Keys:

  • core.hooksPath=<data_dir>/hooks
  • receive.fsckObjects=true, transfer.fsckObjects=true
  • core.fsync and core.fsyncMethod set explicitly. The default committed,-loose-object risks losing recent work (research §16).
  • uploadpack.allowFilter=true, uploadpack.allowRefInWant=true
  • receive.advertisePushOptions=true
  • protocol.file.allow=user (research §4)
  • Later: uploadpack.hideRefs and transfer.hideRefs for internal refs (M4 refs/pull/* as needed)

7.3 Repo path layout

<data_dir>/repos/<id[0..2]>/<id[2..4]>/<id>.git, where <id> is the lowercased ULID. Created with git init --bare through the builder. Renames are DB-only, because the path depends on the id, not the name.

7.4 Maintenance and caches (M2 and later)

  • git maintenance with incremental repack and loose-objects, or git repack --geometric=2 --write-midx.
  • Commit-graph with changed-path Bloom filters (makes git log -- path fast).
  • Bitmaps for clone speed.
  • A pool of long-running git cat-file --batch-command processes, one per repo, for hot reads (research §7).
  • "Last commit per file" cache keyed by (commit, path) for tree views (research §5).
  • Archive cache, with no promise of stable checksums across git versions.
  • Forks: full copies first. Alternates or object pools are deferred, and pools must never be pruned (research §5).
  • Ref updates use git update-ref --stdin for atomicity.

7.5 Minimum git version

  • Boot calls require_min_version(env, (2,38,0)). Boot fails if git is older.
  • 2.38 is required for merge-tree --write-tree (M4).
  • In practice, track the latest git release for security fixes. Keep it patched (CVE-2024-32002 and CVE-2025-48384 are submodule hook-execution bugs, research §4).

8. Data safety and reliability

8.1 Topology

  • Primary (writes, serves users).
  • Standby at a different provider. It receives git mirrors, DB snapshots, and bundles.
  • Immutable backups at a third location and provider.
  • Provider diversity covers provider, payment method, contact email, and jurisdiction. Research §11 notes that account termination can happen without warning. Lesson from SourceHut (2024): the standby was under 30s behind, but backups were up to 12h old with no off-site copy. Lesson from OVH Strasbourg (2021): backups at the same site burned with the primary.

8.2 Git replication (single writer)

  1. After post-receive, a replication job takes a per-repo lock and runs git push --mirror to each replica over SSH. Failures are recorded and retried with backoff. Push mirrors run after each push (M5).
  2. Hourly reconcile: compare a ref checksum (sha256 of sorted git for-each-ref --format='%(objectname) %(refname)') between primary and replica. On mismatch, re-mirror and run git fsck.
  3. Daily git bundle create --all per repo to off-provider storage, then git bundle verify.
  4. Weekly git fsck --full on replicas from cron. git maintenance has no fsck task.
  5. Never rsync or cp a live repo or a live DB.

8.3 Metadata DB (Turso)

  • Turso supports only synchronous OFF or FULL. Use WAL with FULL.
  • Litestream is not verified on Turso files, and Turso says mixed SQLite and Turso access across processes is unsupported. Spike before relying on it (open question 2).
  • Turso Sync and CDC are documented only for Turso Cloud. Do not depend on them for self-hosting.
  • Verified path: VACUUM INTO snapshots every 5-15 minutes. Ship each snapshot to the standby and to backups. Keep a JSON or F3 export as a second format (F3: friendly forge format, JSON plus attachments, research §9).
  • Integrity: PRAGMA quick_check daily. integrity_check plus foreign_key_check weekly. Alert on any result other than ok.

8.4 Integrity, backups, and drills

  • Backups follow 3-2-1-1-0: 3 copies, 2 media, 1 offsite, 1 immutable, 0 verify errors.
  • Immutable storage: Backblaze B2 Object Lock in compliance mode, Cloudflare R2 bucket locks, or Wasabi compliance mode.
  • The server holds no delete credentials. Either a separate machine pulls the backups, or the server pushes with append-only keys (rest-server --append-only, or BorgBase append-only).
  • Tools: restic (mature, check --read-data-subset), rustic (Rust, restic-compatible, newer), borg 1.4 (stable).
  • Keys: add a second restic key and store keys offline. Lost key means lost data.
  • Instance key forge.key (encrypts sealed values) is backed up separately and escrowed offline. Without it, sealed secrets are gone.
  • Restore drill monthly: automated restore to a fresh VPS. Time it. That time is the real RTO.
  • Dead-man switch: healthchecks.io pings with /start and /fail for each backup and integrity job. A missed ping alerts.
  • Filesystem: btrfs (or ZFS with at least 4 GB RAM) for checksums, scrub, and snapshots. ext4 or XFS with tested backups is acceptable on small VPSes.
  • App files: write temp file, fsync it, rename it, fsync the directory.
  • After any unclean shutdown, run git fsck (GitLab runbook, research §11).

8.5 Manual failover with fencing

  1. Stop writes on the primary (block the HTTP and SSH listeners, or put the instance in read-only mode).
  2. Fence the old primary: revoke its replication and backup credentials and make sure it cannot accept writes, so it cannot come back as a second writer. Do this before promotion.
  3. Confirm the standby's last VACUUM INTO snapshot and last ref reconcile. Record the data-loss window.
  4. Restore the DB snapshot and repos on the standby. Run quick_check and fsck.
  5. Promote the standby. Switch DNS. DNS and health checks alone do not enforce a single writer, so fencing in step 2 is required.
  6. Record the incident and the actual RTO.

9. Security checklist

Do:

  • Put --end-of-options before user revisions and -- before paths. Reject names and refs that start with -.
  • Route all git calls through GitCommand.
  • Check every path component for symlinks when rendering, archiving, or writing.
  • Use exact, case-sensitive key lookup (Gitea auth bypass, research §2).
  • Disable ssh-rsa with SHA-1. Set handshake and auth timeouts (Gogs DoS, research §2).
  • Accept only GIT_PROTOCOL from SSH env requests (Gogs CVE-2024-39930, research §2).
  • Store tokens and session tokens as sha256 hashes. Compare in constant time. Show tokens once.
  • Keep tokens out of URLs and logs.
  • Return 404 for both missing and forbidden private repos.
  • Normalize names with NFKC and case-fold. Keep a unique index on the folded form. Keep a reserved-names list. Reject names ending in .git or .atom.
  • Serve raw and user content from a separate origin with nosniff and a strict CSP. Use Content-Disposition: attachment for HTML and SVG.
  • Sanitize rendered markdown with comrak then ammonia. Allow only http, https, and mailto links.
  • Use linear-time regex only (Rust regex crate).
  • Cap request bodies on non-git routes, diff sizes, blame and log time, and archive sizes. Git routes are the exception, with streaming and no cap.
  • For SSRF (mirrors, webhooks, migrations): resolve DNS, pin the IP, block private and link-local ranges, re-check after every redirect, and block file://.
  • Run as a dedicated service user. Keep the git binary patched.
  • Set receive.fsckObjects.
  • Rate limit per user and per IP, with writes weighted heavier. Gate open registration behind email verification.

Don't:

  • Don't use a shell, sh -c, or string-built commands.
  • Don't use dumb HTTP or http.getanyfile.
  • Don't recurse into submodules or trust repo-local config.
  • Don't let users write hooks or config.
  • Don't hard-delete users. Reassign their issues and comments to a ghost user (M3).
  • Don't let the browser use the app origin for raw content.
  • Don't bypass branch rules through a non-PR path (the Gitea gap: required checks only applied to PR merges, research §3).
  • Don't rely on Turso Sync or Litestream until spiked.

10. Milestones

M1: clone and push work (this plan)

  • Scaffold the workspace (cargo add, latest versions), config template, .gitignore, systemd unit, SCSS build.
  • Local accounts with argon2id. First user becomes admin, via admin create-user or /setup while zero users exist. Registration is closed by default.
  • Sessions, CSRF on every POST, __Host- cookie. Access tokens (forge_) and SSH keys with exact blob match.
  • Repo create with the ID-based path. Visibility, collaborators, archive flag.
  • Smart HTTP (info/refs, upload-pack, receive-pack, gzip, streaming) and SSH (exec allow-list, GIT_PROTOCOL, piping, exit status).
  • Pre-receive policy (archived, default-branch delete and force-push, bad ref names) and post-receive push_events. Minimal UI: setup, login, dashboard, new repo, repo page, settings, admin users, /healthz.
  • Tests: unit tests in core and git; end-to-end git_e2e.rs over HTTPS and SSH, with negative cases (bad token, cross-user private 404, forced default-branch push rejected, unknown SSH command rejected, traversal rejected, POST without CSRF rejected).

M2: repo browsing and raw domain (implemented)

Built:

  • forge-git (browse.rs, gix, blocking, called through spawn_blocking): ref listing, revision resolution to a full OID, splitting <ref>/<path>, tree and blob reads from the object database, commit reads, and path validation. Symlinks are shown as their target text and never resolved.
  • forge-git (history.rs, git CLI, OIDs only): paginated log, commit_diff against the first parent with rename detection and the limits in section 10 below, one-walk last_commits, and archive_command.
  • forge-core (raw_token.rs): sign_raw / verify_raw for private raw links (10 minute TTL).
  • forge-web render/: markdown.rs (comrak, then ammonia, relative links rewritten to the repo tree, blob, and raw bases) and highlight.rs (syntect, class-based output, served at /assets/highlight.css).
  • forge-web raw.rs: the user-content listener, bound by boot.rs on user_content_listen, with explicit routes only.
  • forge-web pages: repo (home and README), tree, blob, refs (branches and tags), commits, commit, archive.

Routes on the app origin:

  • /{owner}/{repo}, /{owner}/{repo}/tree/{*rest}, /{owner}/{repo}/blob/{*rest}
  • /{owner}/{repo}/commits/{*rest} (?page=, 50 per page), /{owner}/{repo}/commit/{oid}
  • /{owner}/{repo}/archive/{ref}.tar.gz and .zip
  • /{owner}/{repo}/branches, /{owner}/{repo}/tags

Routes on the user-content origin: GET /{owner}/{repo}/{*rest}, where rest is <ref>/<path>.

Limits: blob view 1 MiB; highlighting skipped above 512 KiB or 20,000 lines; raw files up to 8 MiB held in memory, larger ones streamed; diffs 300 files, 2,000 lines per file, 20,000 lines total; last-commit lookup is one bounded git log walk (2,000 commits, 3 second timeout), with no cache.

Known gaps:

  • Footnote ids are not prefixed user-content- (heading ids are).

  • No Dockerfile grammar in syntax highlighting.

  • A background commit-graph and maintenance run after push is being added. It is not in place yet.

  • The app CSP header is computed once per process (OnceLock), so a change to user_content_base_url needs a restart.

  • Tree, blob, commits, and diff views through gix, with git log and git diff as fallback where needed.

  • README rendering with comrak, then ammonia. Relative links and images rewritten to repo paths.

  • Syntax highlighting (syntect, or tree-sitter with matching core versions).

  • Raw file serving on user_content_base_url, with the headers in section 5.5.

  • Symlink-safe path resolution, and cat-file --batch-command pool.

  • Caches: last commit per file, archives. Maintenance schedule and commit-graph.

M3: issues

  • Issues with per-repo numbers, state, author, timestamps, and edit history.
  • Comments on issues (polymorphic comments table), with ghost-user reassignment on account deletion.
  • Labels and milestones, with per-repo scope.
  • Issue list and detail pages, filters, and permissions through authorize().
  • Markdown bodies sanitized as in M2.

M4: pull requests

  • PR head stored as refs/pull/N/head in the base repo, updated on every push to the head branch in post-receive.
  • Mergeability checked with git merge-tree --write-tree, recomputed when base or head moves. Optional refs/pull/N/merge.
  • Three-dot diffs from the merge base, rename detection (-M), binary detection, lazy-loaded large files.
  • Reviews (approve, request changes, comment) and review comments anchored to commit, path, line, side, and hunk. Comments are marked outdated, not deleted.
  • Merge strategies: merge commit (commit-tree from merge-tree output), squash, rebase, fast-forward only.
  • range-diff for "what changed since the last review." AGit (refs/for/<branch>) is considered later.

M5: webhooks, push mirrors, replication, backups, export

  • Webhooks: HMAC-SHA256 signature over the raw body, delivery id header, 10s timeout, retry queue, delivery log.
  • Pull mirrors on an interval with a minimum; push mirrors with sealed credentials, run after each push with backoff. SSRF rules from section 9 apply.
  • Git replication per section 8.2: per-repo lock, push --mirror, hourly reconcile, daily bundles, weekly fsck --full on replicas.
  • DB: VACUUM INTO snapshots every 5-15 minutes, integrity checks, JSON or F3 export.
  • Backups per section 8.4: immutable tier, append-only push, restore drills, dead-man alerts.
  • Manual failover runbook per section 8.5, tested once end to end.

M6: custom CI runner and commit-status API

  • Commit-status API (commit_statuses: commit oid, context, state, target URL, creator). Required checks enforced on push and PR merge, not only on PR merge (research §3).
  • Webhooks from M5 drive CI job dispatch.
  • Custom runner: a separate process with a narrow protocol, that reports status back through the API. Runner credentials are scoped tokens.
  • Consider a Gitea-compatible API subset later, which would let Woodpecker, Renovate, and tea work (research §9). Decide when the API surface is known.
  • Log retention and secret masking for job output.

11. Open questions

  1. Project name. "forge" is a placeholder. Renaming is a find-and-replace, but pick it before public URLs and crate names are used.
  2. Does Litestream work with Turso files? Unverified. Turso says mixed SQLite and Turso access across processes is unsupported. Run a spike before relying on it for replication. Fallback is the VACUUM INTO path in section 8.3.
  3. Hosting providers. Primary, standby, and backup providers must differ in provider, payment method, contact email, and jurisdiction (section 8.1). Candidates are not chosen. Confirm after reviewing the provider risk notes in research §11 (some are unverified).