Tahinli/forge
public0 watching
★ 0
| 1 | # Wave C: orgs, email, wiki and polish (contract between workers) |
| 2 | |
| 3 | The global rules in `~/.cache/forge-orch/waveb-common.md` apply. Background research: |
| 4 | - `~/.cache/forge-orch/orgs-study.md`: every place where the owner is assumed to be a user. |
| 5 | - `~/.cache/forge-orch/email-study.md`: the im and iz mail designs. |
| 6 | - `~/.cache/forge-orch/polish-audit.md`: what exists already. |
| 7 | |
| 8 | ## Shared-file rules (many workers in parallel) |
| 9 | - **Migrations:** each worker owns exactly one number. Add one line to the array in `crates/forge-core/src/store/migrate.rs`, re-reading the file first. |
| 10 | |
| 11 | | Migration | Owner | |
| 12 | |---|---| |
| 13 | | 0025_orgs | O1 | |
| 14 | | 0026_mail | M1 | |
| 15 | | 0027_wiki | W1 | |
| 16 | | 0028_discussions | D1 | |
| 17 | | 0029_snippets | S1 | |
| 18 | | 0030_ui_prefs | T1 | |
| 19 | | 0031_advisories | A1 | |
| 20 | | 0032_repo_transfers | P1 | |
| 21 | - **Store code:** put it in a new `crates/forge-core/src/store/<feature>.rs`. Don't grow `store/mod.rs`, apart from the `mod` line and changes your section explicitly requires. |
| 22 | - **Shared files** (`pages/mod.rs` route lists, `views/mod.rs` (shell and repo tabs), `names.rs` reserved words, `lib.rs`, `style/main.scss`): small hunks, and re-read the file right before every edit. |
| 23 | - **SCSS:** each feature appends its own clearly commented block at the end of `style/main.scss`, or a partial if the file already uses partials. |
| 24 | - **Audit:** every state-changing action writes an audit entry, as the existing features do. Use new action names, and grep the tests that assert audit counts. One of them asserts a total (48); if you add actions, update it. |
| 25 | - **No JS anywhere.** Use forms and links only. |
| 26 | - **Tests:** unit tests in the store module, plus one e2e file per feature in `crates/forge-web/tests/<feature>_e2e.rs` using `tests/common/mod.rs`. |
| 27 | |
| 28 | ## O1: organizations core (wave 1) |
| 29 | **Model:** an org is a row in `users` with `kind = 'org'`. That keeps one namespace (`/{name}` works for both), keeps `repos.owner_id → users(id)`, and leaves every `owner_name` join unchanged. |
| 30 | |
| 31 | **Migration `0025_orgs.sql`:** |
| 32 | ```sql |
| 33 | ALTER TABLE users ADD COLUMN kind TEXT NOT NULL DEFAULT 'user' CHECK (kind IN ('user','org')); |
| 34 | ALTER TABLE users ADD COLUMN description TEXT; -- only if no such column exists (check 0002_profile) |
| 35 | CREATE TABLE org_settings (org_id TEXT PRIMARY KEY REFERENCES users(id) ON DELETE CASCADE, |
| 36 | default_repo_role TEXT NOT NULL DEFAULT 'read' CHECK (default_repo_role IN ('none','read','write')), |
| 37 | members_can_create_repos INTEGER NOT NULL DEFAULT 1); |
| 38 | CREATE TABLE org_members (org_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE, |
| 39 | user_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE, |
| 40 | role TEXT NOT NULL CHECK (role IN ('owner','member')), created_at TEXT NOT NULL, |
| 41 | PRIMARY KEY (org_id, user_id)); |
| 42 | CREATE TABLE teams (id TEXT PRIMARY KEY, org_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE, |
| 43 | name TEXT NOT NULL, name_folded TEXT NOT NULL, description TEXT NOT NULL DEFAULT '', |
| 44 | role TEXT NOT NULL CHECK (role IN ('read','triage','write','maintain','admin')), |
| 45 | all_repos INTEGER NOT NULL DEFAULT 0, created_at TEXT NOT NULL, UNIQUE (org_id, name_folded)); |
| 46 | CREATE TABLE team_members (team_id TEXT NOT NULL REFERENCES teams(id) ON DELETE CASCADE, |
| 47 | user_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE, PRIMARY KEY (team_id, user_id)); |
| 48 | CREATE TABLE team_repos (team_id TEXT NOT NULL REFERENCES teams(id) ON DELETE CASCADE, |
| 49 | repo_id TEXT NOT NULL REFERENCES repos(id) ON DELETE CASCADE, PRIMARY KEY (team_id, repo_id)); |
| 50 | ``` |
| 51 | **Org rows:** |
| 52 | - They get an unusable `password_hash` (`"!org"`). |
| 53 | - **Hard rule:** an org can never authenticate. Every authentication path must reject `kind = 'org'` explicitly: |
| 54 | - password login, sessions, tokens, SSH keys (including the add-key paths), LFS tokens, TOTP; |
| 55 | - admin `create-user`/`reset-password`/`set-username` (refuse orgs); |
| 56 | - the CI actor. |
| 57 | Add a test for each. |
| 58 | - The `User` struct gets `kind: AccountKind { User, Org }`, with a `is_org()` helper. |
| 59 | |
| 60 | **`store/orgs.rs`:** |
| 61 | - `create_org(name, creator_id)`: the creator becomes owner, and the org gets the same name validation as a username, plus `org_settings` defaults. |
| 62 | - `org_members`, `add_org_member`, `set_org_member_role`, and `remove_org_member`, which refuses to remove the last owner. |
| 63 | - Teams CRUD, team members and team repos. |
| 64 | - `orgs_for_user`. |
| 65 | - `delete_org`, which refuses while the org owns repos. |
| 66 | - `org_repo_role(org_id, user_id, repo_id) -> Option<Role>`: owner gives admin. Otherwise the max of `default_repo_role` (for members) and the roles of the user's teams that include the repo, or that have `all_repos`. |
| 67 | |
| 68 | **authz:** extend the existing authorization input (see `crates/forge-core/src/authz*` and its callers in `access.rs`) with `org_role: Option<Role>`, and load it in the same place collaborators are loaded. The effective role is the max of the collaborator role and the org role. An owner check (`owner_id == actor.id`) never matches an org. |
| 69 | |
| 70 | **Audit every existing site that compares `owner_id == user.id`** (orgs-study lists them: `internal.rs:526/820`, `repo.rs:282`, `admin_users.rs`, `notify.rs`) and route it through the effective role instead. |
| 71 | |
| 72 | **Notifications:** wherever "the repo owner" is a recipient, an org owner expands to the org's owners. |
| 73 | |
| 74 | **User deletion:** refuse to delete a user who is the last owner of any org. Show the message on the admin page and the CLI. |
| 75 | |
| 76 | **Tests:** |
| 77 | - the role math; |
| 78 | - auth refusal for orgs; |
| 79 | - the last-owner rules; |
| 80 | - a store-level `repo_by_owner_and_name` for an org-owned repo. |
| 81 | |
| 82 | ## O2: organizations UI (wave 2, after O1) |
| 83 | **Routes:** the prefix `org` is already reserved. |
| 84 | - `GET/POST /org/new`. Allowed for everyone, unless the config `[orgs] creation = "admin"`; the default is `"all"`. |
| 85 | - `/org/{name}/members`: add by username (owners only), change role, remove, leave. |
| 86 | - `/org/{name}/teams`, `/org/{name}/teams/{team}`: members, repos, role. |
| 87 | - `/org/{name}/settings`: description, default role, members-can-create, delete. |
| 88 | |
| 89 | **Other pages:** |
| 90 | - `/{name}` (`profile.rs`): when the target is an org, show the repos visible to the viewer, the members list and the teams count. |
| 91 | - New repo form: an owner `<select>` listing self plus the orgs where the viewer may create. |
| 92 | - Transfer to an org is handled by P1, not here. O2 does not edit `repo_rename.rs`. |
| 93 | - `explore`: orgs are listed separately or marked. |
| 94 | - Admin users page: shows the kind. |
| 95 | |
| 96 | **Webhooks:** the payload `repository.owner` stays the name string. Add `"owner_kind": "user"|"org"`. |
| 97 | |
| 98 | **E2e:** create an org, add a member, create a team with write on a repo, then: |
| 99 | - that member can push over HTTPS; |
| 100 | - a non-member can't see a private org repo (404); |
| 101 | - the org can't log in. |
| 102 | |
| 103 | ## M1: mail core (wave 1). Follow iz's outbox and im's token design (see the study). |
| 104 | - **Crate:** `lettre` (latest) with `smtp-transport, builder, tokio1-rustls, ring, webpki-roots, pool`, and `default-features = false`. |
| 105 | - **`0026_mail.sql`:** |
| 106 | - `mail_settings`, a singleton row: `host, port, security ('starttls'|'tls'|'none'), username, password_sealed BLOB, from_addr, from_name, enabled`; |
| 107 | - `mail_outbox (id, idem_key TEXT UNIQUE, to_addr, subject, body_text, created_at, attempts, next_attempt_at, lease_until, sent_at, last_error)`; |
| 108 | - `users.email_verified_at TEXT`. |
| 109 | - **Sealing:** the password is sealed with the existing instance key (forge.key, XChaCha20-Poly1305). Find the existing secret-sealing helper (used by repo secrets, CI and mirrors) and reuse it. Debug impls redact the password. |
| 110 | - **Transport:** a `Mailer` trait (`async fn send(&self, msg) -> Result`). The implementations are: |
| 111 | - `SmtpMailer` (lettre, 30 s timeout); |
| 112 | - `LogMailer`, which writes to `<data_dir>/mail-log/` and is enabled by `[mail] transport = "log"` in forge.toml, for dev and tests; |
| 113 | - a test `FakeMailer` that collects messages. |
| 114 | - **`enqueue(idem_key, to, subject, body)`:** idempotent. The background worker leases rows and sends them, retrying up to 5 times with exponential backoff (1m, 5m, 30m, 2h, 12h), then gives up. Wire it into boot the same way the mirror worker is. |
| 115 | - **Admin page `/admin/mail`:** the settings form, plus a "send test mail" button, and a list of the last 50 outbox rows (status and error, never the body). |
| 116 | - **Templates:** plain text, built in one module `crates/forge-web/src/mail/templates.rs` as functions returning `(subject, body)`. i18n will hook in later through that module only. |
| 117 | - **Tests:** outbox idempotency, the retry schedule, the lease, sealing round-trip, and Debug redaction. |
| 118 | |
| 119 | ## M2: mail flows (wave 2, after M1) |
| 120 | - **Password reset** (`/forgot`, `/reset/{token}`): |
| 121 | - Tokens are 32 random bytes, stored as sha256, with a 60-minute TTL. |
| 122 | - Single use, enforced by an atomic `UPDATE … SET used_at=? WHERE token_hash=? AND used_at IS NULL AND expires_at>? RETURNING user_id`. |
| 123 | - Mail goes only to **verified** addresses. |
| 124 | - `/forgot` always shows the same response. The lookup and enqueue happen in a spawned task, so response timing doesn't reveal whether the account exists. |
| 125 | - A successful reset revokes all sessions and is audited. |
| 126 | - Rate-limit with the existing `throttle.rs`. |
| 127 | - **Email verification:** setting or changing the email (profile and setup) sends a verify link (24 h TTL, single use) and clears `email_verified_at`. The settings page shows the status and a "resend" button. |
| 128 | - **Invites:** `registration = "invite"`. Admins invite by email from `/admin/users`, with a 7-day single-use token. `/register?invite=` creates the account with that email pre-verified. |
| 129 | - **Notification emails:** |
| 130 | - A per-user setting, on by default, that applies only to verified addresses. |
| 131 | - Send for the existing notification kinds. Read `notify.rs` for the list. |
| 132 | - Each mail has a link and the threading headers `Message-ID`, `In-Reply-To` and `References`, keyed per issue or PR. |
| 133 | - The idempotency key is the notification id. |
| 134 | - **E2e** with `transport = "log"` or `FakeMailer`: the reset flow end to end, token reuse refused, an expired token refused, an unverified address gets no mail, and the invite flow. |
| 135 | |
| 136 | ## W1: wiki (wave 1) |
| 137 | - **Storage:** a bare repo at `<repo path minus .git>.wiki.git`, created lazily on the first page save. Add a helper next to `repo_path` in `forge-git/src/repo.rs`. |
| 138 | - **`0027_wiki.sql`:** `repos.wiki_enabled INTEGER NOT NULL DEFAULT 1`. |
| 139 | - **Pages:** |
| 140 | - `/{o}/{r}/wiki` shows `Home`, or "create the first page"; |
| 141 | - `/{o}/{r}/wiki/{page}`; |
| 142 | - `/{o}/{r}/wiki/{page}/edit`; |
| 143 | - `/{o}/{r}/wiki/-/new`; |
| 144 | - `/{o}/{r}/wiki/-/pages`; |
| 145 | - `/{o}/{r}/wiki/{page}/history`. |
| 146 | Pages are `<Title-With-Dashes>.md` files on `main`. Markdown is rendered with the existing renderer, and wiki links `[[Page]]` resolve. |
| 147 | - **Web edits:** commit via plumbing with the forge-git builder (`hash-object -w --stdin`, `mktree` or `read-tree`/`update-index` on a temp index, `commit-tree`, `update-ref` with the old value). The author is the user's name and email. A conflicting concurrent edit gets an error page, not a lost write. |
| 148 | - **Permissions:** read needs repo Read. Edit and push need repo Write. Hide the tab when the wiki is disabled. |
| 149 | - **Git access:** |
| 150 | - Clone and push over HTTPS at `/{o}/{r}.wiki.git` (and `/{o}/{r}.wiki`), and over SSH at `o/r.wiki.git`. |
| 151 | - Extend `forge-ssh/src/parse.rs` and `git_http.rs` resolution. |
| 152 | - Repo names must not end in `.wiki`; add that to `names.rs`. |
| 153 | - Pre-receive applies to wiki pushes, but skips the default-branch protections. |
| 154 | - **Coverage:** |
| 155 | - repo delete and trash move the wiki too; |
| 156 | - backup bundles and verify include the wiki when present (`verify.rs` hard-codes `repos/{id}.bundle`, so add `repos/{id}.wiki.bundle`); |
| 157 | - restore and `admin fsck` handle it too; |
| 158 | - replication carries it, so read `replication/` and extend it the same way as the main repo; |
| 159 | - mirrors don't mirror wikis (document this). |
| 160 | - **E2e:** |
| 161 | - create a page on the web, clone the wiki over HTTPS and see it, push a change and see it on the web; |
| 162 | - a Read-only user can't edit; |
| 163 | - backup then verify includes the wiki. |
| 164 | |
| 165 | ## D1: discussions (wave 1) |
| 166 | - **`0028_discussions.sql`:** |
| 167 | - `repos.discussions_enabled INTEGER NOT NULL DEFAULT 0`; |
| 168 | - `discussions (id, repo_id, number, category ('general','ideas','qa','announcements'), title, body, author_id, answer_comment_id, locked, created_at, updated_at)`. Number them with the same per-repo counter as issues and PRs if one exists, or their own counter otherwise; check how issues number. |
| 169 | - `discussion_comments (id, discussion_id, parent_id NULL /* one level of replies */, author_id, body, created_at, updated_at, deleted_at)`. |
| 170 | - **Pages:** |
| 171 | - `/{o}/{r}/discussions`, with a category filter; |
| 172 | - `/{o}/{r}/discussions/new`; |
| 173 | - `/{o}/{r}/discussions/{n}`, showing replies, "mark as answer" in `qa` (author or Triage+), lock (Triage+), edit and delete own. |
| 174 | - Only Maintain+ may post in `announcements`. |
| 175 | - Reuse reactions and the markdown renderer. Mentions notify, the same way issues do. |
| 176 | - A repo settings toggle and a repo tab. |
| 177 | - **E2e:** create, reply, nested reply, answer, lock refusal, and the toggle hiding the tab. |
| 178 | |
| 179 | ## S1: snippets (wave 1) |
| 180 | - **`0029_snippets.sql`:** |
| 181 | - `snippets (id, owner_id, title, description, visibility ('public','unlisted','private'), created_at, updated_at)`; |
| 182 | - `snippet_files (snippet_id, name, content, position)`; |
| 183 | - `snippet_revisions (id, snippet_id, author_id, created_at, files_json)`. |
| 184 | Limits: 10 files, 1 MiB total. |
| 185 | - **Routes:** use a reserved prefix, adding `snippets` to `names.rs` if needed. |
| 186 | - `/snippets`: your snippets plus public ones; |
| 187 | - `/snippets/new`; |
| 188 | - `/snippets/{id}`, with syntax highlighting via the existing highlighter, and markdown rendered for `.md`; |
| 189 | - `/snippets/{id}/edit`; |
| 190 | - `/snippets/{id}/revisions`; |
| 191 | - raw on the user-content origin at `/-/snippets/{id}/{file}`. Private snippets use the same token mechanism as private raw files. |
| 192 | - `unlisted` means viewable by link, not listed. |
| 193 | - **E2e:** create, edit (revision recorded), private snippet hidden from others (404), raw served with the right content type. |
| 194 | |
| 195 | ## T1: theme and language preferences (wave 1) |
| 196 | - **`0030_ui_prefs.sql`:** |
| 197 | - `users.theme TEXT NOT NULL DEFAULT 'auto' CHECK (theme IN ('auto','light','dark'))`; |
| 198 | - `users.language TEXT NOT NULL DEFAULT 'auto'`. |
| 199 | - **Theme:** `views/mod.rs` `shell` emits `<html data-theme="light|dark">` when the preference isn't auto. |
| 200 | - SCSS: the existing dark palette under `@media (prefers-color-scheme: dark)` must apply only to `:root:not([data-theme="light"])`, and also under `:root[data-theme="dark"]`. The same goes for the highlight CSS. |
| 201 | - The settings page `/settings/appearance` has theme and language selects. I18 provides the language list; until then, offer only `auto` and `en`. |
| 202 | - Anonymous visitors get a footer form POSTing to `/-/theme`, which sets a `forge_theme` cookie (Lax, 1 year). The cookie is honoured when no user is signed in. |
| 203 | - **E2e:** the setting persists, the html attribute is rendered, and the anonymous cookie works. |
| 204 | |
| 205 | ## I18: i18n infrastructure (wave 1). Follow im's design (`~/Documents/Code/Rust/im/crates/im-web/src/i18n.rs`). |
| 206 | - `crates/forge-web/src/i18n.rs` defines `Lang { En, Tr }`, a `Key` enum, and an exhaustive `t(lang, key) -> &'static str`, plus `fmt` helpers for strings with arguments. |
| 207 | - **Language selection:** the user's `language` column (T1 adds it; until it lands, read it defensively or wait), else `Accept-Language`, else En. `shell` sets `<html lang>`. |
| 208 | - **Wave 1 scope:** the shell, nav, footer, login, setup, register, settings pages, and the 404 and error pages. Turkish translations must be real Turkish (the user is Turkish). |
| 209 | - Expose `Lang` to page code through a `Cx` helper (`lang(cx)`). |
| 210 | - **Wave 3 (later):** one worker per page group converts the rest. Don't do that now. |
| 211 | - **Tests:** an exhaustiveness test (every Key non-empty in every Lang), Accept-Language parsing, and an e2e that the Turkish login page renders Turkish. |
| 212 | |
| 213 | ## A1: security advisories (wave 1). Opt-in, because it downloads data. |
| 214 | - **Config:** `[advisories] enabled = false`, `ecosystems = ["crates.io","npm","Go"]`, `refresh_hours = 24`. |
| 215 | - **Data:** OSV per-ecosystem dumps from `https://osv-vulnerabilities.storage.googleapis.com/<ecosystem>/all.zip`, downloaded with the existing HTTP client setup (timeouts, size cap 500 MiB) into `<data_dir>/advisories/<eco>/`. Parse only `id`, `summary`, `aliases`, `severity`, `affected[].package.{ecosystem,name}`, `affected[].ranges` (SEMVER and ECOSYSTEM events) and `affected[].versions`. |
| 216 | - **`0031_advisories.sql`:** |
| 217 | - `advisories (id, ecosystem, package, summary, severity, ranges_json, versions_json, modified)`; |
| 218 | - `repo_dependencies (repo_id, ecosystem, package, version, manifest_path)`; |
| 219 | - `repo_alerts (repo_id, advisory_id, package, version, state ('open','dismissed','fixed'), created_at, updated_at)`. |
| 220 | - **Scanning:** after a push to the default branch (hook into `post_update.rs`), read `Cargo.lock`, `package-lock.json` (v2 and v3 `packages`) and `go.sum` from the tree (root and up to depth 3). Recompute dependencies, open new alerts, and mark disappeared ones fixed. Rescan all repos after each advisory refresh. |
| 221 | - **Version matching:** semver for crates.io and npm, and Go pseudo-versions as semver. Use the `semver` crate. |
| 222 | - **Page:** `/{o}/{r}/security` (Write+ only) lists alerts with dismiss and reopen. New alerts notify the repo's admins. |
| 223 | - **Tests:** parsers for all three lockfiles, range matching (introduced, fixed, last_affected), and an e2e with a hand-written OSV fixture dir and a pushed `Cargo.lock`. |
| 224 | |
| 225 | ## P1: repo transfer acceptance (wave 2, after O1) |
| 226 | - **`0032_repo_transfers.sql`:** `repo_transfers (repo_id PRIMARY KEY, from_owner_id, to_owner_id, initiated_by, created_at, expires_at)`. |
| 227 | - **Flow:** |
| 228 | - A transfer now creates a pending row with a 7-day expiry and notifies the target (the user, or an org's owners). |
| 229 | - The target accepts or declines on `/{o}/{r}/transfer` or the notification. Accepting runs the existing `transfer_repo`. |
| 230 | - The initiator may cancel. |
| 231 | - Admin transfers skip acceptance. |
| 232 | - **Org targets:** the target may be an org where the actor is an owner (this moved here from O2). For an org target, any owner of the org may accept. |
| 233 | - **E2e:** initiate, the repo is unchanged until accepted, accept moves it and the redirect works, decline, expiry, and a transfer to an org. |
| 234 | |
| 235 | ## Round-2 follow-ups (from the round-1 reports) |
| 236 | - **F-A (advisories):** |
| 237 | - add a Security tab in the repo tabs, shown to Write+ only and only when advisories are enabled; |
| 238 | - new alerts create a notification for the repo admins (owner, or the org owners plus admin collaborators) through `notify.rs`, plus a test. |
| 239 | - **F-D (discussions):** |
| 240 | - the notifications inbox lists `discussion_notifications` (mentions, and replies to your discussion); |
| 241 | - lock is enforced in the store too, not only in the pages; |
| 242 | - deleting a user reassigns their discussions and comments to the ghost user, the same way issues do (`store/user_delete.rs`); |
| 243 | - tests for each. |
| 244 | - **F-W (wiki):** |
| 245 | - CI job tokens can read (clone) the wiki of the job's repo; |
| 246 | - wiki pushes write an audit entry; |
| 247 | - document in `docs/` (wherever mirrors are documented) that mirrors don't include wikis; |
| 248 | - add a backup round-trip test that includes a wiki. |