Dependabot #190-#194 combined with Zustand 5.1.0, soupsieve 4.1, coverage 7.17.3, Ruff 0.17.5 and a cargo update (wry 0.57.2, tauri-plugin-opener 2.7.3), each past a 3-day cooldown; Wrangler 4.161.0 with regenerated Worker types; tapable held at 2.3.3. Version 4.6.2 with a short changelog entry. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
215 lines
69 KiB
Markdown
215 lines
69 KiB
Markdown
# Repo notes for Claude
|
||
|
||
Quick context so future Claude sessions don't have to re-derive it.
|
||
|
||
## What this repo is
|
||
|
||
A curated, auto-updating catalog of free itch.io games, published at **https://freeitchgames.win** (read-only for
|
||
everyone; the old github.io URL only redirects). Three layers share one JSON catalog:
|
||
|
||
1. **Python pipeline** — the data layer (GitHub Actions).
|
||
- `data_game/game_info_NNN.json` — chunked records, max 500 per file, 6,069 games on 2026-11-16. Serialization is fixed: `json.dumps(..., ensure_ascii=False, indent=4)`, no trailing newline (`json_io.dumps`) — the Worker port (`webapp/worker/catalog.ts`) emits identical bytes (`tests/fixtures/golden/`).
|
||
- `data_game/index.json` — chunk manifest (total + per-file counts). `data_game/count_history.json` — date-keyed `{date,total}` series ("games over time" chart). `scripts/deleted_games.json` — removal log (public "Removed" page). `scripts/temp_link.json` — ingest queue.
|
||
- `scripts/` — scanners never write the catalog: `update_info.py` (queued URLs) and `refresh.py` (rotating 1/7-per-day check; `--full` re-scrape) emit a URL-keyed **patch** (`patch.py`); `apply_patch.py` applies it to the latest `main` and runs `validate.py`; `bash/commit_push.sh` does fetch → reset → apply → commit → push with retries. Shared: `scraper.py` (session, `Pacer` + 429 back-off, parsing: the info panel incl. itch.io's 2025–26 rows — `AI Disclosure` ("AI Assisted", then what for) / `Content` linking `/games/tag-no-ai` → `ai_disclosure` + `ai_content`, `Accessibility` → `accessibility`; `description` = the creator's tagline (`og:description`), else the first sentence), `data_store.py` (chunks stay in place, rebalance only past thresholds, write-if-changed), `json_io.py`, `canonical.py` (canonical itch.io URL; vectors in `tests/fixtures/url_vectors.json`, shared with the Worker). Pipeline bookkeeping lives in `scripts/state/` (not published), including `ingest_outcomes.json` (how each queued link ended — added / duplicate / paid / dead / private / deleted / gave_up; patch key `ingest_outcomes`; the last 90 days, ≤ 5,000) that the admin's tracking and the cron's reconcile read. Tests: `pytest` (network blocked; `pytest --cov --cov-report=xml` → `coverage.xml` for SonarQube Cloud, gotcha 41). Lint: `ruff check scripts tests webapp/scripts`, `ruff format`, `vulture` (config in root `pyproject.toml`). Local env: Python 3.15 (`.github/python-version`, gotcha 39; ruff `target-version = "py315"`), `python -m venv .venv && .venv/Scripts/pip install -r requirements-dev.txt` (3.15 is in UTF-8 mode by default — no `PYTHONUTF8` needed, the workflows no longer set it). Both requirements files are hash-locked pip-tools compiles of `requirements*.in` — edit the `.in`, then recompile (gotcha 40).
|
||
2. **Cloudflare Worker + static assets** — `webapp/worker/` + `webapp/wrangler.jsonc`, deployed by **Cloudflare Workers Builds** from `main` (root dir `webapp`, build `npm run build`, deploy `npx wrangler deploy`; no deploy workflow). The catalog is bundled into the site at build time (`vite-plugins/catalog-data.ts` → `dist/data/pack/` compact packs + the legacy readable JSON in `dist/data/`, plus `data/urls.json` and `sitemap.xml`), so every data commit rebuilds the site.
|
||
3. **Web app + Tauri apps** — `webapp/src/` (React 19.3 + TypeScript 7.0 + Vite 8/rolldown + Tailwind + shadcn/ui + React Router 8), and the **Tauri 2 desktop + Android** read-only viewers in `webapp/src-tauri/` (same code; they fetch `https://freeitchgames.win/data/pack` and load covers straight from img.itch.zone). Plus `webapp/admin/` — the maintainer-only admin app (second Vite entry, served at `/admin/`).
|
||
|
||
Adding games: the public **Suggest** page (`/suggest`, Turnstile) and RSS discovery (Worker cron) feed a **D1 review queue** (the browser extension's `POST /api/ingest` was removed in 4.4.1; the extension `poli0981/itch-f2p-extension` and the Telegram bot `poli0981/telegram-scraper-bot` are archived repos); approving in `/admin` commits the URLs to `scripts/temp_link.json` as a GitHub App, which triggers `update.yml`. Since 4.4 the cron also approves by rule (`/admin` → Settings, on by default: unflagged `rss` / `form` items nobody rejected within 24 h, only those found after the switch-on baseline `since`, ≤ 100 per run) and queues the new free games of **followed itch.io collections** directly (removed or flagged ones go to the queue) — the policies say so (ToS §5, Privacy §2.4, Disclaimer §4). The Add page takes any text, a file or a collection, **Check**s it first (dry run), sends batches of 200 and follows each game's ingest (`/api/admin/track`).
|
||
|
||
## How to work in webapp/
|
||
|
||
```sh
|
||
cd webapp # Node 26 (.node-version; engines >=26)
|
||
npm ci
|
||
npm run dev # Vite on http://localhost:5173 (proxies /api and /img to wrangler dev on :8787)
|
||
npx wrangler dev # the Worker + built assets (run `npm run build` first); .dev.vars holds local secrets
|
||
npm run build # → webapp/dist/ (public app + /admin/ + /data + sitemap); Tauri builds use the same dir
|
||
npm test # vitest (worker/, src/lib); worker/test/* D1 tests spawn a local workerd via getPlatformProxy
|
||
npm run test:coverage # the same with v8 coverage → coverage/lcov.info (repo-root paths, for SonarQube Cloud)
|
||
npm run lint && npm run knip && npx tsc -b
|
||
npm run cf:types # after editing wrangler.jsonc (ignores .dev.vars via types.env; CI runs it with --check)
|
||
npm run cf:check # wrangler deploy --dry-run with the project's wrangler (what CI runs; after a build)
|
||
npx wrangler d1 migrations apply freeitchgames --local # local D1 (use --remote by hand for production)
|
||
npm run tauri:dev # native window; needs Rust toolchain
|
||
npm run tauri:build # native installers
|
||
npm run tauri icon -- path/to/source-1024.png # regenerate src-tauri/icons/
|
||
npm run tauri -- android init # generate src-tauri/gen/android (gitignored)
|
||
npm run tauri -- android build --apk --target aarch64 # arm64-v8a release APK (unsigned; CI post-signs)
|
||
```
|
||
|
||
Local admin without Access: put `DEV_ADMIN_EMAIL=you@example.com` in `webapp/.dev.vars` — honoured only when the host is localhost. Turnstile test keys work locally (`TURNSTILE_SITEKEY=1x00000000000000000000AA`, `TURNSTILE_SECRET=1x0000000000000000000000000000000AA`, `SITE_ORIGIN=http://localhost:<port>` in `.dev.vars`); add a 32+ character `GATE_SECRET` to try the verification gate under `npx wrangler dev` (on localhost the cookie is `fig_gate` without `Secure`, and no bot token is needed). `npm run dev` never sees the gate — except that its `/img` proxy to `wrangler dev` does. Android details live in `webapp/TAURI.md` → **Android APK**. Path alias: `@/*` → `webapp/src/*`.
|
||
|
||
## Worker (`webapp/worker/`)
|
||
|
||
- `index.ts` routes: `/img/*` (img.ts, behind the gate), `/api/health`, `/api/verify` (gate.ts), `/api/suggest` (suggest.ts), `/api/admin/*` + `/admin*` (admin.ts), the readable `/data/*.json` (legacy-data.ts), everything else that misses an asset → `spa.ts`. `run_worker_first` also lists `/` (otherwise served straight from `dist/index.html`) so the gate sees it, and the five legacy catalog files one by one (never a `!` pattern — gotcha 26). `scheduled()` (cron `23 */4 * * *`), four steps each in its own try/catch: `discover.ts` (one itch.io RSS feed per run, 6 h back-off on 429) → `QueueStore.reconcile` (queued → ingested / failed with the pipeline's outcome, read at head through the GitHub App; else in the catalog → ingested, > 3 days → failed `timeout`) → `automation.ts` (auto-approve + followed collections, ≤ 30 itch.io pages a run, ONE `[auto]` commit, D1 written only after it lands; the run's summary in settings `automation_last_run`) → `purge` (Suggest notes 180 d). The account is on **Workers Paid** (30 s CPU): admin catalog edits parse and rewrite ~3.8 MB in the Worker (≈ 25–30 ms for 100 removals).
|
||
- `ratelimit.ts` (since 4.5): every Worker request is counted first, per IP (IPv6 /64, `rateKey`) and per Cloudflare location (approximate, fails open), in one `ratelimits` binding per kind of traffic (counted per machine of the data centre: it holds for one-connection clients, the WAF holds for floods — gotcha 43) — `RL_SITE` 120/60 s (pages, public `/api`, legacy `/data`, misses), `RL_IMG` 600, `RL_ADMIN` 300; `RL_SUGGEST` 5 and `RL_GATE` 20 still apply on top to those POSTs. Verified bots (the gate's header) are not counted. Over a limit: 429 + `Retry-After: 60` — JSON `rate_limited` for `/api`, `/img`, files and `/data` (with CORS), the script-less EN/VI page of `gate-page.ts` for app pages. Keep every `period` 60 and a unique `namespace_id` in 88xx (staging 89xx): namespaces are **account-wide** and other Workers on the account (repolens, ticktune…) use 1001–1006, so the old 1001 / 1003 shared their counters (`ratelimit.test.ts` reads `wrangler.jsonc`). Layer 1 is the zone's WAF rate-limiting rule (Free plan: one rule, IP, 10 s; `docs/operations.md` §1 step 15): only it saves the Worker request a flood costs, and only it covers static assets.
|
||
- `img.ts`: `/img/<160|640>/<path on img.itch.zone>` — allow-listed against `data/urls.json`, edge cache → R2 `THUMBS` (`w{w}/<sha256>.webp`) → `cf.image` transform (stored once) → fallback streams the original with a 1-day cache (quota error 9422). The fallback original and a missing upstream image are also kept in `caches.default` for a day (GET only); widths are the exact strings `160` / `640` (a `Map`, no `Number()` aliases); background puts log failures. Upstream errors → controlled 502.
|
||
- `spa.ts`: `not_found_handling` is `"none"`, so misses reach the Worker: `/assets/*`, `/data/*` (a pack from an older deploy included — `no-store`, CORS) and file-like paths get a real 404 (`no-store`); app routes get the shell with 200; unknown paths and game slugs not in the catalog get the shell with **404**; with the gate on, a visitor without a pass gets the verification page instead (checked before the shell is fetched, so never a 304). Keep its `ROUTES` in sync with `src/App.tsx` and `src/hooks/useSeoHead.ts`.
|
||
- `gate.ts` + `gate-page.ts`: the **verification gate** (web only). App pages and `/img` need a pass: the `__Host-fig_gate` cookie (HMAC of its issue time, HttpOnly, `BOT_GATE_TTL_HOURS`, default 48) set by `POST /api/verify` after a Turnstile check (action `gate`, `RL_GATE`, same-origin); `GET /api/verify` reports `{enabled, valid, via, ttl}`. The page is self-contained (inline CSS/JS with a per-response CSP nonce, EN/VI, the site's og tags, 200 + noindex). Verified bots pass with `x-fig-known-bot: <GATE_BOT_TOKEN>` from a zone Transform Rule; clients that only claim to be a crawler get 503. `/assets`, `/data`, `/api/*`, `/admin*` and the other static files stay open. `turnstile.ts`: siteverify shared with Suggest (hostname + action checked; the test secrets skip both).
|
||
- `access.ts`: verifies `Cf-Access-Jwt-Assertion` with jose (issuer + AUD). Maintainer = email in secret `ADMIN_EMAILS` + same-origin writes (the only identity: the service-token ingest is gone since 4.4.1).
|
||
- `admin.ts`: API (queue list / approve / reject / check (dry run) / add / track / edit one or ≤ 100 (`/games/bulk`) / remove one or ≤ 100 (`/games/remove`; after the commit the audit row and the R2 cleanup are best effort → `warnings`, never a 500) / restore / activity / settings / itch.io collections) and the `/admin` shell with its own CSP (theme-script hash included). Request limits shared with the admin app: `limits.ts`. `collections.ts` reads public itch.io collections (gotcha 36); `automation.ts` holds the cron's rules (`planCollectionItems`: only URLs D1 has never seen and without a flag are queued directly — no re-add loop for games the ingest dropped). The maintainer's email goes only to the D1 audit log, never into a commit (the repo is public).
|
||
- `repo.ts` + `github.ts` + `catalog.ts`: writes as a **GitHub App** (installation token, cached per isolate) with GraphQL `createCommitOnBranch` + `expectedHeadOid` (verified commits; `commitWithRetry` rebuilds from the new head on a race). Reads use the raw media type (no base64/UTF-8 mojibake). `catalog.ts` ports `data_store.py` byte-for-byte (golden fixtures).
|
||
- `legacy-data.ts`: the readable catalog JSON (`index.json`, `game_info_NNN.json`, `count_history.json`, `deleted_games.json`, `urls.json`) goes only to apps 4.0–4.1.1 — `Origin` `tauri://localhost` / `http(s)://tauri.localhost` → `env.ASSETS` passthrough + `Vary: Origin`; anyone else → **410** JSON with the repository links (`no-store`, CORS, a sampled `console.info` of the Origin). A nudge, not a security boundary: the data is public in the repo.
|
||
- `queue.ts`: URL classification (catalog → blocklist → pending → previously deleted) + the D1 store (`worker/migrations/`, additive only — never edit an applied migration; `0002` added `candidates.outcome`, `settings` and `watched_collections`; the `idempotency` table and `'ext'` in the `candidates.source` CHECK are unused leftovers of the ingest API removed in 4.4.1; URL lists bind in `IN` slices of 90, gotcha 37). `data.ts`: catalog facts read from the site's own `/data` via `env.ASSETS`, cached per isolate (loaded values only — see gotchas).
|
||
- Configuration: non-secret settings are `vars` in `wrangler.jsonc` (empty until setup); secrets (`GH_APP_PRIVATE_KEY` — the PEM GitHub downloads (PKCS#1) or PKCS#8; `github.ts` converts, `ADMIN_EMAILS`, `TURNSTILE_SECRET`, `GATE_SECRET`, `GATE_BOT_TOKEN`) via `wrangler secret put`, typed in `worker/env.ts` as optional. A feature whose settings are missing answers **503** (the gate just stays off); the public site needs none of them. `env.staging` = Worker `free-games-itchio-list-staging` on `staging.freeitchgames.win` (no cron, own R2/D1/rate-limit namespaces).
|
||
|
||
## Web app structure (`webapp/src/`)
|
||
|
||
- `routes/*.tsx` — one file per route (`/`, `/games`, `/games/:slug`, `/charts`, `/removed` (`/deleted` redirects), `/suggest`, `/settings`, `/about`, `/errors/:code`); `App.tsx` wires them; `components/app-router.tsx` picks **BrowserRouter on the web, HashRouter in Tauri**. `lib/legacy.ts` rewrites v3 `#/x` links to `/x` (crafted hashes go home) and deletes v3 PAT/GPG storage keys.
|
||
- Data: `lib/config.ts` (`PACK_BASE` = `/data/pack` on the web, `https://freeitchgames.win/data/pack` in Tauri). **Packs** (since 4.2): `lib/data/pack-format.ts` (pure, shared with the build plugin: records by column, dictionaries for repeated strings and every string list; `manifest-v1.json` names per chunk a `main` pack — every field but `description` — and a `desc` pack, plus `deleted` / `history`; files are `<sha256 prefix>.bin`, deflate-raw, served `immutable`). `lib/data/catalog.ts` loads them (`['db','all']` →`{ games, index: {total_games, last_updated}, chunks }`; the manifest revalidated per load and shared 10 s; a main-pack 404 → manifest `cache: 'reload'` + one retry; an unchanged manifest returns the previous result). `lib/data/compression.ts`: `DecompressionStream('deflate-raw')`, else lazy `fflate`. `Game` has **no `description`**: `hooks/useDescriptions.ts` loads `['desc', <file>]` on demand (game page: its chunk; Games search: all, `filterGames(…, descriptions)`).
|
||
- Cache: react-query persisted to IndexedDB (`webapp.query-cache`, 7 days) by `lib/data/persister.ts` — byte 0 = format (1 deflate-raw, 0 plain), then the JSON; a legacy string is dropped; writes throttled 1 s and skipped when no data object changed. Persisted: `db`, `deleted`, `count-history` and only the `desc` files the current catalog names (`shouldPersistQuery`); stale `desc` queries are removed after a `db` success. `CACHE_BUSTER = ${APP.version}:data-v2`. **`gcTime` must stay >= persist `maxAge`.**
|
||
- Layout: `components/site/*` (sticky top nav + global search with the `/` shortcut + display menu + phone sheet menu; footer) — the window scrolls, not `<main>`. `/` is the Welcome page (`routes/welcome.tsx`).
|
||
- `/games`: search, filters, sort and page live in the URL — `lib/game-filters.ts` (`parseQuery` / `toParams` drop defaults; params `q genre platform status tag lang input engine ai access rating browser sort page`; `ai` treats "N/A" / a missing field as its own "Not disclosed" value and lists its options in a fixed order (`aiRank`); unit-tested) — rendered by `components/games/*` (filter chips with facet counts that ignore their own filter, a desktop ARIA table / phone list chosen by `useIsMobile`, 100 per page; the detail page's breadcrumb links back to the same list via router state). Counts show the whole catalog ("2,729 of 2,811") because 18+ games are hidden by default.
|
||
- `components/charts/` (19 charts, 4 lazy tabs loaded with `lib/lazy-preload.ts` — preloaded when idle or on hover/focus, rendered without a Suspense fallback once loaded — and kept mounted once shown (gotcha 33; skeletons in `chart-skeletons.tsx` match the cards), every series spreading `SERIES_ANIMATION` (300 ms, no delay; a test checks), **no barrel `index.ts`**; every Recharts tooltip/cursor comes from `chart-theme.ts` + `chart-tooltip.tsx`, every pie legend from `LEGEND_BASE` there; every chart container spreads `chartAria(id)` (named by its card's h2 + description + arrow-key hint), every `<Pie>` has `rootTabIndex={-1}`, every `ChartCard` gets an `id` and a `table` (a lazy `<details>` "Data table" built with `lib/chart-table.ts`), and the tooltip reads its text into the card's polite live region through `ChartAnnounce` (a ref, no re-render — gotcha 33); `chart-theme.test.ts` checks all four; below `md` the charts get 12 px legends / 11 px ticks from a media block in `index.css`, the tab bar is one row of equal tabs, and Languages / Avg session switch to horizontal bars via `useIsMobile`), `components/game-thumb.tsx` (`lib/thumbnail.ts` → `/img/160|640/...` on the web, the original URL in Tauri), `components/legal-gate.tsx` (a non-dismissable Radix dialog **over** the page, so deep links land where they pointed; links flagged `inGate` in `lib/about.ts`; re-prompts when `LEGAL_VERSION` in `stores/prefs.ts` changes; no "Leave" button in Tauri).
|
||
- Styling: **Tailwind v4** via `@tailwindcss/vite` (no PostCSS, no `tailwind.config`); tokens are hex CSS variables in `src/index.css` (`:root` / `.dark`, `@theme inline`; `--brand-soft`, `--thumb`, `--success`, `--scrollbar-thumb(-hover/-active)` besides the shadcn set) and must match the approved mockup — except where WCAG 2.2 AA needed more (4.6.0): `--primary` #6a48f5 (white text 5.44:1), `--primary-text` for purple text (#9a83ff in dark), `--input` #8a8a93 / #6b6b75 for field borders and the off switch (3:1), `--destructive-solid` for filled destructive buttons; `--brand` keeps the mockup's #7c5cff for the logo (as do the favicon, `og.png`, the app icons and the gate page's logo). `webapp/scripts/check-contrast.test.mjs` fails CI when a token pair or a chart colour (`components/charts/palette.ts`) drops below 4.5:1 (text) / 3:1 (borders, ring, chart marks); `lib/color.ts` does the maths. A `@media (forced-colors: active)` block gives selected tabs / pressed toggles / the current page / the active option and switches the system highlight (Windows contrast themes). Themed scrollbars (`@layer base`, only under `(hover: hover) and (pointer: fine) and (forced-colors: none)`): `::-webkit-scrollbar*` for Chromium/Safari, `scrollbar-color` / `scrollbar-width: thin` for Firefox only (gotcha 28); "Hide scrollbars" = unlayered `html[data-scrollbars='hidden']` rules (`hooks/useScrollbarsEffect.ts`). On mouse devices `<html>` reserves the scrollbar gutter, released while a dialog locks scrolling (gotcha 35). The theme: `stores/theme.ts` (synced across tabs, site ↔ admin), `hooks/useThemeEffect.ts` (`.dark` + the `theme-color` metas), an inline pre-paint script in both HTML entries (gotcha 34), and the display menu's button shows `components/theme-icon.ts`'s icon for the theme in use. Self-hosted Geist / Geist Mono (`@fontsource-variable/*`, CSP `font-src 'self'`). Numbers and dates go through `lib/format.ts` (`useFormat`: en-GB / vi-VN). `webapp/scripts/gen_assets.py` redraws favicon, PWA icons and `og.png` from the same tokens and the Geist fonts in `node_modules`.
|
||
- `lib/gate.ts` (web only): when a tab comes back into view or a cover fails and the verification pass may have run out (`webapp.gate` hint), it asks `GET /api/verify` and reloads into the verification page; `lib/reload.ts` is the shared at-most-every-30-s reload guard.
|
||
- `hooks/useSeoHead.ts` sets canonical + robots per route (index.html has no static canonical/og:url on purpose).
|
||
- Accessibility (statement: root `ACCESSIBILITY.md`; target WCAG 2.2 AA): after an in-app navigation `App.tsx`'s ScrollManager moves focus to the new page's h1 (`lib/route-focus.ts` decides: not on first load or query-only changes; a #hash focuses its section; Back focuses the link last used on that entry), waiting for lazy routes with `components/focus-target.ts` (`whenPresent`, 10 s; `focusElement` adds `tabindex=-1` + `data-focus-target`, which hides the ring). Dialogs without a trigger return focus themselves (18+: the opener, `requestNsfw(returnFocus)`; legal gate: the page h1); menus that navigate `preventDefault` their `onCloseAutoFocus` (gotcha 46). The "/" shortcut is a pref (`slashShortcut`, Settings → Accessibility; WCAG 2.1.4). `CardTitle` renders an h2. `ExtLink` appends "(opens in a new tab)" / "(opens in your browser)" for screen readers. English-only strings shown on a Vietnamese page carry `lang="en"`; `<html lang>` turns `vi` only once the VI dictionary is in. `html { scroll-padding-top }` keeps focus clear of the sticky header. No a11y linter: `eslint-plugin-jsx-a11y` peers ESLint ≤ 9.
|
||
- `lib/i18n/` — `en.ts` is the source of truth; `vi.ts` is `Record<MessageKey, string>` and must never be static-imported (lazy chunk).
|
||
- `stores/` — Zustand: `prefs` (`webapp.prefs`: language, density, `hideScrollbars`, legal acceptance…), `theme` (`webapp.theme`).
|
||
- `lib/about.ts` — keep `THIRD_PARTY` up to date when adding npm deps; `LEGAL_LINKS`; `APP.version`.
|
||
- Code split (`vite.config.ts` rolldown `codeSplitting.groups`): `vendor-react` (priority 100 — must claim React first), `vendor-query`, `vendor-ui`. There is deliberately **no vendor-charts group** (it pulled React/clsx in and put Recharts on the first load); Recharts stays in the lazy chart chunks. Initial load ≈ 189 KB gzip (JS + CSS, gzip -9, measured for 4.6.0; 4.2 added ~4.6 KB for the pack loader, persister and `useQueries`, 4.6 ~3 KB for focus management and the accessibility strings) plus ~51 KB of Geist woff2; catalog data on a first visit ≈ 264 KB (main packs + log + history), descriptions ≈ 170 KB on demand. `fflate` is a lazy chunk, never preloaded. Web build `target` is Safari/iOS 15.4 (Radix class static blocks otherwise break iOS 15); Tauri uses the default.
|
||
|
||
## Editable vs read-only fields
|
||
|
||
Each game has **20** fields (plus optional `added_at` / `updated_at`, appended by the pipeline, and since 4.3 the optional scraper fields `ai_disclosure` (`No AI|AI Assisted|N/A`) / `ai_content` / `accessibility`, which the daily refresh adds to older records — schema changes are additive only, installed v3 apps may still read the raw JSON). Only **3** are maintainer-editable: `safe_virus` (`?|Yes|No|Caution`), `notes`, `nsfw` (`Yes|No`) — via `/admin`. The other 17 are scraper output; no scrape (not even `--full`) overwrites the 3 editable fields.
|
||
|
||
## GitHub Actions
|
||
|
||
| Workflow | Trigger | Notes |
|
||
|---|---|---|
|
||
| `update.yml` | Push to `main` touching `scripts/temp_link.json` + daily 01:23 UTC + `workflow_dispatch` (`url`) | Ingest queue → patch → `commit_push.sh`. Workflow-level `concurrency: ingest` with **`queue: max`** (the default queue replaces a pending run, which would drop its dispatched `url`). A dispatched `url` is processed first and written to `temp_link.json` (`queue_add`) until it reaches a final outcome. Failed fetches count as an attempt only ≥ 6 h apart (`last_attempt`; 3 → given up), and a 429 counts as none. GitHub-App pushes (the admin Worker and its cron) trigger it; its own `GITHUB_TOKEN` push does not (no loop). Keep the filename. |
|
||
| `refresh.yml` | Daily 02:47 UTC + `workflow_dispatch` (`budget`, `url`, `allow_mass_removal`, `full`) | scan (read-only, 55-min step timeout, checks out the branch as it is when the job starts — not the queued SHA — so runs waiting in the queue see the previous run's state; patch artifact uploaded `if: always()`) → apply (`!cancelled()`: a failed / timed-out scan's checkpoint is applied, a cancelled run's is not) → notify. Workflow-level `concurrency: refresh-pipeline` + `queue: max` runs one whole scan → apply at a time (scheduled, manual and force-update runs alike). Removal needs the same 404/410, paid or `private` (itch.io's password form, `is_password_protected`; ingest drops such links as final) strike seen again ≥ 20 h after the first sighting (`strike_at`). Checkpoints never carry removals. **Mass-change guard:** a run whose removals or new strikes exceed max(10, 10 %) of the games it checked withholds all its removals and marks them — and, on a strike spike, all its strikes — `withheld_at`; later runs keep withholding those (they take at most half a run's budget) until a healthy check clears them, and alert (`mass_change`) — that pattern means itch.io markup changed; dispatch with `allow_mass_removal` only after checking on itch.io. `apply_patch.py` re-checks the same per-run limit and max(25, 5 %) of the catalog unless `ALLOW_MASS_REMOVAL=1`. |
|
||
| `force_update.yml` | `workflow_dispatch` ONLY (`url`, `budget`, `allow_mass_removal`) | Dispatches `refresh.yml` with `full: true` (`actions: write`; it used to call it through `uses:`, which failed at startup with no annotation): re-scrapes every scraper field for one URL or the next batch (rotates on `full` timestamps). Dispatch it N times in a row to cover N consecutive batches (FIFO, each from the latest main). Always preserves `safe_virus` / `notes` / `nsfw`. Canonical tool for repairing mojibake. |
|
||
| `notify-failure.yml` | `workflow_call` | Discord (`DISCORD_CI_WEBHOOK`) on failure / **cancelled** (job-level timeouts end as cancelled) / rate-limited / mass change / failed Cloudflare build. |
|
||
| `python-ci.yml` / `webapp-ci.yml` | PRs (+ pushes for Python) | ruff + format + vulture + pytest + `validate.py` (both requirements locks installed at once, so drifting pins fail) / `npm ci --ignore-scripts` + `npm run check:tauri` (gotcha 31) + `npm run cf:types -- --check` + lint + knip + vitest + build + `npm run cf:check` (`wrangler deploy --dry-run`). SonarQube Cloud runs in `sonarcloud.yml`. |
|
||
| `sonarcloud.yml` | Push to `main` + PRs (both skip data-only and `.md`-only changes) + `workflow_dispatch` | SonarQube Cloud CI analysis (gotcha 41): `token` (no `SONAR_TOKEN` — Dependabot and fork PRs — → notice, the rest skipped) → `coverage` (pytest-cov + Vitest v8, no secrets; artifact kept 1 day) → `scan` (`sonarqube-scan-action`, full history, `sonar.projectVersion` from `webapp/package.json`; the only step that uses the token). |
|
||
| `qodana.yml` | Push to `main` + PRs (both skip data-only and `.md`-only changes) + `workflow_dispatch` | Qodana (JetBrains) analysis (gotcha 49): `tokens` (no Qodana secrets — Dependabot and fork PRs — → notice, the rest skipped) → three jobs, one per Qodana Cloud project and token: `typescript` (`qodana-js`, native, `webapp/qodana.yaml`), `python` (`qodana-python`, native, root `qodana.yaml`), `rust` (`qodana-rust`, Docker image, `webapp/src-tauri/qodana.yaml`). Whole-project scans (`pr-mode: false`), no PR comment; each `qodana.yaml` has a quality gate of 0 problems (`failureConditions`), so a finding fails its job. |
|
||
| `deploy-status.yml` | Hourly + manual | Not a deploy. `bash/cf_build_status.sh` walks main's first-parent history to the latest finished `Workers Builds: free-games-itchio-list` check run (Cloudflare posts it only when a build ends; branch commits carry preview builds under the same name) and alerts once per failed build (cache key per check-run id, saved after the Discord post). Scheduled because GitHub never starts `check_run` workflows for `GITHUB_TOKEN` pushes (the data commits). |
|
||
| `release_desktop.yml` | Tag `v*` push, or manual | `create-release` makes the single draft (id passed to tauri-action v1 `releaseId`); release builds use **no** npm/cargo caches (cache-poisoning). Win + macOS aarch64 + macOS x86_64 (cross-compile) + Linux (`ubuntu-22.04` for a low glibc floor). Uploads `.dmg` / `.app.tar.gz` / `.pkg` (macOS), `.msi` / `.exe` (Windows), `.deb` / `.rpm` / `.AppImage` (Linux). macOS `.pkg` `--identifier` reads `tauri.conf.json`. |
|
||
| `release_android.yml` | Tag `v*` push, or manual | Tauri **Android** build → arm64-v8a APK, **post-signed** with `zipalign`+`apksigner` (gen/ stays gitignored). Tag → attaches to the same draft Release; manual dispatch → workflow artifact. Secrets: `ANDROID_KEYSTORE_BASE64` / `ANDROID_KEYSTORE_PASSWORD` / `ANDROID_KEY_ALIAS`. |
|
||
| `announce-release.yml`, `notify-release-pipeline.yml` | Release published / pipeline runs | Reusable workflows from `poli0981/.github`, SHA-pinned — but at that pin they still check out their notifier script from that repo's `main`. |
|
||
| `announce-discussion.yml` | Release published; push to `main` touching `CHANGELOG.md` / `changelog/**` | Local. The release goes to Discussions → Announcements, a changelog push to → General unless the head commit says `[skip-discuss]`. |
|
||
|
||
All workflows: `permissions: {}` at the top, per-job grants, actions pinned by commit SHA. Python installs are hash-locked and wheels only (`pip install --require-hashes --only-binary :all:`, gotcha 40); npm installs skip install scripts (`npm ci --ignore-scripts` — esbuild's and workerd's launchers find their platform binary without them).
|
||
|
||
## Release / tag process
|
||
|
||
```sh
|
||
git fetch origin
|
||
git tag -s vX.Y.Z origin/main -m "summary"
|
||
git push origin vX.Y.Z
|
||
# release_desktop.yml's create-release job opens ONE draft; desktop + Android jobs upload into it.
|
||
# When every asset is there, edit the notes and publish (this fires the announcements):
|
||
gh release edit vX.Y.Z --draft=false
|
||
```
|
||
|
||
Draft-first: **never** `gh release create` by hand before CI — tauri-action v1 refuses to upload into a published release, and announcements would go out before installers exist. Merge PRs into `main` with squash or a merge commit, never by fast-forward pushing a branch: its commits' preview-build check runs would then sit on main's own line and confuse `deploy-status.yml`. Tag from `main` only. Release notes go to `changelog/<major>.x/ver<major>.<minor>.x.md` plus a row in the `CHANGELOG.md` index (each row keeps the anchor of the old single-file heading as `<a name>`: published release notes link to them). Versions live in `webapp/src/lib/about.ts` (`APP.version`), `webapp/package.json`, `webapp/src-tauri/tauri.conf.json` and `Cargo.toml` (+ `Cargo.lock`) — keep them equal. GPG signing is wired through gpg-agent (`commit.gpgsign=true`, so `git tag -s` and `git commit` both sign; the pinentry must be answered — in unattended sessions commits fail rather than going unsigned). Signing key since 2026-10-10: `838884804847BE7AE5FCF0A96AD1AB0921B96825` (ed25519 certify-only primary, generated locally and never published before; signing subkey `ED0BFD3005662D5D`; `user.signingkey` = the primary fingerprint, global + repo). The old key `03F965C2E2DB5C6B` is **revoked** (compromised: it sat in the browser extension's storage, Sept 2026); tags up to v4.3.0 and older commits carry it. The owner's offline backup and revocation certificates live outside the repo.
|
||
|
||
## Licenses & policies
|
||
|
||
Code **MIT** (`LICENSE`); catalog data **CC BY 4.0** for the maintainer's contribution — excluding creators' descriptions, cover images, names/trademarks (`data_game/LICENSE.md`, `LICENSES/CC-BY-4.0.txt`); docs **CC BY 4.0**; map + non-affiliation in `NOTICE.md`. Policies (EN in `docs/`, VI mirrors in `docs/i18n/vi/`, English controls): ToS (website + data + content removal), EULA (desktop/Android binaries only; also `webapp/src-tauri/installer/EULA.rtf`), Privacy Policy, Disclaimer, `SECURITY.md`, `CODE_OF_CONDUCT.md`. `ACCESSIBILITY.md` (root: GitHub's accessibility community health file; VI mirror `docs/i18n/vi/ACCESSIBILITY.md`) is a statement, not a policy to accept — no `LEGAL_VERSION` bump when it changes, but keep its §3 (what works) and §6 (known limitations) in step with the code; barriers come in through `.github/ISSUE_TEMPLATE/accessibility.yml` (label `accessibility`) or contact@. When a policy changes, bump `LEGAL_VERSION` (`webapp/src/stores/prefs.ts`) and the dates in `about.legal.desc` (EN/VI) so the legal gate asks again. Contact addresses `contact@ / legal@ / privacy@ / security@ / takedown@poli0981.dev` are aliases of the maintainer's Google Workspace mailbox (not Cloudflare Email Routing; the Privacy Policy lists Google as the email processor). The maintainer's site https://poli0981.dev/ and channel hub https://poli0981.dev/links/ are linked from the About page (`DEV` in `lib/about.ts`).
|
||
|
||
## Known gotchas (cost a release each — don't relearn)
|
||
|
||
1. **`universal-apple-darwin` + `bundle_dmg.sh` is fragile** in headless CI. Use per-arch (`aarch64-apple-darwin`, `x86_64-apple-darwin`).
|
||
2. **`macos-13` is retired** from free GitHub-hosted runners. Cross-compile x86_64 from `macos-latest`.
|
||
3. **Tauri icons must exist before build.** `npm run tauri icon -- path/to/source.png`; the 1024×1024 source is gitignored, generated icons are committed.
|
||
4. **Tauri doesn't ship `.pkg`.** Built with `pkgbuild` after tauri-action and uploaded via `gh release upload`; the step uses `if: ${{ !cancelled() && … }}` so a DMG flake doesn't drop the PKG while a cancel still stops it.
|
||
5. **Vite v8 `base`** only accepts `'./'`, an absolute URL, or an empty string. Web uses `'/'` (BrowserRouter deep links), Tauri `'./'`.
|
||
6. **TypeScript 7 has no JS API (expected in 7.1), and typescript-eslint needs one** (peer `typescript <6.1`). So in `webapp/package.json` `typescript` is an alias of `@typescript/typescript6` (the 6.0 API for typescript-eslint and `scripts/i18n-edit.mjs`; bin `tsc6`) and TS 7 is `@typescript/native` (`npm:typescript@^7`), whose `tsc` is what `npm run build` / `npx tsc -b` run — `npx tsc --version` must say 7.x. Drop the alias once typescript-eslint accepts TS 7; never let Dependabot move the alias to 7. `baseUrl` is gone since TS 6: use `paths` alone.
|
||
7. **lucide-react v1+** removed brand icons (`Github`). Use `Library`, `Database`, etc.
|
||
8. **`bundle.windows.nsis.license` does NOT exist in `@tauri-apps/cli` 2.11** — use top-level `bundle.licenseFile`, and it must be **RTF** (WiX `.msi` requires it; NSIS auto-detects). File: `webapp/src-tauri/installer/EULA.rtf`.
|
||
9. **The bundle `identifier` must be alphanumeric** (`com.poli0981.freegamesitchio`): Android rejects `-`, Tauri's validator rejects `_`; one identifier serves every platform.
|
||
10. **Android signing: `zipalign` BEFORE `apksigner`**; never edit the generated Gradle; never lose the keystore. `versionCode = major*1e6 + minor*1e3 + patch` from `tauri.conf.json`. **`minSdkVersion` 30 (Android 11) is the install gate** — change it only in `tauri.conf.json` (`bundle.android.minSdkVersion`); no runtime OS check by design (reach ≈ 86.9 % mid-2026; see `docs/pc_spec.md`). The Android JDK setup must NOT use `cache: gradle` (gen/ doesn't exist at checkout).
|
||
11. **`webapp/dist/` is gitignored and `assets.directory` must be `./dist`.** In Sept 2026 a config-less deploy served the raw `webapp/` tree (source exposed).
|
||
12. **A Worker with static assets gets no `ctx.access`** — verify the Access JWT yourself (access.ts). **`workers_dev` and `preview_urls` must stay `false`**: those hostnames bypass Access.
|
||
13. **`public/_headers` applies only to responses served straight from assets**, not to responses the Worker builds (set headers in `worker/http.ts`). Responses fetched through `env.ASSETS` do carry them.
|
||
14. **With `run_worker_first` as an array, the asset layer's SPA fallback answers every miss before the Worker** — a stale hashed chunk got `index.html` + the year-long immutable cache. Hence `not_found_handling: "none"` + `spa.ts`.
|
||
15. **Never share an in-flight promise across requests in a Worker** (cache only resolved values): a pending subrequest dies with the request that started it (client disconnect) and every later awaiter hangs.
|
||
16. **itch.io cover URLs are signed per size**, so you can't rewrite `/original/` to a smaller variant — resizing goes through Image Transformations (`cf.image`) and R2.
|
||
17. **GitHub Actions concurrency keeps at most one *pending* run per group by default** and cancels the older pending one — use `queue: max` where a queued run carries unique input. A called workflow must not request its caller's group (deadlock → cancelled). **A queued run checks out the SHA from when it was triggered** (`workflow_dispatch`: the branch head at dispatch time), so runs waiting behind each other read the same state unless the job checks out `ref: ${{ github.ref }}` (refresh.yml's scan does).
|
||
18. **Pushes made with `GITHUB_TOKEN` trigger no workflows** (no ingest loop), and GitHub never starts `check_run` workflows for them (hence the scheduled `deploy-status.yml`). Workers Builds posts its check run only when a build ends.
|
||
19. **`wrangler types` reads `.dev.vars`** — `npm run cf:types` points it at the empty `types.env` so the committed `worker-configuration.d.ts` never depends on local secrets. Don't use `secrets.required` until every environment has the secrets (deploys fail without them).
|
||
20. **D1 migrations are applied by hand** (`npx wrangler d1 migrations apply freeitchgames --remote`), never in the build — and before deploying code that needs them. Production `freeitchgames` was created by command (APAC); its `database_id` is in `wrangler.jsonc` (not a secret). **R2 must be enabled on the account** (dashboard, payment method) before `wrangler r2 bucket create`; a deploy whose bucket is missing fails.
|
||
21. **React Router 8 wraps navigations in `startTransition` by default**: `useSearchParams` then returns the previous URL until the transition commits, so URL-driven state (the Games filters) overwrote itself on quick clicks. Both routers in `components/app-router.tsx` pass `useTransitions={false}`.
|
||
22. **`window.confirm` / `alert` / `prompt` do nothing in the macOS app** (wry's WebKit answers "Cancel"). Use in-app dialogs (`components/nsfw-dialog.tsx` + `requestNsfw()` in `lib/nsfw.ts`).
|
||
23. **The design tokens are hex: write `var(--x)`, never `hsl(var(--x))`.** The invalid colour painted the Recharts hover band black and left tooltips transparent (fixed in 4.1.0).
|
||
24. **Recharts 3 makes charts focusable** (`accessibilityLayer`, plus `tabindex` on inner `<g>`s): a mouse click focused the SVG, Chromium drew its black focus ring and the tooltip stuck on the first item. `ChartCard` keeps mouse clicks from moving focus; keyboard focus keeps the `--ring` outline.
|
||
25. **The verification gate fails open, and its off switch is deleting `GATE_SECRET`** (`BOT_GATE` changed in the dashboard is undone by the next deploy). It also needs `GATE_BOT_TOKEN` + the zone Transform Rule — without them Google would get the challenge; a client that only claims to be a crawler gets a 503, never the noindex page. Don't gate `/data`: every app version reads it — `/data/pack/*` (4.2+), and the legacy JSON (4.0–4.1.1, Tauri origins only; browsers get 410). Keep those legacy files in `dist/`: the Worker also reads `urls.json` and `deleted_games.json` through `env.ASSETS`.
|
||
26. **A negative `run_worker_first` pattern (`!/data/pack/*`) sends a missing file straight to the asset worker**, whose 404 still gets the `_headers` rules — for a pack, the year-long `immutable`, so a browser would cache the 404 of a name that can come back (rollback). List the Worker-first files positively; pack misses then reach `spa.ts` (`no-store`).
|
||
27. **A `desc` pack matches its games by row, so only within one build.** Load the `desc` named in the same `db` result; on a 404 refetch `db` (`useDescriptions.ts`), never re-resolve a desc file against a newer manifest for games already on screen.
|
||
28. **In Chromium 121+, `scrollbar-color` or `scrollbar-width` on an element switches its `::-webkit-scrollbar` styling off.** Keep the standard properties Firefox-only (`@supports not selector(::-webkit-scrollbar)`), except `scrollbar-width: none` for hiding.
|
||
29. **Pack names are content hashes: the build must stay deterministic** (column and dictionary order, zlib level 9 — a Node/zlib upgrade only re-downloads once), **and no hash may reach the JS bundle** (data-only deploys keep JS byte-identical; the manifest is fetched, never imported). `_headers` joins a header set by two matching rules with a comma — give each path one `Cache-Control`.
|
||
30. **itch.io shows the `Published` and `Updated` info rows only for a few weeks** after the event (every `Published` date in a 250-page sample of Oct 2026 was under four weeks old); an explicit `Release date` row stays. That is why `release_date` is N/A for most older games, and why a `--full` re-scrape keeps a known `release_date` / `thumbnail` instead of writing N/A (`KEEP_KNOWN_FIELDS` in `refresh.py`).
|
||
31. **Tauri crates and their npm packages must share major.minor** (`tauri` ↔ `@tauri-apps/api`, `tauri-plugin-x` ↔ `@tauri-apps/plugin-x`): `tauri build` refuses to build otherwise, and only the release workflows run it — every v4.3.0 release build first failed on `tauri-plugin-opener` 2.6.0 vs `@tauri-apps/plugin-opener` 2.7.0 after Dependabot bumped the npm side alone (#119). Webapp CI's `npm run check:tauri` (`webapp/scripts/check-tauri-versions.mjs`, lock files only) now fails such a PR; Dependabot's cargo updates run weekly on Mondays like npm so both groups arrive together. Fix a one-sided bump with `cargo update -p <crate>` in `webapp/src-tauri/`.
|
||
32. **Recharts hides category labels silently and styles legends its own way.** The default `interval="preserveEnd"` keeps a label only if its line box (font size × 1.5) plus `minTickGap` (5) fits its row, and the first one only if half of it fits above: the 15-row Genres chart showed 7 labels, without the two largest genres — category axes set `interval={0}`. A label wider than the axis wraps onto two lines, measured at the 16px body size unless `style` carries a size with a unit (`singleLineTick()` in `chart-theme.ts`). `<Legend>` sorts alphabetically (`itemSorter: 'value'`) and colours each label like its slice (amber/green text on white fails contrast) — pies spread `LEGEND_BASE`.
|
||
33. **Recharts' ResponsiveContainer unmounts its chart when it measures 0×0** (`display: none`, e.g. Radix's `hidden` panel): a hidden tab must keep its size. The Charts page force-mounts visited tabs and hides the inactive ones with `absolute inset-x-0 top-0 h-0 overflow-hidden invisible` + `inert` (no scroll height, no focus); switching back otherwise rebuilt every chart and replayed its animation. Its panel elements are memoised, so a tab switch re-renders only the tab bar (a re-render restarts Recharts tweens).
|
||
34. **The inline theme script's sha256 is in three CSPs**: `public/_headers`, `src-tauri/tauri.conf.json` and `THEME_SCRIPT_HASH` in `worker/admin.ts`. Change the script and `scripts/check-csp-hashes.test.mjs` fails with the hash to paste. A wrong hash only blocks the script (the old light flash). Keep `%` and `import(` out of it (Vite rewrites them in HTML).
|
||
35. **`scrollbar-gutter: stable` vs dialogs**: fixed overlays stop at the reserved gutter, and react-remove-scroll-bar adds the scrollbar width as body margin on top. Hence `html:not(:has(> body[data-scroll-locked]))` — the gutter is released while a dialog locks scrolling, which is then exactly the old behaviour.
|
||
36. **itch.io collection JSON** (`/c/<id>/<slug>?format=json&page=N`): `&page=1` redirects to the HTML page (page 1 has no `page`); a wrong or missing slug redirects to the canonical URL **without** the query string (re-request with the slug from `Location`); an unknown id answers 500; link attributes come in any order; past the last page `num_items` is 0. Zero cells while `num_items > 0` means changed markup: `collections.ts` reports it (422) and adds nothing.
|
||
37. **D1 counts every statement of a batch** towards its 1,000 queries per invocation: bind URL lists in `IN (…)` slices of ≤ 90 (100 bound parameters at most), as `setStatus` / `reconcile` / `markIngested` do, never one statement per URL.
|
||
38. **npm 11.20+ (Node 26) gates dependency install scripts on `allowScripts` in `webapp/package.json`.** esbuild and workerd are approved by name (their postinstall only checks the platform binary). A new dependency with an install script shows up in `npm install-scripts ls`; review it, then `npm install-scripts approve <pkg> --no-allow-scripts-pin` (name-only, so Dependabot bumps stay covered).
|
||
39. **Dependabot's pip updater reads `.python-version` and aborts on a Python it doesn't support** (3.10–3.14 as of Oct 2026): a `3.15` there stopped pip version updates and the pip dependency graph ("Dependabot detected the following Python requirement … '3.15.*'"). So the CI pin lives in `.github/python-version`, which only `setup-python`'s `python-version-file` reads (python-ci, refresh ×2, update); move it back to `.python-version` once Dependabot lists 3.15.
|
||
40. **The Python requirements are hash-locked pip-tools compiles** — `requirements.txt` ← `requirements.in`, `requirements-dev.txt` ← `requirements-dev.in` (`-c requirements.txt`) — installed everywhere with `--require-hashes --only-binary :all:`, so a new release of any dependency, indirect ones included, can't reach the jobs that push to `main`, and no sdist build script runs. After editing a `.in`, recompile with **pip-tools on Python 3.14**: `pip-compile --generate-hashes --allow-unsafe --strip-extras requirements.in`, then the same for `requirements-dev.in`. Dependabot takes the Python version from the `# … autogenerated by pip-compile with Python X.Y` header and supports ≤ 3.14 (gotcha 39); the hashes cover every file of a release, so CI's 3.15 still finds its cp315 wheels. `colorama` is listed in `requirements-dev.in` because pytest needs it on Windows only: a lock compiled on Linux (Dependabot's) would otherwise leave it out and the hash-checked local install would fail. Keep those pip commands in a block scalar (`run: |`): in a plain YAML scalar the `: ` of `:all: -r` makes the whole workflow file invalid — GitHub then records a 0-job failed push run named after the file, and a PR simply shows no check for that workflow.
|
||
41. **SonarQube Cloud analyses in CI** (`sonarcloud.yml` + `sonar-project.properties`; project `poli0981_free-games-itchio-list`, public). Automatic Analysis is switched off on sonarcloud.io — switched back on, it makes the CI scan fail. PRs get a "SonarCloud Code Analysis" check from the SonarQube Cloud app; Dependabot and fork PRs get no `SONAR_TOKEN`, so their analysis is skipped with a notice (a Dependabot secret of that name would enable it). Tests run in the `coverage` job without the token; besides the `token` job's presence check, only the `scan` step gets it, in a job that runs no project code. Binary images and the D1 migrations (SQLite, claimed by the Oracle PL/SQL analyzer, which then warned on every run) are excluded from the analysis. Coverage: `pytest --cov` (`[tool.coverage.run]` in `pyproject.toml`) and `npm run test:coverage` (`test.coverage` in `vite.config.ts`; the lcov reporter's `projectRoot` is the repo root, so the scanner resolves the paths); `@vitest/coverage-v8` needs vitest's exact version (Dependabot group `vitest`). The built-in "Sonar way" gate fails a PR under 80 % coverage on new code (from 20 new lines up), so code without unit tests by design is in `sonar.coverage.exclusions` — keep that list in step with what the tests reach. `sonar.projectVersion` comes from `webapp/package.json`: on `main`, "new code" (previous version) starts over at each version bump. `sonar.python.version` stays 3.14 until SonarQube Cloud's analyzer supports 3.15 (it warns "No explicit support for version 3.15"). The free plan keeps only `main` and PRs: a manual run on another branch uploads a branch analysis nobody can open ("not allowed to access data from non main branches"). Mark a false positive on the issue itself. Two of its suggested fixes are wrong here: S2871 wants `localeCompare` in `.sort()`, but chunk names, migration files and `unblocked.json` keep code-unit order (= Python's `sorted()`) — pass `compareCodeUnits` (`worker/text.ts`); S8906 wants `select_one('div.a.b')` for bs4's `class_=[a, b]`, which means *either* class — write a selector list `div.a, div.b`. Trim trailing characters with `trimEnd()` (`worker/text.ts`), not `/x+$/` (S8786: quadratic backtracking on a long run). Known false positive: css:S8776 on Tailwind v4's `@utility … { &::-webkit-scrollbar … }` (the `&` is required for variants).
|
||
42. **miniflare (wrangler's local runtime) pins `undici` exactly, so Dependabot can't fix an undici alert** until a wrangler release moves the pin. The stopgap is `overrides.undici` in `webapp/package.json` (4.4.3–4.5.0 carried `^7.30.1` over wrangler 4.152's 7.29.1; dropped with wrangler 4.156, whose miniflare pins 7.30.1). Only miniflare uses undici; nothing of it reaches the site, the Worker or the apps. Remove such an override once `npm ls undici` without it shows the fixed version, and before miniflare moves to a new undici major (the override would hold it back).
|
||
43. **The Workers rate-limit binding counts per machine, not per data centre.** Measured on production right after the 4.5.0 deploy (CDG): 1,000 requests in 53 s, each `curl` on a new connection, against `RL_SITE` 120/60 s → **no 429 at all**; 200 requests on **one** keep-alive connection → 93 × 200, then 429s. A data centre has many machines, each counts locally and syncs late, so a client that keeps one connection (browsers over HTTP/2, scripts with keep-alive) is held to the limit, while one that opens a new connection per request is spread across machines and barely counted. The zone's WAF rule counts across the data centre: it is the limit that holds against floods. Test the Worker limits over one connection (`curl URL URL …` in one invocation, or `--next`), never with `xargs curl`; under `wrangler dev` (miniflare: one process) every request counts. The WAF rule (300 / 10 s, set 2026-10-27) needs > 30 requests a second, which `xargs curl` doesn't reach from a home line: use the runbook's Node `fetch` one-liner (800 requests in 4.9 s → 111 × 200, 201 × Worker 429, 488 × WAF 1015).
|
||
|
||
44. **Tailwind v4's `outline-hidden` adds a visible 2 px outline in forced-colours mode.** On a focus variant that is the point (inputs keep a system-coloured focus ring in Windows contrast themes); used unconditionally on a container (`<main tabIndex={-1}>`, the games list, popovers) it drew a permanent frame there. Containers use `outline-none`, fields `focus-visible:outline-hidden`.
|
||
45. **Vitest stubs CSS imports — `./index.css?raw` is an empty string in tests.** A test that reads a stylesheet lives in `webapp/scripts/*.test.mjs` and uses `node:fs` (`check-contrast.test.mjs`; `src/` has no Node types).
|
||
46. **Radix FocusScope gives focus back to the trigger in a `setTimeout` after unmount.** A link inside a menu or sheet that navigates would have focus pulled back to the menu button, away from the new page's h1: `SheetContent` / `PopoverContent` call `e.preventDefault()` in `onCloseAutoFocus` when the path changed. A modal opened without a Radix trigger (the 18+ dialog) drops focus to `<body>` unless `onCloseAutoFocus` focuses something. Radix 1.3.1 sets no `aria-modal` (it hides the rest with `aria-hidden`).
|
||
47. **Name Recharts charts with `aria-labelledby`, not their `title` prop** (`title` also shows a native browser tooltip over the whole chart); `Treemap`'s props type has no `role`. A custom tooltip loses Recharts' live region, and the tooltip box is `visibility: hidden` while inactive, so announcements go through the card (`ChartAnnounce`).
|
||
48. **The tool permissions refuse loading third-party code into the preview page** (an axe-core `<script>` from a CDN). Check pages with the browser's accessibility tree and DOM queries instead; Lighthouse / axe DevTools are for the owner's own browser.
|
||
49. **Qodana takes one linter per Qodana Cloud project**, hence three projects (`itch-typescript` wwgZ1, `itch-python` GgGnX, `itch-rust` P2Rd7; team "My first team", public) and three secrets (`QODANA_TOKEN_TS_JS` / `_PYTHON` / `_RUST`). All three linters need an **Ultimate** licence (a trial in Nov 2026; the Rust linter is Early Access and its image carries its own evaluation licence, to 2027-01-10): when it ends the scans fail on the licence — renew, or delete `qodana.yml` and the three `qodana.yaml`. What the first runs cost: (a) **native `qodana-python` finds no interpreter by itself** (`SDKs.json` empty), and then calls every `X | None` an invalid type hint and a dataclass a class without `__init__` — the job builds `.venv` and passes `QODANA_PYTHON_PATH`; (b) **the `qodana-rust` image has no C linker and none of Tauri's system libraries, and its container runs as the runner's user**, who can't write the image's `CARGO_HOME` or `apt-get install` (`cargo metadata` / `cargo check` fail → "Cargo project loading" sanity problem, analysis degraded, `run` unresolved) — the job passes `-u root -e CARGO_HOME=/data/cache/cargo`, `webapp/src-tauri/qodana.yaml` installs the packages in `bootstrap`, and a stand-in `webapp/dist/index.html` lets `tauri::generate_context!()` expand; (c) `--project-dir <subdir> --repository-root .` is how a subdirectory project keeps its git data; (d) the default cache keys are the same for every job of a run — each job sets its own. `gh run view --log` returns nothing until the whole run has finished; the report (SARIF + IDE logs) is the `qodana-report-*` artifact, a zip inside the artifact. Inspections switched off on purpose are listed with their reason in `webapp/qodana.yaml` (throw-to-own-catch in Worker handlers, kept regex escapes, non-ASCII keys); deliberate uses of deprecated DOM APIs carry `// noinspection JSDeprecatedSymbols`. Recharts' `<Cell>` is deprecated (gone in 4.0): a pie slice's colour is the `fill` of its data entry (`paletteFills` in `components/charts/palette.ts`).
|
||
|
||
## Safe-edit rules
|
||
|
||
- Don't write `data_game/` from new code paths directly — go through a patch + `apply_patch.py` (or the Worker's `repo.ts`), so concurrent writers never clobber each other and `validate.py` gates every commit.
|
||
- Don't commit `webapp/dist/`, `webapp/src-tauri/target/`, `webapp/src-tauri/gen/`, `webapp/src-tauri/icon-source.png`, `.dev.vars`, `username.txt`, `assets/qr/`. `Cargo.lock` **is** committed (since v4).
|
||
- Never commit secrets (GitHub App key, admin emails, Turnstile secret, tokens). Data workflows push with the job's `GITHUB_TOKEN`; the Worker writes as the GitHub App.
|
||
- When adding an npm dep, also add it to `webapp/src/lib/about.ts` so the About page lists it. A Python dep goes into `requirements*.in`, then recompile the locks (gotcha 40).
|
||
|
||
## Current state (Nov 2026)
|
||
|
||
**Maintenance mode since v4.6.1 (2026-11-11) — the owner's decision.** The project is stable and feature-complete. Work is limited to security fixes (CVEs), runtime / dependency updates (the weekly sweep) and repairs when the site or the data pipeline breaks; **no new features, redesigns or refactors**. Issues and PRs are accepted for the data only (add, fix, remove a game — README → Project status, CONTRIBUTING); accessibility barriers and security reports keep their channels (kept because ACCESSIBILITY.md §7 and SECURITY.md promise them). Feature requests and code PRs are closed without review. Don't propose features in this repo.
|
||
|
||
**v4.6.2** (2026-11-16, PR #195): dependency-only release — the 2026-11-16 sweep below. Released because wry 0.57.2 fixes Android crashes and a memory leak, macOS panics and a Windows focus issue, which reach the apps only through a release (web-only updates redeploy on merge and need none). The owner wants a deps-only release's notes short: one "Update deps:" list (changelog + GitHub release). README count 6,000+ (6,069 games that day). No policy change (`LEGAL_VERSION` unchanged).
|
||
|
||
**v4.6.1** (2026-11-11): maintenance mode + Qodana. Docs (EN + VI): "Project status" in the READMEs, CONTRIBUTING rewritten around the data-only scope, SECURITY note, runbook §2 step 5 (Qodana) and §4. Templates: `bug_report.yml` is now "Report a problem"; `feature_request.yml`, `feedback.yml`, `general_question.yml` and the PR templates `bug_fix.md`, `new_features.md`, `documentation.md` are deleted; the issue chooser links Discussions. README badges (status, Python CI, SonarQube quality gate + coverage, Qodana, WCAG target) and the game count (5,900+; 5,907 games that day). `qodana.yml` + three `qodana.yaml` (gotcha 49). Code changes are only the first Qodana findings: pie charts without `<Cell>`, `ElementRef` → `ComponentRef`, `FormEvent` → `SubmitEvent`, an awaited `res.json()`, three Python nits, the unused `serde_json` crate dropped from the Tauri shell. No policy change (`LEGAL_VERSION` unchanged).
|
||
|
||
**v4.6.0** (2026-11-09): accessibility. Root `ACCESSIBILITY.md` + VI mirror (the owner had tested the website and the Windows app with the keyboard only and Windows Narrator; this release fixed what a WCAG 2.2 AA review found; after it, the owner's VoiceOver test on a real iPhone 14 Pro in Safari, Chrome and Brave passed), the `accessibility.yml` issue form and labels `accessibility` / `feedback`, a default `.github/pull_request_template.md` (GitHub's community profile ignores `PULL_REQUEST_TEMPLATE/`). Code: focus management (route h1, Back to the used link, dialogs), the "/" shortcut switch, header search options / count, h2 card titles, `lang` marking, screen-reader text for counts / loading / new tabs, contrast tokens + `check-contrast.test.mjs`, forced-colours CSS, chart names / arrow-key announcements / data tables. Docs sync: dead `@freeitchgames.win` addresses in `data_game/LICENSE.md`, stale dates / VI revision notes / library versions, `check:tauri` in the check lists, the About page's removal link, `remove_game.yml` → label `delete-game`. Not done by choice: desktop zoom (`zoomHotkeysEnabled` off, `maximizable: false`, min 900×600 — a known limitation in the statement). No policy change (`LEGAL_VERSION` unchanged). Ships the two deps sweeps below.
|
||
|
||
**Deps sweeps since v4.5.0** (11-02 and 11-09 shipped in 4.6.0 — the catalog had 5,818 games on 2026-11-09 —, 11-16 in 4.6.2). Each combines the week's Dependabot PRs plus what Dependabot didn't open (its 5-PR limit, pip / actions / cargo), with the 3-day cooldown applied by hand to every ecosystem (cargo and actions have none in `dependabot.yml`):
|
||
- 2026-11-02, PR #176: Dependabot #171–#175 (react-router 8.5.1, TanStack Query 5.105.0, lucide-react 1.64.0, knip 6.44.0, typescript-eslint 8.72.1, wrangler 4.156.0 + regenerated types), jose 6.2.14, `overrides.undici` dropped (gotcha 42), pytest 9.2.0 / Ruff 0.17.3, sonarqube-scan-action 8.4.0, `cargo update`.
|
||
- 2026-11-09, PR #182: Dependabot #177–#181 (TanStack Query 5.106.1, vitest 5.1.0, vite 8.3.6, @vitejs/plugin-react 6.2.1, Tailwind 4.3.4, eslint 10.14.0, knip 6.44.1, wrangler 4.158.0 + regenerated types, jose 6.2.15, globals 17.14.0), lucide-react 1.69.0, soupsieve 4.0.1 (a major whose breaking changes only touch `<template>` / `<iframe>` contents), coverage 7.17.2 / Ruff 0.17.4, setup-java 6.2.0, sonarqube-scan-action 8.4.1, `cargo update` (wasm-bindgen 0.2.130 family …).
|
||
- 2026-11-16, PR #195: Dependabot #190–#194 (TanStack Query 5.107.1, @tauri-apps/plugin-opener 2.7.3, Tailwind 4.3.5, @vitejs/plugin-react 6.3.0, knip 6.45.1, typescript-eslint 8.72.2, vite 8.3.8, wrangler 4.161.0 + regenerated types, jose 6.2.17, lucide-react 1.72.0), zustand 5.1.0 (`persist` still hydrates sync storage synchronously; `rehydrate()` now returns a `Promise`), soupsieve 4.1 / coverage 7.17.3 / Ruff 0.17.5, `cargo update` (wry 0.57.2 — it pulls `dom_query` 0.29 next to tauri-utils' 0.28 —, tauri-plugin-opener 2.7.3; brotli, bumpalo, cc, ctor, jiff, mio held at older versions by the cooldown); transitive npm bumps checked too (tapable 2.3.4 held at 2.3.3 in the lock; `npm audit signatures` all verified). Held for the next sweep: eslint 10.15.0, eslint-plugin-react-refresh 0.5.8, lucide-react 1.73.0, wrangler 4.162.0, soupsieve 4.1.1, sonarqube-scan-action 8.5.0, setup-java 6.3.0. Rust stable 1.100.0 (2026-11-12) is what the release builds now get (`toolchain: stable`).
|
||
|
||
Node 26.13.1 (LTS; the 2026-11-11 security release — http2 / TLS server fixes) with its npm 11.20.0 is current: CI and Workers Builds follow `.node-version` / `NODE_VERSION` `26` by themselves; only local installs need upgrading.
|
||
|
||
`npm audit`: 9 moderate, all minimatch < 11.0.1 (GHSA-g3hv-75wg-m249) under eslint / typescript-eslint — dev-only, no fix until they accept minimatch 11. `cargo audit`: 0 vulnerabilities, 5 upstream warnings. Security settings unchanged: CodeQL default setup not configured, Dependabot security updates off; 0 open alerts, SonarQube Cloud 0 issues.
|
||
|
||
**v4.5.0** (2026-10-27): rate limits for the whole site — `worker/ratelimit.ts` + `RL_SITE` / `RL_IMG` / `RL_ADMIN`, every namespace moved to 8801–8806 / staging 8901–8906 (account-wide ids; 1001 / 1003 were shared with other Workers), the 429 page, admin wording, `docs/operations.md` §1 step 15 (the WAF rule: `not cf.client.bot`, not `/assets/` or `/data/pack/`, 300 requests / 10 s, block 10 s — set by the owner in the dashboard). No policy change (`LEGAL_VERSION` unchanged; SECURITY.md EN + VI describe the limits).
|
||
|
||
**v4.4.3** (2026-10-26): dependency and security updates only — the week's npm updates (Dependabot #159–#163 in one PR, plus lucide-react / Zustand; `knip` kept at 6.42.1 by the 3-day cooldown), wrangler 4.152 with regenerated `worker-configuration.d.ts` (a wrangler bump fails `cf:types -- --check` until it is regenerated — why Dependabot's build-tooling PR failed), `overrides.undici` (gotcha 42), zod 4.6.7, `cargo update` (tauri 2.12.5, tokio 1.54.1; `cargo audit`: 0 vulnerabilities, 5 upstream warnings — glib, zbus family, proc-macro-error), certifi / coverage / Ruff via pip-tools on 3.14, `actions/setup-python` 7.1.0. Repository security state that day: CodeQL default setup **not configured** (last analyses May 2026, 0 open alerts), Dependabot **security updates disabled** in Settings (version updates run), secret scanning + push protection on, SonarQube Cloud gate OK with 0 issues.
|
||
|
||
**SonarQube Cloud CI** (2026-10-22, shipped in 4.4.3's notes): CI-based analysis with test coverage replaced Automatic Analysis — `sonarcloud.yml`, `sonar-project.properties` (tests marked as tests, coverage exclusions, `.sonarcloud.properties` removed), `pytest-cov` + `@vitest/coverage-v8` (gotcha 41).
|
||
|
||
**Toolchain** (2026-10-22, #149 + #154, shipped in 4.4.2): TypeScript 7.0.2 (`@typescript/native`; `typescript` = the 6.0 API alias, gotcha 6), Node 26 (`webapp/.node-version`, `engines >=26`, `@types/node` 26, Workers Builds `NODE_VERSION` raised to 26 by the owner in the dashboard — `docs/operations.md` §1 step 1), Python 3.15 (`.github/python-version` — gotcha 39, ruff `py315`, `urllib3` declared since `scraper.py` imports it). No source change was needed: TS 7 reports 0 errors, the suites pass on Node 26.12 and Python 3.15.1.
|
||
|
||
**v4.4.2** (2026-10-22): SonarQube Cloud cleanup — the scan's 57 issues: 45 fixed in code, the 11 on the saved itch.io test pages excluded by `.sonarcloud.properties`, css:S8776 (Tailwind `@utility`) left for the owner to mark a false positive (gotcha 41). Worker paths and the gate cookie's padding are trimmed by `trimEnd()` (a `/\/+$/` on the public 404 path was quadratic), the feed / collection parsers have no backtracking regexes left, `compareCodeUnits` makes the code-unit sorts explicit, the NSFW notices are one selector list (each tested). Supply chain: hash-locked Python deps + wheels only in every workflow (gotcha 40), `npm ci --ignore-scripts` in CI and the release builds, `npm run cf:check` instead of `npx wrangler`. `scripts/i18n-edit.mjs` reads its operations from stdin (`node scripts/i18n-edit.mjs < ops.json`). Ships the toolchain upgrade and Dependabot #141–#148, #150, #152, #153.
|
||
|
||
**v4.4.1** (2026-10-16): the browser extension and the Telegram bot are retired — both repos (`poli0981/itch-f2p-extension`, `poli0981/telegram-scraper-bot`) were already public archives; an Announcements discussion says so. Removed: `POST /api/ingest` (`ingest.ts`, `requireService`, `RL_INGEST`, `ACCESS_AUD_INGEST`, the idempotency store — it never received a game: 0 `ext` rows in production D1), the admin's "Browser extension" auto-approve source and "Extension" label, the About page's Telegram numeric-ID note. The policies (Privacy, ToS, EULA + installer RTF, Disclaimer, Security; EN + VI) only lost the extension / ingest wording: "Last updated" 2026-10-16, `LEGAL_VERSION` deliberately unchanged (nothing new to accept, ToS §14). Unused repo secrets `GH_TOKEN`, `TELEGRAM_BOT_TOKEN`, `DISCUSSION_*` deleted. Ships Dependabot #131 (vite 8.3.4), #135 (tauri crates), #136 (serde_json), #137 (ruff).
|
||
|
||
**v4.4.0** (2026-10-13, PR #133): admin automation. Add page: any text / file / itch.io collection → **Check** (dry run) → batches of 200 → per-game ingest tracking (`/api/admin/track`, from the new `scripts/state/ingest_outcomes.json`). The cron runs the maintainer's automatic rules in one `[auto]` commit: auto-approve (on by default, `rss` + `form`, unflagged, after 24 h, only items found after the `since` baseline, ≤ 100 a run) and **followed collections** (≤ 30 pages a run; never-seen unflagged free games queued directly, removed or flagged ones to review). Bulk remove / edit ≤ 100 in one commit (preset reasons translated on the Removed page), New games, Activity, Settings; queue batches of 200. D1 migration `0002_automation.sql` applied to production before the merge. Policies rewritten for approval by rules (ToS §5, Privacy §2.3/§2.4, Disclaimer §4, EN + VI; `LEGAL_VERSION` 2026-10-13). Web: the theme button shows the theme in use (a v4 regression), a pre-paint theme script (no dark flash; hash in three CSPs), Charts tabs kept mounted (switching back reuses every chart, no replayed animation), 300 ms entrances, scrollbar gutter. Also ships #130 (2026-10-11, the Charts page on phones: one-row tab bar, 12 px legends / 11 px ticks, horizontal Languages / Avg session bars, all 15 Genre labels, data-order pie legends, translated removal reasons, locale axis numbers).
|
||
|
||
**v4.3.0** (2026-10-10, PR #117; released the same day): itch.io's newer info rows in the catalog — `ai_disclosure` / `ai_content` (generative-AI disclosure) and `accessibility` — on the game page, as `/games` filters (`ai`, `access`) and as charts (Overview: "Generative AI"; Discovery: "Accessibility Features", "What AI Was Used For"); `description` = the creator's tagline. The whole catalog was re-scraped the same day with `force_update.yml` (6 × 600 + 10, ≈14.5 games/min; an itch.io outage ~05:20–06:50Z only cost retries): 3,236 of 3,251 games carry the new fields — No AI 53 %, AI Assisted 11 %, not disclosed 36 %; accessibility on 14 % (Subtitles 257, One button 119); the other 15 are 14 dead-page strikes awaiting confirmation and one password-protected page (`everium.itch.io/underthethorns`; it failed every check until #129 taught the refresh to strike such pages). Fixed on the way, same day: #118 (`force_update.yml` had never worked in v4 — it now dispatches `refresh.yml`, whose scan reads the latest main so queued batches rotate), #119 (`tauri-plugin-opener` 2.7: every first v4.3.0 release build had failed; the tag was moved before publishing), #120 (the 5 open Dependabot alerts; `check:tauri` CI guard; weekly cargo Dependabot), then Dependabot #122–#127.
|
||
|
||
**v4.2.0** (2026-10-03, PR #103): catalog packs + lazy descriptions (first visit ≈ 264 KB instead of ≈ 520 KB), compressed IndexedDB cache, `/data/*.json` only for apps 4.0–4.1.1 (410 + repository links for everyone else), `/img` cache fixes, themed scrollbars + "Hide scrollbars", policies (`LEGAL_VERSION` 2026-10-03: data reuse goes through the repository). Checked on the live site the same day: TLS 1.2/1.3 only, HSTS preload header (domain not on the preload list), Cloudflare compression already on (zstd/br), Smart Tiered Cache on but irrelevant here (`docs/operations.md` §1 step 12).
|
||
|
||
**v4.1.1** (2026-09-29): contact addresses moved to `@poli0981.dev` (Google Workspace), About page links poli0981.dev + /links/, `LEGAL_VERSION` 2026-09-29.
|
||
|
||
**v4.1.0** (2026-09-24): the Turnstile verification gate for the web app, readable chart tooltips, the changelog split into `changelog/`, a docs refresh, updated Privacy/ToS/EULA (`LEGAL_VERSION` 2026-09-24). The gate ships switched on in `wrangler.jsonc` but stays inert until `GATE_BOT_TOKEN`, the Transform Rule and `GATE_SECRET` are in place (`docs/operations.md` §1 step 14) — `curl -s https://freeitchgames.win/api/verify` says `"enabled"`.
|
||
|
||
**v4.0.0** (2026-09-18) moved the site to the Cloudflare Worker at https://freeitchgames.win (PRs #81, #82, #88). The whole chain is live: Access for `/admin`, the GitHub App (verified commits to `scripts/temp_link.json` → `update.yml`), Turnstile, `ADMIN_EMAILS`; D1 `freeitchgames`, R2 `freeitchgames-thumbs`, Image Transformations (source `img.itch.zone`); Web Analytics is injected by the zone. **Still open (owner):** the legal-text review items; add the new GPG key's public half on GitHub (until then new signatures show Unverified) and remove the revoked `03F965C2E2DB5C6B` there; delete the Zero Trust Access app "Ingest" and its service token (unused since 4.4.1).
|