README's link, the export-ignore beside CHANGELOG.md's, the watch-paths note in DEPLOYMENT.md, and a CLAUDE.md rule for where release notes go and how a release moves [Unreleased] into its own file. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
302 lines
16 KiB
Markdown
302 lines
16 KiB
Markdown
# Deployment
|
||
|
||
The site is a Cloudflare Worker serving a Vite build. Pushing to `main` is what
|
||
deploys — there is deliberately no deploy workflow in `.github/workflows/`.
|
||
|
||
---
|
||
|
||
## How it fits together
|
||
|
||
```
|
||
push to main
|
||
└─> Cloudflare Workers Builds (root directory: web)
|
||
npm ci → npm run build → wrangler deploy
|
||
│ └─> Worker "free-steam-games-list"
|
||
│ ├─ static assets from web/dist
|
||
│ ├─ /api/data/* → proxies GitHub raw
|
||
│ ├─ /img/* → proxies Valve's CDN
|
||
│ ├─ /api/human-check → Turnstile siteverify (web only)
|
||
│ ├─ /admin → Access-gated admin SPA (embedded)
|
||
│ └─ /api/ingest/* → Access service token
|
||
├─ vite build → web/dist (the public site)
|
||
└─ vite build -c admin/… → web/worker/generated/admin-bundle.ts
|
||
```
|
||
|
||
**The build runs on Node 26.** The Worker's dashboard sets the build variable
|
||
`NODE_VERSION=26` (Settings → Build → Variables and secrets); Cloudflare's
|
||
image would otherwise use its default, Node 24. `web/.node-version` says the
|
||
same, and CI and `web/shared/toolchain.test.ts` follow that file, but
|
||
Cloudflare does not document which directory it reads it from or which of the
|
||
two wins, so keep the variable and change both together. The first lines of a
|
||
build log show the Node actually used.
|
||
|
||
`web/wrangler.jsonc` is the only deployment config. Several fields in it are
|
||
load-bearing and easy to break:
|
||
|
||
- **`name` must stay `free-steam-games-list`.** That is the Worker holding the
|
||
`free-steam-games.win` custom domain. Deploying under any other name creates
|
||
a second, domainless Worker and leaves the live site on the old build, with
|
||
a completely green build log.
|
||
- **`assets.directory` must be `./dist`, not `.`.** Pointing it at the source
|
||
directory serves an unbuilt `index.html` that references `/src/main.tsx` —
|
||
a blank page — and publishes the whole source tree. This happened.
|
||
- **`triggers.crons` drives the reconciler.** Remove it and approvals never
|
||
leave the `approved` state: nothing else promotes a row to `committed`, and
|
||
nothing else ages out an approval the pipeline silently dropped. The queue
|
||
keeps working and the failure is invisible until someone opens /admin.
|
||
- **`vars.ACCESS_AUD_ADMIN` / `vars.ACCESS_AUD_INGEST` are per-route
|
||
allowlists.** Every Access application has its own AUD; add one without
|
||
listing it here and every request through it is refused with
|
||
`access: aud mismatch`. They are scoped by route on purpose — see
|
||
docs/ADMIN.md.
|
||
- **The admin SPA is part of the Worker script, not of `dist/`.**
|
||
`npm run build` runs `build:admin` after the site build, which writes the
|
||
gitignored `worker/generated/admin-bundle.ts` that `worker/index.ts` imports.
|
||
A `wrangler deploy` without that step fails to bundle, and that is the
|
||
intended failure: the alternative, serving the admin from `dist/`, would hand
|
||
its code to anyone, to the service worker's precache and to the packaged
|
||
apps. `scripts/verify-dist.mjs` fails the CI build if any admin code reaches
|
||
`dist/`. The bundle adds about 0.5 MB to the Worker upload.
|
||
|
||
## Data does not redeploy the site
|
||
|
||
The catalogue is proxied at `/api/data/*`, not built into `dist/`. A pipeline
|
||
commit to `data/**` therefore changes the site's content without any rebuild.
|
||
|
||
Keep it that way. If the data ever moves into the build output, the Worker's
|
||
build watch paths must include `data/**` — otherwise the site silently serves
|
||
a frozen dataset with no error anywhere, and a health check that reads the repo
|
||
rather than production will report green while it happens.
|
||
|
||
### The one exception: prerendered game pages
|
||
|
||
Every `/games/<appid>` page is prerendered on the web build from `data/` as it
|
||
stood at build time, so a crawler - which never accepts the terms and never
|
||
loads the catalogue - gets real content. Only the slow-changing fields are
|
||
baked in (name, description, artwork, studios, genre, platforms, tags); player
|
||
counts and reviews always come from `/api/data/*` in the browser.
|
||
|
||
That snapshot is therefore **as fresh as the last deploy**, and it should stay
|
||
that way:
|
||
|
||
- **The Workers Builds watch paths must exclude `data/**`** (dashboard →
|
||
Worker → Settings → Build → Build watch paths; the configuration is under
|
||
"Which commits deploy" below). Every bot data commit that deploys - several
|
||
a day - rebuilds and re-uploads all ~5,700 files (measured on 2026-10-02: a
|
||
4-file data commit re-uploaded 5,681 assets, because `kit.version.name` is a
|
||
timestamp) and gives every open tab a "Reload" prompt.
|
||
- A game added after the last deploy has no file. The host answers with the
|
||
SPA fallback and `lib/fallback-route.ts` renders it client-side, so it works;
|
||
it simply is not indexable until the next deploy.
|
||
- A game removed since the last deploy keeps its file until then. Once the live
|
||
catalogue loads, the page shows "Game not found" with `noindex`.
|
||
- `F2P_SKIP_GAME_PRERENDER=1 npm run build` skips them for quick local builds.
|
||
The Tauri builds never prerender them.
|
||
|
||
### Which commits deploy (build watch paths)
|
||
|
||
Since 2026-10-02 a commit that touches only documentation does not deploy -
|
||
but **data commits still did**: `3f9e8cff` ("Refresh store data
|
||
[2026-10-02]", four files under `data/`) deployed a new Worker version
|
||
(`712b4ab8`) at 16:54Z. The configuration below stops both. Skipping
|
||
everything outside `web/` is right, with ONE trap: **the web build reads the
|
||
legal documents.** `web/build/legal-versions.ts`
|
||
hashes them for the consent gate and `web/src/lib/server/markdown.ts`
|
||
renders them at `/legal/*`:
|
||
|
||
`LICENSE`, `LICENSE-DATA`, `docs/DISCLAIMER.md`, `docs/ToS.md`,
|
||
`docs/EULA.md`, `docs/PRIVACY_POLICY.md`, `docs/ACKNOWLEDGEMENTs.md`,
|
||
`docs/Contact.md`
|
||
|
||
A commit that changes only those - the ToS edit Phase B needs, for one - would
|
||
not reach the site, and the consent gate would keep showing the old text and
|
||
hash until the next code deploy. Either keep those files triggering a build,
|
||
or redeploy by hand after merging such a change. Excludes are applied BEFORE
|
||
includes, so an exclude like `docs/*` swallows them whatever the include list
|
||
says. A configuration that does both jobs:
|
||
|
||
- Include: `web/*`, `LICENSE`, `LICENSE-DATA`, `docs/DISCLAIMER.md`,
|
||
`docs/ToS.md`, `docs/EULA.md`, `docs/PRIVACY_POLICY.md`,
|
||
`docs/ACKNOWLEDGEMENTs.md`, `docs/Contact.md`
|
||
- Exclude: `web/src-tauri/*`, `web/README.md`
|
||
|
||
Everything not included - `data/**`, `scripts/**`, the other docs,
|
||
`CHANGELOG.md` and `changelog/`, `CLAUDE.md`, `README.md` - then never deploys. Check it once
|
||
by merging a commit that touches only `docs/plan/` (no build) and one that
|
||
touches only `docs/ToS.md` (a build), and by the next bot data commit: its
|
||
check runs on GitHub must have no `Workers Builds: free-steam-games-list`.
|
||
A check that IS there names a Version ID - that commit deployed.
|
||
|
||
## Verifying a deploy
|
||
|
||
```bash
|
||
curl -sI https://free-steam-games.win/ | head -3
|
||
curl -s https://free-steam-games.win/ | grep -o '<html lang="[a-z]*"'
|
||
curl -so /dev/null -w '%{http_code}\n' https://free-steam-games.win/api/data/data/index.json
|
||
curl -so /dev/null -w '%{http_code} %{size_download}b\n' https://free-steam-games.win/img/t/570/header.jpg
|
||
curl -so /dev/null -w '%{http_code}\n' https://free-steam-games.win/admin # expect 302 to Access
|
||
curl -so /dev/null -w '%{http_code}\n' https://free-steam-games.win/api/ingest/ping # expect 302 to Access
|
||
```
|
||
|
||
The image should come back around 3–4 KB: the Worker prefers Steam's small
|
||
capsule asset over the full header.
|
||
|
||
For the AVIF path, copy a game page's hero URL (`/img/d/<appid>/…?t=…`, the
|
||
`?t=` matters - it is part of the R2 key) and request it with
|
||
`-H "Accept: image/avif"`: once the cron has minted it the answer is
|
||
`image/avif` with `X-Img-Final: 1`; before that it is the JPEG with
|
||
`Cache-Control: public, max-age=86400` and no `X-Img-Final`.
|
||
|
||
The website itself loads that art from the bucket's custom domain instead
|
||
(`docs/SECURITY_SETUP.md` section 5): in a game page's Network panel the hero
|
||
is `image/avif` from `https://media.free-steam-games.win/img/v2/460q80/…`,
|
||
with `cf-cache-status: HIT` on a reload. A `/img/d/…` request right after it
|
||
means the media load failed and the fallback ran - expected for a game the
|
||
cron has not reached yet, a fault for one it has.
|
||
|
||
**A stale service worker will show you the old app after a deploy.** The site
|
||
is a PWA, so a browser that visited before the deploy serves its cached shell
|
||
until the service worker updates. If you are checking whether a deploy landed,
|
||
use a fresh profile or a hard reload — otherwise you will diagnose a
|
||
deployment problem that does not exist.
|
||
|
||
## Local development
|
||
|
||
```bash
|
||
cd web
|
||
npm ci
|
||
npm run dev # Vite + HMR; /api/* and /img/* proxy to the live Worker
|
||
npx wrangler dev # the real Worker against web/dist — required for worker/ changes
|
||
```
|
||
|
||
`npm run dev` cannot exercise `worker/` source. It proxies those paths to
|
||
production so the app has real data and images; changes under `web/worker/`
|
||
need `wrangler dev`.
|
||
|
||
Stop `wrangler dev` before running `npm ci`. On Windows the install deletes
|
||
`node_modules` and fails partway if a dev server holds files open.
|
||
|
||
## Checks
|
||
|
||
`.github/workflows/web-ci.yml` runs on pull requests and on pushes to `main`
|
||
touching `web/**`, `.github/workflows/**` or `sonar-project.properties`:
|
||
`npm ci --ignore-scripts`, the typechecks (app, Worker and tests, on
|
||
TypeScript 7; the admin's svelte-check on TypeScript 6), `npm test` (which
|
||
includes the toolchain-pin and data-write-group checks over the workflows),
|
||
the build, `scripts/verify-dist.mjs --web` over the
|
||
prerendered HTML, `npm run cf:check` (`wrangler deploy --dry-run`), and
|
||
`knip`. `python-tests.yml` runs the pipeline's pytest suite on changes under
|
||
`scripts/` and the requirements files, installing them exactly as the data jobs
|
||
do (`--require-hashes --only-binary :all:`).
|
||
|
||
It uses a plain `npm ci`, never `npm ci || npm install`. That fallback is what
|
||
hid a lockfile conflict which broke installs for a month. `--ignore-scripts`
|
||
keeps every dependency's install script from running with the job's token;
|
||
none of the three packages that have one needs it (the comment in
|
||
`web-ci.yml` says why), and no workflow runs `npx`, which can fetch a package
|
||
the lockfile never named.
|
||
|
||
CodeQL (`codeql.yml`) and SonarCloud (`sonarcloud.yml`) both look at pull
|
||
requests and `main`. Each posts a check on the pull request (`CodeQL`,
|
||
`SonarCloud Code Analysis`) about the issues the change brings in; the full list
|
||
for `main` lives in the repository's Security tab and on the SonarCloud
|
||
dashboard. `sonar-project.properties` at the root names the project and decides
|
||
what SonarCloud reads as code, what as tests, and what not at all (the dataset,
|
||
generated Android resources, the SQLite migrations); the scanner reads it from
|
||
the commit it analyses.
|
||
|
||
SonarCloud runs in CI, with the repository secret `SONAR_TOKEN` (a SonarCloud
|
||
token that may analyse the project). Its automatic analysis must stay off
|
||
(project Administration > Analysis Method): with both on, every CI scan fails.
|
||
Commits that touch only what the data jobs write (`data/`, `games/`, the
|
||
`.jsonl` queues and logs under `scripts/`) are not scanned. A pull request from
|
||
a fork or from Dependabot gets no repository secrets, so its scan job is
|
||
skipped; giving Dependabot its own `SONAR_TOKEN` (Settings > Secrets >
|
||
Dependabot) would have its pull requests analysed too.
|
||
|
||
Qodana (`qodana.yml`) runs JetBrains' inspections on pull requests and
|
||
`main` as well, one job per Qodana Cloud project: Python over `scripts/`,
|
||
JavaScript/TypeScript (with the Svelte plugin) over `web/`, Rust over
|
||
`web/src-tauri/`. Each project has its own token, kept in the repository
|
||
secrets `QODANA_TOKEN_PYTHON`, `QODANA_TOKEN_TS_JS` and `QODANA_TOKEN_RUST`
|
||
(Qodana Cloud > the project > its setup page). Its settings are the
|
||
`qodana.yaml` in that directory: a gate of zero problems, so a new finding
|
||
fails the job, and every inspection or path left out, with the reason.
|
||
Commits that touch only data or Markdown are not scanned, and a pull request
|
||
without the secrets (a fork, Dependabot) skips the scan. The results are on
|
||
Qodana Cloud, in the job's annotations and in its `qodana-report-*`
|
||
artifact. All three linters need a Qodana Ultimate licence; without one every
|
||
job fails, and `qodana.yml` and the three `qodana.yaml` files are what to
|
||
remove.
|
||
|
||
Before the scan the job runs both test suites with coverage: pytest with
|
||
`--cov` (configured in `pyproject.toml`, written to `coverage.xml`) and
|
||
`npm run test:coverage` in `web/` (configured in `vite.config.ts`, written to
|
||
`web/coverage/lcov.info`). SonarCloud's default "Sonar way" quality gate asks
|
||
for 80% coverage on new code (it ignores that condition below 20 new lines),
|
||
so a pull request that adds code its tests do not reach fails that check. To
|
||
see the figure locally, run
|
||
`npm run test:coverage` in `web/`, or
|
||
`python -m pytest scripts/tests --cov` after installing `requirements-test.txt`.
|
||
|
||
### A red "Workers Builds" check on a PR branch is usually not yours
|
||
|
||
Cloudflare posts its own check alongside those. On a pull-request branch it has
|
||
been seen **failing with zero duration** — `started_at` equal to `completed_at`,
|
||
so nothing was compiled and there is no log reachable from GitHub — while the
|
||
identical commits built and deployed from `main` minutes later. Measured on
|
||
PR #130 (2026-09-20), twice.
|
||
|
||
Before spending time on it, look at the GitHub Actions `check` job: it already
|
||
runs the same `npm run build`, `verify-dist.mjs` and `wrangler deploy
|
||
--dry-run`. If that is green, the build is sound and the Cloudflare check is
|
||
about the preview build, not your diff. The only place the real answer lives is
|
||
the build log in the Cloudflare dashboard, which the check links to.
|
||
|
||
## Rolling back
|
||
|
||
Roll back the Worker deployment in the Cloudflare dashboard
|
||
(*Workers → free-steam-games-list → Deployments*). Reverting the commit also
|
||
works but takes a full build cycle.
|
||
|
||
GitHub Pages is **not** a fallback. It now serves only a redirect to this
|
||
domain; the app cannot run there because it is built with `base: "/"`.
|
||
|
||
## Secrets
|
||
|
||
`wrangler.jsonc` holds no secrets — only the D1 `database_id`, which is an
|
||
account-scoped identifier and safe to commit. Anything sensitive goes through
|
||
`npx wrangler secret put`: the GitHub App's three (see [ADMIN.md](./ADMIN.md))
|
||
and `TURNSTILE_SECRET` for the human check (see
|
||
[SECURITY_SETUP.md](./SECURITY_SETUP.md) section 12). The Worker declares them
|
||
in `web/worker/env.d.ts`.
|
||
|
||
For `wrangler dev`, put local values in `web/.dev.vars` (gitignored);
|
||
`web/.dev.vars.example` holds Cloudflare's always-pass Turnstile test secret.
|
||
|
||
## Cost notes
|
||
|
||
**Images.** AVIF copies of the header art are minted by the cron into the R2
|
||
bucket `f2p-media` (`web/worker/lib/img-mint.ts`); `/img/*` only reads them, so
|
||
requests cannot spend anything - there is nothing for an allowlist to guard.
|
||
Images Paid bills $0.50 per 1,000 unique transformations after 5,000 a month
|
||
and has **no spend cap**; `IMG_TRANSFORM_MONTHLY_CAP` (a D1 counter) is the
|
||
cap. See SECURITY_SETUP.md section 5.
|
||
|
||
**The bucket must exist before any deploy that binds it.** `wrangler deploy`
|
||
fails with code 10085 otherwise (the old Worker stays live). It was created
|
||
with `npx wrangler r2 bucket create f2p-media`; keep its r2.dev URL disabled
|
||
and attach no custom domain - the Worker is the only reader.
|
||
|
||
**R2** stays inside the free tier: ~11k objects at ~3-18 KB, about 12 list
|
||
calls per cron tick only while minting, and reads only on an edge-cache miss.
|
||
|
||
**Shard compression is a dashboard setting.** Cloudflare does not compress
|
||
`application/x-ndjson` on its own, so the plain shards went out uncompressed
|
||
(~9 MiB per full load) until the Compression Rule `ndjson-shards` was added on
|
||
2026-10-02: Rules → Compression Rules, `http.request.uri.path.extension eq
|
||
"jsonl"` → Custom: Brotli, Gzip. It applies to Worker responses (data_001:
|
||
1,497,828 → 225,826 bytes). The packed `/api/data/p1/*.bin` files are
|
||
compressed before they are encrypted and are deliberately not `.jsonl`, so the
|
||
rule never touches them.
|