Files
poli0981andClaude Opus 5.5 d482013cf7 docs: name changelog/ where CHANGELOG.md is named
README's link, the export-ignore beside CHANGELOG.md's, the watch-paths
note in DEPLOYMENT.md, and a CLAUDE.md rule for where release notes go
and how a release moves [Unreleased] into its own file.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 09:02:17 +07:00

302 lines
16 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Deployment
The site is a Cloudflare Worker serving a Vite build. Pushing to `main` is what
deploys — there is deliberately no deploy workflow in `.github/workflows/`.
---
## How it fits together
```
push to main
└─> Cloudflare Workers Builds (root directory: web)
npm ci → npm run build → wrangler deploy
│ └─> Worker "free-steam-games-list"
│ ├─ static assets from web/dist
│ ├─ /api/data/* → proxies GitHub raw
│ ├─ /img/* → proxies Valve's CDN
│ ├─ /api/human-check → Turnstile siteverify (web only)
│ ├─ /admin → Access-gated admin SPA (embedded)
│ └─ /api/ingest/* → Access service token
├─ vite build → web/dist (the public site)
└─ vite build -c admin/… → web/worker/generated/admin-bundle.ts
```
**The build runs on Node 26.** The Worker's dashboard sets the build variable
`NODE_VERSION=26` (Settings → Build → Variables and secrets); Cloudflare's
image would otherwise use its default, Node 24. `web/.node-version` says the
same, and CI and `web/shared/toolchain.test.ts` follow that file, but
Cloudflare does not document which directory it reads it from or which of the
two wins, so keep the variable and change both together. The first lines of a
build log show the Node actually used.
`web/wrangler.jsonc` is the only deployment config. Several fields in it are
load-bearing and easy to break:
- **`name` must stay `free-steam-games-list`.** That is the Worker holding the
`free-steam-games.win` custom domain. Deploying under any other name creates
a second, domainless Worker and leaves the live site on the old build, with
a completely green build log.
- **`assets.directory` must be `./dist`, not `.`.** Pointing it at the source
directory serves an unbuilt `index.html` that references `/src/main.tsx` —
a blank page — and publishes the whole source tree. This happened.
- **`triggers.crons` drives the reconciler.** Remove it and approvals never
leave the `approved` state: nothing else promotes a row to `committed`, and
nothing else ages out an approval the pipeline silently dropped. The queue
keeps working and the failure is invisible until someone opens /admin.
- **`vars.ACCESS_AUD_ADMIN` / `vars.ACCESS_AUD_INGEST` are per-route
allowlists.** Every Access application has its own AUD; add one without
listing it here and every request through it is refused with
`access: aud mismatch`. They are scoped by route on purpose — see
docs/ADMIN.md.
- **The admin SPA is part of the Worker script, not of `dist/`.**
`npm run build` runs `build:admin` after the site build, which writes the
gitignored `worker/generated/admin-bundle.ts` that `worker/index.ts` imports.
A `wrangler deploy` without that step fails to bundle, and that is the
intended failure: the alternative, serving the admin from `dist/`, would hand
its code to anyone, to the service worker's precache and to the packaged
apps. `scripts/verify-dist.mjs` fails the CI build if any admin code reaches
`dist/`. The bundle adds about 0.5 MB to the Worker upload.
## Data does not redeploy the site
The catalogue is proxied at `/api/data/*`, not built into `dist/`. A pipeline
commit to `data/**` therefore changes the site's content without any rebuild.
Keep it that way. If the data ever moves into the build output, the Worker's
build watch paths must include `data/**` — otherwise the site silently serves
a frozen dataset with no error anywhere, and a health check that reads the repo
rather than production will report green while it happens.
### The one exception: prerendered game pages
Every `/games/<appid>` page is prerendered on the web build from `data/` as it
stood at build time, so a crawler - which never accepts the terms and never
loads the catalogue - gets real content. Only the slow-changing fields are
baked in (name, description, artwork, studios, genre, platforms, tags); player
counts and reviews always come from `/api/data/*` in the browser.
That snapshot is therefore **as fresh as the last deploy**, and it should stay
that way:
- **The Workers Builds watch paths must exclude `data/**`** (dashboard →
Worker → Settings → Build → Build watch paths; the configuration is under
"Which commits deploy" below). Every bot data commit that deploys - several
a day - rebuilds and re-uploads all ~5,700 files (measured on 2026-10-02: a
4-file data commit re-uploaded 5,681 assets, because `kit.version.name` is a
timestamp) and gives every open tab a "Reload" prompt.
- A game added after the last deploy has no file. The host answers with the
SPA fallback and `lib/fallback-route.ts` renders it client-side, so it works;
it simply is not indexable until the next deploy.
- A game removed since the last deploy keeps its file until then. Once the live
catalogue loads, the page shows "Game not found" with `noindex`.
- `F2P_SKIP_GAME_PRERENDER=1 npm run build` skips them for quick local builds.
The Tauri builds never prerender them.
### Which commits deploy (build watch paths)
Since 2026-10-02 a commit that touches only documentation does not deploy -
but **data commits still did**: `3f9e8cff` ("Refresh store data
[2026-10-02]", four files under `data/`) deployed a new Worker version
(`712b4ab8`) at 16:54Z. The configuration below stops both. Skipping
everything outside `web/` is right, with ONE trap: **the web build reads the
legal documents.** `web/build/legal-versions.ts`
hashes them for the consent gate and `web/src/lib/server/markdown.ts`
renders them at `/legal/*`:
`LICENSE`, `LICENSE-DATA`, `docs/DISCLAIMER.md`, `docs/ToS.md`,
`docs/EULA.md`, `docs/PRIVACY_POLICY.md`, `docs/ACKNOWLEDGEMENTs.md`,
`docs/Contact.md`
A commit that changes only those - the ToS edit Phase B needs, for one - would
not reach the site, and the consent gate would keep showing the old text and
hash until the next code deploy. Either keep those files triggering a build,
or redeploy by hand after merging such a change. Excludes are applied BEFORE
includes, so an exclude like `docs/*` swallows them whatever the include list
says. A configuration that does both jobs:
- Include: `web/*`, `LICENSE`, `LICENSE-DATA`, `docs/DISCLAIMER.md`,
`docs/ToS.md`, `docs/EULA.md`, `docs/PRIVACY_POLICY.md`,
`docs/ACKNOWLEDGEMENTs.md`, `docs/Contact.md`
- Exclude: `web/src-tauri/*`, `web/README.md`
Everything not included - `data/**`, `scripts/**`, the other docs,
`CHANGELOG.md` and `changelog/`, `CLAUDE.md`, `README.md` - then never deploys. Check it once
by merging a commit that touches only `docs/plan/` (no build) and one that
touches only `docs/ToS.md` (a build), and by the next bot data commit: its
check runs on GitHub must have no `Workers Builds: free-steam-games-list`.
A check that IS there names a Version ID - that commit deployed.
## Verifying a deploy
```bash
curl -sI https://free-steam-games.win/ | head -3
curl -s https://free-steam-games.win/ | grep -o '<html lang="[a-z]*"'
curl -so /dev/null -w '%{http_code}\n' https://free-steam-games.win/api/data/data/index.json
curl -so /dev/null -w '%{http_code} %{size_download}b\n' https://free-steam-games.win/img/t/570/header.jpg
curl -so /dev/null -w '%{http_code}\n' https://free-steam-games.win/admin # expect 302 to Access
curl -so /dev/null -w '%{http_code}\n' https://free-steam-games.win/api/ingest/ping # expect 302 to Access
```
The image should come back around 3–4 KB: the Worker prefers Steam's small
capsule asset over the full header.
For the AVIF path, copy a game page's hero URL (`/img/d/<appid>/…?t=…`, the
`?t=` matters - it is part of the R2 key) and request it with
`-H "Accept: image/avif"`: once the cron has minted it the answer is
`image/avif` with `X-Img-Final: 1`; before that it is the JPEG with
`Cache-Control: public, max-age=86400` and no `X-Img-Final`.
The website itself loads that art from the bucket's custom domain instead
(`docs/SECURITY_SETUP.md` section 5): in a game page's Network panel the hero
is `image/avif` from `https://media.free-steam-games.win/img/v2/460q80/…`,
with `cf-cache-status: HIT` on a reload. A `/img/d/…` request right after it
means the media load failed and the fallback ran - expected for a game the
cron has not reached yet, a fault for one it has.
**A stale service worker will show you the old app after a deploy.** The site
is a PWA, so a browser that visited before the deploy serves its cached shell
until the service worker updates. If you are checking whether a deploy landed,
use a fresh profile or a hard reload — otherwise you will diagnose a
deployment problem that does not exist.
## Local development
```bash
cd web
npm ci
npm run dev # Vite + HMR; /api/* and /img/* proxy to the live Worker
npx wrangler dev # the real Worker against web/dist — required for worker/ changes
```
`npm run dev` cannot exercise `worker/` source. It proxies those paths to
production so the app has real data and images; changes under `web/worker/`
need `wrangler dev`.
Stop `wrangler dev` before running `npm ci`. On Windows the install deletes
`node_modules` and fails partway if a dev server holds files open.
## Checks
`.github/workflows/web-ci.yml` runs on pull requests and on pushes to `main`
touching `web/**`, `.github/workflows/**` or `sonar-project.properties`:
`npm ci --ignore-scripts`, the typechecks (app, Worker and tests, on
TypeScript 7; the admin's svelte-check on TypeScript 6), `npm test` (which
includes the toolchain-pin and data-write-group checks over the workflows),
the build, `scripts/verify-dist.mjs --web` over the
prerendered HTML, `npm run cf:check` (`wrangler deploy --dry-run`), and
`knip`. `python-tests.yml` runs the pipeline's pytest suite on changes under
`scripts/` and the requirements files, installing them exactly as the data jobs
do (`--require-hashes --only-binary :all:`).
It uses a plain `npm ci`, never `npm ci || npm install`. That fallback is what
hid a lockfile conflict which broke installs for a month. `--ignore-scripts`
keeps every dependency's install script from running with the job's token;
none of the three packages that have one needs it (the comment in
`web-ci.yml` says why), and no workflow runs `npx`, which can fetch a package
the lockfile never named.
CodeQL (`codeql.yml`) and SonarCloud (`sonarcloud.yml`) both look at pull
requests and `main`. Each posts a check on the pull request (`CodeQL`,
`SonarCloud Code Analysis`) about the issues the change brings in; the full list
for `main` lives in the repository's Security tab and on the SonarCloud
dashboard. `sonar-project.properties` at the root names the project and decides
what SonarCloud reads as code, what as tests, and what not at all (the dataset,
generated Android resources, the SQLite migrations); the scanner reads it from
the commit it analyses.
SonarCloud runs in CI, with the repository secret `SONAR_TOKEN` (a SonarCloud
token that may analyse the project). Its automatic analysis must stay off
(project Administration > Analysis Method): with both on, every CI scan fails.
Commits that touch only what the data jobs write (`data/`, `games/`, the
`.jsonl` queues and logs under `scripts/`) are not scanned. A pull request from
a fork or from Dependabot gets no repository secrets, so its scan job is
skipped; giving Dependabot its own `SONAR_TOKEN` (Settings > Secrets >
Dependabot) would have its pull requests analysed too.
Qodana (`qodana.yml`) runs JetBrains' inspections on pull requests and
`main` as well, one job per Qodana Cloud project: Python over `scripts/`,
JavaScript/TypeScript (with the Svelte plugin) over `web/`, Rust over
`web/src-tauri/`. Each project has its own token, kept in the repository
secrets `QODANA_TOKEN_PYTHON`, `QODANA_TOKEN_TS_JS` and `QODANA_TOKEN_RUST`
(Qodana Cloud > the project > its setup page). Its settings are the
`qodana.yaml` in that directory: a gate of zero problems, so a new finding
fails the job, and every inspection or path left out, with the reason.
Commits that touch only data or Markdown are not scanned, and a pull request
without the secrets (a fork, Dependabot) skips the scan. The results are on
Qodana Cloud, in the job's annotations and in its `qodana-report-*`
artifact. All three linters need a Qodana Ultimate licence; without one every
job fails, and `qodana.yml` and the three `qodana.yaml` files are what to
remove.
Before the scan the job runs both test suites with coverage: pytest with
`--cov` (configured in `pyproject.toml`, written to `coverage.xml`) and
`npm run test:coverage` in `web/` (configured in `vite.config.ts`, written to
`web/coverage/lcov.info`). SonarCloud's default "Sonar way" quality gate asks
for 80% coverage on new code (it ignores that condition below 20 new lines),
so a pull request that adds code its tests do not reach fails that check. To
see the figure locally, run
`npm run test:coverage` in `web/`, or
`python -m pytest scripts/tests --cov` after installing `requirements-test.txt`.
### A red "Workers Builds" check on a PR branch is usually not yours
Cloudflare posts its own check alongside those. On a pull-request branch it has
been seen **failing with zero duration** — `started_at` equal to `completed_at`,
so nothing was compiled and there is no log reachable from GitHub — while the
identical commits built and deployed from `main` minutes later. Measured on
PR #130 (2026-09-20), twice.
Before spending time on it, look at the GitHub Actions `check` job: it already
runs the same `npm run build`, `verify-dist.mjs` and `wrangler deploy
--dry-run`. If that is green, the build is sound and the Cloudflare check is
about the preview build, not your diff. The only place the real answer lives is
the build log in the Cloudflare dashboard, which the check links to.
## Rolling back
Roll back the Worker deployment in the Cloudflare dashboard
(*Workers → free-steam-games-list → Deployments*). Reverting the commit also
works but takes a full build cycle.
GitHub Pages is **not** a fallback. It now serves only a redirect to this
domain; the app cannot run there because it is built with `base: "/"`.
## Secrets
`wrangler.jsonc` holds no secrets — only the D1 `database_id`, which is an
account-scoped identifier and safe to commit. Anything sensitive goes through
`npx wrangler secret put`: the GitHub App's three (see [ADMIN.md](./ADMIN.md))
and `TURNSTILE_SECRET` for the human check (see
[SECURITY_SETUP.md](./SECURITY_SETUP.md) section 12). The Worker declares them
in `web/worker/env.d.ts`.
For `wrangler dev`, put local values in `web/.dev.vars` (gitignored);
`web/.dev.vars.example` holds Cloudflare's always-pass Turnstile test secret.
## Cost notes
**Images.** AVIF copies of the header art are minted by the cron into the R2
bucket `f2p-media` (`web/worker/lib/img-mint.ts`); `/img/*` only reads them, so
requests cannot spend anything - there is nothing for an allowlist to guard.
Images Paid bills $0.50 per 1,000 unique transformations after 5,000 a month
and has **no spend cap**; `IMG_TRANSFORM_MONTHLY_CAP` (a D1 counter) is the
cap. See SECURITY_SETUP.md section 5.
**The bucket must exist before any deploy that binds it.** `wrangler deploy`
fails with code 10085 otherwise (the old Worker stays live). It was created
with `npx wrangler r2 bucket create f2p-media`; keep its r2.dev URL disabled
and attach no custom domain - the Worker is the only reader.
**R2** stays inside the free tier: ~11k objects at ~3-18 KB, about 12 list
calls per cron tick only while minting, and reads only on an edge-cache miss.
**Shard compression is a dashboard setting.** Cloudflare does not compress
`application/x-ndjson` on its own, so the plain shards went out uncompressed
(~9 MiB per full load) until the Compression Rule `ndjson-shards` was added on
2026-10-02: Rules → Compression Rules, `http.request.uri.path.extension eq
"jsonl"` → Custom: Brotli, Gzip. It applies to Worker responses (data_001:
1,497,828 → 225,826 bytes). The packed `/api/data/p1/*.bin` files are
compressed before they are encrypted and are deliberately not `.jsonl`, so the
rule never touches them.