Dependabot #190-#194 combined with Zustand 5.1.0, soupsieve 4.1, coverage 7.17.3, Ruff 0.17.5 and a cargo update (wry 0.57.2, tauri-plugin-opener 2.7.3), each past a 3-day cooldown; Wrangler 4.161.0 with regenerated Worker types; tapable held at 2.3.3. Version 4.6.2 with a short changelog entry.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The first SonarQube Cloud scan reported 57 issues. 45 are fixed in code, the 11 on the saved itch.io test pages leave the scan via .sonarcloud.properties, and css:S8776 (Tailwind's @utility nesting) is a false positive to mark on SonarQube Cloud.
Worker: trailing slashes and the gate cookie's padding are trimmed by trimEnd() instead of /\/+$/ (quadratic on the public 404 path); the feed and collection parsers lose their backtracking regexes; compareCodeUnits keeps the code-unit sorts (the rule's localeCompare is locale-dependent); Number.*, DEFAULT_SIZES, crypto.randomUUID; i18n-edit.mjs reads stdin. Pipeline: the NSFW notices are one selector list, either class (the rule's ".a.b" would need both), each tested; plain if-guards; update_info.main returns None; no plain-http adapter; [[ ]] in cf_build_status.sh.
Supply chain: requirements*.txt are hash-locked pip-tools compiles of requirements*.in (on Python 3.14, which Dependabot reads from the header), installed with --require-hashes --only-binary :all: in block scalars; npm ci --ignore-scripts in CI and the release builds (verified by manual release runs on all platforms); npm run cf:check.
Release 4.4.2: also ships the toolchain upgrade (#149, #154) and Dependabot #141-#148, #150, #152, #153. CLAUDE.md gotchas 40 and 41.
[skip-discuss]
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
TypeScript 7.0.2 (the native compiler) now runs `tsc -b` in `npm run build`.
It has no JavaScript API until 7.1, and typescript-eslint (peer
typescript <6.1) and scripts/i18n-edit.mjs need one, so the two are
installed side by side: `@typescript/native` = npm:typescript@^7.0.2
(bin tsc) and `typescript` = npm:@typescript/typescript6@^6.0.2 (the 6.0
API, bin tsc6). No tsconfig or source change: TS 7 reports 0 errors.
Node 26: webapp/.node-version 24 -> 26 (CI and both release workflows read
it), engines >=26, @types/node 26. npm 11.20 gates dependency install
scripts on `allowScripts`, so esbuild and workerd are approved by name.
Python 3.15: .python-version 3.14 -> 3.15, ruff target py315, urllib3
declared (scraper.py imports it). UTF-8 mode is the default since 3.15,
so the workflows drop PYTHONUTF8.
Dependabot: keep the `typescript` alias off new majors, group
@typescript/*, @types/node up to 26. Docs and CLAUDE.md follow (gotcha 6
rewritten, gotcha 38 for allowScripts).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* feat(pipeline): rotating refresh + patch/apply pipeline, drop lists/
Replace check_paid / check_alive / update_reviews / update_status: each
crawled all 2,600+ pages sequentially, overran its job timeout, saved
only at the end and has lost every run since late June (unreported:
timeouts end as "cancelled").
- scripts/refresh.py: daily slice of ~1/7 of the catalog (pending
strikes first, then least recently checked), one GET per game for
alive / paid / rating / status (+ release_date, thumbnail, updated_at
backfill). Removal needs the same 404/410 or paid strike on two
different UTC days. Paced requests, capped Retry-After back-off, stop
after 3x 429, checkpoints and an in-script deadline. --full replaces
force_update.py (rotating batches, editable fields preserved).
- Scanners emit a URL-keyed patch (patch.py); apply_patch.py applies it
to the latest main and runs validate.py; bash/commit_push.sh fetches,
re-applies and retries pushes, so concurrent writers never clobber
each other.
- update_info.py: canonical URLs (canonical.py), skip games in the
deleted log, keep transient failures queued (max 3 runs, tracked in
scripts/state/), stamp added_at, never drop links queued meanwhile.
- data_store.py: games stay in their chunk, re-slice only past
thresholds, write only changed bytes; index/count_history change only
with data (no more no-op commits).
- validate.py (schema, enums, canonical/unique URLs, index consistency)
+ normalize one invalid safe_virus ("y" -> "Yes").
- tests/: 85 network-free pytest cases, HTML fixtures (from sample/),
shared URL vectors and golden chunk/serialization fixtures for the
future Worker port.
- Workflows: refresh.yml (scan -> apply -> notify), update.yml (runs on
queue change), force_update.yml (refresh.yml full mode),
notify-failure.yml (Discord on failure/cancel/rate limit),
python-ci.yml, webapp-ci.yml, deploy-status.yml; draft-first release
with one create-release job and tauri-action v1, no caches in release
builds; SHA-pinned actions, permissions: {} defaults, reusable
workflows pinned with explicit secrets, injection fixes; dependabot.yml.
- Remove lists/, generate_md, generate_table, log_deleted,
deleted_games.txt, the Telegram bot-ingest path and old bash wrappers;
update README (EN/VI), CONTRIBUTING, templates, CLAUDE.md, CHANGELOG
and mark the retired Telegram path in the policies.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(pipeline): review findings on the refresh / ingest rewrite
Removals
- Mass-change guard: a run whose removals or new strikes exceed
max(10, 10%) of the games it checked withholds all its removals and
alerts (mass_change). Withheld games are marked withheld_at and stay
withheld in later (even single-URL or rate-limited) runs until a healthy
check clears them or a run is dispatched with allow_mass_removal.
- Checkpoints never carry removals, so a scan that dies before its guard
cannot delete games; apply_patch re-checks the per-run limit and caps
any patch at max(25, 5%) of the catalog (ALLOW_MASS_REMOVAL=1 overrides).
- Strikes mature by time, not calendar day: a second sighting must come
>= 20 h after the first (strike_at), so two checks around midnight UTC no
longer remove a game.
- Overlapping scans can't undo each other: refresh_state merges keep the
newest check and the newest full stamp in either apply order, and a
removal is skipped when a newer check has landed since.
- A corrupt count_history.json fails the apply instead of being replaced
by a one-row history.
Ingest
- A second spelling of a queued game no longer drops the first copy
before it reaches a final outcome.
- A dispatched url is processed first and kept in temp_link.json
(queue_add) until it finishes, so a retry, deadline or rate-limit stop
keeps it.
- Failures count as an attempt only >= 6 h apart and a 429 counts as none,
so a burst of queued runs can't burn a link's three attempts.
- Non-string queue entries are cleaned on every apply instead of crashing
it; {"url": ...} objects are unwrapped.
CI
- queue: max on the ingest, refresh-apply and force-update concurrency
groups: the default queue replaced (and silently dropped) pending runs.
- Seed scripts/state/*.json and mkdir -p before git add (the pathspec
failed on an empty patch until the directory existed on main).
- Cancelling a refresh / force-update / desktop release really stops it
(!cancelled() instead of always()).
- deploy-status.yml becomes an hourly check of main's latest finished
"Workers Builds: free-games-itchio-list" run (bash/cf_build_status.sh):
GitHub never starts check_run workflows for GITHUB_TOKEN data commits.
- The reusable release notifiers still take their script from
poli0981/.github@main; the caller comments now say so.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(pipeline): close the remaining mass-removal and monitor gaps
From the second review round of the pipeline fixes:
- A strike spike now marks every strike of that run withheld_at (the
checkpoint of a run that dies mid-spike too), so a later short,
rate-limited or single-URL run can't remove them without an
allow_mass_removal dispatch.
- Withheld games take at most half of a run's budget (all of it for the
allow_mass_removal run that confirms them), and a withheld full-mode
game moves its `full` stamp, so neither rotation stalls on them.
- refresh.yml runs one scan -> apply pipeline at a time (workflow-level
concurrency refresh-pipeline, queue: max), so an older removal can't be
applied after a newer healthy check of the same game.
- deploy-status.yml alerts once per failed Cloudflare build, keyed on its
check-run id and marked only after the Discord post, instead of by age
(which could drop an alert after a missed run, or repeat it); the
monitor walks main's first-parent history so preview builds of merged
branch commits are never mistaken for the site's build.
- CLAUDE.md: merge PRs with squash or a merge commit, never by fast-forward.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
When game_info.json reaches 500 entries, data is automatically split
into numbered files (game_info_001.json, game_info_002.json, etc.).
New data_store.py module handles load/save/rebalance for all scripts.
Also removes CSV export pipeline (export_csv.py, pandas dependency)
and cleans up unused files.
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>