Retire the browser extension's ingest API; release 4.4.1 (#138)

The browser extension (poli0981/itch-f2p-extension) and the Telegram bot (poli0981/telegram-scraper-bot) are archived. Removed: POST /api/ingest (ingest.ts, requireService, the idempotency store, RL_INGEST, ACCESS_AUD_INGEST; it never received a game), the admin's "Browser extension" auto-approve source and "Extension" queue label, and the About page's Telegram numeric-ID note. No D1 migration: the idempotency table and the 'ext' CHECK stay.

Policies (Privacy, ToS, EULA + installer RTF, Disclaimer, Security; EN + VI) drop the extension / ingest wording; Last updated 2026-10-16, LEGAL_VERSION unchanged. Docs, CLAUDE.md and pipeline comments updated.

Release 4.4.1 (also ships #131, #135, #136, #137).

[skip-discuss]

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
poli0981andClaude Opus 5.5 authored and GitHub committed 2026-10-16 17:56:46 +07:00
1 parent a5bf9d9f14
commit f199e2ea17
53 files changed
+185 -434

No files matched your search

+1 -1
View File
@@ -1,5 +1,5 @@
# Lint + tests for the data pipeline, and schema validation of the catalog.
# Pushes by GitHub Apps (browser extension, admin Worker) also trigger this,
# Pushes by the admin Worker's GitHub App also trigger this,
# so every externally written data change is validated.
name: Python CI
+2 -2
View File
@@ -1,5 +1,5 @@
# Ingest queued links from scripts/temp_link.json (written by the browser
# extension, and later the admin Worker) plus an optional dispatched URL.
# Ingest queued links from scripts/temp_link.json (written by the admin
# Worker and its cron) plus an optional dispatched URL.
#
# Runs right away when temp_link.json changes on main (pushes made by GitHub
# Apps trigger it; this workflow's own GITHUB_TOKEN push does not, so no loop),
+2 -1
View File
@@ -3,12 +3,13 @@
All notable changes to this project, newest first. The notes of each release live in
[`changelog/`](changelog/), one file per minor version: `changelog/<major>.x/ver<major>.<minor>.x.md`.
**Latest release: [4.4.0](changelog/4.x/ver4.4.x.md#440---2026-10-13-admin-automation-bulk-actions-smoother-charts) (2026-10-13) — Admin automation, bulk actions, smoother charts.**
**Latest release: [4.4.1](changelog/4.x/ver4.4.x.md#441---2026-10-16-extension-and-bot-retired) (2026-10-16) — Extension and bot retired.**
## 4.x
| Version | Date | Highlights |
|---|---|---|
| [4.4.1](changelog/4.x/ver4.4.x.md#441---2026-10-16-extension-and-bot-retired) | 2026-10-16 | Extension and bot retired (archived repos), ingest API removed |
| [4.4.0](changelog/4.x/ver4.4.x.md#440---2026-10-13-admin-automation-bulk-actions-smoother-charts) | 2026-10-13 | Admin automation, bulk actions, smoother charts |
| [4.3.0](changelog/4.x/ver4.3.x.md#430---2026-10-10-ai-disclosure-accessibility-refreshed-catalog) | 2026-10-10 | AI disclosure, accessibility, refreshed catalog |
| [4.2.0](changelog/4.x/ver4.2.x.md#420---2026-10-03-compact-catalog-data-themed-scrollbars) | 2026-10-03 | Compact catalog data, themed scrollbars |
+8 -6
View File
@@ -14,7 +14,7 @@ everyone; the old github.io URL only redirects). Three layers share one JSON cat
2. **Cloudflare Worker + static assets** — `webapp/worker/` + `webapp/wrangler.jsonc`, deployed by **Cloudflare Workers Builds** from `main` (root dir `webapp`, build `npm run build`, deploy `npx wrangler deploy`; no deploy workflow). The catalog is bundled into the site at build time (`vite-plugins/catalog-data.ts` → `dist/data/pack/` compact packs + the legacy readable JSON in `dist/data/`, plus `data/urls.json` and `sitemap.xml`), so every data commit rebuilds the site.
3. **Web app + Tauri apps** — `webapp/src/` (React 19.3 + TypeScript 6.0 + Vite 8/rolldown + Tailwind + shadcn/ui + React Router 8), and the **Tauri 2 desktop + Android** read-only viewers in `webapp/src-tauri/` (same code; they fetch `https://freeitchgames.win/data/pack` and load covers straight from img.itch.zone). Plus `webapp/admin/` — the maintainer-only admin app (second Vite entry, served at `/admin/`).
Adding games: the public **Suggest** page (`/suggest`, Turnstile), the maintainer's browser extension (`POST /api/ingest`, Access service token; repo `poli0981/itch-f2p-extension`), and RSS discovery (Worker cron) all feed a **D1 review queue**; approving in `/admin` commits the URLs to `scripts/temp_link.json` as a GitHub App, which triggers `update.yml`. Since 4.4 the cron also approves by rule (`/admin` → Settings, on by default: unflagged `rss` / `form` items nobody rejected within 24 h, only those found after the switch-on baseline `since`, ≤ 100 per run) and queues the new free games of **followed itch.io collections** directly (removed or flagged ones go to the queue) — the policies say so (ToS §5, Privacy §2.4, Disclaimer §4). The Add page takes any text, a file or a collection, **Check**s it first (dry run), sends batches of 200 and follows each game's ingest (`/api/admin/track`).
Adding games: the public **Suggest** page (`/suggest`, Turnstile) and RSS discovery (Worker cron) feed a **D1 review queue** (the browser extension's `POST /api/ingest` was removed in 4.4.1; the extension `poli0981/itch-f2p-extension` and the Telegram bot `poli0981/telegram-scraper-bot` are archived repos); approving in `/admin` commits the URLs to `scripts/temp_link.json` as a GitHub App, which triggers `update.yml`. Since 4.4 the cron also approves by rule (`/admin` → Settings, on by default: unflagged `rss` / `form` items nobody rejected within 24 h, only those found after the switch-on baseline `since`, ≤ 100 per run) and queues the new free games of **followed itch.io collections** directly (removed or flagged ones go to the queue) — the policies say so (ToS §5, Privacy §2.4, Disclaimer §4). The Add page takes any text, a file or a collection, **Check**s it first (dry run), sends batches of 200 and follows each game's ingest (`/api/admin/track`).
## How to work in webapp/
@@ -39,15 +39,15 @@ Local admin without Access: put `DEV_ADMIN_EMAIL=you@example.com` in `webapp/.de
## Worker (`webapp/worker/`)
- `index.ts` routes: `/img/*` (img.ts, behind the gate), `/api/health`, `/api/verify` (gate.ts), `/api/suggest` (suggest.ts), `/api/ingest` (ingest.ts), `/api/admin/*` + `/admin*` (admin.ts), the readable `/data/*.json` (legacy-data.ts), everything else that misses an asset → `spa.ts`. `run_worker_first` also lists `/` (otherwise served straight from `dist/index.html`) so the gate sees it, and the five legacy catalog files one by one (never a `!` pattern — gotcha 26). `scheduled()` (cron `23 */4 * * *`), four steps each in its own try/catch: `discover.ts` (one itch.io RSS feed per run, 6 h back-off on 429) → `QueueStore.reconcile` (queued → ingested / failed with the pipeline's outcome, read at head through the GitHub App; else in the catalog → ingested, > 3 days → failed `timeout`) → `automation.ts` (auto-approve + followed collections, ≤ 30 itch.io pages a run, ONE `[auto]` commit, D1 written only after it lands; the run's summary in settings `automation_last_run`) → `purge` (idempotency 7 d, Suggest notes 180 d). The account is on **Workers Paid** (30 s CPU): admin catalog edits parse and rewrite ~3.8 MB in the Worker (≈ 25–30 ms for 100 removals).
- `index.ts` routes: `/img/*` (img.ts, behind the gate), `/api/health`, `/api/verify` (gate.ts), `/api/suggest` (suggest.ts), `/api/admin/*` + `/admin*` (admin.ts), the readable `/data/*.json` (legacy-data.ts), everything else that misses an asset → `spa.ts`. `run_worker_first` also lists `/` (otherwise served straight from `dist/index.html`) so the gate sees it, and the five legacy catalog files one by one (never a `!` pattern — gotcha 26). `scheduled()` (cron `23 */4 * * *`), four steps each in its own try/catch: `discover.ts` (one itch.io RSS feed per run, 6 h back-off on 429) → `QueueStore.reconcile` (queued → ingested / failed with the pipeline's outcome, read at head through the GitHub App; else in the catalog → ingested, > 3 days → failed `timeout`) → `automation.ts` (auto-approve + followed collections, ≤ 30 itch.io pages a run, ONE `[auto]` commit, D1 written only after it lands; the run's summary in settings `automation_last_run`) → `purge` (Suggest notes 180 d). The account is on **Workers Paid** (30 s CPU): admin catalog edits parse and rewrite ~3.8 MB in the Worker (≈ 25–30 ms for 100 removals).
- `img.ts`: `/img/<160|640>/<path on img.itch.zone>` — allow-listed against `data/urls.json`, edge cache → R2 `THUMBS` (`w{w}/<sha256>.webp`) → `cf.image` transform (stored once) → fallback streams the original with a 1-day cache (quota error 9422). The fallback original and a missing upstream image are also kept in `caches.default` for a day (GET only); widths are the exact strings `160` / `640` (a `Map`, no `Number()` aliases); background puts log failures. Upstream errors → controlled 502.
- `spa.ts`: `not_found_handling` is `"none"`, so misses reach the Worker: `/assets/*`, `/data/*` (a pack from an older deploy included — `no-store`, CORS) and file-like paths get a real 404 (`no-store`); app routes get the shell with 200; unknown paths and game slugs not in the catalog get the shell with **404**; with the gate on, a visitor without a pass gets the verification page instead (checked before the shell is fetched, so never a 304). Keep its `ROUTES` in sync with `src/App.tsx` and `src/hooks/useSeoHead.ts`.
- `gate.ts` + `gate-page.ts`: the **verification gate** (web only). App pages and `/img` need a pass: the `__Host-fig_gate` cookie (HMAC of its issue time, HttpOnly, `BOT_GATE_TTL_HOURS`, default 48) set by `POST /api/verify` after a Turnstile check (action `gate`, `RL_GATE`, same-origin); `GET /api/verify` reports `{enabled, valid, via, ttl}`. The page is self-contained (inline CSS/JS with a per-response CSP nonce, EN/VI, the site's og tags, 200 + noindex). Verified bots pass with `x-fig-known-bot: <GATE_BOT_TOKEN>` from a zone Transform Rule; clients that only claim to be a crawler get 503. `/assets`, `/data`, `/api/*`, `/admin*` and the other static files stay open. `turnstile.ts`: siteverify shared with Suggest (hostname + action checked; the test secrets skip both).
- `access.ts`: verifies `Cf-Access-Jwt-Assertion` with jose (issuer + AUD). Maintainer = email in secret `ADMIN_EMAILS` + same-origin writes; ingest = service-token `common_name`.
- `access.ts`: verifies `Cf-Access-Jwt-Assertion` with jose (issuer + AUD). Maintainer = email in secret `ADMIN_EMAILS` + same-origin writes (the only identity: the service-token ingest is gone since 4.4.1).
- `admin.ts`: API (queue list / approve / reject / check (dry run) / add / track / edit one or ≤ 100 (`/games/bulk`) / remove one or ≤ 100 (`/games/remove`; after the commit the audit row and the R2 cleanup are best effort → `warnings`, never a 500) / restore / activity / settings / itch.io collections) and the `/admin` shell with its own CSP (theme-script hash included). Request limits shared with the admin app: `limits.ts`. `collections.ts` reads public itch.io collections (gotcha 36); `automation.ts` holds the cron's rules (`planCollectionItems`: only URLs D1 has never seen and without a flag are queued directly — no re-add loop for games the ingest dropped). The maintainer's email goes only to the D1 audit log, never into a commit (the repo is public).
- `repo.ts` + `github.ts` + `catalog.ts`: writes as a **GitHub App** (installation token, cached per isolate) with GraphQL `createCommitOnBranch` + `expectedHeadOid` (verified commits; `commitWithRetry` rebuilds from the new head on a race). Reads use the raw media type (no base64/UTF-8 mojibake). `catalog.ts` ports `data_store.py` byte-for-byte (golden fixtures).
- `legacy-data.ts`: the readable catalog JSON (`index.json`, `game_info_NNN.json`, `count_history.json`, `deleted_games.json`, `urls.json`) goes only to apps 4.0–4.1.1 — `Origin` `tauri://localhost` / `http(s)://tauri.localhost` → `env.ASSETS` passthrough + `Vary: Origin`; anyone else → **410** JSON with the repository links (`no-store`, CORS, a sampled `console.info` of the Origin). A nudge, not a security boundary: the data is public in the repo.
- `queue.ts`: URL classification (catalog → blocklist → pending → previously deleted) + the D1 store (`worker/migrations/`, additive only — never edit an applied migration; `0002` added `candidates.outcome`, `settings` and `watched_collections`; URL lists bind in `IN` slices of 90, gotcha 37). `data.ts`: catalog facts read from the site's own `/data` via `env.ASSETS`, cached per isolate (loaded values only — see gotchas).
- `queue.ts`: URL classification (catalog → blocklist → pending → previously deleted) + the D1 store (`worker/migrations/`, additive only — never edit an applied migration; `0002` added `candidates.outcome`, `settings` and `watched_collections`; the `idempotency` table and `'ext'` in the `candidates.source` CHECK are unused leftovers of the ingest API removed in 4.4.1; URL lists bind in `IN` slices of 90, gotcha 37). `data.ts`: catalog facts read from the site's own `/data` via `env.ASSETS`, cached per isolate (loaded values only — see gotchas).
- Configuration: non-secret settings are `vars` in `wrangler.jsonc` (empty until setup); secrets (`GH_APP_PRIVATE_KEY` — the PEM GitHub downloads (PKCS#1) or PKCS#8; `github.ts` converts, `ADMIN_EMAILS`, `TURNSTILE_SECRET`, `GATE_SECRET`, `GATE_BOT_TOKEN`) via `wrangler secret put`, typed in `worker/env.ts` as optional. A feature whose settings are missing answers **503** (the gate just stays off); the public site needs none of them. `env.staging` = Worker `free-games-itchio-list-staging` on `staging.freeitchgames.win` (no cron, own R2/D1/rate-limit namespaces).
## Web app structure (`webapp/src/`)
@@ -74,7 +74,7 @@ Each game has **20** fields (plus optional `added_at` / `updated_at`, appended b
| Workflow | Trigger | Notes |
|---|---|---|
| `update.yml` | Push to `main` touching `scripts/temp_link.json` + daily 01:23 UTC + `workflow_dispatch` (`url`) | Ingest queue → patch → `commit_push.sh`. Workflow-level `concurrency: ingest` with **`queue: max`** (the default queue replaces a pending run, which would drop its dispatched `url`). A dispatched `url` is processed first and written to `temp_link.json` (`queue_add`) until it reaches a final outcome. Failed fetches count as an attempt only ≥ 6 h apart (`last_attempt`; 3 → given up), and a 429 counts as none. GitHub-App pushes (extension, admin Worker) trigger it; its own `GITHUB_TOKEN` push does not (no loop). Keep the filename. |
| `update.yml` | Push to `main` touching `scripts/temp_link.json` + daily 01:23 UTC + `workflow_dispatch` (`url`) | Ingest queue → patch → `commit_push.sh`. Workflow-level `concurrency: ingest` with **`queue: max`** (the default queue replaces a pending run, which would drop its dispatched `url`). A dispatched `url` is processed first and written to `temp_link.json` (`queue_add`) until it reaches a final outcome. Failed fetches count as an attempt only ≥ 6 h apart (`last_attempt`; 3 → given up), and a 429 counts as none. GitHub-App pushes (the admin Worker and its cron) trigger it; its own `GITHUB_TOKEN` push does not (no loop). Keep the filename. |
| `refresh.yml` | Daily 02:47 UTC + `workflow_dispatch` (`budget`, `url`, `allow_mass_removal`, `full`) | scan (read-only, 55-min step timeout, checks out the branch as it is when the job starts — not the queued SHA — so runs waiting in the queue see the previous run's state; patch artifact uploaded `if: always()`) → apply (`!cancelled()`: a failed / timed-out scan's checkpoint is applied, a cancelled run's is not) → notify. Workflow-level `concurrency: refresh-pipeline` + `queue: max` runs one whole scan → apply at a time (scheduled, manual and force-update runs alike). Removal needs the same 404/410, paid or `private` (itch.io's password form, `is_password_protected`; ingest drops such links as final) strike seen again ≥ 20 h after the first sighting (`strike_at`). Checkpoints never carry removals. **Mass-change guard:** a run whose removals or new strikes exceed max(10, 10 %) of the games it checked withholds all its removals and marks them — and, on a strike spike, all its strikes — `withheld_at`; later runs keep withholding those (they take at most half a run's budget) until a healthy check clears them, and alert (`mass_change`) — that pattern means itch.io markup changed; dispatch with `allow_mass_removal` only after checking on itch.io. `apply_patch.py` re-checks the same per-run limit and max(25, 5 %) of the catalog unless `ALLOW_MASS_REMOVAL=1`. |
| `force_update.yml` | `workflow_dispatch` ONLY (`url`, `budget`, `allow_mass_removal`) | Dispatches `refresh.yml` with `full: true` (`actions: write`; it used to call it through `uses:`, which failed at startup with no annotation): re-scrapes every scraper field for one URL or the next batch (rotates on `full` timestamps). Dispatch it N times in a row to cover N consecutive batches (FIFO, each from the latest main). Always preserves `safe_virus` / `notes` / `nsfw`. Canonical tool for repairing mojibake. |
| `notify-failure.yml` | `workflow_call` | Discord (`DISCORD_CI_WEBHOOK`) on failure / **cancelled** (job-level timeouts end as cancelled) / rate-limited / mass change / failed Cloudflare build. |
@@ -153,6 +153,8 @@ Code **MIT** (`LICENSE`); catalog data **CC BY 4.0** for the maintainer's contri
## Current state (Oct 2026)
**v4.4.1** (2026-10-16): the browser extension and the Telegram bot are retired — both repos (`poli0981/itch-f2p-extension`, `poli0981/telegram-scraper-bot`) were already public archives; an Announcements discussion says so. Removed: `POST /api/ingest` (`ingest.ts`, `requireService`, `RL_INGEST`, `ACCESS_AUD_INGEST`, the idempotency store — it never received a game: 0 `ext` rows in production D1), the admin's "Browser extension" auto-approve source and "Extension" label, the About page's Telegram numeric-ID note. The policies (Privacy, ToS, EULA + installer RTF, Disclaimer, Security; EN + VI) only lost the extension / ingest wording: "Last updated" 2026-10-16, `LEGAL_VERSION` deliberately unchanged (nothing new to accept, ToS §14). Unused repo secrets `GH_TOKEN`, `TELEGRAM_BOT_TOKEN`, `DISCUSSION_*` deleted. Ships Dependabot #131 (vite 8.3.4), #135 (tauri crates), #136 (serde_json), #137 (ruff).
**v4.4.0** (2026-10-13, PR #133): admin automation. Add page: any text / file / itch.io collection → **Check** (dry run) → batches of 200 → per-game ingest tracking (`/api/admin/track`, from the new `scripts/state/ingest_outcomes.json`). The cron runs the maintainer's automatic rules in one `[auto]` commit: auto-approve (on by default, `rss` + `form`, unflagged, after 24 h, only items found after the `since` baseline, ≤ 100 a run) and **followed collections** (≤ 30 pages a run; never-seen unflagged free games queued directly, removed or flagged ones to review). Bulk remove / edit ≤ 100 in one commit (preset reasons translated on the Removed page), New games, Activity, Settings; queue batches of 200. D1 migration `0002_automation.sql` applied to production before the merge. Policies rewritten for approval by rules (ToS §5, Privacy §2.3/§2.4, Disclaimer §4, EN + VI; `LEGAL_VERSION` 2026-10-13). Web: the theme button shows the theme in use (a v4 regression), a pre-paint theme script (no dark flash; hash in three CSPs), Charts tabs kept mounted (switching back reuses every chart, no replayed animation), 300 ms entrances, scrollbar gutter. Also ships #130 (2026-10-11, the Charts page on phones: one-row tab bar, 12 px legends / 11 px ticks, horizontal Languages / Avg session bars, all 15 Genre labels, data-order pie legends, translated removal reasons, locale axis numbers).
**v4.3.0** (2026-10-10, PR #117; released the same day): itch.io's newer info rows in the catalog — `ai_disclosure` / `ai_content` (generative-AI disclosure) and `accessibility` — on the game page, as `/games` filters (`ai`, `access`) and as charts (Overview: "Generative AI"; Discovery: "Accessibility Features", "What AI Was Used For"); `description` = the creator's tagline. The whole catalog was re-scraped the same day with `force_update.yml` (6 × 600 + 10, ≈14.5 games/min; an itch.io outage ~05:20–06:50Z only cost retries): 3,236 of 3,251 games carry the new fields — No AI 53 %, AI Assisted 11 %, not disclosed 36 %; accessibility on 14 % (Subtitles 257, One button 119); the other 15 are 14 dead-page strikes awaiting confirmation and one password-protected page (`everium.itch.io/underthethorns`; it failed every check until #129 taught the refresh to strike such pages). Fixed on the way, same day: #118 (`force_update.yml` had never worked in v4 — it now dispatches `refresh.yml`, whose scan reads the latest main so queued batches rotate), #119 (`tauri-plugin-opener` 2.7: every first v4.3.0 release build had failed; the tag was moved before publishing), #120 (the 5 open Dependabot alerts; `check:tauri` CI guard; weekly cargo Dependabot), then Dependabot #122–#127.
@@ -163,4 +165,4 @@ Code **MIT** (`LICENSE`); catalog data **CC BY 4.0** for the maintainer's contri
**v4.1.0** (2026-09-24): the Turnstile verification gate for the web app, readable chart tooltips, the changelog split into `changelog/`, a docs refresh, updated Privacy/ToS/EULA (`LEGAL_VERSION` 2026-09-24). The gate ships switched on in `wrangler.jsonc` but stays inert until `GATE_BOT_TOKEN`, the Transform Rule and `GATE_SECRET` are in place (`docs/operations.md` §1 step 14) — `curl -s https://freeitchgames.win/api/verify` says `"enabled"`.
**v4.0.0** (2026-09-18) moved the site to the Cloudflare Worker at https://freeitchgames.win (PRs #81, #82, #88). The whole chain is live: Access for `/admin`, the GitHub App (verified commits to `scripts/temp_link.json` → `update.yml`), Turnstile, `ADMIN_EMAILS`; D1 `freeitchgames`, R2 `freeitchgames-thumbs`, Image Transformations (source `img.itch.zone`); Web Analytics is injected by the zone. `/api/ingest` stays unused — the browser extension is archived. **Still open (owner):** the legal-text review items; add the new GPG key's public half on GitHub (until then new signatures show Unverified) and remove the revoked `03F965C2E2DB5C6B` there; deleting the unused repo secrets `GH_TOKEN`, `TELEGRAM_BOT_TOKEN`, `DISCUSSION_*`.
**v4.0.0** (2026-09-18) moved the site to the Cloudflare Worker at https://freeitchgames.win (PRs #81, #82, #88). The whole chain is live: Access for `/admin`, the GitHub App (verified commits to `scripts/temp_link.json` → `update.yml`), Turnstile, `ADMIN_EMAILS`; D1 `freeitchgames`, R2 `freeitchgames-thumbs`, Image Transformations (source `img.itch.zone`); Web Analytics is injected by the zone. **Still open (owner):** the legal-text review items; add the new GPG key's public half on GitHub (until then new signatures show Unverified) and remove the revoked `03F965C2E2DB5C6B` there; delete the Zero Trust Access app "Ingest" and its service token (unused since 4.4.1).
+1 -2
View File
@@ -18,11 +18,10 @@ Got a whole list? The **[Add Games (bulk list)]** issue template takes up to 15
#### 1b. Other ways games get in
- The Maintainer's companion browser extension ([itch-f2p-extension](https://github.com/poli0981/itch-f2p-extension), separate repo, GPL-3.0) sends links through an authenticated API while browsing itch.io. It is the Maintainer's own tool, not a public submission channel.
- The site reads itch.io's public RSS feeds of new and popular free games every 4 hours (one feed per run) and queues what it finds.
- The site also reads the public itch.io collections the Maintainer follows and queues their new free games directly (one removed before, or with a warning flag, goes to the review queue instead).
The extension and the RSS feeds land in the same review queue as the Suggest page. (The old Telegram bot path is retired.)
The RSS feeds land in the same review queue as the Suggest page. (The Maintainer's browser extension and the old Telegram bot are retired; their repositories, [itch-f2p-extension](https://github.com/poli0981/itch-f2p-extension) and [telegram-scraper-bot](https://github.com/poli0981/telegram-scraper-bot), are archived.)
### 2. Remove Games or Fix Their Data
+1 -1
View File
@@ -45,7 +45,7 @@ The process is described in the [Terms of Use](docs/ToS.md).
| **Catalog data** | `data_game/`, `scripts/deleted_games.json`, and the files built from them under <https://freeitchgames.win/data> | CC BY 4.0, for the Maintainer's contribution only. Creator descriptions, cover images, game names and trademarks are **excluded** | [`data_game/LICENSE.md`](data_game/LICENSE.md), [`LICENSES/CC-BY-4.0.txt`](LICENSES/CC-BY-4.0.txt) |
| **Documentation** | `*.md` files in the repository root and in `docs/` (README, policies, guides and their Vietnamese translations) | CC BY 4.0 | [`LICENSES/CC-BY-4.0.txt`](LICENSES/CC-BY-4.0.txt) |
| **Third-party components** | Libraries, frameworks and tools the project uses | Their own licenses | [`docs/THIRD_PARTY.md`](docs/THIRD_PARTY.md) and the [About page](https://freeitchgames.win/about) |
| **Browser extension** | [poli0981/itch-f2p-extension](https://github.com/poli0981/itch-f2p-extension), a separate repository | GPL-3.0 | That repository |
| **Browser extension** (archived) | [poli0981/itch-f2p-extension](https://github.com/poli0981/itch-f2p-extension), a separate repository, archived and no longer used | GPL-3.0 | That repository |
When you reuse the documentation, credit poli0981 (SkullMute), link to this repository and to CC BY 4.0,
and say if you changed it. For the data, use the credit line in
+12 -11
View File
@@ -93,21 +93,22 @@ still gets security hardening and that we actually test on (≈87% of active dev
## Add a game
There are four ways a game gets into the catalog. Each is approved by the Maintainer, one by one in the
There are three ways a game gets into the catalog. Each is approved by the Maintainer, one by one in the
review queue or through automatic rules the Maintainer sets.
1. **Suggest page**: <https://freeitchgames.win/suggest>. Paste an itch.io game link and, if you like, a
short note (up to 500 characters; notes are deleted after 180 days). A Cloudflare Turnstile check and a
rate limit keep spam out. Please don't put personal data in the note.
2. **Browser extension**: the Maintainer's own
[itch-f2p-extension](https://github.com/poli0981/itch-f2p-extension) (separate repo, GPL-3.0) sends
links through an authenticated API while browsing itch.io.
3. **Automatic discovery**: every 4 hours the site reads one of itch.io's public RSS feeds of new and
2. **Automatic discovery**: every 4 hours the site reads one of itch.io's public RSS feeds of new and
popular free games (one feed per run) and queues what it finds.
4. **Followed collections**: the site also reads the public itch.io collections the Maintainer follows and
3. **Followed collections**: the site also reads the public itch.io collections the Maintainer follows and
queues their new free games directly (one removed before, or with a warning flag, goes to the review
queue instead).
The Maintainer's browser extension and the old Telegram bot are retired; their repositories
([itch-f2p-extension](https://github.com/poli0981/itch-f2p-extension),
[telegram-scraper-bot](https://github.com/poli0981/telegram-scraper-bot)) are archived.
The Maintainer reviews the queue in a private admin area (protected by Cloudflare Access); an item
without a warning flag that nobody rejected within a waiting time (24 hours by default) is approved
automatically. Approved links are handed to the data pipeline, which scrapes each game page; paid games
@@ -125,8 +126,7 @@ Details: [Terms of Use](docs/ToS.md) and [Disclaimer](docs/DISCLAIMER.md).
## How the data stays fresh
```
Suggest page ──┐
Extension ──┼─→ review queue ─→ Maintainer approves ─→ scripts/temp_link.json
Suggest page ──┬─→ review queue ─→ Maintainer approves ─→ scripts/temp_link.json
RSS feeds ──┘ (admin, or auto rules) │
Followed collections ─────────────────────────────────────────┤
▼
@@ -162,7 +162,7 @@ The pipeline is polite to itch.io: it identifies itself as
- **Web app** ([`webapp/src/`](webapp/src/)): React + TypeScript + Vite + Tailwind CSS, TanStack
Query, React Router, Zustand. English and Vietnamese UI.
- **Cloudflare Worker** ([`webapp/worker/`](webapp/worker/)): serves the site and `/data`, the verification gate
(Cloudflare Turnstile), the image proxy (`/img`, resized WebP covers stored on Cloudflare R2), the Suggest and ingest APIs, the review queue
(Cloudflare Turnstile), the image proxy (`/img`, resized WebP covers stored on Cloudflare R2), the Suggest API, the review queue
(Cloudflare D1), the admin API and the RSS discovery schedule. Config:
[`webapp/wrangler.jsonc`](webapp/wrangler.jsonc).
- **Admin app** ([`webapp/admin/`](webapp/admin/)): the Maintainer-only review screen at `/admin`, behind
@@ -334,8 +334,9 @@ Free itch.io games catalog by poli0981 (SkullMute) — https://freeitchgames.win
```
Third-party components keep their own licenses: see [`docs/THIRD_PARTY.md`](docs/THIRD_PARTY.md) and the
site's About page. The browser extension is a separate repo under GPL-3.0. Contributions are accepted under
the same license as the part they change (inbound = outbound): MIT for code, CC BY 4.0 for data and docs.
site's About page. The archived browser extension is a separate repo under GPL-3.0. Contributions are
accepted under the same license as the part they change (inbound = outbound): MIT for code, CC BY 4.0 for
data and docs.
**Not affiliated with itch.io.** This is an independent fan project. It is not affiliated with, endorsed by,
or sponsored by itch.io, Leaf Corcoran or itch corp. "itch.io" and all game names, logos and trademarks
+8 -8
View File
@@ -71,12 +71,13 @@ Lưu ý: **chỉ arm64-v8a** (mọi điện thoại từ ~2017; không hỗ tr
## Thêm game
Có bốn cách để một game vào danh mục. Mỗi game đều được Maintainer duyệt, từng game một trong hàng chờ duyệt hoặc qua các quy tắc tự động do Maintainer đặt ra.
Có ba cách để một game vào danh mục. Mỗi game đều được Maintainer duyệt, từng game một trong hàng chờ duyệt hoặc qua các quy tắc tự động do Maintainer đặt ra.
1. **Trang Suggest**: <https://freeitchgames.win/suggest>. Dán link game itch.io và, nếu muốn, một ghi chú ngắn (tối đa 500 ký tự; ghi chú bị xóa sau 180 ngày). Một bước kiểm tra Cloudflare Turnstile và giới hạn tần suất giúp chặn spam. Vui lòng không ghi dữ liệu cá nhân vào ghi chú.
2. **Extension trình duyệt**: [itch-f2p-extension](https://github.com/poli0981/itch-f2p-extension) của chính Maintainer (repo riêng, GPL-3.0) gửi link qua một API có xác thực trong lúc duyệt itch.io.
3. **Tự động phát hiện**: cứ 4 giờ một lần, website đọc một trong các RSS feed công khai của itch.io về game miễn phí mới và phổ biến (mỗi lượt một feed) rồi đưa những gì tìm được vào hàng chờ.
4. **Collection được theo dõi**: website cũng đọc các collection công khai trên itch.io mà Maintainer theo dõi và đưa thẳng game miễn phí mới của chúng vào hàng chờ ingest (game từng bị gỡ, hoặc có cờ cảnh báo, thì vào hàng chờ duyệt).
2. **Tự động phát hiện**: cứ 4 giờ một lần, website đọc một trong các RSS feed công khai của itch.io về game miễn phí mới và phổ biến (mỗi lượt một feed) rồi đưa những gì tìm được vào hàng chờ.
3. **Collection được theo dõi**: website cũng đọc các collection công khai trên itch.io mà Maintainer theo dõi và đưa thẳng game miễn phí mới của chúng vào hàng chờ ingest (game từng bị gỡ, hoặc có cờ cảnh báo, thì vào hàng chờ duyệt).
Extension trình duyệt của Maintainer và Telegram bot cũ đã ngừng hoạt động; repo của chúng ([itch-f2p-extension](https://github.com/poli0981/itch-f2p-extension), [telegram-scraper-bot](https://github.com/poli0981/telegram-scraper-bot)) đã được lưu trữ (archive).
Maintainer duyệt hàng chờ trong một khu vực quản trị riêng (được bảo vệ bởi Cloudflare Access); mục không có cờ cảnh báo mà không ai từ chối trong thời gian chờ (mặc định 24 giờ) được duyệt tự động. Link được duyệt sẽ được chuyển cho pipeline dữ liệu, pipeline này cào từng trang game; game trả phí bị loại.
@@ -87,8 +88,7 @@ Muốn **gỡ** một game, hoặc sửa dữ liệu của nó (nhất là các
## Dữ liệu được cập nhật thế nào
```
Trang Suggest ─┐
Extension ─┼─→ hàng chờ duyệt ─→ Maintainer duyệt ─→ scripts/temp_link.json
Trang Suggest ─┬─→ hàng chờ duyệt ─→ Maintainer duyệt ─→ scripts/temp_link.json
RSS feed ─┘ (admin, hoặc quy tắc tự động) │
Collection theo dõi ──────────────────────────────────────────┤
▼
@@ -110,7 +110,7 @@ Pipeline cư xử lịch sự với itch.io: nó tự xưng danh là `FreeItchGa
## Bên trong dự án
- **Web app** ([`webapp/src/`](webapp/src/)): React + TypeScript + Vite + Tailwind CSS, TanStack Query, React Router, Zustand. Giao diện tiếng Anh và tiếng Việt.
- **Cloudflare Worker** ([`webapp/worker/`](webapp/worker/)): phục vụ website và `/data`, cổng xác minh (Cloudflare Turnstile), proxy ảnh (`/img`, ảnh bìa WebP đã thu nhỏ lưu trên Cloudflare R2), API Suggest và ingest, hàng chờ duyệt (Cloudflare D1), API quản trị và lịch phát hiện qua RSS. Cấu hình: [`webapp/wrangler.jsonc`](webapp/wrangler.jsonc).
- **Cloudflare Worker** ([`webapp/worker/`](webapp/worker/)): phục vụ website và `/data`, cổng xác minh (Cloudflare Turnstile), proxy ảnh (`/img`, ảnh bìa WebP đã thu nhỏ lưu trên Cloudflare R2), API Suggest, hàng chờ duyệt (Cloudflare D1), API quản trị và lịch phát hiện qua RSS. Cấu hình: [`webapp/wrangler.jsonc`](webapp/wrangler.jsonc).
- **App quản trị** ([`webapp/admin/`](webapp/admin/)): màn hình duyệt chỉ dành cho Maintainer tại `/admin`, đứng sau Cloudflare Access. Nó ghi vào repo này dưới danh nghĩa một GitHub App.
- **Vỏ native** ([`webapp/src-tauri/`](webapp/src-tauri/)): Tauri 2 cho Windows, macOS, Linux và Android. Xem [`webapp/TAURI.md`](webapp/TAURI.md).
- **Pipeline dữ liệu** ([`scripts/`](scripts/), Python 3.14): scraper, refresh, patch/apply và validate, chạy bằng GitHub Actions.
@@ -263,7 +263,7 @@ Khi tái sử dụng dữ liệu, vui lòng ghi công như sau:
Free itch.io games catalog by poli0981 (SkullMute) — https://freeitchgames.win — CC BY 4.0
```
Các thành phần bên thứ ba giữ giấy phép riêng của chúng: xem [`docs/THIRD_PARTY.md`](docs/THIRD_PARTY.md) và trang About của website. Extension trình duyệt là một repo riêng theo GPL-3.0. Đóng góp được nhận theo cùng giấy phép với phần mà nó thay đổi (inbound = outbound): MIT cho mã nguồn, CC BY 4.0 cho dữ liệu và tài liệu.
Các thành phần bên thứ ba giữ giấy phép riêng của chúng: xem [`docs/THIRD_PARTY.md`](docs/THIRD_PARTY.md) và trang About của website. Extension trình duyệt (đã archive) là một repo riêng theo GPL-3.0. Đóng góp được nhận theo cùng giấy phép với phần mà nó thay đổi (inbound = outbound): MIT cho mã nguồn, CC BY 4.0 cho dữ liệu và tài liệu.
**Không liên kết với itch.io.** Đây là một dự án độc lập do người hâm mộ làm. Dự án không liên kết với, không được chứng thực hay tài trợ bởi itch.io, Leaf Corcoran hoặc itch corp. "itch.io" cùng mọi tên game, logo và nhãn hiệu thuộc về chủ sở hữu tương ứng.
+3 -3
View File
@@ -1,6 +1,6 @@
# Security Policy
Last updated: 2026-09-29
Last updated: 2026-10-16
Applies from: the release of version 4.0.0.
@@ -24,7 +24,7 @@ A good report includes:
**In scope**
- The website https://freeitchgames.win and its Cloudflare Worker, including the image proxy (`/img`), the Suggest API (`/api/suggest`), the ingest API used by the Maintainer's browser extension (`/api/ingest`), and the admin app and API (`/admin`, `/api/admin`).
- The website https://freeitchgames.win and its Cloudflare Worker, including the image proxy (`/img`), the Suggest API (`/api/suggest`), and the admin app and API (`/admin`, `/api/admin`).
- The data pipeline (`scripts/`) and the GitHub Actions workflows (`.github/workflows/`).
- The desktop apps and the Android APK published on [GitHub Releases](https://github.com/poli0981/free-games-itchio-list/releases).
- The published data (`data_game/`, `scripts/deleted_games.json`, and https://freeitchgames.win/data) — for example, a way to get content into the catalog without the Maintainer's approval.
@@ -37,7 +37,7 @@ A good report includes:
- Volumetric denial-of-service (DoS/DDoS) and load testing.
- Reports from automated scanners without a working proof of concept.
The browser extension (poli0981/itch-f2p-extension) lives in its own repository; report issues in the extension itself there. Issues in the ingest API it talks to are in scope here.
The browser extension (poli0981/itch-f2p-extension) and the Telegram bot (poli0981/telegram-scraper-bot) are archived and no longer supported, so they are out of scope.
## 3. Supported versions
+1 -1
View File
@@ -3,7 +3,7 @@
#
# Usage: bash bash/commit_push.sh PATCH.json "commit message"
#
# Every data writer (refresh, ingest, admin Worker, browser extension) pushes
# Every data writer (refresh, ingest, the admin Worker and its cron) pushes
# to main. Instead of rebasing whole-file JSON rewrites (which conflict), each
# attempt starts from a fresh origin/main and re-applies the URL-keyed patch;
# apply_patch.py is idempotent and validates before anything is committed.
+43 -1
View File
@@ -1,6 +1,48 @@
# Changelog — 4.4.x
Release notes for 4.4.0. Every version: [CHANGELOG.md](../../CHANGELOG.md).
Release notes for 4.4.0 and 4.4.1, newest first. Every version: [CHANGELOG.md](../../CHANGELOG.md).
## [4.4.1] - 2026-10-16 (Extension and bot retired)
The maintainer's browser extension and the old Telegram bot were hardly used, and both repositories
are now public archives (read-only):
[itch-f2p-extension](https://github.com/poli0981/itch-f2p-extension) and
[telegram-scraper-bot](https://github.com/poli0981/telegram-scraper-bot). This version removes what was
left of them here. Games still come in through the Suggest page, itch.io's RSS feeds and the itch.io
collections the maintainer follows.
### Upgrade notes
- **Nothing to accept again.** The Privacy Policy, Terms of Use, EULA, Disclaimer and Security Policy
(EN + VI) only lose their mentions of the extension and its API; nothing new is collected, so the
legal check does not ask again.
- **Desktop and Android apps**: install 4.4.1 over 4.4.0 as usual.
- **Running your own copy of the Worker?** The `RL_INGEST` rate limit and the `ACCESS_AUD_INGEST`
variable are gone from `wrangler.jsonc`; delete the Access application that protected
`/api/ingest` and its service token. No D1 migration: the `idempotency` table stays and is no longer
used.
### Removed
- **The ingest API** (`POST /api/ingest`), which the browser extension used to send links to the review
queue with an Access service token. It never received a game since 4.0; the address now answers 404
like any unknown API path.
- **Admin:** the "Browser extension" choice in the auto-approve settings and the "Extension" source in
the review queue.
- **About page:** the old note asking to send a Telegram numeric ID privately (that was for the bot).
The Telegram link stays as a way to reach the maintainer.
### Changed
- **Policies** (2026-10-16, EN + VI): the Privacy Policy no longer lists the extension as a source of
games or its duplicate-submission records (7 days); the Terms of Use, the EULA (also in the
installers) and the Security Policy no longer name the ingest API; the Disclaimer lists the three
remaining ways games arrive. The extension and the bot are out of the Security Policy's scope.
- **Docs**: the READMEs, contributing guide, notices, acknowledgements and operations guide describe
three ways games get in, and mark the extension and the bot as archived.
- **Dependencies**: Vite 8.3.4 (#131); Tauri 2.12.2 with `tauri-build` 2.7.2, `tauri-plugin-opener`
2.7.1 and `tauri-plugin-single-instance` 2.5.3 (#135); `serde_json` 1.0.154 (#136); Ruff 0.17 for
the pipeline's lint (#137).
## [4.4.0] - 2026-10-13 (Admin automation, bulk actions, smoother charts)
+2 -2
View File
@@ -27,9 +27,9 @@ The website, the Worker and the desktop / Android apps build on a stack of open-
When you add a new dep to `webapp/package.json`, also add it to the `THIRD_PARTY` array so the About page lists it, and to [`THIRD_PARTY.md`](THIRD_PARTY.md). House rules are in [CONTRIBUTING.md](../CONTRIBUTING.md).
## Browser extension — the other way in
## Retired helpers
[poli0981/itch-f2p-extension](https://github.com/poli0981/itch-f2p-extension) (GPL-3.0, separate repo) lets me queue itch.io games straight from the browser. It sends them through an authenticated API into the same review queue as the public [Suggest page](https://freeitchgames.win/suggest) and the RSS discovery. Nothing goes live until I approve it. (The old Telegram bot did this job in v3 and is now retired.)
Two side projects used to feed the catalog: the browser extension [poli0981/itch-f2p-extension](https://github.com/poli0981/itch-f2p-extension) (GPL-3.0) and, in v3, the Telegram bot [poli0981/telegram-scraper-bot](https://github.com/poli0981/telegram-scraper-bot) (MIT). Both were hardly used, so they are archived now (October 2026). Games come in through the public [Suggest page](https://freeitchgames.win/suggest), the RSS discovery and the itch.io collections I follow, and nothing goes live until it is approved.
## Find me / Support me
Real channels exist now (still introvert max level, replies slow):
+2 -2
View File
@@ -1,6 +1,6 @@
# Disclaimer
Last updated: 2026-10-13
Last updated: 2026-10-16
Applies from: the release of version 4.0.0.
@@ -31,7 +31,7 @@ The Project is independent. It is **not affiliated with, endorsed by, or sponsor
The Project **does not make, host, distribute, sell or endorse** the games it lists. Every game links to its own page on itch.io, and any download happens there, under itch.io's terms and the creator's own terms or license.
Games reach the catalog through the public Suggest page, the Maintainer's browser extension, itch.io's public RSS feeds, or public itch.io collections the Maintainer follows. Each one is approved by the Maintainer, one by one or through automatic rules the Maintainer sets. That approval is **not** a security audit, a quality check or an endorsement.
Games reach the catalog through the public Suggest page, itch.io's public RSS feeds, or public itch.io collections the Maintainer follows. Each one is approved by the Maintainer, one by one or through automatic rules the Maintainer sets. That approval is **not** a security audit, a quality check or an endorsement.
## 5. No warranty as to the games
+2 -2
View File
@@ -1,6 +1,6 @@
# End-User License Agreement (EULA)
Last updated: 2026-10-03
Last updated: 2026-10-16
Applies from: the release of version 4.2.0.
@@ -82,7 +82,7 @@ You agree not to:
- Present a modified build as an official release, for example by distributing it under the project's name and icon, or with the official app identifier `com.poli0981.freegamesitchio`, without clearly marking it as unofficial. Forks are welcome under the MIT License; just make it obvious they are Yours.
- Use the Maintainer's name, the GitHub handle `poli0981`, or the alias "SkullMute" to endorse or promote derivative works without prior written permission.
- Use the App, or a modified version of it, in a way that breaks the Website's [Terms of Use](ToS.md), for example automated bulk requests to the Website (the data is in the Repository), or attempts to reach the admin area or the ingest API without authorization.
- Use the App, or a modified version of it, in a way that breaks the Website's [Terms of Use](ToS.md), for example automated bulk requests to the Website (the data is in the Repository), or attempts to reach the admin area without authorization.
- Use the App or its installers to host, distribute or facilitate malware, phishing, or content that violates applicable law.
These rules are about how You use the App and the project's name and services. They sit alongside the MIT License and do not limit the rights it grants over the source code.
+5 -6
View File
@@ -1,6 +1,6 @@
# Privacy Policy
Last updated: 2026-10-13
Last updated: 2026-10-16
Applies from: 2026-10-13 on the Website, and from version 4.4.0 of the Apps.
@@ -36,7 +36,7 @@ The controller of the personal data described in this Policy is **the Maintainer
### 1.3 What this Policy does not cover
This Policy does not cover services that handle data on their own account: itch.io (every game link and every download goes there), GitHub for your GitHub account and your activity on github.com, Cloudflare's use of data for its own purposes as described in its privacy policy, the games themselves, or the Maintainer's browser extension, which is a separate project with its own documentation. Their own policies apply (links in section 4).
This Policy does not cover services that handle data on their own account: itch.io (every game link and every download goes there), GitHub for your GitHub account and your activity on github.com, Cloudflare's use of data for its own purposes as described in its privacy policy, or the games themselves. Their own policies apply (links in section 4).
## 2. What is processed and why
@@ -70,7 +70,7 @@ When you use the Suggest page (https://freeitchgames.win/suggest):
### 2.4 The review queue and the admin area
- **How games arrive**: suggestions from the Suggest page; submissions from the Maintainer's browser extension through an authenticated API (the game URL, an optional title and note, and the extension's service-token ID; the records that prevent duplicate submissions are kept for 7 days); automatic discovery from itch.io's public RSS feeds; and new free games in public itch.io collections the Maintainer follows (the Website reads those public pages; nothing about you is involved).
- **How games arrive**: suggestions from the Suggest page; automatic discovery from itch.io's public RSS feeds; and new free games in public itch.io collections the Maintainer follows (the Website reads those public pages; nothing about you is involved).
- **The review queue** (Cloudflare D1) holds candidate game URLs with their source, the title, cover-image URL and genre hint taken from itch.io, automatic flags, notes, timestamps and the Maintainer's decision. Games enter the Catalog when the Maintainer approves them: one by one, or through automatic rules the Maintainer sets and can change at any time (items without a warning flag that nobody rejected within a waiting time; new free games in collections the Maintainer follows). Automatic approvals are recorded in the audit log like the Maintainer's own. Rejected URLs are kept indefinitely as a blocklist; only the URL is needed to avoid reviewing the same game again.
- **The admin area** (https://freeitchgames.win/admin) is for the Maintainer only. It is protected by Cloudflare Access (GitHub login or a one-time PIN sent by email), which sets a `CF_Authorization` session cookie in the Maintainer's browser. Admin actions, including changes to the automatic rules, are recorded in an audit log together with the Maintainer's email address. The audit log is never published.
@@ -145,7 +145,6 @@ Cloudflare and Google may process data anywhere on their global networks (see se
| IP address used for rate limiting and Turnstile | Used transiently; never stored by the Project |
| Suggest note | Deleted automatically after **180 days** |
| Suggested or discovered game URLs, with source and timestamps | In the review queue while needed to avoid reviewing the same game twice. Approved URLs become part of the public Catalog; rejected URLs are kept indefinitely as a blocklist |
| Duplicate-submission records from the browser extension | **7 days** |
| Admin audit log (with the Maintainer's email address) | Kept as the record of admin changes; never published |
| Catalog entries (including developer and publisher names) | While the game is listed. Removal records stay in the public removed-games list, and the Repository's Git history keeps past versions (see section 7.3) |
| Emails, including removal and copyright requests | As long as needed to handle the request |
@@ -219,8 +218,8 @@ Under the GDPR and the UK GDPR, you can also ask for processing to be restricted
- The Website is served only over HTTPS.
- Automated traffic has to pass a Cloudflare Turnstile check before the Website's pages and cover images load (section 2.1).
- Data minimization: no accounts, no IP addresses stored by the Project's code, Suggest notes deleted after 180 days, duplicate-submission records after 7 days.
- The admin area and the ingest API are protected by Cloudflare Access, and the Website's server code verifies the Access token itself on every protected request.
- Data minimization: no accounts, no IP addresses stored by the Project's code, Suggest notes deleted after 180 days.
- The admin area is protected by Cloudflare Access, and the Website's server code verifies the Access token itself on every protected request.
- Secrets (such as the GitHub App's private key) are kept in Cloudflare's secret storage, never in the Repository. The admin area writes to the Repository as a GitHub App, with verified commits.
- A strict Content Security Policy limits scripts to the Website itself plus Cloudflare Web Analytics and Turnstile.
- GitHub Actions are pinned to exact commit SHAs with least-privilege permissions, and Dependabot keeps dependencies up to date.
+2 -2
View File
@@ -1,6 +1,6 @@
# Terms of Use
Last updated: 2026-10-13
Last updated: 2026-10-16
Applies from: 2026-10-13 on the Website, and from version 4.4.0 of the Apps.
@@ -62,7 +62,7 @@ You agree **not** to:
1. **Bulk-scrape** the Website or its API, including the files under `/data`. Get the data from the Repository instead; it is cheaper for everyone.
2. **Abuse the image proxy** (`/img`): it exists only to show cover images on the Website. Don't hotlink `/img` URLs from other sites or apps, and don't use it to fetch images in bulk.
3. **Access restricted areas without authorization**: `/admin`, the admin API (`/api/admin/…`) and the ingest API (`/api/ingest`) are for the Maintainer only. Good-faith security research is welcome under the rules of the [Security Policy](../SECURITY.md).
3. **Access restricted areas without authorization**: `/admin` and the admin API (`/api/admin/…`) are for the Maintainer only. Good-faith security research is welcome under the rules of the [Security Policy](../SECURITY.md).
4. **Spam the Suggest form**, or circumvent the Website's verification check, Turnstile or the rate limits (for example with scripts, shared verification cookies or rotating IP addresses).
5. **Submit links to malware**, scams, phishing, or illegal content, anywhere in the Project.
6. **Misrepresent affiliation**: don't claim to be the Maintainer or the Project, don't present a fork or copy as the official Project, don't claim that the Project is run or endorsed by itch.io, and don't use the names "poli0981" or "SkullMute" to endorse your own work without prior written permission.
+1 -2
View File
@@ -22,11 +22,10 @@ Có cả một danh sách? Issue template **[Add Games (bulk list)]** nhận t
#### 1b. Những cách khác để game được thêm vào
- Extension trình duyệt đi kèm của Maintainer ([itch-f2p-extension](https://github.com/poli0981/itch-f2p-extension), repo riêng, GPL-3.0) gửi link qua một API có xác thực trong lúc duyệt itch.io. Đây là công cụ riêng của Maintainer, không phải kênh gửi công khai.
- Website đọc các RSS feed công khai của itch.io về game miễn phí mới và phổ biến mỗi 4 giờ (mỗi lượt một feed) và đưa những gì tìm được vào hàng chờ.
- Website cũng đọc các collection công khai trên itch.io mà Maintainer theo dõi và đưa thẳng game miễn phí mới của chúng vào hàng chờ ingest (game từng bị gỡ, hoặc có cờ cảnh báo, thì vào hàng chờ duyệt).
Extension và RSS feed đều vào cùng hàng chờ duyệt với trang Suggest. (Đường thêm game qua Telegram bot cũ đã ngừng hoạt động.)
RSS feed vào cùng hàng chờ duyệt với trang Suggest. (Extension trình duyệt của Maintainer và Telegram bot cũ đã ngừng hoạt động; repo của chúng, [itch-f2p-extension](https://github.com/poli0981/itch-f2p-extension) và [telegram-scraper-bot](https://github.com/poli0981/telegram-scraper-bot), đã được lưu trữ (archive).)
### 2. Gỡ game hoặc sửa dữ liệu game
+2 -2
View File
@@ -1,6 +1,6 @@
# Tuyên bố miễn trừ trách nhiệm (Disclaimer)
Cập nhật lần cuối: 2026-10-13
Cập nhật lần cuối: 2026-10-16
Áp dụng từ: khi phát hành phiên bản 4.0.0.
@@ -33,7 +33,7 @@ Dự án hoạt động độc lập. Dự án **không liên kết với, khôn
Dự án **không làm, không lưu trữ, không phân phối, không bán và không chứng thực** các game được liệt kê. Mỗi game liên kết tới trang riêng của nó trên itch.io, và mọi lượt tải về đều diễn ra tại đó, theo điều khoản của itch.io và điều khoản hoặc giấy phép riêng của người làm game.
Game được đưa vào catalog qua trang Suggest công khai, tiện ích trình duyệt của Người duy trì, các RSS feed công khai của itch.io, hoặc các collection công khai trên itch.io mà Người duy trì theo dõi. Mỗi game đều được Người duy trì duyệt, từng game một hoặc qua các quy tắc tự động do Người duy trì đặt ra. Việc duyệt đó **không phải** là kiểm tra bảo mật, đánh giá chất lượng hay sự chứng thực.
Game được đưa vào catalog qua trang Suggest công khai, các RSS feed công khai của itch.io, hoặc các collection công khai trên itch.io mà Người duy trì theo dõi. Mỗi game đều được Người duy trì duyệt, từng game một hoặc qua các quy tắc tự động do Người duy trì đặt ra. Việc duyệt đó **không phải** là kiểm tra bảo mật, đánh giá chất lượng hay sự chứng thực.
## 5. Không bảo đảm về các game
+2 -2
View File
@@ -1,6 +1,6 @@
# Thỏa thuận cấp phép cho người dùng cuối (EULA)
Cập nhật lần cuối: 2026-10-03
Cập nhật lần cuối: 2026-10-16
Áp dụng từ: khi phát hành phiên bản 4.2.0.
@@ -86,7 +86,7 @@ Bạn đồng ý không:
- Trình bày một bản build đã sửa đổi như một bản phát hành chính thức, ví dụ phân phối nó dưới tên và biểu tượng của dự án, hoặc với app identifier chính thức `com.poli0981.freegamesitchio`, mà không ghi rõ đó là bản không chính thức. Fork luôn được chào đón theo Giấy phép MIT; chỉ cần làm rõ rằng đó là bản của Bạn.
- Sử dụng tên Người duy trì, GitHub handle `poli0981`, hoặc alias "SkullMute" để chứng thực hoặc quảng bá tác phẩm phái sinh mà không có sự cho phép trước bằng văn bản.
- Sử dụng Ứng dụng, hoặc một phiên bản đã sửa đổi của nó, theo cách vi phạm [Điều khoản sử dụng](ToS.md) của Website, ví dụ gửi yêu cầu tự động hàng loạt tới Website (dữ liệu có sẵn trong Repository), hoặc cố truy cập khu vực quản trị (admin) hay ingest API khi không được phép.
- Sử dụng Ứng dụng, hoặc một phiên bản đã sửa đổi của nó, theo cách vi phạm [Điều khoản sử dụng](ToS.md) của Website, ví dụ gửi yêu cầu tự động hàng loạt tới Website (dữ liệu có sẵn trong Repository), hoặc cố truy cập khu vực quản trị (admin) khi không được phép.
- Sử dụng Ứng dụng hoặc các bộ cài của nó để lưu trữ, phân phối hoặc tạo điều kiện phân phối malware, trang phishing, hoặc nội dung vi phạm pháp luật hiện hành.
Các quy tắc này là về cách Bạn sử dụng Ứng dụng cũng như tên và dịch vụ của dự án. Chúng tồn tại song song với Giấy phép MIT và không hạn chế các quyền mà giấy phép đó trao đối với mã nguồn.
+5 -6
View File
@@ -1,6 +1,6 @@
# Chính sách quyền riêng tư (Privacy Policy)
Cập nhật lần cuối: 2026-10-13
Cập nhật lần cuối: 2026-10-16
Áp dụng từ: ngày 2026-10-13 đối với Website, và từ phiên bản 4.4.0 đối với các Ứng dụng.
@@ -36,7 +36,7 @@ Bên kiểm soát dữ liệu cá nhân được mô tả trong Chính sách nà
### 1.3 Những gì Chính sách này không áp dụng
Chính sách này không áp dụng cho các dịch vụ tự xử lý dữ liệu với tư cách của chính họ: itch.io (mọi link game và mọi lượt tải đều dẫn tới đó), GitHub đối với tài khoản GitHub và hoạt động của bạn trên github.com, việc Cloudflare dùng dữ liệu cho mục đích riêng của họ như mô tả trong chính sách quyền riêng tư của Cloudflare, bản thân các game, và tiện ích trình duyệt của Người duy trì (một dự án riêng, có tài liệu riêng). Chính sách của từng bên sẽ được áp dụng (link ở mục 4).
Chính sách này không áp dụng cho các dịch vụ tự xử lý dữ liệu với tư cách của chính họ: itch.io (mọi link game và mọi lượt tải đều dẫn tới đó), GitHub đối với tài khoản GitHub và hoạt động của bạn trên github.com, việc Cloudflare dùng dữ liệu cho mục đích riêng của họ như mô tả trong chính sách quyền riêng tư của Cloudflare, và bản thân các game. Chính sách của từng bên sẽ được áp dụng (link ở mục 4).
## 2. Dữ liệu nào được xử lý và vì sao
@@ -70,7 +70,7 @@ Khi bạn dùng trang Suggest (https://freeitchgames.win/suggest):
### 2.4 Hàng chờ duyệt và khu vực quản trị
- **Game đến từ đâu**: đề xuất từ trang Suggest; dữ liệu gửi từ tiện ích trình duyệt của Người duy trì qua một API có xác thực (URL game, tiêu đề và ghi chú tùy chọn, và ID service token của tiện ích; các bản ghi chống gửi trùng được giữ 7 ngày); tự động phát hiện từ các RSS feed công khai của itch.io; và game miễn phí mới trong các collection công khai trên itch.io mà Người duy trì theo dõi (Website đọc các trang công khai đó; không liên quan gì tới dữ liệu của bạn).
- **Game đến từ đâu**: đề xuất từ trang Suggest; tự động phát hiện từ các RSS feed công khai của itch.io; và game miễn phí mới trong các collection công khai trên itch.io mà Người duy trì theo dõi (Website đọc các trang công khai đó; không liên quan gì tới dữ liệu của bạn).
- **Hàng chờ duyệt** (Cloudflare D1) chứa các URL game ứng viên cùng nguồn gửi, tiêu đề, URL ảnh bìa và gợi ý thể loại lấy từ itch.io, các cờ tự động, ghi chú, mốc thời gian và quyết định của Người duy trì. Game được đưa vào Danh mục khi Người duy trì duyệt: từng game một, hoặc qua các quy tắc tự động do Người duy trì đặt ra và có thể thay đổi bất cứ lúc nào (mục không có cờ cảnh báo mà không ai từ chối trong thời gian chờ; game miễn phí mới trong các collection Người duy trì theo dõi). Các lượt duyệt tự động được ghi vào nhật ký kiểm tra như thao tác của chính Người duy trì. Các URL bị từ chối được giữ vô thời hạn làm danh sách chặn; chỉ cần URL là đủ để không phải xét lại cùng một game.
- **Khu vực quản trị** (https://freeitchgames.win/admin) chỉ dành cho Người duy trì. Khu vực này được bảo vệ bằng Cloudflare Access (đăng nhập bằng GitHub hoặc mã PIN dùng một lần gửi qua email); Cloudflare Access đặt cookie phiên `CF_Authorization` trong trình duyệt của Người duy trì. Các thao tác quản trị, kể cả thay đổi quy tắc tự động, được ghi vào nhật ký kiểm tra (audit log) kèm địa chỉ email của Người duy trì. Nhật ký này không bao giờ được công bố.
@@ -145,7 +145,6 @@ Cloudflare và Google có thể xử lý dữ liệu ở bất kỳ đâu trên
| Địa chỉ IP dùng cho giới hạn tần suất và Turnstile | Chỉ dùng tạm thời; Dự án không bao giờ lưu |
| Ghi chú trên trang Suggest | Tự động xóa sau **180 ngày** |
| URL game được đề xuất hoặc được phát hiện, kèm nguồn và mốc thời gian | Nằm trong hàng chờ duyệt chừng nào còn cần để không phải xét cùng một game hai lần. URL được duyệt trở thành một phần của Danh mục công khai; URL bị từ chối được giữ vô thời hạn làm danh sách chặn |
| Bản ghi chống gửi trùng từ tiện ích trình duyệt | **7 ngày** |
| Nhật ký kiểm tra quản trị (kèm địa chỉ email của Người duy trì) | Được giữ làm hồ sơ các thay đổi quản trị; không bao giờ công bố |
| Mục trong Danh mục (bao gồm tên nhà phát triển và nhà phát hành) | Khi game còn được liệt kê. Bản ghi gỡ bỏ vẫn nằm trong danh sách game đã gỡ công khai, và lịch sử Git của Repository giữ các phiên bản trước (xem mục 7.3) |
| Email, bao gồm yêu cầu gỡ game và yêu cầu liên quan đến bản quyền | Trong thời gian cần thiết để xử lý yêu cầu |
@@ -219,8 +218,8 @@ Theo GDPR và UK GDPR, bạn cũng có thể yêu cầu hạn chế việc xử
- Website chỉ được phục vụ qua HTTPS.
- Truy cập tự động phải qua bước kiểm tra Cloudflare Turnstile trước khi các trang và ảnh bìa của Website được tải (mục 2.1).
- Tối thiểu hóa dữ liệu: không tài khoản, mã nguồn của Dự án không lưu địa chỉ IP, ghi chú Suggest bị xóa sau 180 ngày, bản ghi chống gửi trùng bị xóa sau 7 ngày.
- Khu vực quản trị và API ingest được bảo vệ bằng Cloudflare Access, và mã máy chủ của Website tự xác minh token Access ở mọi request được bảo vệ.
- Tối thiểu hóa dữ liệu: không tài khoản, mã nguồn của Dự án không lưu địa chỉ IP, ghi chú Suggest bị xóa sau 180 ngày.
- Khu vực quản trị được bảo vệ bằng Cloudflare Access, và mã máy chủ của Website tự xác minh token Access ở mọi request được bảo vệ.
- Các thông tin bí mật (như private key của GitHub App) được giữ trong kho lưu trữ bí mật của Cloudflare, không bao giờ nằm trong Repository. Khu vực quản trị ghi vào Repository dưới danh nghĩa một GitHub App, với các commit đã được xác minh (verified).
- Content Security Policy chặt chẽ chỉ cho phép script từ chính Website, cộng với Cloudflare Web Analytics và Turnstile.
- Các GitHub Actions được ghim theo commit SHA cụ thể với quyền tối thiểu cần thiết, và Dependabot giữ cho các dependency luôn được cập nhật.
+3 -3
View File
@@ -1,6 +1,6 @@
# Chính sách bảo mật (Security Policy)
Cập nhật lần cuối: 2026-09-29
Cập nhật lần cuối: 2026-10-16
Áp dụng từ: khi phát hành phiên bản 4.0.0.
@@ -26,7 +26,7 @@ Một báo cáo tốt nên có:
**Trong phạm vi**
- Website https://freeitchgames.win và Cloudflare Worker của nó, bao gồm proxy ảnh (`/img`), API Suggest (`/api/suggest`), API ingest mà tiện ích trình duyệt của Người duy trì sử dụng (`/api/ingest`), cùng ứng dụng và API quản trị (`/admin`, `/api/admin`).
- Website https://freeitchgames.win và Cloudflare Worker của nó, bao gồm proxy ảnh (`/img`), API Suggest (`/api/suggest`), cùng ứng dụng và API quản trị (`/admin`, `/api/admin`).
- Pipeline dữ liệu (`scripts/`) và các workflow GitHub Actions (`.github/workflows/`).
- Các ứng dụng desktop và file APK Android được phát hành trên [GitHub Releases](https://github.com/poli0981/free-games-itchio-list/releases).
- Dữ liệu được công bố (`data_game/`, `scripts/deleted_games.json` và https://freeitchgames.win/data) — ví dụ: một cách đưa nội dung vào catalog mà không cần Người duy trì phê duyệt.
@@ -39,7 +39,7 @@ Một báo cáo tốt nên có:
- Tấn công từ chối dịch vụ theo lưu lượng (DoS/DDoS) và kiểm thử tải.
- Báo cáo từ công cụ quét tự động mà không có proof of concept chạy được.
Tiện ích trình duyệt (poli0981/itch-f2p-extension) nằm trong repository riêng; lỗi của chính tiện ích xin báo ở đó. Lỗi trong API ingest mà tiện ích kết nối tới thì thuộc phạm vi ở đây.
Tiện ích trình duyệt (poli0981/itch-f2p-extension) và Telegram bot (poli0981/telegram-scraper-bot) đã được lưu trữ (archive) và không còn được hỗ trợ, nên nằm ngoài phạm vi.
## 3. Phiên bản được hỗ trợ
+2 -2
View File
@@ -1,6 +1,6 @@
# Điều khoản sử dụng (Terms of Use)
Cập nhật lần cuối: 2026-10-13
Cập nhật lần cuối: 2026-10-16
Áp dụng từ: ngày 2026-10-13 đối với Website, và từ phiên bản 4.4.0 đối với các Ứng dụng.
@@ -66,7 +66,7 @@ Bạn đồng ý **không**:
1. **Scrape hàng loạt** Website hoặc API của nó, kể cả các file trong `/data`. Hãy lấy dữ liệu từ Repository; cách đó tốn ít tài nguyên hơn cho tất cả mọi người.
2. **Lạm dụng image proxy** (`/img`): proxy này chỉ để hiển thị ảnh bìa trên Website. Đừng hotlink các URL `/img` từ website hay ứng dụng khác, và đừng dùng nó để tải ảnh hàng loạt.
3. **Truy cập trái phép các khu vực bị hạn chế**: `/admin`, API quản trị (`/api/admin/…`) và API ingest (`/api/ingest`) chỉ dành cho Người duy trì. Nghiên cứu bảo mật thiện chí luôn được hoan nghênh, theo các quy tắc trong [Security Policy](SECURITY.md).
3. **Truy cập trái phép các khu vực bị hạn chế**: `/admin` và API quản trị (`/api/admin/…`) chỉ dành cho Người duy trì. Nghiên cứu bảo mật thiện chí luôn được hoan nghênh, theo các quy tắc trong [Security Policy](SECURITY.md).
4. **Spam form Suggest**, hoặc tìm cách vượt qua bước xác minh của Website, Turnstile hay giới hạn tần suất (ví dụ bằng script, dùng chung cookie xác minh hoặc xoay vòng địa chỉ IP).
5. **Gửi liên kết tới malware**, lừa đảo, phishing hoặc nội dung bất hợp pháp, ở bất kỳ đâu trong Dự án.
6. **Mạo nhận liên kết**: đừng tự nhận là Người duy trì hay là Dự án, đừng giới thiệu một bản fork hay bản sao như thể đó là Dự án chính thức, đừng nói rằng Dự án do itch.io vận hành hoặc bảo trợ, và đừng dùng tên "poli0981" hay "SkullMute" để quảng bá sản phẩm của bạn khi chưa có sự cho phép trước bằng văn bản.
+9 -11
View File
@@ -8,11 +8,11 @@ secret: secrets are only ever typed into Cloudflare (`wrangler secret put`) or a
| Piece | Where | What it does |
|---|---|---|
| Worker `free-games-itchio-list` | Cloudflare Workers, custom domain `freeitchgames.win` | Serves the built app + the catalog packs `/data/pack/*` (static assets), the readable `/data/*.json` to apps 4.0–4.1.1 only ([`legacy-data.ts`](../webapp/worker/legacy-data.ts)), `/img` (resized covers), the verification gate (`/api/verify`), `/api/suggest`, `/api/ingest`, `/api/admin/*`, `/admin`, and runs the RSS cron. Config: [`webapp/wrangler.jsonc`](../webapp/wrangler.jsonc). |
| Worker `free-games-itchio-list` | Cloudflare Workers, custom domain `freeitchgames.win` | Serves the built app + the catalog packs `/data/pack/*` (static assets), the readable `/data/*.json` to apps 4.0–4.1.1 only ([`legacy-data.ts`](../webapp/worker/legacy-data.ts)), `/img` (resized covers), the verification gate (`/api/verify`), `/api/suggest`, `/api/admin/*`, `/admin`, and runs the RSS cron. Config: [`webapp/wrangler.jsonc`](../webapp/wrangler.jsonc). |
| Workers Builds | Cloudflare dashboard → the Worker → Settings → Build | Builds and deploys `main` on every push (data commits included). |
| R2 `freeitchgames-thumbs` | Cloudflare R2 | Resized WebP covers (`w160/…`, `w640/…`). |
| D1 `freeitchgames` | Cloudflare D1 | Review queue, feed state, audit log, idempotency keys ([`webapp/worker/migrations/`](../webapp/worker/migrations)). |
| Access apps "Admin" / "Ingest" | Cloudflare Zero Trust | Login for `/admin` + `/api/admin/*`; service token for `/api/ingest` (browser extension). |
| D1 `freeitchgames` | Cloudflare D1 | Review queue, feed state, audit log, automation settings, followed collections ([`webapp/worker/migrations/`](../webapp/worker/migrations); the `idempotency` table of the retired ingest API stays, empty). |
| Access app "Admin" | Cloudflare Zero Trust | Login for `/admin` + `/api/admin/*`. |
| Turnstile widget | Cloudflare Turnstile | The verification gate in front of the web app (action `gate`) and anti-spam on `/suggest` (action `suggest`). |
| Transform Rule "Verified bots skip the gate" | Cloudflare → Rules → Transform Rules | Adds `x-fig-known-bot: <GATE_BOT_TOKEN>` to requests from Cloudflare-verified bots, so search engines and link previews skip the gate (section 1, step 14). |
| GitHub App | GitHub → Settings → Developer settings | The Worker's identity for commits (queue approvals, edits, removals). |
@@ -49,17 +49,15 @@ Do these once (and again with the `-staging` names for the staging Worker).
- Access → Applications → Self-hosted **"Admin"**: `freeitchgames.win/admin`,
`freeitchgames.win/admin/*`, `freeitchgames.win/api/admin/*`; policy *Allow* → your email(s);
session 24 h. Copy its **AUD** tag.
- Self-hosted **"Ingest"**: `freeitchgames.win/api/ingest`; policy *Service Auth* → a service token
(Access → Service credentials) named e.g. `itch-f2p-extension`, with an expiry; copy the AUD tag.
The extension sends `CF-Access-Client-Id` / `CF-Access-Client-Secret`.
- 4.4.1 removed the browser extension's ingest API: delete the old self-hosted **"Ingest"** app
(`freeitchgames.win/api/ingest`) and its service token (Access → Service credentials) if they still exist.
- Staging: one self-hosted app covering all of `staging.freeitchgames.win`.
7. **Turnstile**: add a widget (Managed) for `freeitchgames.win` and `staging.freeitchgames.win`. The
same widget serves the verification gate (action `gate`) and the Suggest form (action `suggest`);
the Worker checks the token, the hostname and the action, so a token from one can't be used for
the other.
8. **Fill in `wrangler.jsonc` vars** (not secret) and commit: `ACCESS_TEAM_DOMAIN`
(`https://<team>.cloudflareaccess.com`, no trailing slash), `ACCESS_AUD_ADMIN`, `ACCESS_AUD_INGEST`,
`GH_APP_ID`, `GH_APP_INSTALLATION_ID`, `TURNSTILE_SITEKEY` — in both the top level and
(`https://<team>.cloudflareaccess.com`, no trailing slash), `ACCESS_AUD_ADMIN`, `GH_APP_ID`, `GH_APP_INSTALLATION_ID`, `TURNSTILE_SITEKEY` — in both the top level and
`env.staging`. Then `npm run cf:types`. The production values are filled in (2026-09-18). Change
them in the file, not only in the dashboard: every deploy — so every data commit — replaces the
Worker's plain-text variables with the file's values. Secrets are not affected.
@@ -84,7 +82,7 @@ Do these once (and again with the `-staging` names for the staging Worker).
12. **Zone settings**: SSL/TLS Full (strict), Always Use HTTPS, minimum TLS 1.2, TLS 1.3 on; Smart
Tiered Cache; Browser Cache TTL "Respect existing headers"; Rocket Loader and Email Obfuscation
**off** (they rewrite HTML and break the CSP); WAF managed rules on; one rate-limiting rule for
`/api/*`; Bot Fight Mode **off** (it can't be skipped for the extension or the apps; the
`/api/*`; Bot Fight Mode **off** (it can't be skipped for the apps; the
verification gate covers the web app instead). Keep HSTS
`preload` only if every subdomain (including `staging`) will always be HTTPS.
- Measured 2026-10-03: TLS 1.0/1.1 are refused, TLS 1.3 and HTTP/3 answer, `http://` redirects
@@ -150,8 +148,8 @@ Do these once (and again with the `-staging` names for the staging Worker).
policies don't claim them). Rulesets: target the default branch (block deletion and force
pushes); protect tags `v*`. Merge PRs with squash or a merge commit — never fast-forward.
3. **Secrets**: keep `DISCORD_CI_WEBHOOK`, the release webhooks and the Android keystore secrets.
`GH_TOKEN`, `TELEGRAM_BOT_TOKEN` and the three `DISCUSSION_*` secrets are still set but no workflow
uses them any more (checked 2026-09-24): delete them.
`GH_TOKEN`, `TELEGRAM_BOT_TOKEN` and the three `DISCUSSION_*` secrets, which no workflow used any
more, were deleted on 2026-10-16.
4. **Signing key**: the old key `03F965C2E2DB5C6B` was exposed; sign with the replacement key.
## 3. Releasing a version
+1 -1
View File
@@ -1,7 +1,7 @@
"""
canonical.py — One canonical form for itch.io game URLs.
Every ingest channel (extension, admin, Suggest form, RSS, workflow input) and
Every ingest channel (admin, Suggest form, RSS, collections, workflow input) and
the pipeline itself compare URLs through canonicalize(), so the same game can
never be queued twice under two spellings. The Worker ports this function; both
implementations are tested against tests/fixtures/url_vectors.json.
+2 -2
View File
@@ -3,8 +3,8 @@ patch.py — The hand-off format between pipeline scans and the commit step.
Scanners (refresh.py, update_info.py) never write the catalog directly. They
emit a patch; apply_patch.py applies it idempotently to whatever `main` looks
like at push time, so concurrent writers (other workflows, the admin Worker,
the browser extension) never overwrite each other.
like at push time, so concurrent writers (other workflows, the admin Worker
and its cron) never overwrite each other.
Patch v1:
{
+1 -1
View File
@@ -24,7 +24,7 @@ Flow:
which the admin page reads to follow a batch it queued.
apply_patch.py then applies the patch to the latest `main`, so links queued
while this ran (extension, admin) are never overwritten.
while this ran (admin, cron) are never overwritten.
Usage: python scripts/update_info.py --out patch.json [--deadline-min 50]
"""
+6 -8
View File
@@ -3,8 +3,8 @@
One codebase for everything people see of the catalog:
- the website **<https://freeitchgames.win>** — read-only for everyone (no accounts, no sign-in);
- its **Cloudflare Worker** — image proxy, the public Suggest form, the browser-extension ingest API,
the admin API and the scheduled RSS discovery;
- its **Cloudflare Worker** — image proxy, the public Suggest form, the admin API and the scheduled RSS
discovery;
- the **admin** app at `/admin/` — Maintainer only, behind Cloudflare Access;
- the **desktop and Android apps** — the same public app wrapped with Tauri 2 (see [`TAURI.md`](TAURI.md)).
@@ -145,7 +145,6 @@ Worker. It runs for `wrangler.jsonc` `assets.run_worker_first` (`/`, `/img/*`, `
| `/img/<160\|640>/<path>` | [worker/img.ts](worker/img.ts) | Resized WebP covers, stored in R2; only thumbnails that are in the catalog (`/data/urls.json`), upstream fixed to `img.itch.zone` |
| `/api/health` | [worker/index.ts](worker/index.ts) | Health check |
| `/api/suggest` | [worker/suggest.ts](worker/suggest.ts) | Suggest form: same-origin POST, per-IP rate limit, Turnstile, canonical itch.io URL, note ≤ 500 chars |
| `/api/ingest` | [worker/ingest.ts](worker/ingest.ts) | Browser extension: Access service token, `Idempotency-Key`, rate limit per token |
| `/api/admin/*` | [worker/admin.ts](worker/admin.ts) | Admin API (queue, check, add, track, edit and remove one or up to 100 games, restore, activity, settings, itch.io collections) |
| `/admin`, `/admin/*` | [worker/admin.ts](worker/admin.ts) | Admin app shell, after the Access check |
| `/data/*.json` (not `/data/pack/`) | [worker/legacy-data.ts](worker/legacy-data.ts) | The readable catalog JSON, passed only to apps 4.0–4.1.1 (Tauri `Origin`); everyone else gets 410 + the repository links |
@@ -156,13 +155,12 @@ Worker. It runs for `wrangler.jsonc` `assets.run_worker_first` (`/`, `/img/*`, `
bookkeeping settles queued rows with the pipeline's ingest outcomes (queued → ingested / failed);
[`automation.ts`](worker/automation.ts) applies the Maintainer's automatic rules (auto-approve of
unflagged items after the waiting time, new free games of followed collections, read with
[`collections.ts`](worker/collections.ts)) in **one** commit; then retention (idempotency records after
7 days, Suggest notes after 180).
[`collections.ts`](worker/collections.ts)) in **one** commit; then retention (Suggest notes after 180
days).
**Bindings**: `ASSETS`, `THUMBS` (R2 `freeitchgames-thumbs`), `DB` (D1 `freeitchgames`),
`RL_SUGGEST` / `RL_INGEST` / `RL_GATE` (rate limits). **Vars**: `SITE_ORIGIN`, `GITHUB_REPO`, `BOT_UA`, `GH_APP_ID`,
`GH_APP_INSTALLATION_ID`, `ACCESS_TEAM_DOMAIN`, `ACCESS_AUD_ADMIN`, `ACCESS_AUD_INGEST`,
`TURNSTILE_SITEKEY`. **Secrets**: `GH_APP_PRIVATE_KEY`, `ADMIN_EMAILS`, `TURNSTILE_SECRET`
`RL_SUGGEST` / `RL_GATE` (rate limits). **Vars**: `SITE_ORIGIN`, `GITHUB_REPO`, `BOT_UA`, `GH_APP_ID`,
`GH_APP_INSTALLATION_ID`, `ACCESS_TEAM_DOMAIN`, `ACCESS_AUD_ADMIN`, `TURNSTILE_SITEKEY`. **Secrets**: `GH_APP_PRIVATE_KEY`, `ADMIN_EMAILS`, `TURNSTILE_SECRET`
(`DEV_ADMIN_EMAIL` is for local runs only). Logs: Workers Observability, 10% head sampling.
`canonical.ts` and `catalog.ts` are ports of `scripts/canonical.py` and `scripts/data_store.py` +
+1 -1
View File
@@ -61,7 +61,7 @@ async function api<T>(method: string, path: string, body?: unknown): Promise<T>
}
export type CandidateStatus = 'pending' | 'rejected' | 'queued' | 'ingested' | 'failed'
export type CandidateSource = 'rss' | 'form' | 'ext' | 'admin'
export type CandidateSource = 'rss' | 'form' | 'admin'
export interface Candidate {
url: string
+2 -2
View File
@@ -18,7 +18,7 @@ const STATUSES: { id: CandidateStatus; label: string }[] = [
{ id: 'ingested', label: 'Ingested' },
]
const SOURCE: Record<Candidate['source'], string> = { rss: 'RSS', form: 'Suggest form', ext: 'Extension', admin: 'Admin' }
const SOURCE: Record<Candidate['source'], string> = { rss: 'RSS', form: 'Suggest form', admin: 'Admin' }
const FLAG: Record<string, { label: string; variant: 'destructive' | 'secondary' | 'outline' }> = {
previously_deleted: { label: 'previously removed', variant: 'destructive' },
@@ -226,7 +226,7 @@ export function QueuePage() {
</td>
<td className="py-2 text-xs text-muted-foreground">
{source(c)}
{c.submitter && c.source !== 'form' && c.source !== 'admin' && <div>{c.submitter}</div>}
{c.submitter && c.source === 'rss' && <div>{c.submitter}</div>}
{c.decided_by === 'auto' && <div>approved automatically</div>}
</td>
<td className="py-2 text-xs tabular-nums text-muted-foreground">{c.discovered_at.slice(0, 10)}</td>
-1
View File
@@ -11,7 +11,6 @@ import { Notice, type NoticeState } from './ui'
const SOURCES: { id: CandidateSource; label: string }[] = [
{ id: 'rss', label: 'RSS discovery' },
{ id: 'form', label: 'Suggest form' },
{ id: 'ext', label: 'Browser extension' },
]
export function SettingsPage() {
+2 -2
View File
@@ -1,12 +1,12 @@
{
"name": "webapp",
"version": "4.4.0",
"version": "4.4.1",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "webapp",
"version": "4.4.0",
"version": "4.4.1",
"license": "MIT",
"dependencies": {
"@fontsource-variable/geist": "^5.3.0",
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "webapp",
"private": true,
"version": "4.4.0",
"version": "4.4.1",
"license": "MIT",
"type": "module",
"engines": {
+1 -1
View File
@@ -1033,7 +1033,7 @@ dependencies = [
[[package]]
name = "free-games-itchio-webapp"
version = "4.4.0"
version = "4.4.1"
dependencies = [
"serde",
"serde_json",
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "free-games-itchio-webapp"
version = "4.4.0"
version = "4.4.1"
description = "Itch.io free games catalog viewer"
authors = ["poli0981"]
license = "MIT"
+2 -2
View File
@@ -2,7 +2,7 @@
\viewkind4\uc1\pard\f0\fs18
{\b\fs22 END-USER LICENSE AGREEMENT (EULA)}\par
Itch.io Free Games DB - desktop and Android apps\par
Last updated: 2026-10-03\par
Last updated: 2026-10-16\par
Applies from: the release of version 4.2.0.\par
\par
This EULA applies only to the desktop and Android apps of Itch.io Free Games DB: the installers (.msi, .exe, .dmg, .pkg, .app.tar.gz, .deb, .rpm, .AppImage) and the Android .apk published on GitHub Releases. It does NOT cover:\par
@@ -66,7 +66,7 @@ You accept this EULA by installing or using the App, for example by accepting it
You agree not to:\par
- Present a modified build as an official release, for example by distributing it under the project's name and icon, or with the official app identifier com.poli0981.freegamesitchio, without clearly marking it as unofficial. Forks are welcome under the MIT License; just make it obvious they are Yours.\par
- Use the Maintainer's name, the GitHub handle poli0981, or the alias "SkullMute" to endorse or promote derivative works without prior written permission.\par
- Use the App, or a modified version of it, in a way that breaks the Website's Terms of Use, for example automated bulk requests to the Website (the data is in the Repository), or attempts to reach the admin area or the ingest API without authorization.\par
- Use the App, or a modified version of it, in a way that breaks the Website's Terms of Use, for example automated bulk requests to the Website (the data is in the Repository), or attempts to reach the admin area without authorization.\par
- Use the App or its installers to host, distribute or facilitate malware, phishing, or content that violates applicable law.\par
These rules are about how You use the App and the project's name and services. They sit alongside the MIT License and do not limit the rights it grants over the source code.\par
\par
+1 -1
View File
@@ -1,7 +1,7 @@
{
"$schema": "https://schema.tauri.app/config/2",
"productName": "Itch.io Free Games DB",
"version": "4.4.0",
"version": "4.4.1",
"identifier": "com.poli0981.freegamesitchio",
"build": {
"beforeDevCommand": "npm run dev",
+2 -2
View File
@@ -8,7 +8,7 @@ export interface ThirdParty {
export const APP = {
name: 'Free Itch Games',
version: '4.4.0',
version: '4.4.1',
repo: 'https://github.com/poli0981/free-games-itchio-list',
license: 'code MIT · data CC BY 4.0',
} as const
@@ -72,7 +72,7 @@ export const SOCIAL_LINKS: SocialLink[] = [
{ platform: 'discord-repo', label: 'Discord — Repo discussion', handle: '#general', url: 'https://discord.gg/2aNR3aVt', group: 'community' },
{ platform: 'discord-game', label: 'Discord — Game chat', handle: '#general', url: 'https://discord.gg/kDM9GMu5vm', group: 'community' },
// Messaging — DM only; never post your Telegram numeric ID to public channels.
// Messaging
{ platform: 'telegram-user', label: 'Telegram (DM)', handle: '@SkullMute0011', url: 'https://t.me/SkullMute0011', group: 'messaging' },
// Support — mirrors .github/FUNDING.yml (single source of truth on GitHub side).
+1 -2
View File
@@ -75,7 +75,6 @@ export const en = {
"about.legal.viNote": "— Vietnamese translations of the policies above (community-readable, English remains the controlling version for legal interpretation).",
"about.social.allLinks": "Every channel in one place:",
"about.social.desc": "DMs open on most. Replies slow (introvert max level). Pick whichever channel fits.",
"about.social.messagingNote": "Please DM your Telegram numeric ID privately. Never post it to Discord/X/public channels.",
"about.social.paypalNote": "Receipt shows real legal name (DungDang212) — that’s me.",
"about.social.supportNote": "Mirrors .github/FUNDING.yml — same handles GitHub uses for the Sponsor button.",
"about.social.title": "Find me elsewhere",
@@ -335,7 +334,7 @@ export const en = {
"settings.appearance.scrollbarsDesc": "Scrolling still works with the mouse wheel, touchpad, touch and keyboard.",
"settings.appearance.desc": "Theme, language, row density and scrollbars. Saved in this browser only.",
"welcome.feature.open.text": "The whole catalog is open data under CC BY 4.0, in the GitHub repository. Reuse it, chart it, build on it.",
"about.legal.desc": "All policies are Markdown files in the repository — plain language, full terms. Updated 2026-10-13.",
"about.legal.desc": "All policies are Markdown files in the repository — plain language, full terms. Updated 2026-10-16.",
"games.filter.ai": "Generative AI",
"games.filter.access": "Accessibility",
"ai.none": "No AI used",
+1 -2
View File
@@ -77,7 +77,6 @@ export const vi: Record<MessageKey, string> = {
"about.legal.viNote": "— bản dịch tiếng Việt của các chính sách trên (để cộng đồng dễ đọc; bản tiếng Anh vẫn là bản có hiệu lực khi diễn giải pháp lý).",
"about.social.allLinks": "Mọi kênh ở một chỗ:",
"about.social.desc": "DM mở trên hầu hết các kênh. Trả lời chậm (hướng nội cấp tối đa). Chọn kênh nào tiện cho bạn.",
"about.social.messagingNote": "Vui lòng DM riêng Telegram numeric ID của bạn. Đừng bao giờ đăng nó lên Discord/X/kênh công khai.",
"about.social.paypalNote": "Biên lai hiển thị tên pháp lý thật (DungDang212) — chính là mình.",
"about.social.supportNote": "Phản chiếu .github/FUNDING.yml — cùng các handle GitHub dùng cho nút Sponsor.",
"about.social.title": "Tìm mình ở nơi khác",
@@ -337,7 +336,7 @@ export const vi: Record<MessageKey, string> = {
"settings.appearance.scrollbarsDesc": "Vẫn cuộn được bằng con lăn chuột, touchpad, cảm ứng và bàn phím.",
"settings.appearance.desc": "Giao diện, ngôn ngữ, độ dày dòng và thanh cuộn. Chỉ lưu trong trình duyệt này.",
"welcome.feature.open.text": "Toàn bộ danh mục là dữ liệu mở theo giấy phép CC BY 4.0, nằm trong repo GitHub. Bạn cứ dùng lại, vẽ biểu đồ hay xây dựng tiếp trên đó.",
"about.legal.desc": "Tất cả chính sách là file Markdown trong repo — dễ hiểu, đầy đủ điều khoản. Cập nhật 2026-10-13.",
"about.legal.desc": "Tất cả chính sách là file Markdown trong repo — dễ hiểu, đầy đủ điều khoản. Cập nhật 2026-10-16.",
"games.filter.ai": "AI tạo sinh",
"games.filter.access": "Hỗ trợ tiếp cận",
"ai.none": "Không dùng AI",
-3
View File
@@ -1,5 +1,4 @@
import {
Bot,
Bug,
CircleDollarSign,
Cloud,
@@ -61,7 +60,6 @@ const SOCIAL_ICON: Record<string, LucideIcon> = {
'discord-repo': MessagesSquare,
'discord-game': MessagesSquare,
'telegram-user': Send,
'telegram-bot': Bot,
'github-sponsors': HeartHandshake,
patreon: Heart,
kofi: Coffee,
@@ -116,7 +114,6 @@ function ThirdPartySection({ category }: { category: ThirdParty['category'] }) {
}
const GROUP_NOTE: Partial<Record<SocialGroup, MessageKey>> = {
messaging: 'about.social.messagingNote',
support: 'about.social.supportNote',
}
+2 -6
View File
@@ -1,11 +1,10 @@
/* eslint-disable */
// Generated by Wrangler by running `wrangler types --env-file=types.env --strict-vars=false` (hash: 0e8d5d41bc6d6c15e84d899a70e4324f)
// Generated by Wrangler by running `wrangler types --env-file=types.env --strict-vars=false` (hash: bb5186e82d126ae7e5c424393b53d523)
// Runtime types generated with workerd@1.20261006.1 2026-09-15
interface __BaseEnv_Env {
THUMBS: R2Bucket;
DB: D1Database;
RL_SUGGEST: RateLimit;
RL_INGEST: RateLimit;
RL_GATE: RateLimit;
ASSETS: Fetcher;
SITE_ORIGIN: string;
@@ -15,7 +14,6 @@ interface __BaseEnv_Env {
GH_APP_INSTALLATION_ID: string;
ACCESS_TEAM_DOMAIN: string;
ACCESS_AUD_ADMIN: string;
ACCESS_AUD_INGEST: string;
TURNSTILE_SITEKEY: string;
BOT_GATE: string;
BOT_GATE_TTL_HOURS: string;
@@ -28,7 +26,6 @@ declare namespace Cloudflare {
THUMBS: R2Bucket;
DB: D1Database;
RL_SUGGEST: RateLimit;
RL_INGEST: RateLimit;
RL_GATE: RateLimit;
ASSETS: Fetcher;
SITE_ORIGIN: string;
@@ -38,7 +35,6 @@ declare namespace Cloudflare {
GH_APP_INSTALLATION_ID: string;
ACCESS_TEAM_DOMAIN: string;
ACCESS_AUD_ADMIN: string;
ACCESS_AUD_INGEST: string;
TURNSTILE_SITEKEY: string;
BOT_GATE: string;
BOT_GATE_TTL_HOURS: string;
@@ -50,7 +46,7 @@ type StringifyValues<EnvType extends Record<string, unknown>> = {
[Binding in keyof EnvType]: EnvType[Binding] extends string ? EnvType[Binding] : string;
};
declare namespace NodeJS {
interface ProcessEnv extends StringifyValues<Pick<Cloudflare.Env, "SITE_ORIGIN" | "GITHUB_REPO" | "BOT_UA" | "GH_APP_ID" | "GH_APP_INSTALLATION_ID" | "ACCESS_TEAM_DOMAIN" | "ACCESS_AUD_ADMIN" | "ACCESS_AUD_INGEST" | "TURNSTILE_SITEKEY" | "BOT_GATE" | "BOT_GATE_TTL_HOURS">> {}
interface ProcessEnv extends StringifyValues<Pick<Cloudflare.Env, "SITE_ORIGIN" | "GITHUB_REPO" | "BOT_UA" | "GH_APP_ID" | "GH_APP_INSTALLATION_ID" | "ACCESS_TEAM_DOMAIN" | "ACCESS_AUD_ADMIN" | "TURNSTILE_SITEKEY" | "BOT_GATE" | "BOT_GATE_TTL_HOURS">> {}
}
declare module "*.txt" {
const value: string;
+2 -20
View File
@@ -1,12 +1,11 @@
import { SignJWT, exportJWK, generateKeyPair } from 'jose'
import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest'
import { AccessError, requireMaintainer, requireService, type AccessEnv } from './access'
import { AccessError, requireMaintainer, type AccessEnv } from './access'
const TEAM = 'https://example.cloudflareaccess.com'
const env: AccessEnv = {
ACCESS_TEAM_DOMAIN: TEAM,
ACCESS_AUD_ADMIN: 'aud-admin',
ACCESS_AUD_INGEST: 'aud-ingest',
ADMIN_EMAILS: 'Owner@Example.com, second@example.com',
SITE_ORIGIN: 'https://freeitchgames.win',
}
@@ -60,7 +59,7 @@ describe('requireMaintainer', () => {
.setExpirationTime('5m')
.sign(other.privateKey)
await expect(requireMaintainer(request(forged), env)).rejects.toThrow('invalid')
const wrongAud = await token({ email: 'owner@example.com' }, 'aud-ingest')
const wrongAud = await token({ email: 'owner@example.com' }, 'aud-other')
await expect(requireMaintainer(request(wrongAud), env)).rejects.toThrow('invalid')
const wrongIss = await token({ email: 'owner@example.com' }, 'aud-admin', 'https://evil.example')
await expect(requireMaintainer(request(wrongIss), env)).rejects.toThrow('invalid')
@@ -84,20 +83,3 @@ describe('requireMaintainer', () => {
await expect(requireMaintainer(request(jwt, 'POST', env.SITE_ORIGIN), env)).resolves.toBeTruthy()
})
})
describe('requireService', () => {
it('accepts a service token and names it by client id', async () => {
const jwt = await token({ common_name: 'client-123.access' }, 'aud-ingest')
await expect(requireService(request(jwt, 'POST'), env)).resolves.toEqual({
kind: 'service',
id: 'client-123.access',
})
})
it('rejects user tokens and admin-audience tokens', async () => {
const user = await token({ email: 'owner@example.com', common_name: 'x' }, 'aud-ingest')
await expect(requireService(request(user), env)).rejects.toThrow('not a service token')
const adminAud = await token({ common_name: 'client-123.access' }, 'aud-admin')
await expect(requireService(request(adminAud), env)).rejects.toThrow('invalid')
})
})
+3 -14
View File
@@ -1,6 +1,5 @@
/**
* Cloudflare Access verification for /admin and /api/admin/* (maintainer) and
* /api/ingest (service tokens, e.g. the browser extension).
* Cloudflare Access verification for /admin and /api/admin/* (maintainer).
*
* Access sits in front of those paths, but a Worker with static assets does
* not receive `ctx.access`, and requests that bypass the zone (workers.dev,
@@ -13,14 +12,13 @@ import { createRemoteJWKSet, jwtVerify, type JWTPayload } from 'jose'
export interface AccessEnv {
ACCESS_TEAM_DOMAIN: string // https://<team>.cloudflareaccess.com
ACCESS_AUD_ADMIN: string
ACCESS_AUD_INGEST: string
ADMIN_EMAILS: string // comma-separated; a secret, not in the repo
SITE_ORIGIN: string
}
export interface Identity {
kind: 'maintainer' | 'service'
/** Email for people, service-token client id (common_name) for services. */
kind: 'maintainer'
/** The maintainer's email. */
id: string
}
@@ -68,12 +66,3 @@ export async function requireMaintainer(request: Request, env: AccessEnv): Promi
}
return { kind: 'maintainer', id: email }
}
/** A service token of the "Ingest" Access application (Service Auth policy). */
export async function requireService(request: Request, env: AccessEnv): Promise<Identity> {
const payload = await verify(request, env, env.ACCESS_AUD_INGEST)
// Service-token JWTs carry the client id as common_name and no email.
const id = String(payload.common_name ?? '')
if (!id || payload.email) throw new AccessError('not a service token')
return { kind: 'service', id }
}
+1 -1
View File
@@ -8,7 +8,7 @@
* - Writes: GraphQL `createCommitOnBranch` with `expectedHeadOid`. Commits made
* by an app through the API are signed by GitHub ("Verified"), and the
* expected head makes a write fail instead of silently overwriting a commit
* another writer (pipeline, extension) pushed in between; callers retry.
* another writer (the data workflows) pushed in between; callers retry.
*/
import { SignJWT, importPKCS8 } from 'jose'
-5
View File
@@ -115,11 +115,6 @@ export async function readJson(request: Request, maxBytes: number): Promise<unkn
}
}
export async function sha256Hex(text: string): Promise<string> {
const digest = await crypto.subtle.digest('SHA-256', new TextEncoder().encode(text))
return [...new Uint8Array(digest)].map((b) => b.toString(16).padStart(2, '0')).join('')
}
export function isObject(v: unknown): v is Record<string, unknown> {
return typeof v === 'object' && v !== null && !Array.isArray(v)
}
-10
View File
@@ -9,7 +9,6 @@
* /img/* resized covers, gated too (img.ts)
* /api/verify verification gate status + Turnstile check (gate.ts)
* /api/suggest public suggestion form (suggest.ts)
* /api/ingest browser extension, Access service token (ingest.ts)
* /api/admin/* maintainer API, Access + JWT check (admin.ts)
* /admin, /admin/* admin app shell, maintainer only (admin.ts)
* /data/*.json readable catalog JSON, for apps 4.0–4.1.1 only (legacy-data.ts)
@@ -26,7 +25,6 @@ import type { WorkerEnv } from './env'
import { checkPass, gateConfig, gateNavigation, gateRequired, handleVerify } from './gate'
import { json } from './http'
import { handleImg } from './img'
import { accessService, handleIngest } from './ingest'
import { handleLegacyData } from './legacy-data'
import { QueueStore } from './queue'
import { repoOps } from './repo'
@@ -67,14 +65,6 @@ export default {
limiter: env.RL_SUGGEST,
})
}
if (path === '/api/ingest') {
return handleIngest(request, {
store: queueStore(env),
data,
limiter: env.RL_INGEST,
authenticate: accessService(env),
})
}
if (path.startsWith('/api/admin/')) return handleAdminApi(request, url, env, queueStore(env))
if (path.startsWith('/api/')) return json({ error: 'not_found' }, 404)
if (path === '/admin' || path.startsWith('/admin/')) return serveAdminApp(request, url, env)
-133
View File
@@ -1,133 +0,0 @@
/**
* POST /api/ingest — machine submissions (the browser extension) into the
* review queue. Replaces the extension's direct commits with a GitHub PAT.
*
* Auth: a Cloudflare Access service token of the "Ingest" application
* (the client sends CF-Access-Client-Id/Secret; Access adds the JWT that
* requireService verifies). The submitter recorded is the token's client
* id, never something from the body.
*
* Body (≤ 64 KB, ≤ 150 items), with a required Idempotency-Key header:
* {"items": [{"url": "...", "title"?: "...", "note"?: "..."}]} or {"urls": ["..."]}
* A retry with the same key and body gets the stored answer; the same key
* with a different body is refused. Kept for 7 days.
*/
import { AccessError, requireService, type AccessEnv } from './access'
import { canonicalize } from './canonical'
import { catalogUrls, deletedGames, type DataSource } from './data'
import { ACCESS_CONFIG, missingConfig, type WorkerEnv } from './env'
import { HttpError, errorResponse, isObject, json, readBodyCapped, sha256Hex } from './http'
import { autoFlags, classify, type QueueStore } from './queue'
const MAX_BYTES = 64 * 1024
const MAX_ITEMS = 150
const KEY = /^[A-Za-z0-9_-]{8,128}$/
interface Item {
url: string
title: string | null
note: string | null
}
function text(value: unknown, max: number): string | null {
return typeof value === 'string' && value.trim() ? value.trim().slice(0, max) : null
}
function parseItems(body: unknown): Item[] {
if (!isObject(body)) throw new HttpError(400, 'invalid_body')
const raw = Array.isArray(body.items) ? body.items : Array.isArray(body.urls) ? body.urls : null
if (!raw || raw.length === 0) throw new HttpError(400, 'invalid_body', 'Send `items` or `urls`')
if (raw.length > MAX_ITEMS) throw new HttpError(413, 'too_many_items', `At most ${MAX_ITEMS} items`)
return raw.map((entry: unknown) => {
if (typeof entry === 'string') return { url: entry, title: null, note: null }
if (isObject(entry) && typeof entry.url === 'string') {
return { url: entry.url, title: text(entry.title, 200), note: text(entry.note, 500) }
}
throw new HttpError(400, 'invalid_item')
})
}
export interface IngestDeps {
store: QueueStore | null
data: DataSource
limiter: Pick<RateLimit, 'limit'> | undefined
/** Verifies the Access JWT; returns the service-token client id. */
authenticate: (request: Request) => Promise<string>
}
export function accessService(env: WorkerEnv) {
return async (request: Request): Promise<string> => {
if (missingConfig(env, [...ACCESS_CONFIG, 'ACCESS_AUD_INGEST']).length > 0) {
throw new HttpError(503, 'not_configured')
}
try {
return (await requireService(request, env as WorkerEnv & AccessEnv)).id
} catch (e) {
if (e instanceof AccessError) throw new HttpError(403, 'forbidden')
throw e
}
}
}
export async function handleIngest(request: Request, deps: IngestDeps): Promise<Response> {
try {
if (request.method !== 'POST') throw new HttpError(405, 'method_not_allowed')
const client = await deps.authenticate(request)
if (!deps.store) throw new HttpError(503, 'not_configured')
if (deps.limiter && !(await deps.limiter.limit({ key: client })).success) {
throw new HttpError(429, 'rate_limited')
}
const key = request.headers.get('idempotency-key') ?? ''
if (!KEY.test(key)) throw new HttpError(400, 'idempotency_key_required')
const raw = new TextDecoder().decode(await readBodyCapped(request, MAX_BYTES))
const bodyHash = await sha256Hex(raw)
const previous = await deps.store.idempotent(client, key)
if (previous) {
if (previous.body_hash !== bodyHash) throw new HttpError(422, 'idempotency_key_reused')
return json(JSON.parse(previous.response), 200, { 'Idempotent-Replayed': 'true' })
}
let body: unknown
try {
body = JSON.parse(raw)
} catch {
throw new HttpError(400, 'invalid_json')
}
const items = parseItems(body)
const canonical = items.map((i) => canonicalize(i.url)).filter((u): u is string => u !== null)
const [catalog, deleted, known] = await Promise.all([
catalogUrls(deps.data),
deletedGames(deps.data),
deps.store.statuses(canonical),
])
const results = classify(
items.map((i) => i.url),
{ catalog: new Set(catalog.keys()), deleted, known },
)
const fresh = results.flatMap((r, i) =>
r.store && r.canonical
? [
{
url: r.canonical,
source: 'ext' as const,
title: items[i].title,
note: items[i].note,
submitter: client,
flags: autoFlags(r.outcome, items[i].title),
},
]
: [],
)
await deps.store.insert(fresh)
const response = JSON.stringify({
received: fresh.length,
results: results.map((r) => ({ url: r.input, canonical: r.canonical, outcome: r.outcome })),
})
await deps.store.remember(client, key, bodyHash, response)
return json(JSON.parse(response))
} catch (e) {
return errorResponse(e)
}
}
+6 -24
View File
@@ -8,7 +8,8 @@
*/
import { canonicalize } from './canonical'
export type CandidateSource = 'rss' | 'form' | 'ext' | 'admin'
/** The table's CHECK still allows 'ext' (the browser extension, retired in 4.4.1); nothing writes it. */
export type CandidateSource = 'rss' | 'form' | 'admin'
export type CandidateStatus = 'pending' | 'rejected' | 'queued' | 'ingested' | 'failed'
/**
@@ -167,7 +168,6 @@ export interface AuditRow {
/** D1 binds at most 100 parameters per statement. */
const PARAM_SLICE = 90
const QUEUED_TIMEOUT_MS = 3 * 86_400_000
const IDEMPOTENCY_TTL_MS = 7 * 86_400_000
const FORM_NOTE_TTL_MS = 180 * 86_400_000
export function isoSeconds(date: Date): string {
@@ -188,7 +188,7 @@ function safeJson(text: string): unknown {
}
}
export const AUTOMATION_SOURCES: readonly CandidateSource[] = ['rss', 'form', 'ext']
export const AUTOMATION_SOURCES: readonly CandidateSource[] = ['rss', 'form']
/** Stored settings with every field checked; anything unusable falls back to `fallback`. */
function normalizeAutomation(value: unknown, fallback: AutomationSettings): AutomationSettings {
@@ -487,29 +487,11 @@ export class QueueStore {
return results
}
/** Retention: idempotency records after 7 days, Suggest-form notes after 180. */
/** Retention: Suggest-form notes after 180 days. */
async purge(now = new Date()): Promise<void> {
await this.db.batch([
this.db
.prepare('DELETE FROM idempotency WHERE created_at < ?')
.bind(isoSeconds(new Date(now.getTime() - IDEMPOTENCY_TTL_MS))),
this.db
.prepare("UPDATE candidates SET note = NULL WHERE source = 'form' AND note IS NOT NULL AND discovered_at < ?")
.bind(isoSeconds(new Date(now.getTime() - FORM_NOTE_TTL_MS))),
])
}
async idempotent(client: string, key: string): Promise<{ body_hash: string; response: string } | null> {
return this.db
.prepare('SELECT body_hash, response FROM idempotency WHERE client = ? AND key = ?')
.bind(client, key)
.first<{ body_hash: string; response: string }>()
}
async remember(client: string, key: string, bodyHash: string, response: string, now = new Date()): Promise<void> {
await this.db
.prepare('INSERT OR IGNORE INTO idempotency (client, key, body_hash, response, created_at) VALUES (?, ?, ?, ?, ?)')
.bind(client, key, bodyHash, response, isoSeconds(now))
.prepare("UPDATE candidates SET note = NULL WHERE source = 'form' AND note IS NOT NULL AND discovered_at < ?")
.bind(isoSeconds(new Date(now.getTime() - FORM_NOTE_TTL_MS)))
.run()
}
+2 -3
View File
@@ -12,7 +12,6 @@ const env = {
SITE_ORIGIN: ORIGIN,
ACCESS_TEAM_DOMAIN: TEAM,
ACCESS_AUD_ADMIN: 'aud-admin',
ACCESS_AUD_INGEST: 'aud-ingest',
ADMIN_EMAILS: 'owner@example.com',
ASSETS: {
fetch: async (req: Request) =>
@@ -73,8 +72,8 @@ describe('admin authentication', () => {
expect(await res.text()).toBe('')
})
it('refuses a token minted for the ingest application', async () => {
expect((await get('/admin/', await jwt('owner@example.com', 'aud-ingest'))).status).toBe(403)
it('refuses a token minted for another Access application', async () => {
expect((await get('/admin/', await jwt('owner@example.com', 'aud-other'))).status).toBe(403)
})
it('answers 503 until Access is configured, never serving the shell', async () => {
+1
View File
@@ -342,6 +342,7 @@ describe('admin API', () => {
for (const bad of [
{ enabled: 'yes', sources: [], hours: 1 },
{ enabled: true, sources: ['admin'], hours: 1 },
{ enabled: true, sources: ['ext'], hours: 1 }, // the retired browser extension
{ enabled: true, sources: ['rss', 'rss'], hours: 1 },
{ enabled: true, sources: [], hours: 0 },
{ enabled: true, sources: [], hours: 1.5 },
+19 -20
View File
@@ -12,7 +12,7 @@ beforeAll(async () => {
afterAll(() => dispose())
beforeEach(async () => {
await db.batch(
['candidates', 'feed_state', 'audit_log', 'idempotency', 'settings', 'watched_collections'].map((t) =>
['candidates', 'feed_state', 'audit_log', 'settings', 'watched_collections'].map((t) =>
db.prepare(`DELETE FROM ${t}`),
),
)
@@ -46,10 +46,10 @@ describe('QueueStore', () => {
expect((await store.get([url('a')]))[0]).toMatchObject({ status: 'rejected', source: 'rss', title: 'Old' })
await store.setStatus([url('a')], 'failed', 'reconcile', hours(2))
await store.insert([{ url: url('a'), source: 'ext', note: 'again' }], hours(3))
await store.insert([{ url: url('a'), source: 'admin', note: 'again' }], hours(3))
expect((await store.get([url('a')]))[0]).toMatchObject({
status: 'pending',
source: 'ext',
source: 'admin',
title: 'Old', // kept when the new submission has none
note: 'again',
decided_at: null,
@@ -116,7 +116,7 @@ describe('QueueStore', () => {
[
{ url: url('ok'), source: 'rss' },
{ url: url('form'), source: 'form' },
{ url: url('ext'), source: 'ext' },
{ url: url('admin'), source: 'admin' },
{ url: url('flagged'), source: 'rss', flags: ['demo'] },
],
hours(1),
@@ -146,6 +146,15 @@ describe('QueueStore', () => {
expect(await store.lastRun()).toEqual({ approved: 3 })
})
it('drops sources it no longer knows from stored settings (the retired extension)', async () => {
const stored = { enabled: true, sources: ['rss', 'ext'], hours: 8, since: '2026-09-18T00:00:00Z' }
await db
.prepare("INSERT INTO settings (key, value, updated_at) VALUES ('automation', ?, ?)")
.bind(JSON.stringify(stored), '2026-09-18T00:00:00Z')
.run()
expect(await store.automation(hours(1))).toEqual({ ...stored, sources: ['rss'] })
})
it('keeps watched collections and picks the due ones, longest-unread first', async () => {
await store.watch('1', 'one', 'me', T0)
await store.watch('2', 'two', 'me', hours(1))
@@ -178,25 +187,15 @@ describe('QueueStore', () => {
expect(older.map((r) => r.id)).toEqual([all[4].id])
})
it('purges old idempotency records and old Suggest-form notes', async () => {
await store.remember('ext', 'key-old-0001', 'h', '{}', T0)
await store.remember('ext', 'key-new-0001', 'h', '{}', hours(24 * 6))
it('purges Suggest-form notes after 180 days, and only those', async () => {
await store.insert([{ url: url('form'), source: 'form', note: 'contact me' }], T0)
await store.insert([{ url: url('ext'), source: 'ext', note: 'kept' }], T0)
await store.insert([{ url: url('recent'), source: 'form', note: 'still here' }], hours(24 * 2))
await store.insert([{ url: url('admin'), source: 'admin', note: 'kept' }], T0)
await store.purge(hours(24 * 181))
expect(await store.idempotent('ext', 'key-old-0001')).toBeNull()
expect(await store.idempotent('ext', 'key-new-0001')).toBeNull() // 175 days old by then
const notes = new Map((await store.get([url('form'), url('ext')])).map((r) => [r.url, r.note]))
const notes = new Map((await store.get([url('form'), url('recent'), url('admin')])).map((r) => [r.url, r.note]))
expect(notes.get(url('form'))).toBeNull()
expect(notes.get(url('ext'))).toBe('kept')
})
it('keeps idempotency records per client and ignores a duplicate insert', async () => {
await store.remember('ext-a', 'same-key-01', 'h1', '{"n":1}', T0)
await store.remember('ext-a', 'same-key-01', 'h2', '{"n":2}', T0)
await store.remember('ext-b', 'same-key-01', 'h3', '{"n":3}', T0)
expect(await store.idempotent('ext-a', 'same-key-01')).toEqual({ body_hash: 'h1', response: '{"n":1}' })
expect(await store.idempotent('ext-b', 'same-key-01')).toEqual({ body_hash: 'h3', response: '{"n":3}' })
expect(notes.get(url('recent'))).toBe('still here')
expect(notes.get(url('admin'))).toBe('kept')
})
it('upserts feed state and writes the audit log', async () => {
@@ -1,8 +1,6 @@
import { afterAll, beforeAll, beforeEach, describe, expect, it, vi } from 'vitest'
import { resetDataCache } from '../data'
import type { WorkerEnv } from '../env'
import { HttpError } from '../http'
import { handleIngest } from '../ingest'
import { QueueStore } from '../queue'
import { handleSuggest } from '../suggest'
import { testDb } from './d1'
@@ -18,7 +16,7 @@ beforeAll(async () => {
afterAll(() => dispose())
beforeEach(async () => {
resetDataCache()
await db.batch(['candidates', 'idempotency'].map((t) => db.prepare(`DELETE FROM ${t}`)))
await db.prepare('DELETE FROM candidates').run()
store = new QueueStore(db)
})
@@ -101,78 +99,3 @@ describe('POST /api/suggest', () => {
expect(res.status).toBe(503)
})
})
describe('POST /api/ingest', () => {
const deps = (over: Partial<Parameters<typeof handleIngest>[1]> = {}) => ({
store,
data: fakeData(['https://dev.itch.io/listed']),
limiter: fakeLimiter(),
authenticate: async () => 'ext-client.access',
...over,
})
const ingestRequest = (body: unknown, key = 'key-0000-0001') =>
new Request(`${ORIGIN}/api/ingest`, {
method: 'POST',
headers: { 'content-type': 'application/json', 'idempotency-key': key },
body: typeof body === 'string' ? body : JSON.stringify(body),
})
it('queues items and reports each outcome, attributed to the token', async () => {
const res = await handleIngest(
ingestRequest({
items: [
{ url: 'https://dev.itch.io/fresh', title: 'Fresh Demo', note: 'from a jam' },
{ url: 'https://dev.itch.io/listed' },
'https://DEV.itch.io/fresh/',
'nope',
],
}),
deps(),
)
expect(res.status).toBe(200)
expect(await res.json()).toEqual({
received: 1,
results: [
{ url: 'https://dev.itch.io/fresh', canonical: 'https://dev.itch.io/fresh', outcome: 'queued' },
{ url: 'https://dev.itch.io/listed', canonical: 'https://dev.itch.io/listed', outcome: 'duplicate_catalog' },
{ url: 'https://DEV.itch.io/fresh/', canonical: 'https://dev.itch.io/fresh', outcome: 'duplicate_in_request' },
{ url: 'nope', canonical: null, outcome: 'invalid_url' },
],
})
const [row] = await store.get(['https://dev.itch.io/fresh'])
expect(row).toMatchObject({ source: 'ext', submitter: 'ext-client.access', title: 'Fresh Demo', note: 'from a jam' })
expect(JSON.parse(row.flags)).toEqual(['demo'])
})
it('replays a retry with the same key and refuses the key with another body', async () => {
const first = await handleIngest(ingestRequest({ urls: ['https://dev.itch.io/a'] }), deps())
const again = await handleIngest(ingestRequest({ urls: ['https://dev.itch.io/a'] }), deps())
expect(again.headers.get('idempotent-replayed')).toBe('true')
expect(await again.json()).toEqual(await first.json())
const other = await handleIngest(ingestRequest({ urls: ['https://dev.itch.io/b'] }), deps())
expect(other.status).toBe(422)
})
it.each([
['no idempotency key', ingestRequest({ urls: ['https://dev.itch.io/a'] }, ''), {}, 400],
['too many items', ingestRequest({ urls: Array.from({ length: 151 }, (_, i) => `https://dev.itch.io/g${i}`) }), {}, 413],
['a body over 64 KB', ingestRequest({ urls: ['https://dev.itch.io/a'], pad: 'x'.repeat(70_000) }), {}, 413],
['invalid JSON', ingestRequest('{nope'), {}, 400],
['an empty list', ingestRequest({ items: [] }), {}, 400],
['the rate limit', ingestRequest({ urls: ['https://dev.itch.io/a'] }), { limiter: fakeLimiter(0) }, 429],
[
'a failed Access check',
ingestRequest({ urls: ['https://dev.itch.io/a'] }),
{
authenticate: async () => {
throw new HttpError(403, 'forbidden')
},
},
403,
],
])('refuses %s', async (_name, request, over, status) => {
const res = await handleIngest(request, deps(over as Parameters<typeof deps>[0]))
expect(res.status).toBe(status)
expect(await store.counts()).toMatchObject({ pending: 0 })
})
})
+4 -8
View File
@@ -16,7 +16,7 @@
"name": "free-games-itchio-list",
"main": "./worker/index.ts",
"compatibility_date": "2026-09-15",
// workers.dev / preview URLs would bypass Cloudflare Access (admin, ingest).
// workers.dev / preview URLs would bypass Cloudflare Access (the admin).
"workers_dev": false,
"preview_urls": false,
"assets": {
@@ -64,12 +64,11 @@
"migrations_dir": "./worker/migrations"
}
],
// Per-key limits (IP for the Suggest form and the verification gate — an IPv6
// client by its /64 — and the service-token id for ingest). The gate's limit
// is higher: carrier-grade NAT puts many phones behind one address.
// Per-IP limits for the Suggest form and the verification gate (an IPv6
// client by its /64). The gate's limit is higher: carrier-grade NAT puts many
// phones behind one address. (Namespace 1002 was the retired ingest API's.)
"ratelimits": [
{ "name": "RL_SUGGEST", "namespace_id": "1001", "simple": { "limit": 5, "period": 60 } },
{ "name": "RL_INGEST", "namespace_id": "1002", "simple": { "limit": 20, "period": 60 } },
{ "name": "RL_GATE", "namespace_id": "1003", "simple": { "limit": 20, "period": 60 } }
],
// RSS discovery (one feed per run) + queue bookkeeping, worker/index.ts scheduled().
@@ -84,7 +83,6 @@
"GH_APP_INSTALLATION_ID": "162777505",
"ACCESS_TEAM_DOMAIN": "https://polished-hall-48fd.cloudflareaccess.com", // no trailing slash
"ACCESS_AUD_ADMIN": "fc4d79904ffb90f7405cabb0135e15940c7bad48ba344309adc085d71622f482", // "Admin" Access app
"ACCESS_AUD_INGEST": "85662a9f2112d3c5912699f10a59ad9702ef44c05f4e186eb4f550bf1b163ff6", // "Ingest" Access app
"TURNSTILE_SITEKEY": "0x4AAAAAAE7oBxYiQZ_lVEBW",
// Verification gate (worker/gate.ts). "on" still needs the GATE_SECRET and
// GATE_BOT_TOKEN secrets; the instant off switch is deleting GATE_SECRET.
@@ -107,7 +105,6 @@
],
"ratelimits": [
{ "name": "RL_SUGGEST", "namespace_id": "2001", "simple": { "limit": 5, "period": 60 } },
{ "name": "RL_INGEST", "namespace_id": "2002", "simple": { "limit": 20, "period": 60 } },
{ "name": "RL_GATE", "namespace_id": "2003", "simple": { "limit": 20, "period": 60 } }
],
// No RSS polling from staging: production is the only itch.io client.
@@ -120,7 +117,6 @@
"GH_APP_INSTALLATION_ID": "",
"ACCESS_TEAM_DOMAIN": "",
"ACCESS_AUD_ADMIN": "",
"ACCESS_AUD_INGEST": "",
"TURNSTILE_SITEKEY": "",
"BOT_GATE": "off",
"BOT_GATE_TTL_HOURS": "48"