Compare commits

...
Author SHA1 Message Date
poli0981andClaude Opus 5.5 70ec107443 Move the Python pin out of Dependabot's reach
Dependabot's pip updater reads .python-version and supports Python
3.10-3.14 only, so the 3.15 pin from the toolchain upgrade aborted both
the pip version updates and the pip dependency graph ("Dependabot
detected the following Python requirement for your project: '3.15.*'").

The pin now lives in .github/python-version, read only by setup-python's
python-version-file (python-ci, refresh scan + apply, update); Dependabot
falls back to its default 3.14, which every pinned package supports.
Move it back once Dependabot lists 3.15 (CLAUDE.md gotcha 39).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 08:54:45 +07:00
9 changed files with 15 additions and 12 deletions

No files matched your search

File renamed without changes.
+5 -3
View File
@@ -11,7 +11,7 @@ on:
- 'data_game/**'
- 'requirements*.txt'
- 'pyproject.toml'
- '.python-version'
- '.github/python-version'
- '.github/workflows/python-ci.yml'
push:
branches: [main]
@@ -21,7 +21,7 @@ on:
- 'data_game/**'
- 'requirements*.txt'
- 'pyproject.toml'
- '.python-version'
- '.github/python-version'
- '.github/workflows/python-ci.yml'
permissions: {}
@@ -45,7 +45,9 @@ jobs:
- name: Set up Python
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version-file: .python-version
# Not .python-version: Dependabot's pip updater reads that file and
# supports Python <= 3.14 only (CLAUDE.md gotcha 39).
python-version-file: .github/python-version
cache: pip
cache-dependency-path: requirements-dev.txt
+2 -2
View File
@@ -70,7 +70,7 @@ jobs:
- name: Set up Python
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version-file: .python-version
python-version-file: .github/python-version
cache: pip
cache-dependency-path: requirements.txt
@@ -125,7 +125,7 @@ jobs:
- name: Set up Python
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version-file: .python-version
python-version-file: .github/python-version
cache: pip
cache-dependency-path: requirements.txt
+1 -1
View File
@@ -50,7 +50,7 @@ jobs:
- name: Set up Python
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version-file: .python-version
python-version-file: .github/python-version
cache: pip
cache-dependency-path: requirements.txt
+3 -2
View File
@@ -10,7 +10,7 @@ everyone; the old github.io URL only redirects). Three layers share one JSON cat
1. **Python pipeline** — the data layer (GitHub Actions).
- `data_game/game_info_NNN.json` — chunked records, max 500 per file, 3,251 games on 2026-10-10. Serialization is fixed: `json.dumps(..., ensure_ascii=False, indent=4)`, no trailing newline (`json_io.dumps`) — the Worker port (`webapp/worker/catalog.ts`) emits identical bytes (`tests/fixtures/golden/`).
- `data_game/index.json` — chunk manifest (total + per-file counts). `data_game/count_history.json` — date-keyed `{date,total}` series ("games over time" chart). `scripts/deleted_games.json` — removal log (public "Removed" page). `scripts/temp_link.json` — ingest queue.
- `scripts/` — scanners never write the catalog: `update_info.py` (queued URLs) and `refresh.py` (rotating 1/7-per-day check; `--full` re-scrape) emit a URL-keyed **patch** (`patch.py`); `apply_patch.py` applies it to the latest `main` and runs `validate.py`; `bash/commit_push.sh` does fetch → reset → apply → commit → push with retries. Shared: `scraper.py` (session, `Pacer` + 429 back-off, parsing: the info panel incl. itch.io's 2025–26 rows — `AI Disclosure` ("AI Assisted", then what for) / `Content` linking `/games/tag-no-ai` → `ai_disclosure` + `ai_content`, `Accessibility` → `accessibility`; `description` = the creator's tagline (`og:description`), else the first sentence), `data_store.py` (chunks stay in place, rebalance only past thresholds, write-if-changed), `json_io.py`, `canonical.py` (canonical itch.io URL; vectors in `tests/fixtures/url_vectors.json`, shared with the Worker). Pipeline bookkeeping lives in `scripts/state/` (not published), including `ingest_outcomes.json` (how each queued link ended — added / duplicate / paid / dead / private / deleted / gave_up; patch key `ingest_outcomes`; the last 90 days, ≤ 5,000) that the admin's tracking and the cron's reconcile read. Tests: `pytest` (network blocked). Lint: `ruff check scripts tests webapp/scripts`, `ruff format`, `vulture` (config in root `pyproject.toml`). Local env: Python 3.15 (`.python-version`; ruff `target-version = "py315"`), `python -m venv .venv && .venv/Scripts/pip install -r requirements-dev.txt` (3.15 is in UTF-8 mode by default — no `PYTHONUTF8` needed, the workflows no longer set it).
- `scripts/` — scanners never write the catalog: `update_info.py` (queued URLs) and `refresh.py` (rotating 1/7-per-day check; `--full` re-scrape) emit a URL-keyed **patch** (`patch.py`); `apply_patch.py` applies it to the latest `main` and runs `validate.py`; `bash/commit_push.sh` does fetch → reset → apply → commit → push with retries. Shared: `scraper.py` (session, `Pacer` + 429 back-off, parsing: the info panel incl. itch.io's 2025–26 rows — `AI Disclosure` ("AI Assisted", then what for) / `Content` linking `/games/tag-no-ai` → `ai_disclosure` + `ai_content`, `Accessibility` → `accessibility`; `description` = the creator's tagline (`og:description`), else the first sentence), `data_store.py` (chunks stay in place, rebalance only past thresholds, write-if-changed), `json_io.py`, `canonical.py` (canonical itch.io URL; vectors in `tests/fixtures/url_vectors.json`, shared with the Worker). Pipeline bookkeeping lives in `scripts/state/` (not published), including `ingest_outcomes.json` (how each queued link ended — added / duplicate / paid / dead / private / deleted / gave_up; patch key `ingest_outcomes`; the last 90 days, ≤ 5,000) that the admin's tracking and the cron's reconcile read. Tests: `pytest` (network blocked). Lint: `ruff check scripts tests webapp/scripts`, `ruff format`, `vulture` (config in root `pyproject.toml`). Local env: Python 3.15 (`.github/python-version`, gotcha 39; ruff `target-version = "py315"`), `python -m venv .venv && .venv/Scripts/pip install -r requirements-dev.txt` (3.15 is in UTF-8 mode by default — no `PYTHONUTF8` needed, the workflows no longer set it).
2. **Cloudflare Worker + static assets** — `webapp/worker/` + `webapp/wrangler.jsonc`, deployed by **Cloudflare Workers Builds** from `main` (root dir `webapp`, build `npm run build`, deploy `npx wrangler deploy`; no deploy workflow). The catalog is bundled into the site at build time (`vite-plugins/catalog-data.ts` → `dist/data/pack/` compact packs + the legacy readable JSON in `dist/data/`, plus `data/urls.json` and `sitemap.xml`), so every data commit rebuilds the site.
3. **Web app + Tauri apps** — `webapp/src/` (React 19.3 + TypeScript 7.0 + Vite 8/rolldown + Tailwind + shadcn/ui + React Router 8), and the **Tauri 2 desktop + Android** read-only viewers in `webapp/src-tauri/` (same code; they fetch `https://freeitchgames.win/data/pack` and load covers straight from img.itch.zone). Plus `webapp/admin/` — the maintainer-only admin app (second Vite entry, served at `/admin/`).
@@ -144,6 +144,7 @@ Code **MIT** (`LICENSE`); catalog data **CC BY 4.0** for the maintainer's contri
36. **itch.io collection JSON** (`/c/<id>/<slug>?format=json&page=N`): `&page=1` redirects to the HTML page (page 1 has no `page`); a wrong or missing slug redirects to the canonical URL **without** the query string (re-request with the slug from `Location`); an unknown id answers 500; link attributes come in any order; past the last page `num_items` is 0. Zero cells while `num_items > 0` means changed markup: `collections.ts` reports it (422) and adds nothing.
37. **D1 counts every statement of a batch** towards its 1,000 queries per invocation: bind URL lists in `IN (…)` slices of ≤ 90 (100 bound parameters at most), as `setStatus` / `reconcile` / `markIngested` do, never one statement per URL.
38. **npm 11.20+ (Node 26) gates dependency install scripts on `allowScripts` in `webapp/package.json`.** esbuild and workerd are approved by name (their postinstall only checks the platform binary). A new dependency with an install script shows up in `npm install-scripts ls`; review it, then `npm install-scripts approve <pkg> --no-allow-scripts-pin` (name-only, so Dependabot bumps stay covered).
39. **Dependabot's pip updater reads `.python-version` and aborts on a Python it doesn't support** (3.10–3.14 as of Oct 2026): a `3.15` there stopped pip version updates and the pip dependency graph ("Dependabot detected the following Python requirement … '3.15.*'"). So the CI pin lives in `.github/python-version`, which only `setup-python`'s `python-version-file` reads (python-ci, refresh ×2, update); move it back to `.python-version` once Dependabot lists 3.15.
## Safe-edit rules
@@ -154,7 +155,7 @@ Code **MIT** (`LICENSE`); catalog data **CC BY 4.0** for the maintainer's contri
## Current state (Oct 2026)
**Toolchain** (2026-10-22, on `main`, not released yet — goes into the next release's notes): TypeScript 7.0.2 (`@typescript/native`; `typescript` = the 6.0 API alias, gotcha 6), Node 26 (`webapp/.node-version`, `engines >=26`, `@types/node` 26, Workers Builds `NODE_VERSION` raised to 26 by the owner in the dashboard — `docs/operations.md` §1 step 1), Python 3.15 (`.python-version`, ruff `py315`, `urllib3` declared since `scraper.py` imports it). No source change was needed: TS 7 reports 0 errors, the suites pass on Node 26.12 and Python 3.15.1.
**Toolchain** (2026-10-22, on `main`, not released yet — goes into the next release's notes): TypeScript 7.0.2 (`@typescript/native`; `typescript` = the 6.0 API alias, gotcha 6), Node 26 (`webapp/.node-version`, `engines >=26`, `@types/node` 26, Workers Builds `NODE_VERSION` raised to 26 by the owner in the dashboard — `docs/operations.md` §1 step 1), Python 3.15 (`.github/python-version` — gotcha 39, ruff `py315`, `urllib3` declared since `scraper.py` imports it). No source change was needed: TS 7 reports 0 errors, the suites pass on Node 26.12 and Python 3.15.1.
**v4.4.1** (2026-10-16): the browser extension and the Telegram bot are retired — both repos (`poli0981/itch-f2p-extension`, `poli0981/telegram-scraper-bot`) were already public archives; an Announcements discussion says so. Removed: `POST /api/ingest` (`ingest.ts`, `requireService`, `RL_INGEST`, `ACCESS_AUD_INGEST`, the idempotency store — it never received a game: 0 `ext` rows in production D1), the admin's "Browser extension" auto-approve source and "Extension" label, the About page's Telegram numeric-ID note. The policies (Privacy, ToS, EULA + installer RTF, Disclaimer, Security; EN + VI) only lost the extension / ingest wording: "Last updated" 2026-10-16, `LEGAL_VERSION` deliberately unchanged (nothing new to accept, ToS §14). Unused repo secrets `GH_TOKEN`, `TELEGRAM_BOT_TOKEN`, `DISCUSSION_*` deleted. Ships Dependabot #131 (vite 8.3.4), #135 (tauri crates), #136 (serde_json), #137 (ruff).
+1 -1
View File
@@ -80,7 +80,7 @@ Details live in the [Terms of Use](docs/ToS.md) and the [Disclaimer](docs/DISCLA
Full instructions (Worker, local D1, secrets, Tauri, Android) are in **[`docs/dev_env.md`](docs/dev_env.md)**; the Tauri/Android build notes are in [`webapp/TAURI.md`](webapp/TAURI.md). The short version:
**Python pipeline** (Python 3.15, see `.python-version`):
**Python pipeline** (Python 3.15, see `.github/python-version`):
```sh
python -m venv .venv
+1 -1
View File
@@ -24,7 +24,7 @@ TypeScript* at `webapp/node_modules/@typescript/old` (TypeScript 6.0, which repo
| Stack | Required | Used for |
|-------|----------|----------|
| Python | 3.15 (`.python-version`) | Data pipeline (`scripts/`) and its tests |
| Python | 3.15 (`.github/python-version`) | Data pipeline (`scripts/`) and its tests |
| Node.js | 26+ (CI uses `webapp/.node-version`) | Web app, admin app, Worker, Tauri front end |
| npm | ships with Node.js | Dependencies (`npm ci` from `package-lock.json`) |
| Wrangler | project dependency (`npx wrangler`) | Local Worker, local D1, deploys |
+1 -1
View File
@@ -84,7 +84,7 @@ Chi tiết có trong [Điều khoản sử dụng](ToS.md) và [Tuyên bố mi
Hướng dẫn đầy đủ (Worker, D1 cục bộ, secret, Tauri, Android) nằm trong **[`docs/dev_env.md`](../../dev_env.md)** ([VI](dev_env.md)); ghi chú build Tauri/Android nằm trong [`webapp/TAURI.md`](../../../webapp/TAURI.md). Bản ngắn gọn:
**Pipeline Python** (Python 3.15, xem `.python-version`):
**Pipeline Python** (Python 3.15, xem `.github/python-version`):
```sh
python -m venv .venv
+1 -1
View File
@@ -26,7 +26,7 @@ tới `webapp/node_modules/@typescript/old` (TypeScript 6.0, báo cùng các l
| Stack | Yêu cầu | Dùng cho |
|-------|---------|----------|
| Python | 3.15 (`.python-version`) | Pipeline dữ liệu (`scripts/`) và test của nó |
| Python | 3.15 (`.github/python-version`) | Pipeline dữ liệu (`scripts/`) và test của nó |
| Node.js | 26+ (CI dùng `webapp/.node-version`) | Web app, admin app, Worker, phần front end của Tauri |
| npm | đi kèm Node.js | Dependency (`npm ci` theo `package-lock.json`) |
| Wrangler | dependency của dự án (`npx wrangler`) | Worker chạy local, D1 local, deploy |