Files
poli0981andClaude Opus 5 72d00c1014 fix(ci): drop the expired PAT, close an unauthenticated write path, land snapshots
The scheduled pipeline has been failing daily since 2026-08-13 (last successful
`Auto update` was e8b7d1b2). Root cause: `secrets.GH_TOKEN`, a PAT, expired.
Every workflow passing it to actions/checkout dies at fetch with exit 128.

No workflow actually needs a PAT. Each already declares
`permissions: {contents: write}`, so the built-in GITHUB_TOKEN has push rights —
proven by mark-dead-games.yml, the only green data workflow and the only one
with a bare checkout. Removed the `token:` input from all 12 sites. This also
ends the silent-expiry failure mode rather than rescheduling it a year out.

`workflow_run` triggers are unaffected: GITHUB_TOKEN suppression applies to
push/issues/release events, and update-daily.yml plus the three notify-*
workflows all key off workflow_run, which the Actions service emits on run
completion regardless of the checkout token.

Also in this commit:

- ingest-from-issue.yml gated on `author_association`. Without it, any user
  could open an issue titled [add-game] and have their JSON committed and
  pushed to main with `contents: write`. A title match is not an authorization
  check. Its `&&`/`||` commit chain is also parenthesised — left-associativity
  meant it pushed even when there was nothing staged.

- bot-ingest.yml's final step deleted. It referenced `secrets.BOT_TOKEN` while
  the rest of the file uses TELEGRAM_BOT_TOKEN, `cat`-ed a result.txt no step
  creates, ran under `if: always()`, and interpolated a secret straight into a
  shell command line. The Python step above it already does the same Telegram
  callback correctly. `secrets.BOT_TOKEN` is now unreferenced.

- The snapshot feature is committed. snapshot-daily.yml gains the
  `concurrency: {group: data-write}` guard every other data-writing workflow
  has — without it a 23:00 run can push against jobs that take ~2 hours. The
  `pip install requests` in bash/snapshot.sh is dropped: snapshot.py and
  scripts/core/* import only stdlib.

- New web-ci.yml. deploy-pages.yml was the only thing running `npm run
  typecheck`, so deleting it during the Cloudflare migration would leave the
  frontend unchecked. Uses a plain `npm ci` — the old `npm ci || npm install`
  fallback is what let the lockfile conflict above go unnoticed.

- New dependabot.yml. NOTE: an in-repo config replaces the one in repo
  settings and supersedes open PR #111. Groups security updates so the
  transitive backlog collapses into one PR instead of one per package.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-10 13:46:38 +07:00

75 lines
2.3 KiB
Python

#!/usr/bin/env python3
"""Daily snapshot — capture per-game time-series data.
Writes one JSONL line per game with fields needed for trend charts:
{appid, name, current_players, peak_today, reviews_pct, captured_at}
Output: data/snapshots/YYYY-MM-DD.jsonl (one file per UTC day, idempotent — overwrites same-day).
Manual fields and full game records are NOT duplicated here; consumers join on appid
against the live shards if they need the rest.
"""
import json
import os
import re
import sys
from datetime import datetime, timezone
from pathlib import Path
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from core.data_store import load_main, extract_appid
SNAPSHOTS_DIR = Path("data/snapshots")
PCT_RE = re.compile(r"(\d{1,3})%")
def _parse_players(s: str) -> int:
if not s:
return 0
cleaned = s.replace(",", "").strip()
return int(cleaned) if cleaned.isdigit() else 0
def _review_pct(reviews: str) -> int | None:
if not reviews:
return None
m = PCT_RE.search(reviews)
return int(m.group(1)) if m else None
def main() -> int:
games = load_main()
if not games:
print("No games loaded — aborting snapshot.")
return 1
SNAPSHOTS_DIR.mkdir(parents=True, exist_ok=True)
today = datetime.now(timezone.utc).strftime("%Y-%m-%d")
out_path = SNAPSHOTS_DIR / f"{today}.jsonl"
captured_at = datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
written = 0
tmp = out_path.with_suffix(".jsonl.tmp")
with tmp.open("w", encoding="utf-8") as f:
for g in games:
appid = extract_appid(g.get("link", "") or "")
if not appid:
continue
row = {
"appid": appid,
"name": g.get("name", ""),
"current_players": _parse_players(g.get("current_players", "")),
"peak_today": _parse_players(g.get("peak_today", "")),
"reviews_pct": _review_pct(g.get("reviews", "")),
"type_game": g.get("type_game", ""),
"status": g.get("status", "active"),
"captured_at": captured_at,
}
f.write(json.dumps(row, ensure_ascii=False) + "\n")
written += 1
tmp.replace(out_path)
print(f"[OK] {out_path} ({written} rows)")
return 0
if __name__ == "__main__":
sys.exit(main())