51 Commits
Author SHA1 Message Date
poli0981andClaude Opus 5.5 529b8603c3 ci(pipeline): queue data-write runs and start each on the latest main
- queue: max on every data-write workflow (GitHub, 2026-05-07): up to
  100 runs wait, first in, first out. The default kept one, and a run
  queueing behind a running job cancelled it.
- ref: main on every checkout. A run's commit is fixed when it is
  created, so a run that waited behind another data job wrote shards
  from before that job's commit; Auto Update JSON failed four days in
  ten on the rebase conflict in data/index.json.
- timeout-minutes everywhere, and JOB_BUDGET_MINUTES at least 20 below
  it for the long loops (never for the ingest, which empties its queue).
- pipeline.test.ts fails on a data-write workflow missing any of these.
- The admin's Pipeline and Automation pages, docs/ADMIN.md and the
  auto-approve comment no longer say a push evicts a waiting job; the
  idle gate stays, so a batch's ingest starts at once.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 22:22:08 +07:00
poli0981andClaude Opus 5.5 6e7d1bcf15 chore(pipeline): retire Check Dead Links
It sent a HEAD request to every store page every five days, 5.5 hours at
9,847 games, and flagged 404/410. Steam sends a delisted app's page to
the store front, which answers 200: scripts/dead_links.log never existed
and no game ever became "delisted". The purge's GetItems screen sees
delisted games (unlisted) instead.

Removes the workflow, bash/dead_links.sh, scripts/check_dead_links.py
and DEAD_LINKS_LOG, and the workflow from web/shared/pipeline.ts and the
failure notifier's list. check_store_page() stays: it is the health
check's last resort.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 22:22:08 +07:00
poli0981andClaude Opus 5.5 1da4d5940e feat(pipeline): keep the store, peaks and re-fetch runs bounded
- Refresh Store Data: the page and appdetails cycles grow past 2,000
  pages / 500 appdetails a day (7 -> 14, 30 -> 60) instead of the run,
  the slice comes in a daily order, and the store step has a budget. A
  game appdetails no longer calls free is listed in a ::warning::, the
  case the purge's two-signal rule cannot see; nothing is written.
- Refresh All-Time Peaks: 30 -> 60 days past 400 pages a day, never
  shorter, so SteamCharts is still asked at most once a month a game;
  the budget stops it with a warning like a refusal does.
- Force Re-fetch: --part K/N, oldest last_updated first, each game
  cleared on a copy that replaces it only once appdetails has answered;
  no final loop stamping every game. Parts queue one after another.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 22:22:08 +07:00
poli0981andClaude Opus 5.5 74e1845fc4 feat(purge): remove a game only when two of Steam's answers agree
The purge asked appdetails about every game, 5.3 hours at 9,847, and one
answer was enough to remove one - appdetails has answered success:false
for live games in bursts (2026-09-27).

GetItems now screens the whole catalogue in a few minutes (unknown,
hidden, unlisted, coming soon, not is_free; never type), and
check_game_health() must confirm each flag; "unavailable" is asked again
five minutes later. A failed batch or a network error never reads as
gone, blocklisted games are not asked about, and more than 500 flags or
100 removals stops the run before anything is written. Removals are
logged as before. A notice names earlier removals Steam has since undone.

The dry run on the real catalogue: 8 flagged, 3 removed (the three
unlisted games), 5 kept because appdetails calls them free. Daily now.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 22:21:59 +07:00
poli0981andClaude Opus 5.5 49f0c56ed7 feat(pipeline): read reviews for every game from Steam's store API
Update Reviews asked appreviews about one game at a time, two seconds a
game. At 9,847 games it ran into GitHub's six-hour limit on 2026-10-23
and was stopped with nothing saved; five of its ten runs before that
were cancelled while waiting.

- SteamClient.fetch_items(): GetItems for a whole catalogue, a failed
  batch costing only its own ids (the anti-cheat sweep now uses it).
- update_reviews.py reads review_summary() for every game, ~2 minutes,
  daily. Steam floors its percentage; reviews_from_summary() floors
  appreviews in integers too, so an ingest and the refresh agree. A
  count of 0 is never written on GetItems' word (it said 0 for a game
  with thousands): a stored percentage or "N/A" is asked on appreviews,
  at most 300 a run. last_updated moves on a change, and on a 14-20 day
  heartbeat that keeps the site's 30-day "stale" badge quiet without
  rewriting every record every run.
- core/budget.py: JOB_BUDGET_MINUTES. Past it process_batch() starts no
  further game, so a job saves what it did instead of being killed;
  a ::warning:: when a loop used three quarters of its budget.
- rotation.cycle_for() / daily_order(): cycles that grow with the
  catalogue as multiples of their base, and an order that changes daily.
- The player-count loops skip the batch pauses meant for store pages
  (Top Online ~27 min instead of 48) and take a deadline, like
  update_all_full().

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 22:21:59 +07:00
poli0981andClaude Opus 5.5 988e4f4ce6 ci: pin third-party actions, install without lifecycle scripts, hash Python deps
SonarCloud's GitHub Actions and shell findings, all supply-chain hardening
for jobs that hold a write token (and, in the release workflows, the
signing keys):

- The six third-party actions in release-desktop.yml and release-android.yml
  are pinned to the commit their major tag points at today, with the version
  in a comment for Dependabot. dtolnay/rust-toolchain is pinned to a master
  commit and given `toolchain: stable`: its `stable` branch is regenerated
  on top of master, so a commit of it can vanish.
- Every `npm ci` is `npm ci --ignore-scripts`. Only fsevents, workerd and
  wrangler's esbuild have install scripts; the last two find their binary in
  their optional platform package without them, and fsevents is only file
  watching. `npx` is gone from the workflows (`npm run cf:check`,
  `npm test -- <file>`).
- requirements.txt pins every runtime package, transitive ones included
  (certifi, charset-normalizer, idna), with the sha256 of every file on
  PyPI; every install is `--require-hashes --only-binary :all:`. The new
  requirements-test.txt holds pytest the same way. python-tests.yml runs the
  data jobs' exact install line first, so a Dependabot bump is proven before
  a data job meets it.
- `[[ ]]` instead of `[ ]` in the bash wrappers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 09:31:40 +07:00
poli0981andClaude Opus 5.5 3d0e22cd76 fix(pipeline): read anti-cheat from Steam's own disclosure; retire the extension's paths
None of the 4,704 games added since September had an anti-cheat. The only
code that read the anti-cheat a developer declares on Steam was the browser
extension (archived; last push 2026-06-25); the pipeline matched appdetails'
categories, which only ever say "Valve Anti-Cheat enabled".

- core/anticheat.py (new, pure): the store page's div.anticheat_section
  (its HEADING decides kernel, never the class; a Boot Protection row is a
  flag) and GetItems' extra_details.anticheat; labels from
  ANTI_CHEAT_PATTERNS, else Steam's name cut at 40 characters; same_family()
  ("EAC" = "Easy Anti-Cheat", "ACE" = "Anti-Cheat Expert (ACE)"); and
  apply_anti_cheat(), the one rule.
- fetch_full() fills, never replaces, from the page it already fetches (or
  GetItems for a gated page), else the VAC category, now found in any
  position (with a stored "" only the first category was checked).
- refresh_anticheat.py (new): GetItems for every game (~160 requests), a
  store page only for a disclosure without a name. Steam wins; an override
  wins over Steam and is not fought (OverrideGuard); no disclosure never
  clears; a failed batch is unknown. Runs at the end of the daily Refresh
  Store Data job with its own commit; only=anticheat runs it alone. That job
  now checks out main.
- The queue takes links only: merge_extension_data(), EXTENSION_FIELDS,
  ARRAY_FIELDS and the schema.ts mirrors go. translate_notes() becomes
  clean_notes(): the extension's leading "Has paid DLC" and "[hover-add]"
  are dropped (517 notes, once, never news).
- refetch_all.py keeps "-" instead of resetting it to ""; a retired override
  whose was equals its value no longer bumps last_updated on every save.
- "denuvo" joins Denuvo AC on both sides; anti-cheat.test.ts holds the TS
  patterns and enum to constants.py.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 09:38:39 +07:00
poli0981andClaude Opus 5.5 986077d3f1 feat(pipeline): data/blocklist.jsonl - save_main drops and logs removed games
A game the maintainer removes is now a standing instruction in Git, like an
override, instead of a one-time deletion that the browser extension or the
next ingest could undo.

- core/blocklist.py reads data/blocklist.jsonl (one line per game, written by
  /admin and by delete_game.py). save_main() applies it FIRST on every write,
  for all 14 callers, without touching the caller's list, and whatever
  apply_human_overrides says.
- A dropped game is logged once in scripts/removed_games.jsonl as
  status_code "admin_removed", built from the blocklist line alone with
  removed_at = blocked_at, so writing it again changes no byte. A later purge
  line for the same game keeps winning, as dedup_removed() decides.
- More than 500 drops in one save raises BlocklistError before anything is
  written: a damaged file fails the job instead of emptying the catalogue.
- remove_games.py applies the blocklist now and heals a missing log line.
  bash/ingest.sh runs it before ingest_new.py, and ingest-new.yml also
  triggers on blocklist pushes and checks out the tip of main. One workflow
  on purpose: data-write keeps one pending run, so a second workflow and the
  ingest would keep evicting each other.
- ingest_new.py skips blocked links before any Steam request, and a link
  that is not text ({"link": 730}) no longer crashes the run (and every
  three-hourly retry after it).
- Discovery counts blocked appids as known.
- delete_game.py is a non-interactive CLI writing the same bytes as /admin
  (golden fixture), with --list and --check; check-overrides.yml runs --check.
- mark-dead-games.yml commits through bash/commit_push.sh, like every other
  data job: it rebases and retries, and commits the log with the shards.
- dependabot.yml ignores Svelte-era framework majors and @types/node majors
  instead of the React-era names.

With an empty blocklist every path is a no-op.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 12:33:53 +07:00
poli0981andClaude Opus 5.5 f997d79054 feat(data): all_time_peak, from SteamCharts and our own samples
The dataset only knew how many people play a game now. The only peak it had,
peak_today, is not today's: nothing resets it, so it is the highest of our
sparse samples (online games 3x a week, the rest twice a month) since the
last refetch - The First Descendant reads 8,052 against a real all-time peak
of 264,860.

all_time_peak is the most players a game has had at once, and it only ever
goes up (core/peaks.py):
- refresh_peaks.py reads SteamCharts' recorded all-time peak, the only public
  source older than our sampling (SteamDB forbids scraping; Valve's APIs only
  know today). It takes a crc32 thirtieth of the catalogue a day, so each game
  is asked once a month, ~186 pages 2-3 s apart. SteamCharts has no API, no
  terms and no robots.txt, so it is read politely (core/steamcharts.py): a
  User-Agent that names the project, 500/404 read as "not tracked", and a
  403, 429 or Cloudflare challenge stops the run with a warning, keeping
  everything raised so far. Measured: 9 of 10 games with a few dozen players
  or more are tracked; most of the 0-3 player long tail is not.
- Every run also raises the field from our own samples (peak_today,
  current_players) without a request, and apply_players() raises it on every
  sample - never from a 0. That seeds 2,329 games on the first run.
- Nothing else is written, last_updated included (player counts never moved
  it), and refetch_all.py does not clear the field: it cannot be fetched again.

"Refresh All-Time Peaks" runs daily at 19:30 UTC, between Refresh Store Data
and the snapshot, in the data-write group, committing through
commit_push.sh; a manual run with "backfill" asks about every game (~4 h).
The crc32 rotation moves to core/rotation.py, shared with
refresh_store_data.py. test_refresh_peaks.py holds the rules.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-03 07:15:39 +07:00
poli0981andClaude Opus 5.5 de9d380994 feat(pipeline): refresh tags, languages and store details on a rotation
update_data.py only enriches records that are incomplete, and fetcher.py's
apply_details()/apply_scraped() only fill EMPTY fields. Once a game had its
tags, languages, platforms or header image, nothing looked at them again
short of the manual refetch_all.py. A dry run on three games found all three
with tags that had drifted since they were first fetched.

scripts/refresh_store_data.py re-reads them a slice at a time, daily:
- store page (tags, languages, language_details, has_paid_dlc): every game
  once every 7 days;
- appdetails (name, header_image, description, developer, publisher,
  release_date, platforms, metacritic, drm_notes): every game once every
  30 days.
Measured on the live catalogue: 600-675 store pages and 121-166 appdetails
calls a day, about 40 minutes at the client's throttle, instead of ~8,800
requests in one sitting.

A game is due when crc32(appid) % cycle == day % cycle. Not appid % cycle:
Steam appids are nearly all multiples of 10, which would leave 27 of the 30
monthly slices empty.

These fields were fill-if-empty until now, so the refresh keeps the same
care: it replaces a value only with a non-empty one (a failed request or an
age-gated page changes nothing), never writes MANUAL_FIELDS or notes, reads
has_paid_dlc only from a real store page, compares description as stored
(truncated to DESCRIPTION_MAX), bumps last_updated only on a real change,
and writes through save_main() so overrides and index.json behave as for
every other job. scripts/tests/test_refresh_store_data.py pins each rule and
the even spread of the slices.

"Refresh Store Data" runs at 16:00 UTC: scheduled runs here start 2-5 hours
late, the morning data jobs run 2-4 hours, and the data-write group keeps
only one pending run, so the slot is the one nothing long overlaps. It is in
data-write, commits through bash/commit_push.sh, and is on the CI-failure
notification list. README, CLAUDE.md and the changelog describe it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 22:40:48 +07:00
poli0981andClaude Opus 5.5 5e689e7aca fix(pipeline): rebase and retry when a data job's push is rejected
purge-unhealthy ran for four hours on 2026-09-21 and then failed on
"push rejected (fetch first)", throwing the whole run away. The data-write
concurrency group serialises the WORKFLOWS, but main still moves under a
long job: a merged pull request, an /admin approval or override the Worker
commits, the extension's push to scripts/temp_info.jsonl.

bash/commit_push.sh does what every wrapper did - add, commit only if
something changed - and then pushes with up to five attempts, rebasing onto
origin/main between them. None of the other writers touch the shards, so
the rebase is clean; a genuine conflict still fails the job. All nine data
wrappers now call it, with their commit messages unchanged (snapshot.sh
keeps its data/snapshots/ pathspec).

Tested against a scratch remote: with main moved by another commit, the
first push is rejected, the rebase lands the job's commit on top, and the
retry succeeds; with nothing to commit it prints "No changes".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 22:40:23 +07:00
poli0981andClaude Opus 5 192f17622f chore: drop the bulk generated markdown, keep the two leaderboards
games/ held 1.8 MB of markdown regenerated from data/*.jsonl on every data
change: 18 all-games_part*.md files, anti-cheat-list.md, a README index of
them, and a .gen-hash checksum whose only purpose was to make regenerating
them cheaper. The web app has been the way to browse the catalogue since v3.0,
so these were a second rendering of the same records that had to be rebuilt,
committed and reviewed forever.

Kept: games/top-online.md and games/top-offline.md.

Removed with them, because nothing else used them:
  scripts/generate_tables.py, scripts/generate_anti_cheat_list.py
  .github/workflows/update-daily.yml ("Generate Markdown Tables")
  .github/workflows/anti-cheat-list.yml
  bash/daily.sh, bash/anti_cheat.sh
and the generate_tables call inside ingest.sh, purge.sh, refetch_all.sh and
ingest-from-issue.yml.

Nothing depended on the anti-cheat markdown: /charts/anti-cheat reads
data/*.jsonl, not that file.

Four references would have broken, two of them user-visible:

- web/public/sitemap.xml advertised all-games_part1.md to search engines, and
  About.tsx had an "All games (markdown)" button. Both would now 404. The
  sitemap entry is gone; the button points at data/ instead.
- .github/workflows/mark-dead-games.yml ended with
  `gh workflow run "Generate Markdown Tables"`. That command exits non-zero
  against a workflow that does not exist, so every successful dead-games run
  would have finished red.
- notify-ci-failure.yml's allowlist is matched by exact workflow NAME, and the
  file's own comment warns that a stale entry breaks silently.

Also fixed the stale poli0981.github.io URL in the bug-report template while
removing its games/*.md option.

Verified: typecheck, build, all 24 workflow YAML files parse, sitemap.xml is
well-formed, and no reference to the removed files survives outside CHANGELOG
and docs/plan (both historical).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-10 23:23:41 +07:00
poli0981andClaude Opus 5 eae8f6354d fix: detect MIME-type chunk failures, install pinned Python deps everywhere
isChunkLoadError only matched the three "failed to fetch dynamically imported
module" style messages. Those cover a 404 on a stale chunk, but not what a
host with an SPA fallback returns: a request for a deleted hashed chunk is
answered with index.html at HTTP 200 and Content-Type: text/html, so the
browser rejects it on MIME type instead. The message looks nothing like a
fetch failure, so the reload-once recovery would silently stop working the
moment the site moves behind Cloudflare's not_found_handling. Added the
"failed to load module script" / "expected a javascript module script"
patterns and lifted the regex to a named constant.

Python installs now go through requirements.txt at all 11 sites — nine
bash/*.sh scripts and the two inline workflow steps — instead of a bare
`pip install requests`. The bare install silently ignored the urllib3 pin,
so CI was resolving a dependency the requirements file was trying to
constrain. mark-dead-games.yml already did this correctly.

requirements.txt and requirements-dev.txt move from `>=x,<y` ranges to exact
`==` pins: the pipeline should install identical versions on every run, and
Dependabot does not raise PRs for range-specified dependencies, so the new
pip ecosystem entry in dependabot.yml would otherwise have had nothing to do.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-10 14:06:50 +07:00
poli0981andClaude Opus 5 72d00c1014 fix(ci): drop the expired PAT, close an unauthenticated write path, land snapshots
The scheduled pipeline has been failing daily since 2026-08-13 (last successful
`Auto update` was e8b7d1b2). Root cause: `secrets.GH_TOKEN`, a PAT, expired.
Every workflow passing it to actions/checkout dies at fetch with exit 128.

No workflow actually needs a PAT. Each already declares
`permissions: {contents: write}`, so the built-in GITHUB_TOKEN has push rights —
proven by mark-dead-games.yml, the only green data workflow and the only one
with a bare checkout. Removed the `token:` input from all 12 sites. This also
ends the silent-expiry failure mode rather than rescheduling it a year out.

`workflow_run` triggers are unaffected: GITHUB_TOKEN suppression applies to
push/issues/release events, and update-daily.yml plus the three notify-*
workflows all key off workflow_run, which the Actions service emits on run
completion regardless of the checkout token.

Also in this commit:

- ingest-from-issue.yml gated on `author_association`. Without it, any user
  could open an issue titled [add-game] and have their JSON committed and
  pushed to main with `contents: write`. A title match is not an authorization
  check. Its `&&`/`||` commit chain is also parenthesised — left-associativity
  meant it pushed even when there was nothing staged.

- bot-ingest.yml's final step deleted. It referenced `secrets.BOT_TOKEN` while
  the rest of the file uses TELEGRAM_BOT_TOKEN, `cat`-ed a result.txt no step
  creates, ran under `if: always()`, and interpolated a secret straight into a
  shell command line. The Python step above it already does the same Telegram
  callback correctly. `secrets.BOT_TOKEN` is now unreferenced.

- The snapshot feature is committed. snapshot-daily.yml gains the
  `concurrency: {group: data-write}` guard every other data-writing workflow
  has — without it a 23:00 run can push against jobs that take ~2 hours. The
  `pip install requests` in bash/snapshot.sh is dropped: snapshot.py and
  scripts/core/* import only stdlib.

- New web-ci.yml. deploy-pages.yml was the only thing running `npm run
  typecheck`, so deleting it during the Cloudflare migration would leave the
  frontend unchecked. Uses a plain `npm ci` — the old `npm ci || npm install`
  fallback is what let the lockfile conflict above go unnoticed.

- New dependabot.yml. NOTE: an in-repo config replaces the one in repo
  settings and supersedes open PR #111. Groups security updates so the
  transitive backlog collapses into one PR instead of one per package.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-10 13:46:38 +07:00
github-actions[bot] 009ddfe718 Anti-cheat list [2026-05-24 07:07] 2026-05-24 07:07:13 +00:00
poli0981andClaude Opus 4.7 fe6e93e1bd v3.2.4 (features): removed-games dedup + anti-cheat list + top-online 100 + delisted charts (#75)
- `scripts/core/data_store.py` adds `dedup_removed(records)` — keeps latest entry per appid (by `removed_at`). Call sites updated in `purge_unhealthy.py:41-43` and `ingest_new.py:106-108` so every write through the existing `save_jsonl` atomic-rename transaction is dedup-clean. One-shot `scripts/dedup_removed_games.py` ran the historical cleanup (43 → 37 records, 6 duplicates dropped).
- `scripts/core/constants.py:81` bumps `TOP_ONLINE_LIMIT` 50 → 100. Top Offline already 100; `TopOnline.tsx` and the `topOnline.{subtitle,cardTitle}` strings updated to match.
- `scripts/generate_anti_cheat_list.py` + `bash/anti_cheat.sh` + `.github/workflows/anti-cheat-list.yml` — weekly-cron pipeline that buckets active games by canonical AC name (`ANTI_CHEAT_PATTERNS`, substring-match for free-text values, `Other` bucket for unrecognised) and emits `games/anti-cheat-list.md` with TOC + per-family 6-column tables sorted by current_players desc. First run: 54 games across 8 known families + 17 in `Other`.
- `web/src/pages/AntiCheatList.tsx` — `/charts/anti-cheat/list` page mirrors the MD: collapsible per-family sections + sticky AC-name sidebar nav. TS port of the Python canonicalisation table in-file so behaviour stays in sync.
- `web/src/hooks/useRemovedGames.ts` + `DelistedTimeline.tsx` + `DelistedReasonBreakdown.tsx` + `pages/charts/Delisted.tsx` — fetches `removed_games.jsonl` from raw.githubusercontent.com, dedups client-side, renders stacked-bar timeline by month/`status_code` + donut by `status_code`. Available at `/charts/delisted`.
- `web/src/components/charts/KpiCards.tsx` — new optional `removedCount` prop adds a 7th "Removed (all-time)" KPI when supplied; grid bumps `xl:grid-cols-6 → 7`. `Dashboard.tsx` wires `useRemovedGames().data?.length` through.
- Sidebar nav: new `AC Index` link (under existing Anti-Cheat) and `Delisted` link (under DRM/DLC).
- i18n: `antiCheatList.*`, `charts.delisted.*`, `nav.{antiCheatList,delisted}` keys added to `en.json` + `vi.json`.

Verification: `npm run build` (2418 modules), `npm run typecheck`, Python imports clean, dedup migration idempotent on re-run.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 00:21:03 +07:00
github-actions[bot] 71010491f9 Top offline [2026-05-11 04:56] 2026-05-11 04:56:34 +00:00
poli0981andClaude Opus 4.7 70d9730ce9 v3.2.0: SEO + Donate + auto-Discussion + Health bucket + dev docs (#64)
* v3.2.0: SEO, Donate page, Discussion auto-post, Health bucket split, dev docs

Bundles 8 features for v3.2.0 (per the planning round):

1. SEO basics — OG / Twitter card / canonical / JSON-LD in index.html;
   robots.txt + sitemap.xml in web/public; useDocumentTitle hook wired
   into every page so the tab title reflects the current route +
   language.

2. Auto-post a GitHub Discussion on release `published` —
   .github/workflows/announce-release-discussion.yml. Uses GraphQL
   `createDiscussion`, idempotent (skips if a discussion with the same
   title already exists), prefers the `Announcements` category with
   fallback to `General`.

3. Dev docs — docs/pc_spec.md, docs/dev_env.md, VI mirrors under
   docs/i18n/vi/, web/src-tauri/TAURI.md (build prerequisites + signing
   notes). AUTHORS.md gets a "Dev info" table; README.md links the new
   docs.

4. Donate page (/donate) with 5 platform cards
   (GitHub Sponsors / Ko-fi / BMC / Patreon / PayPal) sourced from
   .github/FUNDING.yml. New nav item, EN + VI i18n keys.

5. Telegram bot user guide — new docs/telegram-bot.md pointer page
   linking the external USER_GUIDE.md, bot source repo, and the
   in-repo bot-ingest.yml workflow. CONTRIBUTING.md and About page get
   the same link.

6. Health bucket split — separates "Online + AC + kernel unknown" from
   "Online · anti-cheat unknown / none" so games with empty `anti_cheat`
   no longer disappear from the report.

7. Steam Desktop direct link — game detail drawer now shows two buttons:
   `steam://store/<appid>` (Steam client) and the regular
   store.steampowered.com URL. Helper at web/src/lib/steam-link.ts.

8. Bi-weekly offline player leaderboard — scripts/top_offline.py +
   bash/offline.sh + .github/workflows/top-offline.yml (cron 1st + 15th
   of each month, plus workflow_dispatch). Output:
   games/top-offline.md, top 100 single-player F2P concurrents.
   Health page gets a manual trigger button.

Verification: `cd web && npm run typecheck && npm run build` — both
pass; PWA generates 17 precache entries; manifest.webmanifest emitted
by VitePWA as before.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(top-offline): make update_players_only honour offline filter

`update_players_only(games)` had a hardcoded `type_game == "online"`
filter, so passing it the full games list from top_offline.py refreshed
ZERO offline player counts and the resulting top-offline.md was sorted
on stale data (last touched by the daily update_data.py).

Extend the function with a backward-compatible `type_filter` kwarg
("online" default, plus "offline" and "all"). top_online.py keeps its
existing call shape; top_offline.py now passes type_filter="offline"
which excludes online AND empty-type rows (unclassified is not
"offline").

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore(release): bump to v3.2.0 / desktop v1.2.0 + CHANGELOG

- web/package.json + lockfile: 1.1.0 → 1.2.0
- web/src-tauri/tauri.conf.json: 1.1.0 → 1.2.0
- web/src-tauri/Cargo.toml: 1.1.0 → 1.2.0
- README badge: 3.1.0 → 3.2.0
- CHANGELOG.md: new v3.2.0 entry covering all 8 features in this PR

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-11 09:50:12 +07:00
poli0981andClaude Opus 4.6 abc4681086 Sharded data store + cleanup
- Replace single data.jsonl with sharded data/ directory (800 records/file)
- load_main()/save_main() handle sharding transparently
- Add migration script (migrate_to_shards.py)
- Remove CSV export (export_data.py, csv_export.sh, update_csv.yml, data.csv)
- Remove legacy scripts (check_duplicate.py, migrate_schema.py, migrate.sh)
- Clean dependencies: remove DateTime, jsonlines, pandas from requirements
- Remove jsonlines from all bash pip installs

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-16 19:28:58 +07:00
github-actions[bot] 64e51fb428 Dead link check [2026-04-01] 2026-04-01 05:23:25 +00:00
github-actions[bot] 516cf7a14d Force re-fetch all games → v2.1 schema [2026-03-26] 2026-03-26 15:37:34 +00:00
poli0981 124b2aa28a feat: migrate to v2.1 schema and add scripts for data management (#38) 2026-03-26 21:55:52 +07:00
github-actions[bot] 3297f95987 Auto update reviews [2026-03-13] by GitHub Action 2026-03-13 07:13:43 +00:00
github-actions[bot] 806bb6bad9 Purge unhealthy games [2026-03-13] by GitHub Action 2026-03-13 04:37:33 +00:00
github-actions[bot] f1731c5f63 Auto update CSV [2026-03-13] by GitHub Action 2026-03-13 04:07:56 +00:00
poli0981 bff1716c8c Add health checker and purge functionality for unhealthy games 2026-03-13 10:44:59 +07:00
github-actions[bot] 783c8171ff Auto update data [2026-03-13] by GitHub Action 2026-03-13 01:45:09 +00:00
github-actions[bot] 22a1132b8b Auto update tables [2026-03-12] by GitHub Action 2026-03-12 09:23:39 +00:00
github-actions[bot] 4b0375e347 Update top online [2026-03-12 08:47] by GitHub Action 2026-03-12 08:47:16 +00:00
github-actions[bot] d87328dc91 Ingest new games [2026-03-12 08:40] by GitHub Action 2026-03-12 08:40:13 +00:00
poli0981 72c0ae2501 New feature (#33)
* Add workflows to check dead links and update reviews

* Add new scripts for dead link check, ingest new games, and update reviews; enhance existing scripts with improved error handling and logging

* Refactor scripts for game management and data processing

- Enhanced `delete_game.py` to support interactive deletion of games by link, appid, or name, with improved user feedback.
- Updated `export_data.py` to export data from JSONL to CSV format, with error handling for missing files.
- Revamped `generate_tables.py` to create markdown tables from JSONL data, including genre-specific files and a README index.
- Introduced `ingest_new.py` for ingesting new game links from a temporary file, validating and normalizing links before fetching data.
- Removed `steam_fetcher.py` as its functionality is now integrated into other scripts.
- Overhauled `top_online.py` to generate a leaderboard for online games with fresh player counts and detailed statistics.
- Updated `update_data.py` to streamline the process of fetching and saving game details.
- Added `update_reviews.py` for periodic review updates of all games.

* Add temp_info.jsonl file for temporary data storage
2026-03-12 10:08:54 +07:00
poli0981 d04dd61355 perf: use data.jsonl replace data.json 2026-01-19 21:56:23 +07:00
github-actions[bot] 74b76341e7 Auto update CSV [2026-01-07] by GitHub Action 2026-01-07 01:23:38 +00:00
poli0981 ace445da91 Modify git commit message in csv_export.sh 2026-01-07 08:19:15 +07:00
poli0981 b131bde8ed Add automated CSV export from JSON data with GitHub Actions 2026-01-07 08:08:24 +07:00
poli0981 d57986fa00 Change Git config to use GitHub Actions bot
Updated Git configuration for GitHub Actions and modified commit message format.
2026-01-06 01:43:18 +07:00
poli0981 5b63cffea3 Update Git configuration for GitHub Actions 2026-01-06 01:34:25 +07:00
poli0981 151c6241d9 Fix indentation in online.sh for git config 2026-01-06 01:32:03 +07:00
poli0981 300894088b Update git config for GitHub Actions 2026-01-06 01:31:48 +07:00
poli0981 7e3387901f Merge remote-tracking branch 'origin/main' 2026-01-02 06:19:07 +07:00
poli0981 ca4c7d1bba fix: update file paths in generate_tables.py and remove unnecessary script listing in daily.sh 2026-01-02 06:18:51 +07:00
poli0981 c1f72d81b5 Auto update tables online top & stats [2026-01-01] 2026-01-01 23:14:24 +00:00
poli0981 3d27b77023 fix: update daily.sh to add all changes and improve commit handling 2026-01-02 06:13:41 +07:00
poli0981 0e1b94ff66 fix: streamline commit process and update echo messages 2026-01-02 06:10:19 +07:00
poli0981 b4300016a1 fix: correct echo message in empty commit handling 2026-01-02 06:08:01 +07:00
poli0981 48d4be04dd update git configuration and enhance JSON commit message 2026-01-02 06:04:36 +07:00
poli0981 0dfaacd9d2 fix 2026-01-02 06:01:56 +07:00
poli0981 543790b03f JSON updated [] 2026-01-01 22:51:56 +00:00
poli0981 824c622964 update 2026-01-02 05:48:16 +07:00
poli0981 d0f93f98da Modify commit command to exclude git pull
Removed the git pull --rebase command from the commit process.
2026-01-02 05:30:11 +07:00