- queue: max on every data-write workflow (GitHub, 2026-05-07): up to
100 runs wait, first in, first out. The default kept one, and a run
queueing behind a running job cancelled it.
- ref: main on every checkout. A run's commit is fixed when it is
created, so a run that waited behind another data job wrote shards
from before that job's commit; Auto Update JSON failed four days in
ten on the rebase conflict in data/index.json.
- timeout-minutes everywhere, and JOB_BUDGET_MINUTES at least 20 below
it for the long loops (never for the ingest, which empties its queue).
- pipeline.test.ts fails on a data-write workflow missing any of these.
- The admin's Pipeline and Automation pages, docs/ADMIN.md and the
auto-approve comment no longer say a push evicts a waiting job; the
idle gate stays, so a batch's ingest starts at once.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
It sent a HEAD request to every store page every five days, 5.5 hours at
9,847 games, and flagged 404/410. Steam sends a delisted app's page to
the store front, which answers 200: scripts/dead_links.log never existed
and no game ever became "delisted". The purge's GetItems screen sees
delisted games (unlisted) instead.
Removes the workflow, bash/dead_links.sh, scripts/check_dead_links.py
and DEAD_LINKS_LOG, and the workflow from web/shared/pipeline.ts and the
failure notifier's list. check_store_page() stays: it is the health
check's last resort.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Refresh Store Data: the page and appdetails cycles grow past 2,000
pages / 500 appdetails a day (7 -> 14, 30 -> 60) instead of the run,
the slice comes in a daily order, and the store step has a budget. A
game appdetails no longer calls free is listed in a ::warning::, the
case the purge's two-signal rule cannot see; nothing is written.
- Refresh All-Time Peaks: 30 -> 60 days past 400 pages a day, never
shorter, so SteamCharts is still asked at most once a month a game;
the budget stops it with a warning like a refusal does.
- Force Re-fetch: --part K/N, oldest last_updated first, each game
cleared on a copy that replaces it only once appdetails has answered;
no final loop stamping every game. Parts queue one after another.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The purge asked appdetails about every game, 5.3 hours at 9,847, and one
answer was enough to remove one - appdetails has answered success:false
for live games in bursts (2026-09-27).
GetItems now screens the whole catalogue in a few minutes (unknown,
hidden, unlisted, coming soon, not is_free; never type), and
check_game_health() must confirm each flag; "unavailable" is asked again
five minutes later. A failed batch or a network error never reads as
gone, blocklisted games are not asked about, and more than 500 flags or
100 removals stops the run before anything is written. Removals are
logged as before. A notice names earlier removals Steam has since undone.
The dry run on the real catalogue: 8 flagged, 3 removed (the three
unlisted games), 5 kept because appdetails calls them free. Daily now.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Update Reviews asked appreviews about one game at a time, two seconds a
game. At 9,847 games it ran into GitHub's six-hour limit on 2026-10-23
and was stopped with nothing saved; five of its ten runs before that
were cancelled while waiting.
- SteamClient.fetch_items(): GetItems for a whole catalogue, a failed
batch costing only its own ids (the anti-cheat sweep now uses it).
- update_reviews.py reads review_summary() for every game, ~2 minutes,
daily. Steam floors its percentage; reviews_from_summary() floors
appreviews in integers too, so an ingest and the refresh agree. A
count of 0 is never written on GetItems' word (it said 0 for a game
with thousands): a stored percentage or "N/A" is asked on appreviews,
at most 300 a run. last_updated moves on a change, and on a 14-20 day
heartbeat that keeps the site's 30-day "stale" badge quiet without
rewriting every record every run.
- core/budget.py: JOB_BUDGET_MINUTES. Past it process_batch() starts no
further game, so a job saves what it did instead of being killed;
a ::warning:: when a loop used three quarters of its budget.
- rotation.cycle_for() / daily_order(): cycles that grow with the
catalogue as multiples of their base, and an order that changes daily.
- The player-count loops skip the batch pauses meant for store pages
(Top Online ~27 min instead of 48) and take a deadline, like
update_all_full().
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
SonarCloud's GitHub Actions and shell findings, all supply-chain hardening
for jobs that hold a write token (and, in the release workflows, the
signing keys):
- The six third-party actions in release-desktop.yml and release-android.yml
are pinned to the commit their major tag points at today, with the version
in a comment for Dependabot. dtolnay/rust-toolchain is pinned to a master
commit and given `toolchain: stable`: its `stable` branch is regenerated
on top of master, so a commit of it can vanish.
- Every `npm ci` is `npm ci --ignore-scripts`. Only fsevents, workerd and
wrangler's esbuild have install scripts; the last two find their binary in
their optional platform package without them, and fsevents is only file
watching. `npx` is gone from the workflows (`npm run cf:check`,
`npm test -- <file>`).
- requirements.txt pins every runtime package, transitive ones included
(certifi, charset-normalizer, idna), with the sha256 of every file on
PyPI; every install is `--require-hashes --only-binary :all:`. The new
requirements-test.txt holds pytest the same way. python-tests.yml runs the
data jobs' exact install line first, so a Dependabot bump is proven before
a data job meets it.
- `[[ ]]` instead of `[ ]` in the bash wrappers.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
None of the 4,704 games added since September had an anti-cheat. The only
code that read the anti-cheat a developer declares on Steam was the browser
extension (archived; last push 2026-06-25); the pipeline matched appdetails'
categories, which only ever say "Valve Anti-Cheat enabled".
- core/anticheat.py (new, pure): the store page's div.anticheat_section
(its HEADING decides kernel, never the class; a Boot Protection row is a
flag) and GetItems' extra_details.anticheat; labels from
ANTI_CHEAT_PATTERNS, else Steam's name cut at 40 characters; same_family()
("EAC" = "Easy Anti-Cheat", "ACE" = "Anti-Cheat Expert (ACE)"); and
apply_anti_cheat(), the one rule.
- fetch_full() fills, never replaces, from the page it already fetches (or
GetItems for a gated page), else the VAC category, now found in any
position (with a stored "" only the first category was checked).
- refresh_anticheat.py (new): GetItems for every game (~160 requests), a
store page only for a disclosure without a name. Steam wins; an override
wins over Steam and is not fought (OverrideGuard); no disclosure never
clears; a failed batch is unknown. Runs at the end of the daily Refresh
Store Data job with its own commit; only=anticheat runs it alone. That job
now checks out main.
- The queue takes links only: merge_extension_data(), EXTENSION_FIELDS,
ARRAY_FIELDS and the schema.ts mirrors go. translate_notes() becomes
clean_notes(): the extension's leading "Has paid DLC" and "[hover-add]"
are dropped (517 notes, once, never news).
- refetch_all.py keeps "-" instead of resetting it to ""; a retired override
whose was equals its value no longer bumps last_updated on every save.
- "denuvo" joins Denuvo AC on both sides; anti-cheat.test.ts holds the TS
patterns and enum to constants.py.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A game the maintainer removes is now a standing instruction in Git, like an
override, instead of a one-time deletion that the browser extension or the
next ingest could undo.
- core/blocklist.py reads data/blocklist.jsonl (one line per game, written by
/admin and by delete_game.py). save_main() applies it FIRST on every write,
for all 14 callers, without touching the caller's list, and whatever
apply_human_overrides says.
- A dropped game is logged once in scripts/removed_games.jsonl as
status_code "admin_removed", built from the blocklist line alone with
removed_at = blocked_at, so writing it again changes no byte. A later purge
line for the same game keeps winning, as dedup_removed() decides.
- More than 500 drops in one save raises BlocklistError before anything is
written: a damaged file fails the job instead of emptying the catalogue.
- remove_games.py applies the blocklist now and heals a missing log line.
bash/ingest.sh runs it before ingest_new.py, and ingest-new.yml also
triggers on blocklist pushes and checks out the tip of main. One workflow
on purpose: data-write keeps one pending run, so a second workflow and the
ingest would keep evicting each other.
- ingest_new.py skips blocked links before any Steam request, and a link
that is not text ({"link": 730}) no longer crashes the run (and every
three-hourly retry after it).
- Discovery counts blocked appids as known.
- delete_game.py is a non-interactive CLI writing the same bytes as /admin
(golden fixture), with --list and --check; check-overrides.yml runs --check.
- mark-dead-games.yml commits through bash/commit_push.sh, like every other
data job: it rebases and retries, and commits the log with the shards.
- dependabot.yml ignores Svelte-era framework majors and @types/node majors
instead of the React-era names.
With an empty blocklist every path is a no-op.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The dataset only knew how many people play a game now. The only peak it had,
peak_today, is not today's: nothing resets it, so it is the highest of our
sparse samples (online games 3x a week, the rest twice a month) since the
last refetch - The First Descendant reads 8,052 against a real all-time peak
of 264,860.
all_time_peak is the most players a game has had at once, and it only ever
goes up (core/peaks.py):
- refresh_peaks.py reads SteamCharts' recorded all-time peak, the only public
source older than our sampling (SteamDB forbids scraping; Valve's APIs only
know today). It takes a crc32 thirtieth of the catalogue a day, so each game
is asked once a month, ~186 pages 2-3 s apart. SteamCharts has no API, no
terms and no robots.txt, so it is read politely (core/steamcharts.py): a
User-Agent that names the project, 500/404 read as "not tracked", and a
403, 429 or Cloudflare challenge stops the run with a warning, keeping
everything raised so far. Measured: 9 of 10 games with a few dozen players
or more are tracked; most of the 0-3 player long tail is not.
- Every run also raises the field from our own samples (peak_today,
current_players) without a request, and apply_players() raises it on every
sample - never from a 0. That seeds 2,329 games on the first run.
- Nothing else is written, last_updated included (player counts never moved
it), and refetch_all.py does not clear the field: it cannot be fetched again.
"Refresh All-Time Peaks" runs daily at 19:30 UTC, between Refresh Store Data
and the snapshot, in the data-write group, committing through
commit_push.sh; a manual run with "backfill" asks about every game (~4 h).
The crc32 rotation moves to core/rotation.py, shared with
refresh_store_data.py. test_refresh_peaks.py holds the rules.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
update_data.py only enriches records that are incomplete, and fetcher.py's
apply_details()/apply_scraped() only fill EMPTY fields. Once a game had its
tags, languages, platforms or header image, nothing looked at them again
short of the manual refetch_all.py. A dry run on three games found all three
with tags that had drifted since they were first fetched.
scripts/refresh_store_data.py re-reads them a slice at a time, daily:
- store page (tags, languages, language_details, has_paid_dlc): every game
once every 7 days;
- appdetails (name, header_image, description, developer, publisher,
release_date, platforms, metacritic, drm_notes): every game once every
30 days.
Measured on the live catalogue: 600-675 store pages and 121-166 appdetails
calls a day, about 40 minutes at the client's throttle, instead of ~8,800
requests in one sitting.
A game is due when crc32(appid) % cycle == day % cycle. Not appid % cycle:
Steam appids are nearly all multiples of 10, which would leave 27 of the 30
monthly slices empty.
These fields were fill-if-empty until now, so the refresh keeps the same
care: it replaces a value only with a non-empty one (a failed request or an
age-gated page changes nothing), never writes MANUAL_FIELDS or notes, reads
has_paid_dlc only from a real store page, compares description as stored
(truncated to DESCRIPTION_MAX), bumps last_updated only on a real change,
and writes through save_main() so overrides and index.json behave as for
every other job. scripts/tests/test_refresh_store_data.py pins each rule and
the even spread of the slices.
"Refresh Store Data" runs at 16:00 UTC: scheduled runs here start 2-5 hours
late, the morning data jobs run 2-4 hours, and the data-write group keeps
only one pending run, so the slot is the one nothing long overlaps. It is in
data-write, commits through bash/commit_push.sh, and is on the CI-failure
notification list. README, CLAUDE.md and the changelog describe it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
purge-unhealthy ran for four hours on 2026-09-21 and then failed on
"push rejected (fetch first)", throwing the whole run away. The data-write
concurrency group serialises the WORKFLOWS, but main still moves under a
long job: a merged pull request, an /admin approval or override the Worker
commits, the extension's push to scripts/temp_info.jsonl.
bash/commit_push.sh does what every wrapper did - add, commit only if
something changed - and then pushes with up to five attempts, rebasing onto
origin/main between them. None of the other writers touch the shards, so
the rebase is clean; a genuine conflict still fails the job. All nine data
wrappers now call it, with their commit messages unchanged (snapshot.sh
keeps its data/snapshots/ pathspec).
Tested against a scratch remote: with main moved by another commit, the
first push is rejected, the rebase lands the job's commit on top, and the
retry succeeds; with nothing to commit it prints "No changes".
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
games/ held 1.8 MB of markdown regenerated from data/*.jsonl on every data
change: 18 all-games_part*.md files, anti-cheat-list.md, a README index of
them, and a .gen-hash checksum whose only purpose was to make regenerating
them cheaper. The web app has been the way to browse the catalogue since v3.0,
so these were a second rendering of the same records that had to be rebuilt,
committed and reviewed forever.
Kept: games/top-online.md and games/top-offline.md.
Removed with them, because nothing else used them:
scripts/generate_tables.py, scripts/generate_anti_cheat_list.py
.github/workflows/update-daily.yml ("Generate Markdown Tables")
.github/workflows/anti-cheat-list.yml
bash/daily.sh, bash/anti_cheat.sh
and the generate_tables call inside ingest.sh, purge.sh, refetch_all.sh and
ingest-from-issue.yml.
Nothing depended on the anti-cheat markdown: /charts/anti-cheat reads
data/*.jsonl, not that file.
Four references would have broken, two of them user-visible:
- web/public/sitemap.xml advertised all-games_part1.md to search engines, and
About.tsx had an "All games (markdown)" button. Both would now 404. The
sitemap entry is gone; the button points at data/ instead.
- .github/workflows/mark-dead-games.yml ended with
`gh workflow run "Generate Markdown Tables"`. That command exits non-zero
against a workflow that does not exist, so every successful dead-games run
would have finished red.
- notify-ci-failure.yml's allowlist is matched by exact workflow NAME, and the
file's own comment warns that a stale entry breaks silently.
Also fixed the stale poli0981.github.io URL in the bug-report template while
removing its games/*.md option.
Verified: typecheck, build, all 24 workflow YAML files parse, sitemap.xml is
well-formed, and no reference to the removed files survives outside CHANGELOG
and docs/plan (both historical).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
isChunkLoadError only matched the three "failed to fetch dynamically imported
module" style messages. Those cover a 404 on a stale chunk, but not what a
host with an SPA fallback returns: a request for a deleted hashed chunk is
answered with index.html at HTTP 200 and Content-Type: text/html, so the
browser rejects it on MIME type instead. The message looks nothing like a
fetch failure, so the reload-once recovery would silently stop working the
moment the site moves behind Cloudflare's not_found_handling. Added the
"failed to load module script" / "expected a javascript module script"
patterns and lifted the regex to a named constant.
Python installs now go through requirements.txt at all 11 sites — nine
bash/*.sh scripts and the two inline workflow steps — instead of a bare
`pip install requests`. The bare install silently ignored the urllib3 pin,
so CI was resolving a dependency the requirements file was trying to
constrain. mark-dead-games.yml already did this correctly.
requirements.txt and requirements-dev.txt move from `>=x,<y` ranges to exact
`==` pins: the pipeline should install identical versions on every run, and
Dependabot does not raise PRs for range-specified dependencies, so the new
pip ecosystem entry in dependabot.yml would otherwise have had nothing to do.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The scheduled pipeline has been failing daily since 2026-08-13 (last successful
`Auto update` was e8b7d1b2). Root cause: `secrets.GH_TOKEN`, a PAT, expired.
Every workflow passing it to actions/checkout dies at fetch with exit 128.
No workflow actually needs a PAT. Each already declares
`permissions: {contents: write}`, so the built-in GITHUB_TOKEN has push rights —
proven by mark-dead-games.yml, the only green data workflow and the only one
with a bare checkout. Removed the `token:` input from all 12 sites. This also
ends the silent-expiry failure mode rather than rescheduling it a year out.
`workflow_run` triggers are unaffected: GITHUB_TOKEN suppression applies to
push/issues/release events, and update-daily.yml plus the three notify-*
workflows all key off workflow_run, which the Actions service emits on run
completion regardless of the checkout token.
Also in this commit:
- ingest-from-issue.yml gated on `author_association`. Without it, any user
could open an issue titled [add-game] and have their JSON committed and
pushed to main with `contents: write`. A title match is not an authorization
check. Its `&&`/`||` commit chain is also parenthesised — left-associativity
meant it pushed even when there was nothing staged.
- bot-ingest.yml's final step deleted. It referenced `secrets.BOT_TOKEN` while
the rest of the file uses TELEGRAM_BOT_TOKEN, `cat`-ed a result.txt no step
creates, ran under `if: always()`, and interpolated a secret straight into a
shell command line. The Python step above it already does the same Telegram
callback correctly. `secrets.BOT_TOKEN` is now unreferenced.
- The snapshot feature is committed. snapshot-daily.yml gains the
`concurrency: {group: data-write}` guard every other data-writing workflow
has — without it a 23:00 run can push against jobs that take ~2 hours. The
`pip install requests` in bash/snapshot.sh is dropped: snapshot.py and
scripts/core/* import only stdlib.
- New web-ci.yml. deploy-pages.yml was the only thing running `npm run
typecheck`, so deleting it during the Cloudflare migration would leave the
frontend unchecked. Uses a plain `npm ci` — the old `npm ci || npm install`
fallback is what let the lockfile conflict above go unnoticed.
- New dependabot.yml. NOTE: an in-repo config replaces the one in repo
settings and supersedes open PR #111. Groups security updates so the
transitive backlog collapses into one PR instead of one per package.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
- `scripts/core/data_store.py` adds `dedup_removed(records)` — keeps latest entry per appid (by `removed_at`). Call sites updated in `purge_unhealthy.py:41-43` and `ingest_new.py:106-108` so every write through the existing `save_jsonl` atomic-rename transaction is dedup-clean. One-shot `scripts/dedup_removed_games.py` ran the historical cleanup (43 → 37 records, 6 duplicates dropped).
- `scripts/core/constants.py:81` bumps `TOP_ONLINE_LIMIT` 50 → 100. Top Offline already 100; `TopOnline.tsx` and the `topOnline.{subtitle,cardTitle}` strings updated to match.
- `scripts/generate_anti_cheat_list.py` + `bash/anti_cheat.sh` + `.github/workflows/anti-cheat-list.yml` — weekly-cron pipeline that buckets active games by canonical AC name (`ANTI_CHEAT_PATTERNS`, substring-match for free-text values, `Other` bucket for unrecognised) and emits `games/anti-cheat-list.md` with TOC + per-family 6-column tables sorted by current_players desc. First run: 54 games across 8 known families + 17 in `Other`.
- `web/src/pages/AntiCheatList.tsx` — `/charts/anti-cheat/list` page mirrors the MD: collapsible per-family sections + sticky AC-name sidebar nav. TS port of the Python canonicalisation table in-file so behaviour stays in sync.
- `web/src/hooks/useRemovedGames.ts` + `DelistedTimeline.tsx` + `DelistedReasonBreakdown.tsx` + `pages/charts/Delisted.tsx` — fetches `removed_games.jsonl` from raw.githubusercontent.com, dedups client-side, renders stacked-bar timeline by month/`status_code` + donut by `status_code`. Available at `/charts/delisted`.
- `web/src/components/charts/KpiCards.tsx` — new optional `removedCount` prop adds a 7th "Removed (all-time)" KPI when supplied; grid bumps `xl:grid-cols-6 → 7`. `Dashboard.tsx` wires `useRemovedGames().data?.length` through.
- Sidebar nav: new `AC Index` link (under existing Anti-Cheat) and `Delisted` link (under DRM/DLC).
- i18n: `antiCheatList.*`, `charts.delisted.*`, `nav.{antiCheatList,delisted}` keys added to `en.json` + `vi.json`.
Verification: `npm run build` (2418 modules), `npm run typecheck`, Python imports clean, dedup migration idempotent on re-run.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* v3.2.0: SEO, Donate page, Discussion auto-post, Health bucket split, dev docs
Bundles 8 features for v3.2.0 (per the planning round):
1. SEO basics — OG / Twitter card / canonical / JSON-LD in index.html;
robots.txt + sitemap.xml in web/public; useDocumentTitle hook wired
into every page so the tab title reflects the current route +
language.
2. Auto-post a GitHub Discussion on release `published` —
.github/workflows/announce-release-discussion.yml. Uses GraphQL
`createDiscussion`, idempotent (skips if a discussion with the same
title already exists), prefers the `Announcements` category with
fallback to `General`.
3. Dev docs — docs/pc_spec.md, docs/dev_env.md, VI mirrors under
docs/i18n/vi/, web/src-tauri/TAURI.md (build prerequisites + signing
notes). AUTHORS.md gets a "Dev info" table; README.md links the new
docs.
4. Donate page (/donate) with 5 platform cards
(GitHub Sponsors / Ko-fi / BMC / Patreon / PayPal) sourced from
.github/FUNDING.yml. New nav item, EN + VI i18n keys.
5. Telegram bot user guide — new docs/telegram-bot.md pointer page
linking the external USER_GUIDE.md, bot source repo, and the
in-repo bot-ingest.yml workflow. CONTRIBUTING.md and About page get
the same link.
6. Health bucket split — separates "Online + AC + kernel unknown" from
"Online · anti-cheat unknown / none" so games with empty `anti_cheat`
no longer disappear from the report.
7. Steam Desktop direct link — game detail drawer now shows two buttons:
`steam://store/<appid>` (Steam client) and the regular
store.steampowered.com URL. Helper at web/src/lib/steam-link.ts.
8. Bi-weekly offline player leaderboard — scripts/top_offline.py +
bash/offline.sh + .github/workflows/top-offline.yml (cron 1st + 15th
of each month, plus workflow_dispatch). Output:
games/top-offline.md, top 100 single-player F2P concurrents.
Health page gets a manual trigger button.
Verification: `cd web && npm run typecheck && npm run build` — both
pass; PWA generates 17 precache entries; manifest.webmanifest emitted
by VitePWA as before.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(top-offline): make update_players_only honour offline filter
`update_players_only(games)` had a hardcoded `type_game == "online"`
filter, so passing it the full games list from top_offline.py refreshed
ZERO offline player counts and the resulting top-offline.md was sorted
on stale data (last touched by the daily update_data.py).
Extend the function with a backward-compatible `type_filter` kwarg
("online" default, plus "offline" and "all"). top_online.py keeps its
existing call shape; top_offline.py now passes type_filter="offline"
which excludes online AND empty-type rows (unclassified is not
"offline").
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore(release): bump to v3.2.0 / desktop v1.2.0 + CHANGELOG
- web/package.json + lockfile: 1.1.0 → 1.2.0
- web/src-tauri/tauri.conf.json: 1.1.0 → 1.2.0
- web/src-tauri/Cargo.toml: 1.1.0 → 1.2.0
- README badge: 3.1.0 → 3.2.0
- CHANGELOG.md: new v3.2.0 entry covering all 8 features in this PR
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Add workflows to check dead links and update reviews
* Add new scripts for dead link check, ingest new games, and update reviews; enhance existing scripts with improved error handling and logging
* Refactor scripts for game management and data processing
- Enhanced `delete_game.py` to support interactive deletion of games by link, appid, or name, with improved user feedback.
- Updated `export_data.py` to export data from JSONL to CSV format, with error handling for missing files.
- Revamped `generate_tables.py` to create markdown tables from JSONL data, including genre-specific files and a README index.
- Introduced `ingest_new.py` for ingesting new game links from a temporary file, validating and normalizing links before fetching data.
- Removed `steam_fetcher.py` as its functionality is now integrated into other scripts.
- Overhauled `top_online.py` to generate a leaderboard for online games with fresh player counts and detailed statistics.
- Updated `update_data.py` to streamline the process of fetching and saving game details.
- Added `update_reviews.py` for periodic review updates of all games.
* Add temp_info.jsonl file for temporary data storage