Files
poli0981andClaude Opus 5.5 ec1bf939f6 docs: split CHANGELOG.md into one file per release
changelog/v<major>/<major>.<minor>.x/<version>.md for each of the 42
releases from v1.0.0 to v4.7.1, and changelog/beta/ for pre-releases
(none has been published; its README says how one would be filed).
CHANGELOG.md becomes the index: what the layout is, [Unreleased] exactly
as it was, and a table per major version (version, date, edition) that
links each file.

Each file is its section, copied byte for byte except that its heading
is the file's H1 (so the sub-headings move up one level, outside code
fences), relative links gain ../../../ so they name the same repository
paths, and the --- separators between sections are gone. Checked by
reversing all three changes on every file and comparing with the
original section: 40 of 40 equal. Two releases had no section: v1.0.0
(its GitHub release notes, which the changelog pointed to as "Initial
stable") and v1.1.3 (its tag message).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 09:02:37 +07:00

5.3 KiB
Raw Permalink Blame History

v2.1.0 - 2026-03-28 (The "Extension-Ready + Performance" Edition)

All versions

💥 Breaking Changes

  • desc field removed from schema. Use description only. Migration script handles upgrade automatically.
  • free_type field removed (unreliable from API, extension handles classification client-side).
  • jsonlines pip dependency dropped – all I/O now uses stdlib json (faster, zero deps).
  • fetcher.py function renamed: _process_batch() → process_batch() (public API).

🏗️ Schema v2.1 (Extension-Compatible)

  • 7 new fields added to every game record, matching Chrome extension output:
    • publisher (list) – e.g. ["Square Enix", "Feral Interactive (Mac)"]
    • platforms (list) – e.g. ["Windows", "macOS", "Linux"]
    • languages (list) – e.g. ["English", "French", "Japanese"]
    • language_details (list[dict]) – per-language {name, interface, audio, subtitles} booleans
    • tags (list) – all user-defined Steam tags including hidden overflow
    • anti_cheat_note (str) – raw anti-cheat name from store page
    • is_kernel_ac (bool|null) – true = kernel-level, false = non-kernel, null = unknown
  • developer normalized: string → list everywhere (API, extension, migration).
  • desc merged into description, then dropped. One field, no duplication.

🔌 Extension Data Integration

  • New merge_extension_data() in data_store.py – smart merge with 3-tier priority:
    • MANUAL_FIELDS (notes, safe, type_game, genre, anti_cheat): only fill if empty (preserves user edits).
    • ARRAY_FIELDS (developer, publisher, platforms, languages, tags): replace wholesale (extension data is richer).
    • Other fields: overwrite if extension provides non-empty value.
  • ingest_new.py now accepts full extension output (20+ fields per entry) – merges before fetch, re-applies manual overrides after fetch.
  • Ingest summary shows field counts: dev=DONTNOD plat=[Windows, macOS, Linux] langs=6 tags=20.

🔍 HTML Scraper (New Module)

  • New scraper.py – extracts data unavailable from Steam API:
    • Language table: parses <table class="game_language_options"> → accurate Interface/Audio/Subtitles per language (API only gives a broken HTML string with no subtitle info).
    • DLC pricing: parses #gameAreaDLCSection → checks data-price-final per DLC row. Only marks has_paid_dlc=true if at least one DLC has actual non-zero price (API just lists DLC IDs with no prices).
    • Tags: parses .app_tag elements including display:none overflow tags (API appdetails doesn't return user tags at all).
  • All 3 extracted from single GET request to store page – one HTML fetch per game, not three.
  • 8 regex patterns pre-compiled at module level.

⚡ Performance Optimizations

  • Eliminated double-fetch in ingest: health_checker now caches API response in HealthResult.data. ingest_new.py passes prefetched_details=health.data to fetch_full() → skips redundant fetch_app_details() call. Saves 1 API request per game (300 games = ~450s saved).
  • Dropped jsonlines dependency: load_jsonl() uses json.loads() line-by-line, save_jsonl() uses json.dumps() + f.write() directly. ~20-30% faster I/O for large files, zero pip deps for core.
  • Pre-compiled regex: scraper.py compiles all 8 patterns at import time, not per-call. 300 games = 8 compiles instead of 2400.
  • Faster is_empty(): uses frozenset lookup for empty string variants, early returns for None/list.
  • SteamClient.__slots__: reduced memory per instance, faster attribute access.
  • extract_appid() regex simplified: /app/(\d+) instead of full domain match – shorter, same correctness.
  • make_skeleton() template: shallow-copies from frozen template dict instead of building from scratch.
  • generate_tables.py: lines.extend(generator) instead of append() loop.

🔄 New Scripts & Workflows

  • refetch_all.py + refetch-all.yml – manual-only force re-fetch of ALL games into v2.1 schema. Creates backup → migrates → clears fetchable fields → re-fetches from Steam API + HTML scrape. Preserves manual fields (notes, safe, type_game). Requires typing "yes" to confirm. Timeout: 120 minutes.
  • migrate_schema.py + migrate-schema.yml – upgrades existing data.jsonl records: adds missing fields, normalizes types, drops deprecated fields. Idempotent.

🐛 Bug Fixes

  • languages corrupted: API HTML string "Brazil<br>languages with full audio" was parsed as "Brazillanguages...". Fixed by switching to HTML table scraping.
  • subtitles always false: API has no subtitle breakdown. Fixed by parsing store page language table ( Interface/Audio/Subtitles columns).
  • has_paid_dlc false positives: Games with free-only DLC were marked true. Fixed by scraping actual data-price-final attributes.
  • description duplicated: Both desc and description stored same text. Unified to description only.

🧹 Improved

  • Tables: Publisher column, Platforms column, Language count, kernel anti-cheat badge (🔴/🟢).
  • health_checker.py: @dataclass(slots=True) for leaner objects.
  • top_online.py: tier thresholds as tuple list (cleaner).
  • All scripts: consistent sys.path.insert(0, ...) for imports.