Files
free-steam-games-list/scripts/update_data.py
T
poli0981andClaude Opus 5.5 49f0c56ed7 feat(pipeline): read reviews for every game from Steam's store API
Update Reviews asked appreviews about one game at a time, two seconds a
game. At 9,847 games it ran into GitHub's six-hour limit on 2026-10-23
and was stopped with nothing saved; five of its ten runs before that
were cancelled while waiting.

- SteamClient.fetch_items(): GetItems for a whole catalogue, a failed
  batch costing only its own ids (the anti-cheat sweep now uses it).
- update_reviews.py reads review_summary() for every game, ~2 minutes,
  daily. Steam floors its percentage; reviews_from_summary() floors
  appreviews in integers too, so an ingest and the refresh agree. A
  count of 0 is never written on GetItems' word (it said 0 for a game
  with thousands): a stored percentage or "N/A" is asked on appreviews,
  at most 300 a run. last_updated moves on a change, and on a 14-20 day
  heartbeat that keeps the site's 30-day "stale" badge quiet without
  rewriting every record every run.
- core/budget.py: JOB_BUDGET_MINUTES. Past it process_batch() starts no
  further game, so a job saves what it did instead of being killed;
  a ::warning:: when a loop used three quarters of its budget.
- rotation.cycle_for() / daily_order(): cycles that grow with the
  catalogue as multiples of their base, and an order that changes daily.
- The player-count loops skip the batch pauses meant for store pages
  (Top Online ~27 min instead of 48) and take a deadline, like
  update_all_full().

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 22:21:59 +07:00

19 lines
593 B
Python
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
#!/usr/bin/env python3
"""Full data update – enriches incomplete records."""
import sys, os
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from core.budget import Deadline
from core.data_store import load_main, save_main, migrate_record
from core.fetcher import update_all_full
def main():
deadline = Deadline.from_env()
games = load_main()
print(f"Loaded {len(games)} games")
for g in games: migrate_record(g)
update_all_full(games, deadline=deadline)
save_main(games)
print(f"\n✓ Saved {len(games)} games")
if __name__ == "__main__": main()