Files
poli0981andClaude Opus 5.5 1da4d5940e feat(pipeline): keep the store, peaks and re-fetch runs bounded
- Refresh Store Data: the page and appdetails cycles grow past 2,000
  pages / 500 appdetails a day (7 -> 14, 30 -> 60) instead of the run,
  the slice comes in a daily order, and the store step has a budget. A
  game appdetails no longer calls free is listed in a ::warning::, the
  case the purge's two-signal rule cannot see; nothing is written.
- Refresh All-Time Peaks: 30 -> 60 days past 400 pages a day, never
  shorter, so SteamCharts is still asked at most once a month a game;
  the budget stops it with a warning like a refusal does.
- Force Re-fetch: --part K/N, oldest last_updated first, each game
  cleared on a copy that replaces it only once appdetails has answered;
  no final loop stamping every game. Parts queue one after another.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 22:22:08 +07:00

11 lines
458 B
Bash
Executable File

#!/bin/bash
set -euo pipefail
# REFETCH_PART is the manual run's "part" box, K/N (scripts/refetch_all.py
# checks it): the whole catalogue no longer fits in one job.
part="${REFETCH_PART:-1/1}"
echo "── Force re-fetch, part ${part} ──"
pip install --quiet --require-hashes --only-binary :all: -r requirements.txt
python scripts/refetch_all.py --part "$part"
bash "$(dirname "$0")/commit_push.sh" "Force re-fetch, part ${part} [$(date +'%Y-%m-%d')]"