- Refresh Store Data: the page and appdetails cycles grow past 2,000 pages / 500 appdetails a day (7 -> 14, 30 -> 60) instead of the run, the slice comes in a daily order, and the store step has a budget. A game appdetails no longer calls free is listed in a ::warning::, the case the purge's two-signal rule cannot see; nothing is written. - Refresh All-Time Peaks: 30 -> 60 days past 400 pages a day, never shorter, so SteamCharts is still asked at most once a month a game; the budget stops it with a warning like a refusal does. - Force Re-fetch: --part K/N, oldest last_updated first, each game cleared on a copy that replaces it only once appdetails has answered; no final loop stamping every game. Parts queue one after another. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
61 lines
2.4 KiB
YAML
61 lines
2.4 KiB
YAML
name: Refresh Store Data
|
|
|
|
# The store fields update_data.py fills once and never revisits (it only
|
|
# enriches INCOMPLETE records): tags, languages and the paid-DLC flag from the
|
|
# store page, every game once a week; name, header image, description,
|
|
# developer/publisher, release date, platforms, Metacritic and DRM notes from
|
|
# appdetails, every game once a month. Each daily run takes one slice of each
|
|
# rotation - about 1,400 store pages and 330 appdetails calls at 9,800 games,
|
|
# an hour. Past 2,000 pages or 500 appdetails a day a cycle doubles instead of
|
|
# the run growing, so it stays under an hour and a half.
|
|
# scripts/refresh_store_data.py explains the rotation and the rules (never
|
|
# blanks a value, never touches MANUAL_FIELDS).
|
|
#
|
|
# Then the anti-cheat sweep, committed separately: every game's anti-cheat as
|
|
# its developer declares it on Steam, about 200 GetItems requests plus a store
|
|
# page for each disclosure Steam does not name there (~70), a few minutes.
|
|
# scripts/refresh_anticheat.py has its rules: Steam wins, an override wins
|
|
# over Steam, and no disclosure never clears a value. `only: anticheat` runs
|
|
# just the sweep.
|
|
#
|
|
# 16:00 UTC is the quiet part of this repository's day: scheduled runs here
|
|
# start up to an hour late, and the morning jobs are done by then. A data job
|
|
# that is still running only delays this one - the data-write group queues
|
|
# runs (queue: max) instead of evicting the one waiting.
|
|
|
|
on:
|
|
schedule: [{cron: '0 16 * * *'}]
|
|
workflow_dispatch:
|
|
inputs:
|
|
only:
|
|
description: 'One part only: a store group, or the anti-cheat sweep; leave empty for all'
|
|
required: false
|
|
default: ''
|
|
type: choice
|
|
options: ['', 'page', 'details', 'anticheat']
|
|
|
|
permissions: {contents: write}
|
|
|
|
concurrency:
|
|
group: data-write
|
|
cancel-in-progress: false
|
|
queue: max
|
|
|
|
jobs:
|
|
refresh:
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 150
|
|
steps:
|
|
# main, not the triggering commit: a run that waited its turn in
|
|
# data-write must work on the data as it is now, not as it was queued.
|
|
- uses: actions/checkout@v6
|
|
with: {ref: main}
|
|
- uses: actions/setup-python@v6
|
|
with: {python-version: '3.15'}
|
|
- run: chmod +x ./bash/refresh_store.sh && ./bash/refresh_store.sh
|
|
env:
|
|
REFRESH_ONLY: ${{ inputs.only }}
|
|
# The store refresh's budget (core/budget.py); the anti-cheat sweep
|
|
# after it takes a few minutes and has none.
|
|
JOB_BUDGET_MINUTES: '100'
|