A game the maintainer removes is now a standing instruction in Git, like an
override, instead of a one-time deletion that the browser extension or the
next ingest could undo.
- core/blocklist.py reads data/blocklist.jsonl (one line per game, written by
/admin and by delete_game.py). save_main() applies it FIRST on every write,
for all 14 callers, without touching the caller's list, and whatever
apply_human_overrides says.
- A dropped game is logged once in scripts/removed_games.jsonl as
status_code "admin_removed", built from the blocklist line alone with
removed_at = blocked_at, so writing it again changes no byte. A later purge
line for the same game keeps winning, as dedup_removed() decides.
- More than 500 drops in one save raises BlocklistError before anything is
written: a damaged file fails the job instead of emptying the catalogue.
- remove_games.py applies the blocklist now and heals a missing log line.
bash/ingest.sh runs it before ingest_new.py, and ingest-new.yml also
triggers on blocklist pushes and checks out the tip of main. One workflow
on purpose: data-write keeps one pending run, so a second workflow and the
ingest would keep evicting each other.
- ingest_new.py skips blocked links before any Steam request, and a link
that is not text ({"link": 730}) no longer crashes the run (and every
three-hourly retry after it).
- Discovery counts blocked appids as known.
- delete_game.py is a non-interactive CLI writing the same bytes as /admin
(golden fixture), with --list and --check; check-overrides.yml runs --check.
- mark-dead-games.yml commits through bash/commit_push.sh, like every other
data job: it rebases and retries, and commits the log with the shards.
- dependabot.yml ignores Svelte-era framework majors and @types/node majors
instead of the React-era names.
With an empty blocklist every path is a no-op.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
64 lines
2.5 KiB
Python
64 lines
2.5 KiB
Python
#!/usr/bin/env python3
|
|
"""
|
|
Apply data/blocklist.jsonl now: take every blocked game out of data/ and log it.
|
|
|
|
save_main() already leaves blocked games out of EVERY write, so any data job
|
|
would do this on its own. This script is what makes it happen promptly: the
|
|
"Ingest New Game Links" workflow runs it first whenever /admin (or
|
|
delete_game.py) commits to the blocklist, and on its three-hourly schedule.
|
|
|
|
It also heals the log: a blocked game that is no longer in data/ but has no
|
|
line in scripts/removed_games.jsonl gets one. That covers a removal applied by
|
|
a job whose commit left the log out. Every blocklist line was written for a game
|
|
the catalogue held at the time (/admin and delete_game.py both refuse
|
|
anything else), so each logged line describes a real removal.
|
|
|
|
python scripts/remove_games.py # apply and log
|
|
python scripts/remove_games.py --max-drop 900 # a deliberate mass removal
|
|
|
|
Run from the repository root (DATA_DIR is relative).
|
|
"""
|
|
import argparse
|
|
import os
|
|
import sys
|
|
|
|
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
|
|
|
from core.blocklist import load_blocklist, removed_row
|
|
from core.constants import BLOCKLIST_JSONL, BLOCKLIST_MAX_DROP
|
|
from core.data_store import extract_appid, load_main, log_admin_removals, save_main
|
|
|
|
|
|
def main(argv=None) -> int:
|
|
ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
|
|
ap.add_argument("--max-drop", type=int, default=BLOCKLIST_MAX_DROP,
|
|
help=f"refuse to remove more games than this in one run (default {BLOCKLIST_MAX_DROP})")
|
|
args = ap.parse_args(argv)
|
|
|
|
blocked = load_blocklist()
|
|
if not blocked:
|
|
print(f"{BLOCKLIST_JSONL}: nothing blocked.")
|
|
return 0
|
|
|
|
games = load_main()
|
|
live = {extract_appid(g.get("link", "")) for g in games}
|
|
present = sorted(a for a in blocked if a in live)
|
|
|
|
if present:
|
|
# save_main() does the removal and logs each game it leaves out.
|
|
save_main(games, max_blocked_drop=args.max_drop)
|
|
print(f"Removed {len(present)} blocked game(s) from data/: {', '.join(present[:20])}"
|
|
+ (" …" if len(present) > 20 else ""))
|
|
else:
|
|
print("No blocked game is in data/.")
|
|
|
|
healed = log_admin_removals([removed_row(e) for a, e in blocked.items() if a not in live])
|
|
if healed:
|
|
print(f"Logged {healed} earlier removal(s) that were missing from scripts/removed_games.jsonl.")
|
|
print(f"Blocklist: {len(blocked)} game(s).")
|
|
return 0
|
|
|
|
|
|
if __name__ == "__main__":
|
|
sys.exit(main())
|