release(v3.5.0): charts expansion, game-count history, pipeline cleanup (#46)
- Add 6 charts (game-count line, genre treemap, KPI cards, deletions over time, deletion reasons, most-rated) and split charts.tsx into a charts/ module (one file per chart, shared chart-card + palette). - Track catalog size over time: data_store.py appends data_game/ count_history.json daily; backfill_count_history.py seeds it from git. - Add scripts/json_io.py with shared load/save and a URL-keyed dedup_deleted so the deleted-games log can't double-log re-added games. - Add a scroll-to-top button and a ruff config (pyproject.toml). - Refactor docs for v3.5.0 (CHANGELOG, README badges, CLAUDE.md). Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
67ad34c3fb
commit
ac8db6ab1d
49 files changed
+1255
-410
No files matched your search
@@ -2,6 +2,36 @@
|
||||
|
||||
All notable changes to this project will be documented here.
|
||||
|
||||
## [3.5.0] - 2026-05-20 (Charts expansion + game-count history + pipeline cleanup)
|
||||
|
||||
### Added
|
||||
|
||||
- **Six new charts on `/charts`.**
|
||||
- **Game count over time** — a `LineChart` of the catalog size by day, backed by a new `data_game/count_history.json` (date-keyed `{date, total}` series). [`scripts/data_store.py`](scripts/data_store.py) `_append_count_history()` upserts today's total on every catalog write (last-write-wins per UTC day). [`scripts/backfill_count_history.py`](scripts/backfill_count_history.py) is a one-time dev tool that reconstructs the series from git history (`data_game/index.json`, plus the pre-refactor `scripts/game_info.json`). The webapp loads it via `loadCountHistory` / `useCountHistory`.
|
||||
- **Genre treemap** — a Recharts `Treemap` with a custom cell renderer that hides labels on rectangles too small to fit them, so it stays legible on mobile.
|
||||
- **KPI summary cards** — total games / online / NSFW / deleted / average rating, above the Overview grid.
|
||||
- **Deletions over time** (bar, by month), **Deletion reasons** (pie: became-paid vs page-removed), and **Most rated games** (top 10 by `rating_count`).
|
||||
- **Scroll-to-top button** — a floating bottom-right button that appears after scrolling the main pane. New [`webapp/src/components/scroll-to-top.tsx`](webapp/src/components/scroll-to-top.tsx).
|
||||
- **Python linting** — `ruff` config in a new root `pyproject.toml`; run `ruff check scripts/`.
|
||||
|
||||
### Changed
|
||||
|
||||
- **`charts.tsx` split into a module.** The monolith is now a `webapp/src/components/charts/` folder — one file per chart, a shared `chart-card.tsx` (`ChartCard` + `PALETTE`), and an `index.ts` barrel. 16 charts across the 4 tabs.
|
||||
- **`scripts/json_io.py`** — new shared module. `load_json` / `save_json` (previously copy-pasted into both `check_alive.py` and `check_paid.py`) live here now, alongside `dedup_deleted`.
|
||||
- README + README.vi badges bumped to `3.5.0`.
|
||||
|
||||
### Fixed
|
||||
|
||||
- **The deleted-games log can no longer accumulate duplicate URLs.** `check_alive.py` and `check_paid.py` appended to `scripts/deleted_games.json` with no guard — a game removed, re-added by the scraper, then removed again would be logged twice. The new `dedup_deleted()` (keyed by URL, keeping the earliest `deleted_at`) runs on every write and inside `log_deleted.py`. Two genuinely different games that share a title but have different urls are kept separate.
|
||||
|
||||
### Notes
|
||||
|
||||
- No Tauri / Rust binary boundary change — `Cargo.toml` and `tauri.conf.json` stay at `0.1.1`, no `THIRD_PARTY` entries change (Recharts already shipped; `Treemap` / `LineChart` need no new dependency).
|
||||
- `data_game/count_history.json` is auto-committed by the scrape workflows — the bash wrappers already `git add data_game/` wholesale, so no workflow changes were needed.
|
||||
- The dead-code sweep found the project already tidy (TS strict mode + ESLint catch unused webapp code). `webapp/src/lib/tauri-scrape.ts` `tauriScrapePreview` was reviewed and intentionally kept as a Phase-8b stub.
|
||||
|
||||
---
|
||||
|
||||
## [3.4.1] - 2026-05-17 (Hotfix: DataTable column alignment + Discussion-announce auto-discovery)
|
||||
|
||||
### Fixed
|
||||
|
||||
@@ -7,8 +7,9 @@ Quick context so future Claude sessions don't have to re-derive it.
|
||||
Two layers stacked on the same JSON catalog of free itch.io games:
|
||||
|
||||
1. **Python pipeline** — the data layer.
|
||||
- `data_game/game_info_NNN.json` — chunked records, max 500 per file, ~514 games as of v3.1.1.
|
||||
- `scripts/` — `scraper.py` (shared session + parsing), `data_store.py` (chunk load/save/rebalance), `update_info.py`, `check_paid.py`, `check_alive.py`, `generate_md.py`, `log_deleted.py`.
|
||||
- `data_game/game_info_NNN.json` — chunked records, max 500 per file, ~1,470 games as of v3.5.0.
|
||||
- `data_game/index.json` — chunk manifest (total + per-file counts). `data_game/count_history.json` — date-keyed `{date,total}` series powering the "games over time" chart.
|
||||
- `scripts/` — `scraper.py` (shared session + parsing), `data_store.py` (chunk load/save/rebalance, writes `index.json` + `count_history.json`), `json_io.py` (shared `load_json`/`save_json` + `dedup_deleted`), `update_info.py`, `check_paid.py`, `check_alive.py`, `check_duplicate.py`, `update_reviews.py`, `update_status.py`, `force_update.py`, `generate_md.py`, `log_deleted.py`, `backfill_count_history.py` (one-time count-history seed). Lint: `ruff check scripts/` (config in root `pyproject.toml`).
|
||||
- `lists/{genre}.md` — auto-generated, **never edit by hand**.
|
||||
- `bash/` — wrappers used by the GitHub Actions workflows.
|
||||
2. **Webapp** — `webapp/` (React 19 + TypeScript 6 + Vite 8 + Tailwind v3 + shadcn/ui, Tauri 2 desktop wrapper in `webapp/src-tauri/`). Reads from `raw.githubusercontent.com`; writes via GitHub API with an encrypted PAT.
|
||||
@@ -36,7 +37,7 @@ Path alias: `@/*` → `webapp/src/*`. Vite `define`s `__BUILD_DATE__` so About p
|
||||
- `src/routes/*.tsx` — one file per route; `App.tsx` wires `react-router-dom` HashRouter.
|
||||
- `src/components/data-table/*` — `DataTable` (TanStack Table + Virtual + pagination), `data-table-toolbar`, `data-table-pagination`, `faceted-filter`, `columns`. Row keys are URL-based (`getRowId: g => g.url`) so selection survives sort/filter/page changes.
|
||||
- `src/components/ui/*` — shadcn primitives (Button, Card, Input, Label, Badge, Skeleton, Separator, Tabs, Popover, Checkbox, Select, Switch, Textarea, Dialog).
|
||||
- `src/components/charts.tsx` — 9 Recharts charts grouped into 4 tabs.
|
||||
- `src/components/charts/` — chart module (one file per chart, shared `chart-card.tsx` with `ChartCard` + `PALETTE`, `index.ts` barrel). 16 charts across 4 tabs.
|
||||
- `src/lib/github/` — `client.ts` (Octokit factory), `data-store.ts` (TS port of `scripts/data_store.py`), `contents.ts` (single-file commits — routed through Git Data API so they can be signed), `git-data.ts` (atomic multi-file commit; optional signer for GPG), `signer.ts` (builds a `CommitSigner` from gpg store + auth + prefs), `workflow.ts` (dispatch + poll).
|
||||
- `src/lib/gpg/` — `canonicalize.ts` (pure: builds the byte string Git would hash for a commit), `sign.ts` (lazy-imports `openpgp`, exports `loadPrivateKey` / `signCommit` / `verifyDetached`), `storage.ts` (encrypted-key + metadata in localStorage).
|
||||
- `src/lib/crypto.ts` — AES-GCM 256 + PBKDF2-SHA256 100k rounds. **Uint8Array generic must be `<ArrayBuffer>`** under TS 6, not `<ArrayBufferLike>` — Web Crypto rejects the latter.
|
||||
@@ -107,6 +108,6 @@ Fine-grained PAT scoped to **only this repo** with:
|
||||
- Don't commit a real PAT, ever. Settings page handles it client-side; CI uses `secrets.GH_TOKEN`.
|
||||
- When adding an npm dep, also add it to `webapp/src/lib/about.ts` so the About page lists it.
|
||||
|
||||
## Current state (as of v3.1.1)
|
||||
## Current state (as of v3.5.0)
|
||||
|
||||
`main` is the trunk. All feature work has merged. Tags `v3.0.0`, `v3.0.1`, `v3.1.0`, `v3.1.1` exist (all GPG-signed). The webapp is published at `https://poli0981.github.io/free-games-itchio-list/app/` once the user enables Pages → Source = GitHub Actions in repo Settings.
|
||||
`main` is the trunk. All feature work has merged. Tags `v3.0.0` through `v3.5.0` exist (all GPG-signed). The webapp is published at `https://poli0981.github.io/free-games-itchio-list/app/`.
|
||||
@@ -1,6 +1,6 @@
|
||||
# Free Itch.io Games List
|
||||
|
||||
[](https://github.com/poli0981/free-games-itchio-list/)
|
||||
[](https://github.com/poli0981/free-games-itchio-list/)
|
||||
[](https://github.com/poli0981/free-games-itchio-list/stargazers)
|
||||
[](https://github.com/poli0981/free-games-itchio-list/network/members)
|
||||
[](https://github.com/poli0981/free-games-itchio-list/commits/main)
|
||||
@@ -42,7 +42,7 @@ are added.
|
||||
|
||||
A React + TypeScript SPA in [`webapp/`](webapp/) provides a browsable UI on top of the same JSON
|
||||
catalog: virtualized DataTable for all 500+ games, faceted filters (genre / status / platforms /
|
||||
NSFW), 9 charts (Recharts), bulk edit/delete via the GitHub Git Data API, and a one-click "add"
|
||||
NSFW), 16 charts (Recharts), bulk edit/delete via the GitHub Git Data API, and a one-click "add"
|
||||
flow that dispatches the scraper workflow with a URL input.
|
||||
|
||||
- **Web build**: deployed to GitHub Pages by [`.github/workflows/deploy_webapp.yml`](.github/workflows/deploy_webapp.yml)
|
||||
|
||||
+2
-2
@@ -1,6 +1,6 @@
|
||||
# Danh sách Game Itch.io Miễn Phí
|
||||
|
||||
[](https://github.com/poli0981/free-games-itchio-list/)
|
||||
[](https://github.com/poli0981/free-games-itchio-list/)
|
||||
[](https://opensource.org/licenses/MIT)
|
||||
[](README.md)
|
||||
|
||||
@@ -35,7 +35,7 @@ Các bảng được tự tạo và chia theo thể loại chính (tối đa 300
|
||||
|
||||
## Webapp (duyệt + sửa + analytics)
|
||||
|
||||
Một SPA React + TypeScript trong [`webapp/`](webapp/) cung cấp giao diện duyệt trên cùng catalog JSON: DataTable virtualized cho 500+ game, faceted filter (thể loại / trạng thái / nền tảng / NSFW), 9 chart (Recharts), bulk edit/delete qua GitHub Git Data API, và một flow "add" một-click dispatch workflow scraper với input URL.
|
||||
Một SPA React + TypeScript trong [`webapp/`](webapp/) cung cấp giao diện duyệt trên cùng catalog JSON: DataTable virtualized cho 500+ game, faceted filter (thể loại / trạng thái / nền tảng / NSFW), 16 chart (Recharts), bulk edit/delete qua GitHub Git Data API, và một flow "add" một-click dispatch workflow scraper với input URL.
|
||||
|
||||
- **Bản web**: deploy lên GitHub Pages bởi [`.github/workflows/deploy_webapp.yml`](.github/workflows/deploy_webapp.yml) — push vào `main` chạm `webapp/` sẽ tự ship. (Setup một lần: repo Settings → Pages → Source = "GitHub Actions".)
|
||||
- **Bản desktop (tùy chọn)**: cùng code React đóng gói thành app native Tauri 2 cho Windows / macOS / Linux. Xem [`webapp/TAURI.md`](webapp/TAURI.md) để biết yêu cầu và `npm run tauri:dev`.
|
||||
|
||||
@@ -0,0 +1,202 @@
|
||||
[
|
||||
{
|
||||
"date": "2025-12-27",
|
||||
"total": 6
|
||||
},
|
||||
{
|
||||
"date": "2025-12-28",
|
||||
"total": 6
|
||||
},
|
||||
{
|
||||
"date": "2026-01-01",
|
||||
"total": 6
|
||||
},
|
||||
{
|
||||
"date": "2026-01-05",
|
||||
"total": 47
|
||||
},
|
||||
{
|
||||
"date": "2026-01-11",
|
||||
"total": 80
|
||||
},
|
||||
{
|
||||
"date": "2026-01-12",
|
||||
"total": 78
|
||||
},
|
||||
{
|
||||
"date": "2026-01-16",
|
||||
"total": 96
|
||||
},
|
||||
{
|
||||
"date": "2026-01-17",
|
||||
"total": 109
|
||||
},
|
||||
{
|
||||
"date": "2026-01-18",
|
||||
"total": 120
|
||||
},
|
||||
{
|
||||
"date": "2026-01-19",
|
||||
"total": 132
|
||||
},
|
||||
{
|
||||
"date": "2026-03-11",
|
||||
"total": 174
|
||||
},
|
||||
{
|
||||
"date": "2026-03-13",
|
||||
"total": 187
|
||||
},
|
||||
{
|
||||
"date": "2026-03-14",
|
||||
"total": 222
|
||||
},
|
||||
{
|
||||
"date": "2026-03-21",
|
||||
"total": 221
|
||||
},
|
||||
{
|
||||
"date": "2026-03-24",
|
||||
"total": 266
|
||||
},
|
||||
{
|
||||
"date": "2026-03-26",
|
||||
"total": 268
|
||||
},
|
||||
{
|
||||
"date": "2026-03-27",
|
||||
"total": 396
|
||||
},
|
||||
{
|
||||
"date": "2026-03-28",
|
||||
"total": 396
|
||||
},
|
||||
{
|
||||
"date": "2026-03-29",
|
||||
"total": 398
|
||||
},
|
||||
{
|
||||
"date": "2026-03-30",
|
||||
"total": 398
|
||||
},
|
||||
{
|
||||
"date": "2026-03-31",
|
||||
"total": 398
|
||||
},
|
||||
{
|
||||
"date": "2026-04-01",
|
||||
"total": 397
|
||||
},
|
||||
{
|
||||
"date": "2026-04-03",
|
||||
"total": 396
|
||||
},
|
||||
{
|
||||
"date": "2026-04-07",
|
||||
"total": 395
|
||||
},
|
||||
{
|
||||
"date": "2026-04-09",
|
||||
"total": 390
|
||||
},
|
||||
{
|
||||
"date": "2026-04-13",
|
||||
"total": 389
|
||||
},
|
||||
{
|
||||
"date": "2026-04-16",
|
||||
"total": 392
|
||||
},
|
||||
{
|
||||
"date": "2026-04-17",
|
||||
"total": 393
|
||||
},
|
||||
{
|
||||
"date": "2026-04-19",
|
||||
"total": 433
|
||||
},
|
||||
{
|
||||
"date": "2026-04-20",
|
||||
"total": 470
|
||||
},
|
||||
{
|
||||
"date": "2026-04-21",
|
||||
"total": 469
|
||||
},
|
||||
{
|
||||
"date": "2026-04-23",
|
||||
"total": 499
|
||||
},
|
||||
{
|
||||
"date": "2026-04-24",
|
||||
"total": 519
|
||||
},
|
||||
{
|
||||
"date": "2026-04-25",
|
||||
"total": 518
|
||||
},
|
||||
{
|
||||
"date": "2026-04-27",
|
||||
"total": 517
|
||||
},
|
||||
{
|
||||
"date": "2026-04-29",
|
||||
"total": 517
|
||||
},
|
||||
{
|
||||
"date": "2026-05-01",
|
||||
"total": 514
|
||||
},
|
||||
{
|
||||
"date": "2026-05-03",
|
||||
"total": 514
|
||||
},
|
||||
{
|
||||
"date": "2026-05-04",
|
||||
"total": 518
|
||||
},
|
||||
{
|
||||
"date": "2026-05-05",
|
||||
"total": 520
|
||||
},
|
||||
{
|
||||
"date": "2026-05-07",
|
||||
"total": 537
|
||||
},
|
||||
{
|
||||
"date": "2026-05-08",
|
||||
"total": 538
|
||||
},
|
||||
{
|
||||
"date": "2026-05-09",
|
||||
"total": 543
|
||||
},
|
||||
{
|
||||
"date": "2026-05-10",
|
||||
"total": 576
|
||||
},
|
||||
{
|
||||
"date": "2026-05-11",
|
||||
"total": 576
|
||||
},
|
||||
{
|
||||
"date": "2026-05-13",
|
||||
"total": 663
|
||||
},
|
||||
{
|
||||
"date": "2026-05-15",
|
||||
"total": 779
|
||||
},
|
||||
{
|
||||
"date": "2026-05-16",
|
||||
"total": 1076
|
||||
},
|
||||
{
|
||||
"date": "2026-05-17",
|
||||
"total": 1425
|
||||
},
|
||||
{
|
||||
"date": "2026-05-19",
|
||||
"total": 1467
|
||||
}
|
||||
]
|
||||
@@ -32,6 +32,7 @@ checked into the repo.
|
||||
```sh
|
||||
pip install -r scripts/requirements.txt # if/when present, else: beautifulsoup4 requests
|
||||
python scripts/update_info.py
|
||||
ruff check scripts/ # lint Python (pip install ruff; config in root pyproject.toml)
|
||||
```
|
||||
|
||||
The pipeline reads `scripts/temp_link.json` (a JSON array of itch.io URLs)
|
||||
|
||||
@@ -35,6 +35,7 @@ VS Code, Sublime, vim, v.v. đều dùng được; không có gì JetBrains-spec
|
||||
```sh
|
||||
pip install beautifulsoup4 requests
|
||||
python scripts/update_info.py
|
||||
ruff check scripts/ # lint Python (pip install ruff; cấu hình ở pyproject.toml gốc)
|
||||
```
|
||||
|
||||
Pipeline đọc `scripts/temp_link.json` (mảng JSON các URL itch.io)
|
||||
|
||||
@@ -0,0 +1,12 @@
|
||||
# Project tooling configuration.
|
||||
#
|
||||
# Ruff is the only tool configured here: the repo has no Python build
|
||||
# system — the scraper pipeline runs the scripts/ modules directly.
|
||||
# Run it with: ruff check scripts/
|
||||
|
||||
[tool.ruff]
|
||||
line-length = 100
|
||||
target-version = "py310"
|
||||
|
||||
[tool.ruff.lint]
|
||||
select = ["E", "F", "I", "W", "UP", "B"]
|
||||
@@ -0,0 +1,125 @@
|
||||
"""
|
||||
backfill_count_history.py — One-time seed for data_game/count_history.json.
|
||||
|
||||
Walks the git history of the game catalogue and reconstructs the total
|
||||
game count per day, so the "games over time" chart has data from day one.
|
||||
|
||||
Two eras are covered:
|
||||
- scripts/game_info.json — the pre-refactor flat array (its length).
|
||||
- data_game/index.json — the current multi-file layout (total_games,
|
||||
or the sum of files[].count for the oldest shape).
|
||||
|
||||
For each calendar day the latest commit wins. This is a developer tool:
|
||||
run it once locally before tagging a release, then commit the result.
|
||||
Any unreadable commit is skipped with a warning — it never aborts.
|
||||
|
||||
Usage (from the repo root):
|
||||
python scripts/backfill_count_history.py
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
sys.path.insert(0, os.path.dirname(__file__))
|
||||
from json_io import save_json
|
||||
|
||||
OUTPUT_FILE = os.path.join("data_game", "count_history.json")
|
||||
|
||||
# (path, kind) pairs, oldest era first.
|
||||
SOURCES = [
|
||||
("scripts/game_info.json", "array"),
|
||||
("data_game/index.json", "index"),
|
||||
]
|
||||
|
||||
|
||||
def run_git(args: list[str]) -> str:
|
||||
"""Run a git command from the repo root and return stdout."""
|
||||
result = subprocess.run(
|
||||
["git", *args],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
encoding="utf-8",
|
||||
check=True,
|
||||
)
|
||||
return result.stdout
|
||||
|
||||
|
||||
def git_log_commits(path: str):
|
||||
"""Yield (sha, iso_datetime) for every commit that touched `path`."""
|
||||
try:
|
||||
out = run_git(["log", "--format=%H%x09%cI", "--", path])
|
||||
except subprocess.CalledProcessError as exc:
|
||||
print(f" ! git log failed for {path}: {exc}")
|
||||
return
|
||||
for line in out.splitlines():
|
||||
line = line.strip()
|
||||
if "\t" not in line:
|
||||
continue
|
||||
sha, iso = line.split("\t", 1)
|
||||
yield sha, iso
|
||||
|
||||
|
||||
def extract_total(data, kind: str) -> int:
|
||||
"""Compute the game count from a parsed JSON blob of the given kind."""
|
||||
if kind == "array":
|
||||
if not isinstance(data, list):
|
||||
raise ValueError("expected a JSON array")
|
||||
return len(data)
|
||||
# kind == "index"
|
||||
if "total_games" in data:
|
||||
return int(data["total_games"])
|
||||
files = data.get("files")
|
||||
if files:
|
||||
return sum(int(f["count"]) for f in files)
|
||||
raise ValueError("index has no total_games and no files")
|
||||
|
||||
|
||||
def commit_total(sha: str, path: str, kind: str) -> int | None:
|
||||
"""Return the game count at `sha` for `path`, or None if unreadable."""
|
||||
try:
|
||||
blob = run_git(["show", f"{sha}:{path}"])
|
||||
return extract_total(json.loads(blob), kind)
|
||||
except (
|
||||
subprocess.CalledProcessError,
|
||||
json.JSONDecodeError,
|
||||
KeyError,
|
||||
TypeError,
|
||||
ValueError,
|
||||
) as exc:
|
||||
print(f" ! skip {sha[:9]} {path}: {exc}")
|
||||
return None
|
||||
|
||||
|
||||
def main() -> None:
|
||||
records: list[tuple[str, str, int]] = [] # (iso, date, total)
|
||||
for path, kind in SOURCES:
|
||||
commits = list(git_log_commits(path))
|
||||
print(f"{path}: {len(commits)} commit(s)")
|
||||
for sha, iso in commits:
|
||||
total = commit_total(sha, path, kind)
|
||||
if total is not None:
|
||||
records.append((iso, iso[:10], total))
|
||||
|
||||
if not records:
|
||||
print("No history found — nothing written.")
|
||||
return
|
||||
|
||||
# Latest commit of each calendar day wins.
|
||||
records.sort(key=lambda r: r[0])
|
||||
by_date: dict[str, int] = {}
|
||||
for _iso, date, total in records:
|
||||
by_date[date] = total
|
||||
|
||||
history = [{"date": d, "total": by_date[d]} for d in sorted(by_date)]
|
||||
save_json(OUTPUT_FILE, history)
|
||||
print(
|
||||
f"Wrote {len(history)} day(s) -> {OUTPUT_FILE} "
|
||||
f"({history[0]['date']}: {history[0]['total']} ... "
|
||||
f"{history[-1]['date']}: {history[-1]['total']})"
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
+12
-26
@@ -12,20 +12,20 @@ Flow:
|
||||
Uses lighter delays than full scrape since we don't parse HTML.
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
|
||||
sys.path.insert(0, os.path.dirname(__file__))
|
||||
from scraper import (
|
||||
create_session,
|
||||
check_url_alive,
|
||||
light_delay,
|
||||
batch_pause,
|
||||
should_batch_pause,
|
||||
now_iso,
|
||||
)
|
||||
from data_store import load_all_games, save_all_games
|
||||
from json_io import dedup_deleted, load_json, save_json
|
||||
from scraper import (
|
||||
batch_pause,
|
||||
check_url_alive,
|
||||
create_session,
|
||||
light_delay,
|
||||
now_iso,
|
||||
should_batch_pause,
|
||||
)
|
||||
|
||||
DELETED_LOG = "scripts/deleted_games.json"
|
||||
|
||||
@@ -33,20 +33,6 @@ DELETED_LOG = "scripts/deleted_games.json"
|
||||
DEAD_CODES = {404, 410} # Not Found, Gone
|
||||
|
||||
|
||||
def load_json(path: str, default=None):
|
||||
if default is None:
|
||||
default = []
|
||||
if os.path.exists(path):
|
||||
with open(path, "r", encoding="utf-8") as f:
|
||||
return json.load(f)
|
||||
return default
|
||||
|
||||
|
||||
def save_json(path: str, data) -> None:
|
||||
with open(path, "w", encoding="utf-8") as f:
|
||||
json.dump(data, f, ensure_ascii=False, indent=4)
|
||||
|
||||
|
||||
def main() -> None:
|
||||
games: list[dict] = load_all_games()
|
||||
if not games:
|
||||
@@ -70,7 +56,7 @@ def main() -> None:
|
||||
|
||||
if status is None:
|
||||
# Connection failed entirely — keep game
|
||||
print(f" → Connection error, keeping game.")
|
||||
print(" → Connection error, keeping game.")
|
||||
keep.append(game)
|
||||
errors += 1
|
||||
elif status in DEAD_CODES:
|
||||
@@ -96,7 +82,7 @@ def main() -> None:
|
||||
|
||||
# Save
|
||||
save_all_games(keep)
|
||||
save_json(DELETED_LOG, deleted_log)
|
||||
save_json(DELETED_LOG, dedup_deleted(deleted_log))
|
||||
|
||||
print(
|
||||
f"\nDone — {removed} removed (dead), {errors} errors, "
|
||||
@@ -105,4 +91,4 @@ def main() -> None:
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
main()
|
||||
+13
-27
@@ -9,38 +9,24 @@ Flow:
|
||||
5. Save updated games (auto-rebalanced chunks)
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
|
||||
sys.path.insert(0, os.path.dirname(__file__))
|
||||
from scraper import (
|
||||
create_session,
|
||||
check_still_free,
|
||||
polite_delay,
|
||||
batch_pause,
|
||||
should_batch_pause,
|
||||
now_iso,
|
||||
)
|
||||
from data_store import load_all_games, save_all_games
|
||||
from json_io import dedup_deleted, load_json, save_json
|
||||
from scraper import (
|
||||
batch_pause,
|
||||
check_still_free,
|
||||
create_session,
|
||||
now_iso,
|
||||
polite_delay,
|
||||
should_batch_pause,
|
||||
)
|
||||
|
||||
DELETED_LOG = "scripts/deleted_games.json"
|
||||
|
||||
|
||||
def load_json(path: str, default=None):
|
||||
if default is None:
|
||||
default = []
|
||||
if os.path.exists(path):
|
||||
with open(path, "r", encoding="utf-8") as f:
|
||||
return json.load(f)
|
||||
return default
|
||||
|
||||
|
||||
def save_json(path: str, data) -> None:
|
||||
with open(path, "w", encoding="utf-8") as f:
|
||||
json.dump(data, f, ensure_ascii=False, indent=4)
|
||||
|
||||
|
||||
def main() -> None:
|
||||
games: list[dict] = load_all_games()
|
||||
if not games:
|
||||
@@ -64,7 +50,7 @@ def main() -> None:
|
||||
|
||||
if result is None:
|
||||
# Network error — don't delete, keep game
|
||||
print(f" → Network error, keeping game.")
|
||||
print(" → Network error, keeping game.")
|
||||
keep.append(game)
|
||||
errors += 1
|
||||
elif result is True:
|
||||
@@ -72,7 +58,7 @@ def main() -> None:
|
||||
keep.append(game)
|
||||
else:
|
||||
# Became paid → remove
|
||||
print(f" ✘ Game became paid — removing.")
|
||||
print(" ✘ Game became paid — removing.")
|
||||
deleted_log.append({
|
||||
"url": url,
|
||||
"name": name,
|
||||
@@ -89,7 +75,7 @@ def main() -> None:
|
||||
|
||||
# Save
|
||||
save_all_games(keep)
|
||||
save_json(DELETED_LOG, deleted_log)
|
||||
save_json(DELETED_LOG, dedup_deleted(deleted_log))
|
||||
|
||||
print(
|
||||
f"\nDone — {removed} removed (paid), {errors} network errors, "
|
||||
@@ -98,4 +84,4 @@ def main() -> None:
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
main()
|
||||
+32
-2
@@ -20,6 +20,7 @@ DATA_DIR = "data_game"
|
||||
FILE_PREFIX = "game_info_"
|
||||
FILE_SUFFIX = ".json"
|
||||
INDEX_FILE = os.path.join(DATA_DIR, "index.json")
|
||||
COUNT_HISTORY_FILE = os.path.join(DATA_DIR, "count_history.json")
|
||||
CHUNK_SIZE = 500
|
||||
|
||||
|
||||
@@ -38,15 +39,16 @@ def load_all_games() -> list[dict]:
|
||||
"""Load every chunk file and return a single flat list."""
|
||||
games: list[dict] = []
|
||||
for path in _list_chunk_files():
|
||||
with open(path, "r", encoding="utf-8") as f:
|
||||
with open(path, encoding="utf-8") as f:
|
||||
games.extend(json.load(f))
|
||||
return games
|
||||
|
||||
|
||||
def _write_index(chunks: list[tuple[str, int]]) -> None:
|
||||
"""Write data_game/index.json with metadata about all chunk files."""
|
||||
total = sum(count for _, count in chunks)
|
||||
index = {
|
||||
"total_games": sum(count for _, count in chunks),
|
||||
"total_games": total,
|
||||
"max_per_file": CHUNK_SIZE,
|
||||
"last_updated": datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ"),
|
||||
"files": [
|
||||
@@ -56,6 +58,34 @@ def _write_index(chunks: list[tuple[str, int]]) -> None:
|
||||
}
|
||||
with open(INDEX_FILE, "w", encoding="utf-8") as f:
|
||||
json.dump(index, f, ensure_ascii=False, indent=4)
|
||||
_append_count_history(total)
|
||||
|
||||
|
||||
def _append_count_history(total: int) -> None:
|
||||
"""Record today's total game count in data_game/count_history.json.
|
||||
|
||||
Date-keyed upsert: each UTC day holds one row. Multiple pipeline runs
|
||||
on the same day overwrite that day's count (last write wins), so the
|
||||
series stays one point per day. A corrupt or unreadable file falls
|
||||
back to an empty history rather than aborting the save.
|
||||
"""
|
||||
today = datetime.now(timezone.utc).strftime("%Y-%m-%d")
|
||||
history: list[dict] = []
|
||||
if os.path.exists(COUNT_HISTORY_FILE):
|
||||
try:
|
||||
with open(COUNT_HISTORY_FILE, encoding="utf-8") as f:
|
||||
history = json.load(f)
|
||||
except (json.JSONDecodeError, OSError):
|
||||
history = []
|
||||
for row in history:
|
||||
if row.get("date") == today:
|
||||
row["total"] = total
|
||||
break
|
||||
else:
|
||||
history.append({"date": today, "total": total})
|
||||
history.sort(key=lambda r: r.get("date", ""))
|
||||
with open(COUNT_HISTORY_FILE, "w", encoding="utf-8") as f:
|
||||
json.dump(history, f, ensure_ascii=False, indent=4)
|
||||
|
||||
|
||||
def save_all_games(games: list[dict]) -> None:
|
||||
|
||||
@@ -28,14 +28,14 @@ import os
|
||||
import sys
|
||||
|
||||
sys.path.insert(0, os.path.dirname(__file__))
|
||||
from data_store import load_all_games, save_all_games
|
||||
from scraper import (
|
||||
create_session,
|
||||
scrape_game_info,
|
||||
polite_delay,
|
||||
batch_pause,
|
||||
create_session,
|
||||
polite_delay,
|
||||
scrape_game_info,
|
||||
should_batch_pause,
|
||||
)
|
||||
from data_store import load_all_games, save_all_games
|
||||
|
||||
# Fields the scraper does NOT own — never overwrite, regardless of scrape result.
|
||||
PRESERVE_FIELDS = ("safe_virus", "notes", "nsfw")
|
||||
|
||||
@@ -0,0 +1,46 @@
|
||||
"""
|
||||
json_io.py — Shared JSON helpers for the scraper pipeline.
|
||||
|
||||
Small utilities used by several scripts:
|
||||
- load_json / save_json: read/write JSON files (UTF-8, pretty-printed).
|
||||
- dedup_deleted: de-duplicate the deleted-games log by URL.
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
|
||||
|
||||
def load_json(path: str, default=None):
|
||||
"""Load a JSON file, returning `default` (default: []) when it doesn't exist."""
|
||||
if default is None:
|
||||
default = []
|
||||
if os.path.exists(path):
|
||||
with open(path, encoding="utf-8") as f:
|
||||
return json.load(f)
|
||||
return default
|
||||
|
||||
|
||||
def save_json(path: str, data) -> None:
|
||||
"""Write `data` to `path` as pretty-printed UTF-8 JSON."""
|
||||
with open(path, "w", encoding="utf-8") as f:
|
||||
json.dump(data, f, ensure_ascii=False, indent=4)
|
||||
|
||||
|
||||
def dedup_deleted(entries: list[dict]) -> list[dict]:
|
||||
"""De-duplicate the deleted-games log by URL.
|
||||
|
||||
Keeps the earliest `deleted_at` per URL, so a game that is removed,
|
||||
re-added by the scraper, then removed again is logged only once.
|
||||
Entries sharing a name but with different urls are kept separate
|
||||
(they are genuinely different games). Entries without a url are dropped.
|
||||
The result is sorted ascending by `deleted_at`.
|
||||
"""
|
||||
by_url: dict[str, dict] = {}
|
||||
for entry in entries:
|
||||
url = entry.get("url", "")
|
||||
if not url:
|
||||
continue
|
||||
current = by_url.get(url)
|
||||
if current is None or entry.get("deleted_at", "") < current.get("deleted_at", ""):
|
||||
by_url[url] = entry
|
||||
return sorted(by_url.values(), key=lambda e: e.get("deleted_at", ""))
|
||||
@@ -6,6 +6,10 @@ Produces a human-readable log file of all games removed from the list.
|
||||
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
|
||||
sys.path.insert(0, os.path.dirname(__file__))
|
||||
from json_io import dedup_deleted
|
||||
|
||||
DELETED_JSON = "scripts/deleted_games.json"
|
||||
DELETED_TXT = "deleted_games.txt"
|
||||
@@ -16,9 +20,11 @@ def main() -> None:
|
||||
print("No deleted_games.json found — nothing to export.")
|
||||
return
|
||||
|
||||
with open(DELETED_JSON, "r", encoding="utf-8") as f:
|
||||
with open(DELETED_JSON, encoding="utf-8") as f:
|
||||
entries: list[dict] = json.load(f)
|
||||
|
||||
entries = dedup_deleted(entries)
|
||||
|
||||
if not entries:
|
||||
print("deleted_games.json is empty — nothing to export.")
|
||||
# Still write an empty-state file
|
||||
@@ -48,8 +54,8 @@ def main() -> None:
|
||||
f.write(f" Reason: {reason}\n")
|
||||
f.write("\n")
|
||||
|
||||
print(f"Exported {len(entries)} deleted game(s) → {DELETED_TXT}")
|
||||
print(f"Exported {len(entries)} deleted game(s) -> {DELETED_TXT}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
main()
|
||||
+2
-2
@@ -13,10 +13,10 @@ Provides:
|
||||
import random
|
||||
import time
|
||||
from datetime import datetime, timezone
|
||||
from html import unescape
|
||||
|
||||
import requests
|
||||
from bs4 import BeautifulSoup
|
||||
from html import unescape
|
||||
from requests.adapters import HTTPAdapter
|
||||
from urllib3.util.retry import Retry
|
||||
|
||||
@@ -402,4 +402,4 @@ def check_still_free(session: requests.Session, url: str) -> bool | None:
|
||||
return is_free_game(soup)
|
||||
except requests.RequestException as e:
|
||||
print(f" [NET ERROR] {url}: {e}")
|
||||
return None
|
||||
return None
|
||||
@@ -14,14 +14,14 @@ import os
|
||||
import sys
|
||||
|
||||
sys.path.insert(0, os.path.dirname(__file__))
|
||||
from data_store import get_all_urls, load_all_games, save_all_games
|
||||
from scraper import (
|
||||
create_session,
|
||||
scrape_game_info,
|
||||
polite_delay,
|
||||
batch_pause,
|
||||
create_session,
|
||||
polite_delay,
|
||||
scrape_game_info,
|
||||
should_batch_pause,
|
||||
)
|
||||
from data_store import load_all_games, save_all_games, get_all_urls
|
||||
|
||||
TEMP_LINK = "scripts/temp_link.json"
|
||||
|
||||
@@ -30,7 +30,7 @@ def main() -> None:
|
||||
new_links: list[str] = []
|
||||
|
||||
if os.path.exists(TEMP_LINK):
|
||||
with open(TEMP_LINK, "r", encoding="utf-8") as f:
|
||||
with open(TEMP_LINK, encoding="utf-8") as f:
|
||||
new_links = json.load(f)
|
||||
|
||||
# Append a single URL passed via workflow_dispatch input (webapp uses this).
|
||||
@@ -72,7 +72,7 @@ def main() -> None:
|
||||
continue
|
||||
|
||||
if not info.get("is_free", True):
|
||||
print(f" → Paid game, skipping.")
|
||||
print(" → Paid game, skipping.")
|
||||
skipped_paid += 1
|
||||
polite_delay()
|
||||
continue
|
||||
|
||||
@@ -16,14 +16,14 @@ import os
|
||||
import sys
|
||||
|
||||
sys.path.insert(0, os.path.dirname(__file__))
|
||||
from data_store import load_all_games, save_all_games
|
||||
from scraper import (
|
||||
create_session,
|
||||
scrape_game_info,
|
||||
polite_delay,
|
||||
batch_pause,
|
||||
create_session,
|
||||
polite_delay,
|
||||
scrape_game_info,
|
||||
should_batch_pause,
|
||||
)
|
||||
from data_store import load_all_games, save_all_games
|
||||
|
||||
|
||||
def main() -> None:
|
||||
|
||||
@@ -15,14 +15,14 @@ import os
|
||||
import sys
|
||||
|
||||
sys.path.insert(0, os.path.dirname(__file__))
|
||||
from data_store import load_all_games, save_all_games
|
||||
from scraper import (
|
||||
create_session,
|
||||
scrape_game_info,
|
||||
polite_delay,
|
||||
batch_pause,
|
||||
create_session,
|
||||
polite_delay,
|
||||
scrape_game_info,
|
||||
should_batch_pause,
|
||||
)
|
||||
from data_store import load_all_games, save_all_games
|
||||
|
||||
|
||||
def main() -> None:
|
||||
|
||||
+1
-1
@@ -31,7 +31,7 @@ show the build date.
|
||||
| `/games` | [src/routes/games.tsx](src/routes/games.tsx) | Virtualized DataTable + faceted filters; bulk edit / delete |
|
||||
| `/games/:slug` | [src/routes/game-detail.tsx](src/routes/game-detail.tsx) | 23-field read view + edit form for `safe_virus` / `notes` / `nsfw` |
|
||||
| `/add` | [src/routes/add.tsx](src/routes/add.tsx) | Single URL or bulk paste → dispatch scraper workflow |
|
||||
| `/charts` | [src/routes/charts.tsx](src/routes/charts.tsx) | 9 Recharts visualizations across 4 tabs |
|
||||
| `/charts` | [src/routes/charts.tsx](src/routes/charts.tsx) | 16 charts + KPI cards across 4 tabs |
|
||||
| `/workflows` | [src/routes/workflows.tsx](src/routes/workflows.tsx) | Workflow run history + manual dispatch |
|
||||
| `/deleted` | [src/routes/deleted.tsx](src/routes/deleted.tsx) | Tombstone log of removed games |
|
||||
| `/settings` | [src/routes/settings.tsx](src/routes/settings.tsx) | PAT lock/unlock, theme, sidebar prefs |
|
||||
|
||||
+5
-2
@@ -1,6 +1,7 @@
|
||||
import { lazy, Suspense, useEffect } from 'react'
|
||||
import { lazy, Suspense, useEffect, useRef } from 'react'
|
||||
import { Routes, Route, useLocation } from 'react-router-dom'
|
||||
import { Sidebar, MobileTopBar } from '@/components/sidebar'
|
||||
import ScrollToTop from '@/components/scroll-to-top'
|
||||
import { Skeleton } from '@/components/ui/skeleton'
|
||||
import { useThemeEffect } from '@/hooks/useThemeEffect'
|
||||
import { useDensityEffect } from '@/hooks/useDensityEffect'
|
||||
@@ -45,13 +46,14 @@ function ScrollToHash() {
|
||||
export default function App() {
|
||||
useThemeEffect()
|
||||
useDensityEffect()
|
||||
const mainRef = useRef<HTMLElement>(null)
|
||||
return (
|
||||
<div className="flex h-[100dvh] overflow-hidden">
|
||||
<ScrollToHash />
|
||||
<Sidebar />
|
||||
<div className="flex min-w-0 flex-1 flex-col">
|
||||
<MobileTopBar />
|
||||
<main className="flex-1 overflow-y-auto">
|
||||
<main ref={mainRef} className="flex-1 overflow-y-auto">
|
||||
<Suspense fallback={<RouteFallback />}>
|
||||
<Routes>
|
||||
<Route path="/" element={<Dashboard />} />
|
||||
@@ -67,6 +69,7 @@ export default function App() {
|
||||
</Routes>
|
||||
</Suspense>
|
||||
</main>
|
||||
<ScrollToTop targetRef={mainRef} />
|
||||
</div>
|
||||
</div>
|
||||
)
|
||||
|
||||
@@ -1,310 +0,0 @@
|
||||
import { useMemo } from 'react'
|
||||
import {
|
||||
Bar,
|
||||
BarChart,
|
||||
CartesianGrid,
|
||||
Cell,
|
||||
Legend,
|
||||
Pie,
|
||||
PieChart,
|
||||
ResponsiveContainer,
|
||||
Tooltip,
|
||||
XAxis,
|
||||
YAxis,
|
||||
} from 'recharts'
|
||||
import { Card, CardContent, CardHeader, CardTitle } from '@/components/ui/card'
|
||||
import { Badge } from '@/components/ui/badge'
|
||||
import {
|
||||
classifyOnline,
|
||||
countBy,
|
||||
countByArray,
|
||||
ratingHistogram,
|
||||
topMadeWith,
|
||||
topN,
|
||||
type CountEntry,
|
||||
} from '@/lib/analytics'
|
||||
import type { DeletedGameEntry, Game } from '@/types/game'
|
||||
import { formatNumber } from '@/lib/utils'
|
||||
|
||||
const PALETTE = [
|
||||
'hsl(217 91% 60%)',
|
||||
'hsl(330 81% 60%)',
|
||||
'hsl(142 71% 45%)',
|
||||
'hsl(38 92% 50%)',
|
||||
'hsl(262 83% 58%)',
|
||||
'hsl(199 89% 48%)',
|
||||
'hsl(20 90% 55%)',
|
||||
'hsl(173 58% 39%)',
|
||||
'hsl(280 65% 60%)',
|
||||
'hsl(160 60% 45%)',
|
||||
]
|
||||
|
||||
interface ChartCardProps {
|
||||
title: string
|
||||
description?: string
|
||||
children: React.ReactNode
|
||||
}
|
||||
|
||||
function ChartCard({ title, description, children }: ChartCardProps) {
|
||||
return (
|
||||
<Card>
|
||||
<CardHeader>
|
||||
<CardTitle className="text-base">{title}</CardTitle>
|
||||
{description && (
|
||||
<p className="text-xs text-muted-foreground">{description}</p>
|
||||
)}
|
||||
</CardHeader>
|
||||
<CardContent className="h-72">{children}</CardContent>
|
||||
</Card>
|
||||
)
|
||||
}
|
||||
|
||||
export function GenreChart({ games }: { games: Game[] }) {
|
||||
const data = useMemo(() => topN(countBy(games, 'genre'), 15), [games])
|
||||
return (
|
||||
<ChartCard title="Genres" description="Top 15 by game count">
|
||||
<ResponsiveContainer width="100%" height="100%">
|
||||
<BarChart data={data} layout="vertical" margin={{ left: 20 }}>
|
||||
<CartesianGrid horizontal={false} strokeDasharray="3 3" />
|
||||
<XAxis type="number" />
|
||||
<YAxis dataKey="key" type="category" width={120} tick={{ fontSize: 12 }} />
|
||||
<Tooltip
|
||||
cursor={{ fill: 'hsl(var(--accent))' }}
|
||||
contentStyle={{ background: 'hsl(var(--popover))', border: '1px solid hsl(var(--border))' }}
|
||||
/>
|
||||
<Bar dataKey="count" fill={PALETTE[0]} radius={[0, 4, 4, 0]} />
|
||||
</BarChart>
|
||||
</ResponsiveContainer>
|
||||
</ChartCard>
|
||||
)
|
||||
}
|
||||
|
||||
export function PlatformChart({ games }: { games: Game[] }) {
|
||||
const data = useMemo(() => countByArray(games, 'platforms'), [games])
|
||||
return (
|
||||
<ChartCard title="Platforms" description="Counted per game (a game may target several)">
|
||||
<ResponsiveContainer width="100%" height="100%">
|
||||
<PieChart>
|
||||
<Pie
|
||||
data={data}
|
||||
dataKey="count"
|
||||
nameKey="key"
|
||||
innerRadius="50%"
|
||||
outerRadius="80%"
|
||||
paddingAngle={2}
|
||||
>
|
||||
{data.map((_, i) => (
|
||||
<Cell key={i} fill={PALETTE[i % PALETTE.length]} />
|
||||
))}
|
||||
</Pie>
|
||||
<Tooltip
|
||||
contentStyle={{
|
||||
background: 'hsl(var(--popover))',
|
||||
border: '1px solid hsl(var(--border))',
|
||||
}}
|
||||
/>
|
||||
<Legend />
|
||||
</PieChart>
|
||||
</ResponsiveContainer>
|
||||
</ChartCard>
|
||||
)
|
||||
}
|
||||
|
||||
export function OnlineOfflineChart({ games }: { games: Game[] }) {
|
||||
const split = useMemo(() => classifyOnline(games), [games])
|
||||
const data = [
|
||||
{ key: 'Online (HTML5)', count: split.online },
|
||||
{ key: 'Offline only', count: split.offline },
|
||||
]
|
||||
return (
|
||||
<ChartCard title="Online vs Offline" description="HTML5 means playable in the browser">
|
||||
<ResponsiveContainer width="100%" height="100%">
|
||||
<PieChart>
|
||||
<Pie data={data} dataKey="count" nameKey="key" innerRadius="50%" outerRadius="80%">
|
||||
<Cell fill={PALETTE[2]} />
|
||||
<Cell fill={PALETTE[3]} />
|
||||
</Pie>
|
||||
<Tooltip
|
||||
contentStyle={{
|
||||
background: 'hsl(var(--popover))',
|
||||
border: '1px solid hsl(var(--border))',
|
||||
}}
|
||||
/>
|
||||
<Legend />
|
||||
</PieChart>
|
||||
</ResponsiveContainer>
|
||||
</ChartCard>
|
||||
)
|
||||
}
|
||||
|
||||
export function LanguagesChart({ games }: { games: Game[] }) {
|
||||
const data = useMemo(() => topN(countByArray(games, 'languages'), 10), [games])
|
||||
return (
|
||||
<ChartCard title="Top 10 Languages">
|
||||
<ResponsiveContainer width="100%" height="100%">
|
||||
<BarChart data={data}>
|
||||
<CartesianGrid strokeDasharray="3 3" />
|
||||
<XAxis dataKey="key" tick={{ fontSize: 11 }} interval={0} angle={-25} textAnchor="end" height={60} />
|
||||
<YAxis />
|
||||
<Tooltip
|
||||
cursor={{ fill: 'hsl(var(--accent))' }}
|
||||
contentStyle={{ background: 'hsl(var(--popover))', border: '1px solid hsl(var(--border))' }}
|
||||
/>
|
||||
<Bar dataKey="count" fill={PALETTE[4]} radius={[4, 4, 0, 0]} />
|
||||
</BarChart>
|
||||
</ResponsiveContainer>
|
||||
</ChartCard>
|
||||
)
|
||||
}
|
||||
|
||||
export function MadeWithChart({ games }: { games: Game[] }) {
|
||||
const data = useMemo(() => topMadeWith(games, 8), [games])
|
||||
return (
|
||||
<ChartCard title="Game Engines" description="Primary engine per game (made_with[0])">
|
||||
<ResponsiveContainer width="100%" height="100%">
|
||||
<PieChart>
|
||||
<Pie data={data} dataKey="count" nameKey="key" innerRadius="50%" outerRadius="80%" paddingAngle={2}>
|
||||
{data.map((_, i) => (
|
||||
<Cell key={i} fill={PALETTE[i % PALETTE.length]} />
|
||||
))}
|
||||
</Pie>
|
||||
<Tooltip
|
||||
contentStyle={{
|
||||
background: 'hsl(var(--popover))',
|
||||
border: '1px solid hsl(var(--border))',
|
||||
}}
|
||||
/>
|
||||
<Legend />
|
||||
</PieChart>
|
||||
</ResponsiveContainer>
|
||||
</ChartCard>
|
||||
)
|
||||
}
|
||||
|
||||
export function RatingChart({ games }: { games: Game[] }) {
|
||||
const data = useMemo(() => ratingHistogram(games, 10), [games])
|
||||
return (
|
||||
<ChartCard title="Rating Histogram" description="Bins of 0.5 stars from 0 to 5">
|
||||
<ResponsiveContainer width="100%" height="100%">
|
||||
<BarChart data={data}>
|
||||
<CartesianGrid strokeDasharray="3 3" />
|
||||
<XAxis dataKey="key" tick={{ fontSize: 10 }} />
|
||||
<YAxis />
|
||||
<Tooltip
|
||||
cursor={{ fill: 'hsl(var(--accent))' }}
|
||||
contentStyle={{ background: 'hsl(var(--popover))', border: '1px solid hsl(var(--border))' }}
|
||||
/>
|
||||
<Bar dataKey="count" fill={PALETTE[5]} radius={[4, 4, 0, 0]} />
|
||||
</BarChart>
|
||||
</ResponsiveContainer>
|
||||
</ChartCard>
|
||||
)
|
||||
}
|
||||
|
||||
export function StatusChart({ games }: { games: Game[] }) {
|
||||
const data = useMemo(() => countBy(games, 'status'), [games])
|
||||
return (
|
||||
<ChartCard title="Status Mix" description="Released vs prototype/in-progress">
|
||||
<ResponsiveContainer width="100%" height="100%">
|
||||
<PieChart>
|
||||
<Pie data={data} dataKey="count" nameKey="key" outerRadius="80%">
|
||||
{data.map((_, i) => (
|
||||
<Cell key={i} fill={PALETTE[i % PALETTE.length]} />
|
||||
))}
|
||||
</Pie>
|
||||
<Tooltip
|
||||
contentStyle={{
|
||||
background: 'hsl(var(--popover))',
|
||||
border: '1px solid hsl(var(--border))',
|
||||
}}
|
||||
/>
|
||||
<Legend />
|
||||
</PieChart>
|
||||
</ResponsiveContainer>
|
||||
</ChartCard>
|
||||
)
|
||||
}
|
||||
|
||||
export function AvgSessionChart({ games }: { games: Game[] }) {
|
||||
const data = useMemo(() => topN(countBy(games, 'average_session'), 8), [games])
|
||||
return (
|
||||
<ChartCard title="Average Session Length">
|
||||
<ResponsiveContainer width="100%" height="100%">
|
||||
<BarChart data={data}>
|
||||
<CartesianGrid strokeDasharray="3 3" />
|
||||
<XAxis dataKey="key" tick={{ fontSize: 10 }} interval={0} angle={-20} textAnchor="end" height={60} />
|
||||
<YAxis />
|
||||
<Tooltip
|
||||
cursor={{ fill: 'hsl(var(--accent))' }}
|
||||
contentStyle={{ background: 'hsl(var(--popover))', border: '1px solid hsl(var(--border))' }}
|
||||
/>
|
||||
<Bar dataKey="count" fill={PALETTE[6]} radius={[4, 4, 0, 0]} />
|
||||
</BarChart>
|
||||
</ResponsiveContainer>
|
||||
</ChartCard>
|
||||
)
|
||||
}
|
||||
|
||||
export function TopTagsChart({ games }: { games: Game[] }) {
|
||||
const data = useMemo(() => topN(countByArray(games, 'tags'), 30), [games])
|
||||
return (
|
||||
<ChartCard title="Top 30 Tags" description="Word-cloud-style ranking (sized by count)">
|
||||
<div className="flex h-full flex-wrap content-start items-baseline gap-2 overflow-y-auto p-2">
|
||||
{data.map((t) => {
|
||||
const fontSize = computeTagFontSize(t.count, data[0]?.count ?? 1)
|
||||
return (
|
||||
<span
|
||||
key={t.key}
|
||||
style={{ fontSize: `${fontSize}px`, lineHeight: 1.1 }}
|
||||
className="font-medium text-foreground/80 hover:text-foreground"
|
||||
title={`${t.count} games`}
|
||||
>
|
||||
{t.key}
|
||||
</span>
|
||||
)
|
||||
})}
|
||||
</div>
|
||||
</ChartCard>
|
||||
)
|
||||
}
|
||||
|
||||
function computeTagFontSize(count: number, max: number): number {
|
||||
const min = 11
|
||||
const cap = 28
|
||||
const ratio = count / Math.max(1, max)
|
||||
return Math.round(min + (cap - min) * ratio)
|
||||
}
|
||||
|
||||
export function RecentDeletedChart({ deleted }: { deleted: DeletedGameEntry[] }) {
|
||||
const recent = useMemo(
|
||||
() =>
|
||||
[...deleted]
|
||||
.sort((a, b) => (b.deleted_at ?? '').localeCompare(a.deleted_at ?? ''))
|
||||
.slice(0, 12),
|
||||
[deleted],
|
||||
)
|
||||
return (
|
||||
<ChartCard title="Recent Deletions" description={`Last ${recent.length} of ${formatNumber(deleted.length)} total`}>
|
||||
<div className="h-full overflow-y-auto pr-2">
|
||||
<ul className="space-y-1.5 text-sm">
|
||||
{recent.length === 0 ? (
|
||||
<p className="text-muted-foreground">No deletions logged.</p>
|
||||
) : (
|
||||
recent.map((d) => (
|
||||
<li key={`${d.url}-${d.deleted_at}`} className="flex items-center justify-between gap-2 border-b py-1 last:border-0">
|
||||
<span className="truncate font-medium">{d.name}</span>
|
||||
<Badge variant="outline" className="flex-shrink-0 text-xs">
|
||||
{d.reason}
|
||||
</Badge>
|
||||
</li>
|
||||
))
|
||||
)}
|
||||
</ul>
|
||||
</div>
|
||||
</ChartCard>
|
||||
)
|
||||
}
|
||||
|
||||
export function topGenresList(entries: CountEntry[]): string {
|
||||
return entries.slice(0, 3).map((e) => e.key).join(', ')
|
||||
}
|
||||
@@ -0,0 +1,26 @@
|
||||
import { useMemo } from 'react'
|
||||
import { Bar, BarChart, CartesianGrid, ResponsiveContainer, Tooltip, XAxis, YAxis } from 'recharts'
|
||||
import { countBy, topN } from '@/lib/analytics'
|
||||
import type { Game } from '@/types/game'
|
||||
import { ChartCard } from './chart-card'
|
||||
import { PALETTE } from './palette'
|
||||
|
||||
export function AvgSessionChart({ games }: { games: Game[] }) {
|
||||
const data = useMemo(() => topN(countBy(games, 'average_session'), 8), [games])
|
||||
return (
|
||||
<ChartCard title="Average Session Length">
|
||||
<ResponsiveContainer width="100%" height="100%">
|
||||
<BarChart data={data}>
|
||||
<CartesianGrid strokeDasharray="3 3" />
|
||||
<XAxis dataKey="key" tick={{ fontSize: 10 }} interval={0} angle={-20} textAnchor="end" height={60} />
|
||||
<YAxis />
|
||||
<Tooltip
|
||||
cursor={{ fill: 'hsl(var(--accent))' }}
|
||||
contentStyle={{ background: 'hsl(var(--popover))', border: '1px solid hsl(var(--border))' }}
|
||||
/>
|
||||
<Bar dataKey="count" fill={PALETTE[6]} radius={[4, 4, 0, 0]} />
|
||||
</BarChart>
|
||||
</ResponsiveContainer>
|
||||
</ChartCard>
|
||||
)
|
||||
}
|
||||
@@ -0,0 +1,21 @@
|
||||
import { Card, CardContent, CardHeader, CardTitle } from '@/components/ui/card'
|
||||
|
||||
interface ChartCardProps {
|
||||
title: string
|
||||
description?: string
|
||||
children: React.ReactNode
|
||||
}
|
||||
|
||||
export function ChartCard({ title, description, children }: ChartCardProps) {
|
||||
return (
|
||||
<Card>
|
||||
<CardHeader>
|
||||
<CardTitle className="text-base">{title}</CardTitle>
|
||||
{description && (
|
||||
<p className="text-xs text-muted-foreground">{description}</p>
|
||||
)}
|
||||
</CardHeader>
|
||||
<CardContent className="h-72">{children}</CardContent>
|
||||
</Card>
|
||||
)
|
||||
}
|
||||
@@ -0,0 +1,30 @@
|
||||
import { useMemo } from 'react'
|
||||
import { Cell, Legend, Pie, PieChart, ResponsiveContainer, Tooltip } from 'recharts'
|
||||
import { deletionReasonCounts } from '@/lib/analytics'
|
||||
import type { DeletedGameEntry } from '@/types/game'
|
||||
import { ChartCard } from './chart-card'
|
||||
import { PALETTE } from './palette'
|
||||
|
||||
export function DeletionReasonsChart({ deleted }: { deleted: DeletedGameEntry[] }) {
|
||||
const data = useMemo(() => deletionReasonCounts(deleted), [deleted])
|
||||
return (
|
||||
<ChartCard title="Deletion Reasons" description="Why games left the catalog">
|
||||
<ResponsiveContainer width="100%" height="100%">
|
||||
<PieChart>
|
||||
<Pie data={data} dataKey="count" nameKey="key" innerRadius="50%" outerRadius="80%" paddingAngle={2}>
|
||||
{data.map((_, i) => (
|
||||
<Cell key={i} fill={PALETTE[i % PALETTE.length]} />
|
||||
))}
|
||||
</Pie>
|
||||
<Tooltip
|
||||
contentStyle={{
|
||||
background: 'hsl(var(--popover))',
|
||||
border: '1px solid hsl(var(--border))',
|
||||
}}
|
||||
/>
|
||||
<Legend />
|
||||
</PieChart>
|
||||
</ResponsiveContainer>
|
||||
</ChartCard>
|
||||
)
|
||||
}
|
||||
@@ -0,0 +1,26 @@
|
||||
import { useMemo } from 'react'
|
||||
import { Bar, BarChart, CartesianGrid, ResponsiveContainer, Tooltip, XAxis, YAxis } from 'recharts'
|
||||
import { deletionsByMonth } from '@/lib/analytics'
|
||||
import type { DeletedGameEntry } from '@/types/game'
|
||||
import { ChartCard } from './chart-card'
|
||||
import { PALETTE } from './palette'
|
||||
|
||||
export function DeletionsOverTimeChart({ deleted }: { deleted: DeletedGameEntry[] }) {
|
||||
const data = useMemo(() => deletionsByMonth(deleted), [deleted])
|
||||
return (
|
||||
<ChartCard title="Deletions Over Time" description="Games removed per month">
|
||||
<ResponsiveContainer width="100%" height="100%">
|
||||
<BarChart data={data}>
|
||||
<CartesianGrid strokeDasharray="3 3" />
|
||||
<XAxis dataKey="key" tick={{ fontSize: 10 }} interval={0} angle={-25} textAnchor="end" height={60} />
|
||||
<YAxis allowDecimals={false} />
|
||||
<Tooltip
|
||||
cursor={{ fill: 'hsl(var(--accent))' }}
|
||||
contentStyle={{ background: 'hsl(var(--popover))', border: '1px solid hsl(var(--border))' }}
|
||||
/>
|
||||
<Bar dataKey="count" fill={PALETTE[1]} radius={[4, 4, 0, 0]} />
|
||||
</BarChart>
|
||||
</ResponsiveContainer>
|
||||
</ChartCard>
|
||||
)
|
||||
}
|
||||
@@ -0,0 +1,29 @@
|
||||
import { CartesianGrid, Line, LineChart, ResponsiveContainer, Tooltip, XAxis, YAxis } from 'recharts'
|
||||
import type { CountHistoryPoint } from '@/types/game'
|
||||
import { ChartCard } from './chart-card'
|
||||
import { PALETTE } from './palette'
|
||||
|
||||
export function GameCountChart({ history }: { history: CountHistoryPoint[] }) {
|
||||
return (
|
||||
<ChartCard title="Game Count Over Time" description="Total games in the catalog by day">
|
||||
{history.length === 0 ? (
|
||||
<p className="flex h-full items-center justify-center text-sm text-muted-foreground">
|
||||
No history data yet.
|
||||
</p>
|
||||
) : (
|
||||
<ResponsiveContainer width="100%" height="100%">
|
||||
<LineChart data={history} margin={{ left: 4, right: 8 }}>
|
||||
<CartesianGrid strokeDasharray="3 3" />
|
||||
<XAxis dataKey="date" tick={{ fontSize: 10 }} minTickGap={24} />
|
||||
<YAxis tick={{ fontSize: 11 }} width={48} allowDecimals={false} domain={['auto', 'auto']} />
|
||||
<Tooltip
|
||||
cursor={{ stroke: 'hsl(var(--accent))' }}
|
||||
contentStyle={{ background: 'hsl(var(--popover))', border: '1px solid hsl(var(--border))' }}
|
||||
/>
|
||||
<Line type="monotone" dataKey="total" stroke={PALETTE[0]} strokeWidth={2} dot={false} />
|
||||
</LineChart>
|
||||
</ResponsiveContainer>
|
||||
)}
|
||||
</ChartCard>
|
||||
)
|
||||
}
|
||||
@@ -0,0 +1,26 @@
|
||||
import { useMemo } from 'react'
|
||||
import { Bar, BarChart, CartesianGrid, ResponsiveContainer, Tooltip, XAxis, YAxis } from 'recharts'
|
||||
import { countBy, topN } from '@/lib/analytics'
|
||||
import type { Game } from '@/types/game'
|
||||
import { ChartCard } from './chart-card'
|
||||
import { PALETTE } from './palette'
|
||||
|
||||
export function GenreChart({ games }: { games: Game[] }) {
|
||||
const data = useMemo(() => topN(countBy(games, 'genre'), 15), [games])
|
||||
return (
|
||||
<ChartCard title="Genres" description="Top 15 by game count">
|
||||
<ResponsiveContainer width="100%" height="100%">
|
||||
<BarChart data={data} layout="vertical" margin={{ left: 20 }}>
|
||||
<CartesianGrid horizontal={false} strokeDasharray="3 3" />
|
||||
<XAxis type="number" />
|
||||
<YAxis dataKey="key" type="category" width={120} tick={{ fontSize: 12 }} />
|
||||
<Tooltip
|
||||
cursor={{ fill: 'hsl(var(--accent))' }}
|
||||
contentStyle={{ background: 'hsl(var(--popover))', border: '1px solid hsl(var(--border))' }}
|
||||
/>
|
||||
<Bar dataKey="count" fill={PALETTE[0]} radius={[0, 4, 4, 0]} />
|
||||
</BarChart>
|
||||
</ResponsiveContainer>
|
||||
</ChartCard>
|
||||
)
|
||||
}
|
||||
@@ -0,0 +1,59 @@
|
||||
import { useMemo } from 'react'
|
||||
import { ResponsiveContainer, Tooltip, Treemap } from 'recharts'
|
||||
import { genreTreemapData } from '@/lib/analytics'
|
||||
import type { Game } from '@/types/game'
|
||||
import { ChartCard } from './chart-card'
|
||||
import { PALETTE } from './palette'
|
||||
|
||||
interface TreemapCellProps {
|
||||
x?: number
|
||||
y?: number
|
||||
width?: number
|
||||
height?: number
|
||||
index?: number
|
||||
depth?: number
|
||||
name?: string
|
||||
}
|
||||
|
||||
// Custom cell renderer: hides the label on rectangles too small to fit it,
|
||||
// so the treemap stays legible on narrow (mobile) viewports.
|
||||
function TreemapCell(props: TreemapCellProps) {
|
||||
const { x = 0, y = 0, width = 0, height = 0, index = 0, depth = 1, name = '' } = props
|
||||
if (depth !== 1 || width <= 0 || height <= 0) return null
|
||||
return (
|
||||
<g>
|
||||
<rect
|
||||
x={x}
|
||||
y={y}
|
||||
width={width}
|
||||
height={height}
|
||||
fill={PALETTE[index % PALETTE.length]}
|
||||
stroke="hsl(var(--background))"
|
||||
strokeWidth={2}
|
||||
/>
|
||||
{width > 56 && height > 24 && (
|
||||
<text x={x + 6} y={y + 18} fill="hsl(var(--background))" fontSize={12} fontWeight={600}>
|
||||
{name}
|
||||
</text>
|
||||
)}
|
||||
</g>
|
||||
)
|
||||
}
|
||||
|
||||
export function GenreTreemapChart({ games }: { games: Game[] }) {
|
||||
const data = useMemo(() => genreTreemapData(games, 12), [games])
|
||||
// Recharts' Treemap data prop demands an index signature; our typed nodes
|
||||
// are valid treemap data at runtime, so cast through unknown.
|
||||
const treemapData = data as unknown as React.ComponentProps<typeof Treemap>['data']
|
||||
return (
|
||||
<ChartCard title="Genre Treemap" description="Top 12 genres sized by game count">
|
||||
<ResponsiveContainer width="100%" height="100%">
|
||||
<Treemap data={treemapData} dataKey="size" nameKey="name" content={<TreemapCell />}>
|
||||
<Tooltip
|
||||
contentStyle={{ background: 'hsl(var(--popover))', border: '1px solid hsl(var(--border))' }}
|
||||
/>
|
||||
</Treemap>
|
||||
</ResponsiveContainer>
|
||||
</ChartCard>
|
||||
)
|
||||
}
|
||||
@@ -0,0 +1,16 @@
|
||||
export { GenreChart } from './genre-chart'
|
||||
export { PlatformChart } from './platform-chart'
|
||||
export { OnlineOfflineChart } from './online-offline-chart'
|
||||
export { LanguagesChart } from './languages-chart'
|
||||
export { MadeWithChart } from './made-with-chart'
|
||||
export { RatingChart } from './rating-chart'
|
||||
export { StatusChart } from './status-chart'
|
||||
export { AvgSessionChart } from './avg-session-chart'
|
||||
export { TopTagsChart } from './top-tags-chart'
|
||||
export { RecentDeletedChart } from './recent-deleted-chart'
|
||||
export { GameCountChart } from './game-count-chart'
|
||||
export { GenreTreemapChart } from './genre-treemap-chart'
|
||||
export { KpiCards } from './kpi-cards'
|
||||
export { DeletionsOverTimeChart } from './deletions-over-time-chart'
|
||||
export { DeletionReasonsChart } from './deletion-reasons-chart'
|
||||
export { TopRatedCountChart } from './top-rated-count-chart'
|
||||
@@ -0,0 +1,33 @@
|
||||
import { useMemo } from 'react'
|
||||
import { Card, CardContent } from '@/components/ui/card'
|
||||
import { computeKpis } from '@/lib/analytics'
|
||||
import { formatNumber } from '@/lib/utils'
|
||||
import type { DeletedGameEntry, Game } from '@/types/game'
|
||||
|
||||
interface KpiCardsProps {
|
||||
games: Game[]
|
||||
deleted: DeletedGameEntry[]
|
||||
}
|
||||
|
||||
export function KpiCards({ games, deleted }: KpiCardsProps) {
|
||||
const kpis = useMemo(() => computeKpis(games, deleted), [games, deleted])
|
||||
const items = [
|
||||
{ label: 'Total games', value: formatNumber(kpis.totalGames) },
|
||||
{ label: 'Online (HTML5)', value: formatNumber(kpis.onlineCount) },
|
||||
{ label: 'NSFW', value: formatNumber(kpis.nsfwCount) },
|
||||
{ label: 'Deleted (total)', value: formatNumber(kpis.totalDeleted) },
|
||||
{ label: 'Avg rating', value: kpis.avgRating.toFixed(2) },
|
||||
]
|
||||
return (
|
||||
<div className="grid grid-cols-2 gap-4 md:grid-cols-5">
|
||||
{items.map((item) => (
|
||||
<Card key={item.label}>
|
||||
<CardContent className="p-4">
|
||||
<p className="text-xs text-muted-foreground">{item.label}</p>
|
||||
<p className="mt-1 text-2xl font-bold tracking-tight">{item.value}</p>
|
||||
</CardContent>
|
||||
</Card>
|
||||
))}
|
||||
</div>
|
||||
)
|
||||
}
|
||||
@@ -0,0 +1,26 @@
|
||||
import { useMemo } from 'react'
|
||||
import { Bar, BarChart, CartesianGrid, ResponsiveContainer, Tooltip, XAxis, YAxis } from 'recharts'
|
||||
import { countByArray, topN } from '@/lib/analytics'
|
||||
import type { Game } from '@/types/game'
|
||||
import { ChartCard } from './chart-card'
|
||||
import { PALETTE } from './palette'
|
||||
|
||||
export function LanguagesChart({ games }: { games: Game[] }) {
|
||||
const data = useMemo(() => topN(countByArray(games, 'languages'), 10), [games])
|
||||
return (
|
||||
<ChartCard title="Top 10 Languages">
|
||||
<ResponsiveContainer width="100%" height="100%">
|
||||
<BarChart data={data}>
|
||||
<CartesianGrid strokeDasharray="3 3" />
|
||||
<XAxis dataKey="key" tick={{ fontSize: 11 }} interval={0} angle={-25} textAnchor="end" height={60} />
|
||||
<YAxis />
|
||||
<Tooltip
|
||||
cursor={{ fill: 'hsl(var(--accent))' }}
|
||||
contentStyle={{ background: 'hsl(var(--popover))', border: '1px solid hsl(var(--border))' }}
|
||||
/>
|
||||
<Bar dataKey="count" fill={PALETTE[4]} radius={[4, 4, 0, 0]} />
|
||||
</BarChart>
|
||||
</ResponsiveContainer>
|
||||
</ChartCard>
|
||||
)
|
||||
}
|
||||
@@ -0,0 +1,30 @@
|
||||
import { useMemo } from 'react'
|
||||
import { Cell, Legend, Pie, PieChart, ResponsiveContainer, Tooltip } from 'recharts'
|
||||
import { topMadeWith } from '@/lib/analytics'
|
||||
import type { Game } from '@/types/game'
|
||||
import { ChartCard } from './chart-card'
|
||||
import { PALETTE } from './palette'
|
||||
|
||||
export function MadeWithChart({ games }: { games: Game[] }) {
|
||||
const data = useMemo(() => topMadeWith(games, 8), [games])
|
||||
return (
|
||||
<ChartCard title="Game Engines" description="Primary engine per game (made_with[0])">
|
||||
<ResponsiveContainer width="100%" height="100%">
|
||||
<PieChart>
|
||||
<Pie data={data} dataKey="count" nameKey="key" innerRadius="50%" outerRadius="80%" paddingAngle={2}>
|
||||
{data.map((_, i) => (
|
||||
<Cell key={i} fill={PALETTE[i % PALETTE.length]} />
|
||||
))}
|
||||
</Pie>
|
||||
<Tooltip
|
||||
contentStyle={{
|
||||
background: 'hsl(var(--popover))',
|
||||
border: '1px solid hsl(var(--border))',
|
||||
}}
|
||||
/>
|
||||
<Legend />
|
||||
</PieChart>
|
||||
</ResponsiveContainer>
|
||||
</ChartCard>
|
||||
)
|
||||
}
|
||||
@@ -0,0 +1,33 @@
|
||||
import { useMemo } from 'react'
|
||||
import { Cell, Legend, Pie, PieChart, ResponsiveContainer, Tooltip } from 'recharts'
|
||||
import { classifyOnline } from '@/lib/analytics'
|
||||
import type { Game } from '@/types/game'
|
||||
import { ChartCard } from './chart-card'
|
||||
import { PALETTE } from './palette'
|
||||
|
||||
export function OnlineOfflineChart({ games }: { games: Game[] }) {
|
||||
const split = useMemo(() => classifyOnline(games), [games])
|
||||
const data = [
|
||||
{ key: 'Online (HTML5)', count: split.online },
|
||||
{ key: 'Offline only', count: split.offline },
|
||||
]
|
||||
return (
|
||||
<ChartCard title="Online vs Offline" description="HTML5 means playable in the browser">
|
||||
<ResponsiveContainer width="100%" height="100%">
|
||||
<PieChart>
|
||||
<Pie data={data} dataKey="count" nameKey="key" innerRadius="50%" outerRadius="80%">
|
||||
<Cell fill={PALETTE[2]} />
|
||||
<Cell fill={PALETTE[3]} />
|
||||
</Pie>
|
||||
<Tooltip
|
||||
contentStyle={{
|
||||
background: 'hsl(var(--popover))',
|
||||
border: '1px solid hsl(var(--border))',
|
||||
}}
|
||||
/>
|
||||
<Legend />
|
||||
</PieChart>
|
||||
</ResponsiveContainer>
|
||||
</ChartCard>
|
||||
)
|
||||
}
|
||||
@@ -0,0 +1,14 @@
|
||||
// Shared chart colour palette. Kept in its own file so chart-card.tsx
|
||||
// stays a component-only module (React Fast Refresh friendly).
|
||||
export const PALETTE = [
|
||||
'hsl(217 91% 60%)',
|
||||
'hsl(330 81% 60%)',
|
||||
'hsl(142 71% 45%)',
|
||||
'hsl(38 92% 50%)',
|
||||
'hsl(262 83% 58%)',
|
||||
'hsl(199 89% 48%)',
|
||||
'hsl(20 90% 55%)',
|
||||
'hsl(173 58% 39%)',
|
||||
'hsl(280 65% 60%)',
|
||||
'hsl(160 60% 45%)',
|
||||
]
|
||||
@@ -0,0 +1,37 @@
|
||||
import { useMemo } from 'react'
|
||||
import { Cell, Legend, Pie, PieChart, ResponsiveContainer, Tooltip } from 'recharts'
|
||||
import { countByArray } from '@/lib/analytics'
|
||||
import type { Game } from '@/types/game'
|
||||
import { ChartCard } from './chart-card'
|
||||
import { PALETTE } from './palette'
|
||||
|
||||
export function PlatformChart({ games }: { games: Game[] }) {
|
||||
const data = useMemo(() => countByArray(games, 'platforms'), [games])
|
||||
return (
|
||||
<ChartCard title="Platforms" description="Counted per game (a game may target several)">
|
||||
<ResponsiveContainer width="100%" height="100%">
|
||||
<PieChart>
|
||||
<Pie
|
||||
data={data}
|
||||
dataKey="count"
|
||||
nameKey="key"
|
||||
innerRadius="50%"
|
||||
outerRadius="80%"
|
||||
paddingAngle={2}
|
||||
>
|
||||
{data.map((_, i) => (
|
||||
<Cell key={i} fill={PALETTE[i % PALETTE.length]} />
|
||||
))}
|
||||
</Pie>
|
||||
<Tooltip
|
||||
contentStyle={{
|
||||
background: 'hsl(var(--popover))',
|
||||
border: '1px solid hsl(var(--border))',
|
||||
}}
|
||||
/>
|
||||
<Legend />
|
||||
</PieChart>
|
||||
</ResponsiveContainer>
|
||||
</ChartCard>
|
||||
)
|
||||
}
|
||||
@@ -0,0 +1,26 @@
|
||||
import { useMemo } from 'react'
|
||||
import { Bar, BarChart, CartesianGrid, ResponsiveContainer, Tooltip, XAxis, YAxis } from 'recharts'
|
||||
import { ratingHistogram } from '@/lib/analytics'
|
||||
import type { Game } from '@/types/game'
|
||||
import { ChartCard } from './chart-card'
|
||||
import { PALETTE } from './palette'
|
||||
|
||||
export function RatingChart({ games }: { games: Game[] }) {
|
||||
const data = useMemo(() => ratingHistogram(games, 10), [games])
|
||||
return (
|
||||
<ChartCard title="Rating Histogram" description="Bins of 0.5 stars from 0 to 5">
|
||||
<ResponsiveContainer width="100%" height="100%">
|
||||
<BarChart data={data}>
|
||||
<CartesianGrid strokeDasharray="3 3" />
|
||||
<XAxis dataKey="key" tick={{ fontSize: 10 }} />
|
||||
<YAxis />
|
||||
<Tooltip
|
||||
cursor={{ fill: 'hsl(var(--accent))' }}
|
||||
contentStyle={{ background: 'hsl(var(--popover))', border: '1px solid hsl(var(--border))' }}
|
||||
/>
|
||||
<Bar dataKey="count" fill={PALETTE[5]} radius={[4, 4, 0, 0]} />
|
||||
</BarChart>
|
||||
</ResponsiveContainer>
|
||||
</ChartCard>
|
||||
)
|
||||
}
|
||||
@@ -0,0 +1,41 @@
|
||||
import { useMemo } from 'react'
|
||||
import { Badge } from '@/components/ui/badge'
|
||||
import { formatNumber } from '@/lib/utils'
|
||||
import type { DeletedGameEntry } from '@/types/game'
|
||||
import { ChartCard } from './chart-card'
|
||||
|
||||
export function RecentDeletedChart({ deleted }: { deleted: DeletedGameEntry[] }) {
|
||||
const recent = useMemo(
|
||||
() =>
|
||||
[...deleted]
|
||||
.sort((a, b) => (b.deleted_at ?? '').localeCompare(a.deleted_at ?? ''))
|
||||
.slice(0, 12),
|
||||
[deleted],
|
||||
)
|
||||
return (
|
||||
<ChartCard
|
||||
title="Recent Deletions"
|
||||
description={`Last ${recent.length} of ${formatNumber(deleted.length)} total`}
|
||||
>
|
||||
<div className="h-full overflow-y-auto pr-2">
|
||||
<ul className="space-y-1.5 text-sm">
|
||||
{recent.length === 0 ? (
|
||||
<p className="text-muted-foreground">No deletions logged.</p>
|
||||
) : (
|
||||
recent.map((d) => (
|
||||
<li
|
||||
key={`${d.url}-${d.deleted_at}`}
|
||||
className="flex items-center justify-between gap-2 border-b py-1 last:border-0"
|
||||
>
|
||||
<span className="truncate font-medium">{d.name}</span>
|
||||
<Badge variant="outline" className="flex-shrink-0 text-xs">
|
||||
{d.reason}
|
||||
</Badge>
|
||||
</li>
|
||||
))
|
||||
)}
|
||||
</ul>
|
||||
</div>
|
||||
</ChartCard>
|
||||
)
|
||||
}
|
||||
@@ -0,0 +1,30 @@
|
||||
import { useMemo } from 'react'
|
||||
import { Cell, Legend, Pie, PieChart, ResponsiveContainer, Tooltip } from 'recharts'
|
||||
import { countBy } from '@/lib/analytics'
|
||||
import type { Game } from '@/types/game'
|
||||
import { ChartCard } from './chart-card'
|
||||
import { PALETTE } from './palette'
|
||||
|
||||
export function StatusChart({ games }: { games: Game[] }) {
|
||||
const data = useMemo(() => countBy(games, 'status'), [games])
|
||||
return (
|
||||
<ChartCard title="Status Mix" description="Released vs prototype/in-progress">
|
||||
<ResponsiveContainer width="100%" height="100%">
|
||||
<PieChart>
|
||||
<Pie data={data} dataKey="count" nameKey="key" outerRadius="80%">
|
||||
{data.map((_, i) => (
|
||||
<Cell key={i} fill={PALETTE[i % PALETTE.length]} />
|
||||
))}
|
||||
</Pie>
|
||||
<Tooltip
|
||||
contentStyle={{
|
||||
background: 'hsl(var(--popover))',
|
||||
border: '1px solid hsl(var(--border))',
|
||||
}}
|
||||
/>
|
||||
<Legend />
|
||||
</PieChart>
|
||||
</ResponsiveContainer>
|
||||
</ChartCard>
|
||||
)
|
||||
}
|
||||
@@ -0,0 +1,32 @@
|
||||
import { useMemo } from 'react'
|
||||
import { Bar, BarChart, CartesianGrid, ResponsiveContainer, Tooltip, XAxis, YAxis } from 'recharts'
|
||||
import { topByRatingCount } from '@/lib/analytics'
|
||||
import type { Game } from '@/types/game'
|
||||
import { ChartCard } from './chart-card'
|
||||
import { PALETTE } from './palette'
|
||||
|
||||
export function TopRatedCountChart({ games }: { games: Game[] }) {
|
||||
const data = useMemo(() => topByRatingCount(games, 10), [games])
|
||||
return (
|
||||
<ChartCard title="Most Rated Games" description="Top 10 by number of ratings">
|
||||
<ResponsiveContainer width="100%" height="100%">
|
||||
<BarChart data={data} layout="vertical" margin={{ left: 20 }}>
|
||||
<CartesianGrid horizontal={false} strokeDasharray="3 3" />
|
||||
<XAxis type="number" allowDecimals={false} />
|
||||
<YAxis
|
||||
dataKey="key"
|
||||
type="category"
|
||||
width={140}
|
||||
tick={{ fontSize: 11 }}
|
||||
tickFormatter={(v: string) => (v.length > 20 ? `${v.slice(0, 19)}…` : v)}
|
||||
/>
|
||||
<Tooltip
|
||||
cursor={{ fill: 'hsl(var(--accent))' }}
|
||||
contentStyle={{ background: 'hsl(var(--popover))', border: '1px solid hsl(var(--border))' }}
|
||||
/>
|
||||
<Bar dataKey="count" fill={PALETTE[3]} radius={[0, 4, 4, 0]} />
|
||||
</BarChart>
|
||||
</ResponsiveContainer>
|
||||
</ChartCard>
|
||||
)
|
||||
}
|
||||
@@ -0,0 +1,34 @@
|
||||
import { useMemo } from 'react'
|
||||
import { countByArray, topN } from '@/lib/analytics'
|
||||
import type { Game } from '@/types/game'
|
||||
import { ChartCard } from './chart-card'
|
||||
|
||||
export function TopTagsChart({ games }: { games: Game[] }) {
|
||||
const data = useMemo(() => topN(countByArray(games, 'tags'), 30), [games])
|
||||
return (
|
||||
<ChartCard title="Top 30 Tags" description="Word-cloud-style ranking (sized by count)">
|
||||
<div className="flex h-full flex-wrap content-start items-baseline gap-2 overflow-y-auto p-2">
|
||||
{data.map((t) => {
|
||||
const fontSize = computeTagFontSize(t.count, data[0]?.count ?? 1)
|
||||
return (
|
||||
<span
|
||||
key={t.key}
|
||||
style={{ fontSize: `${fontSize}px`, lineHeight: 1.1 }}
|
||||
className="font-medium text-foreground/80 hover:text-foreground"
|
||||
title={`${t.count} games`}
|
||||
>
|
||||
{t.key}
|
||||
</span>
|
||||
)
|
||||
})}
|
||||
</div>
|
||||
</ChartCard>
|
||||
)
|
||||
}
|
||||
|
||||
function computeTagFontSize(count: number, max: number): number {
|
||||
const min = 11
|
||||
const cap = 28
|
||||
const ratio = count / Math.max(1, max)
|
||||
return Math.round(min + (cap - min) * ratio)
|
||||
}
|
||||
@@ -0,0 +1,34 @@
|
||||
import { useEffect, useState } from 'react'
|
||||
import { ArrowUp } from 'lucide-react'
|
||||
import { Button } from '@/components/ui/button'
|
||||
|
||||
interface ScrollToTopProps {
|
||||
// The scroll container to watch — the app scrolls <main>, not window.
|
||||
targetRef: React.RefObject<HTMLElement | null>
|
||||
}
|
||||
|
||||
export default function ScrollToTop({ targetRef }: ScrollToTopProps) {
|
||||
const [visible, setVisible] = useState(false)
|
||||
|
||||
useEffect(() => {
|
||||
const el = targetRef.current
|
||||
if (!el) return
|
||||
const onScroll = () => setVisible(el.scrollTop > 400)
|
||||
onScroll()
|
||||
el.addEventListener('scroll', onScroll, { passive: true })
|
||||
return () => el.removeEventListener('scroll', onScroll)
|
||||
}, [targetRef])
|
||||
|
||||
if (!visible) return null
|
||||
|
||||
return (
|
||||
<Button
|
||||
size="icon"
|
||||
aria-label="Scroll to top"
|
||||
onClick={() => targetRef.current?.scrollTo({ top: 0, behavior: 'smooth' })}
|
||||
className="fixed bottom-6 right-6 z-50 h-11 w-11 rounded-full shadow-lg"
|
||||
>
|
||||
<ArrowUp className="h-5 w-5" />
|
||||
</Button>
|
||||
)
|
||||
}
|
||||
@@ -1,5 +1,10 @@
|
||||
import { useQuery } from '@tanstack/react-query'
|
||||
import { loadAllGames, loadIndex, loadDeletedLog } from '@/lib/github/data-store'
|
||||
import {
|
||||
loadAllGames,
|
||||
loadIndex,
|
||||
loadDeletedLog,
|
||||
loadCountHistory,
|
||||
} from '@/lib/github/data-store'
|
||||
|
||||
export function useIndex() {
|
||||
return useQuery({
|
||||
@@ -21,3 +26,10 @@ export function useDeletedGames() {
|
||||
queryFn: loadDeletedLog,
|
||||
})
|
||||
}
|
||||
|
||||
export function useCountHistory() {
|
||||
return useQuery({
|
||||
queryKey: ['count-history'],
|
||||
queryFn: loadCountHistory,
|
||||
})
|
||||
}
|
||||
@@ -8,7 +8,7 @@ export interface ThirdParty {
|
||||
|
||||
export const APP = {
|
||||
name: 'Itch.io Free Games DB',
|
||||
version: '3.4.1',
|
||||
version: '3.5.0',
|
||||
repo: 'https://github.com/poli0981/free-games-itchio-list',
|
||||
license: 'MIT',
|
||||
buildDate: typeof __BUILD_DATE__ !== 'undefined' ? __BUILD_DATE__ : 'dev',
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import type { Game } from '@/types/game'
|
||||
import type { Game, DeletedGameEntry } from '@/types/game'
|
||||
|
||||
export interface CountEntry {
|
||||
key: string
|
||||
@@ -64,7 +64,7 @@ export function computeOverview(games: Game[]): OverviewStats {
|
||||
}
|
||||
}
|
||||
|
||||
export function parseRating(rating: string): number | null {
|
||||
function parseRating(rating: string): number | null {
|
||||
const n = parseFloat(rating)
|
||||
return isFinite(n) ? n : null
|
||||
}
|
||||
@@ -107,3 +107,81 @@ export function ratingHistogram(games: Game[], bins = 10): CountEntry[] {
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
export interface TreemapNode {
|
||||
name: string
|
||||
size: number
|
||||
}
|
||||
|
||||
export function genreTreemapData(games: Game[], topN = 12): TreemapNode[] {
|
||||
return countBy(games, 'genre')
|
||||
.slice(0, topN)
|
||||
.map((e) => ({ name: e.key, size: e.count }))
|
||||
}
|
||||
|
||||
export interface KpiStats {
|
||||
totalGames: number
|
||||
onlineCount: number
|
||||
nsfwCount: number
|
||||
totalDeleted: number
|
||||
avgRating: number
|
||||
}
|
||||
|
||||
export function computeKpis(games: Game[], deleted: DeletedGameEntry[]): KpiStats {
|
||||
const overview = computeOverview(games)
|
||||
let ratingSum = 0
|
||||
let ratingCount = 0
|
||||
for (const g of games) {
|
||||
const r = parseRating(g.rating)
|
||||
if (r !== null) {
|
||||
ratingSum += r
|
||||
ratingCount += 1
|
||||
}
|
||||
}
|
||||
return {
|
||||
totalGames: overview.total,
|
||||
onlineCount: overview.htmlCount,
|
||||
nsfwCount: overview.nsfwCount,
|
||||
totalDeleted: deleted.length,
|
||||
avgRating: ratingCount ? ratingSum / ratingCount : 0,
|
||||
}
|
||||
}
|
||||
|
||||
export function deletionsByMonth(deleted: DeletedGameEntry[]): CountEntry[] {
|
||||
const m = new Map<string, number>()
|
||||
for (const d of deleted) {
|
||||
const month = (d.deleted_at ?? '').slice(0, 7)
|
||||
if (month.length !== 7) continue
|
||||
m.set(month, (m.get(month) ?? 0) + 1)
|
||||
}
|
||||
return Array.from(m, ([key, count]) => ({ key, count })).sort((a, b) => a.key.localeCompare(b.key))
|
||||
}
|
||||
|
||||
export function deletionReasonCounts(deleted: DeletedGameEntry[]): CountEntry[] {
|
||||
let paid = 0
|
||||
let removed = 0
|
||||
let other = 0
|
||||
for (const d of deleted) {
|
||||
const reason = (d.reason ?? '').toLowerCase()
|
||||
if (reason.includes('paid')) {
|
||||
paid += 1
|
||||
} else if (reason.includes('no longer exists')) {
|
||||
removed += 1
|
||||
} else {
|
||||
other += 1
|
||||
}
|
||||
}
|
||||
const out: CountEntry[] = []
|
||||
if (paid) out.push({ key: 'Became paid', count: paid })
|
||||
if (removed) out.push({ key: 'Page removed', count: removed })
|
||||
if (other) out.push({ key: 'Other', count: other })
|
||||
return out
|
||||
}
|
||||
|
||||
export function topByRatingCount(games: Game[], n = 10): CountEntry[] {
|
||||
return games
|
||||
.map((g) => ({ key: g.name, count: Number(g.rating_count) || 0 }))
|
||||
.filter((e) => e.count > 0)
|
||||
.sort((a, b) => b.count - a.count)
|
||||
.slice(0, n)
|
||||
}
|
||||
@@ -9,6 +9,7 @@ export const RAW_BASE = `https://raw.githubusercontent.com/${REPO.owner}/${REPO.
|
||||
export const PATHS = {
|
||||
index: 'data_game/index.json',
|
||||
chunk: (filename: string) => `data_game/${filename}`,
|
||||
countHistory: 'data_game/count_history.json',
|
||||
deletedJson: 'scripts/deleted_games.json',
|
||||
tempLink: 'scripts/temp_link.json',
|
||||
} as const
|
||||
|
||||
@@ -1,4 +1,10 @@
|
||||
import type { Game, GameDbIndex, ChunkPlan, DeletedGameEntry } from '@/types/game'
|
||||
import type {
|
||||
Game,
|
||||
GameDbIndex,
|
||||
ChunkPlan,
|
||||
DeletedGameEntry,
|
||||
CountHistoryPoint,
|
||||
} from '@/types/game'
|
||||
import { MAX_PER_FILE, PATHS } from '../config'
|
||||
import { fetchRaw } from './raw'
|
||||
|
||||
@@ -25,6 +31,14 @@ export async function loadDeletedLog(): Promise<DeletedGameEntry[]> {
|
||||
}
|
||||
}
|
||||
|
||||
export async function loadCountHistory(): Promise<CountHistoryPoint[]> {
|
||||
try {
|
||||
return await fetchRaw<CountHistoryPoint[]>(PATHS.countHistory)
|
||||
} catch {
|
||||
return []
|
||||
}
|
||||
}
|
||||
|
||||
export function rebalance(games: Game[]): ChunkPlan[] {
|
||||
const numChunks = Math.max(1, Math.ceil(games.length / MAX_PER_FILE))
|
||||
return Array.from({ length: numChunks }, (_, i) => ({
|
||||
|
||||
@@ -2,7 +2,12 @@ import { Skeleton } from '@/components/ui/skeleton'
|
||||
import { Tabs, TabsContent, TabsList, TabsTrigger } from '@/components/ui/tabs'
|
||||
import {
|
||||
AvgSessionChart,
|
||||
DeletionReasonsChart,
|
||||
DeletionsOverTimeChart,
|
||||
GameCountChart,
|
||||
GenreChart,
|
||||
GenreTreemapChart,
|
||||
KpiCards,
|
||||
LanguagesChart,
|
||||
MadeWithChart,
|
||||
OnlineOfflineChart,
|
||||
@@ -10,15 +15,17 @@ import {
|
||||
RatingChart,
|
||||
RecentDeletedChart,
|
||||
StatusChart,
|
||||
TopRatedCountChart,
|
||||
TopTagsChart,
|
||||
} from '@/components/charts'
|
||||
import { useAllGames, useDeletedGames } from '@/hooks/useGames'
|
||||
import { useAllGames, useCountHistory, useDeletedGames } from '@/hooks/useGames'
|
||||
import { useDocumentTitle } from '@/hooks/useDocumentTitle'
|
||||
|
||||
export default function Charts() {
|
||||
useDocumentTitle('Charts')
|
||||
const games = useAllGames()
|
||||
const deleted = useDeletedGames()
|
||||
const history = useCountHistory()
|
||||
|
||||
if (games.isLoading) {
|
||||
return (
|
||||
@@ -44,6 +51,7 @@ export default function Charts() {
|
||||
|
||||
const data = games.data?.games ?? []
|
||||
const deletedList = deleted.data ?? []
|
||||
const historyList = history.data ?? []
|
||||
|
||||
return (
|
||||
<div className="container mx-auto p-6">
|
||||
@@ -57,12 +65,14 @@ export default function Charts() {
|
||||
<TabsTrigger value="discovery">Discovery</TabsTrigger>
|
||||
</TabsList>
|
||||
|
||||
<TabsContent value="overview">
|
||||
<TabsContent value="overview" className="space-y-4">
|
||||
<KpiCards games={data} deleted={deletedList} />
|
||||
<div className="grid gap-4 md:grid-cols-2">
|
||||
<GenreChart games={data} />
|
||||
<StatusChart games={data} />
|
||||
<OnlineOfflineChart games={data} />
|
||||
<MadeWithChart games={data} />
|
||||
<GameCountChart history={historyList} />
|
||||
</div>
|
||||
</TabsContent>
|
||||
|
||||
@@ -76,15 +86,18 @@ export default function Charts() {
|
||||
</TabsContent>
|
||||
|
||||
<TabsContent value="quality">
|
||||
<div className="grid gap-4 md:grid-cols-1">
|
||||
<div className="grid gap-4 md:grid-cols-2">
|
||||
<RatingChart games={data} />
|
||||
<RecentDeletedChart deleted={deletedList} />
|
||||
<TopRatedCountChart games={data} />
|
||||
<DeletionsOverTimeChart deleted={deletedList} />
|
||||
<DeletionReasonsChart deleted={deletedList} />
|
||||
</div>
|
||||
</TabsContent>
|
||||
|
||||
<TabsContent value="discovery">
|
||||
<div className="grid gap-4">
|
||||
<div className="grid gap-4 md:grid-cols-2">
|
||||
<TopTagsChart games={data} />
|
||||
<GenreTreemapChart games={data} />
|
||||
</div>
|
||||
</TabsContent>
|
||||
</Tabs>
|
||||
|
||||
@@ -38,6 +38,11 @@ export interface DeletedGameEntry {
|
||||
deleted_at: string
|
||||
}
|
||||
|
||||
export interface CountHistoryPoint {
|
||||
date: string
|
||||
total: number
|
||||
}
|
||||
|
||||
export interface ChunkPlan {
|
||||
name: string
|
||||
games: Game[]
|
||||
|
||||
Reference in new issue
Block a user