Files
pipdock/docs/DATA-FLOW.md
T
poli0981andClaude Opus 5 4ab948ac7b feat: implement the two owner decisions from spike week
Requires-Python is now enforced by PipDock, not by the engine.

SP-2 found the engines disagree: scipy 1.7.3 declares >=3.7,<3.11, and on
Python 3.12 pip refuses it while uv plans to install it. G5 promises one
behaviour across heads, so the preview must not change shape because the
user flipped the engine radio. crates/pipdock-core/src/compat.rs screens
every candidate against the environment's version before any engine
command runs; rejects surface as PD-PKG-001 with the required range shown
against the found version.

This does not weaken "explain, don't reimplement" — resolution stays
entirely the engine's. compat.rs only filters candidates the engine should
never have been offered, using the slice of PEP 440 that actually appears
in Requires-Python: comparison, wildcard and compatible-release operators.
Specifiers it cannot parse count as compatible, because PipDock failing to
read metadata must never be the reason an installable package is refused.
Nine tests, including the exact scipy case the spike captured and the
boundary cases where an off-by-one would reject a supported Python.

probe.py keeps -I, and the cost is disclosed rather than hidden.

Isolated mode is what stops a poisoned environment injecting code into the
probe, so it stays; dropping to -E was rejected because a package in user
site is part of the untrusted surface the isolation exists to exclude, and
probing twice was rejected as a second interpreter start per scan for a
non-primary case. Instead probe.py now reports hidden_user_site — the
user-site path when it exists, sits off sys.path and is non-empty, else
null. Verified: the system Python reports the path (352 dists), a venv
reports null. The Installed screen shows a note naming the path only when
it is non-null, so users of the affected case are told exactly what is
missing while everyone else sees nothing.

ERROR-CATALOG PD-PKG-001 changes from a stderr classification to a
PipDock-raised code, because engine stderr cannot support it: pip reports
it as plain "No matching distribution found", indistinguishable from
PD-PKG-002, and uv does not enforce it at all.

Docs updated: ARCHITECTURE 3 and 4, DATA-FLOW 2, SECURITY 2, UI-SPEC 4,
ERROR-CATALOG 2, ROADMAP, CLAUDE.md, spikes/README.md. EN and VI strings
added for both notes.

Two non-blocking questions remain in spikes/README.md: search-ranking
scope for M1, and how to obtain the five error fixtures this machine
cannot produce.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-27 18:38:46 +07:00

8.0 KiB
Raw Blame History

PipDock — Data Flow

Version 0.1 · 2026-07-17. This document is the corrected, authoritative version of the original concept flows.

1. Correction log vs. original concept

Original concept Corrected design Why
"Check Conflict → If True → update" No conflict → update. Conflict → explain → 3-way choice Original condition was inverted; binary Yes/No hid the safe middle option
Per-package conflict check Group dry-run resolve of the whole selection Resolving A+B+D together often succeeds where one-by-one is stuck; matches how pip's resolver actually works
"Show the full list of all Python packages" Search-first over a locally cached PyPI name index PyPI hosts ~600 k projects; a browsable list is unusable and unfetchable
(absent) Environment selection + PEP 668 guard as step zero Without it, users mutate system Python by accident — the worst possible outcome
(absent) Snapshot before every batch Makes every flow reversible; prerequisite for user trust

2. Shared preamble (all mutating flows)

[Select env] → probe.py -I → env facts (python ver, PEP 668, hidden_user_site, dist metadata)
     │
     ├─ EXTERNALLY-MANAGED detected → BLOCKED banner (PD-ENV-002)
     │        └─ explicit override in Settings → proceed with persistent warning chip
     ├─ hidden_user_site non-null → Installed screen shows the partial-listing note
     │                              (not a block; see SECURITY §2)
     └─ ok → flow continues; reverse-dep graph built from probe data

Every candidate is then screened against the environment's Python version by
pipdock-core (ARCHITECTURE §3): incompatible ones are dropped from the plan with
PD-PKG-001 before any engine command runs, so the preview is identical under pip
and uv even though the engines themselves disagree.

3. Update flow (state machine)

Idle
 └─ env selected → Scanning            engine.list_outdated()
     └─ SelectionReady                 rows: name, current → latest, pinned rows locked out of Select-all
         └─ user selects + [Update] → Resolving
             │                         engine.resolve(PlanRequest{upgrades, strategy: Compatible})
             ├─ error → Failed(code)   catalog-classified; retry available
             └─ PreviewReady           ResolutionReport rendered as a diff:
                 │                       · clean upgrades  (A 1.2 → 2.0)
                 │                       · new transitive installs
                 │                       · held-back items (B → 0.1.9, latest 0.5.0, blocked by A: "requests<0.2")
                 │                       · impossible items
                 ├─ conflicts exist → ConflictDecision (per held-back/impossible pkg):
                 │       [Keep compatible]  default; accept resolver's version   (held-back only)
                 │       [Skip]             drop pkg from this plan
                 │       [Force latest]     adds per-pkg override; warning names what breaks
                 │     any Force/Skip chosen → re-Resolve with overrides → PreviewReady (loop, max 3 rounds
                 │     then require manual pruning — prevents decision ping-pong)
                 └─ [Confirm] → Snapshotting   freeze + meta written; failure here ABORTS the plan (PD-SNP-001)
                       └─ Executing
                            Phase A: single batch install of pinned set ── ok ──→ PostCheck
                            Phase B (on A-failure): per-pkg sequential, skip-and-continue,
                                     live stderr → console via plan-progress events
                       └─ PostCheck            engine.check(); findings appended to summary
                            └─ Summary         see §6

Cancellation: allowed in Resolving/Executing; kills child process group; completed steps remain; summary marks the rest Skipped(UserCancelled); snapshot offers full revert.

4. Install flow

SearchIdle → (keystrokes) → QueryingIndex (local fuzzy, <50 ms) → Results
   └─ row focus → MetadataFetch (PyPI JSON, cached 24 h) → detail panel
   └─ [Add] → Queue ("dock bay", persists across searches; versions default latest, editable to spec)
Queue → [Install n packages] → Resolving (PlanRequest{installs}) → PreviewReady → … same tail as §3

Installed-tab presentation (per original concept, kept): up-to-date rows dimmed, outdated rows carry an UPDATE badge and are mirrored into the Updates tab; already-installed packages appearing in search results show an INSTALLED ✓ chip (or UPDATE if outdated) instead of [Add].

5. Uninstall flow

Selection → [Uninstall] → GuardCheck: reverse-dep graph over full removal set
   ├─ breaks nothing → Confirm dialog (lists removals)
   └─ breaks {Y,Z} → Warning dialog: "Removing X breaks Y (requires X>=1), Z"
        options: [Cancel] [Remove dependents too] (adds Y,Z to set, re-guard) [Force remove only X]
→ Snapshot → sequential engine.uninstall per pkg (skip-and-continue) → Summary

Rationale: bare pip uninstall performs no dependency check — this guard is a core value-add.

6. Summary model (owner requirement)

{ "planId":"…","phase":"B","results":[
   {"pkg":"httpx","from":"0.27.0","to":"0.28.1","status":"ok"},
   {"pkg":"oldlib","to":"2.0.0","status":"failed","code":"PD-BLD-002",
    "reason":"build backend error in pyproject.toml","stderrTail":"…"}],
  "check":{"ok":true,"findings":[]},
  "counts":{"ok":13,"failed":2,"skipped":1} }

Rendered as: "13 successful, 2 failed, 1 skipped" with an expandable row per failure (catalog code, human reason EN/VI, stderr tail, Copy log, Report bug deep link — see ERROR-CATALOG §4).

7. Engine command mapping

Operation pip (<py> -m pip …) uv (uv pip … --python <py>) Adapter notes
List installed list --format=json list --format=json shapes differ slightly; normalize to Dist
List outdated list --outdated --format=json list --outdated --format=json uv output pinned in SP-1 fixtures
Dry-run resolve install -U --dry-run --quiet --report - (JSON to stdout) uv pip install -U --dry-run (text plan; no stable JSON report) uv adapter parses text; exact format frozen via SP-1 fixtures + snapshot tests
Install pinned install pkg==v [--no-deps in Phase A per-report set] uv pip install pkg==v Phase A passes the full pinned set in one call
Uninstall uninstall -y pkg uv pip uninstall pkg
Env check check uv pip check normalize findings
Freeze (snapshots) freeze --all uv pip freeze pip's --all includes pip/setuptools; record engine in snapshot meta
Upgrade pip install -U pip (n/a — surface "pip upkeep" only when pip engine active or pip present in env)

Version floors: pip ≥ 22.2 for --dry-run --report (offer in-app pip upgrade when older); uv minimum pinned after SP-1.

8. Rollback flow

Snapshot selected → Diff = (current freeze) vs (snapshot freeze)
   → plan: uninstall {added}, install {removed ∪ changed @ snapshot versions}
   → dry-run resolve the restore set (a yanked/deleted release cannot be restored → PD-SNP-002,
     listed explicitly; user may proceed partially)
   → snapshot current state first (rollback is itself reversible) → execute two-phase → summary

9. Data-flow invariants (enforced in pipdock-core, tested)

  1. No mutating engine call without a ResolutionReport accepted in this session.
  2. No mutating engine call without a successful snapshot write.
  3. plan_execute refuses a report older than 10 minutes or if the env's probe hash changed (env drifted → re-resolve).
  4. Every failure surfaced to UI/CLI carries a catalog code.
  5. Pinned packages never appear in a PlanRequest.upgrades unless the user explicitly unpinned them this session.