Files
pipdock/docs/DATA-FLOW.md
T
poli0981andClaude Opus 5 c281d054c0 fix(core): pip upkeep runs pip, whatever engine is configured
`pipdock pip-upgrade --engine uv` failed at PD-ENG-001 — while the line immediately below
had already reached past the abstraction to call PipEngine.info directly for the version.

Upgrading pip is a pip operation by definition. There is no uv way to do it, which is why
UvEngine::upgrade_pip exists only to refuse, and the engine setting is a preference about
the *user's* environments. Dispatching on it meant the one command with no resolver
between the user and the result was also the one that could be blocked by an unrelated
choice. Same reasoning P2 applied to the tools venv: pip is present wherever Python is.

Both heads now call PipEngine::upgrade_pip directly. The uv refusal stays rather than
becoming an unreachable!() — the trait is public and for_id can still hand a caller that
adapter, where a catalog code beats a panic.

That refusal was also the only thing standing in front of this path, so PipEngine now
carries its own PEP 668 guard, copied verbatim from UninstallFlow::start rather than
reworded so the three cannot drift. DATA-FLOW §2's preamble is "all mutating flows" and
replacing pip in site-packages is one; UpdateFlow has refused since S1 and UninstallFlow
since S5, and this had no guard at all. It stayed harmless only while nothing but a CLI
command could reach it — the next commit gives it a button.

Verified by running: `pip-upgrade --engine uv` on a 3.12 venv reports pip 25.0.1 -> 26.2.1
and exits 0, and a second run is idempotent.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-12 22:45:02 +07:00

12 KiB
Raw Blame History

PipDock — Data Flow

Version 0.1 · 2026-07-17. This document is the corrected, authoritative version of the original concept flows.

1. Correction log vs. original concept

Original concept Corrected design Why
"Check Conflict → If True → update" No conflict → update. Conflict → explain → 3-way choice Original condition was inverted; binary Yes/No hid the safe middle option
Per-package conflict check Group dry-run resolve of the whole selection Resolving A+B+D together often succeeds where one-by-one is stuck; matches how pip's resolver actually works
"Show the full list of all Python packages" Search-first over a locally cached PyPI name index PyPI hosts ~600 k projects; a browsable list is unusable and unfetchable
(absent) Environment selection + PEP 668 guard as step zero Without it, users mutate system Python by accident — the worst possible outcome
(absent) Snapshot before every batch Makes every flow reversible; prerequisite for user trust

2. Shared preamble (all mutating flows)

[Select env] → probe.py -I → env facts (python ver, PEP 668, hidden_user_site, dist metadata)
     │
     ├─ EXTERNALLY-MANAGED detected → BLOCKED banner (PD-ENV-002)
     │        └─ explicit override in Settings → proceed with persistent warning chip
     ├─ hidden_user_site non-null → Installed screen shows the partial-listing note
     │                              (not a block; see SECURITY §2)
     └─ ok → flow continues; reverse-dep graph built from probe data

Every candidate is then screened against the environment's Python version by
pipdock-core (ARCHITECTURE §3): incompatible ones are dropped from the plan with
PD-PKG-001 before any engine command runs, so the preview is identical under pip
and uv even though the engines themselves disagree.

3. Update flow (state machine)

Idle
 └─ env selected → Scanning            engine.list_outdated()
     └─ SelectionReady                 rows: name, current → latest, pinned rows locked out of Select-all
         └─ user selects + [Update] → Resolving
             │                         engine.resolve(PlanRequest{upgrades, strategy: Compatible})
             ├─ error → Failed(code)   catalog-classified; retry available
             └─ PreviewReady           ResolutionReport rendered as a diff:
                 │                       · clean upgrades  (A 1.2 → 2.0)
                 │                       · new transitive installs
                 │                       · held-back items (B → 0.1.9, latest 0.5.0, blocked by A: "requests<0.2")
                 │                       · impossible items
                 ├─ conflicts exist → ConflictDecision (per held-back/impossible pkg):
                 │       [Keep compatible]  default; accept resolver's version   (held-back only)
                 │       [Skip]             drop pkg from this plan
                 │       [Force latest]     adds per-pkg override; warning names what breaks
                 │     any Force/Skip chosen → re-Resolve with overrides → PreviewReady (loop, max 3 rounds
                 │     then require manual pruning — prevents decision ping-pong)
                 └─ [Confirm] → Snapshotting   freeze + meta written; failure here ABORTS the plan (PD-SNP-001)
                       └─ Executing
                            Phase A: single batch install of pinned set ── ok ──→ PostCheck
                            Phase B (on A-failure): per-pkg sequential, skip-and-continue,
                                     live stderr → console via plan-progress events
                       └─ PostCheck            engine.check(); findings appended to summary
                            └─ Summary         see §6

Cancellation: allowed in Resolving/Executing; kills child process group; completed steps remain; summary marks the rest Skipped(UserCancelled); snapshot offers full revert.

4. Install flow

SearchIdle → (keystrokes) → QueryingIndex (local fuzzy, <50 ms) → Results
   └─ row focus → MetadataFetch (PyPI JSON, cached 24 h) → detail panel
   └─ [Add] → Queue ("dock bay", persists across searches; versions default latest, editable to spec)
Queue → [Install n packages] → Resolving (PlanRequest{installs}) → PreviewReady → … same tail as §3

Installed-tab presentation (per original concept, kept): up-to-date rows dimmed, outdated rows carry an UPDATE badge and are mirrored into the Updates tab; already-installed packages appearing in search results show an INSTALLED ✓ chip (or UPDATE if outdated) instead of [Add].

5. Uninstall flow

Selection → [Uninstall] → GuardCheck: reverse-dep graph over full removal set
   ├─ breaks nothing → Confirm dialog (lists removals)
   └─ breaks {Y,Z} → Warning dialog: "Removing X breaks Y (requires X>=1), Z"
        options: [Cancel] [Remove dependents too] (adds Y,Z to set, re-guard) [Force remove only X]
→ Snapshot → sequential engine.uninstall per pkg (skip-and-continue) → Summary

Rationale: bare pip uninstall performs no dependency check — this guard is a core value-add.

The dialog is not the enforcement point. Remove dependents too is the caller running the guard again over the widened set, because pulling Y in can break Z and stopping after one level hands the user a set that still breaks something. And the removal itself refuses to run unless the guard was clear or the user explicitly accepted the breakage — flow::GuardAck, checked in UninstallFlow::execute the way SnapshotProof is, so a caller that never looked at the report cannot proceed by omission. Attempting it anyway is PD-RES-004, raised before the snapshot is written so a plan that will not run leaves nothing behind.

A removal is blocked on a PEP 668 environment exactly as an update is: §2's preamble is "all mutating flows", and removing from a system Python is the more dangerous of the two, since no resolver stands between the user and the damage.

6. Summary model (owner requirement)

{ "planId":"…","phase":"B","results":[
   {"pkg":"httpx","from":"0.27.0","to":"0.28.1","status":"ok"},
   {"pkg":"oldlib","to":"2.0.0","status":"failed","code":"PD-BLD-002",
    "reason":"build backend error in pyproject.toml","stderrTail":"…"}],
  "check":{"ok":true,"findings":[]},
  "counts":{"ok":13,"failed":2,"skipped":1} }

Rendered as: "13 successful, 2 failed, 1 skipped" with an expandable row per failure (catalog code, human reason EN/VI, stderr tail, Copy log, Report bug deep link — see ERROR-CATALOG §4).

7. Engine command mapping

Operation pip (<py> -m pip …) uv (uv pip … --python <py>) Adapter notes
List installed (not used — see below) (not used — see below) Engine::list_installed exists and normalizes both shapes to Dist, but only resolve() calls it, to build SP-1's guard set
List outdated list --outdated --format=json list --outdated --format=json uv output pinned in SP-1 fixtures
Dry-run resolve install -U --dry-run --quiet --report - (JSON to stdout) uv pip install -U --dry-run (text plan; no stable JSON report) uv adapter parses text; exact format frozen via SP-1 fixtures + snapshot tests
Install pinned install pkg==v [--no-deps in Phase A per-report set] uv pip install pkg==v Phase A passes the full pinned set in one call
Uninstall uninstall -y pkg uv pip uninstall pkg
Env check check uv pip check normalize findings
Freeze (snapshots) freeze --all uv pip freeze pip's --all includes pip/setuptools; record engine in snapshot meta
Upgrade pip install -U pip (n/a — always runs through the pip adapter, whatever engine is configured)

Pip upkeep does not dispatch on the configured engine (amended by Phase 3 · P1). Upgrading pip is a pip operation by definition; there is no uv way to do it, and UvEngine::upgrade_pip exists only to refuse. Dispatching meant --engine uv failed at PD-ENG-001 for a preference that has nothing to do with the operation, on the one command with no resolver between the user and the result — while the surrounding code was already reaching past the abstraction to read pip's version. Both heads now call PipEngine::upgrade_pip directly. Same reasoning as the Code Health tools venv (CODE-HEALTH-SPEC §2 as amended by P2): pip is present wherever Python is, and a feature must not go dark over an unrelated setting.

The refusal it replaces is also why PipEngine::upgrade_pip carries its own PEP 668 guard: §2's preamble is "all mutating flows", and replacing pip in site-packages is one. UpdateFlow has refused since S1 and UninstallFlow since S5; this path had no guard at all while its only caller was a CLI command with no button behind it.

Neither head lists installed packages through the engine. pipdock list and pkg_list both read probe.py instead, because the probe carries Requires-Dist — which list --format=json does not report, and which the reverse-dependency graph, the uninstall guard and blocker attribution are all built from. Reading the engine here would give the Installed screen a different package set than the guard that protects it. The -I trade-off this buys is documented in ARCHITECTURE §4 and surfaced as hidden_user_site.

Version floors: pip ≥ 22.2 for --dry-run --report (offer in-app pip upgrade when older); uv minimum pinned after SP-1.

8. Rollback flow

Snapshot selected → Diff = (current freeze) vs (snapshot freeze)
   → plan: uninstall {added}, install {removed ∪ changed @ snapshot versions}
   → dry-run resolve the restore set (a yanked/deleted release cannot be restored → PD-SNP-002,
     listed explicitly; user may proceed partially)
   → snapshot current state first (rollback is itself reversible) → execute two-phase → summary

The dry-run resolve step is implemented in neither head, and S6 deliberately did not add it. RollbackFlow::start diffs, plans, and reports unrestorable_lines — the entries no index can supply, which is what the preview lists as PD-SNP-002. What it does not do is ask the engine whether each restore spec is still fetchable, so a release yanked since the snapshot was taken surfaces as a per-package failure at execute time rather than in the preview.

Adding it is a network round trip on an interaction budgeted as instant, and it would resolve against the pre-removal environment — reporting conflicts the two-phase execution never hits, because the removals happen first. It needs its own design, and it belongs with the engine work in M3 rather than bolted onto a preview. Until then the preview's promise is precise: it lists what cannot be restored, not everything that might fail.

9. Data-flow invariants (enforced in pipdock-core, tested)

  1. No mutating engine call without an accepted proof for this session: a ResolutionReport for a resolve-shaped plan (update, install), a GuardReport plus a GuardAck for a removal (§5), a RollbackPlan for a restore (§8). Originally written as "a ResolutionReport", which the uninstall path cannot produce — there is nothing to resolve — leaving the one flow with no preview also the one flow the invariant did not describe.
  2. No mutating engine call without a successful snapshot write.
  3. plan_execute refuses a report older than 10 minutes or if the env's probe hash changed (env drifted → re-resolve).
  4. Every failure surfaced to UI/CLI carries a catalog code.
  5. Pinned packages never appear in a PlanRequest.upgrades unless the user explicitly unpinned them this session.