fix(scripts): the sanitizer stops leaking Chrome profile titles, and every script is checked

sanitize-bookmarks.mjs accepted any JSON object and rewrote only "url" and
"title". Chrome's profile Bookmarks file keeps titles under "name" (plus
meta_info and a base64 sync_metadata blob), so it came out with every title
intact, passed the host-only leak check, and was written as "sanitized" —
the exact file a reporter whose import failed would attach.

- JSON dialects are recognised by shape (BookmarkMagic, Chrome profile,
  Firefox backup); any other JSON is refused. Rewriting is default-deny: a
  small tokenizer tracks the key of every string value and replaces all of
  them except structural keys (types, ids, timestamps, root GUIDs), by span,
  so layout and key order stay byte-identical.
- The backstop now reads the output's TEXT: for HTML it strips markup and
  refuses on any word that is not a placeholder — independent of the
  rewriting rules, unlike the old host-only check whose test shared its
  blind spots. <DD class=…> notes, file: ICON_URIs and <H1>/<TITLE>
  owner names are rewritten.
- Attribute values decode in one pass (CodeQL js/double-escaping), <DD>
  bodies are found in linear time (Sonar S8786), the input is never
  overwritten, size is checked before reading, and bytes are bytes.
- The 627-line file is split into scripts/sanitize/ modules; the
  user-facing entry stays plain .mjs for reporters on any Node.

gen-fixture.mjs validates its flags (whole numbers >= 1, values required, no
unknown flags) and its loop visibly advances (Sonar S2189 blocker).
gen-icons.mjs says what is true: it is the icons' source of truth.

Every script now carries // @ts-check and tsconfig.scripts.json runs them
through the TS 7 tsc in `npm run check`; docs/10's size rule names .mjs.
Regenerated icons are byte-identical.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
poli0981andClaude Opus 5.5 committed 2026-10-29 00:06:52 +07:00
1 parent 94b74f6ac4
commit 0671bb3c33
22 files changed
+1326 -608

No files matched your search

+3 -1
View File
@@ -37,7 +37,9 @@ body:
label: File format
options:
- HTML (the usual "bookmarks.html" export)
- JSON
- Chrome / Edge / Brave profile "Bookmarks" file
- Firefox JSON backup (bookmarks-YYYY-MM-DD.json)
- BookmarkMagic JSON
- CSV
- Not sure
validations:
+1 -1
View File
@@ -41,7 +41,7 @@ jobs:
- run: npx --no-install wxt prepare
- run: npm run lint # biome check .
- run: npm run check # svelte-check (TS 6 API) + tsc (TS 7)
- run: npm run check # svelte-check (TS 6 API) + tsc (TS 7) + scripts checkJs
- run: npm run knip
- run: npm run coverage # vitest --coverage, thresholds per docs/11 §1
- run: npm run guard # the zero-network / no-raw-HTML gate, docs/08 §3
+8 -4
View File
@@ -7,10 +7,14 @@ node_modules/
stats.html
stats-*.json
# Test / coverage
# Test / coverage (Vitest 5 writes reporter output under .vitest/)
coverage/
.vitest/
*.tsbuildinfo
# WXT runner config — holds local browser paths, never shared
web-ext.config.ts
# Editors
.idea/
.vs/
@@ -33,8 +37,8 @@ npm-debug.log*
.env.local
.env.*.local
# Generated fixtures that are too big to commit (tests/fixtures/generate-huge.ts
# writes here on demand — docs/11 §2)
# Generated fixtures that are too big to commit (scripts/gen-fixture.mjs writes
# here on demand — docs/11 §2)
tests/fixtures/generated/
.claude/
.claude/
+9 -2
View File
@@ -21,8 +21,15 @@ node scripts/sanitize-bookmarks.mjs your-bookmarks.html
It needs nothing installed — plain Node. It writes
`your-bookmarks.sanitized.html` next to the original, and prints what it found.
It reads the usual `bookmarks.html` export, a BookmarkMagic JSON file, Chrome's
profile `Bookmarks` file (also Edge, Brave and Vivaldi), a Firefox
`bookmarks-YYYY-MM-DD.json` backup, and our CSV. It refuses anything else rather
than guess, and it refuses to write if its own output still looks personal.
What it **destroys**: every hostname, path, query value and fragment; every
bookmark and folder title; Firefox keywords and tags; favicon data.
bookmark and folder title and description; Firefox keywords and tags; favicon
data; local `file:` paths; in JSON, every string except structural fields
(types, ids, timestamps), including Chrome's `meta_info` and sync blob.
What it **keeps**, because this is where the bug lives: the byte order mark, the
doctype, the `<META>` line, comments, line endings, indentation, tag and
@@ -110,7 +117,7 @@ The full ritual, in order, because it is easy to do four fifths of it:
line-ending normalization cannot rewrite the CRLF or BOM bytes the bug may
depend on — do not "fix" that.
2. **Add the regression test** to `tests/unit/core/fixtures.test.ts`, the only
fixture-driven suite. A browser export goes in its `BROWSER_EXPORTS` array,
parser regression suite. A browser export goes in its `BROWSER_EXPORTS` array,
which drives a shared set of assertions; anything else gets its own case.
There is no shared fixture-loading helper — that file has its own loader.
3. **Fix the parser** in `src/lib/core/parse/`. The test and the fix land in the
+4 -1
View File
@@ -63,11 +63,14 @@
"dev:firefox": "wxt -b firefox", // build sanity only, no store release
"build": "wxt build",
"zip": "wxt zip",
"check": "svelte-check --tsconfig ./tsconfig.json && tsc --noEmit", // TS 6 API, then the TS 7 tsc (§2)
"check": "svelte-check --tsconfig ./tsconfig.json && tsc --noEmit && tsc -p tsconfig.scripts.json",
// TS 6 API, the TS 7 tsc, then scripts/*.mjs via checkJs
"lint": "biome check .",
"lint:fix": "biome check --write .",
"knip": "knip",
"guard": "node scripts/guard.mjs", // the source gate, 08 §3
"check:manifest": "node scripts/check-manifest.mjs", // the built-artifact gate, 08 §3
"icons": "node scripts/gen-icons.mjs", // regenerates the shipped icons (source of truth)
"test": "vitest run",
"test:watch": "vitest",
"coverage": "vitest run --coverage",
+9 -4
View File
@@ -46,11 +46,16 @@ log gives no warning — verify the output tree, not just the exit code.
bookmarkmagic/
├── CLAUDE.md # AI build instructions (root)
├── wxt.config.ts # srcDir + manifest + svelte module
├── biome.json / knip.json / tsconfig.json / vitest.config.ts
├── .nvmrc # 24
├── scripts/
├── biome.json / knip.json / tsconfig.json / tsconfig.scripts.json / vitest.config.ts
├── sonar-project.properties # SonarCloud CI analysis (12 §2.5)
├── .nvmrc / .npmrc # Node 26; min-release-age=7 (01 §2, §6)
├── scripts/ # all plain .mjs, type-checked via checkJs
│ ├── guard.mjs # the source gate (08 §3), run by npm run guard
│ └── gen-icons.mjs # zero-dep placeholder PNG writer (16/32/48/128)
│ ├── check-manifest.mjs # the built-artifact gate (08 §3), npm run check:manifest
│ ├── gen-icons.mjs # zero-dep PNG writer — the icons' source of truth
│ ├── gen-fixture.mjs # large Netscape files for 11 §5/§6, never committed
│ ├── sanitize-bookmarks.mjs # reporter-facing sanitizer (CLI entry, 09 §6)
│ └── sanitize/ # its format modules: detect, mappers, spans, html, json, csv, leaks
├── docs/ # this suite (00–15)
├── legal/ # EULA.md, DISCLAIMER.md, PRIVACY.md, THIRD_PARTY_NOTICES.md
├── src/
+10 -3
View File
@@ -145,9 +145,16 @@ a **minor-version** event at minimum.
**Sample files from reporters.** A bookmark file is a browsing history, so the
policy asks reporters to run `scripts/sanitize-bookmarks.mjs` first — it rewrites
every hostname, path, query value and title while preserving the structure,
encoding, and timestamp magnitudes a parser bug depends on, so the sanitized file
still reproduces. Anything received by email is kept only as long as needed to
every hostname, path, query value, title, note, keyword and tag while preserving
the structure, encoding, and timestamps a parser bug depends on, so the sanitized
file still reproduces. It reads Netscape HTML, BookmarkMagic JSON, Chrome's
profile `Bookmarks` file, Firefox `.json` backups and our CSV, and refuses
anything else. JSON is rewritten default-deny (every string value except a short
list of structural keys). Before writing, it checks its own output — no host but
its `*.example` aliases, and for HTML no text left that is not a placeholder —
and writes nothing if that check fails. Until 2026-10-28 it accepted any JSON
and rewrote only `url`/`title`, so Chrome's profile file (titles under `name`)
came out with every title intact. Anything received by email is kept only as long as needed to
reproduce and fix, then deleted; `legal/PRIVACY.md` states the same to users.
This is the receiving-side counterpart to §3's rule that error messages never
echo more than a 120-character excerpt of file content.
+1 -1
View File
@@ -4,7 +4,7 @@
| Unit | Soft limit | Hard limit |
|---|---|---|
| File (.ts / .svelte) | **300 lines** | **500 lines** — split before merging |
| File (.ts / .svelte / .mjs) | **300 lines** | **500 lines** — split before merging |
| Function / method | 50 lines | 80 lines |
| Svelte component script block | 150 lines | move logic to a `.svelte.ts` store or `core/` |
| Nesting depth | 3 | 4 |
+6 -1
View File
@@ -92,7 +92,9 @@ There is exactly one path, and every step has an owner in the repo:
`tests/fixtures/** -text`, so EOL normalization cannot destroy the CRLF/BOM
bytes the bug may depend on.
3. A regression test lands in `tests/unit/core/fixtures.test.ts` — the only
fixture-driven suite — **in the same PR** as the parser fix (`11 §2`).
parser regression suite — **in the same PR** as the parser fix (`11 §2`). The
sanitizer suite (`tests/unit/scripts/sanitize-bookmarks.test.ts`) also walks
every fixture automatically, so the new file is covered there too.
4. The quirk is noted in the parser source and in `04 §1.2` (`CLAUDE.md`
working style makes this mandatory, not optional).
5. `11 §2`'s inventory and `CHANGELOG.md [Unreleased]` are updated.
@@ -223,3 +225,6 @@ independently of the package (`13 §5`), though the copy should still go through
| 2026-10-28 | **The zero-network promise becomes browser-enforced.** The production manifest gains an extension-page CSP that keeps the MV3 minimum and adds `connect-src 'none'` plus closed image/font/media/frame/form sources — no `default-src`, so styles are untouched; dev builds omit it so HMR works. Reverses `08 §3`'s old "never add a custom CSP", whose reason was that MV3 forbids *loosening*; tightening was always allowed. The rule now: only ever tighten, and `check:manifest` pins the exact string | 08 §3 |
| 2026-10-28 | **Three gates instead of one grep:** the CSP (runtime), `check:manifest` scanning the shipped JS for network APIs (catches dependency code), and a rewritten `guard.mjs` — whole-file matching, identifiers banned outright, computed global access banned, `.html/.css` scanned, no test exemption, fails on zero files. The old one let all 28 audited bypasses through and printed "clean" for a checkout path with a space or diacritic. Vite's modulepreload polyfill (the one network call in the bundle) is off | 08 §3 |
| 2026-10-28 | **The guard's rule list lives in `scripts/guard.mjs` only.** `08 §3` restating it was the "single source" that had already diverged (`\{@html\}` matched only the literal text) | 08 §3 |
| 2026-10-28 | **The sanitizer reads the browser JSON formats and is default-deny for JSON.** It accepted any JSON and rewrote only `url`/`title`, so Chrome's profile `Bookmarks` file — titles under `name`, plus `meta_info` and a base64 sync blob — came out with every title intact and passed its own host check. JSON dialects are now recognised by shape (anything else refused), every string value is rewritten except structural keys, Firefox root GUIDs and the timestamps survive | 09 §6 |
| 2026-10-28 | **The sanitizer's backstop reads the output's text, not just its hosts.** For HTML it strips markup and refuses on any word that is not a placeholder — independent of the rewriting rules, unlike the old check (and its test), which shared their blind spots. It also stopped overwriting the input when the output path equalled it, checks size before reading, decodes attributes once (CodeQL double-escaping) and finds `<DD>` notes in linear time (Sonar S8786), attributes included | 09 §6 |
| 2026-10-28 | **`scripts/` obeys the size rule and is type-checked.** `docs/10 §1` now names `.mjs`; the 627-line sanitizer is split into `scripts/sanitize/` modules; every script carries `// @ts-check` and `tsconfig.scripts.json` runs them through the TS 7 `tsc` in `npm run check`. The sanitizer stays plain `.mjs` on purpose — reporters run it on whatever Node they have | 10 §1 |
+1 -1
View File
@@ -23,7 +23,7 @@
"dev:firefox": "wxt -b firefox",
"build": "wxt build",
"zip": "wxt zip",
"check": "svelte-check --tsconfig ./tsconfig.json && tsc --noEmit",
"check": "svelte-check --tsconfig ./tsconfig.json && tsc --noEmit && tsc -p tsconfig.scripts.json",
"lint": "biome check .",
"lint:fix": "biome check --write .",
"knip": "knip",
+64 -25
View File
@@ -1,4 +1,5 @@
#!/usr/bin/env node
// @ts-check
/**
* Generate a large Netscape bookmark file on demand.
*
@@ -20,7 +21,10 @@
import { mkdirSync, writeFileSync } from 'node:fs';
import { dirname } from 'node:path';
/** Mulberry32 — small, seeded, and good enough for shaping test data. */
/**
* Mulberry32 — small, seeded, and good enough for shaping test data.
* @param {number} seed
*/
function rng(seed) {
let a = seed >>> 0;
return () => {
@@ -31,17 +35,38 @@ function rng(seed) {
};
}
const USAGE =
'usage: node scripts/gen-fixture.mjs --nodes <n> --out <path> [--per-folder <n>] [--seed <n>]';
/**
* Strict flags: whole numbers >= 1, every flag needs a value, no unknown flags.
* The old reader took arguments in blind pairs, so `--nodes 1.5` wrote 2 nodes,
* `--per-folder abc` silently wrote no bookmarks, and one missing value shifted
* every flag after it.
*
* @param {string[]} argv
* @returns {{ nodes: number, perFolder: number, seed: number, out: string } | string}
* the options, or an error message
*/
function parseArgs(argv) {
const out = { nodes: 10_000, perFolder: 50, seed: 1, out: undefined };
const out = { nodes: 10_000, perFolder: 50, seed: 1, out: '' };
/** @type {Record<string, 'nodes' | 'perFolder' | 'seed'>} */
const numeric = { '--nodes': 'nodes', '--per-folder': 'perFolder', '--seed': 'seed' };
for (let i = 0; i < argv.length; i += 2) {
const key = argv[i];
const flag = argv[i] ?? '';
const value = argv[i + 1];
if (key === '--nodes') out.nodes = Number(value);
else if (key === '--per-folder') out.perFolder = Number(value);
else if (key === '--seed') out.seed = Number(value);
else if (key === '--out') out.out = value;
if (value === undefined || value.startsWith('--')) return `${flag} needs a value`;
if (flag === '--out') {
out.out = value;
continue;
}
const key = numeric[flag];
if (key === undefined) return `unknown flag ${flag}`;
const n = Number(value);
if (!Number.isInteger(n) || n < 1) return `${flag} must be a whole number >= 1, got "${value}"`;
out[key] = n;
}
return out;
return out.out === '' ? '--out is required' : out;
}
/**
@@ -61,6 +86,10 @@ const TITLE_POOL = [
'A rather long bookmark title that wraps in most reasonable window widths',
];
/**
* @param {{ nodes: number, perFolder: number, seed: number }} options
* @returns {string}
*/
function build({ nodes, perFolder, seed }) {
const rand = rng(seed);
const lines = [
@@ -72,14 +101,21 @@ function build({ nodes, perFolder, seed }) {
'<DL><p>',
];
let written = 0;
let folderIndex = 0;
/** @param {number} depth */
const indent = (depth) => ' '.repeat(depth);
// Depth 3 so the walk is exercised, but nowhere near the 200 cap — this file
// is for measuring the common case, not the limit.
const emitFolder = (depth) => {
if (written >= nodes) return;
/**
* Write one folder (and, below depth 3, one nested folder chain) using at
* most `budget` nodes. Returns how many it wrote — always at least 1, so the
* caller's loop provably makes progress.
*
* @param {number} depth
* @param {number} written nodes written before this call
* @param {number} budget nodes this call may write
* @returns {number}
*/
const emitFolder = (depth, written, budget) => {
folderIndex++;
const stamp = 1700000000 + folderIndex;
lines.push(
@@ -88,34 +124,37 @@ function build({ nodes, perFolder, seed }) {
}>Folder ${folderIndex}</H3>`,
`${indent(depth)}<DL><p>`,
);
written++;
let used = 1;
const count = Math.min(perFolder, nodes - written);
const count = Math.min(perFolder, budget - used);
for (let i = 0; i < count; i++) {
const n = written + used;
const title = TITLE_POOL[Math.floor(rand() * TITLE_POOL.length)];
const stampB = 1700000000 + written;
lines.push(
`${indent(depth + 1)}<DT><A HREF="https://host${written % 997}.example/p${written}?q=${written}" ` +
`ADD_DATE="${stampB}">${title}</A>`,
`${indent(depth + 1)}<DT><A HREF="https://host${n % 997}.example/p${n}?q=${n}" ` +
`ADD_DATE="${1700000000 + n}">${title}</A>`,
);
written++;
used++;
}
if (depth < 3 && written < nodes) emitFolder(depth + 1);
// Depth 3 so the walk is exercised, but nowhere near the 200 cap — this
// file is for measuring the common case, not the limit.
if (depth < 3 && used < budget) used += emitFolder(depth + 1, written + used, budget - used);
lines.push(`${indent(depth)}</DL><p>`);
return used;
};
while (written < nodes) emitFolder(1);
let written = 0;
while (written < nodes) written += emitFolder(1, written, nodes - written);
lines.push('</DL><p>', '');
return lines.join('\n');
}
const options = parseArgs(process.argv.slice(2));
if (options.out === undefined || !Number.isFinite(options.nodes) || options.nodes < 1) {
console.error(
'usage: node scripts/gen-fixture.mjs --nodes <n> --out <path> [--per-folder <n>] [--seed <n>]',
);
if (typeof options === 'string') {
console.error(`gen-fixture: ${options}
${USAGE}`);
process.exitCode = 1;
} else {
const html = build(options);
+26 -9
View File
@@ -1,20 +1,26 @@
// @ts-check
/**
* Generate placeholder extension icons — a rounded violet square with a white
* bookmark ribbon — at the four sizes Chrome needs. Zero dependencies: PNG is
* written by hand (zlib deflate + a CRC32 table), so no image library enters
* the tree.
* Generate the extension icons — a rounded violet square with a white bookmark
* ribbon — at the four sizes Chrome needs. Zero dependencies: PNG is written by
* hand (zlib deflate + a CRC32 table), so no image library enters the tree.
*
* These are functional placeholders so the manifest loads from Phase 0.
* Replace with the designed icon before a store release (docs/00 §10 item 3,
* docs/13 §2). Run: `npm run icons`.
* THIS SCRIPT IS THE SOURCE OF TRUTH for the shipped icons: the owner adopted
* its output as final on 2026-07-26 (docs/00 §10 item 3). There is no other
* master file. `npm run icons` overwrites src/public/icon/*.png — review the
* diff before committing, because those PNGs are what the store listing shows.
*/
import { mkdirSync, writeFileSync } from 'node:fs';
import { dirname, join } from 'node:path';
import { fileURLToPath } from 'node:url';
import { deflateSync } from 'node:zlib';
/** @typedef {readonly [number, number, number, number]} Rgba */
/** @type {Rgba} */
const ACCENT = [0x6d, 0x4a, 0xff, 255]; // --accent, docs/06 §1
/** @type {Rgba} */
const WHITE = [255, 255, 255, 255];
/** @type {Rgba} */
const TRANSPARENT = [0, 0, 0, 0];
const CRC_TABLE = (() => {
@@ -27,12 +33,14 @@ const CRC_TABLE = (() => {
return table;
})();
/** @param {Uint8Array} buf */
function crc32(buf) {
let c = 0xffffffff;
for (const byte of buf) c = CRC_TABLE[(c ^ byte) & 0xff] ^ (c >>> 8);
for (const byte of buf) c = (CRC_TABLE[(c ^ byte) & 0xff] ?? 0) ^ (c >>> 8);
return (c ^ 0xffffffff) >>> 0;
}
/** @param {string} type @param {Buffer} data */
function chunk(type, data) {
const body = Buffer.concat([Buffer.from(type, 'ascii'), data]);
const len = Buffer.alloc(4);
@@ -42,6 +50,7 @@ function chunk(type, data) {
return Buffer.concat([len, body, crc]);
}
/** @param {number} x @param {number} y @param {number} size @param {number} radius */
function insideRounded(x, y, size, radius) {
const px = x + 0.5;
const py = y + 0.5;
@@ -50,7 +59,10 @@ function insideRounded(x, y, size, radius) {
return dx * dx + dy * dy <= radius * radius;
}
/** Bookmark ribbon: a tall rectangle with a triangular notch cut from the bottom. */
/**
* Bookmark ribbon: a tall rectangle with a triangular notch cut from the bottom.
* @param {number} x @param {number} y @param {number} size
*/
function insideBookmark(x, y, size) {
const u = (x + 0.5) / size;
const v = (y + 0.5) / size;
@@ -64,12 +76,17 @@ function insideBookmark(x, y, size) {
return true;
}
/**
* @param {number} x @param {number} y @param {number} size
* @returns {Rgba}
*/
function colorAt(x, y, size) {
if (!insideRounded(x, y, size, size * 0.2)) return TRANSPARENT;
if (insideBookmark(x, y, size)) return WHITE;
return ACCENT;
}
/** @param {number} size */
function makePng(size) {
const stride = size * 4;
const raw = Buffer.alloc((stride + 1) * size);
+79 -555
View File
@@ -1,546 +1,51 @@
#!/usr/bin/env node
// @ts-check
/**
* Strip personal data from a bookmark file, keeping the bug.
*
* Purpose: make "attach a sanitized sample" (SECURITY.md, the parser issue
* form) something a reporter can actually do, instead of an instruction they
* have to interpret.
* Inputs: a Netscape HTML, BookmarkMagic JSON, or CSV bookmark file.
* Guarantees: every byte outside a replaced address or title is untouched —
* including the BOM, the doctype, line endings, tag and attribute case,
* quoting style, indentation, and every timestamp. Replacements are stable
* per distinct input value, so "how many distinct hosts" and "which two
* bookmarks are duplicates" survive.
* Inputs: a Netscape HTML export, BookmarkMagic JSON, Chrome's profile
* `Bookmarks` file (also Edge/Brave/Vivaldi), a Firefox `.json` backup, or a
* docs/04 §3 CSV.
* Guarantees: every byte outside a replaced address, title or note is untouched
* — the BOM, the doctype, line endings, tag and attribute case, quoting style,
* indentation, JSON key order and escapes, and every timestamp. Replacements
* are stable per distinct input value, so "how many distinct hosts" and
* "which two bookmarks are duplicates" survive. When the output still looks
* personal, nothing is written at all.
*
* Zero dependencies, Node stdlib only, no network. It has to run from a fresh
* clone with nothing installed, which is also why it re-implements a three-line
* format sniff rather than importing `src/lib/core/detect-format.ts`.
* Zero dependencies, Node stdlib only, no network, plain JavaScript: it has to
* run from a fresh clone with nothing installed, on whatever Node the reporter
* has. The format modules live in scripts/sanitize/.
*
* Usage:
* node scripts/sanitize-bookmarks.mjs <input> [output]
* node scripts/sanitize-bookmarks.mjs <input> --stdout
* node scripts/sanitize-bookmarks.mjs <input> --check # report, write nothing
*/
import { readFileSync, writeFileSync } from 'node:fs';
import { basename, extname } from 'node:path';
import { readFileSync, statSync, writeFileSync } from 'node:fs';
import { basename, extname, resolve } from 'node:path';
import { fileURLToPath } from 'node:url';
import { sanitizeCsv } from './sanitize/csv.mjs';
import { detectKind } from './sanitize/detect.mjs';
import { sanitizeHtml } from './sanitize/html.mjs';
import { sanitizeJson } from './sanitize/json.mjs';
import { findLeakedHosts, findLeakedWords } from './sanitize/leaks.mjs';
import { makeTitleMapper, makeUrlMapper } from './sanitize/mappers.mjs';
export { detectKind } from './sanitize/detect.mjs';
export { findLeakedHosts, findLeakedWords } from './sanitize/leaks.mjs';
/** Matches docs/09's parse cap. A 25 MB attachment helps nobody anyway. */
const MAX_BYTES = 25 * 1024 * 1024;
/** 1×1 transparent PNG. Keeps ICON= present, which is what the parser reacts to. */
const BLANK_ICON =
'data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAAC0lEQVR42mNkYAAAAAYAAjCB0C8AAAAASUVORK5CYII=';
// ---------------------------------------------------------------- format sniff
/** The docs/04 §3 CSV header, in order. A file is CSV only if it says so. */
const CSV_HEADER = ['folder_path', 'title', 'url', 'add_date'];
/**
* Identify the format, or refuse.
*
* @param {string} text
* @returns {'html' | 'json' | 'csv' | null} `null` when nothing matched.
*
* Returning `null` rather than guessing is load-bearing. Every sanitizer below
* finds what it must destroy *by format*, so a wrong guess destroys nothing and
* emits the input verbatim — and the file would still be handed to the user as
* "sanitized". The old `return 'csv'` fallback did exactly that for a truncated
* JSON export, an HTML excerpt with no doctype, or a CSV with foreign headers:
* byte-identical output, exit code 0, real hostnames intact.
*
* The people who run this tool are, by definition, the ones whose file did not
* import — so a malformed or hand-trimmed file is the *typical* input here, not
* an edge case. `src/lib/core/detect-format.ts` refuses these same inputs with
* `UNKNOWN_FORMAT`; this must not be weaker than the parser it fronts.
*/
export function detectKind(text) {
const body = text.charCodeAt(0) === 0xfeff ? text.slice(1) : text;
if (/NETSCAPE-Bookmark-file-1/i.test(body.slice(0, 512))) return 'html';
const trimmed = body.trimStart();
if (trimmed.startsWith('{')) {
try {
JSON.parse(trimmed);
return 'json';
} catch {
// Not JSON after all; fall through.
}
}
const firstLine = body.split(/\r?\n/, 1)[0] ?? '';
for (const delimiter of [',', ';']) {
const columns = firstLine
.split(delimiter)
.map((h) => h.trim().replace(/^"|"$/g, '').toLowerCase());
if (columns.length === CSV_HEADER.length && CSV_HEADER.every((c, i) => columns[i] === c)) {
return 'csv';
}
}
if (/<\s*dl[\s>]/i.test(body)) return 'html';
return null;
}
// ------------------------------------------------------------------- mappers
/**
* Replace addresses while preserving everything a parser branches on: the
* scheme, whether there is a port, how deep the path is, whether it ends in a
* slash, how many query parameters there are and in what order, and whether
* there is a fragment. Percent-escapes are carried across literally, so a
* decoding bug still reproduces.
*/
function makeUrlMapper() {
const seen = new Map();
const hosts = new Map();
const hostAlias = (host) => {
let alias = hosts.get(host);
if (alias === undefined) {
const n = hosts.size;
// a.example … z.example, then a1.example, a2.example …
const letter = String.fromCharCode(97 + (n % 26));
const round = Math.floor(n / 26);
alias = round === 0 ? `${letter}.example` : `${letter}${round}.example`;
hosts.set(host, alias);
}
return alias;
};
const escapesIn = (segment) => (segment.match(/%[0-9a-fA-F]{2}/g) ?? []).join('');
const mapPath = (pathname) => {
if (pathname === '' || pathname === '/') return pathname;
const trailing = pathname.endsWith('/');
const parts = pathname.replace(/^\//, '').replace(/\/$/, '').split('/');
const mapped = parts.map((p, i) => (p === '' ? '' : `p${i + 1}${escapesIn(p)}`));
return `/${mapped.join('/')}${trailing ? '/' : ''}`;
};
const mapQuery = (search) => {
if (search === '') return '';
return `?${search
.slice(1)
.split('&')
.map((pair, i) => (pair.includes('=') ? `q${i + 1}=v${i + 1}` : `q${i + 1}`))
.join('&')}`;
};
return (raw) => {
if (raw === '') return raw;
const cached = seen.get(raw);
if (cached !== undefined) return cached;
let out;
let url;
try {
url = new URL(raw);
} catch {
url = undefined;
}
if (url === undefined) {
// Keep it unparseable, and keep distinct values distinct — the dedupe
// path treats an unparseable URL as its own trimmed self.
out = `not-a-url-${seen.size + 1}`;
} else if (url.host === '') {
// Opaque scheme (javascript:, data:, mailto:). The scheme is the whole
// point for the T1/T3 threat cases; the payload is not ours to keep.
out = `${url.protocol}sanitized-${seen.size + 1}`;
} else {
// Credentials are never round-tripped, not even as placeholders.
const port = url.port === '' ? '' : `:${url.port}`;
const hash = url.hash === '' ? '' : '#frag';
out = `${url.protocol}//${hostAlias(url.host.replace(/:\d+$/, ''))}${port}${mapPath(url.pathname)}${mapQuery(url.search)}${hash}`;
}
seen.set(raw, out);
return out;
};
}
/**
* Replace titles while preserving what encoding bugs live in: emptiness, the
* scripts present (so a CJK or RTL or emoji bug still reproduces), and any HTML
* entity references, which are carried across verbatim.
*/
function makeTitleMapper() {
const seen = new Map();
const SCRIPTS = [
[/[一-鿿぀-ヿ]/, '文字'],
[/[֐-ࣿ]/, 'شيء'],
[/[\p{Extended_Pictographic}]/u, '🌸'],
];
return (raw) => {
if (raw.trim() === '') return raw;
const cached = seen.get(raw);
if (cached !== undefined) return cached;
const entities = (raw.match(/&(?:#\d+|#x[0-9a-fA-F]+|[a-zA-Z]+);/g) ?? []).join('');
const extras = SCRIPTS.filter(([re]) => re.test(raw))
.map(([, sample]) => sample)
.join('');
const out = `Title ${seen.size + 1}${extras === '' ? '' : ` ${extras}`}${entities}`;
seen.set(raw, out);
return out;
};
}
// ------------------------------------------------------------ span rewriting
/**
* Apply replacements by character span, right to left.
*
* Everything this tool does is expressed as spans so that the bytes between
* them are provably untouched — that is the property the whole exercise rests
* on, and rebuilding the file from a parsed model would quietly lose it.
*
* @param {string} text
* @param {{start: number, end: number, value: string}[]} spans
*/
function replaceSpans(text, spans) {
// Overlaps are dropped, outermost first. Two patterns can legitimately claim
// the same bytes — an <A> title containing `x="http://…"` matches both the
// title rule and the generic attribute sweep — and applying both would splice
// already-rewritten text using an `end` measured against the original,
// silently eating or stranding bytes. The widest span sanitizes the most.
const ordered = [...spans].sort((a, b) => a.start - b.start || b.end - a.end);
const out = [];
let cursor = 0;
for (const span of ordered) {
if (span.start < cursor) continue;
out.push(text.slice(cursor, span.start), span.value);
cursor = span.end;
}
out.push(text.slice(cursor));
// Joined once rather than spliced per span: the old loop rebuilt the whole
// string for every replacement, which is O(spans x file) and takes minutes on
// a large export.
return out.join('');
}
/**
* Collect spans for one capture group of a global regex. `map` receives the
* captured text.
*
* The group's offset is computed by re-scanning the match, so the regex must
* capture a span that appears only once within it — true for every pattern
* here, since each captures an attribute value or an element's text.
*
* @param {string} text
* @param {RegExp} re
* @param {(captured: string) => string} map
* @param {number} [group]
*/
function spansFrom(text, re, map, group = 1) {
// The `d` flag gives exact per-group offsets. The previous version searched
// the match text for the captured substring, which lands on the wrong copy
// whenever the capture also appears earlier in the match — e.g. a bookmark
// whose title equals part of its own URL.
const indexed = new RegExp(re.source, re.flags.includes('d') ? re.flags : `${re.flags}d`);
const spans = [];
for (const m of text.matchAll(indexed)) {
const captured = m[group] ?? '';
const at = m.indices?.[group];
if (captured === '' || at === undefined) continue;
spans.push({ start: at[0], end: at[1], value: map(captured) });
}
return spans;
}
/**
* The inside of an opening tag, tolerating `>` within quoted attribute values.
*
* `data:text/html,<h1>hi</h1>` is a legal HREF and appears in the T1/T3
* fixtures. A naive `[^>]*` stops inside it, silently leaving that bookmark's
* title unsanitized — which is the one failure mode this tool must not have.
*/
const TAG_INNARDS = '(?:"[^"]*"|\'[^\']*\'|[^>"\'])*';
// ------------------------------------------------------------------ formats
/**
* Decode an HTML attribute value, then re-encode the result the way the source
* file encoded it.
*
* Netscape exports write `&amp;` inside HREF, but not all of them do, and
* whether a parser unescapes attribute values is itself something people report
* bugs about. Emitting a bare `&` where the source had `&amp;` would silently
* repair the file being reported.
*/
function attrRewriter(map) {
return (captured) => {
const decoded = captured
.replace(/&amp;/gi, '&')
.replace(/&lt;/gi, '<')
.replace(/&gt;/gi, '>')
.replace(/&quot;/gi, '"');
const mapped = map(decoded);
const escapesAmp = /&amp;/i.test(captured) || !captured.includes('&');
return escapesAmp ? mapped.replace(/&/g, '&amp;') : mapped;
};
}
/**
* `name = value`, where the value is double-quoted, single-quoted or bare.
*
* Group 2/3/4 is the value, whichever quoting was used. Matching only `"…"`
* left `HREF='https://…'` completely untouched — real exports in the wild use
* single quotes, and that one is a straight leak.
*/
const ANY_ATTR = /\b([a-z_][a-z0-9_-]*)\s*=\s*(?:"([^"]*)"|'([^']*)'|([^\s"'<>=`]+))/dgi;
/**
* Spans for every attribute whose name matches, at any quoting.
*
* @param {string} text
* @param {RegExp} nameRe tested against the lower-cased attribute name
* @param {(value: string) => string} map
*/
function attrSpans(text, nameRe, map) {
const spans = [];
for (const m of text.matchAll(ANY_ATTR)) {
if (!nameRe.test((m[1] ?? '').toLowerCase())) continue;
const at = m.indices?.[2] ?? m.indices?.[3] ?? m.indices?.[4];
const value = m[2] ?? m[3] ?? m[4] ?? '';
if (at === undefined || value === '') continue;
spans.push({ start: at[0], end: at[1], value: map(value) });
}
return spans;
}
/** Attributes handled explicitly; the generic URL sweep must not touch them. */
const HANDLED_ATTRS = new Set(['href', 'icon']);
/**
* Scrub any *other* attribute whose value is a URL with a host.
*
* Firefox alone writes `ICON_URI`, and `FEEDURL` and friends exist in the wild.
* Enumerating them was already wrong once — `ICON_URI` leaked a real hostname
* through a file that had passed every other check — so this sweeps by value
* shape instead, and covers attributes nobody has thought of yet.
*/
function urlAttrSpans(text, mapUrl) {
const spans = [];
const rewrite = attrRewriter(mapUrl);
for (const m of text.matchAll(ANY_ATTR)) {
const name = (m[1] ?? '').toLowerCase();
const value = m[2] ?? m[3] ?? m[4] ?? '';
if (HANDLED_ATTRS.has(name) || value === '') continue;
try {
if (new URL(value).host === '') continue;
} catch {
continue;
}
const at = m.indices?.[2] ?? m.indices?.[3] ?? m.indices?.[4];
if (at === undefined) continue;
spans.push({ start: at[0], end: at[1], value: rewrite(value) });
}
return spans;
}
function sanitizeHtml(text, mapUrl, mapTitle) {
let descriptions = 0;
let keywords = 0;
const spans = [
// HREF and ICON, at any quoting — attribute name case is left as found.
...attrSpans(text, /^href$/, attrRewriter(mapUrl)),
...attrSpans(text, /^icon$/, () => BLANK_ICON),
...urlAttrSpans(text, mapUrl),
// Firefox keywords and tags are user-authored text, so they are personal
// too. Tag count is preserved because it is the only part a parser sees.
...attrSpans(text, /^shortcuturl$/, () => `kw${++keywords}`),
...attrSpans(text, /^tags$/, (v) =>
v
.split(',')
.map((_, i) => `tag${i + 1}`)
.join(','),
),
// Bookmark titles: the text between <A …> and </A>.
// `[\s\S]*?` and not `[^<]*`: a title containing a raw `<` made the whole
// match fail, so that bookmark's title was left in the output untouched.
...spansFrom(text, new RegExp(`<a\\b${TAG_INNARDS}>([\\s\\S]*?)</a>`, 'gi'), mapTitle),
// Folder titles.
...spansFrom(text, new RegExp(`<h3\\b${TAG_INNARDS}>([\\s\\S]*?)</h3>`, 'gi'), mapTitle),
// Folder/bookmark descriptions.
// Runs to the next element, not to the end of the line: a description that
// wrapped onto a second line left everything after the first newline in
// place, and a `<DD>` is exactly where someone writes a private note.
...spansFrom(
text,
// The `\s*` belongs in the lookahead, not the capture: leaving it inside
// eats the newline and indentation before the next tag, which is exactly
// the byte-level shape this tool promises not to touch.
/<dd>([\s\S]*?)(?=\s*<\s*(?:dt|dd|dl|\/dl)\b|\s*$)/gi,
() => `Description ${++descriptions}`,
),
];
return replaceSpans(text, spans);
}
/**
* Rewrite only the *values* of `url` and `title` string properties, by walking
* the raw text. Parsing and re-stringifying would normalize the indentation and
* line endings, which is exactly what a whitespace-sensitive report needs kept.
*/
function sanitizeJson(text, mapUrl, mapTitle) {
const spans = [];
const re = /"(url|title)"\s*:\s*"((?:[^"\\]|\\.)*)"/g;
for (const m of text.matchAll(re)) {
const key = m[1];
const rawValue = m[2] ?? '';
const start = m.index + m[0].length - 1 - rawValue.length;
let decoded;
try {
decoded = JSON.parse(`"${rawValue}"`);
} catch {
decoded = rawValue;
}
const mapped = key === 'url' ? mapUrl(decoded) : mapTitle(decoded);
spans.push({ start, end: start + rawValue.length, value: JSON.stringify(mapped).slice(1, -1) });
}
return replaceSpans(text, spans);
}
/**
* Walk the CSV with a minimal RFC 4180 reader that records the span of each
* field's *content* (inside the quotes, when quoted). Replacements never
* contain a delimiter, quote or newline, so the original quoting survives
* untouched — which matters, because CSV quoting is itself a common bug.
*/
function sanitizeCsv(text, mapUrl, mapTitle) {
const delimiter = (text.split('\n', 1)[0] ?? '').includes(';') ? ';' : ',';
const fields = [];
let row = 0;
let col = 0;
let i = 0;
while (i < text.length) {
const quoted = text[i] === '"';
if (quoted) i++;
const start = i;
while (i < text.length) {
const ch = text[i];
if (quoted) {
if (ch === '"') {
if (text[i + 1] === '"') {
i += 2;
continue;
}
break;
}
i++;
continue;
}
if (ch === delimiter || ch === '\n' || ch === '\r') break;
i++;
}
fields.push({ row, col, start, end: i, quoted });
if (quoted && text[i] === '"') i++;
if (text[i] === delimiter) {
col++;
i++;
} else {
if (text[i] === '\r') i++;
if (text[i] === '\n') i++;
row++;
col = 0;
}
}
// Column names per docs/04 §3 — `folder_path`, not `folder`.
const header = fields.filter((f) => f.row === 0).map((f) => text.slice(f.start, f.end).trim());
const at = (name) => header.findIndex((h) => h.toLowerCase().replace(/^/, '') === name);
const [folderCol, titleCol, urlCol] = [at('folder_path'), at('title'), at('url')];
const spans = [];
for (const f of fields) {
if (f.row === 0 || f.start === f.end) continue;
const raw = text.slice(f.start, f.end);
const value = raw.replace(/""/g, '"');
let mapped;
if (f.col === urlCol) mapped = mapUrl(value);
else if (f.col === titleCol) mapped = mapTitle(value);
else if (f.col === folderCol) mapped = mapFolderPath(value, mapTitle);
else continue;
spans.push({ start: f.start, end: f.end, value: mapped.replace(/"/g, '""') });
}
return replaceSpans(text, spans);
}
/**
* Map each path segment separately so `\/` and `\\` escapes keep their meaning.
*
* A segment that contained a literal slash or backslash keeps one, so the
* escape survives into the output. Without that, `Escaped\/slash` sanitizes to
* a plain name and the escape branch of the CSV path parser — a real reason
* someone would file a report — stops being exercised by the sample.
*/
function mapFolderPath(path, mapTitle) {
const segments = [];
let current = '';
for (let i = 0; i < path.length; i++) {
const ch = path[i];
if (ch === '\\' && (path[i + 1] === '/' || path[i + 1] === '\\')) {
current += path[i + 1];
i++;
continue;
}
if (ch === '/') {
segments.push(current);
current = '';
continue;
}
current += ch;
}
segments.push(current);
return segments
.map((s) => {
if (s === '') return '';
const kept = `${s.includes('/') ? '/' : ''}${s.includes('\\') ? '\\' : ''}`;
return `${mapTitle(s)}${kept}`.replace(/\\/g, '\\\\').replace(/\//g, '\\/');
})
.join('/');
}
// -------------------------------------------------------------- entry points
/**
* Everything in the OUTPUT that still looks like a real address.
*
* This is the backstop, and it is the only reason the rules above can be
* trusted. They are regexes over untrusted markup, so each one has a shape it
* does not match — a single-quoted attribute, a title containing `<`, a JSON
* key nobody enumerated, a `<DD>` that wraps onto a second line. Every one of
* those is a silent leak, and the only way to find them all by inspection is to
* be sure you have thought of every one, which nobody is.
*
* So instead of trusting the rules, the output is checked: any host that is not
* one of our own `*.example` aliases means something got through, and the file
* is refused rather than handed over. A rule this misses degrades into a
* refusal — annoying — instead of into a published browsing history.
*
* @param {string} text sanitized output
* @returns {string[]} distinct suspicious hosts, empty when clean
*/
export function findLeakedHosts(text) {
const leaked = new Set();
for (const m of text.matchAll(/\b[a-z][a-z0-9+.-]*:\/\/([^\s"'<>)\]}\\]+)/gi)) {
const authority = (m[1] ?? '').split(/[/?#]/, 1)[0] ?? '';
// Strip credentials and port; what remains is the host.
const host = authority.split('@').pop()?.replace(/:\d+$/, '').toLowerCase() ?? '';
if (host === '' || /^[a-z]\d*\.example$/.test(host)) continue;
leaked.add(host);
}
return [...leaked];
}
/**
* Sanitize a bookmark file's text.
*
* @param {string} text
* @returns {{ text: string | null, kind: 'html' | 'json' | 'csv' | null, leaked: string[] }}
* @returns {{ text: string | null, kind: import('./sanitize/detect.mjs').Kind | null, leaked: string[] }}
* `text` is null when the format was not recognised, or when the output
* failed its own leak check. Both mean: write nothing.
*/
@@ -550,17 +55,44 @@ export function sanitize(text) {
const mapUrl = makeUrlMapper();
const mapTitle = makeTitleMapper();
const out =
kind === 'html'
? sanitizeHtml(text, mapUrl, mapTitle)
: kind === 'json'
? sanitizeJson(text, mapUrl, mapTitle)
: sanitizeCsv(text, mapUrl, mapTitle);
let out;
if (kind === 'html') out = sanitizeHtml(text, mapUrl, mapTitle);
else if (kind === 'csv') out = sanitizeCsv(text, mapUrl, mapTitle);
else out = sanitizeJson(text, mapUrl, mapTitle);
const leaked = findLeakedHosts(out);
const leaked = [
...findLeakedHosts(out),
...(kind === 'html' ? findLeakedWords(out).map((word) => `text "${word}"`) : []),
];
return { text: leaked.length > 0 ? null : out, kind, leaked };
}
/** @param {string} input */
function refuseUnknown(input) {
console.error(
`${basename(input)} is not a bookmark file this tool recognises, so NOTHING WAS WRITTEN.\n` +
' Expected one of: a Netscape HTML export (the usual bookmarks.html), a BookmarkMagic\n' +
' JSON export, Chrome\'s profile "Bookmarks" file, a Firefox bookmarks-*.json backup,\n' +
' or a CSV whose header is exactly "folder_path,title,url,add_date".\n' +
' Refusing to hand you a file it cannot promise is clean. Please attach the export\n' +
' exactly as your browser wrote it, or describe the problem in the issue instead.',
);
return 1;
}
/** @param {string} input @param {string[]} leaked */
function refuseLeak(input, leaked) {
console.error(
`${basename(input)}: NOTHING WAS WRITTEN — the result still contained ${leaked.length} item(s) ` +
`that look personal, so this file is not safe to attach.\n Found: ${leaked.slice(0, 5).join(', ')}` +
`${leaked.length > 5 ? `, and ${leaked.length - 5} more` : ''}\n` +
' This is a bug in the sanitizer, not in your file. Please report it (without the file)\n' +
' at https://github.com/poli0981/bookmarkmagic/issues — the list above is enough.',
);
return 1;
}
/** @param {string[]} argv */
function main(argv) {
const flags = argv.filter((a) => a.startsWith('--'));
const [input, maybeOutput] = argv.filter((a) => !a.startsWith('--'));
@@ -570,44 +102,28 @@ function main(argv) {
return 1;
}
const raw = readFileSync(input);
if (raw.byteLength > MAX_BYTES) {
// Size first, from the file system: reading a 2 GB file to learn it is too
// big is how the old version ran out of memory before its own check.
const size = statSync(input).size;
if (size > MAX_BYTES) {
console.error(
`${input} is ${(raw.byteLength / 1024 / 1024).toFixed(1)} MB. The extension itself refuses ` +
`${input} is ${(size / 1024 / 1024).toFixed(1)} MB. The extension itself refuses ` +
'anything over 25 MB, and a smaller file that still reproduces the bug is more useful. ' +
'Please trim it first.',
);
return 1;
}
const text = raw.toString('utf8');
const text = readFileSync(input).toString('utf8');
const { text: out, kind, leaked } = sanitize(text);
if (kind === null) {
console.error(
`${basename(input)} is not a bookmark file this tool recognises, so NOTHING WAS WRITTEN.\n` +
' Expected: a Netscape HTML export (the usual bookmarks.html), a BookmarkMagic JSON\n' +
' export, or a CSV whose header is exactly "folder_path,title,url,add_date".\n' +
' Refusing to hand you a file it cannot promise is clean. Please attach the export\n' +
' exactly as your browser wrote it, or describe the problem in the issue instead.',
);
return 1;
}
if (out === null) {
console.error(
`${basename(input)}: NOTHING WAS WRITTEN — the result still contained ${leaked.length} real ` +
`address(es), so this file is not safe to attach.\n Leaked: ${leaked.slice(0, 5).join(', ')}` +
`${leaked.length > 5 ? `, and ${leaked.length - 5} more` : ''}\n` +
' This is a bug in the sanitizer, not in your file. Please report it (without the file)\n' +
' at https://github.com/poli0981/bookmarkmagic/issues — the address above is enough.',
);
return 1;
}
if (kind === null) return refuseUnknown(input);
if (out === null) return refuseLeak(input, leaked);
const eol = text.includes('\r\n') ? 'CRLF' : 'LF';
const bom = text.charCodeAt(0) === 0xfeff ? ', BOM' : '';
console.error(`${basename(input)}: ${kind}, ${eol}${bom} — ${out.length} bytes out, no leaks`);
console.error(
`${basename(input)}: ${kind}, ${eol}${bom} — ${Buffer.byteLength(out)} bytes out, no leaks`,
);
if (flags.includes('--check')) return 0;
if (flags.includes('--stdout')) {
@@ -617,11 +133,19 @@ function main(argv) {
const ext = extname(input);
const target = maybeOutput ?? `${input.slice(0, input.length - ext.length)}.sanitized${ext}`;
// Never over the original: an output path equal to the input used to replace
// the user's only copy of their real bookmarks with the placeholder version.
if (resolve(target) === resolve(input)) {
console.error(
`refusing to overwrite ${input} with its sanitized copy — pick another output path.`,
);
return 1;
}
writeFileSync(target, out);
console.error(`wrote ${target}`);
return 0;
}
if (process.argv[1] !== undefined && process.argv[1] === fileURLToPath(import.meta.url)) {
if (process.argv[1] !== undefined && resolve(process.argv[1]) === fileURLToPath(import.meta.url)) {
process.exitCode = main(process.argv.slice(2));
}
+118
View File
@@ -0,0 +1,118 @@
// @ts-check
/**
* CSV (docs/04 §3): a minimal RFC 4180 reader that records the span of each
* field's *content* (inside the quotes, when quoted). Replacements never
* contain a delimiter, quote or newline, so the original quoting survives —
* which matters, because CSV quoting is itself a common bug.
*/
import { replaceSpans } from './spans.mjs';
/** @typedef {import('./spans.mjs').Span} Span */
/** @typedef {import('./mappers.mjs').Mapper} Mapper */
/** @typedef {{ row: number, col: number, start: number, end: number }} Field */
/**
* @param {string} text
* @param {string} delimiter
* @returns {Field[]}
*/
function readFields(text, delimiter) {
/** @type {Field[]} */
const fields = [];
let row = 0;
let col = 0;
let i = 0;
while (i < text.length) {
const quoted = text[i] === '"';
if (quoted) i++;
const start = i;
while (i < text.length) {
const ch = text[i];
if (quoted) {
if (ch === '"' && text[i + 1] === '"') i += 2;
else if (ch === '"') break;
else i++;
} else if (ch === delimiter || ch === '\n' || ch === '\r') {
break;
} else {
i++;
}
}
fields.push({ row, col, start, end: i });
if (quoted && text[i] === '"') i++;
if (text[i] === delimiter) {
col++;
i++;
} else {
if (text[i] === '\r') i++;
if (text[i] === '\n') i++;
row++;
col = 0;
}
}
return fields;
}
/**
* Map each path segment separately so `\/` and `\\` escapes keep their meaning.
* A segment that contained a literal slash or backslash keeps one, so the
* escape branch of the CSV path parser — a real reason someone files a report —
* is still exercised by the sample.
*
* @param {string} path
* @param {Mapper} mapTitle
*/
function mapFolderPath(path, mapTitle) {
const segments = [];
let current = '';
for (let i = 0; i < path.length; i++) {
const ch = path[i];
if (ch === '\\' && (path[i + 1] === '/' || path[i + 1] === '\\')) {
current += path[i + 1];
i++;
} else if (ch === '/') {
segments.push(current);
current = '';
} else {
current += ch;
}
}
segments.push(current);
return segments
.map((s) => {
if (s === '') return '';
const kept = `${s.includes('/') ? '/' : ''}${s.includes('\\') ? '\\' : ''}`;
return `${mapTitle(s)}${kept}`.replace(/\\/g, '\\\\').replace(/\//g, '\\/');
})
.join('/');
}
/**
* @param {string} text
* @param {Mapper} mapUrl
* @param {Mapper} mapTitle
*/
export function sanitizeCsv(text, mapUrl, mapTitle) {
const delimiter = (text.split('\n', 1)[0] ?? '').includes(';') ? ';' : ',';
const fields = readFields(text, delimiter);
// Column names per docs/04 §3 — `folder_path`, not `folder`.
const header = fields.filter((f) => f.row === 0).map((f) => text.slice(f.start, f.end).trim());
/** @param {string} name */
const at = (name) => header.findIndex((h) => h.toLowerCase().replace(/^/, '') === name);
const [folderCol, titleCol, urlCol] = [at('folder_path'), at('title'), at('url')];
/** @type {Span[]} */
const spans = [];
for (const f of fields) {
if (f.row === 0 || f.start === f.end) continue;
const value = text.slice(f.start, f.end).replace(/""/g, '"');
let mapped;
if (f.col === urlCol) mapped = mapUrl(value);
else if (f.col === titleCol) mapped = mapTitle(value);
else if (f.col === folderCol) mapped = mapFolderPath(value, mapTitle);
else continue;
spans.push({ start: f.start, end: f.end, value: mapped.replace(/"/g, '""') });
}
return replaceSpans(text, spans);
}
+80
View File
@@ -0,0 +1,80 @@
// @ts-check
/**
* Format sniffing for the sanitizer — refuse anything not positively known.
*
* Returning `null` rather than guessing is load-bearing. Every sanitizer finds
* what it must destroy *by format*, so a wrong guess destroys nothing and emits
* the input verbatim — and the file would still be handed over as "sanitized".
* The old `return 'csv'` fallback did exactly that for a truncated JSON export.
*
* The same goes for JSON: the old check accepted ANY object, and the JSON
* rewriter only knew BookmarkMagic's `title`/`url` keys — so Chrome's profile
* `Bookmarks` file, which keeps titles under `name`, "sanitized" with every
* title intact and passed the host check. Each JSON dialect is now recognised
* by shape, and anything else is refused.
*
* Re-implemented rather than imported from `src/lib/core/detect-format.ts`:
* this script must run from a fresh clone with nothing installed.
*/
/** The docs/04 §3 CSV header, in order. A file is CSV only if it says so. */
const CSV_HEADER = ['folder_path', 'title', 'url', 'add_date'];
/** @typedef {'html' | 'json' | 'chrome-json' | 'firefox-json' | 'csv'} Kind */
/**
* @param {unknown} doc
* @returns {'json' | 'chrome-json' | 'firefox-json' | null}
*/
export function jsonDialect(doc) {
if (typeof doc !== 'object' || doc === null || Array.isArray(doc)) return null;
const record = /** @type {Record<string, unknown>} */ (doc);
if (record.format === 'bookmarkmagic') return 'json';
const roots = record.roots;
if (
typeof roots === 'object' &&
roots !== null &&
!Array.isArray(roots) &&
['bookmark_bar', 'other', 'synced'].some((key) => key in roots)
) {
return 'chrome-json';
}
const isPlacesRoot = record.type === 'text/x-moz-place-container' || record.root === 'placesRoot';
if (isPlacesRoot && Array.isArray(record.children)) return 'firefox-json';
return null;
}
/**
* Identify the format, or refuse with `null`.
*
* The people who run this tool are, by definition, the ones whose file did not
* import — so a malformed or hand-trimmed file is the typical input here.
*
* @param {string} text
* @returns {Kind | null}
*/
export function detectKind(text) {
const body = text.charCodeAt(0) === 0xfeff ? text.slice(1) : text;
const trimmed = body.trimStart();
// JSON first: a JSON title may legitimately contain the Netscape doctype
// string, and nothing that starts with `{` is HTML or CSV.
if (trimmed.startsWith('{')) {
try {
return jsonDialect(JSON.parse(trimmed));
} catch {
return null;
}
}
if (/NETSCAPE-Bookmark-file-1/i.test(body.slice(0, 512))) return 'html';
const firstLine = body.split(/\r?\n/, 1)[0] ?? '';
for (const delimiter of [',', ';']) {
const columns = firstLine
.split(delimiter)
.map((h) => h.trim().replace(/^"|"$/g, '').toLowerCase());
if (columns.length === CSV_HEADER.length && CSV_HEADER.every((c, i) => columns[i] === c)) {
return 'csv';
}
}
if (/<\s*dl[\s>]/i.test(body)) return 'html';
return null;
}
+121
View File
@@ -0,0 +1,121 @@
// @ts-check
/**
* Netscape HTML: rewrite addresses, titles, descriptions and user-authored
* attributes by span, leaving every other byte exactly as the browser wrote it.
*/
import { isAddressLike } from './mappers.mjs';
import {
ANY_ATTR,
attrRewriter,
attrSpans,
replaceSpans,
spansFrom,
TAG_INNARDS,
} from './spans.mjs';
/** @typedef {import('./spans.mjs').Span} Span */
/** @typedef {import('./mappers.mjs').Mapper} Mapper */
/** 1×1 transparent PNG. Keeps ICON= present, which is what the parser reacts to. */
const BLANK_ICON =
'data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAAC0lEQVR42mNkYAAAAAYAAjCB0C8AAAAASUVORK5CYII=';
/** Attributes handled explicitly; the generic URL sweep must not touch them. */
const HANDLED_ATTRS = new Set(['href', 'icon']);
/**
* Scrub any *other* attribute whose value is an address. Firefox alone writes
* `ICON_URI`, and `FEEDURL` and friends exist in the wild; enumerating them was
* already wrong once, so this sweeps by value shape. `file:` counts: an
* `ICON_URI="file:///C:/Users/<name>/…"` has no host, and used to pass.
*
* @param {string} text
* @param {Mapper} mapUrl
* @returns {Span[]}
*/
function urlAttrSpans(text, mapUrl) {
/** @type {Span[]} */
const spans = [];
const rewrite = attrRewriter(mapUrl);
for (const m of text.matchAll(ANY_ATTR)) {
const name = (m[1] ?? '').toLowerCase();
const value = m[2] ?? m[3] ?? m[4] ?? '';
if (HANDLED_ATTRS.has(name) || value === '' || !isAddressLike(value)) continue;
const at = m.indices?.[2] ?? m.indices?.[3] ?? m.indices?.[4];
if (at !== undefined) spans.push({ start: at[0], end: at[1], value: rewrite(value) });
}
return spans;
}
/** The next element that ends a description: `<DT>`, `<DD>`, `<DL>` or a close. */
const DESCRIPTION_END = /<\s*\/?\s*(?:dt|dd|dl)\b/gi;
/**
* `<DD>` descriptions, found in one linear pass.
*
* The description runs to the next `<DT>/<DD>/<DL>` (or a closing one), not to
* the end of the line — a wrapped description used to leak its second line —
* and trailing whitespace stays outside the span, because the newline and
* indentation before the next tag are byte-level shape this tool keeps.
*
* Replaces `/<dd>([\s\S]*?)(?=\s*<…>|\s*$)/`, which backtracked polynomially
* (SonarCloud S8786) and matched only a bare `<dd>` — `<DD class=n>Private
* note` passed through untouched.
*
* @param {string} text
* @param {() => string} next
* @returns {Span[]}
*/
function descriptionSpans(text, next) {
/** @type {Span[]} */
const spans = [];
for (const open of text.matchAll(/<dd\b[^>]*>/gi)) {
const start = (open.index ?? 0) + open[0].length;
DESCRIPTION_END.lastIndex = start;
const close = DESCRIPTION_END.exec(text);
let end = close === null ? text.length : close.index;
while (end > start && /\s/.test(text.charAt(end - 1))) end--;
if (end > start) spans.push({ start, end, value: next() });
}
return spans;
}
/** A document heading that says nothing about its owner stays as it is. */
const GENERIC_HEADING = /^\s*bookmarks(?:\s+menu)?\s*$/i;
/**
* @param {string} text
* @param {Mapper} mapUrl
* @param {Mapper} mapTitle
*/
export function sanitizeHtml(text, mapUrl, mapTitle) {
let descriptions = 0;
let keywords = 0;
/** @param {string} heading */
const mapHeading = (heading) => (GENERIC_HEADING.test(heading) ? heading : mapTitle(heading));
const spans = [
// HREF and ICON, at any quoting — attribute name case is left as found.
...attrSpans(text, /^href$/, attrRewriter(mapUrl)),
...attrSpans(text, /^icon$/, () => BLANK_ICON),
...urlAttrSpans(text, mapUrl),
// Firefox keywords and tags are user-authored text, so they are personal
// too. Tag count is preserved because it is the only part a parser sees.
...attrSpans(text, /^shortcuturl$/, () => `kw${++keywords}`),
...attrSpans(text, /^tags$/, (v) =>
v
.split(',')
.map((_, i) => `tag${i + 1}`)
.join(','),
),
// Bookmark and folder titles. `[\s\S]*?`, not `[^<]*`: a title holding a
// raw `<` made the whole match fail and stay in the output.
...spansFrom(text, new RegExp(`<a\\b${TAG_INNARDS}>([\\s\\S]*?)</a>`, 'gi'), mapTitle),
...spansFrom(text, new RegExp(`<h3\\b${TAG_INNARDS}>([\\s\\S]*?)</h3>`, 'gi'), mapTitle),
// `<TITLE>` / `<H1>` — usually "Bookmarks", but an exporter can write the
// owner's name there ("Jane Doe's bookmarks").
...spansFrom(text, /<title\b[^>]*>([^<]*)<\/title>/gi, mapHeading),
...spansFrom(text, /<h1\b[^>]*>([^<]*)<\/h1>/gi, mapHeading),
...descriptionSpans(text, () => `Description ${++descriptions}`),
];
return replaceSpans(text, spans);
}
+141
View File
@@ -0,0 +1,141 @@
// @ts-check
/**
* JSON bookmark files — BookmarkMagic JSON, Chrome's profile `Bookmarks` file,
* and Firefox's `bookmarks-YYYY-MM-DD.json` backup.
*
* Default-deny: every string VALUE is replaced unless its key is on the short
* structural list below. The old rewriter was default-allow — it rewrote only
* `"url"` and `"title"`, so Chrome's titles (under `"name"`), Firefox's `uri`,
* `iconUri` and keywords, Chrome's `meta_info` and the base64 `sync_metadata`
* blob all went through untouched, and the host check could not see a title.
*
* Values are rewritten by span over the raw text (a small tokenizer tracks
* which key each string belongs to), so indentation, key order, escapes and
* line endings stay byte-for-byte as the browser wrote them.
*/
import { isAddressLike, makeGuidMapper } from './mappers.mjs';
import { replaceSpans } from './spans.mjs';
/** @typedef {import('./spans.mjs').Span} Span */
/** @typedef {import('./mappers.mjs').Mapper} Mapper */
/**
* Keys whose string values are structure, not content, and are kept verbatim:
* format markers, type tags, root identifiers, numeric ids and every timestamp
* (Chrome stores dates as decimal strings). Nothing a person typed lives here.
*/
const STRUCTURAL = new Set([
'format',
'generator',
'exportedAt',
'version',
'type',
'root',
'id',
'date_added',
'date_modified',
'date_last_used',
'charset',
'source',
]);
/** Keys whose value is an address, mapped even when it does not parse. */
const ADDRESS = new Set(['url', 'uri', 'iconuri']);
/**
* @typedef {{ kind: 'object' | 'array', key: string | null, expectKey: boolean, current: string | null }} Frame
* @typedef {{ start: number, end: number, raw: string, key: string | null, parentKey: string | null }} StringToken
*/
/**
* Every string VALUE in a JSON text, with the key it sits under and the key of
* the object that holds it (`annos[].value` → key "value", parentKey "annos").
* The input has already passed `JSON.parse`, so it is well-formed.
*
* @param {string} text
* @returns {StringToken[]}
*/
export function stringValues(text) {
/** @type {Frame[]} */
const stack = [];
/** @type {StringToken[]} */
const out = [];
// The key a new child sits under. Array elements inherit the array's key, so
// every object inside `"annos": [...]` reports "annos" as its parent.
const keyFor = () => {
const top = stack.at(-1);
return (top?.kind === 'object' ? top.current : top?.key) ?? null;
};
for (let i = 0; i < text.length; i++) {
const ch = text[i];
if (ch === '{' || ch === '[') {
stack.push({
kind: ch === '{' ? 'object' : 'array',
key: keyFor(),
expectKey: ch === '{',
current: null,
});
} else if (ch === '}' || ch === ']') {
stack.pop();
} else if (ch === ',') {
const top = stack.at(-1);
if (top?.kind === 'object') top.expectKey = true;
} else if (ch === ':') {
const top = stack.at(-1);
if (top !== undefined) top.expectKey = false;
} else if (ch === '"') {
let j = i + 1;
while (j < text.length && text[j] !== '"') j += text[j] === '\\' ? 2 : 1;
const raw = text.slice(i + 1, j);
const top = stack.at(-1);
if (top?.kind === 'object' && top.expectKey) {
top.current = JSON.parse(`"${raw}"`);
} else {
const holder = top?.kind === 'object' ? top : stack.at(-2);
out.push({ start: i + 1, end: j, raw, key: keyFor(), parentKey: holder?.key ?? null });
}
i = j;
}
}
return out;
}
/**
* @param {string} text
* @param {Mapper} mapUrl
* @param {Mapper} mapTitle
*/
export function sanitizeJson(text, mapUrl, mapTitle) {
const mapGuid = makeGuidMapper();
let keywords = 0;
/** @type {Span[]} */
const spans = [];
for (const token of stringValues(text)) {
const key = token.key ?? '';
const lower = key.toLowerCase();
if (STRUCTURAL.has(key)) continue;
// An anno's `name` is a schema id (bookmarkProperties/description); its
// `value` is what the person wrote.
if (token.parentKey === 'annos' && key === 'name') continue;
const value = /** @type {string} */ (JSON.parse(`"${token.raw}"`));
let mapped;
if (lower === 'guid') mapped = mapGuid(value);
else if (lower === 'checksum' || lower === 'checksum_sha256') mapped = '0'.repeat(value.length);
else if (lower === 'sync_metadata') mapped = '';
else if (lower === 'keyword') mapped = value === '' ? '' : `kw${++keywords}`;
else if (lower === 'tags')
mapped =
value === ''
? ''
: value
.split(',')
.map((_, i) => `tag${i + 1}`)
.join(',');
else if (ADDRESS.has(lower) || isAddressLike(value)) mapped = mapUrl(value);
else mapped = mapTitle(value);
spans.push({ start: token.start, end: token.end, value: JSON.stringify(mapped).slice(1, -1) });
}
return replaceSpans(text, spans);
}
+66
View File
@@ -0,0 +1,66 @@
// @ts-check
/**
* The backstop: check the OUTPUT, not the rules that produced it.
*
* The rewriting rules are patterns over untrusted markup, so each one has a
* shape it does not match — a single-quoted attribute, a title containing `<`,
* a JSON key nobody enumerated, a `<DD class=…>`. Every one of those is a silent
* leak, and nobody can honestly claim to have thought of every shape. So the
* output is inspected independently, and anything that still looks personal
* makes the tool refuse instead of handing the file over. A rule this misses
* degrades into a refusal — annoying — instead of a published browsing history.
*/
import { TAG_INNARDS } from './spans.mjs';
/**
* Hosts in the output that are not one of our own `*.example` aliases.
*
* @param {string} text sanitized output
* @returns {string[]} distinct suspicious hosts, empty when clean
*/
export function findLeakedHosts(text) {
const leaked = new Set();
for (const m of text.matchAll(/\b[a-z][a-z0-9+.-]*:\/\/([^\s"'<>)\]}\\]+)/gi)) {
const authority = (m[1] ?? '').split(/[/?#]/, 1)[0] ?? '';
// Strip credentials and port; what remains is the host.
const host = authority.split('@').pop()?.replace(/:\d+$/, '').toLowerCase() ?? '';
if (host === '' || /^[a-z]\d*\.example$/.test(host)) continue;
leaked.add(host);
}
return [...leaked];
}
/**
* Words the sanitized TEXT of a Netscape file may legitimately contain: our
* placeholders, the default heading, and the script samples the title mapper
* adds so an encoding bug still reproduces.
*/
const ALLOWED_WORDS = new Set(['title', 'description', 'bookmarks', 'bookmark', 'menu', 'شيء']);
/**
* Words in the document's text content — everything outside tags and comments —
* that are not placeholders.
*
* Independent of the rewriting rules on purpose: it does not look for `<A>` or
* `<H3>` at all, it strips markup and reads whatever text is left. The old
* backstop matched only `scheme://host`, so an unsanitized title, a `<DD>` note
* or an `<H1>Jane Doe's bookmarks</H1>` passed it, and the test that guarded it
* reused the same pattern and could never notice.
*
* @param {string} html sanitized Netscape output
* @returns {string[]} distinct offending words, empty when clean
*/
export function findLeakedWords(html) {
const text = html
.replace(/<!--[\s\S]*?-->/g, ' ')
// Script and style bodies are code. A browser export never contains them;
// the T1 injection fixtures do, and their payload is the point of the file.
.replace(/<(script|style)\b[^>]*>[\s\S]*?<\/\1\s*>/gi, ' ')
.replace(new RegExp(`<${TAG_INNARDS}>`, 'g'), ' ')
.replace(/&(?:#\d+|#x[0-9a-f]+|[a-z]+);/gi, ' ');
const leaked = new Set();
for (const [word] of text.matchAll(/\p{L}{3,}/gu)) {
if (!ALLOWED_WORDS.has(word.toLowerCase())) leaked.add(word);
}
return [...leaked];
}
+167
View File
@@ -0,0 +1,167 @@
// @ts-check
/**
* Value mappers: deterministic placeholders that keep what parsers branch on.
*
* Replacements are stable per distinct input value, so "how many distinct
* hosts" and "which two bookmarks are duplicates" survive sanitizing.
*/
/** @typedef {(raw: string) => string} Mapper */
/**
* Replace addresses while preserving everything a parser branches on: the
* scheme, whether there is a port, how deep the path is, whether it ends in a
* slash, how many query parameters there are and in what order, and whether
* there is a fragment. Percent-escapes are carried across literally, so a
* decoding bug still reproduces.
*
* @returns {Mapper}
*/
export function makeUrlMapper() {
/** @type {Map<string, string>} */
const seen = new Map();
/** @type {Map<string, string>} */
const hosts = new Map();
/** @param {string} host */
const hostAlias = (host) => {
let alias = hosts.get(host);
if (alias === undefined) {
const n = hosts.size;
// a.example … z.example, then a1.example, a2.example …
const letter = String.fromCharCode(97 + (n % 26));
const round = Math.floor(n / 26);
alias = round === 0 ? `${letter}.example` : `${letter}${round}.example`;
hosts.set(host, alias);
}
return alias;
};
/** @param {string} segment */
const escapesIn = (segment) => (segment.match(/%[0-9a-fA-F]{2}/g) ?? []).join('');
/** @param {string} pathname */
const mapPath = (pathname) => {
if (pathname === '' || pathname === '/') return pathname;
const trailing = pathname.endsWith('/');
const parts = pathname.replace(/^\//, '').replace(/\/$/, '').split('/');
const mapped = parts.map((p, i) => (p === '' ? '' : `p${i + 1}${escapesIn(p)}`));
return `/${mapped.join('/')}${trailing ? '/' : ''}`;
};
/** @param {string} search */
const mapQuery = (search) => {
if (search === '') return '';
return `?${search
.slice(1)
.split('&')
.map((pair, i) => (pair.includes('=') ? `q${i + 1}=v${i + 1}` : `q${i + 1}`))
.join('&')}`;
};
return (raw) => {
if (raw === '') return raw;
const cached = seen.get(raw);
if (cached !== undefined) return cached;
let out;
let url;
try {
url = new URL(raw);
} catch {
url = undefined;
}
if (url === undefined) {
// Keep it unparseable, and keep distinct values distinct — the dedupe
// path treats an unparseable URL as its own trimmed self.
out = `not-a-url-${seen.size + 1}`;
} else if (url.host === '') {
// Opaque or hostless scheme (javascript:, data:, mailto:, file:, place:).
// The scheme is the whole point for the T1/T3 threat cases; the payload —
// which for file: is a local path with a user name in it — is not ours.
out = `${url.protocol}sanitized-${seen.size + 1}`;
} else {
// Credentials are never round-tripped, not even as placeholders.
const port = url.port === '' ? '' : `:${url.port}`;
const hash = url.hash === '' ? '' : '#frag';
out = `${url.protocol}//${hostAlias(url.host.replace(/:\d+$/, ''))}${port}${mapPath(url.pathname)}${mapQuery(url.search)}${hash}`;
}
seen.set(raw, out);
return out;
};
}
/**
* Replace titles while preserving what encoding bugs live in: emptiness, the
* scripts present (so a CJK or RTL or emoji bug still reproduces), and any HTML
* entity references, which are carried across verbatim.
*
* @returns {Mapper}
*/
export function makeTitleMapper() {
/** @type {Map<string, string>} */
const seen = new Map();
/** @type {[RegExp, string][]} */
const SCRIPTS = [
[/[一-鿿぀-ヿ]/, '文字'],
[/[֐-ࣿ]/, 'شيء'],
[/[\p{Extended_Pictographic}]/u, '🌸'],
];
return (raw) => {
if (raw.trim() === '') return raw;
const cached = seen.get(raw);
if (cached !== undefined) return cached;
const entities = (raw.match(/&(?:#\d+|#x[0-9a-fA-F]+|[a-zA-Z]+);/g) ?? []).join('');
const extras = SCRIPTS.filter(([re]) => re.test(raw))
.map(([, sample]) => sample)
.join('');
const out = `Title ${seen.size + 1}${extras === '' ? '' : ` ${extras}`}${entities}`;
seen.set(raw, out);
return out;
};
}
/** Firefox's fixed root GUIDs — structural, and the parser keys off them. */
const RESERVED_GUID = /^(?:root|menu|toolbar|unfiled|mobile|tags)_+$/;
/**
* Replace bookmark GUIDs with stable fakes of the same shape. They are random,
* but they are also the identifier sync uses, so they could tie a public sample
* to an account; the root GUIDs are kept because they carry meaning.
*
* @returns {Mapper}
*/
export function makeGuidMapper() {
/** @type {Map<string, string>} */
const seen = new Map();
return (raw) => {
if (raw === '' || RESERVED_GUID.test(raw)) return raw;
const cached = seen.get(raw);
if (cached !== undefined) return cached;
const n = String(seen.size + 1);
const out = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i.test(raw)
? `00000000-0000-4000-8000-${n.padStart(12, '0')}`
: `g${n.padStart(Math.max(raw.length - 1, n.length), '0')}`;
seen.set(raw, out);
return out;
};
}
/**
* True when a value is a URL whose payload could identify someone: it has a
* host, or it is a `file:` path (`file:///C:/Users/<name>/…` has no host).
*
* @param {string} value
*/
export function isAddressLike(value) {
try {
const url = new URL(value);
return url.host !== '' || url.protocol === 'file:';
} catch {
return false;
}
}
+125
View File
@@ -0,0 +1,125 @@
// @ts-check
/**
* Span rewriting: every change is a [start, end) → value replacement, so the
* bytes between spans are provably untouched. Rebuilding the file from a parsed
* model would quietly normalize them, and the byte-level shape is often the bug.
*/
/** @typedef {{ start: number, end: number, value: string }} Span */
/**
* Apply replacements right to left by character span.
*
* Overlaps are dropped, outermost first. Two patterns can legitimately claim
* the same bytes — an <A> title containing `x="http://…"` matches both the
* title rule and the generic attribute sweep — and applying both would splice
* already-rewritten text using an `end` measured against the original. The
* widest span sanitizes the most.
*
* @param {string} text
* @param {Span[]} spans
*/
export function replaceSpans(text, spans) {
const ordered = [...spans].sort((a, b) => a.start - b.start || b.end - a.end);
const out = [];
let cursor = 0;
for (const span of ordered) {
if (span.start < cursor) continue;
out.push(text.slice(cursor, span.start), span.value);
cursor = span.end;
}
out.push(text.slice(cursor));
// Joined once rather than spliced per span: splicing is O(spans × file),
// which took minutes on a large export.
return out.join('');
}
/**
* Spans for one capture group of a global regex; `map` receives the captured
* text. The `d` flag gives exact per-group offsets — searching the match for
* the captured substring lands on the wrong copy whenever the capture also
* appears earlier in the match (a title equal to part of its own URL).
*
* @param {string} text
* @param {RegExp} re
* @param {(captured: string) => string} map
* @param {number} [group]
* @returns {Span[]}
*/
export function spansFrom(text, re, map, group = 1) {
const indexed = new RegExp(re.source, re.flags.includes('d') ? re.flags : `${re.flags}d`);
/** @type {Span[]} */
const spans = [];
for (const m of text.matchAll(indexed)) {
const captured = m[group] ?? '';
const at = m.indices?.[group];
if (captured === '' || at === undefined) continue;
spans.push({ start: at[0], end: at[1], value: map(captured) });
}
return spans;
}
/**
* The inside of an opening tag, tolerating `>` within quoted attribute values.
* `data:text/html,<h1>hi</h1>` is a legal HREF in the T1/T3 fixtures, and a
* naive `[^>]*` stopped inside it, leaving that bookmark's title unsanitized.
*/
export const TAG_INNARDS = '(?:"[^"]*"|\'[^\']*\'|[^>"\'])*';
/** The four entities Netscape exporters write inside attribute values. */
const ATTR_ENTITY = /&(amp|lt|gt|quot);/gi;
/** @type {Record<string, string>} */
const ATTR_DECODE = { amp: '&', lt: '<', gt: '>', quot: '"' };
/**
* Decode an HTML attribute value, map it, then re-encode it the way the source
* file encoded it.
*
* Decoding is ONE pass. Chained `.replace()` calls (`&amp;` first, then `&lt;`)
* turned `&amp;lt;` into `<` — a double unescape (CodeQL js/double-escaping).
* Netscape exports write `&amp;` inside HREF but not all of them do, and
* emitting a bare `&` where the source had `&amp;` would silently repair the
* file being reported.
*
* @param {(value: string) => string} map
* @returns {(captured: string) => string}
*/
export function attrRewriter(map) {
return (captured) => {
const decoded = captured.replace(
ATTR_ENTITY,
(_m, name) => ATTR_DECODE[name.toLowerCase()] ?? _m,
);
const mapped = map(decoded);
const escapesAmp = /&amp;/i.test(captured) || !captured.includes('&');
return escapesAmp ? mapped.replace(/&/g, '&amp;') : mapped;
};
}
/**
* `name = value`, where the value is double-quoted, single-quoted or bare.
* Group 2/3/4 is the value, whichever quoting was used — matching only `"…"`
* left `HREF='https://…'` completely untouched.
*/
export const ANY_ATTR = /\b([a-z_][a-z0-9_-]*)\s*=\s*(?:"([^"]*)"|'([^']*)'|([^\s"'<>=`]+))/dgi;
/**
* Spans for every attribute whose (lower-cased) name matches, at any quoting.
*
* @param {string} text
* @param {RegExp} nameRe
* @param {(value: string) => string} map
* @returns {Span[]}
*/
export function attrSpans(text, nameRe, map) {
/** @type {Span[]} */
const spans = [];
for (const m of text.matchAll(ANY_ATTR)) {
if (!nameRe.test((m[1] ?? '').toLowerCase())) continue;
const at = m.indices?.[2] ?? m.indices?.[3] ?? m.indices?.[4];
const value = m[2] ?? m[3] ?? m[4] ?? '';
if (at === undefined || value === '') continue;
spans.push({ start: at[0], end: at[1], value: map(value) });
}
return spans;
}
+271
View File
@@ -0,0 +1,271 @@
import { spawnSync } from 'node:child_process';
import { mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { afterEach, describe, expect, it } from 'vitest';
/**
* scripts/sanitize-bookmarks.mjs — the cases the fixture corpus cannot reach:
* the two browser JSON formats (no committed fixture yet when these were
* written), the leaks a 2026-10-28 audit found, and the CLI's own safety rules.
*
* Every personal string in these samples is something the output must NOT
* contain, and each assertion checks the raw output text.
*/
const SCRIPT = join(process.cwd(), 'scripts', 'sanitize-bookmarks.mjs');
const dirs: string[] = [];
function tempFile(name: string, content: string): string {
const dir = mkdtempSync(join(tmpdir(), 'sanitize-'));
dirs.push(dir);
const path = join(dir, name);
writeFileSync(path, content);
return path;
}
function run(args: string[]): { status: number | null; stdout: string; stderr: string } {
const result = spawnSync(process.execPath, [SCRIPT, ...args], { encoding: 'utf8' });
return { status: result.status, stdout: result.stdout, stderr: result.stderr };
}
function sanitizeText(name: string, content: string): { status: number | null; out: string } {
const { status, stdout } = run([tempFile(name, content), '--stdout']);
return { status, out: stdout };
}
afterEach(() => {
for (const dir of dirs.splice(0)) rmSync(dir, { recursive: true, force: true });
});
const CHROME_PROFILE = `{
"checksum": "4f3c2b1a0e9d8c7b6a5f4e3d2c1b0a99",
"roots": {
"bookmark_bar": {
"children": [ {
"date_added": "13344473600000000",
"date_last_used": "0",
"guid": "5e1d9a62-1c3e-4c41-9d0b-6f2b8a1e7c33",
"id": "5",
"meta_info": { "Nickname": "oncology-portal", "Thumbnail": "https://thumbs.clinic.internal/1.png" },
"name": "Dr Smith oncology results portal",
"type": "url",
"url": "https://portal.clinic.internal/results?patient=8812"
} ],
"date_added": "13344473600000000",
"date_modified": "13344473700000000",
"guid": "0bc5d13f-2cba-5d74-951f-3f233fe6c908",
"id": "1",
"name": "Bookmarks bar",
"type": "folder"
},
"other": { "children": [], "date_added": "0", "guid": "82b081ec-3dd3-529c-8475-ab6c344590dd", "id": "2", "name": "Other bookmarks", "type": "folder" },
"synced": { "children": [], "date_added": "0", "guid": "4cf2e351-0e85-532b-bb37-df045d8f8d0f", "id": "3", "name": "Mobile bookmarks", "type": "folder" }
},
"sync_metadata": "Q2hBSkVMa0JBdVlWa1Z3QmxBZ0xpbnVzLnRvcnZhbGRz",
"version": 1
}
`;
const FIREFOX_BACKUP = JSON.stringify({
guid: 'root________',
title: '',
index: 0,
dateAdded: 1700000000000000,
lastModified: 1700000000000000,
id: 1,
typeCode: 2,
type: 'text/x-moz-place-container',
root: 'placesRoot',
children: [
{
guid: 'toolbar_____',
title: 'toolbar',
index: 1,
typeCode: 2,
type: 'text/x-moz-place-container',
root: 'toolbarFolder',
children: [
{
guid: 'Xk2pQ9vLm3Ta',
title: 'Divorce lawyer shortlist',
index: 0,
dateAdded: 1700000000000000,
typeCode: 1,
type: 'text/x-moz-place',
uri: 'https://lawyers.example-private.net/shortlist',
iconUri: 'fake-favicon-uri:https://lawyers.example-private.net/favicon.ico',
keyword: 'lawyerkw',
tags: 'custody,urgent',
charset: 'UTF-8',
annos: [
{
name: 'bookmarkProperties/description',
flags: 0,
expires: 4,
value: 'call before Friday hearing',
},
],
},
],
},
],
});
describe('sanitizer — browser JSON formats', () => {
it("rewrites every personal value in Chrome's profile Bookmarks file", () => {
const { status, out } = sanitizeText('Bookmarks', CHROME_PROFILE);
expect(status).toBe(0);
for (const secret of [
'Dr Smith',
'oncology',
'clinic.internal',
'patient=8812',
'5e1d9a62-1c3e-4c41-9d0b-6f2b8a1e7c33',
'Q2hBSkVMa0JBdVlWa1Z3QmxBZ0xpbnVzLnRvcnZhbGRz',
'4f3c2b1a0e9d8c7b6a5f4e3d2c1b0a99',
]) {
expect(out, secret).not.toContain(secret);
}
});
it('keeps the Chrome structure: keys, types, ids, timestamps and layout', () => {
const { out } = sanitizeText('Bookmarks', CHROME_PROFILE);
const before = JSON.parse(CHROME_PROFILE);
const after = JSON.parse(out);
expect(Object.keys(after)).toEqual(Object.keys(before));
const node = after.roots.bookmark_bar.children[0];
expect(node.type).toBe('url');
expect(node.id).toBe('5');
expect(node.date_added).toBe('13344473600000000');
expect(after.roots.bookmark_bar.date_modified).toBe('13344473700000000');
expect(after.version).toBe(1);
expect(out.split('\n').length).toBe(CHROME_PROFILE.split('\n').length);
expect(after.checksum).toMatch(/^0+$/);
});
it('rewrites every personal value in a Firefox backup and keeps the root GUIDs', () => {
const { status, out } = sanitizeText('bookmarks-2026-10-28.json', FIREFOX_BACKUP);
expect(status).toBe(0);
for (const secret of [
'Divorce',
'lawyer',
'example-private.net',
'lawyerkw',
'custody',
'Friday hearing',
'Xk2pQ9vLm3Ta',
]) {
expect(out, secret).not.toContain(secret);
}
const after = JSON.parse(out);
expect(after.guid).toBe('root________');
expect(after.root).toBe('placesRoot');
expect(after.children[0].guid).toBe('toolbar_____');
const bookmark = after.children[0].children[0];
expect(bookmark.type).toBe('text/x-moz-place');
expect(bookmark.dateAdded).toBe(1700000000000000);
expect(bookmark.tags).toBe('tag1,tag2');
expect(bookmark.annos[0].name).toBe('bookmarkProperties/description');
});
it('refuses JSON that is none of the three known formats', () => {
const { status, out } = sanitizeText(
'x.json',
'{"items":[{"label":"Dr Smith","href":"https://clinic.internal/"}]}',
);
expect(status).toBe(1);
expect(out).toBe('');
});
});
describe('sanitizer — HTML leaks found by the audit', () => {
const html = (body: string) => `<!DOCTYPE NETSCAPE-Bookmark-file-1>\n${body}\n`;
it('sanitizes a <DD> that carries attributes', () => {
const { status, out } = sanitizeText(
'dd.html',
html(
'<DL><p>\n<DT><A HREF="https://a.invalid/">x</A>\n<DD class="n">Private note about my therapist\n</DL><p>',
),
);
expect(status).toBe(0);
expect(out).not.toContain('therapist');
});
it('sanitizes a file: ICON_URI that names a local user', () => {
const { out } = sanitizeText(
'icon.html',
html(
'<DL><p>\n<DT><A HREF="https://a.invalid/" ICON_URI="file:///C:/Users/JaneDoe/icons/x.ico">x</A>\n</DL><p>',
),
);
expect(out).not.toContain('JaneDoe');
});
it('sanitizes a document heading that names its owner', () => {
const { out } = sanitizeText(
'h1.html',
html(
'<TITLE>Jane Doe bookmarks</TITLE>\n<H1>Jane Doe personal bookmarks</H1>\n<DL><p>\n<DT><A HREF="https://a.invalid/">x</A>\n</DL><p>',
),
);
expect(out).not.toContain('Jane');
expect(out).toContain('<TITLE>Title ');
});
it('keeps the generic "Bookmarks" heading as it is', () => {
const { out } = sanitizeText(
'plain.html',
html(
'<TITLE>Bookmarks</TITLE>\n<H1>Bookmarks Menu</H1>\n<DL><p>\n<DT><A HREF="https://a.invalid/">x</A>\n</DL><p>',
),
);
expect(out).toContain('<TITLE>Bookmarks</TITLE>');
expect(out).toContain('<H1>Bookmarks Menu</H1>');
});
it('refuses when personal text sits where no rewriting rule looks', () => {
// Loose text in a <DT> is reached by no rule. The text check catches it
// anyway — that independence is what makes the rules trustworthy.
const { status, out } = sanitizeText(
'loose.html',
html(
'<DL><p>\n<DT>Notes from Jane about the custody case\n<DT><A HREF="https://a.invalid/">x</A>\n</DL><p>',
),
);
expect(status).toBe(1);
expect(out).toBe('');
});
it('decodes an attribute value once, never twice', () => {
// `&amp;lt;` is "&lt;" after ONE decode. The old chained replaces turned it
// into "<", which changed how many query parameters the URL had.
const { out } = sanitizeText(
'amp.html',
html('<DL><p>\n<DT><A HREF="https://a.invalid/p?x=&amp;lt;b">t</A>\n</DL><p>'),
);
expect(out).toContain('HREF="https://a.example/p1?q1=v1&amp;q2"');
});
});
describe('sanitizer — CLI safety', () => {
it('refuses to overwrite the input file', () => {
const content =
'<!DOCTYPE NETSCAPE-Bookmark-file-1>\n<DL><p>\n<DT><A HREF="https://real.example.org/">Mine</A>\n</DL><p>\n';
const path = tempFile('bookmarks.html', content);
const { status, stderr } = run([path, path]);
expect(status).toBe(1);
expect(stderr).toContain('refusing to overwrite');
expect(readFileSync(path, 'utf8')).toBe(content);
});
it('reports the output size in bytes, not characters', () => {
const path = tempFile(
'b.html',
'<!DOCTYPE NETSCAPE-Bookmark-file-1>\n<DL><p>\n<DT><A HREF="https://a.invalid/">ブックマーク</A>\n</DL><p>\n',
);
const { stderr, stdout } = run([path, '--stdout']);
expect(stderr).toContain(`${Buffer.byteLength(stdout)} bytes out`);
});
});
+16
View File
@@ -0,0 +1,16 @@
{
"compilerOptions": {
"target": "ES2025",
"lib": ["ES2025"],
"module": "NodeNext",
"moduleResolution": "NodeNext",
"types": ["node"],
"allowJs": true,
"checkJs": true,
"noEmit": true,
"strict": true,
"skipLibCheck": true,
"noUncheckedIndexedAccess": true
},
"include": ["scripts/**/*.mjs"]
}