When to use

When you want empirical savings data on your specific codebase, not the median from the default benchmark suite.

What it does

Finds a sample of TypeScript/JavaScript files in the project (up to 20, skipping node_modules)
For each file: reads it with the built-in Read tool (byte count via wc -c), then reads it with ashlr__read (returned payload size)
Computes per-file savings percentage and aggregates to a median
Reports the result and writes it to .ashlrcode/benchmark.json

Output

ashlr benchmark  ·  20 files sampled

Results:
  servers/efficiency-server.ts   18.4KB → 2.1KB  −88%
  servers/bash-server.ts         12.1KB → 1.8KB  −85%
  site/app/page.tsx               8.9KB → 1.4KB  −84%
  ...

Median savings:  −82%
Best:            −91%  (servers/genome-server.ts)
Worst:           −31%  (package.json — already small)

Written to .ashlrcode/benchmark.json

Advanced flags

/ashlr-benchmark invokes scripts/run-benchmark.ts directly. You can run it from the shell with additional flags:

bun run scripts/run-benchmark.ts
bun run scripts/run-benchmark.ts --repo /path/to/repo
bun run scripts/run-benchmark.ts --out docs/benchmarks-v2.json
bun run scripts/run-benchmark.ts --compare
bun run scripts/run-benchmark.ts --validate-tokenizer

Flag	Effect
`--compare`	Prints an A/B table: native tool output size vs ashlr output size, per file
`--validate-tokenizer`	Samples 5% of files through the Anthropic API (`measureTokens`) and reports heuristic error vs true token count. Requires `ANTHROPIC_API_KEY`. Skipped silently when the key is absent.
`--dry-run`	Runs the full benchmark but skips writing the output file
`--out <path>`	Write results to a custom path instead of `docs/benchmarks-v2.json`

Confidence intervals

The benchmark computes a 95% bootstrap confidence interval (2.5/97.5 percentile of 1,000 resampled means) for the read-savings ratio. The CLI prints this alongside the headline:

[run-benchmark]   read  median  −82.3%
[run-benchmark]   read  CI      [−80.1%–−84.5%]

The CI is seeded deterministically, so it is reproducible across runs on the same file set.

Methodology

The headline savings number (−57% cross-language) is produced by this benchmark against three public repos (vercel/ai, pandas, tokio). The full methodology is in docs/benchmarks.md:

Samples are drawn with rg --json against the repo root
Raw byte counts are measured with wc -c; returned payload size is measured from the actual tool response
The 2 KB floor is applied: files below this threshold are excluded from the compression headline (they are not candidates for snipCompact)
The --validate-tokenizer flag can verify how closely the chars/4 heuristic tracks true API token counts for your specific codebase

/ashlr-savings — live accumulated savings
/ashlr-badge — generate a badge from the savings data
How session savings are counted — why session numbers can differ from the benchmark headline

/ashlr-benchmark