paniolo stale Guide Research
Paper → Feature → Limit

The bibliography
behind the ledger

Every headline capability in paniolo stale traces to published freshness research or convergent practitioner diagnosis. This page is the trace — keyed by feature, not by paper — and the places we deliberately refuse the literature's shortcuts.

How To Read This Page

Evidence strength, labeled

The product docs cover commands and config. This page covers why the shape exists. Each claim carries one of four labels:

  • Measured — a statistic or result from the paper's own study. The paper measured the problem or a method in its domain; it did not measure our detector's precision on your wiki.
  • Taxonomic — the feature operationalizes a validated vocabulary or split from the literature (for example SUD vs SUNP, freshness vs age).
  • Practitioner — engineering posts that independently converge on the same failure mode (docs and code lack a feedback loop; wrong prose is worse than missing prose). Motivation, not a peer-reviewed effect size.
  • Paniolo ahead / behind — where the ledger has no close analogue in the ingested set, or where the literature offers a mechanism we have not shipped.

One claim we deliberately do not make: that retrieval scores, agent agreement, or a single model edit prove documentation truth. The central product rule — detection nominates; it never proves — is a response to negative results on automated knowledge editing, not a marketing flourish.

Detection

How candidates are nominated

Declared watches

Frontmatter staleness.watches ties prose to repository paths and symbols. A matching code diff nominates a falsifiable allegation — it does not close a verdict.

FeedsAnchorsStrength
declared-watch/1, docs-declared/1 USB predict-then-refresh (IJCAI 2017); DEAN structural outdated-fact detection; Sync-o / Mintlify event-driven doc refresh Taxonomic for predict-then-refresh; practitioner for change-triggered review. Watches are manual binary edges — not a learned update-frequency prior (behind the volatility literature).
Liang, Zhang & Xiao — IJCAI 2017, ijcai.org/proceedings/2017/524 · Tu et al. — DEAN, arxiv.org/abs/2402.03732 · Sync-o · Mintlify

Exact deleted or renamed references

Lexical hits on deleted or renamed code paths and symbols can nominate unwatched prose. Exact identity only — not semantic drift.

FeedsAnchorsStrength
unwatched-ref/1 DEAN outdated structured facts; Doccupine on trusted-but-wrong references Taxonomic problem framing; detector is deterministic string identity
DEAN — arxiv.org/abs/2402.03732 · Doccupine

Parser-proven code comments

Tree-sitter binds comments (and Python docstrings) to owning symbols. Ambiguous ownership creates no allegation. Remediation must preserve the executable token stream.

FeedsAnchorsStrength
comment-assoc/1 SemUpdates text-level semantic update detection (SIGIR 2025); Sugarbug on structural disconnect between work and description Taxonomic for text-level staleness. Comment↔code co-evolution papers exist outside this ingested set; association here is parser-proven engineering, not a Fluri-style mined link.
Ranaut et al. — SemUpdates / text obsoleteness, DOI 10.1145/3726302.3730254

Live-reference verification (the verifier role)

Agents judge claims against current code — or against an explicit claim-scope version — not against remembered snapshots.

FeedsAnchorsStrength
Verifier agent; claim-scope / historical markers DyKnow live-reference checks; SemUpdates SUD/SUNP split; FreshLLMs evidence-at-answer-time; fact-duration / KB stability for time-bounded truth Measured motivation from DyKnow's negative editing results; taxonomic for SUD vs SUNP and freshness/age vocabulary
DyKnow — arxiv.org/abs/2404.08700 · FreshLLMs — arxiv.org/abs/2310.03214 · Fact duration — arxiv.org/abs/2305.14824 · KB stability — arxiv.org/abs/2211.00989
Ledger

Durable work, not a one-shot detector

Falsifiable allegation queue

Content-addressed allegations, immutable evidence, reopen-on-edit, and retained insufficient-evidence. Unknown is not scored as fresh.

FeedsAnchorsStrength
S- allegations, dispositions, scan checkpoints Cho & Garcia-Molina budgeted freshness; Ekline detect→draft→review→measure loop; Doccupine / Sync-o on partially-right pages being worse than missing ones Taxonomic for freshness/age and budgeted revisit. The git-native allegation trail with optimistic revisions has no close analogue in the ingested paper set (ahead).
Cho & Garcia-Molina — ACM TODS 2003 (author preprint mirrored for research) · Ekline · Doccupine

Budgeted phases and deferred tail

maxAllegationsPerPhase, retrieval topK / maxImmediate, and durable deferred work are a refresh-budget problem in the Cho & Garcia-Molina sense.

FeedsAnchorsStrength
Worker budgets, deferred scheduling Page refresh policies; streaming materiality / edit-velocity scores Taxonomic budget framing. Deferred declared candidates are still allegations — scheduling is not shadow-only.
Streaming knowledge compilation — arxiv.org/abs/2606.09877
Adjudication

Why four roles, not one model

Verifier + verdict challenger

Independent processes; the challenger sees the sealed verdict body, never the verifier's hidden reasoning. Disagreement retains work.

FeedsAnchorsStrength
verifier, verdictChallenger DyKnow: automated verification/editing without opposition over-trusts the model; maker/checker harness practice Measured caution from DyKnow. Adversarial separation of sealed verdicts is ahead of the ingested KB-staleness paper set (no close analogue).
DyKnow — arxiv.org/abs/2404.08700

Remediator + patch challenger + merge gate

Remediation is a bounded rewrite (ERASE-style edit, not a stale note). Rust validates the unified diff in memory; merge requires upheld observations and an exact proposal head.

FeedsAnchorsStrength
Prose/comment remediation, merge gate, kill switch ERASE rewrite-don't-append; WikiBigEdit limits of lifelong LLM editing; Mintlify/Ekline human review step (Paniolo substitutes dedicated challengers + repo gates) Measured risk from WikiBigEdit / DyKnow. Allowlists and challengers bound blast radius — they do not claim editing is solved.
ERASE — arxiv.org/abs/2406.11830 · WikiBigEdit — arxiv.org/abs/2503.05683 · Mintlify
Retrieval

Search nominates; it does not judge

qmd as a shadow lane

Lexical, vector, and rerank lanes may measure recall over a holdout. Scores never set dispositions, mint gold labels, or grant merge authority. Production nomination today is declared watches plus exact references until a refreshed holdout clears frozen gates.

FeedsAnchorsStrength
shadow-qmd, live retrieval.shadow, index-integrity fail-closed FreshLLMs (fetch current evidence, don't rewrite weights); appropri8 freshness ≠ relevance; agent-ecosystem / Risingwave on index lag as pipeline staleness Taxonomic for retrieval-as-refresh. Shadow remains non-authoritative by design — live shadow metrics are not admission.
FreshLLMs — arxiv.org/abs/2310.03214 · appropri8

Demand and consumer signals (deferred)

Query-log and behavioral bounce signals appear in the literature and practitioner corpus. They are not a shipped scheduling prior — collection policy and measured benefit remain open.

FeedsAnchorsStrength
Future demand prior; bounded flag observations Topic-aware KB update from query logs; Slite / Kipwise search-bounce; dbi embedding-version outcome metrics Practitioner / taxonomic motivation. Behind as a production scheduler.
Topic-aware KB update — arxiv.org/abs/2208.14601 · Slite

A second pass over recorded history

StateHarness keeps a structured record of what a long-horizon agent has learned and decided, and — the part that maps here — when behavioral signals accumulate, a separate diagnostic session revisits the recorded history and surfaces evidence the working agent overlooked. Their results include an honest trade-off: the extra bookkeeping improved completion and consistency while increasing pass-to-pass regressions on one suite.

That is the stale lane's shape with a different trigger: signals accumulate (commits, config edits), then a second pass audits claims the first pass wrote. The regression caveat is the reason the auditor is a different role with a narrower charter than the writer — and why a remediation that cannot produce evidence stays an allegation. What the CLI does not ship is their in-session runtime state; Paniolo's record is the durable wiki plus the ledger, not a working brief that dies with the session.

StateHarness — arXiv 2026, arxiv.org/abs/2610.17616

Verification scoped to what the update touches

GRACE maintains an evolving instruction as a typed semantic graph and validates each proposed update inside the local neighborhoods of the nodes it modifies — not against the whole corpus. Under a controlled distribution shift the scoped approach reached a 0.661 strict-reliability checkpoint where a flat-text evolving instruction collapsed to 0.191 — and the flat version ended 1.6× longer, accreting prose as it degraded.

Two transfers land here. The first is scope: a stale remediation should be checked against the claim it touches and its linked neighbors, not re-justified against every page in the wiki — the typed neighborhood is what wikilinks give the adjudicator for free. The second is the flat-text result: an instruction store that grows by accretion gets worse and longer, which is why the ledger records allegations about specific claims instead of appending caveats to whole documents. Their evolving artifact is an agent prompt, not a wiki — the graph discipline transfers; the harness does not.

Hsu & Lu — GRACE, arXiv 2026, arxiv.org/abs/2607.09175

The durable-worker's honesty rules — claude.dev

Anthropic's reference implementation for scheduled agents condenses to six rules, three of which are staleness-lane invariants stated plainly: a failed read is not a quiet day (a source that couldn't be fetched must be named in the output, not reported as zero signal); re-check every item right before posting (the world moved while the agent read — "one stale 'still waiting on you' costs more trust than ten missing items"); and confirm the write, then update the ledger (a post counts only on the API's confirmation, and ambiguous outcomes are recorded as "maybe posted" rather than lost or double-reported). Bookmarks per source replace fixed time windows so a late run never leaves a coverage gap.

That is the contract a durable worker owes its reader, and the stale lane already keeps most of it — verdicts cite page bytes at HEAD rather than trusting the nomination's memory, and the ledger is append-only. The rules worth adopting whole are the cursor discipline (a lane that fails mid-scan should freeze its own bookmark instead of silently skipping coverage) and the explicit third state: an ambiguous outcome is recorded, not resolved by guesswork. Their plumbing is Anthropic-specific; the honesty rules are not.

Martin & Avilla — claude.dev Engineering, Oct 2026, claude.dev/blog/building-effective-agent-automations
Honest Gaps

Where the literature is ahead

Declared watches are a defensible deterministic floor. These are the largest research gaps relative to the ingested corpus — not active release promises:

  • Learned volatility prior — fact duration, Wikidata stability (~83% F1 post-hoc in that study), materiality / Abnormal Edit Ratio, and per-entity update frequency all predict who goes stale before checking. The ledger records outcomes that could feed such a prior after opportunity-normalized calibration; it does not ship one yet.
  • Typed propagation — WikiMonitor-Onto found one-hop ontology propagation surfacing indirect stale concepts a per-assertion baseline missed. Ordinary wikilinks do not currently fan out nominations.
  • Remediator risk class — DyKnow and WikiBigEdit warn about LLM edits. Challengers and allowlists bound the blast radius; they do not eliminate the failure class.
  • State-validated recall — MemTrace keeps memory as immutable traces and checks recalled evidence against the current repository state before reuse — staleness detection inside agent memory, not only in documentation. The ledger nominates stale pages; it does not yet validate an agent's recalled facts the same way.
KB stability — arxiv.org/abs/2211.00989 · WikiMonitor-Onto — DOI 10.18178/JAAI.2026.4.3.152-163 · MemTrace — arxiv.org/abs/2610.04838
Practitioner Convergence

Same diagnosis, shared loop

Independent engineering posts converge on the problem the ledger productizes: documentation and code live in disconnected systems; AI raises write velocity without review velocity; a wrong page is trusted where a missing page is not.

Those workflows usually place a person at approval. Paniolo places dedicated verifier, challenger, and remediator agents plus deterministic repository gates — still evidence-gated, still refuse-to-guess when ungrounded.

Read the operator guide, then try a dry-run scan.

npx @paniolo/cli stale scan — detection without agents

Stale guide