Who decided
75 lines?
Fair question — every scanner threshold deserves it. This page is the answer: where each default comes from, what locks it in place, and what we still consider open. If you'd challenge a threshold in an issue, you should be able to read its derivation here first.
Reference values,
not pass/fail gates
- Scores degrade smoothly. Six scoring curves (binary, asymptotic decay, ratio, Goldilocks range, tiered, averaged sub-scores) mean 2,500 words against a 2,000-word reference scores 0.8 — measured deviation, not a verdict.
- Not run is never zero. A dimension that didn't run reports
"status": "not_run"with anullscore and is excluded from the total;scoreScoperecords what the score covers. A dimension that did not run can never make an honest repo look worse. - One failure counts once. Rule-compliance dimensions score from pass ratio only — no stacked penalties for the same finding. The two analysis-based dimensions (sharing, harness wiring) apply their own penalties because their base score isn't findings-derived.
Every default,
sourced
Values live in one place in the codebase (a single thresholds file plus shared constants) so no two rules can disagree about a number. The defaults and where they come from:
| Threshold | Default | Derivation |
|---|---|---|
adapter_max_lines |
75 | Thin-adapter principle: a per-tool file (CLAUDE.md, GEMINI.md) should route to shared guidance, not inline it. Calibrated so mature reference adapters pass and inlined ones fail. |
skill_max_lines |
300 | A skill is a short, loadable checklist; beyond ~300 lines it's a document and should deep-link instead. Calibrated against the reference repo's skills. |
sharing_target_percent |
65 | Share of guidance lines that should live in shared layers vs. per-harness adapters; reference-calibrated against mature multi-harness repos. |
MAX_RECOMMENDED_ALWAYS_LOADED_LINES |
250 | The Codified Context study validated a ~660-line always-loaded constitution over 283 sessions on a 108,256-line codebase — an empirical ceiling for a large repo. 250 is a deliberately tighter advisory default for the typical, smaller repo. |
MAX_RECOMMENDED_ALWAYS_LOADED_FILE_LINES |
150 | Per-file slice of the always-loaded budget — keeps any single always-on file scannable. Advisory (info). |
| Grade bands | 85 / 70 / 50 | Excellent / good / fair cutoffs, validated against the reference repo and the fixture suite below. |
The same study reports context infrastructure at 24.2% of that 108K-line codebase — a published reference point for judging whether a guidance footprint is proportionate, used as context, not as a gate.
~660 lines / 283 sessions / 24.2% — Codified Context, arXiv 2026, arxiv.org/abs/2602.20478Fixtures fail the build
before a threshold drifts
Single-repo calibration risks overfitting to one mature repo's quirks. So the calibration suite holds three synthetic repo shapes at distinct maturity tiers, scanned in CI:
| Fixture | Shape | Expectation |
|---|---|---|
| weak | One bloated, inlined CLAUDE.md; no shared entry, rules doc, skills, or agents | Low score; surfaces actionable findings |
| normal | Shared AGENTS.md + thin CLAUDE.md + one skill | Middle band |
| mature | Thin adapters routing to AGENTS.md + canonical rules, a skills index, routed skill and agent | Excellent band (≥ 85) |
The test asserts the band ordering — weak < normal < mature, with mature clearing the excellent threshold — not brittle exact scores. A threshold change that lets a weak shape score like a mature one breaks the band and fails CI. The fixtures complement (not replace) reference-repo calibration: fixtures catch threshold drift across shapes; the reference repo catches regressions in a known-good one. And the scanner holds itself to its own standard — paniolo-scan must score excellent on its own repo after any scoring change.
What we don't
claim yet
- No per-rule precision/recall corpus. We can tell you why a rule exists and what it checks; we cannot yet quote a measured detection rate. A labeled multi-repo corpus with per-rule precision/recall in CI is the planned extension — prior art exists in this niche and we intend to match it.
- Security-posture severities are conservative pending calibration. The six posture rules ship at info/warn until a real-repo severity pass earns stronger levels.
Publishing the limits is part of the methodology. A threshold you can interrogate is worth more than a score you're asked to trust.
Interrogate it on your own repo.
npx @paniolo/cli scan — every finding carries its reasoning