tomkay commited on
Commit
9cd9644
·
verified ·
1 Parent(s): 9f76180

Add certification & corpus-fit guidance (PB score, 4-bit distractor certificate, chunking prescription)

Browse files
Files changed (1) hide show
  1. README.md +14 -0
README.md CHANGED
@@ -59,3 +59,17 @@ ranked = m.rerank("how does a cross-encoder reranker work?",
59
  - Incorporates the `MiniLM-L6-H384-uncased` backbone under the **MIT License** — see `LICENSE-minilm.txt`.
60
 
61
  © 2026 BAA AI (Black Sheep AI) — baa.ai. Provided "as is" without warranty.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
59
  - Incorporates the `MiniLM-L6-H384-uncased` backbone under the **MIT License** — see `LICENSE-minilm.txt`.
60
 
61
  © 2026 BAA AI (Black Sheep AI) — baa.ai. Provided "as is" without warranty.
62
+
63
+ ## Certification & corpus fit (2026-07)
64
+
65
+ **Position Balance (PB): 0.17** — PB measures how findable a chunk is through its *second* fact when two
66
+ facts share one embedding (second-fact / first-fact top-1 retrieval on an adversarial 1,300-chunk audit;
67
+ fleet range 0.16–0.69). Compact deployment tier: use with strictly atomic chunking (one claim per embedded chunk) and parent-document retrieval.
68
+
69
+ **4-bit quantization: certified lossless under distractor stress.** Paired contested-region robustness
70
+ (gold document injected into pools of up to 100 near-topical distractors, n=300 queries, bootstrap CIs)
71
+ is statistically indistinguishable from fp16 — an axis standard hit@k benchmarks do not measure.
72
+
73
+ **Chunking prescription:** embed one atomic claim per chunk and lead with its key entity; retrieve small,
74
+ return the parent section for context. Basis: single-vector embeddings preserve ~one independent fact per
75
+ chunk regardless of encoder family (measured across 12 encoders).