Exact and hybrid KB retrieval recorded the same Recall@10 of 0.833333 over 155 searchable notes and 18 questions.
Hybrid had higher observed MRR@10 (0.937500 versus 0.892857) and nDCG@10 (0.833884 versus 0.790377), while the paired 95% intervals included no change.
Exact p95 latency was 44.345 milliseconds; hybrid was 62.834 milliseconds on QMD 2.5.3 and a pinned EmbeddingGemma 300M Q8 model.
The corpus is too small to claim that hybrid is generally better than exact search.
Retrieval quality
| Retriever | Recall@10 | MRR@10 | nDCG@10 |
|---|---|---|---|
| exact | 0.833333 | 0.892857 | 0.790377 |
| hybrid | 0.833333 | 0.937500 | 0.833884 |
Latency
| Retriever | p95 ms |
|---|---|
| exact | 44.345 |
| hybrid | 62.834 |
| keyword | 821.370 |
| semantic | 41,000.524 |
Paired hybrid minus exact
| Metric | Observed difference | 95% interval |
|---|---|---|
| Recall@10 | 0 | [0, 0] |
| MRR@10 | +0.044643 | [0, 0.133929] |
| nDCG@10 | +0.043508 | [-0.012752, 0.111832] |
Method
The pilot froze 18 questions against one repository snapshot. Graded relevance judgments were written before rankings were inspected. The evaluator scanned 156 Markdown records and projected 155 searchable notes into QMD after excluding the authored vault index and agent guides. Nine questions were development; nine were held out. The held-out set covered exact identity, conceptual recall, active plans, current decisions, code-path context, source evidence, historical rationale, stale-versus-current conflicts, and one no-answer case.
Metrics use cutoff 10. Recall@10 is the fraction of judged relevant notes that appeared. MRR@10 rewards an earlier first relevant result. nDCG@10 also uses graded relevance and rank. Eight held-out questions were answerable. A 10,000-resample paired bootstrap estimated hybrid minus exact.
The mixed-cache run used QMD 2.5.3 at Hraness compatibility commit aa993dc and a locally verified EmbeddingGemma 300M Q8 model. The machine was an Apple M4 Max with 16 logical CPUs and 128 GiB of memory, running Bun 1.3.14 and Node 24.3.0 on arm64 Darwin 25.5.0.
Limits
These nine-query p95 values mix cold and warm state. They are local diagnostics, not a speed claim. The semantic p95 includes the first in-process model load. Eight answerable questions and one no-answer question cannot establish stable overall or per-class performance. The paired intervals include no change, so the pilot does not show that hybrid retrieval is better. Both retrievers answered the no-answer question, so no-answer accuracy was 0. The result is a frozen Jungle regression baseline. It does not generalize to another vault, model, QMD release, cache state, or machine.
The same figures appear in the measurement section of the kb project page.