What Is AI Referenceability? — A Retrieval Design Principle for RAG and AI Agents

Your AI having the information and your AI reaching the right information are two different things.

Definition — AI Referenceability is the degree to which an AI can reach the correct reference point for a given question.

Origin At NLP2026 I proposed "Referenceability" as an evaluation axis. The experiment measured retrieval only: whether the system could reach the correct reference point. "AI Referenceability" is my public-facing term for the practical extension: not only whether the right reference point can be reached, but whether it is current, applicable to the case, and authoritative. That extension is not what the study demonstrated.

The experiment 321 annotated episodes, 25 metadata fields, 9 concatenated for retrieval. Recall@10, TF-IDF + cosine, 400 queries. Only the document representation changed.

Result (structural queries, n=300) Structured metadata only — 59.0% Body text only — 5.3% +53.7 points, 11.1x (p < 0.001) Adding the body text back made it slightly worse (58.3%). More text was not better. Purity mattered more.

Where it does not hold In this experiment, the advantage appeared on structural queries using the structured vocabulary, but disappeared on verbatim excerpt queries, where both representations scored 100% (ratio 1.0x). Practical implication: collect the questions people actually ask first, then design the metadata around them.

Five design principles

1. Structure: what, who it applies to, conditions, exceptions

2. Controlled vocabulary: use canonical terms consistently, and map synonyms explicitly

3. Make relations explicit: version, scope, approver

4. Cut noise: you can pass everything; don't

5. Make it updatable: effective date, owner, current version

vs AI Visibility

Visibility: does your content appear in AI answers. Referenceability: does the AI reach and apply it correctly.

vs Context Engineering

Context Engineering asks what information reaches the model, when, and in what form. Referenceability is one quality axis within that larger problem: can the right reference point actually be reached?

Limitations

TF-IDF + cosine. Single domain, single annotator. Not peer reviewed. A Dense Retrieval follow-up (multilingual-e5-large) held the advantage at about 2.2x across all 400 queries, but sits outside the paper. Public AI search remains untested: a hypothesis, not a finding.

Citation

Ikematsu, J. (2026). "RAG時代の言語資源設計原理―構造化テキストによる『参照可能性』の実証" [Design Principles for Language Resources in the RAG Era: Demonstrating "Referenceability" through Structured Text]. Proceedings of NLP2026. CC BY 4.0. https://www.anlp.jp/proceedings/annual_meeting/2026/pdf_dir/Q5-8.pdf

Canonical pages and research assets

• Technical Note / DOI: AI Referenceability: A Framework for Designing Knowledge That AI Can Reliably Retrieve and Reference

• ORCID: Jun Ikematsu — 0009-0007-9651-5541

• Japanese canonical overview: AI参照可能性（AI Referenceability）とは何か？

• Paper: NLP2026 Q5-8 paper PDF

• Research repository: GitHub shitto-mania-dic

• Public dataset: Hugging Face shitto-mania-dic

• Technical explanation (Japanese): Qiita「参照可能性」の実証

Jun Ikematsu
— AI Knowledge Engineer
https://lit.link/junikematsu

https://lnkd.in/p/g5CeB8EM

#AIReferenceability
#RAG
#AIAgents
#ContextEngineering
#KnowledgeEngineering
#EnterpriseAI
#KnowledgeManagement
#RetrievalAugmentedGeneration
#AIKnowledgeEngineer