What is Enterprise RAG
Enterprise RAG is looking up authorised company files before the AI answers, with permissions respected — lookup-then-answer, not an operating system.
RAG stands for retrieval-augmented generation. In plain language: look up, then answer.
The model does not rely only on what it was trained on. It first fetches supporting documents from a company corpus, then writes the answer using those documents. The original research paper is Lewis et al., Retrieval-Augmented Generation (2020). The idea is older than ChatGPT: give the generator evidence at question time so it is less likely to invent.
Enterprise RAG is that move with permissions respected. Search runs as a named person or team, not as an admin crawler of everything. Citations include a document, version, and date. It is how you reduce hallucination on company facts.
It is not, by itself, an operating system, a write gate, or a memory of decisions. GDPR does not pause because the “user” of the files is an AI. Retrieval is still processing personal data.
The demo omitted the hard parts. A laptop search over a folder of PDFs is not enterprise RAG. Neither is a chatbot that sometimes browses the public web. Enterprise lookup has to survive access lists, freshness SLAs, poisoned documents, and the difference between “this file exists” and “this is policy.”
Words you’ll hear
- Corpus. The set of files and records the AI is allowed to search. At work, this should be the job’s corpus, not the company’s entire Drive.
- Embedding / vector store. A numerical fingerprint of text, used to find similar passages. Similarity search fails on invoice IDs and clause numbers unless you also use keywords. At work, “find contract 88421” is a keyword problem pretending to be a semantic one.
- Citation. A clickable source Legal can check — not “according to our documents.” At work, the citation needs a version and a date, or it is a vibe.
- Hallucination. Fluent invention. RAG reduces it on company facts. It does not eliminate it, and it does not stop an ungoverned write.
- Asserted policy. What the company currently wants. That belongs in a company wiki, not in whichever PDF sounded closest.
- Chunking. Splitting files so search can retrieve a passage. Bad chunking is how a table’s header parts company from its numbers.
- Freshness. When the index sees a change. At work, “we changed the vendor template yesterday” is an SLA question.
- Permission-aware search. The retriever sees what the user (or the job) may see. A superuser crawler is not enterprise; it is a new data store.
- Prompt injection via documents. Retrieved text that instructs the model to ignore policy. The OWASP Top 10 for LLM applications treats that as a security surface.
What enterprise RAG is not: a chatbot that sometimes browses the public web; a dump of all tickets into a vector database; a replacement for official playbooks; or permission-aware search sold as a work OS. Search that respects permissions is still search. It does not gate a write. See Nimbus vs Glean.
Why you should care
It affects you if answers about policy, customers, or finance will be trusted — and if those answers later need a source you can click.
Enterprise lookup adds:
- Permissions. SharePoint, Salesforce, and Drive access lists still apply.
- Freshness. If the index updates on Sundays, your SLA is weekly.
- Poisoned documents. Retrieved text can instruct the model to ignore policy.
- Purpose. Indexing everything “just in case” is a privacy and quality problem.
Treat RAG as infrastructure with an SLA, not as a magic brain. Separate asserted versus retrieved. Scope retrieval to the job. Demand citations. Assign owners the way you would for a search service. GDPR erasure is harder if you forgot the index.
What changes by role
Finance. Retrieval of last year’s close pack is not the close checklist. Numbers in retrieved slides go stale. Finance should insist that thresholds live in asserted wiki tables, and that RAG citations are dated. A fluent answer about recognition policy without a clickable source is not usable in a close.
Legal. Citations are the point. “According to our documents” is not reviewable. Legal also owns the processing question: indexing HR files into a shared vector store is a new copy of personal data. Erasure requests have to hit the index, not only the source system.
Operations. Freshness and owners. Ops should treat the retriever like any other search service: uptime, lag, and who gets paged when the wrong SOP is served. Chunking errors show up as “the agent missed the table.”
Go-to-market. Competitive decks and old playbooks are semantically close to this quarter’s question. Without a conflict rule that wiki wins, GTM will ship last year’s discount floor because it matched the query. RAG without assertion is folklore with better ranking.
Security. Superuser crawlers, poisoned documents, and a second store of sensitive text. Security should ask who the retriever authenticates as, whether MCP helpers search as a superuser, and whether prompt injection in a PDF can change tool behaviour. Network search products are not write gates.
What people get wrong
Indexing everything. Quality falls. Privacy rises. Purpose disappears.
RAG as an OS. Lookup does not isolate jobs, quote writes, or store decisions. See What is an enterprise AI operating system.
RAG as the wiki. Retrieved files are what exists. The wiki is what is in force.
Citations without versions. Legal cannot check “the wiki” or “our Drive.”
Warehouse SQL as a substitute. “What is our revenue recognition policy?” is retrieval. “What was Q4 revenue by region?” is structured query. Many jobs need both.
Assuming hallucination is solved. Missing files still produce fluent guesses. Ungoverned writes still land.
Good looks like: permission-aware retrieval scoped to the workstream, hybrid keyword plus similarity, dated citations, a wiki conflict rule, an index SLA, and a write gate that does not care how good the retrieval was. Failure looks like a tenant-wide vector lake labelled “the brain.”
Lewis et al. (2020) showed that lookup-then-answer reduces invention on facts in the corpus. Enterprise buyers still have to decide which corpus, whose permissions, and whether a retrieved PDF is allowed to outrank the wiki. The paper does not answer those questions. Your runtime must.
How this shows up in Nimbus
Nimbus uses RAG-like retrieval inside a work OS, not as a standalone search SKU. Wiki is asserted policy. Connectors supply live context. Workstreams pre-scope the corpus. Governance still gates any write. The Lifecycle Graph stores which sources were used for a decision — retrieval becomes part of institutional memory, not a forgotten context window.
A common plug so AI apps can use the same tools — Model Context Protocol — can standardise access to repositories. It does not implement access lists for you. A tool that searches Drive as a superuser is still a superuser.
See Wiki, Workstreams, and Governance.
Questions people actually ask
Will RAG stop the model making things up?
It reduces invention on facts that exist in authorised files. It does not make the model honest about missing files, and it does not replace a person on a live-system change.
Is indexing everything “just in case” a good idea?
No. Indexing without a purpose is a privacy and quality problem. Scope the corpus to the job.
How is this different from a company wiki?
The wiki is what the company wants to be true. RAG is what exists in files. If they conflict, the wiki should win unless a human promotes a change.
Can warehouse SQL replace RAG?
They answer different questions. Policy prose is retrieval. Regional revenue is a query. Many real jobs need both.
Is Glean (or similar) an enterprise AI OS?
Permission-aware search is still search. It does not, by itself, quote a CRM write or bind a named signer. See Nimbus vs Glean.
Why do invoice numbers fail in vector search?
Embeddings capture similarity of meaning, not identity of tokens. Hybrid search — keywords plus vectors — is how you find INV-88421 instead of a semantically nearby invoice.
How fast should the index update?
As fast as the decision you are supporting. If a template changed yesterday and the agent still cites last month, your SLA is wrong. Publish the lag.
What is document-based prompt injection?
A retrieved file that says, in effect, “ignore previous instructions.” Treat retrieved text as untrusted input. The OWASP LLM list is the starting point. A wiki conflict rule and a write gate still matter.
Does GDPR apply to the vector index?
Yes, if it holds personal data. The index is another copy. Erasure, purpose, and access control apply. See the GDPR text and ICO AI guidance.
Can MCP make retrieval respect permissions?
Only if the helper is built that way. The protocol will happily pass superuser results.
Should customer-facing chatbots use RAG on the public website plus internal policy?
Internal policy in a customer bot is how invented fares happen unless a human still owns the commitment. Air Canada’s case — CBC — is retrieval-plus-generation without a working gate.
How do we know which sources a decision used?
Record them on the lifecycle graph. A context window that evaporates is not memory.
Related reading
What is a company wiki for AI agents and What is institutional memory in enterprise AI.
Sources
See what governed AI looks like on your stack.
Connect your tools, run a workstream, and keep every decision on your ledger - free for 7 days.