What is RAG?

The term comes from a 2020 paper by Patrick Lewis and colleagues, posted to arXiv in May 2020 and accepted at NeurIPS 2020. The authors start from a limitation: large pretrained language models keep factual knowledge in their weights, but their ability to access and precisely manipulate it is limited. The paper also lists giving provenance for outputs and updating world knowledge as open problems.

Their proposal pairs two kinds of memory. Parametric memory is a pretrained sequence-to-sequence model whose knowledge lives in its parameters. Non-parametric memory is an external store, in the paper a dense vector index of Wikipedia, searched by a pretrained neural retriever. The retriever finds passages, and the generator conditions on them while writing its answer. The paper compares one variant that uses the same retrieved passages for the whole output with another that can use different passages for each token.

The authors report strong results on three open-domain question answering tasks and say the generated language was more specific, diverse and factual than a parametric-only baseline. Those results are specific to that paper's models and datasets. The abstract does not prescribe a particular retriever or index beyond the dense vector index it describes.