Everybody doing AI these days is well familiar with Retrieval Augmented Generation (RAG), a common method for adding relevant context to the LLM based on the user query.

Back in 2014, we were working on domain adaptation for statistical machine translation, arguably one of the best-studied generation tasks in NLP. The goal was to find techniques to make translation models best-adapted to target domains, while keeping them as small as possible.

One of the methods we proposed was taking the input text at test time, querying a large parallel corpus for relevant examples, and then training a customized translation model augmented with that retrieved data.

There you have it: RAG in 2014, way before the name was even coined.

Check out the details on our algorithm in Chapter 5 of the paper:

Shachar Mirkin and Laurent Besacier. Data Selection for Compact Adapted SMT Models. AMTA 2014.