RAG as a Rewrite-or-Repeat Discipline
January 2, 2024

One practical RAG discipline is a two-way decision: rewrite the existing answer when retrieved context changes it, or repeat the answer when the context adds nothing useful.

The value is not the acronym. The value is the control rule: retrieval should update the answer only when it actually carries new evidence.

RAG noteOriginally posted 2024-01-02; expanded here around the rewrite-or-repeat rule.

Related: Four-Layer Tiny Transformer Training Run

Article focus: retrieval, answer refinement, rewrite/repeat behavior, and context usefulness.

The Rule

The original prompt constrained answer refinement to two outcomes. If the new context is useful, rewrite the answer using it. If the new context is not useful, repeat the original answer.

That keeps retrieval from becoming automatic contamination. A retrieved passage has to earn its way into the answer.

The Small Test

The prompt example used a simple fact: "Three apples were green." The query asked what color the apples were. The model's job was not to discuss context. It was to produce the corrected answer when the context answered the query.

The same discipline scales to larger RAG systems: context should either change the answer for a reason or leave the answer alone.

https://lnkd.in/g6CGSTuH

Links From the Original Post