RAG or fine-tuning? Choosing for your own company data

There are two ways to make a model use your knowledge: you retrieve the right pieces at the moment of the question, or you train the model further on your own material. They solve different problems, and get confused constantly.

What the difference means in practice

As soon as you must point at the source, or access applies per user, fine-tuning drops out.

Retrieval at question time means new knowledge is available immediately and that you can say where every answer came from. Fine-tuning bakes knowledge into the model weights: the source disappears, and a correction requires another training round.

For company numbers, stock levels, contracts and procedures that difference is decisive. Those change, and someone has to be able to retell what an answer rested on.

Fine-tuning is useful for form: a fixed tone, a fixed format, a professional vocabulary the model should use consistently. Form, not facts.

The retrieval chain

Access rights apply during search, not only at display time. Meaning travels with the source.

The step missing most often is the third: sending meaning along. A model that finds three tables with revenue does not know which one is your revenue. If the definition and the owner come with it, it does not choose for itself but follows your agreement. See what is data governance.

When to choose which

• Facts that change: retrieval. Stock, orders, margins, prices.
• Documents with permissions: retrieval, with access per user in the search step.
• Fixed tone or format: fine-tuning, on a small, well chosen set.
• Both: normal. Fine-tune for form, retrieve for facts.

What you need either way is a layer where your data and definitions are already right. See implementing AI and what is a data platform.

In short

For company data, retrieval at question time is almost always the answer, because you must be able to point at the source and enforce access. Fine-tune for form, not for truth.