An assistant recommends a show that is not available in the user's region. Somebody suggests fine-tuning the model. Somebody else suggests RAG.
I would ask to see the failed answer first.
If the assistant did not have the current catalogue, we have a knowledge problem. If it had the catalogue and ignored the required response format, we may have a behaviour problem. Those failures call for different investigations.
RAG and fine-tuning can be useful together. Neither should become the default answer before you know what needs to improve.
What RAG adds
Retrieval-augmented generation gives a model relevant information at response time. That might be approved training material, product documentation, or catalogue information retrieved for the user's question.
It is attractive when the answer depends on information that is private, specialised, or changes frequently. But retrieving a passage does not guarantee that the model uses it correctly.
You still have to check what was retrieved, whether it was relevant, whether it was current, and whether the response was actually supported by it. Retrieval can fail before generation even starts.
What fine-tuning changes
Supervised fine-tuning uses example inputs and desired outputs to adjust a model's task-specific behaviour. It can help with things such as consistent formatting or a response style that is difficult to obtain reliably from prompting alone. OpenAI recommends establishing evaluations before investing in it. 1
It is not a reliable replacement for fetching today's availability or enforcing user access. A model trained on yesterday's catalogue does not become today's source of truth.
OpenAI's accuracy guide distinguishes context optimisation from behaviour optimisation and describes approaches that can be combined, rather than a mandatory sequence of prompt, RAG, and fine-tuning. 2
A fictional OTT assistant example
Imagine a streaming discovery assistant that must suggest relevant titles, avoid spoilers, and respect regional availability.
| Observed failure | First investigation | Possible response |
|---|---|---|
| Recommends an unavailable title | Did it receive authoritative availability? | Query the catalogue service at request time |
| Describes the wrong title | Was the correct item retrieved and identified? | Improve retrieval and item matching |
| Reveals spoilers despite clear instructions | Did the instructions and examples work on held-out cases? | Improve prompting; evaluate behaviour tuning if justified |
| Exposes a restricted item | Was access filtered in application logic? | Fix authorisation, not just model wording |
| Produces inconsistent output fields | Is the requested format enforced and tested? | Evaluate structured output and validation before training |
The table is a diagnostic starting point. It does not prescribe a particular vendor or architecture.
Compare candidates on the same questions
I would keep a baseline before adding more machinery. Try the simplest workable prompt and context setup, then compare alternatives on a held-out set.
For the OTT example, use questions about current availability, ambiguous titles, spoiler-sensitive requests, and unsupported queries. Score whether the answer is useful and supported, not only whether it sounds natural.
Record latency, operational complexity, and cost alongside answer quality. A retrieval-backed system may need content ingestion and freshness checks. A fine-tuned approach may need maintained training examples and model-version evaluation. Both create work beyond the initial demo.
Do not put permissions in the model's hands
“Only show authorised content” is a useful instruction. It is not an access-control system.
Application logic must determine what information the caller can access before that information reaches the model. Similar care applies to logging, retained prompts, and training data. Use authorised material and the relevant privacy requirements.
This matters whether you choose RAG, fine-tuning, both, or neither. Architecture terminology can distract from the ordinary product responsibility of keeping the wrong data out of the wrong conversation.
Choose the smallest change that fixes the measured failure
If a better source lookup solves the availability problem, that is progress. You do not need a more elaborate approach to make the product sound sophisticated.
If the model's behaviour is still unreliable after clear instructions and representative examples, evaluate the next intervention. Make the case with failed examples and a repeatable test, not with enthusiasm for a technique.
I would want the final decision to read: “We changed this because these cases failed, and this version improved them without unacceptable trade-offs.” That is more useful than “we are now using RAG.”