Fine-Tuning

The mechanism is ordinary supervised learning applied to an already-capable model: show input–output pairs, compute loss against the desired output, update weights. What changes is the prior — after training, the behaviour you demonstrated becomes what the model does without being asked. That is why fine-tuning is strong for form and weak for facts. Tone, structure, a house JSON shape, a domain’s conventions, a classification boundary that is hard to describe in words: these are all shape, and they transfer well.

Knowledge is the wrong target. Facts baked into weights cannot be updated without retraining, cannot be cited, and cannot be removed on request. Retrieval handles the same requirement with a source link and an edit path. The practical decision rule is close to mechanical: if the failing behaviour would be fixed by the model *knowing* something, retrieve it; if it would be fixed by the model *behaving* differently every single time, consider fine-tuning.

The cost people underestimate is not the training run — it is the dataset and the evaluation. A few hundred genuinely correct, consistent examples outperform tens of thousands of scraped ones, and without a held-out evaluation set you cannot tell whether the tune helped or simply narrowed the model. You also inherit an ongoing obligation: every base-model upgrade means re-running the pipeline and re-validating the result.