Fine-Tuning LLMs: When and How to Customize a Model

Welcome to Level 6 — the advanced, hands-on track. You already know two ways to shape an LLM's behaviour: prompt engineering (ask better) and RAG (give it your documents). There's a third, more powerful — and more involved — option: fine-tuning, where you actually train the model itself on your examples. It's how you get a model to reliably adopt a specific style, format, or skill. This lesson explains what it is, when it's worth it (often it isn't!), and how modern techniques like LoRA make it surprisingly affordable.

What is fine-tuning?

A base LLM (Lesson 20) is a generalist, trained on the whole internet. Fine-tuning continues its training on a smaller, curated set of your own examples, nudging its weights so it behaves the way you want by default — without you having to spell it out in every prompt. Think of the base model as a talented graduate, and fine-tuning as on-the-job training for your specific role.

The three ways to customize an LLM

Before you fine-tune, make sure it's actually the right tool. Most problems are solved more cheaply by the first two options.

When to fine-tune (and when not to)

Fine-tune when: you need a consistent voice or output format the model won't reliably follow from prompts alone; you have a specialised task (e.g. classifying support tickets a particular way); or you want a smaller, cheaper model to match a bigger one on a narrow job.

Don't fine-tune when: you mainly need up-to-date facts (use RAG — fine-tuning bakes knowledge in and goes stale); you have very little data; or you haven't yet exhausted good prompting. The classic mistake is reaching for fine-tuning first — it's usually the last resort, not the first.

Rule of thumb: RAG teaches the model what to know; fine-tuning teaches it how to behave. Need facts? RAG. Need a style or skill? Fine-tune.

Full fine-tuning vs LoRA

Full fine-tuning updates every weight in the model. For a model with billions of parameters that's slow, needs expensive GPUs, and produces a huge new copy of the model. That's why almost everyone now uses parameter-efficient fine-tuning (PEFT) — and the most popular method is LoRA (Low-Rank Adaptation).

LoRA's trick: freeze the giant base model and train only a tiny pair of "adapter" matrices added alongside it. You end up training a fraction of a percent of the parameters, getting most of the benefit at a tiny fraction of the cost — small enough to fine-tune on a single modest GPU.

See why LoRA is so cheap (runnable)

The whole idea is that a big weight update can be approximated by two small matrices multiplied together. This NumPy snippet counts the parameters — the saving is dramatic.

import numpy as np

d = 1000          # a 1000 x 1000 weight matrix
r = 8             # LoRA rank (small!)

# Full fine-tuning updates the whole matrix:
full_params = d * d

# LoRA trains two thin matrices A (d x r) and B (r x d):
A = np.random.randn(d, r)
B = np.random.randn(r, d)
lora_params = A.size + B.size

# A @ B has the SAME shape as the full update (d x d)...
print("update shape:", (A @ B).shape)
print("full params: ", full_params)      # 1,000,000
print("LoRA params: ", lora_params)      # 16,000
print("trained: {:.1%}".format(lora_params / full_params))  # 1.6%

Same-shaped update, ~1.6% of the parameters. That's why LoRA turned fine-tuning from a data-centre job into something a hobbyist can do.

Preparing your data

Fine-tuning is only as good as your examples. You typically provide hundreds to thousands of high-quality demonstrations of the behaviour you want, usually as JSONL (one JSON example per line):

// each line = one training example (prompt -> ideal answer)
{"messages":[{"role":"user","content":"Summarize: The launch went well..."},
             {"role":"assistant","content":"✅ Launch: success. Next: monitor uptime."}]}
{"messages":[{"role":"user","content":"Summarize: Server crashed twice today..."},
             {"role":"assistant","content":"⚠️ Incident: 2 crashes. Next: root-cause review."}]}

Notice both answers follow the same style — an emoji, a label, a "Next:" line. Show the model that pattern a few hundred times and it learns to produce it automatically. Quality and consistency beat quantity.

The fine-tuning process, and what code looks like

The typical flow: prepare and split your data → pick a base model → run training (often LoRA) → evaluate on held-out examples → deploy. With libraries like Hugging Face transformers + peft, the core is only a few lines (conceptual sketch; needs those libraries and a GPU):

from peft import LoraConfig, get_peft_model

lora = LoraConfig(r=8, lora_alpha=16, target_modules=["q_proj","v_proj"])
model = get_peft_model(base_model, lora)   # wrap the frozen base

model.print_trainable_parameters()          # -> ~0.1-1% trainable
# ...then train on your JSONL data with a normal training loop.

Many providers also offer managed fine-tuning — you upload the JSONL, they handle the GPUs and return a custom model endpoint. That's the easiest on-ramp if you don't want to manage infrastructure.

Costs and pitfalls

• It goes stale: knowledge baked in at training time won't update — that's RAG's job.
• Catastrophic forgetting: over-training on narrow data can make the model worse at general tasks.
• Overfitting: too few or repetitive examples and it memorises instead of generalising (remember Lesson 12).
• Cost & effort: data prep and evaluation are real work — make sure the payoff justifies it.

The habit that saved me the most time in real projects was resisting the urge to fine-tune first — nine times out of ten, a better prompt or a RAG setup solved the problem in an afternoon, where fine-tuning would have cost me days. When I finally did fine-tune, it was for a stubborn formatting requirement prompts couldn't nail, and that's when it earned its keep. My advice: treat fine-tuning as a deliberate choice, not a default.

Key takeaways

• Fine-tuning continues an LLM's training on your examples to change how it behaves by default.
• Customize in order of effort: prompt → RAG → fine-tune. Most problems don't need fine-tuning.
• RAG for knowledge, fine-tuning for style/skill — fine-tuned knowledge goes stale.
• LoRA/PEFT freezes the base model and trains tiny adapters — ~1% of the parameters, hugely cheaper.
• Success depends on high-quality, consistent JSONL data; watch for forgetting, overfitting and cost.

Continue the series: ← Lesson 28: The Future of AI  ·  Next: Lesson 30 — Deploying AI Models: From Notebook to Production (MLOps) →