How to fine-tune a model on your own PC

Two years ago, fine-tuning a language model meant renting a cluster. Today you can do it on the gaming laptop you already own, or a Mac with 16 GB of memory, in an afternoon. Nothing gets uploaded anywhere, and when it is done you have a small model that does one job your way, running offline.

This page walks through it the way you would actually do it the first time, with one running example: a small model that turns messy bug reports into tidy tickets in your team's format. You paste in "login broken again on android, cant click the button after otp, happened twice today", and it gives back a title, steps to reproduce, the affected platform and a severity, the way your team writes them.

You do not need to be a machine learning engineer. You need a reasonable GPU or a recent Mac, a few hundred good examples, and a bit of patience.

First, check you need it#

Fine-tuning teaches a model a behaviour: a format, a tone, a narrow task it should do the same way every time. It is bad at teaching facts. If you want a model that knows your company's refund policy, put the policy in the prompt or use retrieval. If you want a model that always writes tickets in exactly your format, from any mess of input, fine-tuning is the right tool.

Before you train anything, spend fifteen minutes on this test. Take a decent small model, write a clear prompt with your format and two or three examples, and try it on twenty real bug reports. Three things can happen:

• It works almost every time. Stop here. A good prompt is cheaper than a fine-tune, and easier to change.
• It works most of the time but drifts: forgets a field, invents severities, gets wordy on long input. This is exactly what fine-tuning fixes. Carry on.
• It is wrong in ways that need knowledge, such as which team owns which feature. Fine-tuning will not fix that reliably. Put that knowledge in the prompt.

Keep those twenty reports. They are the start of your test set, and you will want them at the end.