Key facts
| Definition | Continuing training on a base model with your own examples to change its behaviour |
| SFT | Supervised fine-tuning on input-output pairs; best for format, tone and narrow tasks |
| DPO | Preference optimization on chosen versus rejected answers; best for alignment and judgement |
| Data needs | SFT: consistent examples. DPO: paired comparisons often collected from user feedback |
| Alternatives | Prompt engineering, few-shot examples and RAG are live and usually the first step |
| Plugsky status | Fine-tuning is a coming-soon endpoint; chat, embeddings and RAG are live |
| Models | Base candidates and stronger models for data generation come from the same 30+ catalogue |
| Ongoing duty | Fine-tuned models need versioning, evaluation and a retraining plan |
TL;DR
- SFT teaches a task; DPO teaches a preference.
- Exhaust prompting and RAG before committing to training.
- Data consistency beats dataset size for SFT.
- DPO needs genuine preference signal, not synthetic noise.
- Plugsky fine-tuning is coming soon; build data with live endpoints now.
How it works, step by step
- Confirm the gap is behavioural — not missing knowledge that RAG would fix.
- Choose the method: SFT for task and format, DPO for judgement between answers.
- Build a representative dataset with held-out validation and test splits.
- Set an evaluation suite that measures the task, not just training loss.
- Generate or label data using a strong model through the chat API, then curate it.
- Plan versioning, rollback and a retraining cadence before deployment.
Try it yourself
Open the LLM token calculator →
SFT versus DPO
SFT is straightforward supervised learning: show the model good input-output pairs until it reproduces the pattern. It excels at format, tone, domain vocabulary and narrow task behaviour. DPO takes pairs — one preferred answer, one rejected — and adjusts the model to favour the preferred style of response. It is the better tool when the model already understands the task but makes poor judgement calls, such as being too verbose or too cautious.
When fine-tuning is the right tool
- Format compliance: strict output schemas where prompting is brittle.
- Domain language: specialist jargon the base model handles awkwardly.
- Behaviour shaping: tone, length or refusal patterns that must be consistent.
- Cost reduction: replacing a large prompted model with a tuned smaller one.
- Privacy of method: encoding a workflow into weights rather than a prompt you share.
Fine-tuning is the wrong fix for missing or changing facts — that is retrieval's job.
Common mistakes
- Fine-tuning to add facts the model could retrieve instead, causing staleness.
- Small, inconsistent datasets that teach noise and format drift.
- No held-out evaluation, so improvement is measured on training examples.
- Skipping DPO when the real problem is preference quality rather than task ability.
- Deploying without version tracking, making rollback impossible.
Fine-tuning and Plugsky
Fine-tuning is documented as a coming-soon endpoint, so Plugsky does not serve managed training today. The practical path now: use live chat completions with a strong model to draft and augment datasets, embeddings to deduplicate and select examples, and function calling to enforce output structure during generation. Those datasets transfer directly when managed training arrives. Until then, close most gaps with better prompts, few-shot examples and RAG, which are live and cheaper to iterate.
Honest comparison
| Method | SFT | DPO | Prompt plus RAG |
|---|---|---|---|
| Training signal | Input-output pairs | Preferred versus rejected pairs | None |
| Best for | New tasks and formats | Judgement and alignment | Knowledge and changing facts |
| Data effort | Consistent labeled examples | Paired preference labels | Documentation only |
| Iteration speed | Days to weeks | Days to weeks | Minutes |
| Plugsky status | Coming soon | Coming soon | Live |
Frequently asked questions
What is fine-tuning?
Continuing training on a base model with your own data to change how it behaves — its format, tone, task skill or judgement — rather than adding new facts to it.
What is the difference between SFT and DPO?
SFT learns from input-output examples and teaches tasks and formats. DPO learns from pairs of preferred and rejected answers and tunes judgement and alignment.
Should I fine-tune or use RAG?
Use RAG for knowledge that changes or must be cited; fine-tune for behaviour that should be consistent, such as format and tone. Many production systems use both.
How much data do I need?
It depends on task narrowness, but consistency and coverage matter more than raw size. Always hold back a validation split and evaluate on real inputs.
Can I fine-tune on Plugsky now?
Fine-tuning is a coming-soon endpoint. Today you can generate and curate datasets with live chat completions and embeddings, then use external training tooling.
What happens to fine-tuned models over time?
They freeze the behaviour at training time, so plan for versioning, periodic re-evaluation and retraining when base models or requirements change.