synthetic-data-generation
verified4f208c66-421e-4193-b6d7-974adeb95600
Generate high-quality synthetic training and eval data for LLMs — Self-Instruct and Evol-Instruct, distillation from a teacher, quality filtering (dedup/length/judge), and JSONL output for TRL/Unsloth fine-tuning.
Metadata
Skill file
# Synthetic Data Generation for LLMs
Use when you need more (or better) training, alignment, or eval data than you can
afford to label by hand. Synthetic data is model-generated data that replaces or
augments scarce, expensive, or privacy-sensitive human labels. The shape is
always the same: *small real seed → frontier teacher → quality filter → JSONL*.
## The main techniques
- **Self-Instruct** — start with a small seed set (often ~150–200 human-written
tasks), ask the teacher to expand them into thousands of new instructions, then
generate a response for each. Ask for novel/diverse task types so you don't
just paraphrase the seeds.
- **Evol-Instruct** — iteratively mutate ("evolve") existing instructions to
higher complexity/depth (used by WizardLM), producing harder, deeper training
tasks.
- **Distillation** — use a frontier model as a *teacher* and transcribe its
outputs (often with chain-of-thought traces) into the training set for a
*student* model (Orca-style). This transfers capability at lower inference cost.
- **Constitutional AI** — generate safety/refusal data by having the model
critique and revise using a set of principles.
- **Domain-specific generation**: function-calling traces for tool-using agents,
RAG QA pairs grounded in your corpus, preference pairs (DPO/IPO) for alignment,
and code + test pairs for coding models. These work best when the target domain
is machine-verifiable.
## Core recipe
1. **Seed.** Collect a small, real, high-quality set (human or from reliable
sources). This is the only part you should never skimp on — seed quality caps
synthetic quality.
2. **Teacher.** Use the strongest model you can afford (frontier API or a big
open model). A weak teacher bakes in weak output.
3. **Generate with structure.** Prompt for a specific output schema (JSONL row),
and for reasoning tasks ask for chain-of-thought first, then the answer.
Produce far more examples than you need and over-sample for diversity.
4. **Filter for quality.** This is the step that separates good data from garbage:
- **Deduplication** — remove near-duplicate instructions (embedding similarity
threshold) that reduce diversity and cause overfitting.
- **Length filtering** — drop very short (likely low-quality) and very long
(likely hallucinated/padded) responses.
- **Quality judging** — use a second LLM to rate each response for relevance
to the instruction, correctness, and helpfulness; keep the top slice.
- **Correctness checking** — for verifiable tasks, check answers against a
knowledge base or tests/critic model.
- **Toxicity/safety filtering** — remove harmful output before it trains harm.
- **Diversity sampling** — keep coverage across task types, not one cluster.
5. **Write JSONL** — the standard `{"instruction", "input", "output"}` (and
optionally `"chosen"/"rejected"` for preference data) format that loads
directly into TRL, Unsloth, or your trainer. Split off a held-out slice as
your eval set.
## Practical levers
- **Generate 2–5x what you'll train on**, filter down. More candidates + stricter
filtering beats "exactly N good ones".
- **Match the distribution to deployment.** If your model will answer customer
tickets, seed with real ticket shapes and styles, not generic trivia.
- **Check coverage:** embed your generated set and confirm it spans your seed
categories (a coverage report beats vibes).
- **Lock reproducibility:** record teacher model + version, temperature, and
prompt template in a config file; version the generated dataset in git like
code. (` datasets.Name` hashes help.)
- **Keep a real eval set.** Never eval on synthetic data alone — hold out some
*human* examples so you aren't grading on metrics your generator optimizes.
## Pitfalls
- **Garbage-in amplification** — a single bad seed can spiral; curate seeds
carefully.
- **Skipping quality filtering** — raw teacher output includes duplicates,
hallucinations, and off-task rows that tank fine-tune quality.
- **Unbalanced distribution** — generating lots of easy generic instructions
produces a model good at trivia and bad at your domain.
- **Evaluating on synthetic-only data** — self-fulfilling and misleading.
- **No dedup** — a handful of duplicated high-value examples get overweighted and
cause the model to memorize rather than generalize.
- **Not shaping to your trainer's format** — produce JSONL exactly as TRL/Unsloth
expect, or schema-conversion bugs corrupt the run.
## Verify
- On a held-out *human* eval set, compare the fine-tuned model's pass rate vs a
no-synthetic baseline — the lift is the real win to report.
- Spot-check 20–50 generated rows by hand: are they grammatically clean, on-task,
and matching the required output schema? Measure a "clean rate".
- Confirm near-duplicate rate is low (embedding-similarity histogram) after dedup.
- Confirm the JSONL loads and trains without corruption (schema-validate first row
and run one short training step).