Atlas · GenAI 2026

LLM Fine-Tuning

Supervised Fine-Tuning (SFT)

conceptPeak: 2023Fine-TuningAI consensus: 3/3

Prerequisites

hardDeep Learning
SFT is supervised training of a neural network — you need DL fundamentals (loss functions, learning rate, overfitting) to do it well
hardTransformer Architecture
SFT trains a Transformer model — understanding the architecture is essential for diagnosing training issues
hardTraining Data Curation
SFT quality is determined by data quality — dataset curation is the gating factor for fine-tuning success

Raschka, S. (2025) 'Build a Large Language Model (From Scratch)' — Manning; Ch.6-7 on SFT. Also: philschmid.de 'How to fine-tune open LLMs in 2025'

Anthropic Opus

SFT to fundament. Raschka (2025) + philschmid.de guide. Wymaga dyscypliny, nie jest trudne

OpenAI Deep Research

Dopasowanie do stylu organizacji [OA#14]

Google Deep Think

Formatowanie danych [G#61]

← is subcategory of: Direct Preference Optimization(3/3)
← is part of: Distributed Training(3/3)
← is part of: Fine-Tuning Evaluation(3/3)
← is an instance of: Hugging Face PEFT(3/3)
→ is subcategory of: Model Fine-Tuning(3/3)
← is subcategory of: LoRA / QLoRA(3/3)
← is subcategory of: RLHF(3/3)
← is subcategory of: Supervised Fine-Tuning (SFT)(3/3)
← is part of: Synthetic Data Generation(3/3)
← is part of: Training Data Curation(3/3)
← is an instance of: Hugging Face(2/3)
← is part of: Catastrophic Forgetting(1/3)
← is subcategory of: Continual Pre-Training(1/3)
← is subcategory of: Knowledge Distillation(1/3)
← is subcategory of: Model Merging(1/3)
← is subcategory of: Model Quantization(1/3)
← is an instance of: Unsloth(0/3)