Atlas · GenAI 2026
LoRA / QLoRA
PEFT / LoRA / QLoRA / DoRA
conceptPeak: 2024Parameter-Efficient Fine-TuningAI consensus: 3/3
Prerequisites
- hardLLM Fine-Tuning
LoRA/QLoRA are parameter-efficient alternatives to full SFT — you must understand what full fine-tuning does before learning to approximate it cheaply
- mediumLinear Algebra
LoRA factorizes weight updates into low-rank matrices (W = BA where B,A are low-rank) — linear algebra explains why this works
Recommended reference
Hu et al. (2022) 'LoRA: Low-Rank Adaptation of Large Language Models' — ICLR; foundational paper. Plus Dettmers et al. (2023) 'QLoRA' paper for 4-bit variant
Notes from AI deep research
Anthropic Opus
Hu (2022) LoRA + Dettmers (2023) QLoRA. Most cited GenAI skill. 7B na 16GB GPU
OpenAI Deep Research
LoRA [OA#15] + QLoRA [OA#16]
Google Deep Think
Adaptery zdejmujące 90% kosztów [G#62]
Related skills
- → is subcategory of: LLM Fine-Tuning(3/3)
- ← is an instance of: Hugging Face PEFT(2/3)
- → is subcategory of: Transfer Learning(1/3)