Atlas · GenAI 2026
Continual Pre-Training
Continuous Pre-Training (CPT)
conceptPeak: 2025Pre-Training & AdaptationAI consensus: 1/3
Prerequisites
CPT extends pretraining on domain data — you must understand pretraining (next-token prediction on a Transformer) to extend it
- mediumDistributed Training
CPT on domain data often requires multi-GPU training — distributed training skills become practical requirements
Recommended reference
Gupta et al. (2023) 'Continual Pre-Training of Large Language Models: How to (Re)warm Your Model?' — practical guide to CPT without catastrophic forgetting
Notes from AI deep research
Anthropic Opus
Dotrenowywanie rdzenia na danych domenowych. Silniejszy niz RAG dla deep domain knowledge
Google Deep Think
Dotrenowywanie na żargonie firmowym [G#65]
Related skills
- → is subcategory of: Model Fine-Tuning(3/3)
- → is subcategory of: LLM Fine-Tuning(1/3)