Atlas · GenAI 2026
Model Merging
Model merging (SLERP, TIES, DARE)
conceptPeak: 2024Model CompositionAI consensus: 1/3
Prerequisites
- hardLinear Algebra
Model merging operates directly on weight tensors using interpolation algorithms — it's pure applied linear algebra
- mediumLoRA / QLoRA
Model merging often combines LoRA adapters or specialized fine-tunes — understanding PEFT helps understand what is being merged
Recommended reference
Yadav et al. (2023) 'TIES-Merging: Resolving Interference When Merging Models' — NeurIPS; plus Yu et al. (2024) 'Language Models are Super Mario' (DARE paper)
Notes from AI deep research
Anthropic Opus
SLERP, TIES, DARE — laczenie wag bez treningu. Zero GPU cost. Ciekawy research direction
Google Deep Think
Łączenie wag bez trenowania [G#68]
Related skills
- → is subcategory of: Model Fine-Tuning(3/3)
- → is subcategory of: LLM Fine-Tuning(1/3)