Synthetic data flywheel

A synthetic data flywheel is a loop in which one model generates training data for another, that model produces better data, and the cycle repeats, improving quality at lower cost. In practice, teacher-student distillation dominates. The concept is in tension with the model collapse hypothesis of Shumailov et al.