Glossary · term

Benchmaxxing

The process in which labs build synthetic data near benchmark distributions, cultivating "intelligence spikes" that cover the test points. "Training on the test set" has become "a new art form." As a result, benchmarks lose their credibility as a proxy for general capabilities.

Karpathy2025Wave 1 · 2023Maturity: 2/5

Maturity rationale

Benchmaxxing — critical term in circulation

References

Author: Andrej Karpathy