Atlas · GenAI 2026
Dataset Engineering
Dataset design & leakage prevention
conceptPeak: 2020Dataset CurationAI consensus: 1/3
Prerequisites
- mediumModel Evaluation
Understanding evaluation metrics is needed to recognize when leakage artificially inflates them
Recommended reference
Kaufman, S. et al. (2012) 'Leakage in Data Mining' — ACM TKDD; seminal paper; updated by Kapoor & Narayanan (2023) 'Leakage and the Reproducibility Crisis in ML'
Notes from AI deep research
Anthropic Opus
Kapoor & Narayanan (2023) pokazali powszechnosc leakage. W LLM eval: test set contamination jest epidemia
OpenAI Deep Research
W ewaluacjach LLM-as-judge łatwo o 'przeciek' [OA#8]