Open Model Lab
Data Efficiency + Scaling Ladder
Measure whether better data mixtures produce better behavior with the same compute.
Gate status
- Target
- August 2027
- Status
- planned
- Report
- Results pending
Success criterion
The effect of data quality on behavior and compute efficiency is quantified.
Focus
- Data mixture.
- Quality filtering.
- Synthetic data.
- Contamination control.
- Small model ladder: 30M, 70M, 150M, optional 350M.
- Score/GPU-hour.
Expected outputs
- data_scaling module.
- Data mixture comparison report.
- Report: which data mixture produces the most behavior gain per GPU-hour?
Stage completion decision
Does data quality improve score/GPU-hour enough to justify the filtering pipeline?