Open Model Lab
Reasoning Behavior + Process Evaluation
Evaluate not only final answers, but where reasoning processes break down.
Gate status
- Target
- April 2027
- Status
- planned
- Report
- Results pending
Success criterion
The system can show flawed processes despite correct answers, or identify breakpoints leading to wrong answers.
Focus
- Outcome-based grading vs process-based grading.
- Math, logic, code reasoning, and planning tasks.
- Self-correction.
- Uncertainty signals.
- Final-answer vs reasoning-path comparison.
Expected outputs
- reasoning_evals module.
- Final answer vs reasoning path comparison.
- Report: why outcome scores are not enough.
Stage completion decision
Do process signals add useful diagnostic value beyond final-answer grading?