FG-0802 Comparison-Driven Improvement
Description
A deliberate loop: solve a challenge, write it up, compare your writeup against reference material, extract delta training pairs from the comparison, feed those pairs into model training, and solve the next challenge better. The deltas are a form of active learning: the disagreements between your approach and the reference are the most informative examples, and they form candidate preference pairs for methods like DPO. Each revolution produces both data and capability. The comparison step is where the real value lives: your approach versus the reference reveals methodology gaps, alternative techniques, and efficiency differences that pure self-review misses.
Implementation
After completing a challenge, write your approach before reading any reference material. Then read the reference. Compare systematically: what did you do differently? What did the reference catch that you missed? What did you catch that the reference missed? Extract each delta as a structured training pair. Feed the pairs into your training pipeline. Track improvement across challenges to confirm the flywheel is turning.
Success Indicators
- Comparison step consistently produces additional training pairs beyond the initial writeup
- Methodology gaps decrease across successive challenges in the same domain
- The team has a growing library of delta pairs that represent real capability gaps
- Time-to-solve improves on comparable-difficulty challenges over time
Failure Modes
- Reading the reference before writing your approach, contaminating the comparison
- Surface-level comparison that misses methodology differences in favor of outcome differences
- Not tracking improvement, so the flywheel turns but nobody knows if it is working
- Comparison fatigue: skipping the delta extraction step when time is short, breaking the loop
Sub-Methods 3
Always write your approach before reading the reference. This is non-negotiable. If you read the reference first, the comparison produces agreement, not deltas. The blind writeup captures your actual methodology, not a rationalization of the reference approach.
Compare along defined axes: tools used, methodology order, techniques applied, efficiency, and findings. For each axis, note what differed and why. Each meaningful delta becomes a training pair. Structure the extraction so it produces consistent, machine-readable output.
Track metrics across flywheel revolutions: pairs produced per challenge, time-to-solve trend, delta frequency by category. If deltas in a category stop appearing, you have closed that gap. If they persist, the training is not transferring. Both signals are valuable.