FG-0805 Feedback Capture Loop
Description
Improvement needs a signal, and the signal is feedback. The Feedback Capture Loop systematically records the human and automated judgments produced during normal work: corrections, approvals and rejections, preference between two outputs, and pass or fail against a verifier. Captured in a consistent format, these judgments become the training signal that drives the flywheels. These are the same kinds of signal that underlie supervised, preference-based, and reinforcement-style model training, gathered through daily practice rather than in a one-time labeling project.
Implementation
Identify the points in your workflow where a judgment is already being made: a human approving or rejecting output, a reviewer choosing between alternatives, a test passing or failing. Capture each as structured feedback at the moment it happens, with enough context to be reusable. Distinguish the kinds of signal, because corrections, preferences, and verdicts feed different downstream uses. Route captured feedback into the curation and training pipelines rather than letting it evaporate in chat logs and review comments.
Success Indicators
- Routine approvals, rejections, and preferences are captured as structured signal, not lost in conversation
- Each feedback record carries enough context to be reused without reconstruction
- Feedback is typed by kind, so it feeds the right pipeline
- Captured feedback reaches the curation pipeline at a measurable rate, not a sampled fraction
Failure Modes
- Feedback lives only in chat history and review threads, where it cannot be aggregated or reused
- Capturing verdicts without context, producing signal that cannot be acted on later
- Treating all feedback as one undifferentiated stream, so preference and correction signals are conflated
- Collecting feedback that never reaches a pipeline, leaving the loop open rather than closed
Sub-Methods 3
Map the moments in the workflow where a human or an automated check already renders a judgment, and capture each one. The cheapest feedback is the judgment that is already being made; you only have to record it rather than manufacture it.
Distinguish corrections (this is the right answer), preferences (this output is better than that one), and verdicts (this passed or failed). Each type feeds a different downstream use, and conflating them produces a stream that no pipeline can cleanly consume.
Route captured feedback into the curation and training pipelines, and confirm it arrives. Feedback that is collected but never consumed is an open loop: it carries the cost of capture and none of the benefit.