Quality
Testing, validation, review processes, and quality assurance. Quality techniques ensure that AI-generated output meets defined standards, that regressions are caught, and that trust in the system's output is earned through verification.
7 methods| ID | Method | Description |
|---|---|---|
| FG-0601 | Testing Pyramid 4 | Implement a multi-tiered testing strategy that validates AI-generated output at every level. The Testing Pyramid applies traditional software testing principles to human-AI collaboration: unit-level validation of individual outputs, integration testing of combined outputs, system testing of complete deliverables, and acceptance testing against human expectations. More tests at the base, fewer at the top, comprehensive coverage throughout. |
| FG-0602 | Code Quality Standards | Establish explicit standards for the quality of AI-generated code. AI agents can write code quickly, but quality requires explicit standards: read existing code before modifying it, follow established patterns, don't introduce new dependencies without justification, and test before committing. Code Quality Standards convert implicit expectations into explicit, enforceable rules that agents follow consistently. |
| FG-0603 | Contributor Review Period | Require new contributors (human or AI) to operate under enhanced review for a defined period before granting full autonomy. The Review Period exists for calibration. New contributors need time to learn project conventions, and the team needs time to assess the contributor's quality, reliability, and alignment with project standards. The review period formalizes this mutual calibration. |
| FG-0604 | Cross-Platform Validation | Test AI-generated artifacts across all target platforms, environments, and configurations. AI agents typically work in one environment but their output may need to function across many. Cross-Platform Validation ensures that output is tested in every context it will encounter, well beyond the development environment. This includes operating systems, browsers, screen sizes, accessibility tools, and deployment environments. |
| FG-0605 | Audit Trail System | Maintain comprehensive, searchable records of all significant actions, decisions, and changes. An Audit Trail supports both compliance and learning. When something goes wrong, the audit trail enables root cause analysis. When something goes right, it enables pattern replication. Every significant action should leave a mark in the record: who did it, what they did, why they did it, and when. |
| FG-0606 | Output Verification | Systematic methods for detecting inaccurate, fabricated, or misleading content in AI-generated output. RLHF-tuned language models tend to be overconfident, their expressed confidence poorly calibrated against actual accuracy: they produce fluent, assured output regardless of whether it is correct. Hallucinated facts, invented citations, plausible-sounding but incorrect technical claims, and subtly wrong data are indistinguishable from correct output without explicit verification. Output verification is the minimum standard for using AI-generated content in any context where accuracy matters. |
| FG-0607 | Structured Model Evaluation 3 | Head-to-head model comparison using consistent prompt sets, automated metrics, and structured JSON results. Multiple candidates are evaluated against the same benchmarks, scored on defined dimensions, and the winner advances. This is model selection through controlled experiment, not vibes. Every evaluation produces a reusable artifact: the prompt set, the scoring rubric, and the results archive. |