9. Metrics

Organizations SHALL establish baselines by measuring for at least 3 Increments before setting targets. Targets are organization-specific, not universal.

9.1 Mandatory (SHALL)

A post-done defect is a defect task explicitly linked to a completed parent task that introduced the defect. The link SHALL be recorded in the task tracker.

9.2 Recommended (SHOULD; SHALL in Team+)

Manual Intervention Rate is computed from recorded Gate Bypasses, not from self-assessment. Section 8.6(j) makes direct modification of an artifact a Gate Bypass that SHALL be recorded, so the boundary of “manual intervention” is set normatively and is no longer left to each organization to define.

Relationship to Gate Bypass rate: 8.6(i) recommends rather than requires that figure, so an organization may hold this metric without it, and this Standard does not fix how it is computed. What the definitions give is a containment of events, not of rates: a modification recorded under 8.6(j) is a Gate Bypass under 8.6(h), so every such record belongs to the population Gate Bypass rate is drawn from, while the reverse does not hold — that population also contains admissions of any other gate’s declared effect. Whether the two rates stand in the same relation depends on the denominator the organization gives Gate Bypass rate, which 8.6(i) leaves to it: they are comparable where that denominator is Tasks, the unit this metric uses, and not otherwise. Figures on different denominators answer different questions and SHALL NOT be added together or presented as one nested inside the other.

NOTE: The record this metric counts is made by the party that acted. It is therefore a measure of what was recorded, and organizations SHALL NOT present it as a measure of what occurred.

An unrecorded modification does not appear in it, and nothing in this standard detects one automatically. Where assurance about the figure itself is required, it comes from comparing the modifications recorded in version control against the gate’s admission records (8.6(g)): every modification within a Task’s scope should fall under an admission the gate recorded, and one that does not is either an admission that was not recorded or a bypass that was not declared. Authorship does not settle it — 8.6(j) makes the route the origin of the content, not the identity of the process that wrote the file, so a comparison of who committed what is blind to precisely the case the clause was written for. This is an audit the organization performs, not a property the metric has.

Adversarial Detection Rate (ADR) measures the density of CRITICAL-severity findings discovered by adversarial review (Section 10.15, L3) per task that underwent L3 review. Target: organizations SHOULD aim for ADR < 0.5 (fewer than one CRITICAL finding per two reviewed tasks). An ADR of 0 indicates either excellent AI output quality or insufficient review rigor — organizations SHOULD distinguish between the two.

For ADR computation, “L3_reviewed_tasks” means tasks that underwent L3 Adversarial Review. Tasks that did not receive L3 review (e.g., Low-risk tasks where L3 was skipped per Section 10.15) are excluded from the denominator. This ensures ADR reflects review effectiveness, not review coverage.

Knowledge Capture Rate target calibration: organizations with established knowledge bases SHOULD set KCR targets that reflect diminishing returns. A target of 1.0 (one entry per task) is appropriate for greenfield projects. For mature projects (>500 tasks), a target of 0.33 (one entry per three tasks) better reflects the natural rate of novel knowledge discovery. Organizations SHALL document their KCR target rationale.

9.3 Collection

Metrics 1–5, 8 and 9 SHALL be collected automatically from task tracking and session data. Metric 8 is derived from recorded Gate Bypasses under 8.6(j) and requires no separate reporting act. Metrics 6–7 require cost tracking integration. Cost Predictability requires Flow Manager assessment at Increment Retrospective.

Metric 10 (ADR) SHALL be collected from adversarial review findings recorded during L3 review (Section 10.15). Organizations SHALL maintain a record of review findings per task, classified by severity, to compute ADR.

Metric computation SHALL include all tasks without epoch-based filtering or exclusion of historical data. Organizations SHALL NOT exclude tasks from metric computation based on creation date, migration status, or tooling version. Rationale: epoch filters introduce complexity, create maintenance burden, and mask data quality issues; they also make a figure unauditable, because the population it was computed over stops being the population the tracker holds. A cumulative figure does not shed its early data on its own: those tasks keep the weight their share of the total gives them, and that share falls only as the total grows — not because the tooling changed. Where the change of tooling is exactly what makes early data unrepresentative, waiting does not fix the figure. That is a reason to state the limitation rather than to filter it away. If early data is known to be unreliable (e.g., pre-automation manual entries), this SHALL be documented as a known limitation, not filtered.

Cost Predictability (metric 6) requires organizations to record planned cost before implementation begins. In practice, planned cost estimation for AI-assisted tasks is unreliable — AI execution time is non-deterministic and model pricing varies. At Team configuration, Cost Predictability SHALL be tracked but baselines may be provisional during the first 3 Increments. At Enterprise configuration, Cost Predictability SHALL be tracked with established baselines. Cost per Task (metric 7) provides a more actionable proxy for cost management in early adoption.

Multi-agent and multi-session configurations: when multiple AI agents work in parallel (multiple concurrent sessions), per-session metrics (throughput, session duration, cost per session) reflect individual agent performance, not aggregate team output. Organizations using multi-agent configurations SHOULD additionally track aggregate metrics at the Increment level. Token and cost attribution in multi-agent scenarios SHOULD be recorded per-agent, with aggregate totals available for Increment-level reporting.