Quality Fragility

Unintended System Conditions. A USC is an unplanned state that builds up when principles are repeatedly violated. It is the default output of the system as currently designed, not a one-off incident or a people problem. Fixes aimed at symptoms improve things briefly before the condition reasserts itself. Which principles feed each condition is NOT stored here: it is derived at runtime from the consequence-to-condition links in the principle files.

Explore all conditions

Quality Fragility

Defects, incidents and rework grow as a share of output, and most problems are found after they are cheap to fix.

Customers find defects before the team does.
Read more

Defects, incidents, and rework are increasing as a proportion of delivery output, and the majority of quality problems are being discovered after the point where they are inexpensive to address. The system's safeguards are positioned too late in the flow. Quality is verified at the end rather than built in throughout. As the system grows, the cost of late-stage quality failures increases non-linearly, incident rates rise, and confidence in releases declines.

In agentic delivery

In human-driven delivery, Quality Fragility happens when testing and review are deferred to the end of the delivery cycle. In an agentic system, quality fragility takes on a new dimension. Agents can produce outputs that pass every defined quality check while still failing on dimensions nobody thought to specify. The quality gates were designed for human-generated work and do not account for the specific failure modes of agent-generated outputs.

  • AI-generated outputs pass all automated validation but fail in production on edge cases no test covered
  • Evaluation frameworks for agent outputs either do not exist or were written once and never updated as agent behavior evolved
  • Quality gates are designed around what humans typically get wrong and miss the failure modes specific to agents
  • The definition of done has not been revisited since agents entered the workflow and no longer captures what quality means in an agentic context
  • Agent-generated changes receive lighter scrutiny than human-generated changes despite introducing different and less understood risk profiles

Principles that feed this condition

Primary

Upstream Quality Reduces Downstream Failure Propagation

  • Defects and wrong assumptions travel through several stages before anyone catches them.
  • Teams spend more and more effort undoing work built on early errors.
  • Release dates slip, and fewer people trust a release will hold up.
  • Extra reviews, approvals and coordination pile up to make up for weak quality signals.
  • The system gets harder and riskier to change.
  • Small upstream errors spread fast across automated agent pipelines.

Greater Observability Enables Safer Expansion of Autonomy

  • Risk from agent actions nobody checked keeps growing, including compliance and ethical risk.
  • Learning about agent behavior has slowed because nobody can inspect it as it runs.
  • The team relies on post-incident reviews instead of preventing failures.

Automate by Task Nature, Not Capability

  • Rule-based work is slower, less predictable and harder to audit than fixed code.
  • Irreversible mistakes show up only after they've spread through downstream systems.

Secondary contributors