starter-kit/methodology/build-methodology-core.md

• ../Up one level
• raw The file itself. Your browser may save it rather than show it.

# Build Methodology — Portable Core (v0.3)

Status: **Portable core. Project-agnostic.** This document defines a reusable, AI-augmented software-development methodology and its verification harness. It contains **no project-specific knowledge** — no stack, no domain, no business rules. A project connects to it by supplying a **Project Manifest** (Section 6); the finance app's manifest (`../design/finance-app-binding.md`) is the first reference instance.

Versioning: semantic. A project **pins** a core version. Improvements are made here and adopted by projects deliberately, never by accident. This is v0.3 — v0.2 plus §9 Change Control, extracted from the reference project's live change practice (batched brief revisions, conflict flagging, back-propagation).

The seam this document defends: **everything here is portable; everything a project changes lives in its manifest.** If a thing cannot be expressed as either core logic or a manifest entry, the seam is in the wrong place — fix the seam, don't leak.

Relationship to a project's operating file (e.g. `CLAUDE.md`): the operating file is the thread-facing SOP; this core is the portable method behind it. The belts map 1:1 — the operating file's *Brief* + *Plan & design* phases = this core's **Design** (Brief / Architect / Visualize); *Build* and *Verify* align directly; *Operate* covers **Deploy**'s post-release sub-stages (observe & recover). One method, two views — not two methods.

---

## 1. First principle: verification is the product

When AI produces artifacts — briefs, designs, code, tests — production becomes cheap and **trust becomes the scarce resource.** So this methodology is not organised around *making* things; it is organised around *verifying* them. Every stage produces an artifact that is untrusted until it passes a gate, and only a passed artifact may become the next stage's input.

The rest of this document is the machinery that makes that discipline concrete and repeatable.

---

## 2. Principles (the portable philosophy)

1. **The brief (plus approved mockups) is the oracle.** Everything downstream is verified against the brief and the acceptance criteria derived from it — never against the implementation. A short, testable brief plus approved mockups is the foundation; agree them before building on them.
2. **Verification is independent and adversarial.** The agent (or person) that verifies is never the one that produced. Verifiers are prompted to *disprove* and *break*, not to confirm. For high-stakes artifacts, multiple verifiers with distinct lenses, with a majority required to pass.
3. **Verify properties, not just examples.** Where a rule must hold for all inputs, test the property (generate many cases), not a handful of hand-picked examples.
4. **Ground-truth oracles beat synthetic tests.** A trusted real-world answer (a legacy system, a reconciled dataset, a known-correct output) is the strongest possible check. Prefer reconciliation against reality over assertions we wrote ourselves.
5. **Layer the verification** into a pyramid (Section 5) that runs on every change — cheap checks first, expensive judgement last.
6. **Traceability is non-negotiable.** Every scope item maps to the acceptance criteria and tests that verify it. Coverage is visible; gaps cannot hide behind claims.
7. **Gates route backward, not forward.** A failed gate returns the artifact to its stage. Nothing flows downstream on a "we'll fix it later."
8. **Extract, don't pre-build** (a rule for evolving *this* core). Add to the portable core only what a real project has already proven. The core grows by extraction from working instances, never by speculation.
9. **Back-propagate every decision.** When a decision is made or changed at any gate, every artifact that consumes it is re-verified **before the next gate**: search for the superseded language, update it, and re-check traceability. Staleness in a consuming artifact is a defect, equal in rank to a failing test — a stale acceptance criterion will pass an implementation the design forbids. (Extracted from the reference project, where a late FX decision left acceptance tests encoding superseded behaviour, and a folder reorganisation left cross-document paths dangling.)

---

## 3. The conveyor belt (four phases, one rhythm)

Four phases — **Design → Build ⇄ Verify → Deploy** — with a **gate at every boundary**. But phases are not monolithic boxes. Each has standardized **sub-stages**, and every sub-stage is itself the same loop:

> **Produce** (an agent generates the artifact) → **Verify** (an *independent* agent/panel + human where stakes require) → **Gate** (pass/fail against a definition of done) → trusted output feeds the next.

That self-similarity **is** the standardization: the same Produce → Verify → Gate rhythm at every level, whether you zoom to a whole phase or a single build slice. Verification is therefore **pervasive** (every sub-stage boundary is a gate), and it **concentrates** in the Verify phase — where built software is checked against everything Design produced.

| Phase | Sub-stage | Produces | Gate (definition of done) |
|---|---|---|---|
| **1 · Design** *(Discovery & Design)* | **Brief** | Brief — *what* it does | Complete, consistent, testable; owner + domain expert sign off. **Becomes the oracle.** |
| | **Architect** | Technical design **+ acceptance criteria + invariants** | Every scope item has an architectural home; no contradictions; criteria + invariants extracted. |
| | **Visualize** | Wireframes → clickable mockups | Every scope item/view reachable; owner approves the experience **before code**. |
| **2 · Build** *(per vertical slice)* | **Plan** | Slice scope + contracts/interfaces | Slice bounded; contracts defined against the design. |
| | **Implement** | Code + the builder's own unit tests | Compiles and runs; unit tests present. |
| | **Integrate** | The slice wired into the system | End-to-end runnable; no regressions surfaced. |
| **3 · Verify** *(the dev-CI/CD; loops with Build per slice)* | **Automated checks** | Static + unit/property + integration results | All green. |
| | **Acceptance & reconciliation** | Brief-derived acceptance tests; ground-truth diff | Scenarios pass; outputs match the oracle. |
| | **Adversarial review** | Independent agent-panel verdicts | Majority pass; no open correctness objection. |
| | **Human sign-off** | Owner + domain-expert approval | Behaviour and (where relevant) domain correctness attested. |
| **4 · Deploy** | **Package & provision** | Reproducible build; config/secrets | Build reproducible; environment ready. |
| | **Release** | Shipped to target | Deployed cleanly. |
| | **Post-deploy verification** | Smoke + health + prod reconciliation | Healthy in production. |
| | **Observe & recover** | Monitoring, verified backups, tested rollback | Recoverable — rollback and restore proven. |

Two things this makes explicit:

- **Build ⇄ Verify is a loop, not a hand-off.** You build a slice, verify it, repeat. The development-CI/CD (Section 5.3) *is* that loop; deployment-CI/CD (Phase 4) is separate and downstream.
- **Design manufactures verifiability.** The Design phase emits the oracle — acceptance criteria, invariants, approved screens — that the Verify phase consumes. Rush Design and Verify has nothing to check against. The phase gates are, in effect, the project's **exit criteria**; a project's phase plan states them concretely (see the manifest, Section 6).

---

## 4. Roles

**Human**
- **Owner / author** — holds intent, makes product decisions, signs the behavioural/UX gate.
- **Domain expert / verifier** — signs the correctness gate for the domain (the human ground-truth oracle). May be the same person on small projects; must exist as a role regardless.

**Agent**
- **Builder** — produces the artifact for a stage.
- **Reviewer panel** — independent verifiers (Section 5.2). Never the builder.
- **Orchestrator** — runs the pipeline: spawns builders and reviewers, collects votes, enforces gates, maintains traceability.

Independence rule: no agent verifies its own output. The orchestrator guarantees this.

---

### 4.1 The delegation boundary

Delegate tasks, not judgment. A **task** has a determinable right answer; a **judgement** depends on
values, undocumented context, or accountability. Write the boundary down where the agents read it:

- **Agents own:** compliance scanning, failure triage and classification, assembling evidence packets,
  routing to reviewers, executing an already-approved merge.
- **Agents stop, always,** before: merging to a protected branch, promoting to production, overriding a
  failed gate, or accepting an ambiguous staging result.
- **Humans own:** contradictory requirements, ambiguous failures, the four gate moments, and any
  trade-off whose answer depends on what the client values.

Escaping the automation trap needs three measurements: a **human baseline** before delegating, the
**verification overhead** counted separately per class of work, and experiments run first where the
probability of success is highest.

---

## 5. The verification engine

### 5.1 The pyramid (runs on every change, cheap → expensive)

1. **Static** — types, lint, schema/migration validity, "does it build."
2. **Unit / property** — pure logic in isolation; invariants expressed as property-based tests.
3. **Integration** — components together; data flows across boundaries.
4. **Acceptance** — end-to-end scenarios generated **from the acceptance criteria** (brief-derived, not implementation-derived).
5. **Reconciliation** — outputs diffed against the project's ground-truth oracle, if it has one (declared in the manifest).
6. **Agentic review** — the independent adversarial panel (5.2).
7. **Human gate** — expert sign-off on behaviour and (where relevant) domain correctness.

A change is not "done" until every applicable layer is green. Layers 1–5 are automated; 6 is agent-run; 7 is human.

### 5.2 The agentic reviewer panel

A fixed set of **roles**; the domain slot is filled by the manifest.

| Reviewer | Adversarial question it must answer |
|---|---|
| **Brief-conformance** | Where does this fail to satisfy the brief / acceptance criteria? What's missing or contradictory? |
| **Adversarial-correctness** | What input or state makes this produce a wrong result or crash? |
| **Security** | How is data exposed, auth bypassed, input trusted that shouldn't be, secret leaked? |
| **Domain-logic** *(manifest slot)* | Where does this violate the project's domain rules? (Content supplied by the project.) |
| **Simplicity / maintainability** *(optional)* | What here is needlessly complex, duplicated, or will rot? |

Rules: reviewers are **independent** of the builder; each is prompted to **find fault**, not approve; for changes touching a declared **invariant** or **critical path**, a **majority pass** is required, and a single credible correctness objection blocks the gate until resolved.

### 5.3 The development CI/CD

Distinct from deployment CI/CD. On every change to a build slice:

```
change → [1] static → [2] property/unit → [3] integration
       → [4] acceptance → [5] reconciliation → [6] agentic panel
       → red/green report → [7] human gate
```

- It runs continuously and is **agentic** (the panel is part of the pipeline, not a manual afterthought).
- It **gates** the human review: a human only looks once the machine layers are green, so human attention is spent on judgement, not catching regressions.
- Deployment CI/CD sits *downstream* and ships only what this pipeline passed.

### 5.4 The oracle ladder (what your claim is worth)

Every layer above compares the system against something. Layers 1 to 4 compare it against a
statement **you wrote**. Only layer 5 compares it against the world. So the strength of the whole
correctness claim is capped by the oracle available, and on genuinely new work there often is none.

Put every claim on a rung **before** you price or promise anything:

| Rung | Oracle | Technique | What you may claim |
|---|---|---|---|
| 1 | A live system doing the job | Differential / shadow running | "Provably equivalent on observed traffic" |
| 2 | A closed, audited record | Reconciliation, with a denominator | "Matches the settled answer, N of N" |
| 3 | A domain expert | Sampled, blind, inter-rater agreement measured first | "Adjudicated by experts who agree with each other" |
| 4 | The spec itself | Executable acceptance criteria | "Meets every stated criterion" |
| 5 | None | A named human decides | "A named person judged it sound" |

**Never borrow a claim from a rung above the one you are on.** Where you are stuck on rung 4 or 5,
manufacture an oracle, cheapest first: metamorphic relations (hours), golden-dataset capture from
adjudications and incidents (days, then free), an independent second implementation of the critical
calculation (a week or two), staged reality such as a canary or parallel run (weeks, and real risk).

### 5.5 Report the denominator

Every check reports **how many things it examined**, not just how many passed. A number without a
denominator is a number about nothing, and a layer that silently drops what it could not reach will
report a clean result for work it never looked at. Three questions catch a lying suite:

1. **Can it fail?** Break something the layer should catch and confirm the pipeline goes red.
2. **What is it comparing against, and does that thing still exist?** An absent reference is a loud
   failure, never a quiet pass.
3. **How many did it check, of how many there were?**

### 5.6 Traceability matrix

A living table, agent-maintained:

| Scope item (brief §) | Acceptance criterion | Verifying test(s) | Oracle | Status | Last verified |
|---|---|---|---|---|---|

It answers one question at any moment: *which scope items are genuinely verified, and which are only claimed?* Uncovered scope items are visible by construction. No silent truncation of coverage.

---

## 6. The Project Manifest (the interface)

A project "connects to the methodology" by supplying this manifest. The core reads it and runs. **This schema is the seam.** Nothing project-specific belongs anywhere else.

```yaml
project:
  name:            # string
  brief:           # path to the brief (the oracle, with the approved mockups)
  design:          # path to the technical design
  acceptance:      # where acceptance criteria live (produced in Design · Architect)

stack:             # tech choices, per layer (build/runtime/test tools)

invariants:        # rules that must always hold; each becomes a property test
  - name:
    statement:     # the rule, precisely
    verified_by:   # property | reconciliation | runtime-check

oracles:           # ground-truth sources for the reconciliation layer
  - type:          # reconciliation | runtime | human
    source:        # what/where
    verifies:      # which outputs it grounds

reviewers:
  domain_slot:     # description + rules the domain-logic reviewer must enforce

gates:
  automated:       # which pyramid layers must be green to pass
  human:
    - who:         # role
      verifies:    # what they sign off

stages:
  visual_design:   # approach (e.g. in-repo mockups | design tool)
  build:           # slicing strategy
  phase_map:       # how the project's phases map onto the belt
```

**Conformance:** a project is "on the methodology" when it supplies a complete manifest and either provides each required hook (oracle, domain reviewer, expert gate) or *explicitly declares its absence* (e.g. "no ground-truth oracle for this project"). Silence is not allowed; absence must be a stated decision.

---

## 7. Artifact templates (portable)

The core ships standard shapes so every project's artifacts are comparable:

- **Brief** — short sections for goal, users/journeys, scope (each with an acceptance criterion), out-of-scope, open items.
- **Acceptance criteria** — `Given / When / Then`, each tagged to a brief item (feeds traceability).
- **Invariant** — `name · statement · verified_by`.
- **Verification report** — per change: pyramid layer results, panel votes with objections, reconciliation diffs, gate decision.
- **Traceability matrix** — Section 5.4.

Templates are project-agnostic; the manifest and stages fill them with content.

---

## 8. How this core evolves

Per Principle 8: this document changes only when a real project proves a need. The workflow: a project hits a gap → solves it in its own binding/adapter first → if the solution is genuinely project-agnostic, it is **extracted** here and the core version bumped. The reference instance (currently the finance app) is always kept passing against the current core, so the core never drifts into theory.

**Change log:** v0.1 initial. v0.2 — Principle 9 (back-propagation), extracted after a fresh verification pass caught stale artifacts. v0.3 — §9 Change Control, extracted from the reference project's lived change practice.

---

## 9. Change control (change orders)

Every project changes mid-flight. The discipline: **a change is a mini-belt run** — it enters at the artifact it modifies and re-runs produce → verify → gate from that point downstream. A change never lands directly on code.

**Rule zero — brief-before-code applies to changes too.** First distinguish:
- **Defect:** the code disagrees with the approved artifacts → fix forward inside Build ⇄ Verify. No change control.
- **Change:** the approved artifacts themselves must move → the process below.

**Impact classes:**

| Class | What moves | Decision gate | Handling |
|---|---|---|---|
| **1 · Cosmetic / clarification** | wording, polish within the established philosophy | none (normal review) | apply; log |
| **2 · Functional, within design** | scope items that fit the existing architecture and invariants | Owner | batch into the next brief revision; update acceptance criteria; back-propagate |
| **3 · Structural / technical** | architecture, data model, invariants, stack | Owner + domain expert (where domain rules are touched) | impact analysis → decision → revise design + criteria + invariants → back-propagate → re-verify affected tests/evals; version-bump the artifacts |
| **4 · Foundational** | invalidates the locked brief's premises (scope pivot, new persona, regulatory shift) | **stop the belt** | return to the Design phase; re-run Brief → Architect → Visualize for the impacted area; explicit re-lock; record a waiver if any rule is bypassed |

**Process (every class):** intake → log in the **change register** → classify (agent proposes; human confirms for class ≥ 2) → impact analysis via the traceability matrix (which artifacts, tests, and tasks consume the changed decision — mechanical, not from memory) → decide at the class's gate → apply + **back-propagate (Principle 9)** → re-run the verification layers affected → close the register entry with commit references.

**Batch discipline:** locked artifacts are never edited piecemeal. Approved non-urgent changes accumulate in the register and land as one **versioned revision** (e.g. brief v1.1 → v1.2) with a single re-verification sweep.

**Conflict rule:** when stakeholder needs collide (e.g. owner vs domain expert), the change is **flagged, never silently applied** — the register records both positions and names the gate that resolves it.

**Mid-build changes:** affected build slices re-enter Build ⇄ Verify; the regression layer is the safety net that proves the rest of the system didn't move.

Register template (`design/change-log.md` in the project): `ID · date · source · description · class · decision gate · artifacts touched · status`.