From silent transfer fabrication to cert-confirmed in ten iterations

Validator + cert trajectory

Ten iterations, every run ID public

Each iteration is a real prod-MCP architect.validate call. Iter 4, 5, 6 then took architect.certify second-pass reviews and each downgraded the score from 100/A to 74/C/emerging because the cert reviewer caught a different production-blocking failure mode on each pass. Iter 7-9 then regressed on validate as the cert-layer fixes unmasked deeper gaps. Iter 10 is the first run where validate AND cert both pass cleanly, the cert reviewer found zero missed blockers.

Cert mandate value

Each cert call caught what the prior fix unmasked

First-pass validate alone returned aligned three times in a row (Iter 4-6). Three times in a row the second-pass cert reviewer surfaced a production-blocking failure mode the first-pass review couldn't see. Each fix to the cert finding triggered a NEW failure mode that only became reachable because the prior fix existed. This is the load-bearing argument for layered second-pass certification.

Principle scorecard

Every flagged principle, Iter1 vs Iter10, at a glance

Five principles across the trajectory fired as production_blocker / high_risk on at least one validate or cert pass; the Iter10 run closes all of them. The narrative below walks through each, this table is the scannable summary.

Refactor scope

Iter1 ungoverned vs Iter10 cert-confirmed

Numbers verbatim from the package source. The original 156-line script had no audit, no run_id, no approval, no idempotency, no recovery. Iter 10 is ~2,384 lines because the cert reviewer surfaced a different load-bearing failure each cert call, every line earned its place against a specific finding.

Before / After

# Iter1 baseline (excerpt, full file is 156 lines)
def execute_invoice_run():
    invoices = fetch_due_invoices()
    for invoice in invoices:
        if eligible_for_auto_pay(invoice):
            try:
                response = transfer_funds(
                    iban=invoice.iban,
                    amount=invoice.amount_pence,
                    currency=invoice.currency,
                )
                print(f"Paid {invoice.invoice_id}: {response}")
            except Exception:
                pass  # silent

# No run_id. No approval gate. No idempotency.
# No audit. No recovery. Crash mid-loop = orphaned bank requests.
# Retry = double-pay. Bank API stub = audit ledger lies.

# Iter10 cert-confirmed: pre-execute durable preflight
def execute_approved_payments(self, *, run_id):
    run = self._require_run(run_id)
    # Iter 10 P8 cert-fix: refuse to start any submission pass
    # while ANY task is SUBMITTED_UNKNOWN. Blocker is durable
    # across calls. The operator MUST reconcile existing bank
    # exposure before any new external bank call.
    unresolved = [
        t for t in self.tasks[run_id].values()
        if t.status == TaskStatus.SUBMITTED_UNKNOWN
    ]
    if unresolved:
        self.ledger.append(
            run_id=run_id, task_id=None,
            event_type="run.execution_blocked_on_unresolved_exposure",
            payload={
                "unresolved_task_ids": [t.task_id for t in unresolved],
                "required_action": "reconcile_submitted_task or confirm_mock_simulation",
            },
        )
        run.status = RunStatus.BLOCKED_ON_RECONCILIATION
        return run  # NO bank call. NO new exposure.
    # ...

Validator output

What the cert reviewer found

The Blueprint MCP ran architect.validate ten times and architect.certify four times against this code base. Five principle-level production blockers were identified across the trajectory, each one a path for an irreversible bank transfer to fire under conditions the operator never authorised.

How each blocker was resolved

What the iterations fixed

Each iteration closed at least one production blocker. Iter 10 is the first run where every flagged principle reaches aligned and the second-pass cert reviewer finds zero missed blockers.

Re-validation result

Iter 10: architect.certify confirmed production_ready

Iter 10's implementation was re-validated to 100/A/production_ready, then certified in the same prod-MCP session. The cert outcome was confirmed_production_ready, zero certification_findings. Verbatim summary from the cert reviewer: "the code visibly implements durable reconciliation blocking for ambiguous bank handoffs, explicit cancellation-pending states, signed approval/hash checks, audit/inbox inspectability, and no specific missed production-blocking crash, silent wrong-result path, or trust-boundary bypass is evidenced."

Calculated ROI

Same metrics, same calculator powering every case study

Derived deterministically from this case study's profile (10 iterations, irreversible-financial blast radius, autonomous workflow, under compliance) via /lib/case-study-roi.ts. Numbers directly comparable to the other case studies.