Skip to main contentSkip to footer
Reviewed

Assessment complete; awaiting evidence revision.

Agent Architecture Review, Validation snapshot

Evaluated 12 May 2026 against the AI Design Blueprint doctrine

High Risk

Status: High Risk

35/100

Grade F

2 aligned6 production blockers2 hardening1 high risk

Blueprint Readiness measures doctrine alignment, not runtime correctness. A production-ready verdict means the architecture embodies the 10 principles; it does not run your tests or types. Layer it on top of your test suite, not in place of it.

Per-principle verdicts

The bridge is a real delegated workflow with useful scoping, typed fetch errors, duplicate guards, and a structured final persistence path. However, the production trust boundary still fails around inspectability: the run is not anchored to a durable run record before execution, the cloned commit SHA and selected file hashes are not captured, skipped reads are silent, and the returned `_log` from validation is ignored. There are also blocker-level gaps in lifecycle visibility, selector policy consistency, explicit blocker handling, and steering.

Per-principle findings

10 principles evaluated. Verdict, severity, evidence and recommendation for each.

P7

high riskproduction blocker92/100

Establish trust through inspectability

The validation result is not reproducibly inspectable. `fetch_public_repo(...)` performs `git clone --depth=1 --single-branch` but never records the resolved commit SHA; `select_agentic_surface(...)` returns selected paths but no file hashes; `_build_implementation_context(...)` silently skips files on `FetchError`; and `run()` ignores the returned `_log` from `validate_code_against_principles(request, api_key)`. The final `UserValidationRun` stores `result_json` and `repository`, but not an audit chain linking repo ref, selected files, file contents/hashes, skipped files, model/tool call log, and final decision. Delta: source availability and final result persistence improve the prior high-…

Recommendation

Move audit authority into a durable ledger keyed by `run_id`: record repo URL, resolved commit SHA, selected paths, per-file hashes, skipped reads, validation request metadata, model/tool call log ID, response hash, and final persistence event before surfacing the result.

P2

needs changesproduction blocker70/100

Ensure that background work remains perceptible

The workflow prints foreground progress such as `✓ Cloned`, `✓ Selected`, and `→ Calling architect.validate`, but it does not create a durable run record before clone/validation and does not persist stage transitions like `cloning_repo`, `selecting_surface`, `validating`, or `persisting_result`. If the process dies during `fetch_public_repo(...)` or `asyncio.run(validate_code_against_principles(...))`, the dashboard has no reliable way to distinguish active, delayed, crashed, or never-started work because `app.validation_run_id` is only set after `db.flush()` near the end. Delta: the CLI progress improves operator visibility versus the prior baseline, but the durable lifecycle primitive is s…

Recommendation

Move lifecycle authority out of the CLI loop: create a persistent workflow/run row keyed by `run_id` before cloning, then update typed stage/status fields at each boundary so users can leave and return without losing continuity.

P8

needs changesproduction blocker65/100

Make hand-offs, approvals, and blockers explicit

Several blockers are explicit (`approved_at is None`, missing `user_id`, missing `repo_url`, duplicate `validation_run_id`, `FetchError.kind`, and `SelectionRejected`). However, some failure paths still bypass typed hand-off handling: `_parse_github_owner_repo(repo_url)` can raise `ValueError` before the `try/except FetchError` block and leave no persisted blocker, while `_build_implementation_context(...)` catches `FetchError` and silently `continue`s without recording skipped files or stopping on an empty/partial bundle. The production DB mutation also happens after validation without a separately recorded `ready_to_persist` or confirmation state beyond the earlier `approved_at` check. Del…

Recommendation

Route parse failures, read failures, empty bundles, and pre-persist confirmation through typed blocker states in the persistent workflow record; do not silently skip selected files or rely on free-text `rejection_reason` for hand-offs.

P9

needs changesproduction blocker60/100

Represent delegated work as a system, not merely as a conversation

The code is organized as a structured pipeline, and `AgenticSurfaceSelection` is a useful in-memory artifact, but the delegated work is not represented as a durable system. There is no persisted workflow object containing stages, dependencies, selected artifacts, blocker states, validation call metadata, and final outcome; only the final `UserValidationRun` is inserted after the LLM call succeeds. Intermediate state remains a CLI transcript plus transient Python variables. Delta: the current implementation improves the prior baseline by separating fetch, selection, and orchestration modules, but the persistent system-of-work primitive is still missing.

Recommendation

Keep the CLI as a runner, but represent the job as a persisted workflow record keyed by `run_id` with stage rows or events for clone, inventory, selection, bundle construction, validation, and result persistence.

P10

needs changesproduction blocker60/100

Optimise for steering, not only initiating

The workflow supports initiation and a `--dry-run`, but not steering while work is underway. Once `run()` enters `fetch_public_repo(...)` or `asyncio.run(validate_code_against_principles(...))`, there is no persisted abort flag, retry-from-stage, selected-surface override, or constraint update mechanism. After success, `app.validation_run_id` prevents duplicate runs but does not provide a controlled retry or correction path. Delta: this maintains the prior steering gap; the implementation is clearer now, but still only supports start-or-restart semantics.

Recommendation

Add a persistent control plane keyed by `run_id` with abort, retry-from-stage, and selected-surface override commands; have the runner poll that state at hard boundaries before clone, before validation, and before DB persistence.

P5

needs changesproduction blocker55/100

Replace implied magic with clear mental models

The selector’s stated mental model does not match its behavior. The docstring and `SelectionRejected` message say the selector requires an agentic-pattern marker, but `select_agentic_surface(...)` computes `has_positive_signal` and then still appends files without positive signal using `priority = 50 if not has_positive_signal else 99`; there is no `continue` for non-agentic names. That means arbitrary non-test source files can be bundled while the system presents the result as an agentic-surface scan. Delta: the current code is more explicit than the prior baseline about limits and byte caps, but this policy/behavior mismatch is a new concrete trust-boundary issue.

Recommendation

Make the selection policy a hard primitive, not a comment: either require `has_positive_signal` before `candidates.append(...)`, or rename and expose the selector as a broad source-file fallback with clear user-facing limitations.

P6

needs changeshardening recommended35/100

Expose meaningful operational state, not internal complexity

`FetchError.kind` provides useful typed internals (`invalid_url`, `clone_failed`, `clone_timeout`, `repo_too_large`, `read_failed`), but the durable surface collapses operational state into `app.rejection_reason` strings like `fetch_failed:{kind}: {message[:300]}` and `bridge_exit:{reason}`. That mixes user state, diagnostic detail, and legacy rejection semantics in one free-text field rather than exposing states such as `blocked_by_repo_access`, `unsupported_language`, or `validation_complete`. Delta: this improves the prior baseline by introducing discriminated fetch failures, but the persisted state model is still stringly typed.

Recommendation

Persist a small typed status enum and optional diagnostic artifact instead of overloading `rejection_reason`; reserve low-level subprocess or filesystem messages for diagnostics while keeping the primary state user-relevant.

P3

needs changeshardening recommended30/100

Align feedback with the user’s level of attention

The foreground CLI feedback is reasonably concise, but applicant/dashboard feedback is encoded indirectly through `app.rejection_reason` strings such as `bridge_exit:selector_rejected` and `fetch_failed:{kind}: {message[:300]}`. There is no distinction between an actively monitoring operator, a passively waiting applicant, and a later diagnostic inspection view. Delta: current code improves the prior result by adding concise operator messages and typed `FetchError.kind`, but the audience-specific feedback layer remains incomplete.

Recommendation

Keep the CLI transcript as an operator view, but persist a small user-facing status and attention level separately from diagnostic text, escalating only actionable states such as inaccessible repo, unsupported language, selector rejection, or validation failure.

P1

aligned

Design for delegation rather than direct manipulation

`run(application_id)` delegates a bounded job from an approved `CohortApplication`: it checks `approved_at`, `user_id`, `repo_url`, and `validation_run_id`, derives a scoped `cohort/{application_id}/{owner}/{repo_name}` namespace, clones only a public GitHub repo, selects a bounded surface via `select_agentic_surface(...)`, and persists the result under the linked `user_id`. This is a genuine delegation pipeline rather than manual step-by-step manipulation. Delta: this materially improves the prior finding by making the delegation scope visible in executable source.

P4

aligned

Apply progressive disclosure to system agency

The persisted result separates primary summary fields from deeper inspection data: `UserValidationRun` stores `assessment_status`, `score`, `grade`, `tier`, `repository`, `latency_ms`, and the full `result_json=response.model_dump_json()`. The validation request also carries `ValidationContext(... files=list(selection.selected_paths))`, giving a path from summary to detailed validator output. Delta: this addresses the prior need for a layered result envelope at the bridge layer, although audit-grade provenance is still handled under Principle 7.

Embed in your README

Two embeddable variants: a small flat shield and a richer score card.

Score card (recommended)

Blueprint Readiness Score card
[![Blueprint Readiness Score card](https://aidesignblueprint.com/api/badge/run/7ef168b8-b9c0-40f0-8dc4-b25e36ae3a09/card.svg)](https://aidesignblueprint.com/en/readiness-review/7ef168b8-b9c0-40f0-8dc4-b25e36ae3a09)

Flat badge

Blueprint Readiness Score badge
[![Blueprint Readiness Score](https://aidesignblueprint.com/api/badge/run/7ef168b8-b9c0-40f0-8dc4-b25e36ae3a09.svg)](https://aidesignblueprint.com/en/readiness-review/7ef168b8-b9c0-40f0-8dc4-b25e36ae3a09)

Start here, part 3 of 3

See how this page and its badge are made, from the first run to the README.

Baseline and iteration details
Baseline: usedDoctrine: same doctrineRace: checked clear

Iteration delta

10 closed this pass0 reopened1 high-risk findings still open

Improvements (10)

P1Design for delegation rather than direct manipulationneeds_changesaligned
P2Ensure that background work remains perceptibleneeds_changesneeds_changes
P3Align feedback with the user’s level of attentionneeds_changesneeds_changes
P4Apply progressive disclosure to system agencyneeds_changesaligned
P5Replace implied magic with clear mental modelsneeds_changesneeds_changes
P6Expose meaningful operational state, not internal complexityneeds_changesneeds_changes
P7Establish trust through inspectabilityhigh_riskhigh_risk
P8Make hand-offs, approvals, and blockers explicithigh_riskneeds_changes
P9Represent delegated work as a system, not merely as a conversationneeds_changesneeds_changes
P10Optimise for steering, not only initiatingneeds_changesneeds_changes
Rubric: 2026-05-04

Run ID: 7ef168b8-b9c0-40f0-8dc4-b25e36ae3a09

A point-in-time review of this artifact against the doctrine, not a security audit or a runtime guarantee.

What a review covers, and how data is handled

Run by agents. Governed by humans. Validated by the AI Design Blueprint.