af-mock-disaster-assistance-gov — Prop-Build Analysis

Document Type: Critical Review & Analysis (companion to prop-build-template.md) Scope: Per-Repo / Per-Module Subject: af-mock-disaster-assistance-gov (FEMA Disaster Assistance Mock Website) Reviewer(s): Claude (automated code review) Date: 2026-04-09 Version: 0.1 Confidence Level: Medium What would raise confidence: running the mock end-to-end against the live AI agents, executing the Playwright/pytest integration suite, and interviewing the author (Gordon Zheng).

Inputs Reviewed:

• Prop-build doc: data/af-mock-disaster-assistance-gov.yaml
• Companion files: data/af-mock-disaster-assistance-gov/{component-tree,data-flow,deployment,runbook,api-examples}.md
• Source: /Users/andres/src/af/af-mock-disaster-assistance-gov/ (js/, tests/, index.html, README.md)

Part A — Per-Repo / Per-Module Analysis

Note: this repo is a mock / test harness used as a deterministic fixture for training and regression-testing AidFinder AI agents. Judged on test-harness quality (determinism, fixture ergonomics, schema stability, observability), not production hardening.

A.1 Executive Summary

• Overall health: Ambitious, well-structured vanilla-ES6 harness: 43 page modules, canonical session schema, fixture injection, sizeable Python/Playwright test suite — solid bones for an AI-agent training fixture.
• Top risk: Left-behind debug console.log statements dumping personalInformation on every save/load (js/storage.js:21,29,69,79,88,90,97; js/state.js:68) — harmless in a mock but a footgun if repurposed for recording agent traces. See A.4.1.
• Top win / thing worth preserving: Schema-first session model (js/store/schema.js) + session-adapter validation + storage.injectSnapshot() fixture pipeline (js/storage.js:44-52) — exactly the right shape for a deterministic agent harness.
• Single recommended next action: Delete debug console.log block in js/storage.js and js/state.js:68; route diagnostics through logger.js.
• Blocking unknowns: Pixel-parity vs. real DisasterAssistance.gov (not verifiable from code); whether Playwright suite is wired into CI; flake rate of schema-migration tests.

A.2 Health Scorecard

Overall score: 3.69 (weighted for the harness role, effective grade is higher: dimensions 1/2/4/8/10/14/15 all score 4+; 7 and 17 are either cosmetic or out-of-scope).

A.3 What's Working Well

• Strength: Schema-first session model with explicit versioning and migrations.

  • Location: js/store/schema.js:1-50, js/store/migrations.js, js/store/session-adapter.js (validateSchema, validatePath, patch).
  • Why it works: Every write funnels through a validated patch() path; _meta._version lets the harness evolve without breaking saved fixtures.
  • Propagate to: af-fema-form-automation and future mock repos.

• Strength: Fixture-injection seam for deterministic agent runs.

  • Location: js/storage.js:44-52 (injectSnapshot), tests/fixtures/*.json, js/utils/fixtures.js.
  • Why it works: Jump straight to any page state without walking the 29-page flow.
  • Propagate to: Any real-site-replica mock.

• Strength: Deterministic-vs-randomized mode via URL flags.

  • Location: js/config.js:11-45 (mode, identityVerificationFailureRate, forceErrors, enableDebugPanel, enableEventLogging).
  • Why it works: A/B correctness vs. robustness tests without code changes.

• Strength: In-browser session editor for debugging agent runs.

  • Location: js/pages/session-editor.js (1295 LOC).
  • Why it works: Operator can inspect/patch live state mid-run.

A.4 What to Improve

A.4.1 P1 — Debug console.log leaking personalInformation on every save/load

• Problem: js/storage.js and js/state.js unconditionally console.log the full personalInformation subtree on every save, load, and set() — bypassing logger.js.
• Evidence: js/storage.js:21-22,28-30,69,79,88-90,97; js/state.js:68.
• Suggested change: Delete; if still useful, route through logger.debug().
• Estimated effort: S.
• Risk if ignored: Console noise; if harness is ever repurposed for real data, a PII-in-logs incident.

A.4.2 P2 — Oversized page modules mixing render, validation, handlers

• Problem: Multiple 500+ LOC pages inline large innerHTML strings alongside validation and handler wiring.
• Evidence: js/pages/session-editor.js 1295, vehicle-damage.js 687, personal-info.js 572 (SSN toggle HTML at lines 70, 367, 370), review-application.js 507, needs.js 413.
• Suggested change: Extract per-page templates into sibling *.template.js; lean more on js/components/form-field.js.
• Estimated effort: M.
• Risk if ignored: Shotgun-surgery pressure grows as the mock tracks upstream UI changes.

A.4.3 P2 — No JS-level unit tests

• Problem: No unit coverage for js/store/schema.js, js/store/session-adapter.js, js/store/migrations.js, js/validation.js — the exact modules correctness depends on.
• Evidence: tests/ is Python-only; package.json "test" is a stub echo.
• Suggested change: Add Vitest or node --test suite for store modules.
• Estimated effort: M.
• Risk if ignored: Schema regressions only caught by slow Playwright runs.

A.5 Things That Don't Make Sense

1. Observation: package.json hard-blocks npm start/npm run dev with echo && exit 1 and declares "engines": { "node": "DO NOT USE - ..." }, yet playwright is a devDep that needs Node.

   • Location: package.json scripts + engines.
   • Hypotheses considered: (a) past http-server ES-module MIME bug; (b) enforcing Python parity with CI.
   • Question for author: What specifically broke under Node servers?

2. Observation: session-editor.js (1295 LOC) is larger than app.js (476) + router.js (338) combined.

   • Location: js/pages/session-editor.js.
   • Hypotheses considered: Debug-tool accumulation point; runbook UI.
   • Question for author: Split into js/components/session-editor/?

A.6 Anti-Patterns Detected

A.6.1 Code-level

• God object / god function — js/pages/session-editor.js (1295 LOC).
• Shotgun surgery
• Feature envy
• Primitive obsession
• Dead code
• Copy-paste / duplication — inline SVG toggles at js/pages/personal-info.js:367,370.
• Magic numbers
• Deep nesting (>3)
• Long parameter lists (>4)
• Boolean-flag parameters

A.6.2 Architectural

• Big ball of mud
• Distributed monolith
• Chatty services
• Leaky abstraction
• Golden hammer
• Vendor lock-in
• Stovepipe
• Missing seams for testing — explicitly not checked; injectSnapshot + forceErrors + mode=deterministic are exactly the seams this asks for.

A.6.3 Data

N/A — no database; localStorage schema is versioned and validated.

A.6.4 Async / Ops

N/A — no queues, no distributed work.

A.6.5 Security

• PII/PHI in logs — console.log of personalInformation in js/storage.js:21,29,69,88 and js/state.js:68. Synthetic today, pattern is the smell.
• Secrets in code / .env committed
• Missing authn/z on internal endpoints — N/A
• Overbroad IAM — N/A
• Unvalidated input crossing trust boundary — N/A
• Missing CSRF/XSS/SQLi/SSRF — not flagging (mock; store is schema-validated).

A.6.6 Detected Instances

A.7 Open Questions

1. Q: Is tests/integration/ wired into CI or run manually?

   • Blocks: A.2 row 15, A.13.
   • Who can answer: Gordon Zheng / QA owner.

2. Q: How is schema-version drift between this mock and af-disaster-assistance-gov-agent coordinated when FEMA changes a field?

   • Blocks: A.10, Part B lockstep risk.
   • Who can answer: Agent-container maintainers.

3. Q: Are tests/fixtures/ guaranteed-synthetic or seeded from real runs?

   • Blocks: A.11 Information Disclosure.

A.8 Difficulties Encountered

• Difficulty: No local execution — Playwright not run, flow not walked.

  • Impact: Cannot verify pixel parity, flake rate, or that migrations.js upgrades legacy fixtures cleanly.
  • Fix: One-command make verify that spins server, runs suite, emits JUnit.

• Difficulty: Page modules mix HTML + handlers + validation; "where is field X written?" needs whole-file reads.

  • Impact: Could not cheaply sample all 43 pages; row 7 is based on 5 largest + core modules.
  • Fix: A.4.2 template extraction + per-page header listing schema paths written.

• Difficulty: No git log review of churn.

  • Impact: A.15 answered from YAML author metadata only.

A.9 Risks & Unknowns

A.9.1 Known risks

A.9.2 Unknown unknowns

• Area not reviewed: Actual correctness vs. real DisasterAssistance.gov.

  • Reason: No access during review; "pixel-perfect" untestable from code.
  • Best guess risk: M — silent divergence trains agents on stale UI.

• Area not reviewed: js/store/migrations.js run end-to-end on legacy fixtures.

  • Reason: Tests not executed.
  • Best guess risk: L–M.

• Area not reviewed: ~37 of 43 page modules (sampled ~6).

  • Reason: Time.
  • Best guess risk: L — sampled patterns are consistent.

• Area not reviewed: dynamic_flow_enhacement/, FEMA-Video-Screen-Cap/, update-mock-webpage/, specs/.

  • Reason: Out of primary code path.
  • Best guess risk: L.

A.10 Technical Debt Register

A.11 Security Posture (lightweight STRIDE)

N/A — single-origin, client-only static mock; no auth, no network, no backend. The one real finding (PII-shaped data in console.log) is captured in A.4.1, A.6.5, and A.10 #1.

A.12 Operational Readiness

N/A — ops concepts (SLOs, on-call, DR, backups) don't apply to a dev-run static fixture. Harness-level observability is captured under A.2 row 14 and A.3.

A.13 Test & Quality Signals

• Coverage: Unknown (no tooling).
• Trend: Unknown.
• Flake rate: Unknown.
• Slowest tests: Likely e2e_full_flow_test.py, test_full_flow_to_success.py.
• Untested critical paths: js/store/schema.js, js/store/session-adapter.js, js/store/migrations.js, js/validation.js.
• Missing test types: [x] unit [ ] integration [ ] e2e [x] contract (no cross-repo contract test) [x] load (N/A) [x] security/fuzz (N/A).

A.14 Performance & Cost Smells

N/A — static site, localStorage, no infra.

A.15 Bus-Factor & Knowledge Risk

• Only-person: authors: ["Gordon Zheng"] in the YAML. Bus factor ≈ 1 for the 29-page → schema mapping.
• What breaks if they vanish: Upstream FEMA UI tracking; schema-drift reconciliation with the agent repo.
• Tribal knowledge: Which pages hand-traced vs. auto-captured; conditional-skip logic; the Node-server taboo.
• Knowledge-transfer actions: One-page "how to add a FEMA page" doc; document FEMA-Video-Screen-Cap/ and update-mock-webpage/ workflows.

A.16 Compliance Gaps

N/A — YAML claims no HIPAA/SOC 2/state-insurance compliance; explicitly a mock.

A.17 Recommendations Summary