af-infra — Prop-Build Analysis

Document Type: Critical Review & Analysis (companion to prop-build-template.md) Scope: Per-Repo Subject: aid-finder/af-infra (Terraform monorepo) Reviewer(s): Claude (automated code review) Date: 2026-04-09 Version: 0.1 Confidence Level: Medium What would raise confidence: access to actual AWS state (drift check), CI run history, an interview with the platform engineer who owns the backend state bucket, and sight of the prod-* environment directories (which do not currently exist in the repo).

Inputs Reviewed:

• Prop-build doc: /Users/andres/src/af/af-analysis/data/af-infra.yaml
• Companion docs: data/af-infra/{api-examples,data-flow,deployment,runbook}.md
• Source tree: /Users/andres/src/af/af-infra/ (HCL, ~4.4k LOC in modules/ + environments/)
• CI: .github/workflows/{plan,apply,lint}.yaml, .github/actions/*
• Backend config: BACKEND.md, each environments/*/providers.tf

A.1 Executive Summary

• Overall health: Solid, conventional Terraform monorepo — clean module boundaries, WAF + KMS + OIDC done right, lint pipeline in place — but the repo's promise of four environments (common/dev/stg/prod) is not yet kept: there is no environments/prod-* directory anywhere in the tree.
• Top risk: The prod environments promised by af-infra.yaml and by .github/workflows/apply.yaml:13 do not exist on disk (ls environments/ returns only common/dev/stg variants). Clicking "Run workflow → env: prod" will fail cold, and no one has prod state, prod tfvars, or prod backups. See A.4.1.
• Top win / thing worth preserving: The modules/service blue/green CodeDeploy + GitHub OIDC pattern (modules/service/main.tf:372-483) is exemplary — scoped IAM policy, no long-lived keys, ignore_changes on task_definition so Terraform and CodeDeploy don't fight. Propagate to any future service module.
• Single recommended next action: Create environments/prod-{backend,analytics,shared} (copy from stg, with prod-sized ACUs, retention, deletion protection, and a distinct state bucket) before any claim of "Active prod" is made externally.
• Blocking unknowns: Whether the existing dev-hih-backend-048599825724-us-east-1 state bucket has versioning + SSE + MFA-delete (cannot verify from code — only the use_lockfile = true line in providers.tf is visible); whether terraform apply has ever been run against a real prod account.

A.2 Health Scorecard

Overall score: 3.50 (14 applicable dimensions averaged, excluding N/A rows 3, 4, 6, 7, 11).

A.3 What's Working Well

• Strength: GitHub OIDC assume-role + narrowly-scoped code_deployer policy per service.

  • Location: modules/service/main.tf:424-483
  • Why it works: No long-lived AWS keys; iam:PassRole is restricted to just the two task roles of that service; subjects = ["${var.github_repository_name}:*"] binds trust to one repo.
  • Propagate to: Any other repo deploying to AWS.

• Strength: Blue/green ECS with CodeDeploy and ignore_changes = [task_definition, load_balancer].

  • Location: modules/service/main.tf:184-197, 297-316
  • Why it works: Terraform owns the infra, CD owns the rollout; avoids drift fight. Auto-rollback on DEPLOYMENT_FAILURE wired in (:378-381).

• Strength: SSM PLACEHOLDER pattern with ignore_changes = [value].

  • Location: modules/service/main.tf:1-12
  • Why it works: Lets Terraform own parameter name/type/ACL while leaving value management to humans/ops — keeps secrets out of state and tfvars.

• Strength: Single lint target — fmt, validate, tflint, terraform-docs --output-check.

  • Location: Makefile:12-18
  • Why it works: Identical local + CI command; docs cannot drift from code.

• Strength: WAF stack in front of CloudFront: managed IP reputation / SQLi / KnownBadInputs / Common + custom rate-based rule + CustomBodySizeLimit.

  • Location: environments/dev-backend/main.tf:116-259

A.4 What to Improve

A.4.1 P0 — Prod environments do not exist

• Problem: af-infra.yaml meta.scope claims "four account-isolated environments (common, dev, stg, prod)" and .github/workflows/apply.yaml:13 offers prod as a workflow_dispatch choice, but environments/ contains only common-*, dev-*, stg-*.
• Evidence: ls environments/ shows no prod-*; .github/workflows/apply.yaml:10-14; af-infra.yaml scope text.
• Suggested change: Either create environments/prod-{backend,analytics,shared} with prod-sized values (deletion_protection=true, higher retention, distinct state bucket under the prod AWS account), or remove prod from the workflow_dispatch enum and update the prop-build doc.
• Estimated effort: M
• Risk if ignored: The documented architecture is a fiction; on-call engineers will assume a prod exists and discover otherwise during an incident.

A.4.2 P0 — Bastion host SSH open to the entire internet

• Problem: The bastion security group allows TCP/22 ingress from 0.0.0.0/0 and ::/0.
• Evidence: modules/bastion-host/main.tf:16-23
• Suggested change: Restrict cidr_blocks to an input variable defaulting to [], or replace the bastion with SSM Session Manager.
• Estimated effort: S
• Risk if ignored: Constant SSH brute-force surface; the private key lives in Terraform state (modules/bastion-host/main.tf:5, create_private_key = true) — state leak = shell into VPC with routes to RDS.

A.4.3 P1 — RDS master password generated by Terraform, stored in state and SSM

• Problem: random_password result is written to state and then to an SSM SecureString; manage_master_user_password = false explicitly opts out of AWS Secrets Manager integration.
• Evidence: modules/rds/main.tf:5-8, 28-29, 83-93
• Suggested change: Set manage_master_user_password = true and let RDS rotate through Secrets Manager.
• Estimated effort: M
• Risk if ignored: Credentials sprawl; manual rotation; state file becomes a secret.

A.4.4 P1 — No security/policy scanning in CI

• Problem: make lint runs fmt, validate, tflint, terraform-docs — no tfsec, checkov, trivy, or OPA/Conftest.
• Evidence: Makefile:12-18; .github/workflows/lint.yaml.
• Suggested change: Add a checkov -d modules -d environments (or tfsec .) step with a curated ignore list.
• Estimated effort: S
• Risk if ignored: Misconfigurations merge unnoticed.

A.4.5 P1 — State bucket is shared across stacks

• Problem: Every dev-* environment points its backend at dev-hih-backend-048599825724-us-east-1 and reads siblings' state via terraform_remote_state. A compromised backend apply can read or clobber analytics state.
• Evidence: environments/dev-backend/providers.tf:3-6; environments/dev-backend/main.tf:1-17
• Suggested change: Per-stack bucket prefixes with IAM conditions, or per-account state buckets.
• Estimated effort: M
• Risk if ignored: Lateral blast radius within an environment during credential compromise.

A.4.6 P2 — environments/dev-backend/main.tf is a 510-line copy of stg-backend/main.tf

• Problem: Parallel structurally-identical files. Classic shotgun surgery. Analytics has same pattern at 640 LOC each.
• Evidence: environments/dev-backend/main.tf and environments/stg-backend/main.tf both 510 LOC.
• Suggested change: Extract modules/stacks/backend so env dirs become thin terraform.tfvars + module "backend" call.
• Estimated effort: L
• Risk if ignored: Drift between environments; review burden; prod (when created) will be a fourth copy.

A.4.7 P2 — No CloudWatch alarms in IaC

• Problem: containerInsights = enabled and WAF emits metrics, but no aws_cloudwatch_metric_alarm resources exist anywhere.
• Evidence: grep -r metric_alarm modules environments returns nothing.
• Suggested change: Add modules/alarms with RDS CPU, ECS task count, ALB 5xx, SQS DLQ depth, WAF blocked request surge.
• Estimated effort: M

A.5 Things That Don't Make Sense

1. Observation: environments/common-shared/main.tf is 6 lines and environments/dev-shared/main.tf is 26 lines — beside 510-line peers.

   • Question for author: Is "shared" intentionally minimal, or a placeholder?

2. Observation: Two CORS rules on the file submission bucket — one allowing only frontend_url, another (dev-only) allowing *.

   • Location: environments/dev-backend/main.tf:300-318
   • Question for author: Why not a per-env allowed_origins list variable?

3. Observation: default_tags is set in every providers.tf but no tagging-compliance policy enforces it.

   • Location: environments/dev-backend/providers.tf:19-25

A.6 Anti-Patterns Detected

A.6.1 Code-level

• Copy-paste / duplication — env files duplicated across dev/stg.
• Magic numbers — WAF priority = 0..5; container 256/512.

A.6.2 Architectural

• Shotgun surgery — any backend-stack change must be made to dev + stg (+ prod once added).
• Missing seams for testing — no terraform test, no Terratest.
• Leaky abstraction — environments/dev-backend/main.tf mixes stack glue with top-level resources rather than delegating to a stack module.

A.6.3 Data

• None observed in IaC repo.

A.6.4 Async / Ops

• Work queues without visibility — aws_sqs_queue.mapping_service_dlq (environments/dev-backend/main.tf:427-430) has no depth alarm.

A.6.5 Security

• Secrets in state — RDS master password (modules/rds/main.tf:5-8, 89-93); bastion private key (modules/bastion-host/main.tf:5).
• Overbroad IAM / SG — bastion SSH 0.0.0.0/0; AWSCodeDeployRoleForECS attached wholesale; shared state bucket.

A.6.6 Detected Instances

A.7 Open Questions

1. Q: Does a prod AWS account exist yet, and if so why is there no environments/prod-* tree?
2. Q: Is the state bucket versioned + SSE-encrypted + MFA-delete?
3. Q: Is there a GitHub Environment protection rule requiring approval before apply.yaml runs against prod?
4. Q: Why manage_master_user_password = false on Aurora (modules/rds/main.tf:29)?

A.8 Difficulties Encountered

• Difficulty: No running state visible; cannot see whether resources match HCL.
  • Impact: Drift and "claimed prod" cannot be verified.
  • Fix: Commit a terraform state list snapshot under doc/.
• Difficulty: Companion docs live in af-analysis, not next to the code.
  • Impact: Bus-factor on discoverability.
• Difficulty: No CHANGELOG; had to read parallel files side-by-side.

A.9 Risks & Unknowns

A.9.1 Known risks

A.9.2 Unknown unknowns

• CodeDeploy rollback runtime behavior (HCL-only review).
• State bucket versioning/encryption/replication (not in backend block).
• Analytics stack (*-analytics/main.tf @ 640 LOC each) — time-boxed.
• .github/actions/for-each-* composite actions.

A.10 Technical Debt Register

A.11 Security Posture (lightweight STRIDE)

A.12 Operational Readiness

A.13 Test & Quality Signals

• Coverage: N/A (lint-only).
• Untested critical paths: modules/service blue/green; VPC peering rollback.
• Missing test types: [x] unit [x] integration [ ] e2e [ ] contract [ ] load [x] security/fuzz.

A.14 Performance & Cost Smells

• Hot paths: ECS Fargate tasks at 256/512 — fine for dev, likely undersized for prod.
• Suspected bottlenecks: Aurora max 1.0 ACU in dev; prod must raise.
• Oversized / idle: Bastion EC2 runs 24/7 (modules/bastion-host/main.tf:50-64); consider schedule-stop or SSM.

A.15 Bus-Factor & Knowledge Risk

• Only-person knowledge: State-bucket bootstrap (who originally ran aws s3 mb?); why manage_master_user_password = false.
• What breaks: Re-bootstrapping state into a new account; interpreting common-shared 6-LOC minimalism.
• Knowledge-transfer actions: doc/bootstrap.md describing state-bucket creation; ADR for the RDS password decision.

A.16 Compliance Gaps

A.17 Recommendations Summary