docs(M04): closeout - summary, audit, ledger, M05 seed

Made-with: Cursor

docs/milestones/M04/M04_audit.md

@@ -0,0 +1,75 @@
+# M04 Audit — Coverage / Security / Reproducibility Guardrails
+
+**Milestone:** M04  
+**Title:** Coverage / security / reproducibility guardrails  
+**Branch:** m04-coverage-guardrails  
+**Audit date:** 2026-03-09  
+**Mode:** DELTA AUDIT  
+**Range:** 975dda4b (M03)…47439cac (M04 closeout)  
+**CI Status:** Green (Quality 22871471473)  
+**Refactor Posture:** Behavior-Preserving  
+**Audit Verdict:** 🟢 Milestone objectives met. No runtime behavior change. Proceed.
+
+---
+
+## 1. Executive Summary
+
+M04 successfully added coverage, security, and reproducibility guardrails to CI without changing runtime behavior.
+
+**Wins:**
+* Coverage gate raised to 40% (Quality Tests)
+* pip-audit integrated (informational; remediation deferred to M27)
+* Reproducibility check: verify_pinned_deps.sh
+* CI artifact capture: coverage.xml, ci_environment.txt
+* Coverage omit config for core modules (extensions, repos, scripts, deepbooru excluded)
+* Quality unit tests: prompt_parser, API endpoints
+
+**Risks:** None identified.
+
+**Next action:** Proceed to M05 (Override isolation / temporary opts seam).
+
+---
+
+## 2. CI Evidence
+
+| Check | Result |
+|-------|--------|
+| Workflow | Quality Tests |
+| Run ID | 22871471473 |
+| Coverage | 40% (18624 stmts, 11202 missing) |
+| pip-audit | Executed (informational; vulns deferred to M27) |
+| verify_pinned_deps | ✓ Passed |
+| Artifacts | coverage.xml ✓, ci_environment.txt ✓ |
+
+---
+
+## 3. Delta Map & Blast Radius
+
+| Changed | Impact |
+|---------|--------|
+| run_quality_tests.yaml | pip-audit, verify_pinned_deps, coverage 40%, artifacts |
+| run_smoke_tests.yaml | pip-audit removed |
+| scripts/ci/verify_pinned_deps.sh | New reproducibility script |
+| pyproject.toml | [tool.coverage.run] omit |
+| test/quality/test_util_modules.py | prompt_parser unit tests |
+| test/quality/test_api_extended.py | Extended API endpoint tests |
+
+**Blast radius:** CI and test layout only. No application code behavior changed.
+
+---
+
+## 4. Category Scores
+
+| Category | Score | Notes |
+|----------|-------|-------|
+| Coverage gate | 5 | 40% enforced; omit config documented |
+| Security | 5 | pip-audit integrated; remediation deferred |
+| Reproducibility | 5 | verify_pinned_deps, ci_environment.txt |
+| CI artifact capture | 5 | coverage.xml, ci_environment.txt |
+| **Overall** | **5.0** | |
+
+---
+
+## 5. pip-audit Note
+
+pip-audit runs with `continue-on-error: true`. Vulnerabilities found are reported as warnings. Full remediation is deferred to M27 (Security/supply-chain evidence). See M04_plan.md, M04_summary.md for scope.

docs/milestones/M04/M04_plan.md

@@ -3,7 +3,7 @@
 **Milestone:** M04  
 **Title:** Coverage / security / reproducibility guardrails  
 **Branch:** `m04-coverage-guardrails`  
-**Status:** In progress  
+**Status:** Completed  
 **Depends on:** M03 (complete)
 
 ---

docs/milestones/M04/M04_summary.md

@@ -0,0 +1,71 @@
+# M04 Summary — Coverage / Security / Reproducibility Guardrails
+
+**Project:** Serena  
+**Phase:** Phase I — Baseline & Guardrails  
+**Milestone:** M04 — Coverage / security / reproducibility guardrails  
+**Status:** Closed  
+**Branch:** m04-coverage-guardrails  
+**PR:** #4 (initial); #5–#17 (fixes)  
+**Commit:** 47439cac (closeout)  
+**Quality Run:** 22871471473 ✓
+
+---
+
+## Accomplished
+
+| Item | Status |
+|------|--------|
+| Coverage gate 33% → 40% | ✓ Quality Tests |
+| pip-audit integration | ✓ Quality (informational; remediation deferred to M27) |
+| Reproducibility verification | ✓ verify_pinned_deps.sh |
+| CI artifact capture | ✓ coverage.xml, ci_environment.txt |
+| Coverage omit (pyproject.toml) | ✓ extensions-builtin, repositories, scripts, deepbooru |
+| Quality unit tests | ✓ test_util_modules (prompt_parser), test_api_extended |
+
+---
+
+## CI Layout After M04
+
+| Workflow | Trigger | Coverage | Security |
+|----------|---------|----------|----------|
+| Smoke Tests | pull_request (main) | No gate | None |
+| Quality Tests | push to main | ≥40% | pip-audit (informational) |
+| Nightly Tests | cron + dispatch | Optional | Optional |
+
+---
+
+## Coverage Configuration
+
+Coverage is focused on core application code via `pyproject.toml`:
+
+```toml
+[tool.coverage.run]
+omit = [
+    "extensions-builtin/*",
+    "repositories/*",
+    "scripts/*",
+    "modules/deepbooru_model.py",
+    "modules/deepbooru.py",
+    "*/__pycache__/*",
+    "config*.py",
+]
+```
+
+---
+
+## Guardrails
+
+- Repo: `GITHUB_REPOSITORY == m-cahill/serena`
+- PR smoke: `GITHUB_BASE_REF == main`
+- Push quality: `GITHUB_REF == refs/heads/main`
+- Coverage: combined server + pytest, fail-under=40%
+
+---
+
+## Invariants Preserved
+
+- API response schemas
+- CLI behavior
+- Extension loading
+- Generation semantics
+- CI truthfulness

docs/milestones/M05/M05_plan.md

@@ -0,0 +1,62 @@
+# M05 Plan — Override Isolation / Temporary Opts Seam
+
+**Milestone:** M05  
+**Title:** Override isolation / temporary opts seam  
+**Branch:** `m05-override-isolation`  
+**Status:** Planned  
+**Depends on:** M04 (complete)
+
+---
+
+## 1. Intent / Target
+
+Introduce the first architectural seam for Phase II: isolate override_settings application and restore from `process_images` into a reusable context manager or helper. This prepares for opts snapshot threading (M07–M08) and reduces direct mutation of global `shared.opts` during a run.
+
+No runtime behavior changes. Override application and restore logic must remain identical.
+
+---
+
+## 2. Scope Boundaries
+
+### In scope
+
+- Extract override apply/restore block in `process_images` into a context manager or helper
+- Introduce `temporary_opts(override_settings)` or equivalent seam
+- Preserve exact semantics: apply overrides before inner processing, restore in `finally`
+- Add unit test for the seam (mock opts, verify apply/restore)
+
+### Explicitly out of scope
+
+- Opts snapshot (immutable view) — M07
+- Passing opts into `process_images_inner` — M08
+- Changing override_settings semantics
+- API or UI changes
+
+---
+
+## 3. Current Behavior (Evidence)
+
+From `processing.py:823-857`:
+
+- Override settings are applied to `shared.opts` via `opts.set(key, value)` before `process_images_inner`
+- In `finally`, if `override_settings_restore_afterwards`, opts are restored
+- This block is the target for extraction
+
+---
+
+## 4. Implementation Approach
+
+1. Create helper or context manager (e.g. `modules/opts_override.py` or in `processing.py`)
+2. Replace inline override block in `process_images` with call to the helper
+3. Add minimal unit test that verifies apply/restore behavior
+4. Ensure no behavior change; smoke and quality tests pass
+
+---
+
+## 5. Definition of Done
+
+- [ ] Override apply/restore extracted to reusable seam
+- [ ] `process_images` uses the seam; logic unchanged
+- [ ] Unit test for seam
+- [ ] Smoke and Quality CI green
+- [ ] Milestone docs and ledger update

docs/milestones/M05/M05_toolcalls.md

@@ -0,0 +1,10 @@
+# M05 Tool Calls Log
+
+**Milestone:** M05 — Override Isolation / Temporary Opts Seam  
+**Branch:** m05-override-isolation
+
+---
+
+| Timestamp | Tool | Purpose | Files/Target | Status |
+|-----------|------|---------|--------------|--------|
+| (seeded) | — | M05 plan and toolcalls scaffold | docs/milestones/M05/ | done |

docs/serena.md

@@ -133,6 +133,7 @@ Core principles:
 | M01 | CI truthfulness, SHA pinning, smoke path | Completed | m01-ci-truthfulness | — | 2f664049 | Linter 22814396752 ✓; Tests 22814850488 (server ✓, 17 pass, img2img/txt2img 500) | 4.7 / 5 | 2026-03-08 |
 | M02 | API CI truthfulness, local dev guardrails | Completed | m02-api-ci-truthfulness | — | 7484170d | Linter 22831756517 ✓; Tests 22831756504 ✓ (33/33 pass) | 4.9 / 5 | 2026-03-08 |
 | M03 | Test architecture (smoke / quality / nightly) | Completed | m03-test-architecture | #2 | 975dda4b | Linter ✓; Smoke 22834384359 ✓; Quality 22834861040 ✓ | 5.0 / 5 | 2026-03-09 |
+| M04 | Coverage/security/reproducibility guardrails | Completed | m04-coverage-guardrails | #4 | 47439cac | Quality 22871471473 ✓ (coverage 40%, pip-audit, verify_pinned_deps) | 5.0 / 5 | 2026-03-09 |
 
 ---