Merge pull request #22 from m-cahill/m07-opts-snapshot

M07: Opts snapshot introduction

docs/milestones/M07/M07_plan.md

@@ -3,10 +3,361 @@
 Project: Serena
 Phase: Phase II — Runtime Seam Preparation
 Milestone: M07
-Title: Opts snapshot introduction  
+Title: **Opts snapshot introduction**
 Posture: Behavior-preserving refactor
-Target: TBD
+Target: Introduce a stable **options snapshot mechanism** for generation runs.
+
+Context: M06 isolated prompt/seed preparation into `prepare_prompt_seed_state(p)`, establishing a second runtime seam.
+
+M07 introduces the **third seam**: a deterministic snapshot of `shared.opts` used for the duration of a generation run.
+
+This prepares the runtime pipeline for dependency injection and removal of global option mutation in later milestones.
+
+The milestone map defines this sequence explicitly:
+
+* M05 — override isolation
+* M06 — prompt/seed prep extraction
+* **M07 — opts snapshot introduction**
+* M08 — snapshot threading into `process_images_inner`
+* M09 — execution context/state seam
 
 ---
 
-*Plan to be populated.*
+# 1. Intent / Target
+
+The current pipeline mutates `shared.opts` during generation via:
+
+```
+override_settings
+```
+
+This occurs in `process_images()` where overrides are temporarily applied and later restored.
+
+This creates problems:
+
+* hidden global mutation
+* non-deterministic runtime state
+* difficult testing
+
+The goal of **M07** is to introduce an **immutable snapshot of opts** for generation runs without changing runtime behavior.
+
+The snapshot is **not yet threaded through the runtime** (that happens in M08).
+
+Instead, this milestone:
+
+1. Creates a function that builds an **opts snapshot object**
+2. Captures it once at the start of `process_images()`
+3. Stores it on the processing object (`p.opts_snapshot`)
+
+The runtime will continue to read from `shared.opts` for now.
+
+This is purely **infrastructure preparation**.
+
+---
+
+# 2. Scope Boundaries
+
+## In Scope
+
+Introduce a new helper module:
+
+```
+modules/opts_snapshot.py
+```
+
+Add:
+
+```
+create_opts_snapshot(opts)
+```
+
+Responsibilities:
+
+* Copy relevant values from `shared.opts`
+* Return an immutable snapshot structure
+* Attach snapshot to `StableDiffusionProcessing`
+
+Example:
+
+```
+p.opts_snapshot = create_opts_snapshot(shared.opts)
+```
+
+Snapshot should include frequently accessed generation settings such as:
+
+* outdir configuration
+* sampler defaults
+* VAE settings
+* precision settings
+* clip skip
+* model settings
+
+Exact contents may initially mirror `opts.data`.
+
+---
+
+## Out of Scope
+
+This milestone **does NOT**:
+
+* replace reads of `shared.opts`
+* remove override logic
+* change how options are applied
+* modify extension access patterns
+* alter UI/API option behavior
+
+Those changes happen later (M08–M09).
+
+---
+
+# 3. Invariants
+
+The following must remain identical.
+
+### Generation behavior
+
+Identical inputs must produce identical outputs.
+
+### Options behavior
+
+* override_settings logic unchanged
+* UI settings unchanged
+* CLI flags unchanged
+
+### Extension compatibility
+
+Extensions accessing:
+
+```
+shared.opts
+```
+
+must continue working.
+
+### API compatibility
+
+txt2img and img2img responses remain unchanged.
+
+### Runtime state
+
+No behavioral change to model loading or VAE logic.
+
+---
+
+# 4. Verification Plan
+
+## CI Gates
+
+All existing CI checks must pass:
+
+* Smoke Tests
+* Linter
+* Quality Tests
+* Coverage ≥ 40%
+
+The CI suite already executes the generation path via API smoke tests.
+
+If option behavior breaks, smoke tests will fail.
+
+---
+
+## Local verification
+
+Recommended commands:
+
+```
+pytest test/smoke
+pytest test/quality
+```
+
+---
+
+# 5. Implementation Steps
+
+## Step 1 — Create snapshot module
+
+Create file:
+
+```
+modules/opts_snapshot.py
+```
+
+Add:
+
+```
+create_opts_snapshot(opts)
+```
+
+Example structure:
+
+```python
+from types import SimpleNamespace
+
+def create_opts_snapshot(opts):
+    return SimpleNamespace(**opts.data.copy())
+```
+
+Important properties:
+
+* shallow copy of `opts.data`
+* no mutation allowed
+
+---
+
+## Step 2 — Capture snapshot
+
+Inside:
+
+```
+process_images_inner(p)
+```
+
+Add:
+
+```
+p.opts_snapshot = create_opts_snapshot(shared.opts)
+```
+
+Placement:
+
+Immediately after prompt/seed preparation.
+
+---
+
+## Step 3 — Preserve override behavior
+
+The snapshot must capture **post-override settings**.
+
+Therefore ensure the snapshot is created **after override_settings are applied**.
+
+Current order:
+
+```
+apply override_settings
+prepare prompts/seeds
+```
+
+New order:
+
+```
+apply override_settings
+prepare prompts/seeds
+capture opts snapshot
+```
+
+---
+
+## Step 4 — Avoid runtime behavior change
+
+The snapshot should **not yet replace any option reads**.
+
+Example:
+
+Keep existing code:
+
+```
+opts.outdir_samples
+```
+
+Do NOT change to:
+
+```
+p.opts_snapshot.outdir_samples
+```
+
+That change occurs in **M08**.
+
+---
+
+## Step 5 — Minimal structural validation
+
+Optionally add a small test:
+
+```
+test_opts_snapshot.py
+```
+
+Validate:
+
+* snapshot contains expected attributes
+* snapshot does not mutate when opts change afterward
+
+This is optional.
+
+---
+
+# 6. Risk & Rollback Plan
+
+Risk: **Very low**
+
+Changes:
+
+```
+modules/opts_snapshot.py
+modules/processing.py
+```
+
+Failure scenarios:
+
+* snapshot created before overrides
+* extension code accidentally reading snapshot
+
+Mitigation:
+
+* ensure snapshot is write-only in M07
+* snapshot stored only on `p`
+
+Rollback:
+
+Revert the commit.
+
+---
+
+# 7. Deliverables
+
+Expected files:
+
+```
+modules/opts_snapshot.py        (new)
+modules/processing.py           (modified)
+docs/milestones/M07/*           (plan, toolcalls, CI reports)
+```
+
+No other modules should change.
+
+---
+
+# 8. Expected Outcome
+
+After M07:
+
+* generation runs capture a **deterministic opts snapshot**
+* runtime seams now include:
+
+```
+temporary_opts()           (M05)
+prepare_prompt_seed_state  (M06)
+opts snapshot              (M07)
+```
+
+These seams prepare the system for:
+
+```
+M08 — snapshot threading into process_images_inner
+M09 — execution context/state seam
+```
+
+Together they begin removing the **global state dependency** identified in the pre-refactor audit.
+
+---
+
+# 9. Success Criteria
+
+M07 is complete when:
+
+* opts snapshot helper exists
+* snapshot attached to `StableDiffusionProcessing`
+* no runtime behavior change
+* CI fully green
+* milestone artifacts generated
+* ledger updated

docs/milestones/M07/M07_run1.md

@@ -0,0 +1,112 @@
+# M07 CI Run 1 — Opts Snapshot Introduction
+
+**Date:** 2026-03-10  
+**Branch:** m07-opts-snapshot  
+**PR:** #22  
+**Trigger:** pull_request (PR to main)
+
+---
+
+## 1. Workflow Identity
+
+| Workflow | Run ID | Trigger | Branch | Commit | Status |
+|----------|--------|---------|--------|--------|--------|
+| Smoke Tests | 22920401686 | pull_request | m07-opts-snapshot | 87c8fe86 | ✓ success |
+| Linter | 22920401661 | pull_request | m07-opts-snapshot | 87c8fe86 | ✓ success |
+
+**Quality Tests:** Not yet run (triggered on push to main; will run post-merge).
+
+---
+
+## 2. Change Context
+
+| Item | Value |
+|------|-------|
+| Milestone | M07 — Opts snapshot introduction |
+| Phase | Phase II — Runtime Seam Preparation |
+| Posture | Behavior-preserving |
+| Refactor target | `modules/opts_snapshot.py` (new), `modules/processing.py` (modified) |
+| Run type | First CI verification of M07 implementation |
+
+---
+
+## 3. Step 1 — Workflow Inventory
+
+### Smoke Tests (22920401686)
+
+| Job / Step | Required? | Purpose | Pass/Fail |
+|------------|-----------|---------|-----------|
+| Verify repository | Yes | Guardrail: m-cahill/serena only | ✓ |
+| Verify base branch | Yes | Guardrail: PR targets main | ✓ |
+| Checkout Code | Yes | Fetch PR branch | ✓ |
+| Set up Python 3.10 | Yes | Runtime | ✓ |
+| Cache models | Yes | Deterministic model path | ✓ |
+| Install test dependencies | Yes | pytest, coverage | ✓ |
+| Install runtime dependencies | Yes | torch, CLIP, open_clip, requirements_versions | ✓ |
+| Create stub repositories | Yes | CI fake inference support | ✓ |
+| Setup environment | Yes | launch.py --exit | ✓ |
+| Smoke startup | Yes | Verify server can start | ✓ |
+| Start test server | Yes | Live server for API tests | ✓ |
+| **Run smoke tests** | **Yes** | **pytest test/smoke** | **✓** |
+| Kill test server | Yes | Cleanup | ✓ |
+| Upload main app output | No (always) | Artifact for debugging | ✓ |
+
+**Duration:** 2m44s
+
+### Linter (22920401661)
+
+| Job | Required? | Purpose | Pass/Fail |
+|-----|-----------|---------|-----------|
+| ruff | Yes | Python lint | ✓ |
+| eslint | Yes | JS lint | ✓ |
+
+---
+
+## 4. Step 2 — Refactor Signal Integrity
+
+### A) Tests
+
+- **Tier:** Smoke only (test/smoke)
+- **Coverage of refactor target:** Smoke tests exercise txt2img and img2img API endpoints, which call `process_images()` → `process_images_inner()` → `prepare_prompt_seed_state(p)` → `create_opts_snapshot(shared.opts)`. The opts snapshot is captured on the critical path before sampling.
+- **Failures:** None
+- **Golden/snapshot:** Smoke tests use CI fake inference (deterministic 1×1 PNG); no golden image comparison. API contract (response schema) is exercised.
+- **Missing:** Quality tier tests will run on push to main (coverage ≥40%, pip-audit, verify_pinned_deps).
+
+### B) Coverage
+
+- Smoke run does not enforce a coverage gate (Quality Tests do, on push to main).
+- Coverage gate: ≥40% (M04 baseline).
+- Post-merge Quality run will report coverage.
+
+---
+
+## 5. Step 3 — Invariant Verification
+
+| Invariant | Verification | Status |
+|-----------|--------------|--------|
+| Generation behavior unchanged | Smoke tests pass; snapshot is write-only | ✓ |
+| Override logic unchanged | temporary_opts() unchanged; snapshot captured after overrides | ✓ |
+| Extension compatibility | shared.opts reads unchanged; p.opts_snapshot not yet read by runtime | ✓ |
+| API compatibility | txt2img/img2img smoke tests pass | ✓ |
+| No opts reads replaced | All opts. / shared.opts reads remain; M08 will thread snapshot | ✓ |
+
+---
+
+## 6. Blast Radius
+
+**Files changed:**
+- `modules/opts_snapshot.py` (new)
+- `modules/processing.py` (modified)
+- `docs/milestones/M07/*`
+
+**No other modules changed.** Invariant registry surfaces (CLI, API, file formats, extension API, generation semantics) preserved.
+
+---
+
+## 7. Verdict
+
+**CI Status:** Green (Linter ✓, Smoke Tests ✓)
+
+**Refactor posture:** Behavior-preserving infrastructure preparation. Snapshot is write-only; no runtime reads replaced.
+
+**Next step:** Await merge permission. Post-merge Quality Tests will run (coverage, pip-audit, verify_pinned_deps). M07 run analysis complete; ready for M07 audit and summary generation after closeout.

docs/milestones/M07/M07_toolcalls.md

@@ -1,5 +1,17 @@
 # M07 Toolcalls — Opts Snapshot Introduction
 
-| Timestamp | Tool | Purpose | Files/Target | Status |
+Implementation toolcalls for Cursor execution.
-|-----------|------|---------|-------------|--------|
+
-| | | | | |
+(To be filled during milestone execution.)
+
+| Timestamp | Tool | Purpose | Files | Status |
+|-----------|------|---------|-------|--------|
+| 2026-03-10 | write | Create M07_plan.md | docs/milestones/M07/M07_plan.md | ✓ |
+| 2026-03-10 | write | Create M07_toolcalls.md | docs/milestones/M07/M07_toolcalls.md | ✓ |
+| 2026-03-10 | write | Create opts_snapshot module | modules/opts_snapshot.py | ✓ |
+| 2026-03-10 | search_replace | Add create_opts_snapshot import | modules/processing.py | ✓ |
+| 2026-03-10 | search_replace | Add opts snapshot capture after prepare_prompt_seed_state | modules/processing.py | ✓ |
+| 2026-03-10 | run | Local pytest (smoke/quality) | — | Skipped: full env (gradio, torch) not installed locally; CI will verify |
+| 2026-03-10 | gh pr create | Create PR #22 | m-cahill/serena | ✓ |
+| 2026-03-10 | gh run list | Monitor CI (Linter, Smoke) | — | ✓ Linter 22920401661, Smoke 22920401686 |
+| 2026-03-10 | write | Create M07_run1.md | docs/milestones/M07/M07_run1.md | ✓ |

modules/opts_snapshot.py

@@ -0,0 +1,15 @@
+"""Opts snapshot for generation runs.
+
+M07: Deterministic snapshot of shared.opts for the duration of a run.
+Behavior-preserving: shallow copy of opts.data.
+Not yet threaded through runtime.
+"""
+from types import SimpleNamespace
+
+
+def create_opts_snapshot(opts):
+    """
+    Create an immutable snapshot of shared.opts for a generation run.
+    Behavior-preserving: shallow copy of opts.data.
+    """
+    return SimpleNamespace(**opts.data.copy())

modules/processing.py

@@ -26,6 +26,7 @@ import modules.paths as paths
 import modules.face_restoration
 import modules.images as images
 import modules.styles
+from modules.opts_snapshot import create_opts_snapshot
 import modules.prompt_seed_prep as prompt_seed_prep
 import modules.runtime_utils as runtime_utils
 import modules.sd_models as sd_models
@@ -891,6 +892,7 @@ def process_images_inner(p: StableDiffusionProcessing) -> Processed:
     p.setup_prompts()
 
     prompt_seed_prep.prepare_prompt_seed_state(p)
+    p.opts_snapshot = create_opts_snapshot(shared.opts)
 
     if os.path.exists(cmd_opts.embeddings_dir) and not p.do_not_reload_embeddings:
         model_hijack.embedding_db.load_textual_inversion_embeddings()