mirrors/stable-diffusion-webui
1
Fork 0
mirror of https://github.com/AUTOMATIC1111/stable-diffusion-webui.git synced 2026-03-24 23:21:09 -07:00
Code Issues Projects Releases Packages Wiki Activity
stable-diffusion-webui/docs/serena.md
Michael Cahill 292115c8f0 docs(M09): closeout — run1, run2, audit, summary, ledger 
Made-with: Cursor
2026-03-11 21:39:12 -07:00

185 lines
9.4 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# Serena — Refactor Program Ledger
**Program name:** Serena
**Source repo:** AUTOMATIC1111/stable-diffusion-webui
**Fork workspace:** m-cahill/serena (origin)
**Source of truth:** This document
**Posture:** Behavior-preserving by default, audit-first, milestone-governed
---
## 1. Project Identity
Serena is a governed refactor program for AUTOMATIC1111/stable-diffusion-webui. The goal is to transform the codebase from its current state (audit score ~2.4–2.6/5) to a 5/5 architecture with clear separation of concerns, testable runtime, and stable extension API.
### Serena Refactor Principles
This refactor program follows strict **behavior-preserving governance**.
Core principles:
1. **Behavior preservation by default**
Existing runtime behavior must remain stable unless explicitly changed.
2. **Small milestones**
Each milestone introduces minimal surface change.
3. **Runtime seams before architecture changes**
Isolation is introduced before structural refactors.
4. **Extension compatibility**
The extension ecosystem must remain functional unless explicitly versioned.
5. **Evidence-based closeout**
Each milestone must end with verifiable CI evidence.
**Source-of-truth hierarchy:**
1. `docs/serena.md` — This ledger (phases, milestones, invariants)
2. `docs/sdwebuirefactoraudit.md` — Baseline audit (architecture, modularity, refactor strategy)
3. `docs/sdwebuiaudit.md` — Supplementary audit (DX, phased plan)
4. Milestone docs under `docs/milestones/MNN/`
---
## 2. Current Baseline
| Item | Value |
|------|-------|
| **Audited baseline SHA** | `82a973c04367123ae98bd9abdf80d9eda9b910e2` |
| **Baseline tag** | `baseline-pre-refactor` (annotated, immutable) |
| **Initial audit score** | 2.4 / 5 (sdwebuirefactoraudit) |
| **Upstream** | https://github.com/AUTOMATIC1111/stable-diffusion-webui.git |
**Top architectural problems (from audit):**
- Global state hub: `shared.opts`, `shared.state`, `shared.sd_model` used by dozens of modules
- No test tiers or coverage gate
- God modules: `processing.py` (~1793 LOC), `ui.py` (~984 LOC), `api/api.py` (~750 LOC)
- Dependency and CI hygiene: mixed pinning, lockfile gitignored, actions use tags not SHA
---
## 3. Proposed Phase Map (Provisional)
*Can evolve after M00 evidence.*
### Phase I — Baseline & Guardrails (M00–M04)
| Milestone | Title |
|-----------|-------|
| M00 | Program kickoff, baseline freeze, phase map, E2E verification |
| M01 | CI truthfulness, SHA pinning, smoke path |
| M02 | Local dev guardrails, CONTRIBUTING, repeatable verification |
| M03 | Test architecture (smoke / quality / nightly) |
| M04 | Coverage/security/reproducibility guardrails |
### Phase II — Runtime Seam Preparation (M05–M09)
| Milestone | Title |
|-----------|-------|
| M05 | Override isolation / temporary opts seam |
| M06 | Prompt/seed prep extraction |
| M07 | Opts snapshot introduction |
| M08 | process_images_inner snapshot threading |
| M09 | Execution context/state seam |
### Phase III — Runner & Service Boundary (M10–M14)
| Milestone | Title |
|-----------|-------|
| M10 | ProcessingRunner skeleton |
| M11 | txt2img via runner |
| M12 | img2img via runner |
| M13 | API adoption of runner |
| M14 | UI adoption of runner |
### Phase IV — Runtime Extraction (M15–M19)
| Milestone | Title |
|-----------|-------|
| M15 | Runtime module extraction |
| M16 | Sampler runner extraction |
| M17 | Decode/save separation |
| M18 | Model provider interface |
| M19 | Runtime tests with mockable boundaries |
### Phase V — UI & Extension Stabilization (M20–M24)
| Milestone | Title |
|-----------|-------|
| M20 | UI tab registry |
| M21 | txt2img/img2img tab modularization |
| M22 | Settings/extensions modularization |
| M23 | Extension API version/contract |
| M24 | Deprecation/compatibility scaffolding |
### Phase VI — Hardening & Reproducibility (M25–M29)
| Milestone | Title |
|-----------|-------|
| M25 | Locked manifests / npm ci / CI env stabilization |
| M26 | Coverage and complexity gates |
| M27 | Security/supply-chain evidence |
| M28 | Health/perf verification |
| M29 | QA/evidence publishing |
### Phase VII — Release Lock / 5.0 Closure (M30–M32)
| Milestone | Title |
|-----------|-------|
| M30 | Architecture lock |
| M31 | Evidence/audit closure |
| M32 | Release-ready 5/5 close |
---
## 4. Milestone Ledger
| Milestone | Title | Status | Branch | PR | Commit | CI Run(s) | Audit Score / Notes | Completed At |
|-----------|-------|--------|--------|-----|--------|-----------|---------------------|--------------|
| M00 | Program kickoff, baseline freeze, phase map, E2E verification | Completed | m00-kickoff-baseline-e2e | — | cdfe1285 | Linter 22794525690 ✓; Tests 22794525698 ✗ (pre-existing CLIP/pkg_resources) | Baseline 2.4/5 | 2025-03-07 |
| M01 | CI truthfulness, SHA pinning, smoke path | Completed | m01-ci-truthfulness | — | 2f664049 | Linter 22814396752 ✓; Tests 22814850488 (server ✓, 17 pass, img2img/txt2img 500) | 4.7 / 5 | 2026-03-08 |
| M02 | API CI truthfulness, local dev guardrails | Completed | m02-api-ci-truthfulness | — | 7484170d | Linter 22831756517 ✓; Tests 22831756504 ✓ (33/33 pass) | 4.9 / 5 | 2026-03-08 |
| M03 | Test architecture (smoke / quality / nightly) | Completed | m03-test-architecture | #2 | 975dda4b | Linter ✓; Smoke 22834384359 ✓; Quality 22834861040 ✓ | 5.0 / 5 | 2026-03-09 |
| M04 | Coverage/security/reproducibility guardrails | Completed | m04-coverage-guardrails | #4 | 47439cac | Quality 22871471473 ✓ (coverage 40%, pip-audit, verify_pinned_deps) | 5.0 / 5 | 2026-03-09 |
| M05 | Override isolation / temporary opts seam | Completed | m05-override-isolation | #18 (+ #19 fix) | ae161cbb | Quality 22888808682 ✓ | 5.0 / 5 | 2026-03-10 |
| M06 | Prompt/seed prep extraction | Completed | m06-prompt-seed-prep | #20 | 6744152a | Quality 22890285319 ✓ | 5.0 / 5 | 2026-03-10 |
| M07 | Opts snapshot introduction | Completed | m07-opts-snapshot | #22 | 8ea50d35 | Quality 22983583947 ✓ | 5.0 / 5 | 2026-03-12 |
| M08 | Opts snapshot threading | Completed | m08-snapshot-threading | #24 | 710a0abd | Quality 22984445599 ✓ | 5.0 / 5 | 2026-03-12 |
| M09 | Execution context introduction | Completed | m09-execution-context | #26 | 2c6a2510 | Quality 22986731960 ✓ | 5.0 / 5 | 2026-03-12 |
**M05:** Introduced `temporary_opts()` context manager — first Phase II runtime seam. Isolates override_settings mutation from global `shared.opts`; preserves behavior (opts.set, setattr restore, k in opts.data). Model/VAE reload and token merging remain in process_images. Enables future opts snapshot injection (M07).
**M06:** Extracted `prepare_prompt_seed_state(p)` into `modules/prompt_seed_prep.py`. Second Phase II runtime seam. Populates p.all_seeds and p.all_subseeds; setup_prompts and fill_fields_from_opts unchanged. Enables M07 opts snapshot and M09 execution context.
**M07:** Introduced `create_opts_snapshot(opts)` in `modules/opts_snapshot.py`. Third Phase II runtime seam. Captures deterministic snapshot of opts.data in process_images_inner after prepare_prompt_seed_state; stored on p.opts_snapshot. Write-only in M07; enables M08 snapshot threading.
**M08:** Threaded p.opts_snapshot into process_images_inner for save-related reads. Fourth Phase II runtime seam. Migrated 12 opts (save_images_before_face_restoration, samples_format, grid_save, etc.) from shared.opts to p.opts_snapshot. save_samples(), sample_hr_pass(), metadata unchanged. Enables M09 execution context.
**M09:** Introduced RuntimeContext in modules/runtime_context.py. Fifth Phase II runtime seam. Attached p.runtime_context in process_images_inner() after opts_snapshot (model, opts_snapshot, device, state, cmd_opts). Write-only in M09; no migration of shared.* reads yet. Completes Phase II — Runtime Seam Preparation. Enables Phase III ProcessingRunner.
---
## 5. Standing Invariants
Repo-wide non-negotiables for this program:
- **No silent behavior drift** — All changes must preserve or explicitly document behavior
- **No CI weakening** — Do not relax checks, thresholds, or truthfulness
- **Preserve extension behavior** — Unless intentionally versioned in a milestone
- **Preserve API/UI semantics** — Unless milestone explicitly approves change
- **Evidence-first closeout** — Every milestone closes with documented evidence
---
## 6. Invariant Registry
These invariants must remain stable throughout the Serena refactor program unless explicitly revised by a milestone.
This registry provides **cross-milestone contract stability**.
| Surface | Description | Verification |
| -------------------- | ------------------------------------------------------- | --------------------------- |
| CLI | Command flags and output behavior remain stable | Snapshot tests |
| API | JSON response schemas remain compatible | Contract tests |
| File formats | Serialized artifacts and saved images remain compatible | Schema validation |
| Public modules | Import surfaces remain available unless versioned | API compatibility tests |
| Extension API | Extension loading behavior and hooks remain stable | Extension integration tests |
| Generation semantics | txt2img / img2img parameter behavior preserved | E2E smoke tests |
Notes:
* Any invariant modification must be documented in the milestone plan.
* Regression verification must be automated where possible.
Reference in a new issue View git blame Copy permalink