mirror of
https://github.com/AUTOMATIC1111/stable-diffusion-webui.git
synced 2026-03-22 06:10:51 -07:00
M14: API runner contract test (verification milestone)
- Add test_api_runner_contract.py: proves API txt2img path invokes ProcessingRunner - No routing changes: API continues to call process_images (orchestration boundary) - Monkeypatch CI env + runner.run; call API method directly - M14_plan.md, M14_toolcalls.md: governance docs Phase III — Runner & Service Boundary Made-with: Cursor
This commit is contained in:
parent
a12028b148
commit
961297f09d
3 changed files with 359 additions and 0 deletions
274
docs/milestones/M14/M14_plan.md
Normal file
274
docs/milestones/M14/M14_plan.md
Normal file
|
|
@ -0,0 +1,274 @@
|
|||
# M14_plan — API Integration Through ProcessingRunner
|
||||
|
||||
## 1. Intent / Target
|
||||
|
||||
**Primary objective:**
|
||||
Ensure API generation paths (`/sdapi/v1/txt2img`, `/sdapi/v1/img2img`) **continue to route through** `process_images` → `ProcessingRunner`, and **lock that behavior with a contract test**.
|
||||
|
||||
This is a **governance milestone** — verification + contract expansion, **not** a routing change.
|
||||
|
||||
### Why this matters
|
||||
|
||||
* M10–M13 established the runner as a **safe execution boundary**
|
||||
* M13 proved txt2img UI already flows through it and is contract-protected
|
||||
* API is the next highest-value surface (external contract)
|
||||
* M14 proves API uses the same boundary and protects it from regression
|
||||
|
||||
**Outcome:**
|
||||
API → runner routing is **provable and protected** by contract test.
|
||||
|
||||
---
|
||||
|
||||
## 2. Scope Boundaries
|
||||
|
||||
### In Scope
|
||||
|
||||
* `modules/api/api.py`
|
||||
|
||||
* `text2imgapi`
|
||||
* `img2imgapi`
|
||||
* `modules/runtime/runner.py`
|
||||
* `modules/processing.py` (only if minimal adapter needed)
|
||||
* New contract tests
|
||||
|
||||
### Out of Scope
|
||||
|
||||
* Queue/background execution (M15)
|
||||
* Runtime extraction (M16+)
|
||||
* UI refactor (Phase V)
|
||||
* Extension behavior changes
|
||||
* Any change to request/response schemas
|
||||
|
||||
---
|
||||
|
||||
## 3. Invariants (Must Not Change)
|
||||
|
||||
From Serena invariant registry:
|
||||
|
||||
### API Contract Invariants
|
||||
|
||||
* JSON request/response schemas unchanged
|
||||
* Status codes unchanged
|
||||
* Error behavior unchanged
|
||||
|
||||
### Runtime Invariants
|
||||
|
||||
* Output images identical (same seed → same result)
|
||||
* Metadata / infotext unchanged
|
||||
* File save paths unchanged
|
||||
|
||||
### System Invariants
|
||||
|
||||
* Extensions behave identically
|
||||
* CLI/UI unaffected
|
||||
* Coverage ≥ 40% maintained
|
||||
|
||||
---
|
||||
|
||||
## 4. Verification Plan
|
||||
|
||||
### Tests (Required)
|
||||
|
||||
#### 1. API → Runner Contract Test (NEW)
|
||||
|
||||
* Monkeypatch `ProcessingRunner.execute`
|
||||
* Call `/sdapi/v1/txt2img`
|
||||
* Assert runner is invoked
|
||||
|
||||
#### 2. Existing API Tests
|
||||
|
||||
* Must pass unchanged
|
||||
* Ensures no schema drift
|
||||
|
||||
#### 3. Smoke Tests
|
||||
|
||||
* Full API roundtrip
|
||||
* Ensures server boot + endpoint works
|
||||
|
||||
---
|
||||
|
||||
### CI Signals (Required)
|
||||
|
||||
* Linter ✓
|
||||
* Smoke Tests ✓
|
||||
* Quality Tests ✓
|
||||
* Coverage gate ≥ 40% ✓
|
||||
|
||||
---
|
||||
|
||||
### Evidence
|
||||
|
||||
* `M14_run1.md` (PR CI)
|
||||
* `M14_run2.md` (post-merge CI)
|
||||
* Contract test proves routing
|
||||
|
||||
---
|
||||
|
||||
## 5. Implementation Steps (Small, Reversible)
|
||||
|
||||
### Step 1 — No Routing Changes
|
||||
|
||||
**KEEP existing API code:**
|
||||
|
||||
```python
|
||||
processed = process_images(p) # ✅ DO NOT CHANGE
|
||||
```
|
||||
|
||||
**DO NOT** replace with `runner.run()`. `process_images` is the orchestration boundary; bypassing it would break override_settings, model reload, script callbacks, and extension behavior.
|
||||
|
||||
---
|
||||
|
||||
### Step 2 — Add Contract Test
|
||||
|
||||
Create:
|
||||
|
||||
```
|
||||
test/quality/test_api_runner_contract.py
|
||||
```
|
||||
|
||||
Test:
|
||||
|
||||
* Monkeypatch `os.getenv("CI")` to `"false"` so real execution path runs
|
||||
* Monkeypatch `ProcessingRunner.run` to track invocation
|
||||
* Call API method directly (not HTTP) to avoid server startup
|
||||
* Assert runner is invoked when API executes
|
||||
|
||||
---
|
||||
|
||||
### Step 3 — Validate Both Paths
|
||||
|
||||
Confirm:
|
||||
|
||||
* `text2imgapi` → `process_images` → runner ✓
|
||||
* `img2imgapi` → `process_images` → runner ✓
|
||||
|
||||
Both already flow through `process_images`; contract test locks txt2img; img2img uses identical pattern.
|
||||
|
||||
---
|
||||
|
||||
### Step 4 — Run CI + Verify
|
||||
|
||||
* All tests pass
|
||||
* No diff in outputs
|
||||
* Coverage unchanged or improved
|
||||
|
||||
---
|
||||
|
||||
## 6. Risk & Rollback Plan
|
||||
|
||||
### Risk Level: LOW
|
||||
|
||||
Why:
|
||||
|
||||
* `process_images` already delegates to runner (M10)
|
||||
* This is a **routing normalization**, not new logic
|
||||
|
||||
---
|
||||
|
||||
### Potential Risks
|
||||
|
||||
| Risk | Mitigation |
|
||||
| --------------------------------- | ------------------ |
|
||||
| API bypasses runner accidentally | Contract test |
|
||||
| Subtle response formatting change | Existing API tests |
|
||||
| Extension interaction edge case | Smoke tests |
|
||||
|
||||
---
|
||||
|
||||
### Rollback Plan
|
||||
|
||||
* Revert API call site change only
|
||||
* No data/model changes required
|
||||
* Single-file rollback (`api/api.py`)
|
||||
|
||||
---
|
||||
|
||||
## 7. Deliverables
|
||||
|
||||
### Code
|
||||
|
||||
* No API routing changes
|
||||
* New contract test only
|
||||
|
||||
### Tests
|
||||
|
||||
* `test_api_runner_contract.py`
|
||||
|
||||
### Docs
|
||||
|
||||
* `docs/milestones/M14/M14_plan.md`
|
||||
* `M14_run1.md`, `M14_run2.md`
|
||||
* `M14_summary.md`
|
||||
* `M14_audit.md`
|
||||
|
||||
### Ledger
|
||||
|
||||
* Add M14 row to `docs/serena.md` with:
|
||||
|
||||
* commit SHA
|
||||
* CI run IDs
|
||||
* audit score
|
||||
|
||||
---
|
||||
|
||||
## 8. Acceptance Criteria
|
||||
|
||||
### Functional
|
||||
|
||||
* API endpoints produce identical outputs
|
||||
* No schema or contract changes
|
||||
|
||||
### Structural
|
||||
|
||||
* API continues to call `process_images` (orchestration boundary)
|
||||
* All generation flows through process_images → runner
|
||||
* Contract test proves API path invokes runner
|
||||
|
||||
### Verification
|
||||
|
||||
* Contract test passes
|
||||
* CI fully green
|
||||
* Coverage ≥ 40%
|
||||
|
||||
---
|
||||
|
||||
## 9. Architectural Outcome
|
||||
|
||||
After M14:
|
||||
|
||||
### Unchanged (Verified)
|
||||
|
||||
```
|
||||
API → process_images → ProcessingRunner → process_images_inner
|
||||
UI → process_images → ProcessingRunner → process_images_inner
|
||||
```
|
||||
|
||||
**Result:**
|
||||
|
||||
* API → runner flow **provably true** and regression-protected
|
||||
* Contract test locks API path
|
||||
* No behavior change; governance milestone only
|
||||
|
||||
---
|
||||
|
||||
## 10. Next Milestone Preview
|
||||
|
||||
### M15 — Background / Queue Runner Preparation
|
||||
|
||||
Will build on M14 by:
|
||||
|
||||
* Introducing async/queued execution
|
||||
* Adding cancellation + lifecycle control
|
||||
* Enabling multi-request orchestration
|
||||
|
||||
---
|
||||
|
||||
# ✅ Final Instruction for Cursor
|
||||
|
||||
Implement M14 exactly as specified:
|
||||
|
||||
* Minimal diff
|
||||
* No behavior change
|
||||
* Add contract test
|
||||
* Verify CI
|
||||
* Produce run, summary, audit, and ledger update
|
||||
21
docs/milestones/M14/M14_toolcalls.md
Normal file
21
docs/milestones/M14/M14_toolcalls.md
Normal file
|
|
@ -0,0 +1,21 @@
|
|||
# M14 Toolcalls
|
||||
|
||||
## Context
|
||||
|
||||
Milestone: M14 — API integration (runner contract enforcement)
|
||||
Phase: Phase III — Runner & Service Boundary
|
||||
|
||||
## Actions
|
||||
|
||||
| Timestamp | Tool | Purpose | Files/Target | Status |
|
||||
|-----------|------|---------|--------------|--------|
|
||||
| (start) | write | Create M14_toolcalls.md | docs/milestones/M14/ | done |
|
||||
| | write | Create M14_plan.md | docs/milestones/M14/ | done |
|
||||
| | search_replace | Update M14_plan (verification-only scope) | docs/milestones/M14/M14_plan.md | done |
|
||||
| | write | Create test_api_runner_contract.py | test/quality/ | done |
|
||||
| | run | Create m14 branch, push, open PR | git | pending |
|
||||
|
||||
## Notes
|
||||
|
||||
- No routing changes applied (behavior preserved)
|
||||
- M14 is verification + contract milestone
|
||||
64
test/quality/test_api_runner_contract.py
Normal file
64
test/quality/test_api_runner_contract.py
Normal file
|
|
@ -0,0 +1,64 @@
|
|||
"""M14 contract test: API txt2img path uses ProcessingRunner.
|
||||
|
||||
Verifies that the API execution path flows through process_images → runner,
|
||||
not direct process_images_inner calls. No routing changes; verification only.
|
||||
"""
|
||||
from threading import Lock
|
||||
|
||||
import pytest
|
||||
|
||||
|
||||
def test_api_txt2img_uses_runner(monkeypatch, initialize):
|
||||
"""API txt2img path invokes ProcessingRunner when process_images is called."""
|
||||
from fastapi import FastAPI
|
||||
|
||||
from modules.api.api import Api
|
||||
from modules.api import models
|
||||
from modules.processing import Processed
|
||||
from modules.runtime.runner import ProcessingRunner
|
||||
|
||||
called = {"run": False}
|
||||
|
||||
# Force real execution path (bypass CI early return)
|
||||
monkeypatch.setenv("CI", "false")
|
||||
|
||||
# Patch runner to track invocation
|
||||
original_run = ProcessingRunner.run
|
||||
|
||||
def tracking_run(self, request):
|
||||
called["run"] = True
|
||||
return original_run(self, request)
|
||||
|
||||
monkeypatch.setattr(ProcessingRunner, "run", tracking_run)
|
||||
|
||||
# Mock process_images_inner to avoid full pipeline
|
||||
def fake_inner(proc):
|
||||
return Processed(proc, [], seed=-1, info="", comments="")
|
||||
|
||||
import modules.processing as proc_mod
|
||||
|
||||
monkeypatch.setattr(proc_mod, "process_images_inner", fake_inner)
|
||||
|
||||
# Mock model reload and token merging to avoid model/device ops
|
||||
import modules.sd_models as sd_models_mod
|
||||
|
||||
monkeypatch.setattr(sd_models_mod, "reload_model_weights", lambda: None)
|
||||
monkeypatch.setattr(sd_models_mod, "apply_token_merging", lambda m, r: None)
|
||||
|
||||
# Call API method directly (no HTTP)
|
||||
app = FastAPI()
|
||||
api = Api(app, Lock())
|
||||
|
||||
req = models.StableDiffusionTxt2ImgProcessingAPI(
|
||||
prompt="test",
|
||||
steps=1,
|
||||
width=64,
|
||||
height=64,
|
||||
)
|
||||
|
||||
result = api.text2imgapi(req)
|
||||
|
||||
assert called["run"] is True
|
||||
assert result is not None
|
||||
assert hasattr(result, "images")
|
||||
assert hasattr(result, "info")
|
||||
Loading…
Add table
Add a link
Reference in a new issue