M12: Add runner instrumentation hooks (on_prepare, on_execute, on_finalize)

- Add optional no-op hooks to ProcessingRunner lifecycle
- Hooks invoked: prepare -> on_prepare -> execute -> on_execute -> finalize -> on_finalize
- Add test_runner_hooks_called contract test
- No runtime behavior change; structural seam for M13+ progress/cancellation

Made-with: Cursor

docs/milestones/M12/M12_plan.md

@@ -1,24 +1,239 @@
 # M12 — Runner Instrumentation Surface
 
-Phase: **Phase III — Runner & Service Boundary**
+Phase: Phase III — Runner & Service Boundary  
 Status: Planned
 
 ---
 
 # 1. Intent / Target
 
-Add optional instrumentation hooks to the lifecycle stages.
+Introduce an **instrumentation hook surface** on the ProcessingRunner lifecycle.
 
-(To be expanded.)
+The runner currently exposes:
+
+prepare → execute → finalize
+
+This milestone introduces **optional lifecycle hooks** that allow later milestones
+to attach progress tracking, tracing, and cancellation signals.
+
+The hooks must default to **no-op behavior** so the processing pipeline remains unchanged.
 
 ---
 
 # 2. Scope Boundaries
 
-### In scope
+## In scope
 
-(To be defined.)
+• Add optional instrumentation hooks to ProcessingRunner  
+• Keep hooks disabled by default  
+• Ensure lifecycle execution order is unchanged  
+• Add contract tests verifying hook invocation
 
-### Out of scope
+## Out of scope
 
-(To be defined.)
+• No runtime behavior change  
+• No progress reporting yet  
+• No cancellation yet  
+• No threading / async  
+• No API changes  
+• No CLI changes  
+
+Instrumentation is only structural in this milestone.
+
+---
+
+# 3. Invariants
+
+| Surface | Invariant | Verification |
+|---------|-----------|--------------|
+| CLI behavior | identical outputs | smoke tests |
+| API responses | unchanged schemas | smoke tests |
+| Processing results | identical images/metadata | quality tests |
+| Runner lifecycle | prepare → execute → finalize | contract tests |
+| Coverage | ≥ 40% | CI gate |
+
+---
+
+# 4. Verification Plan
+
+CI must remain green.
+
+Expected checks:
+
+| Check | Expected |
+|-------|----------|
+| Linter | pass |
+| Smoke Tests | pass |
+| Quality Tests | pass (post-merge) |
+| Coverage | ≥ 40% |
+
+Manual verification:
+
+```bash
+pytest
+```
+
+---
+
+# 5. Implementation Steps
+
+## Step 1 — Add instrumentation hooks
+
+Modify:
+
+```
+modules/runtime/runner.py
+```
+
+Add hook methods:
+
+```python
+class ProcessingRunner:
+
+    def run(self, request):
+        state = self.prepare(request)
+
+        self.on_prepare(state)
+
+        result = self.execute(state)
+
+        self.on_execute(state, result)
+
+        result = self.finalize(state, result)
+
+        self.on_finalize(state, result)
+
+        return result
+
+    def on_prepare(self, state):
+        pass
+
+    def on_execute(self, state, result):
+        pass
+
+    def on_finalize(self, state, result):
+        pass
+```
+
+Hooks must be **no-op by default**.
+
+---
+
+## Step 2 — Ensure lifecycle remains unchanged
+
+Lifecycle order must still be:
+
+```
+prepare → on_prepare → execute → on_execute → finalize → on_finalize
+```
+
+And `execute` must still call:
+
+```
+process_images_inner(state.processing)
+```
+
+---
+
+## Step 3 — Extend contract tests
+
+File:
+
+```
+test/quality/test_processing_runner.py
+```
+
+Add test verifying hook invocation.
+
+Example:
+
+```python
+def test_runner_hooks_called(monkeypatch, initialize):
+
+    calls = []
+
+    class TestRunner(ProcessingRunner):
+
+        def on_prepare(self, state):
+            calls.append("prepare_hook")
+
+        def on_execute(self, state, result):
+            calls.append("execute_hook")
+
+        def on_finalize(self, state, result):
+            calls.append("finalize_hook")
+
+        def execute(self, state):
+            return "result"
+
+    runner = TestRunner()
+    runner.run(ProcessingRequest(processing="dummy"))
+
+    assert calls == ["prepare_hook", "execute_hook", "finalize_hook"]
+```
+
+---
+
+# 6. Risk & Rollback Plan
+
+Risk level: **Low**
+
+Changes are internal to the runner.
+
+Rollback:
+
+1. revert runner instrumentation commit
+2. restore previous runner lifecycle
+3. re-run CI
+
+No runtime data or external interfaces change.
+
+---
+
+# 7. Deliverables
+
+Code:
+
+```
+modules/runtime/runner.py
+```
+
+Tests:
+
+```
+test/quality/test_processing_runner.py
+```
+
+Docs:
+
+```
+docs/milestones/M12/M12_plan.md
+docs/milestones/M12/M12_toolcalls.md
+docs/milestones/M12/M12_run1.md
+docs/milestones/M12/M12_summary.md
+docs/milestones/M12/M12_audit.md
+```
+
+Ledger update:
+
+```
+docs/serena.md
+```
+
+Tag:
+
+```
+v0.0.12-m12
+```
+
+---
+
+# 8. Exit Criteria
+
+M12 closes when:
+
+• PR CI passes
+• post-merge Quality Tests pass
+• instrumentation runner merged
+• ledger updated
+• tag created

docs/milestones/M12/M12_run1.md

@@ -0,0 +1,5 @@
+# M12 Run 1 — CI Analysis
+
+*To be populated after PR CI run completes.*
+
+Workflow identity, change context, and analysis per `docs/prompts/RefactorWorkflowPrompt.md`.

docs/milestones/M12/M12_toolcalls.md

@@ -4,3 +4,11 @@ Implementation toolcalls for Cursor execution.
 
 | Timestamp | Tool | Purpose | Files/Target | Status |
 |-----------|------|---------|--------------|--------|
+| 2026-03-12 | run | Checkout m12-runner-instrumentation branch | git | done |
+| 2026-03-12 | write | Replace M12 plan with full plan | docs/milestones/M12/M12_plan.md | done |
+| 2026-03-12 | search_replace | Add instrumentation hooks to runner.py | modules/runtime/runner.py | done |
+| 2026-03-12 | search_replace | Add test_runner_hooks_called | test/quality/test_processing_runner.py | done |
+| 2026-03-12 | run | Run pytest quality tests | pytest | skipped (local env missing deps; CI will verify) |
+| 2026-03-12 | write | Create M12_run1.md placeholder | docs/milestones/M12/M12_run1.md | done |
+| 2026-03-12 | search_replace | Update ledger with M12 in progress | docs/serena.md | done |
+| 2026-03-12 | run | Commit M12 implementation | git | pending |

docs/serena.md

@@ -142,6 +142,7 @@ Core principles:
 | M09 | Execution context introduction | Completed | m09-execution-context | #26 | 2c6a2510 | Quality 22986731960 ✓ | 5.0 / 5 | 2026-03-12 |
 | M10 | ProcessingRunner skeleton | Completed | m10-processing-runner | #27 (+ #28 fix) | 0d11b587 | Quality 22988627838 ✓ | 5.0 / 5 | 2026-03-12 |
 | M11 | Runner lifecycle surface | Completed | m11-runner-lifecycle | #30 | 08ac1c0e | Quality 22989978348 ✓ | 5.0 / 5 | 2026-03-12 |
+| M12 | Runtime instrumentation hooks | In progress | m12-runner-instrumentation | — | — | — | — | — |
 
 **M05:** Introduced `temporary_opts()` context manager — first Phase II runtime seam. Isolates override_settings mutation from global `shared.opts`; preserves behavior (opts.set, setattr restore, k in opts.data). Model/VAE reload and token merging remain in process_images. Enables future opts snapshot injection (M07).
 
@@ -157,6 +158,8 @@ Core principles:
 
 **M11:** Introduced lifecycle surface on ProcessingRunner: prepare → execute → finalize. run() delegates through stages; pass-through behavior; identical outputs. test_runner_lifecycle_order verifies lifecycle structure. Stable execution surface enables M12 instrumentation, progress hooks, cancellation, queue runners.
 
+**M12:** (In progress) Adds optional instrumentation hooks on ProcessingRunner: on_prepare, on_execute, on_finalize. Hooks no-op by default; lifecycle order prepare → on_prepare → execute → on_execute → finalize → on_finalize. Enables M13+ progress, cancellation, queue runners.
+
 ---
 
 ## 5. Standing Invariants

modules/runtime/runner.py

@@ -2,6 +2,7 @@
 
 M10: Thin adapter around process_images_inner. No behavior changes.
 M11: Lifecycle surface (prepare → execute → finalize). Pass-through behavior.
+M12: Instrumentation hooks (on_prepare, on_execute, on_finalize). No-op by default.
 """
 
 
@@ -16,13 +17,18 @@ class ProcessingRunner:
     """
     Unified execution entrypoint for Serena processing pipeline.
     M11: Exposes lifecycle stages for future instrumentation.
+    M12: Optional instrumentation hooks (no-op by default).
     """
 
     def run(self, request):
         """Execute processing pipeline via lifecycle stages."""
         state = self.prepare(request)
+        self.on_prepare(state)
         result = self.execute(state)
-        return self.finalize(state, result)
+        self.on_execute(state, result)
+        result = self.finalize(state, result)
+        self.on_finalize(state, result)
+        return result
 
     def prepare(self, request):
         """Lifecycle stage 1: prepare request. Pass-through in M11."""
@@ -36,3 +42,12 @@ class ProcessingRunner:
     def finalize(self, state, result):
         """Lifecycle stage 3: finalize. Pass-through in M11."""
         return result
+
+    def on_prepare(self, state):
+        """Instrumentation hook after prepare. No-op by default."""
+
+    def on_execute(self, state, result):
+        """Instrumentation hook after execute. No-op by default."""
+
+    def on_finalize(self, state, result):
+        """Instrumentation hook after finalize. No-op by default."""

test/quality/test_processing_runner.py

@@ -1,7 +1,30 @@
-"""Contract tests for ProcessingRunner (M10 runner skeleton, M11 lifecycle)."""
+"""Contract tests for ProcessingRunner (M10 runner skeleton, M11 lifecycle, M12 hooks)."""
 from modules.runtime.runner import ProcessingRunner, ProcessingRequest
 
 
+def test_runner_hooks_called(monkeypatch, initialize):
+    """ProcessingRunner invokes on_prepare, on_execute, on_finalize in order."""
+    calls = []
+
+    class TestRunner(ProcessingRunner):
+        def on_prepare(self, state):
+            calls.append("prepare_hook")
+
+        def on_execute(self, state, result):
+            calls.append("execute_hook")
+
+        def on_finalize(self, state, result):
+            calls.append("finalize_hook")
+
+        def execute(self, state):
+            return "result"
+
+    runner = TestRunner()
+    runner.run(ProcessingRequest(processing="dummy"))
+
+    assert calls == ["prepare_hook", "execute_hook", "finalize_hook"]
+
+
 def test_runner_lifecycle_order(monkeypatch, initialize):
     """ProcessingRunner invokes prepare → execute → finalize in order."""
     calls = []