Allow launching runs with whole-run and per-stage model overrides

Add `pipelines run --model PROVIDER/MODEL` and repeatable
`--stage-model STAGE=PROVIDER/MODEL`, resolved with precedence
per-stage > whole-run > the model pinned in the pipeline version.

- A shared pure resolver (`pipeline.ParseModelOverride` /
  `pipeline.EffectiveModels`) owns the precedence and validation: every value
  must be a `provider/model` pair (`stage.ValidModel`), every stage key must
  exist in the pinned graph, an unknown key fails naming the valid keys, and a
  per-stage key that names a non-model stage is refused.
- The server validates against the pinned version before admission (no run row
  is ever inserted for an invalid request), persists `model`/`stage_models` on
  the run and `provider`/`model` on each step, applies the effective pair to a
  clone of the stage definition so the stage and its repair/formatter path share
  it, overlays the returned run graph with the effective pair (presentation
  only), and reuses the recorded overrides on retry.
- The run record echoes the overrides it was launched with and each step records
  the provider/model it actually ran with; the TUI Evidence pane renders that
  effective pair. The pinned pipeline version and `show --version N` are never
  modified.
- CLI help, `docs/cli.md` and the packaged authoring skill references document
  the flags and the precedence. A TUI override picker remains a documented
  follow-up; the CLI is the deliverable.

Tests: pure precedence/validation table tests, store round-trip and
request-key-mismatch plus migration, server admission/application/overlay/retry
with a controlled executor, CLI flag/help/malformed-value tests, and a TUI
evidence test.
This commit is contained in:
2026-10-16 23:33:53 +01:00
parent 80232a4ffe
commit fae1005962
19 changed files with 912 additions and 31 deletions
+36 -1
View File
@@ -219,7 +219,7 @@ Submitting activates included schedules, so review them before publishing.
| Browse templates/source | `templates --json`, `template KEY --source` |
| Create or edit a definition | `new`, `edit`, `drafts` |
| Validate and publish | `validate FILE`, `submit FILE` |
| Start a run | `run NAME PROMPT [--title TITLE] --yes --json` |
| Start a run | `run NAME PROMPT [--title TITLE] [--model P/M] [--stage-model S=P/M] --yes --json` |
| Run history/inspection | `runs --json`, `inspect ID --json` |
| Rename a run | `title ID "TITLE" --json` |
| Decide a waiting human stage | `decide ID [STAGE] --option KEY [--feedback TEXT] --json` |
@@ -262,6 +262,41 @@ pipeline are distinguishable. Rename after the fact with `pipelines title
<run-id> "<title>"`; the run's input, steps and execution are unchanged. The
stored prompt is unchanged, and listing/renaming never calls a model.
### Model overrides at launch
A pipeline version pins one model per stage. Without republishing, a run can
choose different models at launch:
```sh
pipelines run "bug fix" "Fix the layout" --json --yes \
--model openai/gpt-5 \
--stage-model review=openai/o4-mini
```
- `--model PROVIDER/MODEL` overrides the model for **every** model stage of that
run.
- `--stage-model STAGE=PROVIDER/MODEL` overrides **one** stage and is repeatable,
so any number of stages can be overridden. A repeated key for the same stage is
last-wins.
- Precedence: a per-stage override beats the whole-run override, which beats the
model pinned in the pipeline version.
Overrides apply to that run only. The pipeline definition and its immutable
version are never modified, `show <name> --version N` still returns the pinned
source and graph, and the run records the `model` and `stage_models` it was
launched with so it stays reproducible; `retry` reuses them. At launch every
value must be a `provider/model` pair of the configured form and every stage key
must exist in the pinned version — an unknown key fails naming the valid keys and
no run is created. A per-stage key that names a non-model stage is refused
because that stage never runs a model; `--model` on a pipeline with no model stage
is accepted and inert. The effective model reaches the stage and its
repair/formatter path from the same resolved definition, and each step's evidence
records the provider/model it actually ran with (`inspect <id> --json` exposes
per-step `provider`/`model`, and the run's graph shows the effective pair). The
CLI is the deliverable; a TUI picker for choosing overrides is a documented
follow-up, not half-built.
On either the runs list or run inspector, press `P` (or use `:pipeline`) to open
the selected run's exact pipeline version. Navigation does not cancel the run.