87 lines
5.7 KiB
Markdown
87 lines
5.7 KiB
Markdown
# Explicit run reports and session export completeness
|
|
|
|
Launch with `pipelines run NAME PROMPT --output-format html` (or `markdown`),
|
|
or choose Report format in the web run dialog. The wire field is
|
|
`output_format: "html" | "markdown"`; omission preserves prior behavior. It is
|
|
stored with the run, compared for request-key replay and inherited by retries.
|
|
This never changes a pipeline definition and is not inferred from prompt text.
|
|
|
|
Before success, the server takes the first nonblank string in the terminal
|
|
output's `markdown`, `report`, `result`, `summary`, `message` fields, renders a
|
|
report deterministically and records the verified artifact on the terminal
|
|
visit. Fixed filenames are `public/report.html` and `public/report.md`.
|
|
Both source and rendered output are capped at 2 MiB. Existing targets and
|
|
symlinked directories are refused; reports are not overwritten. Missing or
|
|
empty textual output, invalid targets and failed verification fail the run.
|
|
HTML has inline styling, no executable raw HTML, and a restrictive content
|
|
security policy preventing external resource loading. This is a structural
|
|
file contract, **not factual validation** or proof that every task was fulfilled.
|
|
|
|
Session export is owned by the runtime, not the model stage. No stage prompt,
|
|
definition or execution contract changes. Ownership is recorded during execution;
|
|
export evidence also snapshots the original session ID/backend/directory before
|
|
the run finishes. Evidence records `status` as `complete`, `empty`, `failed`,
|
|
`partial`, or `unavailable`, plus attempt count and capture time. Zero-message
|
|
readbacks are retried within the runtime's 10-second export budget and then
|
|
recorded as **empty with a warning**, without producing success-looking HTML.
|
|
Known temporary transport/store failures and interrupted JSON with verified
|
|
identity can retry (at most 3 attempts). Wrong-session responses, unsupported
|
|
envelopes and permanent CLI failures do not retry. A 200 ms delay does not prove
|
|
that settling writes caused an invalid readback. Expected array/data envelopes
|
|
are normalized, always retaining exact session identity validation.
|
|
|
|
`pipelines repair-session-export RUN_ID SEQUENCE --json` invokes the
|
|
`repair_session_export` protocol command (`run_id`, `export_sequence`). It is
|
|
read-only to the native session: no generation, resumption, interruption or
|
|
stage execution. It requires a settled run/session and exact original ownership
|
|
metadata, never guesses the backend, and appends a repair attempt to the run.
|
|
Successful readback writes `public/session-repair-SEQUENCE-RANDOM.html` and
|
|
records its bounded inventory entry. Original exports/evidence and run outcomes
|
|
are preserved. Failed/empty/partial attempts remain inspectable warnings; CLI
|
|
JSON returns the repair record itself and exits nonzero for incomplete evidence.
|
|
There are at most 20 manual repair attempts per visit. Repairs cannot guarantee
|
|
recovery if the upstream session or original directory is gone. The web pane
|
|
explains original failure and lists repair attempts/links separately.
|
|
|
|
## Durable automatic recovery
|
|
|
|
Incomplete native exports now enqueue a `session_export_jobs` row in the same
|
|
transaction as `FinishStep`. The snapshot records the exact original session ID,
|
|
backend, directory and harness reference. Runtime startup also reconciles missing
|
|
jobs from persisted ownership/steps, including interrupted initial capture; it
|
|
never guesses missing ownership or edits original evidence. Reconciliation is
|
|
bounded to 256 newly discovered jobs per sweep (startup, then every 30 seconds).
|
|
|
|
A runtime-owned worker claims jobs only for settled runs/steps, with no active
|
|
agent in that run or using that session, and with `cleanup_unconfirmed=0` on
|
|
current main. The runner's in-memory cleanup gate also defers execution without
|
|
spending a retry. Each automatic attempt uses the existing versioned, read-only
|
|
repair path and verifies ownership against the durable snapshot before readback.
|
|
No stage prompt, pipeline definition, stage execution or run outcome changes.
|
|
|
|
The first attempt is due after 5 seconds; further attempts wait 20 and 60 seconds.
|
|
There are at most 3 automatic claims, each with the existing 10-second readback
|
|
budget. A 45-second persisted lease recovers interrupted workers after restart;
|
|
attempt counts/backoff are not reset, including process kills. Saved complete
|
|
repair evidence can acknowledge a job after a crash without exporting again.
|
|
Expired final-attempt leases become exhausted, not an infinite retry loop.
|
|
Wrong-session readbacks never qualify as complete evidence. Failed/empty/partial
|
|
repairs and exhausted-job diagnostics remain visible in run inspection and the
|
|
web visit pane (`session_export_recovery`). Originals remain unchanged.
|
|
|
|
Worker shutdown follows runner context cancellation and current main's
|
|
`Store.Done()` (including cancellation of in-flight readback). The small startup
|
|
helper detects that lifecycle interface without modifying this branch's older
|
|
store/server-lock implementation. Its compatibility fallback exits on a closed
|
|
database; current main's lifecycle/cleanup implementations must be retained when
|
|
integrating. Upstream loss, missing original ownership/directory, inventory
|
|
limits or sessions exceeding export caps can still leave exhausted evidence;
|
|
manual versioned repair remains available. Browser captures for recovery status
|
|
are not captured under the no-live-runs audit constraint.
|
|
|
|
No production service, live run or historical data was modified for this audit.
|
|
Unit/fixture tests cover reports, failure gates, retry inheritance, idempotency,
|
|
migration, transient/permanent export readback, empty-export diagnostics and
|
|
versioned repairs preserving originals. Browser screenshots are not captured; new web states
|
|
remain partial pending a permitted isolated browser review.
|