# Real-provider reference workflows These are small acceptance workflows, not production templates or performance benchmarks. They use real OpenCode inference and independent checks; a model's claim of success is not enough. They never publish to the normal Pipelines database. The [initial live validation results](reference-workflow-results.md) record what passed, the accounting defect discovered, its focused rerun and remaining limits. | Workflow | What is exercised | Independent acceptance evidence | |---|---|---| | Incident handoff | Extract two incidents, then produce a report in a new stage | Exact IDs/severities, counts, unchanged handoff, retained sessions, one local success alert | | Edit and verify | Read/edit a release record, then run a read-only comparison | Exact final file bytes, unchanged sentinel/reference files, harness-run `cmp`, retained read/edit/shell evidence, stage-local diff and no verifier edits | | Missing input | Inspect a deliberately missing required file and take its blocked route | Failed run, blocked outcome, missing file remains missing, retained read attempt, one local failure alert | | Accounting and budget | Generate a tiny answer with a one-token stage budget | Reported USD/tokens retained, budget-exceeded failure, one attempt and stopped session | The suite permits six model stage attempts. Backend compatibility probes or format repairs may add calls. Retries are disabled; each stage has a two-minute timeout and the suite has a twelve-minute context deadline. **Provider charges may apply.** The accounting workflow also sets a $0.25 reported-usage budget. This is not a hard bill ceiling: an in-flight request can overshoot. Other workflows have no spending budget so missing accounting does not mask basic inference/tool compatibility. Notification delivery uses an isolated local HTTP endpoint, not your configured topic or phone. This tests publisher requests and retained receipts, **not real ntfy authentication/storage or phone reception**. The failure case exercises a declared model `blocked` route, not a real provider outage. The tiny budget case checks exhaustion of reported usage; it is not proof that interruption promptly stops a long-running provider response or external shell process. ## Run with native credentials Choose a model you are authorized to charge. Use a private, existing artifacts directory. Do not use normal `go test ./...` to request live inference; opt-in is explicit: ```sh mkdir -p "$HOME/Library/Caches/pipelines/reference-validation" PIPELINES_REFERENCE_LIVE=1 \ PIPELINES_REFERENCE_MODEL=opencode-go/deepseek-v4.1-flash \ PIPELINES_REFERENCE_DIR="$HOME/Library/Caches/pipelines/reference-validation" \ go test ./internal/server -run '^TestReferenceWorkflowsLive$' -count=1 -v -timeout 15m ``` `PIPELINES_OPENCODE` can select an executable. Without the live opt-in, the live test skips; `TestReferenceDefinitions` validates all four definitions without inference. These examples use the configured model selected for the initial Docker trial; other providers/models need their own acceptance run. ## Run current source inside the existing Docker installation The test binary contains the current server source. Copying it into a private test directory avoids replacing/restarting the installed server or touching its database. Choose `GOARCH=arm64` for container `uname -m` equal to `aarch64`, or `GOARCH=amd64` for `x86_64`. ```sh docker exec pipelines-pipelines-1 uname -m GOOS=linux GOARCH=arm64 go test -c -o reference.test ./internal/server docker exec pipelines-pipelines-1 mkdir -p /data/reference-validation docker exec pipelines-pipelines-1 chmod 700 /data/reference-validation docker cp reference.test pipelines-pipelines-1:/data/reference-validation/reference.test docker exec \ -e PIPELINES_REFERENCE_LIVE=1 \ -e PIPELINES_REFERENCE_MODEL=opencode-go/deepseek-v4.1-flash \ -e PIPELINES_REFERENCE_DIR=/data/reference-validation \ pipelines-pipelines-1 /data/reference-validation/reference.test \ -test.run '^TestReferenceWorkflowsLive$' -test.v -test.timeout 15m ``` This uses the container's existing provider credentials. It does not copy host credentials, log into a provider, rebuild the installation image or publish any definitions to `/data/pipelines.db`. ## Retained evidence The suite prints its uniquely created `pipelines-reference-*` directory. It retains: - `runs.db`: isolated definitions, immutable runs/visits, diffs and receipts. - One pinned Markdown definition and run/session JSON per workflow. - Per-workflow fixture directories, including final edited files. - `report.json`: model, verification verdicts, run IDs, durations and diagnostics. Keep the test console log alongside these artifacts. A failed assertion does not delete evidence; OpenCode sessions are also retained. A verified result applies to that model/backend/environment and suite revision, not every provider or task. Suite artifacts contain fixture prompts/results and should stay private. They are not a reconciled provider bill. No result is labeled verified until its independent assertions and session readback checks pass. Unbudgeted tool-free V2 generation may leave an empty OpenCode message history: the pipeline's definition, input/output and command audit are still retained. Budgeted V2 calls use tracked, deny-all prompting so usage and messages are retained. To recheck copied artifacts without inference, set `PIPELINES_REFERENCE_REVIEW_DIR` and run `TestReferenceArtifactsReview`, selecting the relevant case names with `-run` when combining artifacts from separate trials. Missing cases explicitly skip; they are never counted as verified.