Install current verified harness releases inside Docker, preserve persistent accounts and settings, and provide separate interactive and pipeline model defaults with terminal onboarding and fuzzy pickers. Add Docker-aware authoring transport, Pi stage execution, bounded workflow controls, reliable final-result readback and usage settlement, and finite short-lived CLI pipe draining. Keep dated installation evidence and packaged authoring guidance consistent. Verified with a real Docker installation, both harness stages and interactive authors, the built-in plan/implement/review starter, independent host race/vet checks, installer tests, and packaged skill tests. Generated release receipts stay ignored in clean checkouts.
4.7 KiB
Reference validation — October 10, 2026
Execution: existing Docker installation, Linux aarch64, OpenCode v2.0.20,
model opencode-go/deepseek-v4.1-flash. The test binaries were compiled from the
current checkout and run against a separate SQLite database and isolated fixtures.
No definitions were published to the normal server database and no phone alerts
were sent. Notification checks used an isolated local HTTP endpoint.
| Reference workflow | Live result | Duration |
|---|---|---|
| Incident extraction → report | Passed: exact incident evidence/counts, handoff, session readback and local success receipt | 6.63s |
| Edit → independently verify | Passed: exact file bytes, untouched fixtures, harness-run cmp, completed read/edit/shell evidence, retained stage diff |
14.60s |
| Missing required input | Passed: deliberate blocked outcome terminates as failed; missing file not invented; read attempt and local failure receipt retained | 4.21s |
| Accounting and one-token budget | Initially failed the acceptance test; passed after the accounting fix below | 2.43s on focused rerun |
Defect found and fixed
The first accounting run succeeded despite a one-token limit. Its retained V2
session had no messages and zero usage counters, even though /generate
returned a valid result. Polling that session treated placeholder zeros as known
accounting and did not enforce the intended limit.
Budgeted V2 text calls now use tracked /prompt → bounded active-session polling
→ message readback, with all tools denied. Their messages are retained in the
stage audit, usage is read from the tracked session, and a completed
token-budgeted response with zero reported tokens fails closed. Native V1 handling
and unbudgeted text generation keep their existing paths.
A later reliability fix replaced the V2 .../experimental/session/{id}/wait
request, which timed out at the CLI's own ~six-minute limit (HTTP 499) on long
stages, with roughly one-second GET /api/session/active polls under the stage
context/deadline. That change has fake-CLI regression coverage in internal/stage;
it has not yet been exercised by a fresh live run.
The focused live rerun retained 952 reported tokens and $0.00016440, failed
with stage budget exceeded, and stopped its owned session. This is reported
usage, not an independently reconciled provider bill. A one-token limit can be
overshot by a single in-flight request; this run demonstrates exhaustion handling,
not an exact hard cap or proof of immediate mid-stream cancellation.
The three previously passing workflows were not charged again. Their copied artifacts were rechecked with current independent assertions; the fixed accounting artifacts were rechecked separately. There has not been a fresh full-suite live run of all four cases after the fix.
Evidence and reproduction
Retained container directories for this trial:
/data/reference-validation/pipelines-reference-284076920: first complete run./data/reference-validation/pipelines-reference-2247403140: fixed accounting rerun.
Each has isolated SQLite history, pinned definitions, run/session JSON, fixture
files and report.json. Host copies are in the session's approved temporary
directory under reference-first/ and reference-accounting-fixed/.
Commands that performed live inference:
docker exec -e PIPELINES_REFERENCE_LIVE=1 \
-e PIPELINES_REFERENCE_MODEL=opencode-go/deepseek-v4.1-flash \
-e PIPELINES_REFERENCE_DIR=/data/reference-validation \
pipelines-pipelines-1 /data/reference-validation/reference-20261010.test \
-test.run '^TestReferenceWorkflowsLive$' -test.v -test.timeout 15m
docker exec -e PIPELINES_REFERENCE_LIVE=1 \
-e PIPELINES_REFERENCE_MODEL=opencode-go/deepseek-v4.1-flash \
-e PIPELINES_REFERENCE_DIR=/data/reference-validation \
pipelines-pipelines-1 /data/reference-validation/reference-20261010-fixed.test \
-test.run '^TestReferenceWorkflowsLive/accounting-and-budget$' -test.v -test.timeout 5m
TestReferenceArtifactsReview rechecks copied run JSON/fixtures without inference;
PIPELINES_REFERENCE_REVIEW_DIR selects the copy and -run selects its cases.
See reference workflows for a fresh run and its opt-in,
cost, retention and safety boundaries.
Remaining unverified
- Other models/providers and native V1 schema/tool compatibility.
- Real ntfy authentication/storage and actual phone reception.
- Actual provider outages, timeouts, long-stream interruption and retry behavior.
- General workflow correctness beyond these bounded fixtures.
The fix is in the source/test binaries, not deployed to the installed Docker server. Rebuild the installation when ready to use it for ordinary runs.