Improve workflow UI, retain stage diffs, and add portable pipeline transfers
Make OpenCode authoring responsive and expand the managed skill references. Add conflict-aware import/export, real-provider reference workflows, and tracked V2 generation for reliable budget accounting.
This commit is contained in:
@@ -0,0 +1,78 @@
|
||||
# Authoring session review
|
||||
|
||||
## Observed in the supplied October 10 export
|
||||
|
||||
- The initial prompt contained a draft-context path and CLI prefix, but no edit
|
||||
request. Before asking for intent, the agent made eight tool calls: skill load,
|
||||
three file reads, and four CLI commands. It already recognized that a question
|
||||
was needed, but continued investigating.
|
||||
- Those CLI calls listed templates, called `show` without its required argument,
|
||||
fetched an entire template already represented by the draft, and listed unrelated
|
||||
retained drafts. None was needed to ask what should change.
|
||||
- The question presented a generic feature menu instead of a direct edit question.
|
||||
- After the user requested success/error notifications, the agent made twenty
|
||||
discovery calls before its first edit: two globs, two shell commands, nine reads
|
||||
and seven greps. It repeatedly debated adding model/native delivery stages.
|
||||
Its format reference
|
||||
pointed to project-local ntfy docs, causing a repository search and a long read
|
||||
of installation, networking and phone setup information unrelated to this edit.
|
||||
- The agent treated an optional native-publisher hint as a recommended design,
|
||||
overlooking a minimal combination of existing stage hooks.
|
||||
- It eventually chose the correct failure-only research hook and success/failure
|
||||
report hook, validated the draft, and published version 2 after the user's explicit
|
||||
approval. Those approval and immutable-version boundaries were respected.
|
||||
- It also added a report attachment and renamed the output field from `report` to
|
||||
`markdown` even though only a notification was requested. Its final summary
|
||||
overstated this as a guarantee of exactly one notification per run; stage hooks
|
||||
do not cover every cancellation/crash/limit failure or guarantee phone reception.
|
||||
|
||||
The export's creation/update times span 3m56s, including the user's reply. It has
|
||||
no per-message/tool timestamps; it cannot establish time to first interaction or
|
||||
separate inference, tool, network and user-wait latency.
|
||||
|
||||
## Changes
|
||||
|
||||
1. Launch prompts distinguish absent intent from an explicit request. With no
|
||||
request they instruct OpenCode to ask first, before reads or CLI calls. With
|
||||
a request they ask for a brief acknowledgement followed by targeted reading.
|
||||
Launch remains one direct OpenCode command; no extra discovery or inference.
|
||||
2. The skill puts intent before investigation and replaces generic questionnaires
|
||||
with one open-ended question. After the reply it reads context, draft and only
|
||||
relevant references. It preserves the authoritative CLI executable/URL prefix
|
||||
and discourages speculative help calls, bare `show`, catalog comparisons,
|
||||
unrelated draft listings and repeated unchanged reads.
|
||||
3. A packaged `references/notifications.md` gives a self-contained decision guide:
|
||||
hooks for ordinary alerts; native delivery only for explicitly routed delivery.
|
||||
It includes the research/report terminal-path recipe, attachment requirements,
|
||||
failure semantics and cancellation/crash/recovery-loop limitations. Attachments
|
||||
and output-contract changes are added only when requested. No checkout
|
||||
of Pipelines is needed during ordinary authoring.
|
||||
4. Native-delivery examples use the system connection rather than suggesting a
|
||||
second topic/token. Ordinary event alerts leave output contracts unchanged.
|
||||
5. Skill installation includes the new reference and upgrades old managed
|
||||
installations without overwriting modified files. Existing drafts, published
|
||||
versions, schedules, tool permissions and approval boundaries are unchanged.
|
||||
|
||||
## Verification and limits
|
||||
|
||||
New-pipeline guidance also avoids asking for a name/stage count/model/tools before
|
||||
understanding the task. It suggests one minimal graph and retains model inheritance,
|
||||
loading optional notification/example references only when relevant. Regression
|
||||
tests execute both new and edit launch commands against a stub executable and
|
||||
check that each launches exactly once with ask-first guidance, without session,
|
||||
help or API discovery. A command-preparation benchmark measures only local launch
|
||||
preparation, not OpenCode startup or real-provider response latency.
|
||||
|
||||
`go test ./internal/author ./internal/server -run
|
||||
'TestPrompt|TestCommandLaunches|TestInstalledSkill|TestSkillUpgrade|TestResearchTerminalNotificationRecipe'
|
||||
-count=1` passed. These checks execute the captured launch command, inspect generated
|
||||
prompts, upgrade the installed skill in an isolated directory, and execute the
|
||||
research/report notification recipe against SQLite and a mock HTTP ntfy endpoint.
|
||||
The recipe covers success, early/final blocked outcomes, early/final execution
|
||||
errors and failed delivery without changing the run verdict or sending intermediate
|
||||
success events.
|
||||
|
||||
The prompt/skill changes guide model behavior; they do not enforce a model's first
|
||||
action. Real-provider time-to-first-question, compliance with the new instructions,
|
||||
and actual phone reception are **not tested**. No new model inference or phone
|
||||
message is needed to install these authoring changes.
|
||||
+16
-3
@@ -1,5 +1,13 @@
|
||||
# CLI and agent authoring
|
||||
|
||||
In the TUI, publishing is labelled **Create pipeline** for a new definition and
|
||||
**Update pipeline** for a new immutable version. The CLI `submit` command and
|
||||
`:submit` shortcut remain compatible. Templates offer description/graph panes:
|
||||
Tab expands the focused pane, `n/p` selects a template, and `hjkl` pans the focused
|
||||
graph. Create opens one confirmation with an editable, prefilled template name;
|
||||
Enter creates the pipeline without a separate naming or editor step. Duplicate
|
||||
names are rejected rather than updating an existing pipeline.
|
||||
|
||||
`pipelines help` lists commands; `pipelines help edit` (or `edit --help`) shows
|
||||
command-specific flags and examples. Installed commands work from any directory.
|
||||
The installed command is a compiled Go binary. Optional autostart uses the OS user
|
||||
@@ -18,7 +26,8 @@ Search includes pipeline name, version, status, outcome, stage and diagnostic;
|
||||
closing the list. The filter survives inspection/back and live list refreshes.
|
||||
Wide terminals
|
||||
show runs and pipelines side by side, with stacked panels on narrow terminals.
|
||||
Run cards show status, duration, start time and the current stage when active.
|
||||
Home run cards prioritize reported results, status, completed visits and duration;
|
||||
active cards also show the current stage. Start/finish times remain in run details.
|
||||
The usage summary is aggregated over all stored history, including retry attempts:
|
||||
total/active runs, outcomes, success rate, average completed duration, pipeline
|
||||
usage and a zero-filled 14-day daily activity chart (UTC). Success rate is
|
||||
@@ -350,8 +359,12 @@ not a result from a different published definition. Run opens focused on results
|
||||
with output before input and the graph below. Failed or ongoing runs preserve
|
||||
partial output and completed stage reports without labeling them final success.
|
||||
These previews quote recorded output; they do not independently verify model claims.
|
||||
Tab still expands the focused graph, stage list, or result pane; `n`/`p` still
|
||||
navigate recorded visits, including repeated stages.
|
||||
Tab and Shift+Tab toggle **Details ↔ Graph** only. The visit selector lives inside
|
||||
Details; `n`/`p` still navigate recorded visits, including repeated stages, and `0`
|
||||
returns to the overview. `j`/`k` scroll the report; the focused graph uses `hjkl`
|
||||
for panning and `c` to center. Secondary progress/input/evidence/execution sections
|
||||
are collapsed initially. Small inactive graph previews collapse instead of showing
|
||||
severed boxes. See [UI/UX review](ui-ux-review.md) for screen-by-screen feedback.
|
||||
|
||||
## Verification scope
|
||||
|
||||
|
||||
@@ -226,6 +226,16 @@ or the normal `~/.config/opencode/skills/pipelines` default. Only unchanged file
|
||||
previously installed by Pipelines are updated. A user-managed/modified skill is
|
||||
never overwritten; preserve or rename it before running `pipelines skill install`.
|
||||
Installation does not require OpenCode itself. Uninstall retains the skill and drafts.
|
||||
The packaged format, notification, operations, transfer and verification references
|
||||
are self-contained; agents read only those relevant to the task. Authoring
|
||||
does not require Pipelines source in the current repository. Without a task prompt,
|
||||
OpenCode is instructed to ask what to create/change before investigating files or
|
||||
running CLI commands. See [authoring session review](authoring-session-review.md)
|
||||
for the reasoning and verification limits.
|
||||
New features and confirmed limitations are maintained in the packaged skill alongside
|
||||
the topic docs. Skill tests check that every embedded reference is installed and
|
||||
its links resolve. Refresh the global managed skill from current source with
|
||||
`go run ./cmd/pipelines skill install`; this does not deploy server/runtime fixes.
|
||||
|
||||
The bundled examples retain `opencode-go/deepseek-v4.1-flash`. Availability and
|
||||
authentication for that model are runtime requirements, not installation checks.
|
||||
|
||||
+25
-4
@@ -7,6 +7,26 @@ audit as well as the ntfy attachment. A separate HTML report viewer is not inclu
|
||||
|
||||
## Per-stage notifications (recommended)
|
||||
|
||||
### One terminal-path alert without adding stages
|
||||
|
||||
For a sequential workflow such as `research → report`, where intermediate
|
||||
blocked/error routes go straight to `#failure`, add `on=failure` notification
|
||||
hooks to intermediate stages and `on=success,failure` to the final stage.
|
||||
Keep the graph and output contracts unchanged; omit `artifacts` for event-only
|
||||
alerts. This avoids intermediate success alerts while covering early stage
|
||||
errors and the final success/error result. Putting a hook only on the last stage
|
||||
misses earlier failures; putting success hooks on every stage sends progress alerts.
|
||||
|
||||
These are stage hooks, not a universal run-finalizer: they run before the final
|
||||
status is saved, and do not guarantee notifications for cancellation, crashes,
|
||||
persistence failures or runtime step/visit limits. Recovery routes and loops can
|
||||
notify before termination or more than once. An explicit native publisher stage
|
||||
is not necessary for ordinary completion alerts and cannot reliably consume the
|
||||
required `title`/`summary`/`markdown` after an upstream execution error.
|
||||
|
||||
The installed authoring skill includes this recipe and native-delivery details in
|
||||
`references/notifications.md`; authoring does not require a checkout of this project.
|
||||
|
||||
````markdown
|
||||
# stage compose
|
||||
|
||||
@@ -60,7 +80,7 @@ inline credentials. They require `destination` and a `success` edge:
|
||||
|
||||
```configs
|
||||
type=ntfy
|
||||
destination=anime
|
||||
destination=system
|
||||
```
|
||||
|
||||
```edges
|
||||
@@ -72,13 +92,14 @@ error -> #failure
|
||||
Configure the **pipeline server process** outside the pipeline file:
|
||||
|
||||
```sh
|
||||
export PIPELINES_NTFY_ANIME_URL='https://ntfy.your-domain.example/anime'
|
||||
# Set PIPELINES_NTFY_ANIME_TOKEN securely to a publishing user's access token.
|
||||
export PIPELINES_NTFY_URL='https://ntfy.your-domain.example/pipelines'
|
||||
# Set PIPELINES_NTFY_TOKEN_FILE to the existing publishing token file.
|
||||
go run ./cmd/pipelines-server
|
||||
```
|
||||
|
||||
The URL includes one topic, without credentials, query parameters, or a fragment.
|
||||
Other destinations use the same pattern: `destination=reports` reads
|
||||
`destination=system` uses the same connection as hooks. Other destinations use
|
||||
the named pattern: `destination=reports` reads
|
||||
`PIPELINES_NTFY_REPORTS_URL` and `PIPELINES_NTFY_REPORTS_TOKEN` (or `_TOKEN_FILE`
|
||||
pointing to a secret file). Set only one token source. Destination names
|
||||
use lowercase letters, digits, and underscores. A token is optional only when
|
||||
|
||||
@@ -0,0 +1,81 @@
|
||||
# Export and import pipeline definitions
|
||||
|
||||
In the TUI, run `:export`. Search, move with ↑/↓, and use **Space** to select
|
||||
pipelines. **Ctrl+A** selects/deselects the filtered list; selections survive
|
||||
filter changes. **Enter** exports; **Esc** cancels without writing a file.
|
||||
The ZIP is saved in the directory where you launched the client; its path is
|
||||
recorded in the TUI logs (`:logs`). Existing files are never overwritten.
|
||||
|
||||
Run `:import`, enter the ZIP path, select definitions, and confirm publication.
|
||||
For each selected name that already exists, import compares the stored definition
|
||||
with the incoming source. Identical source is skipped automatically; differing
|
||||
source offers:
|
||||
|
||||
- **Keep existing** (`k`): skip this incoming definition.
|
||||
- **Keep both** (`b`): create a copy named `NAME (imported)` (numbered when needed).
|
||||
- **Use imported** (`i`): publish it as a new version of the existing pipeline.
|
||||
All previous versions and run history are retained, not overwritten.
|
||||
- **View diff** (`d`): show a highlighted existing-versus-imported source preview.
|
||||
- **Cancel** (`c` or empty input): abort before any selected definition is published.
|
||||
|
||||
Comparison uses exact source bytes, not exported version numbers: independent
|
||||
servers may have different version numbers for the same definition. Whitespace-only
|
||||
changes are therefore differences too. Choices are resolved before the final
|
||||
publication confirmation; `--yes` does not choose a conflict resolution for you.
|
||||
The destination is the currently connected server, even when it runs on a
|
||||
different computer. The ZIP is read/written on the client computer.
|
||||
|
||||
CLI equivalents:
|
||||
|
||||
```sh
|
||||
# Interactive multi-selection
|
||||
pipelines export --output workflows.zip
|
||||
pipelines import workflows.zip
|
||||
|
||||
# Export every latest definition without a picker
|
||||
pipelines export --all --output workflows.zip
|
||||
|
||||
# Select exact names/IDs without a picker
|
||||
pipelines export "Research" "Review" --output workflows.zip
|
||||
|
||||
# Explicit automation: create everything from the archive
|
||||
pipelines --url ws://other-server:8080/connect import workflows.zip --all --yes --json
|
||||
|
||||
# Automation when differing names exist: choose a policy explicitly
|
||||
pipelines import workflows.zip --all --on-conflict keep --yes --json
|
||||
pipelines import workflows.zip --all --on-conflict both --yes --json
|
||||
pipelines import workflows.zip --all --on-conflict import --yes --json
|
||||
|
||||
# Original one-pipeline Markdown export remains available
|
||||
pipelines export "Research" --version 1 > research.md
|
||||
```
|
||||
|
||||
An archive contains a versioned `manifest.json` with original names, version
|
||||
numbers and SHA-256 checksums, plus one Markdown definition per pipeline. Latest
|
||||
versions are pinned when selection starts; `--version N` exports that exact
|
||||
version for each selected pipeline. No model calls are made.
|
||||
|
||||
Import checks the archive/checksums, destination names and server validation before
|
||||
publishing. New names/copies start at v1; choosing imported increments the existing
|
||||
pipeline's destination version. In JSON mode a differing name without an explicit
|
||||
`--on-conflict` policy fails before publication, rather than asking interactively.
|
||||
Exported version numbers are provenance, not destination version numbers.
|
||||
Confirmation warns that schedules in the definitions will activate. Model/tool
|
||||
availability and notification delivery still need runtime checks on the destination.
|
||||
There is no automatic execution of imported stages.
|
||||
|
||||
Publication uses individual guarded requests, not an atomic batch. New names use
|
||||
create-only guards; updates require the exact existing version used for comparison.
|
||||
Concurrent changes cause a conflict, never a forced overwrite. A network
|
||||
failure or concurrent name/version collision may leave some definitions published. Failures
|
||||
report the confirmed accepted names (also in JSON mode); an ambiguous response may
|
||||
mean the destination accepted a request, so inspect its list before retrying.
|
||||
Already-accepted definitions are retained, never rolled back. JSON reports
|
||||
`imported` (all accepted names), `updated` (new versions), and `skipped` (names with
|
||||
reasons `identical` or `kept-existing`).
|
||||
|
||||
This is a portable definition export, **not a complete database backup**: it does
|
||||
not include older versions, runs/diffs, audit history, retained drafts, repository
|
||||
files, provider credentials or server notification settings. Definitions may
|
||||
themselves contain sensitive prompts; archives are private local files, not
|
||||
encrypted. Limits: 1,000 definitions, 1 MiB per definition and 32 MiB total source.
|
||||
@@ -0,0 +1,80 @@
|
||||
# Reference validation — October 10, 2026
|
||||
|
||||
Execution: existing Docker installation, Linux `aarch64`, OpenCode **v2.0.20**,
|
||||
model **opencode-go/deepseek-v4.1-flash**. The test binaries were compiled from the
|
||||
current checkout and run against a separate SQLite database and isolated fixtures.
|
||||
No definitions were published to the normal server database and no phone alerts
|
||||
were sent. Notification checks used an isolated local HTTP endpoint.
|
||||
|
||||
| Reference workflow | Live result | Duration |
|
||||
|---|---|---|
|
||||
| Incident extraction → report | Passed: exact incident evidence/counts, handoff, session readback and local success receipt | 6.63s |
|
||||
| Edit → independently verify | Passed: exact file bytes, untouched fixtures, harness-run `cmp`, completed read/edit/shell evidence, retained stage diff | 14.60s |
|
||||
| Missing required input | Passed: deliberate blocked outcome terminates as failed; missing file not invented; read attempt and local failure receipt retained | 4.21s |
|
||||
| Accounting and one-token budget | Initially failed the acceptance test; passed after the accounting fix below | 2.43s on focused rerun |
|
||||
|
||||
## Defect found and fixed
|
||||
|
||||
The first accounting run succeeded despite a one-token limit. Its retained V2
|
||||
session had **no messages and zero usage counters**, even though `/generate`
|
||||
returned a valid result. Polling that session treated placeholder zeros as known
|
||||
accounting and did not enforce the intended limit.
|
||||
|
||||
Budgeted V2 text calls now use tracked `/prompt` → `/wait` → message readback,
|
||||
with **all tools denied**. Their messages are retained in the stage audit, usage
|
||||
is read from the tracked session, and a completed token-budgeted response with
|
||||
zero reported tokens fails closed. Native V1 handling and unbudgeted text
|
||||
generation keep their existing paths.
|
||||
|
||||
The focused live rerun retained **952 reported tokens and $0.00016440**, failed
|
||||
with `stage budget exceeded`, and stopped its owned session. This is reported
|
||||
usage, not an independently reconciled provider bill. A one-token limit can be
|
||||
overshot by a single in-flight request; this run demonstrates exhaustion handling,
|
||||
not an exact hard cap or proof of immediate mid-stream cancellation.
|
||||
|
||||
The three previously passing workflows were not charged again. Their copied
|
||||
artifacts were rechecked with current independent assertions; the fixed accounting
|
||||
artifacts were rechecked separately. There has **not** been a fresh full-suite live
|
||||
run of all four cases after the fix.
|
||||
|
||||
## Evidence and reproduction
|
||||
|
||||
Retained container directories for this trial:
|
||||
|
||||
- `/data/reference-validation/pipelines-reference-284076920`: first complete run.
|
||||
- `/data/reference-validation/pipelines-reference-2247403140`: fixed accounting rerun.
|
||||
|
||||
Each has isolated SQLite history, pinned definitions, run/session JSON, fixture
|
||||
files and `report.json`. Host copies are in the session's approved temporary
|
||||
directory under `reference-first/` and `reference-accounting-fixed/`.
|
||||
|
||||
Commands that performed live inference:
|
||||
|
||||
```sh
|
||||
docker exec -e PIPELINES_REFERENCE_LIVE=1 \
|
||||
-e PIPELINES_REFERENCE_MODEL=opencode-go/deepseek-v4.1-flash \
|
||||
-e PIPELINES_REFERENCE_DIR=/data/reference-validation \
|
||||
pipelines-pipelines-1 /data/reference-validation/reference-20261010.test \
|
||||
-test.run '^TestReferenceWorkflowsLive$' -test.v -test.timeout 15m
|
||||
|
||||
docker exec -e PIPELINES_REFERENCE_LIVE=1 \
|
||||
-e PIPELINES_REFERENCE_MODEL=opencode-go/deepseek-v4.1-flash \
|
||||
-e PIPELINES_REFERENCE_DIR=/data/reference-validation \
|
||||
pipelines-pipelines-1 /data/reference-validation/reference-20261010-fixed.test \
|
||||
-test.run '^TestReferenceWorkflowsLive/accounting-and-budget$' -test.v -test.timeout 5m
|
||||
```
|
||||
|
||||
`TestReferenceArtifactsReview` rechecks copied run JSON/fixtures without inference;
|
||||
`PIPELINES_REFERENCE_REVIEW_DIR` selects the copy and `-run` selects its cases.
|
||||
See [reference workflows](reference-workflows.md) for a fresh run and its opt-in,
|
||||
cost, retention and safety boundaries.
|
||||
|
||||
## Remaining unverified
|
||||
|
||||
- Other models/providers and native V1 schema/tool compatibility.
|
||||
- Real ntfy authentication/storage and actual phone reception.
|
||||
- Actual provider outages, timeouts, long-stream interruption and retry behavior.
|
||||
- General workflow correctness beyond these bounded fixtures.
|
||||
|
||||
The fix is in the source/test binaries, **not deployed to the installed Docker
|
||||
server**. Rebuild the installation when ready to use it for ordinary runs.
|
||||
@@ -0,0 +1,97 @@
|
||||
# Real-provider reference workflows
|
||||
|
||||
These are small acceptance workflows, not production templates or performance
|
||||
benchmarks. They use real OpenCode inference and independent checks; a model's
|
||||
claim of success is not enough. They never publish to the normal Pipelines database.
|
||||
The [initial live validation results](reference-workflow-results.md) record what
|
||||
passed, the accounting defect discovered, its focused rerun and remaining limits.
|
||||
|
||||
| Workflow | What is exercised | Independent acceptance evidence |
|
||||
|---|---|---|
|
||||
| Incident handoff | Extract two incidents, then produce a report in a new stage | Exact IDs/severities, counts, unchanged handoff, retained sessions, one local success alert |
|
||||
| Edit and verify | Read/edit a release record, then run a read-only comparison | Exact final file bytes, unchanged sentinel/reference files, harness-run `cmp`, retained read/edit/shell evidence, stage-local diff and no verifier edits |
|
||||
| Missing input | Inspect a deliberately missing required file and take its blocked route | Failed run, blocked outcome, missing file remains missing, retained read attempt, one local failure alert |
|
||||
| Accounting and budget | Generate a tiny answer with a one-token stage budget | Reported USD/tokens retained, budget-exceeded failure, one attempt and stopped session |
|
||||
|
||||
The suite permits six model stage attempts. Backend compatibility probes or format
|
||||
repairs may add calls. Retries are disabled; each stage has a two-minute timeout
|
||||
and the suite has a twelve-minute context deadline. **Provider charges may apply.**
|
||||
The accounting workflow also sets a $0.25 reported-usage budget. This is not a hard
|
||||
bill ceiling: an in-flight request can overshoot. Other workflows have no spending
|
||||
budget so missing accounting does not mask basic inference/tool compatibility.
|
||||
|
||||
Notification delivery uses an isolated local HTTP endpoint, not your configured
|
||||
topic or phone. This tests publisher requests and retained receipts, **not real
|
||||
ntfy authentication/storage or phone reception**. The failure case exercises a
|
||||
declared model `blocked` route, not a real provider outage. The tiny budget case
|
||||
checks exhaustion of reported usage; it is not proof that interruption promptly
|
||||
stops a long-running provider response or external shell process.
|
||||
|
||||
## Run with native credentials
|
||||
|
||||
Choose a model you are authorized to charge. Use a private, existing artifacts
|
||||
directory. Do not use normal `go test ./...` to request live inference; opt-in is
|
||||
explicit:
|
||||
|
||||
```sh
|
||||
mkdir -p "$HOME/Library/Caches/pipelines/reference-validation"
|
||||
PIPELINES_REFERENCE_LIVE=1 \
|
||||
PIPELINES_REFERENCE_MODEL=opencode-go/deepseek-v4.1-flash \
|
||||
PIPELINES_REFERENCE_DIR="$HOME/Library/Caches/pipelines/reference-validation" \
|
||||
go test ./internal/server -run '^TestReferenceWorkflowsLive$' -count=1 -v -timeout 15m
|
||||
```
|
||||
|
||||
`PIPELINES_OPENCODE` can select an executable. Without the live opt-in, the live
|
||||
test skips; `TestReferenceDefinitions` validates all four definitions without
|
||||
inference. These examples use the configured model selected for the initial
|
||||
Docker trial; other providers/models need their own acceptance run.
|
||||
|
||||
## Run current source inside the existing Docker installation
|
||||
|
||||
The test binary contains the current server source. Copying it into a private
|
||||
test directory avoids replacing/restarting the installed server or touching its
|
||||
database. Choose `GOARCH=arm64` for container `uname -m` equal to `aarch64`, or
|
||||
`GOARCH=amd64` for `x86_64`.
|
||||
|
||||
```sh
|
||||
docker exec pipelines-pipelines-1 uname -m
|
||||
GOOS=linux GOARCH=arm64 go test -c -o reference.test ./internal/server
|
||||
docker exec pipelines-pipelines-1 mkdir -p /data/reference-validation
|
||||
docker exec pipelines-pipelines-1 chmod 700 /data/reference-validation
|
||||
docker cp reference.test pipelines-pipelines-1:/data/reference-validation/reference.test
|
||||
docker exec \
|
||||
-e PIPELINES_REFERENCE_LIVE=1 \
|
||||
-e PIPELINES_REFERENCE_MODEL=opencode-go/deepseek-v4.1-flash \
|
||||
-e PIPELINES_REFERENCE_DIR=/data/reference-validation \
|
||||
pipelines-pipelines-1 /data/reference-validation/reference.test \
|
||||
-test.run '^TestReferenceWorkflowsLive$' -test.v -test.timeout 15m
|
||||
```
|
||||
|
||||
This uses the container's existing provider credentials. It does not copy host
|
||||
credentials, log into a provider, rebuild the installation image or publish any
|
||||
definitions to `/data/pipelines.db`.
|
||||
|
||||
## Retained evidence
|
||||
|
||||
The suite prints its uniquely created `pipelines-reference-*` directory. It retains:
|
||||
|
||||
- `runs.db`: isolated definitions, immutable runs/visits, diffs and receipts.
|
||||
- One pinned Markdown definition and run/session JSON per workflow.
|
||||
- Per-workflow fixture directories, including final edited files.
|
||||
- `report.json`: model, verification verdicts, run IDs, durations and diagnostics.
|
||||
|
||||
Keep the test console log alongside these artifacts. A failed assertion does not
|
||||
delete evidence; OpenCode sessions are also retained. A verified result applies
|
||||
to that model/backend/environment and suite revision, not every provider or task.
|
||||
Suite artifacts contain fixture prompts/results and should stay private. They
|
||||
are not a reconciled provider bill. No result is labeled verified until its
|
||||
independent assertions and session readback checks pass.
|
||||
|
||||
Unbudgeted tool-free V2 generation may leave an empty OpenCode message history:
|
||||
the pipeline's definition, input/output and command audit are still retained.
|
||||
Budgeted V2 calls use tracked, deny-all prompting so usage and messages are retained.
|
||||
|
||||
To recheck copied artifacts without inference, set
|
||||
`PIPELINES_REFERENCE_REVIEW_DIR` and run `TestReferenceArtifactsReview`, selecting
|
||||
the relevant case names with `-run` when combining artifacts from separate trials.
|
||||
Missing cases explicitly skip; they are never counted as verified.
|
||||
@@ -0,0 +1,43 @@
|
||||
# Retained stage file changes
|
||||
|
||||
Each newly executed stage visit compares a read-only snapshot of its run directory
|
||||
immediately before execution with a snapshot after execution/cleanup. The server
|
||||
stores the observed changes and unified patches in SQLite alongside the finished
|
||||
step, atomically. Later stages, edits and server restarts do not replace that
|
||||
evidence. Existing historical steps have no retroactive capture.
|
||||
|
||||
On **Run → Details**, select a recorded visit with `n`/`p`. Its **File changes**
|
||||
section is expanded by default: added lines are green, removed lines red, and hunk
|
||||
headers accented. `d` focuses/toggles the diff; ordinary scrolling handles long
|
||||
patches. Tab still switches only Details ↔ Graph. `pipelines inspect RUN_ID --json`
|
||||
includes the same evidence under each step's `file_changes` field.
|
||||
|
||||
## Scope and limitations
|
||||
|
||||
- This compares stage-start contents, not HEAD or the index. Pre-existing dirty
|
||||
content is the baseline. Changes are retained separately for repeated visits.
|
||||
- In a Git working tree, capture includes tracked and nonignored untracked files
|
||||
within the run directory. Previously captured files remain eligible after an
|
||||
index/ignore change. Git is used only for inventory, never to stage or revert.
|
||||
Outside Git, capture walks regular files in the run directory.
|
||||
- Added/deleted files and permission changes are recorded. Renames appear as
|
||||
deletion/addition. Binary changes retain metadata, not binary contents.
|
||||
- Symlinks, special files, common credential/environment/key/database paths,
|
||||
`secrets/`, Git internals and `node_modules/` are excluded. This is a path-based
|
||||
precaution, **not secret-content scanning**; ordinary source patches may contain
|
||||
sensitive text and become part of retained history.
|
||||
- Limits: 10,000 inventoried entries, 1 MiB per file, 32 MiB per snapshot,
|
||||
five seconds per snapshot, 5,000 combined lines per textual comparison,
|
||||
64 KiB per patch and 128 KiB serialized file evidence per visit. Oversized,
|
||||
unreadable, omitted or truncated evidence is explicitly flagged partial.
|
||||
- Failed/canceled stages still retain changes when execution returns. Unconfirmed
|
||||
agent cleanup is flagged partial. A server/process crash cannot guarantee an
|
||||
after-snapshot; recovered steps without capture say “not recorded.”
|
||||
- Capture is not a transactional filesystem snapshot: concurrent manual edits,
|
||||
other processes or other runs may appear in the diff. Changes outside the run
|
||||
directory, ignored new files, and edits reverted before capture are not observed.
|
||||
The view describes observed changes, not proof of exclusive model authorship.
|
||||
- Capture failure does not change the pipeline's declared outcome or permissions.
|
||||
|
||||
Snapshots are transient server memory. The database retains patches/metadata,
|
||||
not full copies of every workspace file. No new model calls are used for capture.
|
||||
+99
-33
@@ -1,44 +1,110 @@
|
||||
# UI/UX review: October 8, 2026
|
||||
# UI/UX review
|
||||
|
||||
Five independent read-only reviewer sessions inspected executed `Model.View`
|
||||
renders of Home, pipeline details, selected/focused/scrolled stages, native
|
||||
delivery, and run inspection at 80/120/160 columns. Fixes were applied between
|
||||
rounds. Snapshot review does not establish color accessibility or phone delivery.
|
||||
An independent reviewer rendered every screen family at 120×36, 80×24 and
|
||||
40×16, alongside light/dark ANSI previews. This was an executed screen/render
|
||||
review, not a real-user usability or screen-reader study. Real-terminal contrast
|
||||
and external editor/OpenCode behavior were not assessed by that review.
|
||||
|
||||
| Round | Findings and changes |
|
||||
## Design decisions
|
||||
|
||||
- **Results before machinery.** Home starts with recent outcomes and reported
|
||||
results. Statistics come afterward; addresses and notification setup live in Status.
|
||||
- **Two Run focus targets.** Tab and Shift+Tab alternate Details ↔ Graph. Recorded
|
||||
visits are a compact selector inside Details, not a third focusable pane.
|
||||
- **Readable reports.** Results use prose, Markdown, lists and labelled nested
|
||||
values instead of raw transport objects. The stored output is unchanged.
|
||||
- **Progressive disclosure.** Progress, task input, evidence and execution metadata
|
||||
are secondary. A stage's reported result is not independently verified success.
|
||||
- **Full-size graphs.** Keep the existing boxes/arrows, centering, dim pending
|
||||
routes and panning. Collapse an unusably small inactive run-graph preview rather
|
||||
than drawing severed box tops. A focused graph remains reachable on narrow screens.
|
||||
- **Contextual controls.** Forms and overlays replace background-page hints.
|
||||
Safety warnings wrap and scroll while confirmation/cancellation remain pinned.
|
||||
|
||||
## Feedback by screen
|
||||
|
||||
| Screen | Reviewer feedback | Response / remaining work |
|
||||
| --- | --- | --- |
|
||||
| Home | Usage and infrastructure outrank results; narrow columns clip reports. | Results first, wider result column, wrapped previews, statistics last, continuation indicator. |
|
||||
| Home empty/loading | Missing statistics look like permanent loading. | No placeholder usage spinner; empty and initial data-loading states are distinct. |
|
||||
| Pipeline overview | Graph is good; identity and metadata compete with useful content. | Keep the graph. Further compacting the toolbar on small screens remains worthwhile. |
|
||||
| Pipeline stage | Five prose columns resemble a spreadsheet. | Full-width stacked fields; capabilities before long instructions; configuration disclosure. |
|
||||
| Runs list | Footer advertises stage/graph controls that do not apply. | List-specific search/select/open/back controls. |
|
||||
| Run overview | Stage sidebar takes too much space and adds a third focus target. | Remove sidebar focus and resizing; compact visit selector and full-width Details. |
|
||||
| Run stage/visit | Summary repeats output; long evidence crowds the result. | Render the result once; collapse secondary sections; preserve each recorded visit. |
|
||||
| Run graph | Inactive previews sever boxes; narrow focused Graph can disappear. | Collapse unusable previews; allocate the viewport to the focused full-size canvas. |
|
||||
| Templates | Definition source dominates purpose and expected result. | Error/loading states corrected. A purpose-first catalog overview remains a follow-up. |
|
||||
| Drafts | Paths and publication metadata dominate the draft's purpose. | Preserve current draft retention/validation. Moving metadata behind disclosure remains a follow-up. |
|
||||
| Status | Narrow cards hide addresses; scrolling does not reveal them. | Wrap card contents and scroll the actual service-card canvas. |
|
||||
| Task form | Tool lists consume prompt space; wrong footer. | Larger task area, compact scope summary, correct submit/cancel controls. |
|
||||
| CLI pipeline picker | Background Home controls remain visible. | Picker-specific filter/select/cancel controls. |
|
||||
| Search | “Omnisearch” promises broader scope than pipeline search. | “Search pipelines” and specific open/cancel instructions. |
|
||||
| Commands | No descriptions or indication of hidden entries. | Selected-command explanation and visible range/count. |
|
||||
| Versions | Selected/latest markers truncate on narrow screens. | Correct view-version footer; further metadata prioritization remains a follow-up. |
|
||||
| Naming/publication | Naming inherits publication shortcuts. | Input-specific name-saving/retention footer. |
|
||||
| Start/retry/cancel | Narrow safety warnings are clipped. | Wrapped, independently scrollable disclosures with pinned controls. |
|
||||
| Editor picker/checking | Unlabelled filter and color-only selection; wrong pending footer. | Labelled filter, textual selection marker, appropriate pending-state footer. |
|
||||
| Logs | Long failure messages are horizontally inaccessible. | Left/right panning; vertical unread tracking is retained. |
|
||||
| Failure/debug | Generic mutation warning does not explain Debug. | Explain pinned definition/evidence, retained draft scope and separation from retry. |
|
||||
|
||||
The second executed review confirmed the improved Home and 80-column run report
|
||||
hierarchy. It identified four remaining narrow-layout issues, addressed afterward:
|
||||
short-screen Details now uses compact headers to expose the actual result;
|
||||
pipeline stage focus collapses its inactive graph preview; draft-delete uses the
|
||||
same wrapped/scrollable safety dialog; and narrow overlay footers keep Escape
|
||||
instructions visible. Dedicated regressions exercise each of these behaviors.
|
||||
|
||||
## Navigation
|
||||
|
||||
Run opens in Details. `n`/`p` selects recorded visits from either pane, including
|
||||
repeated stages; `0` returns to the overview. `j`/`k` scrolls Details or pans Graph,
|
||||
depending on focus. `h`/`l` and arrow keys pan a focused graph; in Details,
|
||||
left/right selects the previous/next visit. `c` centers the focused graph.
|
||||
`o`, `s`, `i`, `t`, `m` disclose result, progress, input, evidence and execution details.
|
||||
|
||||
Home still advertises only `/` search and `:` commands. `q` never exits the client;
|
||||
use `:quit`. Consequential actions still require explicit confirmation.
|
||||
|
||||
## Regression coverage
|
||||
|
||||
`internal/tui/ux_regression_test.go` covers the two-target focus cycle at all three
|
||||
sizes, visible narrow graph focus, readable nested reports without altering data,
|
||||
reachable safety warnings, contextual footers, terminal errors and narrow Status
|
||||
scrolling. Existing graph, repeated-visit, permission, draft and confirmation tests
|
||||
remain applicable. Compiled CLI/TUI PTY integration additionally exercises the
|
||||
actual two-target Tab cycle and editor/publication/debug/retry flows with mocked
|
||||
OpenCode; it does not verify real-provider inference or terminal-specific contrast.
|
||||
|
||||
## Earlier review record: October 8, 2026
|
||||
|
||||
The retained review below predates this redesign. Its findings and verification
|
||||
are historical, not a claim that the current screens have no remaining UX issues.
|
||||
|
||||
> Five independent read-only reviewer sessions inspected executed `Model.View`
|
||||
> renders of Home, pipeline details, selected/focused/scrolled stages, native
|
||||
> delivery, and run inspection at 80/120/160 columns. Fixes were applied between
|
||||
> rounds. Snapshot review does not establish color accessibility or phone delivery.
|
||||
|
||||
| Earlier round | Findings and changes |
|
||||
| --- | --- |
|
||||
| 1 | Added stage row ranges, horizontal graph overflow arrows/pan hint, and full-height column separators. |
|
||||
| 2 | Balanced graph/detail space; expanded focused details; added textual focus/selection and complete selected-run headings; made title truncation explicit. |
|
||||
| 3 | Exposed tools/notify in narrow configuration columns; preserved run statuses separately from shortened names; corrected run page size. |
|
||||
| 4 | Made pinned field headings follow the actual scrolled group; exposed page-scroll keys; preserved HTTP/HTTPS in compact endpoint text. |
|
||||
| 5 | No material remaining usability findings in the reviewed renders. |
|
||||
| 5 | No material remaining usability findings in those reviewed renders. |
|
||||
|
||||
Requested changes: `n`/`p` select next/previous stages on pipeline details,
|
||||
wrap at either end, reset detail scroll, and reveal the selected graph stage.
|
||||
Home `n` still creates a pipeline; text-entry/modal pages retain their own keys.
|
||||
The global ntfy server URL/topic now appear in the top bar beside the server
|
||||
address, instead of a home-page panel. Extremely narrow terminals truncate text
|
||||
with an ellipsis rather than overflowing.
|
||||
That iteration added wrapping `n`/`p` pipeline-stage selection and placed the ntfy
|
||||
URL/topic in the global header. This redesign supersedes that header placement
|
||||
and the multi-column stage layout; it preserves pipeline-stage selection.
|
||||
|
||||
Verification executed after changes:
|
||||
|
||||
- `go test -race -count=1 ./...`
|
||||
- `go vet ./...`
|
||||
- Go server and TUI builds.
|
||||
- Actual TUI through a pseudo-terminal: connected to the local server, opened
|
||||
the published anime pipeline, verified top-bar ntfy URL/topic, `n`/`p`
|
||||
next/previous/wrap, Tab focus, Esc home, and Home `n` opening the draft form.
|
||||
The earlier record reports race tests, vet, builds, and actual PTY inspection of
|
||||
the historical **anime episodes today v1** pipeline (pipeline **8**, no schedule).
|
||||
Its complete model/web/phone execution was **not tested**. It used dated web
|
||||
evidence → verify/enrich → Markdown report → native ntfy attachment through the
|
||||
system connection; the LiveChart schedule was fetched during that investigation.
|
||||
Removing its bundled starter does not remove its published history. Current
|
||||
snapshot fixtures use the generic `examples/three-stage.md` starter.
|
||||
|
||||
`PIPELINES_UX_SNAPSHOT_DIR=<directory> go test ./internal/tui -run
|
||||
TestUIUXReviewSnapshots -count=1` regenerates rendered snapshots without server
|
||||
mutations. Temporary snapshot/reviewer artifacts are under the session's
|
||||
approved temporary directory, not committed runtime data.
|
||||
|
||||
The historical anime pipeline (no longer bundled as a template) used dated web evidence →
|
||||
verify/enrich → Markdown report → native ntfy attachment, using the system
|
||||
connection. It is submitted as **anime episodes today v1**, pipeline **8**, with
|
||||
no schedule. Its full model/web/phone execution has **not been tested**. The
|
||||
LiveChart schedule was fetched during source investigation; the final delivery
|
||||
stage uses the existing native publisher, not credentials handled by a model.
|
||||
Current snapshot fixtures use the generic `examples/three-stage.md` starter;
|
||||
removing a bundled definition does not remove published history.
|
||||
TestUIUXReviewSnapshots -count=1` regenerates the existing snapshots without server
|
||||
mutations. Review artifacts remain in the session's approved temporary directory.
|
||||
|
||||
Reference in New Issue
Block a user