11 KiB
Chat: Pipelines tools (show file / image / HTML, notify phone)
A chat's model can call four tools that act on Pipelines, not on the harness
or the workspace. The tools come from an MCP server that runs inside
pipelines-server. Each call adds an entry to the chat transcript, and the
entry is persisted.
| Tool | Arguments | Effect |
|---|---|---|
show_file |
path, title? |
Shows an existing file inside the operator's home. The path can be absolute, ~/…, or relative to the chat's directory. Images render inline; other files render as a preview card. |
show_image |
url_or_path, title?, alt? |
Shows a home image file (it must be image/*) or an https:// URL. |
show_html |
html, title?, height_hint? |
Renders model-authored HTML or SVG inline in a sandbox, with an Open button for full screen. |
notify |
title, message, priority? |
Sends a phone notification through the existing ntfy "system" destination. Tapping it opens pipelines://chat/<id>. |
Each tool returns one short text line to the model. A refusal comes back as
isError together with the reason.
Security model
- Endpoint. The endpoint is
POST /mcp/chat/<token>on the server's own listener. It speaks JSON-RPC 2.0 over MCP streamable HTTP:initialize,ping,tools/list,tools/call, and notifications (answered202).GETand other methods get405.- The protocol version is negotiated: 2025-11-25, 2025-06-18, 2025-03-26 or 2024-11-05.
- Batches are refused, and request bodies are capped at 4 MiB.
- Requests that carry an
Originheader (from a browser) get403, as do requests from non-loopback peers. Unknown tokens get404.
- Not exposed through the web.
web/serve.mjsand the peer relay forward only an allowlist (/connect,/files,/artifacts/,/chat-uploads,/updates/). Both new tests (web/test/mcpRefused.test.mjsand the peerRelay test) confirm/mcp…never reaches the server, including for a signed-in browser and%6dcp-encoded paths. - Tokens. Each token is 256 random bits, stored in
chat_tool_tokenswith the scope it serves:session:<id>: one chat. Claude Code and Codex use this, and the token stays stable across restarts so detached hosts keep working.pending:: a Codex thread before its id is known. Calls are refused until the token is bound.instance:opencode: the shared OpenCode service. See "OpenCode attribution" below.
- Paths. Paths use the same containment as
/files:expandHomePath, thenresolveInsideHome(resolves every symlink), then "regular file only", then thespecialFilesystemcheck. The item records the real path, and the client loads it from the chat's own machine's/files(or/peers/<id>/filesfor a peer). - Image URLs. Only
httpsURLs without credentials, at most 2048 chars. The client fetches them with a plain network image, without cookies. - HTML views. These use the same inert view as artifact HTML previews:
- On web, an iframe with
sandbox="allow-popups allow-popups-to-escape-sandbox"andsrcdoc. There are no scripts, no same-origin access, no forms and no top navigation. - Links are rewritten first:
#anchorsscroll inside the frame, http(s)/mailto links open a new tab, and every other link is made inert. - On Android/iOS, a WebView with JavaScript disabled that refuses navigation.
- Interactivity is therefore limited to CSS (
:hover,<details>) and SVG. The tool description tells the model this.
- On web, an iframe with
Limits
show_htmldocuments are refused, not truncated, above 512 KiB. The model is told the size and the limit.- At most 100 items are retained per chat. HTML bodies are kept only for the
newest documents within 2 MiB per chat; older documents keep their entry
but are marked
expired, and the card says "This view is no longer retained". notifyallows 5 per minute per chat (a sliding window, held in memory). Rate-limited calls record nothing.- An undelivered notification is recorded with its reason, for example ntfy not configured or switched off.
priorityis recorded, but ntfy always publishes at the default priority, becausestage.PublishNtfyhas no priority parameter.
Protocol and persistence
Items are stored in chat_shown_items (pipeline store). There are two ways
they reach clients:
- The
chat_subscribereply carrieschat_shown: {session_id, items}with every retained item, oldest first. - A pushed
chat_shownframe carries one new item.
The client interleaves items into the transcript by created_at. Items older
than the loaded history wait until that page loads. In a multi-machine chat,
the session id is rewritten by ChatServerGateway, and file URLs resolve
against the chat's own machine endpoint.
The generic tool rows for mcp__pipelines__* (Claude), pipelines_*
(OpenCode tool parts) and pipelines/* (Codex) are hidden; failed calls stay
visible. OpenCode V2 runs MCP tools from its code-mode execute script, and
that row stays visible because it can contain other code.
Per-harness registration
server.ConfigureChatTools(store, addr) runs at startup, before any harness
starts, and installs chattools.Registry. The registry's base URL is the
loopback address of -addr; a wildcard address maps to 127.0.0.1, and a
non-loopback-only bind disables the tools with a log line.
- Claude Code. Started with
--mcp-config '{"mcpServers":{"pipelines":{"type":"http","url":…}}}'and--allowedTools mcp__pipelines. Both direct and hosted chats get the session token.- The config is added to the operator's own MCP servers; it does not replace them.
- If a start fails with these flags, the chat logs the reason and retries
once without them (
session.toolsOff).
- Codex.
thread/startgetsconfig.mcp_servers.pipelines.urlwith a pending token, which is then bound to the returned thread id.- A
thread/resume(after "thread not found") registers the thread's session token. - If the App Server refuses the override, the thread starts or resumes without it, and this is logged.
- Threads already loaded by the Codex desktop app before Pipelines touched them have no tools.
- A
- OpenCode. The spawned chat service gets
mcp.pipelines = {type: "remote", url, enabled: true}throughOPENCODE_CONFIG_CONTENT. This covers bothOpenChatand the owned detachedEnsureOwnedChatService.- The operator's chat overrides win.
- If the service does not start with the tools, it starts without them, and this is logged.
- An operator-supplied endpoint (
-opencode-endpoint) is not reconfigured, so it has no tools.
- Pi. There is no Pi chat harness in this repository (chat backends are opencode, claude and codex), so it is not supported.
OpenCode attribution. One OpenCode service serves every chat, and its MCP
requests carry no session (verified: OpenCode 2.0.32 sends only
User-Agent: opencode/latest/2.0.32/cli and Mcp-Protocol-Version).
A call on the instance token waits up to 3 s for exactly one chat whose retained stream shows that call running. That means one of:
- a
pipelines_<tool>part with identical arguments, or - a running code-mode
executescript containingpipelines.<tool>(. If several such scripts are running, they are told apart by the argument values.
When no chat matches, or several still do, the call is refused. It is never guessed into a chat. Only chats Pipelines is observing can use the tools.
Verification
- Go:
go test ./internal/server/ ./internal/chattools/ ./internal/pipeline/ ./internal/protocol/ ./internal/codex/ ./internal/codexhost/ ./internal/claudehost/: allok(server 72 s).go test ./internal/claude/ ./internal/opencode/:okunderumask 022. Under this sandbox'sumask 077, the existing testsTestServiceTokenIsPassedOnlyFromAPrivateFileandTestStageDiscoveryRequiresPrivateRegistrationBeforeSessionWorkfail, because both depend on 0644 file creation. They are unrelated to this change.
- New Go tests:
chattools/mcp_test.go: handshake, list, call, refusals, harness config.server/chat_tools_test.go: token isolation, pending binding, path containment, image URLs, HTML limit and retention, notify rate limit, instance and code-mode attribution.server/chat_tools_http_test.go: endpoint on the real listener,loopbackBase.claude|codex|opencode/pipelines_tools_test.go: registration config and the refuse-then-start-without fallback.
cd web && node --test test/:ℹ tests 18 … ℹ pass 18 ℹ fail 0.- Flutter:
tool/memwatch/full.sh test/features/chat:memwatch: PASS: 1080 passed, 0 failed, 0 skipped; peak 2204 MB; 89s; exit 0 (test/features/chat).tool/analyze.sh:0 errors, 8 warnings. All 8 are the existinginvalid_null_aware_operatorwarnings in the unchangedintegration_test/discovery_probe_test.dart.
- Real end-to-end check, run on 2026-11-09:
- Setup: a scratch
pipelines-serveron127.0.0.1:53834, withHOME=/tmp/pl-e2e-tools/home, a scratch database,PIPELINES_SHUTDOWN_SESSIONS=stopand ntfy unset. - OpenCode 2.0.32 (its default free model) was asked to
show_file ~/scratch/checker.pngand toshow_htmlan SVG circle. Itsexecutescript returned "Shown in the operator's Pipelines chat: checker.png (217 bytes)" and "Shown … (137 bytes, sandboxed…)". chat_subscribereturned both items. The file item hadpath=/private/tmp/pl-e2e-tools/home/scratch/checker.png,mime=image/pngandsize=217./filesserved that path with200 image/png 217.notifywas recorded as not delivered: "phone notifications (ntfy) are not configured on this Pipelines server".- A POST carrying an
Originheader got403. - The first OpenCode attempt was refused, and this is how code-mode attribution was found to be necessary.
- Setup: a scratch
- Not run end to end:
- Claude Code under the isolated HOME answered "Not logged in", and
~/.config/pipelines/claude-tokenis empty. - Codex has no app-server under the isolated HOME.
- Both registrations are covered only by unit tests against fake CLI arguments and a fake App Server.
- Claude Code under the isolated HOME answered "Not logged in", and
- Screenshots, real Chrome via
tool/captureon the full-app fixture with?chats=1×tamps=1&shown=1&files=1, all innotes/images/chat-pipelines-tools/:cards-light-880.png: image, markdown card with first lines, HTML diagram with Open, and notify row.cards-dark-880.pngcards-cupertino-560.pnghtml-open-light-880.png: full-screen HTML with zoom.image-open-dark-880.png: the existing viewer with download.
- Mobile (Android/iOS WebView, native thumbnails) is unverified: no device run.