# Chat: Pipelines tools (show file / image / HTML, notify phone)
A chat's model can call four tools that act on Pipelines, not on the harness
or the workspace. The tools come from an MCP server that runs inside
`pipelines-server`. Each call adds an entry to the chat transcript, and the
entry is persisted.
| Tool | Arguments | Effect |
| --- | --- | --- |
| `show_file` | `path`, `title?` | Shows an existing file inside the operator's home. The path can be absolute, `~/…`, or relative to the chat's directory. Images render inline; other files render as a preview card. |
| `show_image` | `url_or_path`, `title?`, `alt?` | Shows a home image file (it must be `image/*`) or an `https://` URL. |
| `show_html` | `html`, `title?`, `height_hint?` | Renders model-authored HTML or SVG inline in a sandbox, with an **Open** button for full screen. |
| `notify` | `title`, `message`, `priority?` | Sends a phone notification through the existing ntfy "system" destination. Tapping it opens `pipelines://chat/`. |
Each tool returns one short text line to the model. A refusal comes back as
`isError` together with the reason.
## Security model
- **Endpoint.** The endpoint is `POST /mcp/chat/` on the server's own
listener. It speaks JSON-RPC 2.0 over MCP streamable HTTP: `initialize`,
`ping`, `tools/list`, `tools/call`, and notifications (answered `202`).
- `GET` and other methods get `405`.
- The protocol version is negotiated: 2025-11-25, 2025-06-18, 2025-03-26 or
2024-11-05.
- Batches are refused, and request bodies are capped at 4 MiB.
- Requests that carry an `Origin` header (from a browser) get `403`, as do
requests from non-loopback peers. Unknown tokens get `404`.
- **Not exposed through the web.** `web/serve.mjs` and the peer relay forward
only an allowlist (`/connect`, `/files`, `/artifacts/`, `/chat-uploads`,
`/updates/`). Both new tests (`web/test/mcpRefused.test.mjs` and the
peerRelay test) confirm `/mcp…` never reaches the server, including for a
signed-in browser and `%6dcp`-encoded paths.
- **Tokens.** Each token is 256 random bits, stored in `chat_tool_tokens` with
the scope it serves:
- `session:`: one chat. Claude Code and Codex use this, and the token
stays stable across restarts so detached hosts keep working.
- `pending:`: a Codex thread before its id is known. Calls are refused until
the token is bound.
- `instance:opencode`: the shared OpenCode service. See "OpenCode attribution"
below.
- **Paths.** Paths use the same containment as `/files`:
`expandHomePath`, then `resolveInsideHome` (resolves every symlink), then
"regular file only", then the `specialFilesystem` check. The item records
the real path, and the client loads it from the chat's own machine's
`/files` (or `/peers//files` for a peer).
- **Image URLs.** Only `https` URLs without credentials, at most 2048 chars.
The client fetches them with a plain network image, without cookies.
- **HTML views.** These use the same inert view as artifact HTML previews:
- On web, an iframe with `sandbox="allow-popups allow-popups-to-escape-sandbox"`
and `srcdoc`. There are no scripts, no same-origin access, no forms and no
top navigation.
- Links are rewritten first: `#anchors` scroll inside the frame,
http(s)/mailto links open a new tab, and every other link is made inert.
- On Android/iOS, a WebView with JavaScript disabled that refuses navigation.
- Interactivity is therefore limited to CSS (`:hover`, ``) and SVG.
The tool description tells the model this.
## Limits
- `show_html` documents are refused, not truncated, above **512 KiB**. The
model is told the size and the limit.
- At most **100 items are retained per chat**. HTML bodies are kept only for the
newest documents within **2 MiB per chat**; older documents keep their entry
but are marked `expired`, and the card says "This view is no longer retained".
- `notify` allows **5 per minute per chat** (a sliding window, held in memory).
Rate-limited calls record nothing.
- An undelivered notification is recorded with its reason, for example ntfy
not configured or switched off.
- `priority` is recorded, but ntfy always publishes at the default priority,
because `stage.PublishNtfy` has no priority parameter.
## Protocol and persistence
Items are stored in `chat_shown_items` (pipeline store). There are two ways
they reach clients:
- The `chat_subscribe` reply carries `chat_shown: {session_id, items}` with
every retained item, oldest first.
- A pushed `chat_shown` frame carries one new item.
The client interleaves items into the transcript by `created_at`. Items older
than the loaded history wait until that page loads. In a multi-machine chat,
the session id is rewritten by `ChatServerGateway`, and file URLs resolve
against the chat's own machine endpoint.
The generic tool rows for `mcp__pipelines__*` (Claude), `pipelines_*`
(OpenCode tool parts) and `pipelines/*` (Codex) are hidden; failed calls stay
visible. OpenCode V2 runs MCP tools from its code-mode `execute` script, and
that row stays visible because it can contain other code.
## Per-harness registration
`server.ConfigureChatTools(store, addr)` runs at startup, before any harness
starts, and installs `chattools.Registry`. The registry's base URL is the
loopback address of `-addr`; a wildcard address maps to 127.0.0.1, and a
non-loopback-only bind disables the tools with a log line.
- **Claude Code.** Started with `--mcp-config '{"mcpServers":{"pipelines":{"type":"http","url":…}}}'`
and `--allowedTools mcp__pipelines`. Both direct and hosted chats get the
session token.
- The config is added to the operator's own MCP servers; it does not replace
them.
- If a start fails with these flags, the chat logs the reason and retries
once without them (`session.toolsOff`).
- **Codex.** `thread/start` gets `config.mcp_servers.pipelines.url` with a
pending token, which is then bound to the returned thread id.
- A `thread/resume` (after "thread not found") registers the thread's
session token.
- If the App Server refuses the override, the thread starts or resumes
without it, and this is logged.
- Threads already loaded by the Codex desktop app before Pipelines touched
them have no tools.
- **OpenCode.** The spawned chat service gets
`mcp.pipelines = {type: "remote", url, enabled: true}` through
`OPENCODE_CONFIG_CONTENT`. This covers both `OpenChat` and the owned detached
`EnsureOwnedChatService`.
- The operator's chat overrides win.
- If the service does not start with the tools, it starts without them, and
this is logged.
- An operator-supplied endpoint (`-opencode-endpoint`) is not reconfigured,
so it has no tools.
- **Pi.** There is no Pi chat harness in this repository (chat backends are
opencode, claude and codex), so it is not supported.
**OpenCode attribution.** One OpenCode service serves every chat, and its MCP
requests carry no session (verified: OpenCode 2.0.32 sends only
`User-Agent: opencode/latest/2.0.32/cli` and `Mcp-Protocol-Version`).
A call on the instance token waits up to 3 s for exactly one chat whose
retained stream shows that call running. That means one of:
- a `pipelines_` part with identical arguments, or
- a running code-mode `execute` script containing `pipelines.(`. If
several such scripts are running, they are told apart by the argument
values.
When no chat matches, or several still do, the call is refused. It is never
guessed into a chat. Only chats Pipelines is observing can use the tools.
## Verification
- Go:
- `go test ./internal/server/ ./internal/chattools/ ./internal/pipeline/ ./internal/protocol/ ./internal/codex/ ./internal/codexhost/ ./internal/claudehost/`: all `ok` (server 72 s).
- `go test ./internal/claude/ ./internal/opencode/`: `ok` under `umask 022`.
Under this sandbox's `umask 077`, the existing tests
`TestServiceTokenIsPassedOnlyFromAPrivateFile` and
`TestStageDiscoveryRequiresPrivateRegistrationBeforeSessionWork` fail,
because both depend on 0644 file creation. They are unrelated to this
change.
- New Go tests:
- `chattools/mcp_test.go`: handshake, list, call, refusals, harness config.
- `server/chat_tools_test.go`: token isolation, pending binding, path
containment, image URLs, HTML limit and retention, notify rate limit,
instance and code-mode attribution.
- `server/chat_tools_http_test.go`: endpoint on the real listener,
`loopbackBase`.
- `claude|codex|opencode/pipelines_tools_test.go`: registration config and
the refuse-then-start-without fallback.
- `cd web && node --test test/`: `ℹ tests 18 … ℹ pass 18 ℹ fail 0`.
- Flutter:
- `tool/memwatch/full.sh test/features/chat`:
`memwatch: PASS: 1080 passed, 0 failed, 0 skipped; peak 2204 MB; 89s; exit 0 (test/features/chat)`.
- `tool/analyze.sh`: `0 errors, 8 warnings`. All 8 are the existing
`invalid_null_aware_operator` warnings in the unchanged
`integration_test/discovery_probe_test.dart`.
- Real end-to-end check, run on 2026-11-09:
- Setup: a scratch `pipelines-server` on `127.0.0.1:53834`, with
`HOME=/tmp/pl-e2e-tools/home`, a scratch database,
`PIPELINES_SHUTDOWN_SESSIONS=stop` and ntfy unset.
- OpenCode 2.0.32 (its default free model) was asked to `show_file
~/scratch/checker.png` and to `show_html` an SVG circle. Its `execute`
script returned "Shown in the operator's Pipelines chat: checker.png (217
bytes)" and "Shown … (137 bytes, sandboxed…)".
- `chat_subscribe` returned both items. The file item had
`path=/private/tmp/pl-e2e-tools/home/scratch/checker.png`,
`mime=image/png` and `size=217`. `/files` served that path with
`200 image/png 217`.
- `notify` was recorded as not delivered: "phone notifications (ntfy) are
not configured on this Pipelines server".
- A POST carrying an `Origin` header got `403`.
- The first OpenCode attempt was refused, and this is how code-mode
attribution was found to be necessary.
- Not run end to end:
- Claude Code under the isolated HOME answered "Not logged in", and
`~/.config/pipelines/claude-token` is empty.
- Codex has no app-server under the isolated HOME.
- Both registrations are covered only by unit tests against fake CLI
arguments and a fake App Server.
- Screenshots, real Chrome via `tool/capture` on the full-app fixture with
`?chats=1×tamps=1&shown=1&files=1`, all in
`notes/images/chat-pipelines-tools/`:
- `cards-light-880.png`: image, markdown card with first lines, HTML
diagram with Open, and notify row.
- `cards-dark-880.png`
- `cards-cupertino-560.png`
- `html-open-light-880.png`: full-screen HTML with zoom.
- `image-open-dark-880.png`: the existing viewer with download.
- Mobile (Android/iOS WebView, native thumbnails) is unverified: no device run.