Files
pipelines/app/notes/chat-thinking-level.md
T

99 lines
5.4 KiB
Markdown

# Chat thinking level
A per-chat thinking (reasoning effort) control in the conversation header,
beside the model and mode controls: "Thinking: High". Material opens a menu,
Cupertino an action sheet; both list only the levels the chat's current (or
saved next-turn) model accepts and check the current one. With no levels the
control is not shown. It uses the chat's own gateway, so a chat on another
machine is set on that machine's server.
## Protocol
- Session rows (`chat_sessions`, `chat_create`, session reads) carry
`effort` (current level, `""` = harness default), `efforts`
(`[{id, name, description?}]`) and `default_effort`, only when the server
knows the row model's levels without starting a harness.
- `chat_models` entries carry `efforts`/`default_effort` per model, and the
reply carries `chat_effort` with the chat's current state.
- `chat_effort {session_id, effort}` sets a level. The server validates it
against the current model's levels; the reply echoes
`{session_id, effort, efforts, default_effort}`. Errors: unsupported harness,
model without levels, a level the model does not list.
- Levels apply from the next turn. A running turn keeps its level; the menu
says so and the wide label reads "Thinking: Low · next turn" until the turn
ends.
## Per harness
- **Codex** (app-server 0.164): levels are model/list
`supportedReasoningEfforts` for the chat's model (catalog cached for one
minute so a list does not read it per row), the default is
`defaultReasoningEffort`, and the current level is the thread's own
`reasoningEffort` from thread/read unless Pipelines stored one. The stored
level is sent as turn/start `effort` (which Codex keeps for later turns).
Pipelines stores it in `operator_sessions.chat_effort`, re-applies it
before every turn and after a restart, and `/new` copies it.
- **Claude Code** (2.1.319): levels are the initialize models'
`supportedEffortLevels` where `supportsEffort` is true (Haiku 4.5 has none,
so no control). The level is sent with the `apply_flag_settings` control
request (`{"effortLevel": level}`, `null` restores the default) before the
next turn, only when the process was told another level. It is session-only
and never written to a settings file. Stored and re-applied like Codex.
Claude Code's initialize reports no default level, so with none chosen the
label reads "Thinking: Default". The levels come from a CLI's initialize: after
a server restart, before any Claude chat has started a CLI, rows carry no
levels and the control stays hidden until a CLI initializes (opening the
model chooser or sending a turn does that).
- **OpenCode** (2.0.x, v2 API): levels are the `/api/model` entry's
`variants` for the session's model. The chosen variant is the session
model's `variant`, set with `POST /api/session/{id}/model
{model:{id, providerID, variant}}`, which OpenCode itself persists, so
Pipelines stores nothing. A session without one reports `variant:
"default"`, which is not a listed level and shows as the default. The v1
serve backend has no variants: no control.
- A model switch (Codex/Claude) keeps a stored level the new model accepts;
otherwise it becomes the new model's default (or the harness default). The
open control re-reads the levels from `chat_models` after a model switch.
## Limits
- The create-chat dialog offers no model choice (new chats start on the
server default model), so it shows no level either; set it in the header.
- If the opened row's model had no levels, the control is absent, so a switch
to a model with levels shows the control after the list re-reads (reopen).
- Phone widths show the control as an icon (its semantics label and menu
name the level), like the mode control.
## Verification (2026-11-09)
- Go: `go test ./internal/protocol ./internal/pipeline ./internal/codex
./internal/server` ok. `./internal/claude` and `./internal/opencode`: all new
tests pass; one pre-existing failure each (`TestServiceTokenIsPassedOnlyFromAPrivateFile`,
`TestStageDiscoveryRequiresPrivateRegistrationBeforeSessionWork`), which fail
identically on base 8ca5a1fd.
- Flutter: `tool/memwatch/full.sh test/features/chat`: `PASS: 1123 passed, 0
failed`. `tool/analyze.sh`: clean.
- Live Claude Code (opt-in `PIPELINES_LIVE_CLAUDE_EFFORT=1 go test
./internal/claude -run TestClaudeEffortLive -v`, scratch directory, two Haiku
turns through the adapter, then `get_settings`):
```
chose low: CLI applied model=claude-haiku-5-5 effort=low
chose max: CLI applied model=claude-haiku-5-5 effort=max
haiku levels from initialize: [low medium high xhigh max]
--- PASS: TestClaudeEffortLive (5.09s)
```
- Live Codex, read-only (`PIPELINES_LIVE_CODEX_EFFORT=1 go test
./internal/codex -run TestCodexEffortCatalogLive -v`): e.g.
`gpt-6.1-sol default="low" levels=[low medium high xhigh max ultra]`,
`gpt-6-luna default="medium" levels=[low medium high xhigh max]`. No Codex
turn was run, so turn/start `effort` was verified by the adapter test only.
- Live OpenCode, no inference, isolated HOME with `opencode api --standalone`:
a new session reported `"variant":"default"`; after posting
`variant:"high"` the session read back
`{"id": "inkling-2-small-free", "providerID": "opencode", "variant": "high"}`.
No OpenCode turn was run.
- Not run: a full Pipelines server with the Flutter app against a live
harness. Browser captures used the fixture app (`?chats=2&header=1`).