diff --git a/docs/cli.md b/docs/cli.md index efce5665..3fbabfd3 100644 --- a/docs/cli.md +++ b/docs/cli.md @@ -61,6 +61,42 @@ ntfy hosts are not misrepresented as local. Publishing URLs and credentials are never returned. Missing public phone configuration is labeled explicitly. `pipelines service status` remains the separate OS autostart status command. +## Search + +`pipelines search QUERY [--type KIND] [--limit N] [--json]` runs one read-only +query across the whole system: pipeline names and their versions, run identity, +status and dates, stage and visit identifiers, artifact names and types, and the +text of stage inputs, outputs, evidence and transcripts. It never reads +artifact contents, never makes a per-result follow-up request, and asks for no +confirmation. + +- Free text matches case-insensitively. A bare status word (`queued`, `running`, + `succeeded`, `failed`, `canceled`, `interrupted`) selects those runs, and a lone + date expression (`today`, `yesterday`, `this week` — Monday 00:00 local — or a + literal `YYYY-MM-DD`) selects the runs created in that window; any other text, + including a status word inside a sentence, stays free text. +- `--type` is the only narrowing mechanism and takes one kind: `pipeline`, + `version`, `run`, `step`, `field` or `artifact`. An unknown kind is an error, + never a silently ignored filter. +- Ranking: an exact name match (score 3) beats a name substring (2) beats a + body-text match (1); equal tiers break by recency, then a stable kind/target + order. Every result carries `score`, `matched_field` and `recency` so a client + can re-rank. +- Bounds: the result cap defaults to 50 and clamps to 200 (`0` = default); + every kind scans at most 2000 rows, and field-text scanning stops at 2000 steps + or 1 MiB decoded. `truncated` plus `scanned_steps`, `scanned_bytes`, `cap` and + `scan_bound_bytes` report when a bound applied. +- Targets are stable and navigable: `pipeline:`, `pipeline::v`, + `run:`, `run::`, `run:::`, or an + artifact URL served by the artifact endpoint. + +```sh +pipelines search "release notes" --json +pipelines search failed --json +pipelines search 2026-10-16 --type run --json +pipelines search "acceptance check" --type field --limit 10 +``` + ## Draft-first harness workflow ```sh @@ -194,6 +230,7 @@ Submitting activates included schedules, so review them before publishing. | TUI action | Agent-capable CLI | | --- | --- | | Browse/search pipelines | `list --json`, `show NAME --json` | +| Search everything | `search QUERY --json` (see [Search](#search)) | | Original source and history | `export NAME --version N`, `versions NAME --json` | | Browse templates/source | `templates --json`, `template KEY --source` | | Create or edit a definition | `new`, `edit`, `drafts` | @@ -217,10 +254,11 @@ printf '%s' '{"type":"get_pipeline","pipeline_id":1,"version":1}' | pipelines rp Existing protocol commands: `ping`, `list_pipelines`, `get_pipeline`, `list_templates`, `validate`, `submit`, `run_pipeline`, `list_runs`, `get_run`, -`cancel_run`, `retry_run`, `get_status`. `list_runs` also returns all-history `usage` -aggregates. See `internal/protocol/message.go` for JSON envelopes. Mutating RPCs -have real effects; they are not previews. TUI-only animation, scrolling, and -transient client logs are presentation, not separate server capabilities. +`cancel_run`, `retry_run`, `get_status`, `search`. `list_runs` also returns +all-history `usage` aggregates. See `internal/protocol/message.go` for JSON +envelopes. Mutating RPCs have real effects; they are not previews. TUI-only +animation, scrolling, and transient client logs are presentation, not separate +server capabilities. ## Scripted runs diff --git a/docs/ui-ux-review.md b/docs/ui-ux-review.md index 8982002c..df9a076d 100644 --- a/docs/ui-ux-review.md +++ b/docs/ui-ux-review.md @@ -38,7 +38,7 @@ and external editor/OpenCode behavior were not assessed by that review. | Status | Narrow cards hide addresses; scrolling does not reveal them. | Wrap card contents and scroll the actual service-card canvas. | | Task form | Tool lists consume prompt space; wrong footer. | Larger task area, compact scope summary, correct submit/cancel controls. | | CLI pipeline picker | Background Home controls remain visible. | Picker-specific filter/select/cancel controls. | -| Search | “Omnisearch” promises broader scope than pipeline search. | “Search pipelines” and specific open/cancel instructions. | +| Search | “Omnisearch” promises broader scope than pipeline search. | Keep the TUI omnisearch pipeline-scoped and labelled “Search pipelines” with open/cancel instructions; the broad cross-kind search is the server-backed `pipelines search QUERY [--type KIND]` command, whose grammar, kinds, ranking and bounds are documented in [CLI and agent authoring](cli.md#search). | | Commands | No descriptions or indication of hidden entries. | Selected-command explanation and visible range/count. | | Versions | Selected/latest markers truncate on narrow screens. | Correct view-version footer; further metadata prioritization remains a follow-up. | | Naming/publication | Naming inherits publication shortcuts. | Input-specific name-saving/retention footer. | diff --git a/internal/author/skill/references/operations.md b/internal/author/skill/references/operations.md index 44273aba..a39e844b 100644 --- a/internal/author/skill/references/operations.md +++ b/internal/author/skill/references/operations.md @@ -71,6 +71,21 @@ delete databases, sessions, volumes or credentials to repair a workflow. Use the authoritative `CLI_PREFIX` from the prompt: - `CLI_PREFIX runs --json`: run history (optional `--pipeline NAME` scope). +- `CLI_PREFIX search QUERY [--type KIND] [--limit N] --json`: one read-only query + across pipelines, versions, runs, steps, field text and artifacts. QUERY is free + text, a bare status word (`queued`/`running`/`succeeded`/`failed`/`canceled`/ + `interrupted`) selecting those runs, or a lone date expression (`today`, + `yesterday`, `this week` — Monday 00:00 local — or `YYYY-MM-DD`) selecting the + runs created in that window; anything else stays free text. `--type` is the only + narrowing mechanism and takes one of `pipeline`, `version`, `run`, `step`, + `field`, `artifact`; an unknown kind is an error. Ranking is exact name (3) > + name substring (2) > body text (1), equal tiers by recency, and every hit + reports `score`, `matched_field` and `recency`. The cap defaults to 50 and clamps + to 200; every kind scans at most 2000 rows and field scanning stops at 2000 + steps / 1 MiB and reports + `truncated`/`scanned_steps`/`scanned_bytes`/`cap`/`scan_bound_bytes`. Targets are + stable: `pipeline:`, `pipeline::v`, `run:`, + `run::`, `run:::`, or an artifact URL. - `CLI_PREFIX inspect RUN_ID --json`: selected run, visits and retained evidence. Exact stage outputs are authoritative; a transcript over the 32 KiB retained bound is a valid, credential-redacted preview envelope (with preserved stage diff --git a/internal/cli/app.go b/internal/cli/app.go index de310df8..eccf6bd6 100644 --- a/internal/cli/app.go +++ b/internal/cli/app.go @@ -37,9 +37,10 @@ type options struct { url, name, prompt, template, workspace, paths, key, opencode, model string output string conflict string + kind string all bool json, source, noOpen, yes, allow, createOnly, force, wait bool - version, expected int + version, expected, limit int timeout, waitTimeout time.Duration } @@ -99,6 +100,9 @@ func (a App) flags(command string) (*flag.FlagSet, *options) { f.DurationVar(&o.waitTimeout, "wait-timeout", 0, "Client limit for --wait after the run is admitted (0 waits until terminal); separate from per-request --timeout") case "runs": f.StringVar(&o.name, "pipeline", "", "Filter by exact pipeline name or ID") + case "search": + f.StringVar(&o.kind, "type", "", "Narrow to one kind: "+strings.Join(protocol.SearchKinds, ", ")) + f.IntVar(&o.limit, "limit", 0, "Maximum results (default 50, max 200)") case "retry": f.BoolVar(&o.yes, "yes", false, "Confirm retry without an interactive prompt") f.StringVar(&o.key, "request-key", "", "Idempotency key (default: generated unique key)") @@ -697,6 +701,31 @@ func (a App) Run(ctx context.Context, args []string) error { return err case "import": return a.importBundle(ctx, o, f.Args(), call) + case "search": + query := strings.Join(pos, " ") + if strings.TrimSpace(query) == "" { + return fmt.Errorf("search requires a query; use pipelines help search") + } + r, err := call(protocol.Message{Type: protocol.Search, Search: &protocol.SearchRequest{Query: query, Kind: o.kind, Limit: o.limit}}) + if err != nil { + return err + } + if r.SearchResults == nil { + return fmt.Errorf("server did not return search results; update pipelines-server") + } + if o.json { + return a.output(r.SearchResults) + } + if len(r.SearchResults.Results) == 0 { + fmt.Fprintln(a.Out, "No matches") + } + for _, hit := range r.SearchResults.Results { + fmt.Fprintf(a.Out, "%s\t%s\t%d\t%s\t%s\n", hit.Kind, hit.Target, hit.Score, hit.MatchedField, searchLabel(hit)) + } + if r.SearchResults.Truncated { + fmt.Fprintf(a.Err, "Results truncated (cap %d, scan bound %d bytes)\n", r.SearchResults.Cap, r.SearchResults.ScanBoundBytes) + } + return nil case "show", "versions": value, err := one() if err != nil { @@ -819,6 +848,26 @@ func (a App) output(value any) error { return encoder.Encode(value) } +// searchLabel renders one search hit's compact human line. Each kind exposes a +// different typed payload; the target stays the navigable identity. +func searchLabel(hit protocol.SearchResult) string { + switch { + case hit.Pipeline != nil: + return fmt.Sprintf("%s · v%d · %d stages", hit.Pipeline.Name, hit.Pipeline.Version, hit.Pipeline.Stages) + case hit.Version != nil: + return fmt.Sprintf("pipeline %d · v%d · %s", hit.Version.PipelineID, hit.Version.Version, hit.Version.ContentHash) + case hit.Run != nil: + return fmt.Sprintf("run %d · %s · %s", hit.Run.ID, hit.Run.PipelineName, hit.Run.Status) + case hit.Step != nil: + return fmt.Sprintf("run %d · step %d · %s · %s", hit.Step.RunID, hit.Step.Sequence, hit.Step.Stage, hit.Step.Outcome) + case hit.Field != nil: + return fmt.Sprintf("run %d · step %d · %s: %s", hit.Field.RunID, hit.Field.Sequence, hit.Field.Field, hit.Field.Snippet) + case hit.Artifact != nil: + return fmt.Sprintf("run %d · step %d · %s · %s", hit.Artifact.RunID, hit.Artifact.Sequence, hit.Artifact.Name, hit.Artifact.ContentType) + } + return "" +} + type callFunc func(protocol.Message) (protocol.Message, error) type resolveFunc func(string) (protocol.PipelineSummary, error) diff --git a/internal/cli/commands.go b/internal/cli/commands.go index ba79a79a..d0c5c4cc 100644 --- a/internal/cli/commands.go +++ b/internal/cli/commands.go @@ -10,6 +10,7 @@ var usage = map[string]string{ "validate": "validate [--json]", "submit": "submit [--name NAME] [--create-only] [--yes] [--json]", "list": "list [--json]", + "search": "search QUERY [--type KIND] [--limit N] [--json]", "show": "show [--version N] [--json]", "versions": "versions [--json]", "export": "export [name|id ...] [--output FILE.zip] [--all] [--version N]", @@ -44,6 +45,7 @@ var notes = map[string]string{ "retry": "Start a new attempt of a failed run with the same initial input, directory, pipeline version, and edit scope. Original history is retained. Already-completed side effects are not undone. --wait/--wait-timeout behave as for run: a reached wait limit leaves the new run active; inspect read-only with `inspect --json`, or reattach with --wait and the original --request-key instead of starting a duplicate retry.", "debug": "Open a fresh configured harness conversation to investigate a run using inspect/show/runs CLI queries. --harness opencode|pi|nvim chooses another configured harness or read-only nvim packet. Does not resume implementation or retry. --json returns the prompt without launching anything.", "rpc": "Accepts a JSON protocol envelope; '-' reads stdin. Every TUI server operation is available here too. Mutating commands are not dry runs.", + "search": "One read-only query over pipelines, versions, runs, steps, field text and artifacts. QUERY is free text, a bare status word (queued, running, succeeded, failed, canceled, interrupted) selecting those runs, or a lone date expression (today, yesterday, this week [Monday 00:00 local], or YYYY-MM-DD) selecting the runs created in that window. --type is the only narrowing mechanism and takes one kind: pipeline, version, run, step, field or artifact; an unknown kind is an error. Ranking: an exact name match (3) beats a name substring (2) beats a body-text match (1); equal tiers break by recency, then a stable kind/target tie-break, and every hit reports score, matched_field and recency so a client can re-rank. Bounds: the result cap defaults to 50 and clamps to 200 (0 = default); every kind scans at most 2000 rows and field-text scanning is additionally capped at 2000 steps / 1 MiB; Truncated plus scanned_steps, scanned_bytes and scan_bound_bytes report when a bound applied. Targets are stable: pipeline:, pipeline::v, run:, run::, run:::, or an artifact URL.", } var serviceHelp = map[string]string{ diff --git a/internal/cli/help.go b/internal/cli/help.go index d0090b87..7552a857 100644 --- a/internal/cli/help.go +++ b/internal/cli/help.go @@ -13,6 +13,7 @@ AUTHOR BROWSE list List registered pipelines + search QUERY Search pipelines, versions, runs, steps, field text, artifacts show Inspect graph, settings, source and history versions List immutable versions export [name|id ...] Select/export definitions as ZIP; one name prints Markdown diff --git a/internal/pipeline/runs.go b/internal/pipeline/runs.go index 03c8ca6f..dc8a2ac8 100644 --- a/internal/pipeline/runs.go +++ b/internal/pipeline/runs.go @@ -107,11 +107,19 @@ func (s *Store) CreateRun(ctx context.Context, d *protocol.PipelineDetail, key, return id, true, nil } +// runFrom is the shared pipeline_runs/pipelines join every run-row read builds +// on, so the run projections cannot drift. +const runFrom = `FROM pipeline_runs r JOIN pipelines p ON p.id=r.pipeline_id` + +// runStamps formats a run row's created/finished timestamps; reused by runSelect +// and searchRuns instead of copied per query. +const runStamps = `strftime('%Y-%m-%dT%H:%M:%SZ',r.created_at),COALESCE(strftime('%Y-%m-%dT%H:%M:%SZ',r.finished_at),'')` + const runSelect = `SELECT r.id,r.pipeline_id,p.name,r.version,v.content_hash,r.status,r.outcome,r.current_stage, -strftime('%Y-%m-%dT%H:%M:%SZ',r.created_at),COALESCE(strftime('%Y-%m-%dT%H:%M:%SZ',r.finished_at),''),r.diagnostic, +` + runStamps + `,r.diagnostic, CASE WHEN r.status='succeeded' THEN r.output_json ELSE COALESCE((SELECT output_json FROM run_steps WHERE run_id=r.id AND status='completed' ORDER BY sequence DESC LIMIT 1),'{}') END, (SELECT COUNT(*) FROM run_steps WHERE run_id=r.id AND status='completed') -FROM pipeline_runs r JOIN pipelines p ON p.id=r.pipeline_id JOIN pipeline_versions v ON v.pipeline_id=r.pipeline_id AND v.version=r.version` +` + runFrom + ` JOIN pipeline_versions v ON v.pipeline_id=r.pipeline_id AND v.version=r.version` func scanRun(row interface{ Scan(...any) error }, r *protocol.RunSummary) error { var output string diff --git a/internal/pipeline/search.go b/internal/pipeline/search.go new file mode 100644 index 00000000..dff75a83 --- /dev/null +++ b/internal/pipeline/search.go @@ -0,0 +1,686 @@ +package pipeline + +import ( + "context" + "database/sql" + "encoding/json" + "fmt" + "sort" + "strconv" + "strings" + "time" + "unicode" + + "git.a-wan.net/alan/pipelines/internal/protocol" +) + +// Search answers one ranked, bounded, read-only query over the existing tables. +// It scans no artifact contents and makes no follow-up calls: each kind runs at +// most one query, field text is decoded with decodeValues over a fixed scan +// budget, and ranking/tie-breaking happens in memory over the candidate set. +func (s *Store) Search(ctx context.Context, req protocol.SearchRequest, now time.Time) (*protocol.SearchResponse, error) { + kind := strings.TrimSpace(req.Kind) + if kind != "" && searchKindIndex(kind) < 0 { + return nil, fmt.Errorf("unknown search kind %q; use one of %s", req.Kind, strings.Join(protocol.SearchKinds, ", ")) + } + response := &protocol.SearchResponse{ + Query: req.Query, + Kind: kind, + Results: []protocol.SearchResult{}, + Cap: searchLimit(req.Limit), + ScanBoundBytes: protocol.SearchMaxScanBytes, + } + // An empty query is a well-defined no-op, never a match-everything scan. + if strings.TrimSpace(req.Query) == "" { + return response, nil + } + terms := parseSearchTerms(req.Query, now) + // A bare status word or date selects runs; it is never treated as body text. + selectorOnly := terms.Text == "" + + tx, err := s.db.BeginTx(ctx, &sql.TxOptions{ReadOnly: true}) + if err != nil { + return nil, err + } + defer tx.Rollback() + + var results []protocol.SearchResult + var scan searchScan + for i, candidate := range protocol.SearchKinds { + if kind != "" && searchKindIndex(kind) != i { + continue + } + if selectorOnly && candidate != protocol.SearchKindRun { + continue + } + var hits []protocol.SearchResult + var err error + switch candidate { + case protocol.SearchKindPipeline: + hits, err = searchPipelines(ctx, tx, terms, &scan) + case protocol.SearchKindVersion: + hits, err = searchVersions(ctx, tx, terms, &scan) + case protocol.SearchKindRun: + hits, err = searchRuns(ctx, tx, terms, &scan) + case protocol.SearchKindStep: + hits, err = searchSteps(ctx, tx, terms, &scan) + case protocol.SearchKindField: + hits, err = searchFields(ctx, tx, terms, &scan) + case protocol.SearchKindArtifact: + hits, err = searchArtifacts(ctx, tx, terms, &scan) + } + if err != nil { + return nil, err + } + results = append(results, hits...) + } + if err := tx.Commit(); err != nil { + return nil, err + } + response.ScannedSteps = scan.steps + response.ScannedBytes = scan.bytes + if scan.truncated { + response.Truncated = true + } + + sortSearchResults(results) + if len(results) > response.Cap { + results = results[:response.Cap] + response.Truncated = true + } + if results == nil { + results = []protocol.SearchResult{} + } + response.Results = results + return response, nil +} + +func searchKindIndex(kind string) int { + for i, k := range protocol.SearchKinds { + if k == kind { + return i + } + } + return -1 +} + +// searchLimit clamps a requested cap. Zero, negative or over-max requests fall +// back to the default or the maximum, so the cap is always enforced server-side. +func searchLimit(limit int) int { + if limit <= 0 { + return protocol.SearchDefaultLimit + } + if limit > protocol.SearchMaxLimit { + return protocol.SearchMaxLimit + } + return limit +} + +// searchTerms is the parsed query. Text carries free text; Status and the +// From/To window carry a bare status word or a date expression that selects runs. +type searchTerms struct { + Text string + Status string + HasDate bool + From time.Time + To time.Time +} + +// parseSearchTerms recognises a lone status word and a lone date expression and +// leaves everything else as free text, so ordinary words are never misread as a +// selector. Relative dates resolve against the injected now in its own location; +// the comparison window is converted to UTC to match the stored timestamps. +func parseSearchTerms(query string, now time.Time) searchTerms { + trimmed := strings.TrimSpace(query) + lower := strings.ToLower(trimmed) + for _, status := range protocol.SearchStatuses { + if lower == status { + return searchTerms{Status: status} + } + } + location := now.Location() + day := time.Date(now.Year(), now.Month(), now.Day(), 0, 0, 0, 0, location) + switch lower { + case "today": + return dateTerms(day, day.AddDate(0, 0, 1)) + case "yesterday": + return dateTerms(day.AddDate(0, 0, -1), day) + case "this week": + // Monday 00:00 local is the start of the week. + start := day.AddDate(0, 0, -((int(day.Weekday()) + 6) % 7)) + return dateTerms(start, start.AddDate(0, 0, 7)) + } + if parsed, err := time.ParseInLocation("2006-01-02", trimmed, location); err == nil { + return dateTerms(parsed, parsed.AddDate(0, 0, 1)) + } + return searchTerms{Text: trimmed} +} + +func dateTerms(from, to time.Time) searchTerms { + return searchTerms{HasDate: true, From: from.UTC(), To: to.UTC()} +} + +const searchSQLTime = "2006-01-02 15:04:05" + +// searchStepJoin is the shared run_steps/step_control source and projections the +// step and field scans build on, so those queries cannot drift apart. +const ( + searchStepJoin = `FROM run_steps s LEFT JOIN step_control c ON c.run_id=s.run_id AND c.sequence=s.sequence` + searchStepEvidence = `COALESCE(c.evidence_json,'')` + searchStepRecency = `COALESCE(strftime('%Y-%m-%dT%H:%M:%SZ',s.started_at),'')` +) + +// searchRowBudget bounds how many raw rows one non-body kind scans before it +// stops and reports Truncated, so a LIKE scan over a very large table cannot +// scale without limit. Field text uses the separate SearchMaxScanSteps budget. +const searchRowBudget = 2000 + +// searchLike builds a case-insensitive substring pattern, escaping LIKE +// metacharacters so a query can never broaden the pattern. +func searchLike(text string) string { + escaped := strings.NewReplacer(`\`, `\\`, `%`, `\%`, `_`, `\_`).Replace(strings.ToLower(text)) + return "%" + escaped + "%" +} + +// searchTier ranks a "name" match: exact beats a substring. The third tier (a +// body-text match) is assigned by the caller. +func searchTier(value, text string) (int, bool) { + if text == "" { + return 0, false + } + if strings.EqualFold(value, text) { + return 3, true + } + if strings.Contains(strings.ToLower(value), strings.ToLower(text)) { + return 2, true + } + return 0, false +} + +func searchContains(value, text string) bool { + return text != "" && strings.Contains(strings.ToLower(value), strings.ToLower(text)) +} + +func sortSearchResults(results []protocol.SearchResult) { + sort.SliceStable(results, func(i, j int) bool { + a, b := results[i], results[j] + if a.Score != b.Score { + return a.Score > b.Score + } + if a.Recency != b.Recency { + return a.Recency > b.Recency + } + if a.Kind != b.Kind { + return a.Kind < b.Kind + } + return a.Target < b.Target + }) +} + +func searchPipelines(ctx context.Context, tx *sql.Tx, terms searchTerms, scan *searchScan) ([]protocol.SearchResult, error) { + if terms.Text == "" { + return nil, nil + } + rows, err := tx.QueryContext(ctx, `SELECT p.id,p.name,v.version,json_array_length(v.graph_json,'$.stages'),COALESCE(strftime('%Y-%m-%dT%H:%M:%SZ',v.created_at),'') +FROM pipelines p JOIN pipeline_versions v ON v.pipeline_id=p.id +WHERE v.version=(SELECT MAX(version) FROM pipeline_versions WHERE pipeline_id=p.id) +AND lower(p.name) LIKE ? ESCAPE '\' +ORDER BY p.name,p.id LIMIT ?`, searchLike(terms.Text), searchRowBudget+1) + if err != nil { + return nil, err + } + defer rows.Close() + var results []protocol.SearchResult + scanned := 0 + for rows.Next() { + if scanned == searchRowBudget { + scan.truncated = true + break + } + scanned++ + var id int64 + var name, recency string + var version, stages int + if err := rows.Scan(&id, &name, &version, &stages, &recency); err != nil { + return nil, err + } + score, ok := searchTier(name, terms.Text) + if !ok { + continue + } + results = append(results, protocol.SearchResult{ + Kind: protocol.SearchKindPipeline, Target: searchPipelineTarget(id), + Score: score, MatchedField: "name", Recency: recency, + Pipeline: &protocol.SearchPipeline{ID: id, Name: name, Version: version, Stages: stages}, + }) + } + return results, rows.Err() +} + +func searchVersions(ctx context.Context, tx *sql.Tx, terms searchTerms, scan *searchScan) ([]protocol.SearchResult, error) { + if terms.Text == "" { + return nil, nil + } + query := `SELECT v.pipeline_id,v.version,v.content_hash,COALESCE(strftime('%Y-%m-%dT%H:%M:%SZ',v.created_at),''),v.source +FROM pipeline_versions v WHERE lower(v.source) LIKE ? ESCAPE '\'` + args := []any{searchLike(terms.Text)} + var wanted int64 + numeric := false + if n, err := parsePositiveInt(terms.Text); err == nil { + query += " OR v.version=?" + args = append(args, n) + wanted, numeric = n, true + } + query += " ORDER BY v.pipeline_id,v.version DESC LIMIT ?" + args = append(args, searchRowBudget+1) + rows, err := tx.QueryContext(ctx, query, args...) + if err != nil { + return nil, err + } + defer rows.Close() + var results []protocol.SearchResult + scanned := 0 + for rows.Next() { + if scanned == searchRowBudget { + scan.truncated = true + break + } + scanned++ + var pipelineID int64 + var version int + var hash, recency, source string + if err := rows.Scan(&pipelineID, &version, &hash, &recency, &source); err != nil { + return nil, err + } + score, matched := 1, "source" + if numeric && int64(version) == wanted { + score, matched = 3, "version" + } else if !searchContains(source, terms.Text) { + continue + } + results = append(results, protocol.SearchResult{ + Kind: protocol.SearchKindVersion, Target: searchVersionTarget(pipelineID, version), + Score: score, MatchedField: matched, Recency: recency, + Version: &protocol.SearchVersion{PipelineID: pipelineID, Version: version, ContentHash: hash, CreatedAt: recency}, + }) + } + return results, rows.Err() +} + +func searchRuns(ctx context.Context, tx *sql.Tx, terms searchTerms, scan *searchScan) ([]protocol.SearchResult, error) { + // Reuse runSelect's join and timestamp projection rather than restating the + // run-row SELECT; only the fields search needs are added. + base := `SELECT r.id,p.name,r.status,r.workspace,` + runStamps + `,r.diagnostic ` + runFrom + var query string + var args []any + switch { + case terms.Status != "": + query = base + " WHERE r.status=?" + args = append(args, terms.Status) + case terms.HasDate: + query = base + " WHERE r.created_at>=? AND r.created_at= protocol.SearchMaxScanSteps { + scan.truncated = true + break + } + var runID int64 + var sequence int + var input, output, transcript, evidence, recency string + if err := rows.Scan(&runID, &sequence, &input, &output, &transcript, &evidence, &recency); err != nil { + return nil, err + } + size := len(input) + len(output) + len(transcript) + len(evidence) + if scan.bytes+size > protocol.SearchMaxScanBytes { + scan.truncated = true + break + } + scan.bytes += size + scan.steps++ + for _, field := range []struct { + name string + raw string + }{ + {name: "input", raw: input}, + {name: "output", raw: output}, + {name: "evidence", raw: evidence}, + } { + if field.raw == "" { + continue + } + var values map[string]any + if err := decodeValues(field.raw, &values); err != nil { + continue + } + text, ok := findMatch(values, terms.Text) + if !ok { + continue + } + results = append(results, protocol.SearchResult{ + Kind: protocol.SearchKindField, Target: searchFieldTarget(runID, sequence, field.name), + Score: 1, MatchedField: field.name, Recency: recency, + Field: &protocol.SearchField{RunID: runID, Sequence: sequence, Field: field.name, Snippet: searchSnippet(text, terms.Text)}, + }) + } + if searchContains(transcript, terms.Text) { + results = append(results, protocol.SearchResult{ + Kind: protocol.SearchKindField, Target: searchFieldTarget(runID, sequence, "transcript"), + Score: 1, MatchedField: "transcript", Recency: recency, + Field: &protocol.SearchField{RunID: runID, Sequence: sequence, Field: "transcript", Snippet: searchSnippet(transcript, terms.Text)}, + }) + } + } + return results, rows.Err() +} + +func searchArtifacts(ctx context.Context, tx *sql.Tx, terms searchTerms, scan *searchScan) ([]protocol.SearchResult, error) { + if terms.Text == "" { + return nil, nil + } + pattern := searchLike(terms.Text) + rows, err := tx.QueryContext(ctx, `SELECT a.run_id,a.sequence,a.artifacts_json,COALESCE(strftime('%Y-%m-%dT%H:%M:%SZ',r.created_at),'') +FROM step_artifacts a JOIN pipeline_runs r ON r.id=a.run_id +WHERE lower(a.artifacts_json) LIKE ? ESCAPE '\' +ORDER BY a.run_id DESC,a.sequence DESC LIMIT ?`, pattern, searchRowBudget+1) + if err != nil { + return nil, err + } + defer rows.Close() + var results []protocol.SearchResult + scanned := 0 + for rows.Next() { + if scanned == searchRowBudget { + scan.truncated = true + break + } + scanned++ + var runID int64 + var sequence int + var artifacts, recency string + if err := rows.Scan(&runID, &sequence, &artifacts, &recency); err != nil { + return nil, err + } + var evidence stageArtifactsEvidence + if err := json.Unmarshal([]byte(artifacts), &evidence); err != nil { + continue + } + for _, file := range evidence.Files { + score, matched := 1, "content_type" + if s, ok := searchTier(file.Name, terms.Text); ok { + score, matched = s, "name" + } else if !searchContains(file.ContentType, terms.Text) { + continue + } + results = append(results, protocol.SearchResult{ + Kind: protocol.SearchKindArtifact, Target: file.URL, + Score: score, MatchedField: matched, Recency: recency, + Artifact: &protocol.SearchArtifact{RunID: runID, Sequence: sequence, Name: file.Name, ContentType: file.ContentType, SizeBytes: file.SizeBytes, URL: file.URL}, + }) + } + } + return results, rows.Err() +} + +// findMatch walks decoded JSON values in a stable key order and returns the +// first string leaf containing the query, so a hit and its snippet are +// deterministic rather than map-iteration dependent. +func findMatch(value any, text string) (string, bool) { + switch typed := value.(type) { + case string: + if searchContains(typed, text) { + return typed, true + } + case map[string]any: + keys := make([]string, 0, len(typed)) + for key := range typed { + keys = append(keys, key) + } + sort.Strings(keys) + for _, key := range keys { + if match, ok := findMatch(typed[key], text); ok { + return match, true + } + } + case []any: + for _, item := range typed { + if match, ok := findMatch(item, text); ok { + return match, true + } + } + } + return "", false +} + +// searchSnippet reduces text to a single clean line and centers a short window +// on the first case-insensitive match, folding runes so UTF-8 boundaries are +// never split and the result is always well-formed. +func searchSnippet(text, query string) string { + normalized := protocol.NormalizeText(text) + runes := []rune(normalized) + index := matchRunes(runes, []rune(query)) + if index < 0 { + return truncateRunes(normalized, 80) + } + const margin = 40 + start := index - margin + if start < 0 { + start = 0 + } + end := index + len([]rune(query)) + margin + if end > len(runes) { + end = len(runes) + } + snippet := string(runes[start:end]) + if start > 0 { + snippet = "…" + snippet + } + if end < len(runes) { + snippet = snippet + "…" + } + return snippet +} + +// matchRunes returns the rune index of query in text, case-insensitively. +func matchRunes(text, query []rune) int { + if len(query) == 0 || len(query) > len(text) { + return -1 + } + for i := 0; i+len(query) <= len(text); i++ { + match := true + for j := range query { + if unicode.ToLower(text[i+j]) != unicode.ToLower(query[j]) { + match = false + break + } + } + if match { + return i + } + } + return -1 +} + +func truncateRunes(text string, limit int) string { + runes := []rune(text) + if len(runes) <= limit { + return text + } + return string(runes[:limit]) + "…" +} + +func searchPipelineTarget(id int64) string { return fmt.Sprintf("pipeline:%d", id) } + +func searchVersionTarget(id int64, version int) string { + return fmt.Sprintf("pipeline:%d:v%d", id, version) +} + +func searchRunTarget(id int64) string { return fmt.Sprintf("run:%d", id) } + +func searchStepTarget(id int64, sequence int) string { return fmt.Sprintf("run:%d:%d", id, sequence) } + +func searchFieldTarget(id int64, sequence int, field string) string { + return fmt.Sprintf("run:%d:%d:%s", id, sequence, field) +} + +func parsePositiveInt(text string) (int64, error) { + n, err := strconv.ParseInt(strings.TrimSpace(text), 10, 64) + if err != nil || n <= 0 { + return 0, fmt.Errorf("not a positive integer") + } + return n, nil +} + +func numericEqual(text string, id int64) bool { + n, err := parsePositiveInt(text) + return err == nil && n == id +} + +func durationSeconds(created, finished string) (float64, bool) { + start, err := time.Parse(time.RFC3339, created) + if err != nil { + return 0, false + } + end, err := time.Parse(time.RFC3339, finished) + if err != nil { + return 0, false + } + if end.Before(start) { + return 0, false + } + return end.Sub(start).Seconds(), true +} diff --git a/internal/pipeline/search_live_test.go b/internal/pipeline/search_live_test.go new file mode 100644 index 00000000..37b40f75 --- /dev/null +++ b/internal/pipeline/search_live_test.go @@ -0,0 +1,119 @@ +package pipeline + +import ( + "context" + "database/sql" + "os" + "strings" + "testing" + "time" + + "git.a-wan.net/alan/pipelines/internal/protocol" +) + +// TestLiveSearchRealDatabase is an env-guarded read-only check against a real +// database. It opens the file read-only (no DDL, no journal change, no writes), +// derives queries from the data itself, and reports one result of each kind it +// can observe. Run it with: +// +// PIPELINES_LIVE_SEARCH_DB=/path/to/pipelines.db go test ./internal/pipeline -run LiveSearch -v +func TestLiveSearchRealDatabase(t *testing.T) { + path := os.Getenv("PIPELINES_LIVE_SEARCH_DB") + if path == "" { + t.Skip("set PIPELINES_LIVE_SEARCH_DB to a real pipelines.db") + } + store, err := OpenReadOnly(path) + if err != nil { + t.Fatalf("open read-only: %v", err) + } + defer store.Close() + requireSearchSchema(t, store) + ctx := context.Background() + pipelines, err := store.List(ctx) + if err != nil { + t.Fatal(err) + } + if len(pipelines) == 0 { + t.Skip("database has no pipelines") + } + now := time.Now() + seen := map[string]protocol.SearchResult{} + record := func(query string) { + response, err := store.Search(ctx, protocol.SearchRequest{Query: query}, now) + if err != nil { + t.Fatalf("search %q: %v", query, err) + } + for _, hit := range response.Results { + if _, ok := seen[hit.Kind]; !ok { + seen[hit.Kind] = hit + } + } + } + queries := []string{} + for _, p := range pipelines { + queries = append(queries, p.Name) + } + // Derive stage/field/artifact queries from a bounded number of recent runs. + runs, err := store.ListRuns(ctx) + if err != nil { + t.Fatal(err) + } + for i := 0; i < len(runs) && i < 5; i++ { + detail, err := store.GetRun(ctx, runs[i].ID) + if err != nil { + continue + } + for j := 0; j < len(detail.Steps) && j < 3; j++ { + step := detail.Steps[j] + if step.Stage != "" { + queries = append(queries, step.Stage) + } + for _, artifact := range step.Artifacts { + queries = append(queries, artifact.Name) + } + for _, key := range []string{"summary", "result", "report", "message", "evidence", "reason"} { + if text, ok := step.Output[key].(string); ok { + for _, word := range strings.Fields(text) { + if len([]rune(word)) >= 6 { + queries = append(queries, word) + break + } + } + } + } + } + } + for _, query := range queries { + record(query) + } + if len(seen) == 0 { + t.Fatal("live database returned no matches for any derived query") + } + for _, kind := range protocol.SearchKinds { + if hit, ok := seen[kind]; ok { + t.Logf("kind=%s target=%s score=%d matched=%s", kind, hit.Target, hit.Score, hit.MatchedField) + } else { + t.Logf("kind=%s not observed in this database", kind) + } + } + if _, ok := seen[protocol.SearchKindRun]; !ok { + t.Fatal("live database returned no run matches") + } +} + +// requireSearchSchema skips (rather than fails) when the supplied database +// predates the tables search reads, so pointing the check at a stale local file +// reports the mismatch instead of a confusing "no such table" error. +func requireSearchSchema(t *testing.T, store *Store) { + t.Helper() + for _, table := range []string{"pipelines", "pipeline_versions", "pipeline_runs", "run_steps", "step_control", "step_artifacts"} { + var name string + err := store.db.QueryRow("SELECT name FROM sqlite_master WHERE type='table' AND name=?", table).Scan(&name) + if err == sql.ErrNoRows { + t.Skipf("database lacks the current search schema (missing %q); point PIPELINES_LIVE_SEARCH_DB at a current database", table) + } + if err != nil { + t.Fatal(err) + } + } +} diff --git a/internal/pipeline/search_test.go b/internal/pipeline/search_test.go new file mode 100644 index 00000000..c259dc8c --- /dev/null +++ b/internal/pipeline/search_test.go @@ -0,0 +1,601 @@ +package pipeline + +import ( + "context" + "path/filepath" + "strconv" + "strings" + "testing" + "time" + "unicode/utf8" + + "git.a-wan.net/alan/pipelines/internal/protocol" +) + +// searchFixture builds one pipeline whose name, source, run, stage, field text +// and artifact all contain "zephyr", so a single free-text query exercises +// every kind. +func searchFixture(t *testing.T) (*Store, int64) { + t.Helper() + ctx := context.Background() + store, err := Open(filepath.Join(t.TempDir(), "search.db")) + if err != nil { + t.Fatal(err) + } + t.Cleanup(func() { store.Close() }) + if _, err := store.Submit(ctx, "Zephyr Build", Template([]string{"zephyr"}, true)); err != nil { + t.Fatal(err) + } + items, err := store.List(ctx) + if err != nil { + t.Fatal(err) + } + definition, err := store.Get(ctx, items[0].ID, 1) + if err != nil { + t.Fatal(err) + } + run, _, err := store.CreateRun(ctx, definition, "zephyr-key", "/work/zephyr", false, nil, map[string]any{"prompt": "zephyr task"}) + if err != nil { + t.Fatal(err) + } + if err := store.BeginStep(ctx, run, 1, "zephyr", map[string]any{"prompt": "zephyr task"}); err != nil { + t.Fatal(err) + } + artifactURL := "/artifacts/" + strconv.FormatInt(run, 10) + "/1/zephyr.txt" + if err := store.FinishStep(ctx, run, protocol.RunStep{ + Sequence: 1, Status: "completed", Outcome: "success", Target: protocol.SuccessEnd, + Output: map[string]any{"summary": "zephyr completed"}, + Control: &protocol.ControlEvidence{Visit: 1, MaxVisits: 2, ModelTarget: "zephyr-model"}, + Artifacts: []protocol.StageArtifact{{Name: "zephyr.txt", SizeBytes: 12, ContentType: "text/plain", URL: artifactURL}}, + }); err != nil { + t.Fatal(err) + } + if err := store.FinishRun(ctx, run, "succeeded", "success", "", map[string]any{"summary": "zephyr completed"}); err != nil { + t.Fatal(err) + } + return store, run +} + +func searchKinds(response *protocol.SearchResponse) map[string]bool { + kinds := map[string]bool{} + for _, hit := range response.Results { + kinds[hit.Kind] = true + } + return kinds +} + +func TestSearchReturnsEveryKind(t *testing.T) { + store, run := searchFixture(t) + response, err := store.Search(context.Background(), protocol.SearchRequest{Query: "zephyr"}, time.Now()) + if err != nil { + t.Fatal(err) + } + kinds := searchKinds(response) + for _, kind := range protocol.SearchKinds { + if !kinds[kind] { + t.Fatalf("kind %q missing from %+v", kind, response.Results) + } + } + if response.Cap != protocol.SearchDefaultLimit || response.ScanBoundBytes != protocol.SearchMaxScanBytes { + t.Fatalf("bounds not reported: %+v", response) + } + if response.Truncated { + t.Fatalf("a tiny store must not report truncation: %+v", response) + } + // Every target must point at an addressable entity an existing command resolves. + ctx := context.Background() + for _, hit := range response.Results { + switch hit.Kind { + case protocol.SearchKindPipeline: + id := targetInt(t, hit.Target, "pipeline:") + if _, err := store.Get(ctx, id, 0); err != nil { + t.Fatalf("pipeline target %q does not resolve: %v", hit.Target, err) + } + case protocol.SearchKindVersion: + parts := strings.Split(strings.TrimPrefix(hit.Target, "pipeline:"), ":v") + if len(parts) != 2 { + t.Fatalf("bad version target %q", hit.Target) + } + if _, err := store.Get(ctx, parseInt(t, parts[0]), int(parseInt(t, parts[1]))); err != nil { + t.Fatalf("version target %q does not resolve: %v", hit.Target, err) + } + case protocol.SearchKindRun: + if _, err := store.GetRun(ctx, targetInt(t, hit.Target, "run:")); err != nil { + t.Fatalf("run target %q does not resolve: %v", hit.Target, err) + } + case protocol.SearchKindStep, protocol.SearchKindField: + fields := strings.Split(strings.TrimPrefix(hit.Target, "run:"), ":") + if len(fields) < 2 { + t.Fatalf("bad step/field target %q", hit.Target) + } + detail, err := store.GetRun(ctx, parseInt(t, fields[0])) + if err != nil { + t.Fatalf("step/field target %q does not resolve: %v", hit.Target, err) + } + sequence := parseInt(t, fields[1]) + found := false + for _, step := range detail.Steps { + if int64(step.Sequence) == sequence { + found = true + } + } + if !found { + t.Fatalf("step/field target %q names no recorded step", hit.Target) + } + case protocol.SearchKindArtifact: + if !strings.HasPrefix(hit.Target, "/artifacts/") { + t.Fatalf("artifact target %q is not a served URL", hit.Target) + } + } + } + if run <= 0 { + t.Fatal("fixture run missing") + } +} + +func targetInt(t *testing.T, target, prefix string) int64 { + t.Helper() + if !strings.HasPrefix(target, prefix) { + t.Fatalf("target %q lacks prefix %q", target, prefix) + } + return parseInt(t, strings.TrimPrefix(target, prefix)) +} + +func parseInt(t *testing.T, value string) int64 { + t.Helper() + n, err := parsePositiveInt(value) + if err != nil { + t.Fatalf("bad number %q: %v", value, err) + } + return n +} + +func TestSearchFieldTextCoversInputOutputEvidence(t *testing.T) { + store, _ := searchFixture(t) + response, err := store.Search(context.Background(), protocol.SearchRequest{Query: "zephyr", Kind: protocol.SearchKindField}, time.Now()) + if err != nil { + t.Fatal(err) + } + fields := map[string]bool{} + for _, hit := range response.Results { + if hit.Field == nil { + t.Fatalf("field kind without a field payload: %+v", hit) + } + fields[hit.Field.Field] = true + if !strings.Contains(hit.Field.Snippet, "zephyr") { + t.Fatalf("snippet lacks the match: %+v", hit.Field) + } + } + for _, field := range []string{"input", "output", "evidence"} { + if !fields[field] { + t.Fatalf("field text search missed %q: %+v", field, response.Results) + } + } +} + +func TestSearchRankingExactSubstringBodyAndRecency(t *testing.T) { + ctx := context.Background() + store, err := Open(filepath.Join(t.TempDir(), "ranking.db")) + if err != nil { + t.Fatal(err) + } + defer store.Close() + submit := func(name string) { + if _, err := store.Submit(ctx, name, Template([]string{"one"}, true)); err != nil { + t.Fatal(err) + } + } + submit("alpha") + submit("alphabet") + items, _ := store.List(ctx) + definition, _ := store.Get(ctx, items[0].ID, 1) + run, _, err := store.CreateRun(ctx, definition, "alpha-key", "/work", false, nil, map[string]any{"prompt": "body alpha match"}) + if err != nil { + t.Fatal(err) + } + if err := store.BeginStep(ctx, run, 1, "one", map[string]any{"prompt": "body alpha match"}); err != nil { + t.Fatal(err) + } + if err := store.FinishStep(ctx, run, protocol.RunStep{Sequence: 1, Status: "completed", Outcome: "success", Target: protocol.SuccessEnd}); err != nil { + t.Fatal(err) + } + + response, err := store.Search(ctx, protocol.SearchRequest{Query: "alpha"}, time.Now()) + if err != nil { + t.Fatal(err) + } + seen := map[int]bool{} + for i, hit := range response.Results { + seen[hit.Score] = true + if i > 0 && response.Results[i-1].Score < hit.Score { + t.Fatalf("scores not descending: %+v", response.Results) + } + } + if !seen[3] || !seen[2] || !seen[1] { + t.Fatalf("expected exact/substring/body tiers: %+v", response.Results) + } + // The exact pipeline match outranks the substring pipeline match. + exactIndex, substringIndex := -1, -1 + for i, hit := range response.Results { + if hit.Pipeline == nil { + continue + } + if hit.Pipeline.Name == "alpha" { + exactIndex = i + } + if hit.Pipeline.Name == "alphabet" { + substringIndex = i + } + if hit.Pipeline.Name == "alpha" && hit.Score != 3 { + t.Fatalf("exact name scored %d", hit.Score) + } + if hit.Pipeline.Name == "alphabet" && hit.Score != 2 { + t.Fatalf("substring name scored %d", hit.Score) + } + } + if exactIndex < 0 || substringIndex < 0 || exactIndex > substringIndex { + t.Fatalf("exact name must outrank a name substring: %+v", response.Results) + } + + // Equal-tier ties break by recency. + older, _, err := store.CreateRun(ctx, definition, "alpha-older", "/work", false, nil, map[string]any{"prompt": "alpha"}) + if err != nil { + t.Fatal(err) + } + newer, _, err := store.CreateRun(ctx, definition, "alpha-newer", "/work", false, nil, map[string]any{"prompt": "alpha"}) + if err != nil { + t.Fatal(err) + } + setCreated(t, store, older, "2026-01-01 00:00:00") + setCreated(t, store, newer, "2026-06-01 00:00:00") + response, err = store.Search(ctx, protocol.SearchRequest{Query: "alpha"}, time.Now()) + if err != nil { + t.Fatal(err) + } + var olderIndex, newerIndex = -1, -1 + for i, hit := range response.Results { + if hit.Run == nil || hit.Run.ID == 0 { + continue + } + if hit.Run.ID == older { + olderIndex = i + } + if hit.Run.ID == newer { + newerIndex = i + } + } + if olderIndex < 0 || newerIndex < 0 || newerIndex > olderIndex { + t.Fatalf("recency tie-break lost: older=%d newer=%d", olderIndex, newerIndex) + } +} + +func TestSearchStatusWordsSelectRuns(t *testing.T) { + ctx := context.Background() + store, err := Open(filepath.Join(t.TempDir(), "statuses.db")) + if err != nil { + t.Fatal(err) + } + defer store.Close() + if _, err := store.Submit(ctx, "statuses", Template([]string{"one"}, true)); err != nil { + t.Fatal(err) + } + items, _ := store.List(ctx) + definition, _ := store.Get(ctx, items[0].ID, 1) + // One run per accepted status word, each with a distinct created_at so a + // date selector can never be mistaken for the status selector. + byStatus := map[string]int64{} + for i, status := range protocol.SearchStatuses { + run, _, err := store.CreateRun(ctx, definition, "status-"+status, "/work", false, nil, nil) + if err != nil { + t.Fatal(err) + } + if _, err := store.db.Exec("UPDATE pipeline_runs SET status=?,created_at=? WHERE id=?", status, "2026-01-0"+strconv.Itoa(i+1)+" 09:00:00", run); err != nil { + t.Fatal(err) + } + byStatus[status] = run + } + now := time.Date(2026, 10, 16, 12, 0, 0, 0, time.UTC) + for _, status := range protocol.SearchStatuses { + response, err := store.Search(ctx, protocol.SearchRequest{Query: status}, now) + if err != nil { + t.Fatal(err) + } + var runs []int64 + for _, hit := range response.Results { + if hit.Run == nil { + t.Fatalf("status %q returned a non-run kind %q", status, hit.Kind) + } + runs = append(runs, hit.Run.ID) + } + if len(runs) != 1 || runs[0] != byStatus[status] { + t.Fatalf("status %q selected %v, want [%d]", status, runs, byStatus[status]) + } + } +} + +func TestSearchStatusAndDateSelectors(t *testing.T) { + ctx := context.Background() + store, err := Open(filepath.Join(t.TempDir(), "selector.db")) + if err != nil { + t.Fatal(err) + } + defer store.Close() + if _, err := store.Submit(ctx, "selectors", Template([]string{"one"}, true)); err != nil { + t.Fatal(err) + } + items, _ := store.List(ctx) + definition, _ := store.Get(ctx, items[0].ID, 1) + yesterdayRun, _, err := store.CreateRun(ctx, definition, "yesterday", "/work", false, nil, nil) + if err != nil { + t.Fatal(err) + } + todayRun, _, err := store.CreateRun(ctx, definition, "today", "/work", false, nil, nil) + if err != nil { + t.Fatal(err) + } + if err := store.FinishRun(ctx, yesterdayRun, "failed", "error", "boom", nil); err != nil { + t.Fatal(err) + } + setCreated(t, store, yesterdayRun, "2026-10-15 09:00:00") + setCreated(t, store, todayRun, "2026-10-16 09:00:00") + now := time.Date(2026, 10, 16, 12, 0, 0, 0, time.UTC) + + selects := func(query string) map[int64]bool { + t.Helper() + response, err := store.Search(ctx, protocol.SearchRequest{Query: query}, now) + if err != nil { + t.Fatal(err) + } + ids := map[int64]bool{} + for _, hit := range response.Results { + if hit.Run != nil { + ids[hit.Run.ID] = true + } + } + return ids + } + if ids := selects("failed"); !ids[yesterdayRun] || ids[todayRun] { + t.Fatalf("status selector: %+v", ids) + } + if ids := selects("today"); !ids[todayRun] || ids[yesterdayRun] { + t.Fatalf("today selector: %+v", ids) + } + if ids := selects("yesterday"); !ids[yesterdayRun] || ids[todayRun] { + t.Fatalf("yesterday selector: %+v", ids) + } + if ids := selects("2026-10-15"); !ids[yesterdayRun] || ids[todayRun] { + t.Fatalf("literal date selector: %+v", ids) + } + if ids := selects("this week"); !ids[yesterdayRun] || !ids[todayRun] { + t.Fatalf("this week selector: %+v", ids) + } + // Non-selector text stays free text. + if ids := selects("selectors"); !ids[yesterdayRun] || !ids[todayRun] { + t.Fatalf("free text was misread as a selector: %+v", ids) + } +} + +func TestParseSearchTerms(t *testing.T) { + now := time.Date(2026, 10, 16, 12, 0, 0, 0, time.UTC) + cases := []struct { + query string + status string + hasDate bool + from, to string + text string + }{ + {query: "failed", status: "failed"}, + {query: " RUNNING ", status: "running"}, + {query: "today", hasDate: true, from: "2026-10-16 00:00:00", to: "2026-10-17 00:00:00"}, + {query: "yesterday", hasDate: true, from: "2026-10-15 00:00:00", to: "2026-10-16 00:00:00"}, + {query: "this week", hasDate: true, from: "2026-10-12 00:00:00", to: "2026-10-19 00:00:00"}, + {query: "2026-10-05", hasDate: true, from: "2026-10-05 00:00:00", to: "2026-10-06 00:00:00"}, + {query: "failed to build", text: "failed to build"}, + {query: "build", text: "build"}, + } + for _, test := range cases { + terms := parseSearchTerms(test.query, now) + if terms.Status != test.status || terms.Text != test.text || terms.HasDate != test.hasDate { + t.Fatalf("parse %q = %+v", test.query, terms) + } + if test.hasDate { + if got := terms.From.Format(searchSQLTime); got != test.from { + t.Fatalf("parse %q from = %s, want %s", test.query, got, test.from) + } + if got := terms.To.Format(searchSQLTime); got != test.to { + t.Fatalf("parse %q to = %s, want %s", test.query, got, test.to) + } + } + } +} + +func TestSearchKindFilterAndLimits(t *testing.T) { + store, _ := searchFixture(t) + ctx := context.Background() + response, err := store.Search(ctx, protocol.SearchRequest{Query: "zephyr", Kind: protocol.SearchKindRun}, time.Now()) + if err != nil { + t.Fatal(err) + } + for _, hit := range response.Results { + if hit.Kind != protocol.SearchKindRun { + t.Fatalf("kind filter leaked %q", hit.Kind) + } + } + if _, err := store.Search(ctx, protocol.SearchRequest{Query: "zephyr", Kind: "bogus"}, time.Now()); err == nil { + t.Fatal("invalid kind must return a clear error") + } + for _, test := range []struct { + limit int + want int + }{{limit: 0, want: protocol.SearchDefaultLimit}, {limit: -5, want: protocol.SearchDefaultLimit}, {limit: 1, want: 1}, {limit: 500, want: protocol.SearchMaxLimit}} { + response, err := store.Search(ctx, protocol.SearchRequest{Query: "zephyr", Limit: test.limit}, time.Now()) + if err != nil { + t.Fatal(err) + } + if response.Cap != test.want { + t.Fatalf("limit %d => cap %d, want %d", test.limit, response.Cap, test.want) + } + if len(response.Results) > response.Cap { + t.Fatalf("limit %d returned %d results", test.limit, len(response.Results)) + } + } + // A cap below the candidate count drops matches and must report Truncated. + truncated, err := store.Search(ctx, protocol.SearchRequest{Query: "zephyr", Limit: 1}, time.Now()) + if err != nil { + t.Fatal(err) + } + if len(truncated.Results) != 1 || !truncated.Truncated { + t.Fatalf("result cap must truncate: %+v", truncated) + } +} + +func TestSearchEdgeCases(t *testing.T) { + store, run := searchFixture(t) + ctx := context.Background() + empty, err := store.Search(ctx, protocol.SearchRequest{Query: " "}, time.Now()) + if err != nil || len(empty.Results) != 0 { + t.Fatalf("empty query: %+v %v", empty, err) + } + none, err := store.Search(ctx, protocol.SearchRequest{Query: "no-such-token-xyzzy"}, time.Now()) + if err != nil || len(none.Results) != 0 { + t.Fatalf("no-match query: %+v %v", none, err) + } + if none.Results == nil { + t.Fatal("results must be an empty slice, not nil") + } + // A run without a finish reports no duration. + setFinishedNull(t, store, run) + response, err := store.Search(ctx, protocol.SearchRequest{Query: "zephyr", Kind: protocol.SearchKindRun}, time.Now()) + if err != nil { + t.Fatal(err) + } + for _, hit := range response.Results { + if hit.Run != nil && hit.Run.FinishedAt != "" { + t.Fatalf("null finished_at surfaced a value: %+v", hit.Run) + } + } +} + +func TestSearchSnippetBoundaries(t *testing.T) { + if got := searchSnippet("prefix café résumé naïve suffix", "naïve"); !strings.Contains(got, "naïve") || !utf8.ValidString(got) { + t.Fatalf("unicode snippet malformed: %q", got) + } + if got := searchSnippet("line one\n\tline two\r\n zephyr here", "zephyr"); strings.ContainsAny(got, "\n\r\t") || !strings.Contains(got, "zephyr") { + t.Fatalf("whitespace snippet not normalized: %q", got) + } + if got := searchSnippet(strings.Repeat("é", 200)+"needle"+strings.Repeat("é", 200), "needle"); !utf8.ValidString(got) || !strings.Contains(got, "needle") { + t.Fatalf("boundary snippet malformed: %q", got) + } + if got := searchSnippet("no match here", "absent"); got == "" || !utf8.ValidString(got) { + t.Fatalf("no-match snippet must stay a clean line: %q", got) + } +} + +func TestSearchScanBoundTruncates(t *testing.T) { + ctx := context.Background() + store, err := Open(filepath.Join(t.TempDir(), "scan.db")) + if err != nil { + t.Fatal(err) + } + defer store.Close() + if _, err := store.Submit(ctx, "scan", Template([]string{"one"}, true)); err != nil { + t.Fatal(err) + } + items, _ := store.List(ctx) + definition, _ := store.Get(ctx, items[0].ID, 1) + // An older step that does match, and a newer step whose body alone exceeds + // the scan budget: the budget must stop the scan and be reported. + older, _, err := store.CreateRun(ctx, definition, "scan-older", "/work", false, nil, nil) + if err != nil { + t.Fatal(err) + } + if err := store.BeginStep(ctx, older, 1, "one", nil); err != nil { + t.Fatal(err) + } + if err := store.FinishStep(ctx, older, protocol.RunStep{Sequence: 1, Status: "completed", Outcome: "success", Target: protocol.SuccessEnd, Output: map[string]any{"summary": "needle in older step"}}); err != nil { + t.Fatal(err) + } + newer, _, err := store.CreateRun(ctx, definition, "scan-newer", "/work", false, nil, nil) + if err != nil { + t.Fatal(err) + } + if err := store.BeginStep(ctx, newer, 1, "one", nil); err != nil { + t.Fatal(err) + } + blob := strings.Repeat("x", protocol.SearchMaxScanBytes+1024) + if err := store.FinishStep(ctx, newer, protocol.RunStep{Sequence: 1, Status: "completed", Outcome: "success", Target: protocol.SuccessEnd, Output: map[string]any{"blob": blob}}); err != nil { + t.Fatal(err) + } + response, err := store.Search(ctx, protocol.SearchRequest{Query: "needle"}, time.Now()) + if err != nil { + t.Fatal(err) + } + if !response.Truncated { + t.Fatalf("scan bound not reported: %+v", response) + } + for _, hit := range response.Results { + if hit.Field != nil { + t.Fatalf("scan passed the bound and matched: %+v", hit.Field) + } + } +} + +// TestSearchScanBoundTruncatesByRowCount proves the step-count bound is +// observable: more recorded steps than the budget sets Truncated even though +// every body is tiny and fits the byte budget. +func TestSearchScanBoundTruncatesByRowCount(t *testing.T) { + ctx := context.Background() + store, err := Open(filepath.Join(t.TempDir(), "scanrows.db")) + if err != nil { + t.Fatal(err) + } + defer store.Close() + if _, err := store.Submit(ctx, "scanrows", Template([]string{"one"}, true)); err != nil { + t.Fatal(err) + } + items, _ := store.List(ctx) + definition, _ := store.Get(ctx, items[0].ID, 1) + run, _, err := store.CreateRun(ctx, definition, "scanrows", "/work", false, nil, nil) + if err != nil { + t.Fatal(err) + } + tx, err := store.db.Begin() + if err != nil { + t.Fatal(err) + } + for sequence := 1; sequence <= protocol.SearchMaxScanSteps+1; sequence++ { + if _, err := tx.Exec(`INSERT INTO run_steps(run_id,sequence,stage_key,input_json) VALUES(?,?,?,?)`, run, sequence, "one", "{}"); err != nil { + t.Fatal(err) + } + } + if err := tx.Commit(); err != nil { + t.Fatal(err) + } + response, err := store.Search(ctx, protocol.SearchRequest{Query: "needle"}, time.Now()) + if err != nil { + t.Fatal(err) + } + if !response.Truncated { + t.Fatalf("step-count bound not reported: scanned_steps=%d total=%d", response.ScannedSteps, protocol.SearchMaxScanSteps+1) + } + if response.ScannedSteps != protocol.SearchMaxScanSteps { + t.Fatalf("scanned_steps=%d, want %d", response.ScannedSteps, protocol.SearchMaxScanSteps) + } + if response.ScannedBytes > protocol.SearchMaxScanBytes { + t.Fatalf("scanned_bytes=%d exceeds bound %d", response.ScannedBytes, protocol.SearchMaxScanBytes) + } +} + +func setCreated(t *testing.T, store *Store, runID int64, timestamp string) { + t.Helper() + if _, err := store.db.Exec("UPDATE pipeline_runs SET created_at=? WHERE id=?", timestamp, runID); err != nil { + t.Fatal(err) + } +} + +func setFinishedNull(t *testing.T, store *Store, runID int64) { + t.Helper() + if _, err := store.db.Exec("UPDATE pipeline_runs SET finished_at=NULL,status='running' WHERE id=?", runID); err != nil { + t.Fatal(err) + } +} diff --git a/internal/pipeline/store.go b/internal/pipeline/store.go index afdde185..ea69fb39 100644 --- a/internal/pipeline/store.go +++ b/internal/pipeline/store.go @@ -8,6 +8,7 @@ import ( "fmt" "git.a-wan.net/alan/pipelines/internal/protocol" "git.a-wan.net/alan/pipelines/internal/stage" + "os" "path/filepath" "strings" "sync" @@ -50,6 +51,26 @@ CREATE TABLE IF NOT EXISTS submissions (id INTEGER PRIMARY KEY, name TEXT NOT NU func (s *Store) Close() error { return s.db.Close() } +// OpenReadOnly opens an existing database without running schema DDL or +// changing journal mode, so a live database can be queried read-only while its +// server keeps writing. It fails when the file does not exist. It is used by +// tests that inspect a real database and is never part of server startup. +func OpenReadOnly(path string) (*Store, error) { + if _, err := os.Stat(path); err != nil { + return nil, err + } + db, err := sql.Open("sqlite", "file:"+path+"?mode=ro") + if err != nil { + return nil, err + } + db.SetMaxOpenConns(1) + if _, err := db.Exec("PRAGMA busy_timeout=5000"); err != nil { + db.Close() + return nil, err + } + return &Store{db: db, dir: filepath.Dir(path)}, nil +} + // Directory is the resolved directory holding the database, used to place // adjacent server state such as stage artifacts. func (s *Store) Directory() string { return s.dir } diff --git a/internal/protocol/message.go b/internal/protocol/message.go index 7b63b470..0a945f49 100644 --- a/internal/protocol/message.go +++ b/internal/protocol/message.go @@ -133,6 +133,11 @@ type Message struct { WritePaths []string `json:"write_paths,omitempty"` RequestKey string `json:"request_key,omitempty"` Workspace string `json:"workspace,omitempty"` + // Search is the broad, read-only search traffic over pipelines, versions, + // runs, steps, field text and artifacts. See search.go for the per-type + // payloads and the ranking/kind/bound vocabulary. + Search *SearchRequest `json:"search,omitempty"` + SearchResults *SearchResponse `json:"search_results,omitempty"` // Chat is the brokered OpenCode chat traffic (route B). See chat.go for the // per-type payload structs and the wire shapes they mirror. ChatSessions []ChatSession `json:"chat_sessions,omitempty"` diff --git a/internal/protocol/search.go b/internal/protocol/search.go new file mode 100644 index 00000000..a25586b1 --- /dev/null +++ b/internal/protocol/search.go @@ -0,0 +1,158 @@ +package protocol + +// Search brokered by the Pipelines server: one query can find a pipeline, a +// pipeline version, a run, a recorded step/visit, the text of a step field +// (input/output/evidence/transcript) or a recorded artifact. A client sends +// Message{Type: Search, ID: , Search: } and the server +// echoes the ID with a SearchResults response, matching every other exchange. +const ( + Search = "search" + SearchResults = "search_results" +) + +// Search.Kind narrowing values. They are the only narrowing mechanism and are +// enforced server-side; an empty Kind searches every kind. +const ( + SearchKindPipeline = "pipeline" + SearchKindVersion = "version" + SearchKindRun = "run" + SearchKindStep = "step" + SearchKindField = "field" + SearchKindArtifact = "artifact" +) + +// SearchKinds is the accepted Kind vocabulary in display order; the CLI help +// and notes are kept in step with it. +var SearchKinds = []string{ + SearchKindPipeline, + SearchKindVersion, + SearchKindRun, + SearchKindStep, + SearchKindField, + SearchKindArtifact, +} + +// SearchStatuses is the accepted set of bare status words a query may name to +// select runs. It mirrors the run status vocabulary; a bare word selects runs +// rather than matching as free text. +var SearchStatuses = []string{"queued", "running", "succeeded", "failed", "canceled", "interrupted"} + +// Search budget and cap constants. They are fixed, enforced server-side and +// reported on SearchResponse so a client can see the bound that applied. +const ( + // SearchDefaultLimit is the result cap when a request omits Limit (or sends 0). + SearchDefaultLimit = 50 + // SearchMaxLimit is the largest accepted result cap; larger requests clamp. + SearchMaxLimit = 200 + // SearchMaxScanSteps bounds how many recorded steps one query inspects while + // scanning input/output/transcript field text. + SearchMaxScanSteps = 2000 + // SearchMaxScanBytes bounds the total field bytes (input_json + output_json + + // evidence_json + transcript) one query decodes and searches before it stops + // and reports Truncated. + SearchMaxScanBytes = 1 << 20 +) + +// SearchRequest is the search request body. Query is the free text, a bare +// status word or a date expression; Kind optionally narrows to one SearchKind; +// Limit caps the returned results (0 = server default). +type SearchRequest struct { + Query string `json:"query"` + Kind string `json:"kind,omitempty"` + Limit int `json:"limit,omitempty"` +} + +// SearchResponse is the ranked, bounded set of matches. Results are ordered by +// score, then recency, then a stable kind/target tie-break. Truncated reports +// that either the result cap or the field-scan budget dropped matches; +// ScannedSteps/ScannedBytes and ScanBoundBytes make the body-scan bound +// observable. +type SearchResponse struct { + Query string `json:"query"` + Kind string `json:"kind,omitempty"` + Results []SearchResult `json:"results"` + Truncated bool `json:"truncated,omitempty"` + ScannedSteps int `json:"scanned_steps"` + ScannedBytes int `json:"scanned_bytes"` + Cap int `json:"cap"` + ScanBoundBytes int `json:"scan_bound_bytes"` +} + +// SearchResult is one typed, navigable match. Kind selects which payload is +// set; Target is a stable token an existing CLI command can resolve +// (pipeline:, pipeline::v, run:, run::, +// run:::, or an artifact URL). Score and MatchedField +// expose the ranking tier and why the hit matched so a client can re-rank; +// Recency is the hit's sortable UTC timestamp. +type SearchResult struct { + Kind string `json:"kind"` + Target string `json:"target"` + Score int `json:"score"` + MatchedField string `json:"matched_field"` + Recency string `json:"recency,omitempty"` + Pipeline *SearchPipeline `json:"pipeline,omitempty"` + Version *SearchVersion `json:"version,omitempty"` + Run *SearchRun `json:"run,omitempty"` + Step *SearchStep `json:"step,omitempty"` + Field *SearchField `json:"field,omitempty"` + Artifact *SearchArtifact `json:"artifact,omitempty"` +} + +// SearchPipeline is a pipeline name/identity hit: its latest version and stage count. +type SearchPipeline struct { + ID int64 `json:"id"` + Name string `json:"name"` + Version int `json:"version"` + Stages int `json:"stages"` +} + +// SearchVersion is a pipeline version/source hit. +type SearchVersion struct { + PipelineID int64 `json:"pipeline_id"` + Version int `json:"version"` + ContentHash string `json:"content_hash"` + CreatedAt string `json:"created_at"` +} + +// SearchRun is a run identity/metadata hit. FinishedAt and DurationSeconds are +// omitted for a run without a recorded finish. +type SearchRun struct { + ID int64 `json:"id"` + PipelineName string `json:"pipeline_name"` + Status string `json:"status"` + Workspace string `json:"workspace,omitempty"` + CreatedAt string `json:"created_at"` + FinishedAt string `json:"finished_at,omitempty"` + DurationSeconds float64 `json:"duration_seconds,omitempty"` +} + +// SearchStep is a recorded step/visit hit. +type SearchStep struct { + RunID int64 `json:"run_id"` + Sequence int `json:"sequence"` + Stage string `json:"stage"` + Visit int `json:"visit,omitempty"` + Outcome string `json:"outcome,omitempty"` + Target string `json:"target,omitempty"` + Diagnostic string `json:"diagnostic,omitempty"` +} + +// SearchField is a stage input/output/evidence/transcript text hit with a +// snippet around the first match. +type SearchField struct { + RunID int64 `json:"run_id"` + Sequence int `json:"sequence"` + Field string `json:"field"` + Snippet string `json:"snippet"` +} + +// SearchArtifact is a recorded artifact hit; contents are never read, only the +// recorded metadata. +type SearchArtifact struct { + RunID int64 `json:"run_id"` + Sequence int `json:"sequence"` + Name string `json:"name"` + ContentType string `json:"content_type"` + SizeBytes int64 `json:"size_bytes"` + URL string `json:"url"` +} diff --git a/internal/protocol/search_test.go b/internal/protocol/search_test.go new file mode 100644 index 00000000..7a8d8311 --- /dev/null +++ b/internal/protocol/search_test.go @@ -0,0 +1,82 @@ +package protocol + +import ( + "encoding/json" + "strings" + "testing" +) + +func TestSearchRequestResponseRoundTrip(t *testing.T) { + request := Message{Type: Search, ID: "req-1", Search: &SearchRequest{Query: "release", Kind: SearchKindRun, Limit: 5}} + encoded, err := json.Marshal(request) + if err != nil { + t.Fatal(err) + } + var decoded Message + if err := json.Unmarshal(encoded, &decoded); err != nil { + t.Fatal(err) + } + if decoded.Type != Search || decoded.ID != "req-1" || decoded.Search == nil { + t.Fatalf("request envelope lost: %+v", decoded) + } + if decoded.Search.Query != "release" || decoded.Search.Kind != SearchKindRun || decoded.Search.Limit != 5 { + t.Fatalf("search request lost: %+v", decoded.Search) + } + + response := Message{Type: SearchResults, ID: "req-1", SearchResults: &SearchResponse{ + Query: "release", + Results: []SearchResult{{Kind: SearchKindPipeline, Target: "pipeline:1", Score: 3, MatchedField: "name", Pipeline: &SearchPipeline{ID: 1, Name: "release", Version: 2, Stages: 3}}}, + Cap: 50, + ScanBoundBytes: SearchMaxScanBytes, + }} + encoded, err = json.Marshal(response) + if err != nil { + t.Fatal(err) + } + var echoed Message + if err := json.Unmarshal(encoded, &echoed); err != nil { + t.Fatal(err) + } + if echoed.ID != "req-1" || echoed.Type != SearchResults || echoed.SearchResults == nil { + t.Fatalf("response envelope or ID echo lost: %+v", echoed) + } + got := echoed.SearchResults + if got.Query != "release" || got.Cap != 50 || got.ScanBoundBytes != SearchMaxScanBytes || len(got.Results) != 1 { + t.Fatalf("search response lost: %+v", got) + } + hit := got.Results[0] + if hit.Pipeline == nil || hit.Pipeline.Name != "release" || hit.Pipeline.Version != 2 || hit.Pipeline.Stages != 3 || hit.Score != 3 || hit.MatchedField != "name" || hit.Target != "pipeline:1" { + t.Fatalf("pipeline payload lost: %+v", hit) + } +} + +func TestSearchResultOmitsAbsentPayloadsAndEmptyKind(t *testing.T) { + encoded, err := json.Marshal(SearchResult{Kind: SearchKindRun, Target: "run:1", Score: 1, MatchedField: "status"}) + if err != nil { + t.Fatal(err) + } + for _, key := range []string{"pipeline", "version", "run", "step", "field", "artifact", "recency"} { + if strings.Contains(string(encoded), `"`+key+`":`) { + t.Fatalf("absent %s payload must be omitted: %s", key, encoded) + } + } + var decoded SearchResult + if err := json.Unmarshal([]byte(`{"kind":"run","target":"run:1","score":1,"matched_field":"status"}`), &decoded); err != nil { + t.Fatal(err) + } + if decoded.Pipeline != nil || decoded.Version != nil || decoded.Run != nil || decoded.Step != nil || decoded.Field != nil || decoded.Artifact != nil { + t.Fatalf("absent payloads must decode nil: %+v", decoded) + } + + response := Message{Type: SearchResults, ID: "x", SearchResults: &SearchResponse{Query: "q", Results: []SearchResult{}}} + encoded, err = json.Marshal(response) + if err != nil { + t.Fatal(err) + } + if strings.Contains(string(encoded), `"kind"`) { + t.Fatalf("empty response kind must be omitted: %s", encoded) + } + if !strings.Contains(string(encoded), `"results":[]`) { + t.Fatalf("results must always be present: %s", encoded) + } +} diff --git a/internal/server/search_test.go b/internal/server/search_test.go new file mode 100644 index 00000000..0b4a38b3 --- /dev/null +++ b/internal/server/search_test.go @@ -0,0 +1,105 @@ +package server + +import ( + "bytes" + "context" + "encoding/json" + "net/http/httptest" + "path/filepath" + "strconv" + "strings" + "testing" + + "git.a-wan.net/alan/pipelines/internal/cli" + "git.a-wan.net/alan/pipelines/internal/pipeline" + "git.a-wan.net/alan/pipelines/internal/protocol" +) + +// TestCLISearchEndToEndEveryKind seeds a store whose pipeline, source, run, +// stage, field text and artifact all contain one token, then runs the CLI +// search command over a real Handler and asserts the server returned one match +// of every kind and rejects an unknown --type rather than ignoring it. +func TestCLISearchEndToEndEveryKind(t *testing.T) { + t.Setenv("PIPELINES_HARNESS_CONFIG", filepath.Join(t.TempDir(), "harnesses.json")) + ctx, cancel := context.WithCancel(context.Background()) + store, err := pipeline.Open(filepath.Join(t.TempDir(), "search-e2e.db")) + if err != nil { + t.Fatal(err) + } + defer store.Close() + if _, err := store.Submit(ctx, "Zephyr Build", pipeline.Template([]string{"zephyr"}, true)); err != nil { + t.Fatal(err) + } + items, err := store.List(ctx) + if err != nil { + t.Fatal(err) + } + definition, err := store.Get(ctx, items[0].ID, 1) + if err != nil { + t.Fatal(err) + } + run, _, err := store.CreateRun(ctx, definition, "zephyr-key", "/work/zephyr", false, nil, map[string]any{"prompt": "zephyr task"}) + if err != nil { + t.Fatal(err) + } + if err := store.BeginStep(ctx, run, 1, "zephyr", map[string]any{"prompt": "zephyr task"}); err != nil { + t.Fatal(err) + } + if err := store.FinishStep(ctx, run, protocol.RunStep{ + Sequence: 1, Status: "completed", Outcome: "success", Target: protocol.SuccessEnd, + Output: map[string]any{"summary": "zephyr completed"}, + Artifacts: []protocol.StageArtifact{{Name: "zephyr.txt", SizeBytes: 12, ContentType: "text/plain", URL: "/artifacts/" + strconv.FormatInt(run, 10) + "/1/zephyr.txt"}}, + }); err != nil { + t.Fatal(err) + } + handler := Handler(ctx, store) + defer func() { cancel(); WaitRuns(handler) }() + http := httptest.NewServer(handler) + defer http.Close() + url := "ws" + strings.TrimPrefix(http.URL, "http") + "/connect" + + runCLI := func(args ...string) (string, error) { + var out bytes.Buffer + app := cli.App{Out: &out, Err: &out, In: strings.NewReader("")} + err := app.Run(ctx, append([]string{"--url", url}, args...)) + return out.String(), err + } + output, err := runCLI("search", "zephyr", "--json") + if err != nil { + t.Fatal(err) + } + var response protocol.SearchResponse + if err := json.Unmarshal([]byte(output), &response); err != nil { + t.Fatal(err, output) + } + kinds := map[string]bool{} + for _, hit := range response.Results { + kinds[hit.Kind] = true + if hit.Target == "" || hit.MatchedField == "" || hit.Score == 0 { + t.Fatalf("hit lacks ranking/navigation context: %+v", hit) + } + } + for _, kind := range protocol.SearchKinds { + if !kinds[kind] { + t.Fatalf("kind %q missing from end-to-end search: %s", kind, output) + } + } + + if output, err := runCLI("search", "zephyr", "--type", "run", "--json"); err != nil { + t.Fatal(err) + } else { + var filtered protocol.SearchResponse + if err := json.Unmarshal([]byte(output), &filtered); err != nil { + t.Fatal(err) + } + for _, hit := range filtered.Results { + if hit.Kind != protocol.SearchKindRun { + t.Fatalf("kind filter leaked %q", hit.Kind) + } + } + } + + if _, err := runCLI("search", "zephyr", "--type", "bogus"); err == nil || !strings.Contains(err.Error(), "unknown search kind") { + t.Fatalf("invalid kind must error, got %v", err) + } +} diff --git a/internal/server/websocket.go b/internal/server/websocket.go index e311d376..94c99eed 100644 --- a/internal/server/websocket.go +++ b/internal/server/websocket.go @@ -192,6 +192,22 @@ func HandlerWithOptions(ctx context.Context, options Options, stores ...*pipelin } else { response.Pipelines = items } + } else if request.Type == protocol.Search { + if len(stores) == 0 { + response.Type = protocol.Error + response.Error = "search unavailable: no store" + } else if request.Search == nil { + response.Type = protocol.Error + response.Error = "search requires a query" + } else { + response.Type = protocol.SearchResults + var err error + response.SearchResults, err = stores[0].Search(ctx, *request.Search, time.Now()) + if err != nil { + response.Type = protocol.Error + response.Error = err.Error() + } + } } else if runs != nil && (request.Type == protocol.RunPipeline || request.Type == protocol.RetryRun || request.Type == protocol.ListRuns || request.Type == protocol.GetRun || request.Type == protocol.CancelRun || request.Type == protocol.GetRunEvidence) { var err error switch request.Type {