Add cross-system read-only search
One query now finds anything in the system over the wire and from the CLI. - protocol: add Search/SearchResults message types and a SearchRequest (query, optional kind filter, optional limit) / SearchResponse (typed, ranked matches plus truncation and scan-bound counters) pair, with typed payloads for pipeline, version, run, step/visit, field text and artifact. - store: add Store.Search, a read-only query over pipelines, pipeline_versions, pipeline_runs, run_steps (+step_control) and step_artifacts. Ranks exact name (3) > name substring (2) > body text (1), breaks ties by recency then a stable kind/target order, and reports the matched field. Bounds work with a 50-default/200-max result cap, a 2000-row per-kind scan limit and a 2000-step / 1 MiB field-text scan budget. - terms: a bare status word or lone date expression (today, yesterday, this week, YYYY-MM-DD) selects runs; anything else stays free text. - server: dispatch protocol.Search to stores[0].Search; read-only and unconfirmed. - cli: add `pipelines search QUERY [--type KIND] [--limit N] [--json]`, help text and a BROWSE line. - store: add OpenReadOnly so the live-database check never runs DDL. - docs: document the grammar, kinds, ranking and bounds in docs/cli.md, scope the TUI omnisearch to pipelines in docs/ui-ux-review.md, and add a search bullet to the packaged operations reference. - tests: protocol round trip, store/ranking/term-parsing/limit/scan-bound unit tests, a server end-to-end test, and an env-guarded live check. No new dependencies; go.mod/go.sum unchanged.
This commit is contained in:
+42
-4
@@ -61,6 +61,42 @@ ntfy hosts are not misrepresented as local. Publishing URLs and credentials are
|
||||
never returned. Missing public phone configuration is labeled explicitly.
|
||||
`pipelines service status` remains the separate OS autostart status command.
|
||||
|
||||
## Search
|
||||
|
||||
`pipelines search QUERY [--type KIND] [--limit N] [--json]` runs one read-only
|
||||
query across the whole system: pipeline names and their versions, run identity,
|
||||
status and dates, stage and visit identifiers, artifact names and types, and the
|
||||
text of stage inputs, outputs, evidence and transcripts. It never reads
|
||||
artifact contents, never makes a per-result follow-up request, and asks for no
|
||||
confirmation.
|
||||
|
||||
- Free text matches case-insensitively. A bare status word (`queued`, `running`,
|
||||
`succeeded`, `failed`, `canceled`, `interrupted`) selects those runs, and a lone
|
||||
date expression (`today`, `yesterday`, `this week` — Monday 00:00 local — or a
|
||||
literal `YYYY-MM-DD`) selects the runs created in that window; any other text,
|
||||
including a status word inside a sentence, stays free text.
|
||||
- `--type` is the only narrowing mechanism and takes one kind: `pipeline`,
|
||||
`version`, `run`, `step`, `field` or `artifact`. An unknown kind is an error,
|
||||
never a silently ignored filter.
|
||||
- Ranking: an exact name match (score 3) beats a name substring (2) beats a
|
||||
body-text match (1); equal tiers break by recency, then a stable kind/target
|
||||
order. Every result carries `score`, `matched_field` and `recency` so a client
|
||||
can re-rank.
|
||||
- Bounds: the result cap defaults to 50 and clamps to 200 (`0` = default);
|
||||
every kind scans at most 2000 rows, and field-text scanning stops at 2000 steps
|
||||
or 1 MiB decoded. `truncated` plus `scanned_steps`, `scanned_bytes`, `cap` and
|
||||
`scan_bound_bytes` report when a bound applied.
|
||||
- Targets are stable and navigable: `pipeline:<id>`, `pipeline:<id>:v<version>`,
|
||||
`run:<id>`, `run:<id>:<sequence>`, `run:<id>:<sequence>:<field>`, or an
|
||||
artifact URL served by the artifact endpoint.
|
||||
|
||||
```sh
|
||||
pipelines search "release notes" --json
|
||||
pipelines search failed --json
|
||||
pipelines search 2026-10-16 --type run --json
|
||||
pipelines search "acceptance check" --type field --limit 10
|
||||
```
|
||||
|
||||
## Draft-first harness workflow
|
||||
|
||||
```sh
|
||||
@@ -194,6 +230,7 @@ Submitting activates included schedules, so review them before publishing.
|
||||
| TUI action | Agent-capable CLI |
|
||||
| --- | --- |
|
||||
| Browse/search pipelines | `list --json`, `show NAME --json` |
|
||||
| Search everything | `search QUERY --json` (see [Search](#search)) |
|
||||
| Original source and history | `export NAME --version N`, `versions NAME --json` |
|
||||
| Browse templates/source | `templates --json`, `template KEY --source` |
|
||||
| Create or edit a definition | `new`, `edit`, `drafts` |
|
||||
@@ -217,10 +254,11 @@ printf '%s' '{"type":"get_pipeline","pipeline_id":1,"version":1}' | pipelines rp
|
||||
|
||||
Existing protocol commands: `ping`, `list_pipelines`, `get_pipeline`,
|
||||
`list_templates`, `validate`, `submit`, `run_pipeline`, `list_runs`, `get_run`,
|
||||
`cancel_run`, `retry_run`, `get_status`. `list_runs` also returns all-history `usage`
|
||||
aggregates. See `internal/protocol/message.go` for JSON envelopes. Mutating RPCs
|
||||
have real effects; they are not previews. TUI-only animation, scrolling, and
|
||||
transient client logs are presentation, not separate server capabilities.
|
||||
`cancel_run`, `retry_run`, `get_status`, `search`. `list_runs` also returns
|
||||
all-history `usage` aggregates. See `internal/protocol/message.go` for JSON
|
||||
envelopes. Mutating RPCs have real effects; they are not previews. TUI-only
|
||||
animation, scrolling, and transient client logs are presentation, not separate
|
||||
server capabilities.
|
||||
|
||||
## Scripted runs
|
||||
|
||||
|
||||
@@ -38,7 +38,7 @@ and external editor/OpenCode behavior were not assessed by that review.
|
||||
| Status | Narrow cards hide addresses; scrolling does not reveal them. | Wrap card contents and scroll the actual service-card canvas. |
|
||||
| Task form | Tool lists consume prompt space; wrong footer. | Larger task area, compact scope summary, correct submit/cancel controls. |
|
||||
| CLI pipeline picker | Background Home controls remain visible. | Picker-specific filter/select/cancel controls. |
|
||||
| Search | “Omnisearch” promises broader scope than pipeline search. | “Search pipelines” and specific open/cancel instructions. |
|
||||
| Search | “Omnisearch” promises broader scope than pipeline search. | Keep the TUI omnisearch pipeline-scoped and labelled “Search pipelines” with open/cancel instructions; the broad cross-kind search is the server-backed `pipelines search QUERY [--type KIND]` command, whose grammar, kinds, ranking and bounds are documented in [CLI and agent authoring](cli.md#search). |
|
||||
| Commands | No descriptions or indication of hidden entries. | Selected-command explanation and visible range/count. |
|
||||
| Versions | Selected/latest markers truncate on narrow screens. | Correct view-version footer; further metadata prioritization remains a follow-up. |
|
||||
| Naming/publication | Naming inherits publication shortcuts. | Input-specific name-saving/retention footer. |
|
||||
|
||||
@@ -71,6 +71,21 @@ delete databases, sessions, volumes or credentials to repair a workflow.
|
||||
Use the authoritative `CLI_PREFIX` from the prompt:
|
||||
|
||||
- `CLI_PREFIX runs --json`: run history (optional `--pipeline NAME` scope).
|
||||
- `CLI_PREFIX search QUERY [--type KIND] [--limit N] --json`: one read-only query
|
||||
across pipelines, versions, runs, steps, field text and artifacts. QUERY is free
|
||||
text, a bare status word (`queued`/`running`/`succeeded`/`failed`/`canceled`/
|
||||
`interrupted`) selecting those runs, or a lone date expression (`today`,
|
||||
`yesterday`, `this week` — Monday 00:00 local — or `YYYY-MM-DD`) selecting the
|
||||
runs created in that window; anything else stays free text. `--type` is the only
|
||||
narrowing mechanism and takes one of `pipeline`, `version`, `run`, `step`,
|
||||
`field`, `artifact`; an unknown kind is an error. Ranking is exact name (3) >
|
||||
name substring (2) > body text (1), equal tiers by recency, and every hit
|
||||
reports `score`, `matched_field` and `recency`. The cap defaults to 50 and clamps
|
||||
to 200; every kind scans at most 2000 rows and field scanning stops at 2000
|
||||
steps / 1 MiB and reports
|
||||
`truncated`/`scanned_steps`/`scanned_bytes`/`cap`/`scan_bound_bytes`. Targets are
|
||||
stable: `pipeline:<id>`, `pipeline:<id>:v<version>`, `run:<id>`,
|
||||
`run:<id>:<sequence>`, `run:<id>:<sequence>:<field>`, or an artifact URL.
|
||||
- `CLI_PREFIX inspect RUN_ID --json`: selected run, visits and retained evidence.
|
||||
Exact stage outputs are authoritative; a transcript over the 32 KiB retained
|
||||
bound is a valid, credential-redacted preview envelope (with preserved stage
|
||||
|
||||
+50
-1
@@ -37,9 +37,10 @@ type options struct {
|
||||
url, name, prompt, template, workspace, paths, key, opencode, model string
|
||||
output string
|
||||
conflict string
|
||||
kind string
|
||||
all bool
|
||||
json, source, noOpen, yes, allow, createOnly, force, wait bool
|
||||
version, expected int
|
||||
version, expected, limit int
|
||||
timeout, waitTimeout time.Duration
|
||||
}
|
||||
|
||||
@@ -99,6 +100,9 @@ func (a App) flags(command string) (*flag.FlagSet, *options) {
|
||||
f.DurationVar(&o.waitTimeout, "wait-timeout", 0, "Client limit for --wait after the run is admitted (0 waits until terminal); separate from per-request --timeout")
|
||||
case "runs":
|
||||
f.StringVar(&o.name, "pipeline", "", "Filter by exact pipeline name or ID")
|
||||
case "search":
|
||||
f.StringVar(&o.kind, "type", "", "Narrow to one kind: "+strings.Join(protocol.SearchKinds, ", "))
|
||||
f.IntVar(&o.limit, "limit", 0, "Maximum results (default 50, max 200)")
|
||||
case "retry":
|
||||
f.BoolVar(&o.yes, "yes", false, "Confirm retry without an interactive prompt")
|
||||
f.StringVar(&o.key, "request-key", "", "Idempotency key (default: generated unique key)")
|
||||
@@ -697,6 +701,31 @@ func (a App) Run(ctx context.Context, args []string) error {
|
||||
return err
|
||||
case "import":
|
||||
return a.importBundle(ctx, o, f.Args(), call)
|
||||
case "search":
|
||||
query := strings.Join(pos, " ")
|
||||
if strings.TrimSpace(query) == "" {
|
||||
return fmt.Errorf("search requires a query; use pipelines help search")
|
||||
}
|
||||
r, err := call(protocol.Message{Type: protocol.Search, Search: &protocol.SearchRequest{Query: query, Kind: o.kind, Limit: o.limit}})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if r.SearchResults == nil {
|
||||
return fmt.Errorf("server did not return search results; update pipelines-server")
|
||||
}
|
||||
if o.json {
|
||||
return a.output(r.SearchResults)
|
||||
}
|
||||
if len(r.SearchResults.Results) == 0 {
|
||||
fmt.Fprintln(a.Out, "No matches")
|
||||
}
|
||||
for _, hit := range r.SearchResults.Results {
|
||||
fmt.Fprintf(a.Out, "%s\t%s\t%d\t%s\t%s\n", hit.Kind, hit.Target, hit.Score, hit.MatchedField, searchLabel(hit))
|
||||
}
|
||||
if r.SearchResults.Truncated {
|
||||
fmt.Fprintf(a.Err, "Results truncated (cap %d, scan bound %d bytes)\n", r.SearchResults.Cap, r.SearchResults.ScanBoundBytes)
|
||||
}
|
||||
return nil
|
||||
case "show", "versions":
|
||||
value, err := one()
|
||||
if err != nil {
|
||||
@@ -819,6 +848,26 @@ func (a App) output(value any) error {
|
||||
return encoder.Encode(value)
|
||||
}
|
||||
|
||||
// searchLabel renders one search hit's compact human line. Each kind exposes a
|
||||
// different typed payload; the target stays the navigable identity.
|
||||
func searchLabel(hit protocol.SearchResult) string {
|
||||
switch {
|
||||
case hit.Pipeline != nil:
|
||||
return fmt.Sprintf("%s · v%d · %d stages", hit.Pipeline.Name, hit.Pipeline.Version, hit.Pipeline.Stages)
|
||||
case hit.Version != nil:
|
||||
return fmt.Sprintf("pipeline %d · v%d · %s", hit.Version.PipelineID, hit.Version.Version, hit.Version.ContentHash)
|
||||
case hit.Run != nil:
|
||||
return fmt.Sprintf("run %d · %s · %s", hit.Run.ID, hit.Run.PipelineName, hit.Run.Status)
|
||||
case hit.Step != nil:
|
||||
return fmt.Sprintf("run %d · step %d · %s · %s", hit.Step.RunID, hit.Step.Sequence, hit.Step.Stage, hit.Step.Outcome)
|
||||
case hit.Field != nil:
|
||||
return fmt.Sprintf("run %d · step %d · %s: %s", hit.Field.RunID, hit.Field.Sequence, hit.Field.Field, hit.Field.Snippet)
|
||||
case hit.Artifact != nil:
|
||||
return fmt.Sprintf("run %d · step %d · %s · %s", hit.Artifact.RunID, hit.Artifact.Sequence, hit.Artifact.Name, hit.Artifact.ContentType)
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
type callFunc func(protocol.Message) (protocol.Message, error)
|
||||
type resolveFunc func(string) (protocol.PipelineSummary, error)
|
||||
|
||||
|
||||
@@ -10,6 +10,7 @@ var usage = map[string]string{
|
||||
"validate": "validate <draft.md> [--json]",
|
||||
"submit": "submit <draft.md> [--name NAME] [--create-only] [--yes] [--json]",
|
||||
"list": "list [--json]",
|
||||
"search": "search QUERY [--type KIND] [--limit N] [--json]",
|
||||
"show": "show <name|id> [--version N] [--json]",
|
||||
"versions": "versions <name|id> [--json]",
|
||||
"export": "export [name|id ...] [--output FILE.zip] [--all] [--version N]",
|
||||
@@ -44,6 +45,7 @@ var notes = map[string]string{
|
||||
"retry": "Start a new attempt of a failed run with the same initial input, directory, pipeline version, and edit scope. Original history is retained. Already-completed side effects are not undone. --wait/--wait-timeout behave as for run: a reached wait limit leaves the new run active; inspect read-only with `inspect <run-id> --json`, or reattach with --wait and the original --request-key instead of starting a duplicate retry.",
|
||||
"debug": "Open a fresh configured harness conversation to investigate a run using inspect/show/runs CLI queries. --harness opencode|pi|nvim chooses another configured harness or read-only nvim packet. Does not resume implementation or retry. --json returns the prompt without launching anything.",
|
||||
"rpc": "Accepts a JSON protocol envelope; '-' reads stdin. Every TUI server operation is available here too. Mutating commands are not dry runs.",
|
||||
"search": "One read-only query over pipelines, versions, runs, steps, field text and artifacts. QUERY is free text, a bare status word (queued, running, succeeded, failed, canceled, interrupted) selecting those runs, or a lone date expression (today, yesterday, this week [Monday 00:00 local], or YYYY-MM-DD) selecting the runs created in that window. --type is the only narrowing mechanism and takes one kind: pipeline, version, run, step, field or artifact; an unknown kind is an error. Ranking: an exact name match (3) beats a name substring (2) beats a body-text match (1); equal tiers break by recency, then a stable kind/target tie-break, and every hit reports score, matched_field and recency so a client can re-rank. Bounds: the result cap defaults to 50 and clamps to 200 (0 = default); every kind scans at most 2000 rows and field-text scanning is additionally capped at 2000 steps / 1 MiB; Truncated plus scanned_steps, scanned_bytes and scan_bound_bytes report when a bound applied. Targets are stable: pipeline:<id>, pipeline:<id>:v<version>, run:<id>, run:<id>:<sequence>, run:<id>:<sequence>:<field>, or an artifact URL.",
|
||||
}
|
||||
|
||||
var serviceHelp = map[string]string{
|
||||
|
||||
@@ -13,6 +13,7 @@ AUTHOR
|
||||
|
||||
BROWSE
|
||||
list List registered pipelines
|
||||
search QUERY Search pipelines, versions, runs, steps, field text, artifacts
|
||||
show <name|id> Inspect graph, settings, source and history
|
||||
versions <name|id> List immutable versions
|
||||
export [name|id ...] Select/export definitions as ZIP; one name prints Markdown
|
||||
|
||||
@@ -107,11 +107,19 @@ func (s *Store) CreateRun(ctx context.Context, d *protocol.PipelineDetail, key,
|
||||
return id, true, nil
|
||||
}
|
||||
|
||||
// runFrom is the shared pipeline_runs/pipelines join every run-row read builds
|
||||
// on, so the run projections cannot drift.
|
||||
const runFrom = `FROM pipeline_runs r JOIN pipelines p ON p.id=r.pipeline_id`
|
||||
|
||||
// runStamps formats a run row's created/finished timestamps; reused by runSelect
|
||||
// and searchRuns instead of copied per query.
|
||||
const runStamps = `strftime('%Y-%m-%dT%H:%M:%SZ',r.created_at),COALESCE(strftime('%Y-%m-%dT%H:%M:%SZ',r.finished_at),'')`
|
||||
|
||||
const runSelect = `SELECT r.id,r.pipeline_id,p.name,r.version,v.content_hash,r.status,r.outcome,r.current_stage,
|
||||
strftime('%Y-%m-%dT%H:%M:%SZ',r.created_at),COALESCE(strftime('%Y-%m-%dT%H:%M:%SZ',r.finished_at),''),r.diagnostic,
|
||||
` + runStamps + `,r.diagnostic,
|
||||
CASE WHEN r.status='succeeded' THEN r.output_json ELSE COALESCE((SELECT output_json FROM run_steps WHERE run_id=r.id AND status='completed' ORDER BY sequence DESC LIMIT 1),'{}') END,
|
||||
(SELECT COUNT(*) FROM run_steps WHERE run_id=r.id AND status='completed')
|
||||
FROM pipeline_runs r JOIN pipelines p ON p.id=r.pipeline_id JOIN pipeline_versions v ON v.pipeline_id=r.pipeline_id AND v.version=r.version`
|
||||
` + runFrom + ` JOIN pipeline_versions v ON v.pipeline_id=r.pipeline_id AND v.version=r.version`
|
||||
|
||||
func scanRun(row interface{ Scan(...any) error }, r *protocol.RunSummary) error {
|
||||
var output string
|
||||
|
||||
@@ -0,0 +1,686 @@
|
||||
package pipeline
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"sort"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
"unicode"
|
||||
|
||||
"git.a-wan.net/alan/pipelines/internal/protocol"
|
||||
)
|
||||
|
||||
// Search answers one ranked, bounded, read-only query over the existing tables.
|
||||
// It scans no artifact contents and makes no follow-up calls: each kind runs at
|
||||
// most one query, field text is decoded with decodeValues over a fixed scan
|
||||
// budget, and ranking/tie-breaking happens in memory over the candidate set.
|
||||
func (s *Store) Search(ctx context.Context, req protocol.SearchRequest, now time.Time) (*protocol.SearchResponse, error) {
|
||||
kind := strings.TrimSpace(req.Kind)
|
||||
if kind != "" && searchKindIndex(kind) < 0 {
|
||||
return nil, fmt.Errorf("unknown search kind %q; use one of %s", req.Kind, strings.Join(protocol.SearchKinds, ", "))
|
||||
}
|
||||
response := &protocol.SearchResponse{
|
||||
Query: req.Query,
|
||||
Kind: kind,
|
||||
Results: []protocol.SearchResult{},
|
||||
Cap: searchLimit(req.Limit),
|
||||
ScanBoundBytes: protocol.SearchMaxScanBytes,
|
||||
}
|
||||
// An empty query is a well-defined no-op, never a match-everything scan.
|
||||
if strings.TrimSpace(req.Query) == "" {
|
||||
return response, nil
|
||||
}
|
||||
terms := parseSearchTerms(req.Query, now)
|
||||
// A bare status word or date selects runs; it is never treated as body text.
|
||||
selectorOnly := terms.Text == ""
|
||||
|
||||
tx, err := s.db.BeginTx(ctx, &sql.TxOptions{ReadOnly: true})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer tx.Rollback()
|
||||
|
||||
var results []protocol.SearchResult
|
||||
var scan searchScan
|
||||
for i, candidate := range protocol.SearchKinds {
|
||||
if kind != "" && searchKindIndex(kind) != i {
|
||||
continue
|
||||
}
|
||||
if selectorOnly && candidate != protocol.SearchKindRun {
|
||||
continue
|
||||
}
|
||||
var hits []protocol.SearchResult
|
||||
var err error
|
||||
switch candidate {
|
||||
case protocol.SearchKindPipeline:
|
||||
hits, err = searchPipelines(ctx, tx, terms, &scan)
|
||||
case protocol.SearchKindVersion:
|
||||
hits, err = searchVersions(ctx, tx, terms, &scan)
|
||||
case protocol.SearchKindRun:
|
||||
hits, err = searchRuns(ctx, tx, terms, &scan)
|
||||
case protocol.SearchKindStep:
|
||||
hits, err = searchSteps(ctx, tx, terms, &scan)
|
||||
case protocol.SearchKindField:
|
||||
hits, err = searchFields(ctx, tx, terms, &scan)
|
||||
case protocol.SearchKindArtifact:
|
||||
hits, err = searchArtifacts(ctx, tx, terms, &scan)
|
||||
}
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
results = append(results, hits...)
|
||||
}
|
||||
if err := tx.Commit(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
response.ScannedSteps = scan.steps
|
||||
response.ScannedBytes = scan.bytes
|
||||
if scan.truncated {
|
||||
response.Truncated = true
|
||||
}
|
||||
|
||||
sortSearchResults(results)
|
||||
if len(results) > response.Cap {
|
||||
results = results[:response.Cap]
|
||||
response.Truncated = true
|
||||
}
|
||||
if results == nil {
|
||||
results = []protocol.SearchResult{}
|
||||
}
|
||||
response.Results = results
|
||||
return response, nil
|
||||
}
|
||||
|
||||
func searchKindIndex(kind string) int {
|
||||
for i, k := range protocol.SearchKinds {
|
||||
if k == kind {
|
||||
return i
|
||||
}
|
||||
}
|
||||
return -1
|
||||
}
|
||||
|
||||
// searchLimit clamps a requested cap. Zero, negative or over-max requests fall
|
||||
// back to the default or the maximum, so the cap is always enforced server-side.
|
||||
func searchLimit(limit int) int {
|
||||
if limit <= 0 {
|
||||
return protocol.SearchDefaultLimit
|
||||
}
|
||||
if limit > protocol.SearchMaxLimit {
|
||||
return protocol.SearchMaxLimit
|
||||
}
|
||||
return limit
|
||||
}
|
||||
|
||||
// searchTerms is the parsed query. Text carries free text; Status and the
|
||||
// From/To window carry a bare status word or a date expression that selects runs.
|
||||
type searchTerms struct {
|
||||
Text string
|
||||
Status string
|
||||
HasDate bool
|
||||
From time.Time
|
||||
To time.Time
|
||||
}
|
||||
|
||||
// parseSearchTerms recognises a lone status word and a lone date expression and
|
||||
// leaves everything else as free text, so ordinary words are never misread as a
|
||||
// selector. Relative dates resolve against the injected now in its own location;
|
||||
// the comparison window is converted to UTC to match the stored timestamps.
|
||||
func parseSearchTerms(query string, now time.Time) searchTerms {
|
||||
trimmed := strings.TrimSpace(query)
|
||||
lower := strings.ToLower(trimmed)
|
||||
for _, status := range protocol.SearchStatuses {
|
||||
if lower == status {
|
||||
return searchTerms{Status: status}
|
||||
}
|
||||
}
|
||||
location := now.Location()
|
||||
day := time.Date(now.Year(), now.Month(), now.Day(), 0, 0, 0, 0, location)
|
||||
switch lower {
|
||||
case "today":
|
||||
return dateTerms(day, day.AddDate(0, 0, 1))
|
||||
case "yesterday":
|
||||
return dateTerms(day.AddDate(0, 0, -1), day)
|
||||
case "this week":
|
||||
// Monday 00:00 local is the start of the week.
|
||||
start := day.AddDate(0, 0, -((int(day.Weekday()) + 6) % 7))
|
||||
return dateTerms(start, start.AddDate(0, 0, 7))
|
||||
}
|
||||
if parsed, err := time.ParseInLocation("2006-01-02", trimmed, location); err == nil {
|
||||
return dateTerms(parsed, parsed.AddDate(0, 0, 1))
|
||||
}
|
||||
return searchTerms{Text: trimmed}
|
||||
}
|
||||
|
||||
func dateTerms(from, to time.Time) searchTerms {
|
||||
return searchTerms{HasDate: true, From: from.UTC(), To: to.UTC()}
|
||||
}
|
||||
|
||||
const searchSQLTime = "2006-01-02 15:04:05"
|
||||
|
||||
// searchStepJoin is the shared run_steps/step_control source and projections the
|
||||
// step and field scans build on, so those queries cannot drift apart.
|
||||
const (
|
||||
searchStepJoin = `FROM run_steps s LEFT JOIN step_control c ON c.run_id=s.run_id AND c.sequence=s.sequence`
|
||||
searchStepEvidence = `COALESCE(c.evidence_json,'')`
|
||||
searchStepRecency = `COALESCE(strftime('%Y-%m-%dT%H:%M:%SZ',s.started_at),'')`
|
||||
)
|
||||
|
||||
// searchRowBudget bounds how many raw rows one non-body kind scans before it
|
||||
// stops and reports Truncated, so a LIKE scan over a very large table cannot
|
||||
// scale without limit. Field text uses the separate SearchMaxScanSteps budget.
|
||||
const searchRowBudget = 2000
|
||||
|
||||
// searchLike builds a case-insensitive substring pattern, escaping LIKE
|
||||
// metacharacters so a query can never broaden the pattern.
|
||||
func searchLike(text string) string {
|
||||
escaped := strings.NewReplacer(`\`, `\\`, `%`, `\%`, `_`, `\_`).Replace(strings.ToLower(text))
|
||||
return "%" + escaped + "%"
|
||||
}
|
||||
|
||||
// searchTier ranks a "name" match: exact beats a substring. The third tier (a
|
||||
// body-text match) is assigned by the caller.
|
||||
func searchTier(value, text string) (int, bool) {
|
||||
if text == "" {
|
||||
return 0, false
|
||||
}
|
||||
if strings.EqualFold(value, text) {
|
||||
return 3, true
|
||||
}
|
||||
if strings.Contains(strings.ToLower(value), strings.ToLower(text)) {
|
||||
return 2, true
|
||||
}
|
||||
return 0, false
|
||||
}
|
||||
|
||||
func searchContains(value, text string) bool {
|
||||
return text != "" && strings.Contains(strings.ToLower(value), strings.ToLower(text))
|
||||
}
|
||||
|
||||
func sortSearchResults(results []protocol.SearchResult) {
|
||||
sort.SliceStable(results, func(i, j int) bool {
|
||||
a, b := results[i], results[j]
|
||||
if a.Score != b.Score {
|
||||
return a.Score > b.Score
|
||||
}
|
||||
if a.Recency != b.Recency {
|
||||
return a.Recency > b.Recency
|
||||
}
|
||||
if a.Kind != b.Kind {
|
||||
return a.Kind < b.Kind
|
||||
}
|
||||
return a.Target < b.Target
|
||||
})
|
||||
}
|
||||
|
||||
func searchPipelines(ctx context.Context, tx *sql.Tx, terms searchTerms, scan *searchScan) ([]protocol.SearchResult, error) {
|
||||
if terms.Text == "" {
|
||||
return nil, nil
|
||||
}
|
||||
rows, err := tx.QueryContext(ctx, `SELECT p.id,p.name,v.version,json_array_length(v.graph_json,'$.stages'),COALESCE(strftime('%Y-%m-%dT%H:%M:%SZ',v.created_at),'')
|
||||
FROM pipelines p JOIN pipeline_versions v ON v.pipeline_id=p.id
|
||||
WHERE v.version=(SELECT MAX(version) FROM pipeline_versions WHERE pipeline_id=p.id)
|
||||
AND lower(p.name) LIKE ? ESCAPE '\'
|
||||
ORDER BY p.name,p.id LIMIT ?`, searchLike(terms.Text), searchRowBudget+1)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
var results []protocol.SearchResult
|
||||
scanned := 0
|
||||
for rows.Next() {
|
||||
if scanned == searchRowBudget {
|
||||
scan.truncated = true
|
||||
break
|
||||
}
|
||||
scanned++
|
||||
var id int64
|
||||
var name, recency string
|
||||
var version, stages int
|
||||
if err := rows.Scan(&id, &name, &version, &stages, &recency); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
score, ok := searchTier(name, terms.Text)
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
results = append(results, protocol.SearchResult{
|
||||
Kind: protocol.SearchKindPipeline, Target: searchPipelineTarget(id),
|
||||
Score: score, MatchedField: "name", Recency: recency,
|
||||
Pipeline: &protocol.SearchPipeline{ID: id, Name: name, Version: version, Stages: stages},
|
||||
})
|
||||
}
|
||||
return results, rows.Err()
|
||||
}
|
||||
|
||||
func searchVersions(ctx context.Context, tx *sql.Tx, terms searchTerms, scan *searchScan) ([]protocol.SearchResult, error) {
|
||||
if terms.Text == "" {
|
||||
return nil, nil
|
||||
}
|
||||
query := `SELECT v.pipeline_id,v.version,v.content_hash,COALESCE(strftime('%Y-%m-%dT%H:%M:%SZ',v.created_at),''),v.source
|
||||
FROM pipeline_versions v WHERE lower(v.source) LIKE ? ESCAPE '\'`
|
||||
args := []any{searchLike(terms.Text)}
|
||||
var wanted int64
|
||||
numeric := false
|
||||
if n, err := parsePositiveInt(terms.Text); err == nil {
|
||||
query += " OR v.version=?"
|
||||
args = append(args, n)
|
||||
wanted, numeric = n, true
|
||||
}
|
||||
query += " ORDER BY v.pipeline_id,v.version DESC LIMIT ?"
|
||||
args = append(args, searchRowBudget+1)
|
||||
rows, err := tx.QueryContext(ctx, query, args...)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
var results []protocol.SearchResult
|
||||
scanned := 0
|
||||
for rows.Next() {
|
||||
if scanned == searchRowBudget {
|
||||
scan.truncated = true
|
||||
break
|
||||
}
|
||||
scanned++
|
||||
var pipelineID int64
|
||||
var version int
|
||||
var hash, recency, source string
|
||||
if err := rows.Scan(&pipelineID, &version, &hash, &recency, &source); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
score, matched := 1, "source"
|
||||
if numeric && int64(version) == wanted {
|
||||
score, matched = 3, "version"
|
||||
} else if !searchContains(source, terms.Text) {
|
||||
continue
|
||||
}
|
||||
results = append(results, protocol.SearchResult{
|
||||
Kind: protocol.SearchKindVersion, Target: searchVersionTarget(pipelineID, version),
|
||||
Score: score, MatchedField: matched, Recency: recency,
|
||||
Version: &protocol.SearchVersion{PipelineID: pipelineID, Version: version, ContentHash: hash, CreatedAt: recency},
|
||||
})
|
||||
}
|
||||
return results, rows.Err()
|
||||
}
|
||||
|
||||
func searchRuns(ctx context.Context, tx *sql.Tx, terms searchTerms, scan *searchScan) ([]protocol.SearchResult, error) {
|
||||
// Reuse runSelect's join and timestamp projection rather than restating the
|
||||
// run-row SELECT; only the fields search needs are added.
|
||||
base := `SELECT r.id,p.name,r.status,r.workspace,` + runStamps + `,r.diagnostic ` + runFrom
|
||||
var query string
|
||||
var args []any
|
||||
switch {
|
||||
case terms.Status != "":
|
||||
query = base + " WHERE r.status=?"
|
||||
args = append(args, terms.Status)
|
||||
case terms.HasDate:
|
||||
query = base + " WHERE r.created_at>=? AND r.created_at<?"
|
||||
args = append(args, terms.From.Format(searchSQLTime), terms.To.Format(searchSQLTime))
|
||||
default:
|
||||
query = base + ` WHERE lower(p.name) LIKE ? ESCAPE '\' OR lower(r.workspace) LIKE ? ESCAPE '\' OR lower(r.status) LIKE ? ESCAPE '\' OR lower(r.diagnostic) LIKE ? ESCAPE '\' OR CAST(r.id AS TEXT)=?`
|
||||
pattern := searchLike(terms.Text)
|
||||
args = append(args, pattern, pattern, pattern, pattern, terms.Text)
|
||||
}
|
||||
query += " ORDER BY r.id DESC LIMIT ?"
|
||||
args = append(args, searchRowBudget+1)
|
||||
rows, err := tx.QueryContext(ctx, query, args...)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
var results []protocol.SearchResult
|
||||
scanned := 0
|
||||
for rows.Next() {
|
||||
if scanned == searchRowBudget {
|
||||
scan.truncated = true
|
||||
break
|
||||
}
|
||||
scanned++
|
||||
var id int64
|
||||
var name, status, workspace, created, finished, diagnostic string
|
||||
if err := rows.Scan(&id, &name, &status, &workspace, &created, &finished, &diagnostic); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
score, matched := 1, "status"
|
||||
if terms.Status != "" {
|
||||
matched = "status"
|
||||
} else if terms.HasDate {
|
||||
matched = "created_at"
|
||||
} else if s, ok := searchTier(name, terms.Text); ok {
|
||||
score, matched = s, "pipeline_name"
|
||||
} else if numericEqual(terms.Text, id) {
|
||||
score, matched = 3, "id"
|
||||
} else if searchContains(workspace, terms.Text) {
|
||||
matched = "workspace"
|
||||
} else if searchContains(status, terms.Text) {
|
||||
matched = "status"
|
||||
} else if searchContains(diagnostic, terms.Text) {
|
||||
matched = "diagnostic"
|
||||
} else {
|
||||
continue
|
||||
}
|
||||
run := &protocol.SearchRun{ID: id, PipelineName: name, Status: status, Workspace: workspace, CreatedAt: created, FinishedAt: finished}
|
||||
if created != "" && finished != "" {
|
||||
if secs, ok := durationSeconds(created, finished); ok {
|
||||
run.DurationSeconds = secs
|
||||
}
|
||||
}
|
||||
results = append(results, protocol.SearchResult{
|
||||
Kind: protocol.SearchKindRun, Target: searchRunTarget(id),
|
||||
Score: score, MatchedField: matched, Recency: created, Run: run,
|
||||
})
|
||||
}
|
||||
return results, rows.Err()
|
||||
}
|
||||
|
||||
func searchSteps(ctx context.Context, tx *sql.Tx, terms searchTerms, scan *searchScan) ([]protocol.SearchResult, error) {
|
||||
if terms.Text == "" {
|
||||
return nil, nil
|
||||
}
|
||||
pattern := searchLike(terms.Text)
|
||||
rows, err := tx.QueryContext(ctx, `SELECT s.run_id,s.sequence,s.stage_key,s.outcome,s.target,s.diagnostic,`+searchStepRecency+`,`+searchStepEvidence+`
|
||||
`+searchStepJoin+`
|
||||
WHERE lower(s.stage_key) LIKE ? ESCAPE '\' OR lower(s.outcome) LIKE ? ESCAPE '\' OR lower(s.target) LIKE ? ESCAPE '\' OR lower(s.diagnostic) LIKE ? ESCAPE '\'
|
||||
ORDER BY s.run_id DESC,s.sequence DESC LIMIT ?`, pattern, pattern, pattern, pattern, searchRowBudget+1)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
var results []protocol.SearchResult
|
||||
scanned := 0
|
||||
for rows.Next() {
|
||||
if scanned == searchRowBudget {
|
||||
scan.truncated = true
|
||||
break
|
||||
}
|
||||
scanned++
|
||||
var runID int64
|
||||
var sequence int
|
||||
var stage, outcome, target, diagnostic, recency, control string
|
||||
if err := rows.Scan(&runID, &sequence, &stage, &outcome, &target, &diagnostic, &recency, &control); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
score, matched := 1, "diagnostic"
|
||||
if s, ok := searchTier(stage, terms.Text); ok {
|
||||
score, matched = s, "stage"
|
||||
} else if searchContains(outcome, terms.Text) {
|
||||
matched = "outcome"
|
||||
} else if searchContains(target, terms.Text) {
|
||||
matched = "target"
|
||||
} else if searchContains(diagnostic, terms.Text) {
|
||||
matched = "diagnostic"
|
||||
} else {
|
||||
continue
|
||||
}
|
||||
visit := 0
|
||||
if control != "" && control != "null" {
|
||||
var evidence protocol.ControlEvidence
|
||||
if err := json.Unmarshal([]byte(control), &evidence); err == nil {
|
||||
visit = evidence.Visit
|
||||
}
|
||||
}
|
||||
results = append(results, protocol.SearchResult{
|
||||
Kind: protocol.SearchKindStep, Target: searchStepTarget(runID, sequence),
|
||||
Score: score, MatchedField: matched, Recency: recency,
|
||||
Step: &protocol.SearchStep{RunID: runID, Sequence: sequence, Stage: stage, Visit: visit, Outcome: outcome, Target: target, Diagnostic: diagnostic},
|
||||
})
|
||||
}
|
||||
return results, rows.Err()
|
||||
}
|
||||
|
||||
// searchScan accumulates the shared bounded-scan budget so the response can
|
||||
// report how much body text was inspected and whether any bound was reached.
|
||||
type searchScan struct {
|
||||
steps int
|
||||
bytes int
|
||||
truncated bool
|
||||
}
|
||||
|
||||
func searchFields(ctx context.Context, tx *sql.Tx, terms searchTerms, scan *searchScan) ([]protocol.SearchResult, error) {
|
||||
if terms.Text == "" {
|
||||
return nil, nil
|
||||
}
|
||||
// One extra step is requested so a scan that hits the step bound can see that
|
||||
// more rows remain and report Truncated rather than ending silently.
|
||||
rows, err := tx.QueryContext(ctx, `SELECT s.run_id,s.sequence,s.input_json,s.output_json,s.transcript,`+searchStepEvidence+`,`+searchStepRecency+`
|
||||
`+searchStepJoin+`
|
||||
ORDER BY s.run_id DESC,s.sequence DESC LIMIT ?`, protocol.SearchMaxScanSteps+1)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
var results []protocol.SearchResult
|
||||
for rows.Next() {
|
||||
if scan.steps >= protocol.SearchMaxScanSteps {
|
||||
scan.truncated = true
|
||||
break
|
||||
}
|
||||
var runID int64
|
||||
var sequence int
|
||||
var input, output, transcript, evidence, recency string
|
||||
if err := rows.Scan(&runID, &sequence, &input, &output, &transcript, &evidence, &recency); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
size := len(input) + len(output) + len(transcript) + len(evidence)
|
||||
if scan.bytes+size > protocol.SearchMaxScanBytes {
|
||||
scan.truncated = true
|
||||
break
|
||||
}
|
||||
scan.bytes += size
|
||||
scan.steps++
|
||||
for _, field := range []struct {
|
||||
name string
|
||||
raw string
|
||||
}{
|
||||
{name: "input", raw: input},
|
||||
{name: "output", raw: output},
|
||||
{name: "evidence", raw: evidence},
|
||||
} {
|
||||
if field.raw == "" {
|
||||
continue
|
||||
}
|
||||
var values map[string]any
|
||||
if err := decodeValues(field.raw, &values); err != nil {
|
||||
continue
|
||||
}
|
||||
text, ok := findMatch(values, terms.Text)
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
results = append(results, protocol.SearchResult{
|
||||
Kind: protocol.SearchKindField, Target: searchFieldTarget(runID, sequence, field.name),
|
||||
Score: 1, MatchedField: field.name, Recency: recency,
|
||||
Field: &protocol.SearchField{RunID: runID, Sequence: sequence, Field: field.name, Snippet: searchSnippet(text, terms.Text)},
|
||||
})
|
||||
}
|
||||
if searchContains(transcript, terms.Text) {
|
||||
results = append(results, protocol.SearchResult{
|
||||
Kind: protocol.SearchKindField, Target: searchFieldTarget(runID, sequence, "transcript"),
|
||||
Score: 1, MatchedField: "transcript", Recency: recency,
|
||||
Field: &protocol.SearchField{RunID: runID, Sequence: sequence, Field: "transcript", Snippet: searchSnippet(transcript, terms.Text)},
|
||||
})
|
||||
}
|
||||
}
|
||||
return results, rows.Err()
|
||||
}
|
||||
|
||||
func searchArtifacts(ctx context.Context, tx *sql.Tx, terms searchTerms, scan *searchScan) ([]protocol.SearchResult, error) {
|
||||
if terms.Text == "" {
|
||||
return nil, nil
|
||||
}
|
||||
pattern := searchLike(terms.Text)
|
||||
rows, err := tx.QueryContext(ctx, `SELECT a.run_id,a.sequence,a.artifacts_json,COALESCE(strftime('%Y-%m-%dT%H:%M:%SZ',r.created_at),'')
|
||||
FROM step_artifacts a JOIN pipeline_runs r ON r.id=a.run_id
|
||||
WHERE lower(a.artifacts_json) LIKE ? ESCAPE '\'
|
||||
ORDER BY a.run_id DESC,a.sequence DESC LIMIT ?`, pattern, searchRowBudget+1)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
var results []protocol.SearchResult
|
||||
scanned := 0
|
||||
for rows.Next() {
|
||||
if scanned == searchRowBudget {
|
||||
scan.truncated = true
|
||||
break
|
||||
}
|
||||
scanned++
|
||||
var runID int64
|
||||
var sequence int
|
||||
var artifacts, recency string
|
||||
if err := rows.Scan(&runID, &sequence, &artifacts, &recency); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var evidence stageArtifactsEvidence
|
||||
if err := json.Unmarshal([]byte(artifacts), &evidence); err != nil {
|
||||
continue
|
||||
}
|
||||
for _, file := range evidence.Files {
|
||||
score, matched := 1, "content_type"
|
||||
if s, ok := searchTier(file.Name, terms.Text); ok {
|
||||
score, matched = s, "name"
|
||||
} else if !searchContains(file.ContentType, terms.Text) {
|
||||
continue
|
||||
}
|
||||
results = append(results, protocol.SearchResult{
|
||||
Kind: protocol.SearchKindArtifact, Target: file.URL,
|
||||
Score: score, MatchedField: matched, Recency: recency,
|
||||
Artifact: &protocol.SearchArtifact{RunID: runID, Sequence: sequence, Name: file.Name, ContentType: file.ContentType, SizeBytes: file.SizeBytes, URL: file.URL},
|
||||
})
|
||||
}
|
||||
}
|
||||
return results, rows.Err()
|
||||
}
|
||||
|
||||
// findMatch walks decoded JSON values in a stable key order and returns the
|
||||
// first string leaf containing the query, so a hit and its snippet are
|
||||
// deterministic rather than map-iteration dependent.
|
||||
func findMatch(value any, text string) (string, bool) {
|
||||
switch typed := value.(type) {
|
||||
case string:
|
||||
if searchContains(typed, text) {
|
||||
return typed, true
|
||||
}
|
||||
case map[string]any:
|
||||
keys := make([]string, 0, len(typed))
|
||||
for key := range typed {
|
||||
keys = append(keys, key)
|
||||
}
|
||||
sort.Strings(keys)
|
||||
for _, key := range keys {
|
||||
if match, ok := findMatch(typed[key], text); ok {
|
||||
return match, true
|
||||
}
|
||||
}
|
||||
case []any:
|
||||
for _, item := range typed {
|
||||
if match, ok := findMatch(item, text); ok {
|
||||
return match, true
|
||||
}
|
||||
}
|
||||
}
|
||||
return "", false
|
||||
}
|
||||
|
||||
// searchSnippet reduces text to a single clean line and centers a short window
|
||||
// on the first case-insensitive match, folding runes so UTF-8 boundaries are
|
||||
// never split and the result is always well-formed.
|
||||
func searchSnippet(text, query string) string {
|
||||
normalized := protocol.NormalizeText(text)
|
||||
runes := []rune(normalized)
|
||||
index := matchRunes(runes, []rune(query))
|
||||
if index < 0 {
|
||||
return truncateRunes(normalized, 80)
|
||||
}
|
||||
const margin = 40
|
||||
start := index - margin
|
||||
if start < 0 {
|
||||
start = 0
|
||||
}
|
||||
end := index + len([]rune(query)) + margin
|
||||
if end > len(runes) {
|
||||
end = len(runes)
|
||||
}
|
||||
snippet := string(runes[start:end])
|
||||
if start > 0 {
|
||||
snippet = "…" + snippet
|
||||
}
|
||||
if end < len(runes) {
|
||||
snippet = snippet + "…"
|
||||
}
|
||||
return snippet
|
||||
}
|
||||
|
||||
// matchRunes returns the rune index of query in text, case-insensitively.
|
||||
func matchRunes(text, query []rune) int {
|
||||
if len(query) == 0 || len(query) > len(text) {
|
||||
return -1
|
||||
}
|
||||
for i := 0; i+len(query) <= len(text); i++ {
|
||||
match := true
|
||||
for j := range query {
|
||||
if unicode.ToLower(text[i+j]) != unicode.ToLower(query[j]) {
|
||||
match = false
|
||||
break
|
||||
}
|
||||
}
|
||||
if match {
|
||||
return i
|
||||
}
|
||||
}
|
||||
return -1
|
||||
}
|
||||
|
||||
func truncateRunes(text string, limit int) string {
|
||||
runes := []rune(text)
|
||||
if len(runes) <= limit {
|
||||
return text
|
||||
}
|
||||
return string(runes[:limit]) + "…"
|
||||
}
|
||||
|
||||
func searchPipelineTarget(id int64) string { return fmt.Sprintf("pipeline:%d", id) }
|
||||
|
||||
func searchVersionTarget(id int64, version int) string {
|
||||
return fmt.Sprintf("pipeline:%d:v%d", id, version)
|
||||
}
|
||||
|
||||
func searchRunTarget(id int64) string { return fmt.Sprintf("run:%d", id) }
|
||||
|
||||
func searchStepTarget(id int64, sequence int) string { return fmt.Sprintf("run:%d:%d", id, sequence) }
|
||||
|
||||
func searchFieldTarget(id int64, sequence int, field string) string {
|
||||
return fmt.Sprintf("run:%d:%d:%s", id, sequence, field)
|
||||
}
|
||||
|
||||
func parsePositiveInt(text string) (int64, error) {
|
||||
n, err := strconv.ParseInt(strings.TrimSpace(text), 10, 64)
|
||||
if err != nil || n <= 0 {
|
||||
return 0, fmt.Errorf("not a positive integer")
|
||||
}
|
||||
return n, nil
|
||||
}
|
||||
|
||||
func numericEqual(text string, id int64) bool {
|
||||
n, err := parsePositiveInt(text)
|
||||
return err == nil && n == id
|
||||
}
|
||||
|
||||
func durationSeconds(created, finished string) (float64, bool) {
|
||||
start, err := time.Parse(time.RFC3339, created)
|
||||
if err != nil {
|
||||
return 0, false
|
||||
}
|
||||
end, err := time.Parse(time.RFC3339, finished)
|
||||
if err != nil {
|
||||
return 0, false
|
||||
}
|
||||
if end.Before(start) {
|
||||
return 0, false
|
||||
}
|
||||
return end.Sub(start).Seconds(), true
|
||||
}
|
||||
@@ -0,0 +1,119 @@
|
||||
package pipeline
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"os"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"git.a-wan.net/alan/pipelines/internal/protocol"
|
||||
)
|
||||
|
||||
// TestLiveSearchRealDatabase is an env-guarded read-only check against a real
|
||||
// database. It opens the file read-only (no DDL, no journal change, no writes),
|
||||
// derives queries from the data itself, and reports one result of each kind it
|
||||
// can observe. Run it with:
|
||||
//
|
||||
// PIPELINES_LIVE_SEARCH_DB=/path/to/pipelines.db go test ./internal/pipeline -run LiveSearch -v
|
||||
func TestLiveSearchRealDatabase(t *testing.T) {
|
||||
path := os.Getenv("PIPELINES_LIVE_SEARCH_DB")
|
||||
if path == "" {
|
||||
t.Skip("set PIPELINES_LIVE_SEARCH_DB to a real pipelines.db")
|
||||
}
|
||||
store, err := OpenReadOnly(path)
|
||||
if err != nil {
|
||||
t.Fatalf("open read-only: %v", err)
|
||||
}
|
||||
defer store.Close()
|
||||
requireSearchSchema(t, store)
|
||||
ctx := context.Background()
|
||||
pipelines, err := store.List(ctx)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(pipelines) == 0 {
|
||||
t.Skip("database has no pipelines")
|
||||
}
|
||||
now := time.Now()
|
||||
seen := map[string]protocol.SearchResult{}
|
||||
record := func(query string) {
|
||||
response, err := store.Search(ctx, protocol.SearchRequest{Query: query}, now)
|
||||
if err != nil {
|
||||
t.Fatalf("search %q: %v", query, err)
|
||||
}
|
||||
for _, hit := range response.Results {
|
||||
if _, ok := seen[hit.Kind]; !ok {
|
||||
seen[hit.Kind] = hit
|
||||
}
|
||||
}
|
||||
}
|
||||
queries := []string{}
|
||||
for _, p := range pipelines {
|
||||
queries = append(queries, p.Name)
|
||||
}
|
||||
// Derive stage/field/artifact queries from a bounded number of recent runs.
|
||||
runs, err := store.ListRuns(ctx)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for i := 0; i < len(runs) && i < 5; i++ {
|
||||
detail, err := store.GetRun(ctx, runs[i].ID)
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
for j := 0; j < len(detail.Steps) && j < 3; j++ {
|
||||
step := detail.Steps[j]
|
||||
if step.Stage != "" {
|
||||
queries = append(queries, step.Stage)
|
||||
}
|
||||
for _, artifact := range step.Artifacts {
|
||||
queries = append(queries, artifact.Name)
|
||||
}
|
||||
for _, key := range []string{"summary", "result", "report", "message", "evidence", "reason"} {
|
||||
if text, ok := step.Output[key].(string); ok {
|
||||
for _, word := range strings.Fields(text) {
|
||||
if len([]rune(word)) >= 6 {
|
||||
queries = append(queries, word)
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
for _, query := range queries {
|
||||
record(query)
|
||||
}
|
||||
if len(seen) == 0 {
|
||||
t.Fatal("live database returned no matches for any derived query")
|
||||
}
|
||||
for _, kind := range protocol.SearchKinds {
|
||||
if hit, ok := seen[kind]; ok {
|
||||
t.Logf("kind=%s target=%s score=%d matched=%s", kind, hit.Target, hit.Score, hit.MatchedField)
|
||||
} else {
|
||||
t.Logf("kind=%s not observed in this database", kind)
|
||||
}
|
||||
}
|
||||
if _, ok := seen[protocol.SearchKindRun]; !ok {
|
||||
t.Fatal("live database returned no run matches")
|
||||
}
|
||||
}
|
||||
|
||||
// requireSearchSchema skips (rather than fails) when the supplied database
|
||||
// predates the tables search reads, so pointing the check at a stale local file
|
||||
// reports the mismatch instead of a confusing "no such table" error.
|
||||
func requireSearchSchema(t *testing.T, store *Store) {
|
||||
t.Helper()
|
||||
for _, table := range []string{"pipelines", "pipeline_versions", "pipeline_runs", "run_steps", "step_control", "step_artifacts"} {
|
||||
var name string
|
||||
err := store.db.QueryRow("SELECT name FROM sqlite_master WHERE type='table' AND name=?", table).Scan(&name)
|
||||
if err == sql.ErrNoRows {
|
||||
t.Skipf("database lacks the current search schema (missing %q); point PIPELINES_LIVE_SEARCH_DB at a current database", table)
|
||||
}
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,601 @@
|
||||
package pipeline
|
||||
|
||||
import (
|
||||
"context"
|
||||
"path/filepath"
|
||||
"strconv"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
"unicode/utf8"
|
||||
|
||||
"git.a-wan.net/alan/pipelines/internal/protocol"
|
||||
)
|
||||
|
||||
// searchFixture builds one pipeline whose name, source, run, stage, field text
|
||||
// and artifact all contain "zephyr", so a single free-text query exercises
|
||||
// every kind.
|
||||
func searchFixture(t *testing.T) (*Store, int64) {
|
||||
t.Helper()
|
||||
ctx := context.Background()
|
||||
store, err := Open(filepath.Join(t.TempDir(), "search.db"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
t.Cleanup(func() { store.Close() })
|
||||
if _, err := store.Submit(ctx, "Zephyr Build", Template([]string{"zephyr"}, true)); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
items, err := store.List(ctx)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
definition, err := store.Get(ctx, items[0].ID, 1)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
run, _, err := store.CreateRun(ctx, definition, "zephyr-key", "/work/zephyr", false, nil, map[string]any{"prompt": "zephyr task"})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := store.BeginStep(ctx, run, 1, "zephyr", map[string]any{"prompt": "zephyr task"}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
artifactURL := "/artifacts/" + strconv.FormatInt(run, 10) + "/1/zephyr.txt"
|
||||
if err := store.FinishStep(ctx, run, protocol.RunStep{
|
||||
Sequence: 1, Status: "completed", Outcome: "success", Target: protocol.SuccessEnd,
|
||||
Output: map[string]any{"summary": "zephyr completed"},
|
||||
Control: &protocol.ControlEvidence{Visit: 1, MaxVisits: 2, ModelTarget: "zephyr-model"},
|
||||
Artifacts: []protocol.StageArtifact{{Name: "zephyr.txt", SizeBytes: 12, ContentType: "text/plain", URL: artifactURL}},
|
||||
}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := store.FinishRun(ctx, run, "succeeded", "success", "", map[string]any{"summary": "zephyr completed"}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return store, run
|
||||
}
|
||||
|
||||
func searchKinds(response *protocol.SearchResponse) map[string]bool {
|
||||
kinds := map[string]bool{}
|
||||
for _, hit := range response.Results {
|
||||
kinds[hit.Kind] = true
|
||||
}
|
||||
return kinds
|
||||
}
|
||||
|
||||
func TestSearchReturnsEveryKind(t *testing.T) {
|
||||
store, run := searchFixture(t)
|
||||
response, err := store.Search(context.Background(), protocol.SearchRequest{Query: "zephyr"}, time.Now())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
kinds := searchKinds(response)
|
||||
for _, kind := range protocol.SearchKinds {
|
||||
if !kinds[kind] {
|
||||
t.Fatalf("kind %q missing from %+v", kind, response.Results)
|
||||
}
|
||||
}
|
||||
if response.Cap != protocol.SearchDefaultLimit || response.ScanBoundBytes != protocol.SearchMaxScanBytes {
|
||||
t.Fatalf("bounds not reported: %+v", response)
|
||||
}
|
||||
if response.Truncated {
|
||||
t.Fatalf("a tiny store must not report truncation: %+v", response)
|
||||
}
|
||||
// Every target must point at an addressable entity an existing command resolves.
|
||||
ctx := context.Background()
|
||||
for _, hit := range response.Results {
|
||||
switch hit.Kind {
|
||||
case protocol.SearchKindPipeline:
|
||||
id := targetInt(t, hit.Target, "pipeline:")
|
||||
if _, err := store.Get(ctx, id, 0); err != nil {
|
||||
t.Fatalf("pipeline target %q does not resolve: %v", hit.Target, err)
|
||||
}
|
||||
case protocol.SearchKindVersion:
|
||||
parts := strings.Split(strings.TrimPrefix(hit.Target, "pipeline:"), ":v")
|
||||
if len(parts) != 2 {
|
||||
t.Fatalf("bad version target %q", hit.Target)
|
||||
}
|
||||
if _, err := store.Get(ctx, parseInt(t, parts[0]), int(parseInt(t, parts[1]))); err != nil {
|
||||
t.Fatalf("version target %q does not resolve: %v", hit.Target, err)
|
||||
}
|
||||
case protocol.SearchKindRun:
|
||||
if _, err := store.GetRun(ctx, targetInt(t, hit.Target, "run:")); err != nil {
|
||||
t.Fatalf("run target %q does not resolve: %v", hit.Target, err)
|
||||
}
|
||||
case protocol.SearchKindStep, protocol.SearchKindField:
|
||||
fields := strings.Split(strings.TrimPrefix(hit.Target, "run:"), ":")
|
||||
if len(fields) < 2 {
|
||||
t.Fatalf("bad step/field target %q", hit.Target)
|
||||
}
|
||||
detail, err := store.GetRun(ctx, parseInt(t, fields[0]))
|
||||
if err != nil {
|
||||
t.Fatalf("step/field target %q does not resolve: %v", hit.Target, err)
|
||||
}
|
||||
sequence := parseInt(t, fields[1])
|
||||
found := false
|
||||
for _, step := range detail.Steps {
|
||||
if int64(step.Sequence) == sequence {
|
||||
found = true
|
||||
}
|
||||
}
|
||||
if !found {
|
||||
t.Fatalf("step/field target %q names no recorded step", hit.Target)
|
||||
}
|
||||
case protocol.SearchKindArtifact:
|
||||
if !strings.HasPrefix(hit.Target, "/artifacts/") {
|
||||
t.Fatalf("artifact target %q is not a served URL", hit.Target)
|
||||
}
|
||||
}
|
||||
}
|
||||
if run <= 0 {
|
||||
t.Fatal("fixture run missing")
|
||||
}
|
||||
}
|
||||
|
||||
func targetInt(t *testing.T, target, prefix string) int64 {
|
||||
t.Helper()
|
||||
if !strings.HasPrefix(target, prefix) {
|
||||
t.Fatalf("target %q lacks prefix %q", target, prefix)
|
||||
}
|
||||
return parseInt(t, strings.TrimPrefix(target, prefix))
|
||||
}
|
||||
|
||||
func parseInt(t *testing.T, value string) int64 {
|
||||
t.Helper()
|
||||
n, err := parsePositiveInt(value)
|
||||
if err != nil {
|
||||
t.Fatalf("bad number %q: %v", value, err)
|
||||
}
|
||||
return n
|
||||
}
|
||||
|
||||
func TestSearchFieldTextCoversInputOutputEvidence(t *testing.T) {
|
||||
store, _ := searchFixture(t)
|
||||
response, err := store.Search(context.Background(), protocol.SearchRequest{Query: "zephyr", Kind: protocol.SearchKindField}, time.Now())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
fields := map[string]bool{}
|
||||
for _, hit := range response.Results {
|
||||
if hit.Field == nil {
|
||||
t.Fatalf("field kind without a field payload: %+v", hit)
|
||||
}
|
||||
fields[hit.Field.Field] = true
|
||||
if !strings.Contains(hit.Field.Snippet, "zephyr") {
|
||||
t.Fatalf("snippet lacks the match: %+v", hit.Field)
|
||||
}
|
||||
}
|
||||
for _, field := range []string{"input", "output", "evidence"} {
|
||||
if !fields[field] {
|
||||
t.Fatalf("field text search missed %q: %+v", field, response.Results)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestSearchRankingExactSubstringBodyAndRecency(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
store, err := Open(filepath.Join(t.TempDir(), "ranking.db"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer store.Close()
|
||||
submit := func(name string) {
|
||||
if _, err := store.Submit(ctx, name, Template([]string{"one"}, true)); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
submit("alpha")
|
||||
submit("alphabet")
|
||||
items, _ := store.List(ctx)
|
||||
definition, _ := store.Get(ctx, items[0].ID, 1)
|
||||
run, _, err := store.CreateRun(ctx, definition, "alpha-key", "/work", false, nil, map[string]any{"prompt": "body alpha match"})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := store.BeginStep(ctx, run, 1, "one", map[string]any{"prompt": "body alpha match"}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := store.FinishStep(ctx, run, protocol.RunStep{Sequence: 1, Status: "completed", Outcome: "success", Target: protocol.SuccessEnd}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
response, err := store.Search(ctx, protocol.SearchRequest{Query: "alpha"}, time.Now())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
seen := map[int]bool{}
|
||||
for i, hit := range response.Results {
|
||||
seen[hit.Score] = true
|
||||
if i > 0 && response.Results[i-1].Score < hit.Score {
|
||||
t.Fatalf("scores not descending: %+v", response.Results)
|
||||
}
|
||||
}
|
||||
if !seen[3] || !seen[2] || !seen[1] {
|
||||
t.Fatalf("expected exact/substring/body tiers: %+v", response.Results)
|
||||
}
|
||||
// The exact pipeline match outranks the substring pipeline match.
|
||||
exactIndex, substringIndex := -1, -1
|
||||
for i, hit := range response.Results {
|
||||
if hit.Pipeline == nil {
|
||||
continue
|
||||
}
|
||||
if hit.Pipeline.Name == "alpha" {
|
||||
exactIndex = i
|
||||
}
|
||||
if hit.Pipeline.Name == "alphabet" {
|
||||
substringIndex = i
|
||||
}
|
||||
if hit.Pipeline.Name == "alpha" && hit.Score != 3 {
|
||||
t.Fatalf("exact name scored %d", hit.Score)
|
||||
}
|
||||
if hit.Pipeline.Name == "alphabet" && hit.Score != 2 {
|
||||
t.Fatalf("substring name scored %d", hit.Score)
|
||||
}
|
||||
}
|
||||
if exactIndex < 0 || substringIndex < 0 || exactIndex > substringIndex {
|
||||
t.Fatalf("exact name must outrank a name substring: %+v", response.Results)
|
||||
}
|
||||
|
||||
// Equal-tier ties break by recency.
|
||||
older, _, err := store.CreateRun(ctx, definition, "alpha-older", "/work", false, nil, map[string]any{"prompt": "alpha"})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
newer, _, err := store.CreateRun(ctx, definition, "alpha-newer", "/work", false, nil, map[string]any{"prompt": "alpha"})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
setCreated(t, store, older, "2026-01-01 00:00:00")
|
||||
setCreated(t, store, newer, "2026-06-01 00:00:00")
|
||||
response, err = store.Search(ctx, protocol.SearchRequest{Query: "alpha"}, time.Now())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var olderIndex, newerIndex = -1, -1
|
||||
for i, hit := range response.Results {
|
||||
if hit.Run == nil || hit.Run.ID == 0 {
|
||||
continue
|
||||
}
|
||||
if hit.Run.ID == older {
|
||||
olderIndex = i
|
||||
}
|
||||
if hit.Run.ID == newer {
|
||||
newerIndex = i
|
||||
}
|
||||
}
|
||||
if olderIndex < 0 || newerIndex < 0 || newerIndex > olderIndex {
|
||||
t.Fatalf("recency tie-break lost: older=%d newer=%d", olderIndex, newerIndex)
|
||||
}
|
||||
}
|
||||
|
||||
func TestSearchStatusWordsSelectRuns(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
store, err := Open(filepath.Join(t.TempDir(), "statuses.db"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer store.Close()
|
||||
if _, err := store.Submit(ctx, "statuses", Template([]string{"one"}, true)); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
items, _ := store.List(ctx)
|
||||
definition, _ := store.Get(ctx, items[0].ID, 1)
|
||||
// One run per accepted status word, each with a distinct created_at so a
|
||||
// date selector can never be mistaken for the status selector.
|
||||
byStatus := map[string]int64{}
|
||||
for i, status := range protocol.SearchStatuses {
|
||||
run, _, err := store.CreateRun(ctx, definition, "status-"+status, "/work", false, nil, nil)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := store.db.Exec("UPDATE pipeline_runs SET status=?,created_at=? WHERE id=?", status, "2026-01-0"+strconv.Itoa(i+1)+" 09:00:00", run); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
byStatus[status] = run
|
||||
}
|
||||
now := time.Date(2026, 10, 16, 12, 0, 0, 0, time.UTC)
|
||||
for _, status := range protocol.SearchStatuses {
|
||||
response, err := store.Search(ctx, protocol.SearchRequest{Query: status}, now)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var runs []int64
|
||||
for _, hit := range response.Results {
|
||||
if hit.Run == nil {
|
||||
t.Fatalf("status %q returned a non-run kind %q", status, hit.Kind)
|
||||
}
|
||||
runs = append(runs, hit.Run.ID)
|
||||
}
|
||||
if len(runs) != 1 || runs[0] != byStatus[status] {
|
||||
t.Fatalf("status %q selected %v, want [%d]", status, runs, byStatus[status])
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestSearchStatusAndDateSelectors(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
store, err := Open(filepath.Join(t.TempDir(), "selector.db"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer store.Close()
|
||||
if _, err := store.Submit(ctx, "selectors", Template([]string{"one"}, true)); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
items, _ := store.List(ctx)
|
||||
definition, _ := store.Get(ctx, items[0].ID, 1)
|
||||
yesterdayRun, _, err := store.CreateRun(ctx, definition, "yesterday", "/work", false, nil, nil)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
todayRun, _, err := store.CreateRun(ctx, definition, "today", "/work", false, nil, nil)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := store.FinishRun(ctx, yesterdayRun, "failed", "error", "boom", nil); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
setCreated(t, store, yesterdayRun, "2026-10-15 09:00:00")
|
||||
setCreated(t, store, todayRun, "2026-10-16 09:00:00")
|
||||
now := time.Date(2026, 10, 16, 12, 0, 0, 0, time.UTC)
|
||||
|
||||
selects := func(query string) map[int64]bool {
|
||||
t.Helper()
|
||||
response, err := store.Search(ctx, protocol.SearchRequest{Query: query}, now)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
ids := map[int64]bool{}
|
||||
for _, hit := range response.Results {
|
||||
if hit.Run != nil {
|
||||
ids[hit.Run.ID] = true
|
||||
}
|
||||
}
|
||||
return ids
|
||||
}
|
||||
if ids := selects("failed"); !ids[yesterdayRun] || ids[todayRun] {
|
||||
t.Fatalf("status selector: %+v", ids)
|
||||
}
|
||||
if ids := selects("today"); !ids[todayRun] || ids[yesterdayRun] {
|
||||
t.Fatalf("today selector: %+v", ids)
|
||||
}
|
||||
if ids := selects("yesterday"); !ids[yesterdayRun] || ids[todayRun] {
|
||||
t.Fatalf("yesterday selector: %+v", ids)
|
||||
}
|
||||
if ids := selects("2026-10-15"); !ids[yesterdayRun] || ids[todayRun] {
|
||||
t.Fatalf("literal date selector: %+v", ids)
|
||||
}
|
||||
if ids := selects("this week"); !ids[yesterdayRun] || !ids[todayRun] {
|
||||
t.Fatalf("this week selector: %+v", ids)
|
||||
}
|
||||
// Non-selector text stays free text.
|
||||
if ids := selects("selectors"); !ids[yesterdayRun] || !ids[todayRun] {
|
||||
t.Fatalf("free text was misread as a selector: %+v", ids)
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseSearchTerms(t *testing.T) {
|
||||
now := time.Date(2026, 10, 16, 12, 0, 0, 0, time.UTC)
|
||||
cases := []struct {
|
||||
query string
|
||||
status string
|
||||
hasDate bool
|
||||
from, to string
|
||||
text string
|
||||
}{
|
||||
{query: "failed", status: "failed"},
|
||||
{query: " RUNNING ", status: "running"},
|
||||
{query: "today", hasDate: true, from: "2026-10-16 00:00:00", to: "2026-10-17 00:00:00"},
|
||||
{query: "yesterday", hasDate: true, from: "2026-10-15 00:00:00", to: "2026-10-16 00:00:00"},
|
||||
{query: "this week", hasDate: true, from: "2026-10-12 00:00:00", to: "2026-10-19 00:00:00"},
|
||||
{query: "2026-10-05", hasDate: true, from: "2026-10-05 00:00:00", to: "2026-10-06 00:00:00"},
|
||||
{query: "failed to build", text: "failed to build"},
|
||||
{query: "build", text: "build"},
|
||||
}
|
||||
for _, test := range cases {
|
||||
terms := parseSearchTerms(test.query, now)
|
||||
if terms.Status != test.status || terms.Text != test.text || terms.HasDate != test.hasDate {
|
||||
t.Fatalf("parse %q = %+v", test.query, terms)
|
||||
}
|
||||
if test.hasDate {
|
||||
if got := terms.From.Format(searchSQLTime); got != test.from {
|
||||
t.Fatalf("parse %q from = %s, want %s", test.query, got, test.from)
|
||||
}
|
||||
if got := terms.To.Format(searchSQLTime); got != test.to {
|
||||
t.Fatalf("parse %q to = %s, want %s", test.query, got, test.to)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestSearchKindFilterAndLimits(t *testing.T) {
|
||||
store, _ := searchFixture(t)
|
||||
ctx := context.Background()
|
||||
response, err := store.Search(ctx, protocol.SearchRequest{Query: "zephyr", Kind: protocol.SearchKindRun}, time.Now())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, hit := range response.Results {
|
||||
if hit.Kind != protocol.SearchKindRun {
|
||||
t.Fatalf("kind filter leaked %q", hit.Kind)
|
||||
}
|
||||
}
|
||||
if _, err := store.Search(ctx, protocol.SearchRequest{Query: "zephyr", Kind: "bogus"}, time.Now()); err == nil {
|
||||
t.Fatal("invalid kind must return a clear error")
|
||||
}
|
||||
for _, test := range []struct {
|
||||
limit int
|
||||
want int
|
||||
}{{limit: 0, want: protocol.SearchDefaultLimit}, {limit: -5, want: protocol.SearchDefaultLimit}, {limit: 1, want: 1}, {limit: 500, want: protocol.SearchMaxLimit}} {
|
||||
response, err := store.Search(ctx, protocol.SearchRequest{Query: "zephyr", Limit: test.limit}, time.Now())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if response.Cap != test.want {
|
||||
t.Fatalf("limit %d => cap %d, want %d", test.limit, response.Cap, test.want)
|
||||
}
|
||||
if len(response.Results) > response.Cap {
|
||||
t.Fatalf("limit %d returned %d results", test.limit, len(response.Results))
|
||||
}
|
||||
}
|
||||
// A cap below the candidate count drops matches and must report Truncated.
|
||||
truncated, err := store.Search(ctx, protocol.SearchRequest{Query: "zephyr", Limit: 1}, time.Now())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(truncated.Results) != 1 || !truncated.Truncated {
|
||||
t.Fatalf("result cap must truncate: %+v", truncated)
|
||||
}
|
||||
}
|
||||
|
||||
func TestSearchEdgeCases(t *testing.T) {
|
||||
store, run := searchFixture(t)
|
||||
ctx := context.Background()
|
||||
empty, err := store.Search(ctx, protocol.SearchRequest{Query: " "}, time.Now())
|
||||
if err != nil || len(empty.Results) != 0 {
|
||||
t.Fatalf("empty query: %+v %v", empty, err)
|
||||
}
|
||||
none, err := store.Search(ctx, protocol.SearchRequest{Query: "no-such-token-xyzzy"}, time.Now())
|
||||
if err != nil || len(none.Results) != 0 {
|
||||
t.Fatalf("no-match query: %+v %v", none, err)
|
||||
}
|
||||
if none.Results == nil {
|
||||
t.Fatal("results must be an empty slice, not nil")
|
||||
}
|
||||
// A run without a finish reports no duration.
|
||||
setFinishedNull(t, store, run)
|
||||
response, err := store.Search(ctx, protocol.SearchRequest{Query: "zephyr", Kind: protocol.SearchKindRun}, time.Now())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, hit := range response.Results {
|
||||
if hit.Run != nil && hit.Run.FinishedAt != "" {
|
||||
t.Fatalf("null finished_at surfaced a value: %+v", hit.Run)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestSearchSnippetBoundaries(t *testing.T) {
|
||||
if got := searchSnippet("prefix café résumé naïve suffix", "naïve"); !strings.Contains(got, "naïve") || !utf8.ValidString(got) {
|
||||
t.Fatalf("unicode snippet malformed: %q", got)
|
||||
}
|
||||
if got := searchSnippet("line one\n\tline two\r\n zephyr here", "zephyr"); strings.ContainsAny(got, "\n\r\t") || !strings.Contains(got, "zephyr") {
|
||||
t.Fatalf("whitespace snippet not normalized: %q", got)
|
||||
}
|
||||
if got := searchSnippet(strings.Repeat("é", 200)+"needle"+strings.Repeat("é", 200), "needle"); !utf8.ValidString(got) || !strings.Contains(got, "needle") {
|
||||
t.Fatalf("boundary snippet malformed: %q", got)
|
||||
}
|
||||
if got := searchSnippet("no match here", "absent"); got == "" || !utf8.ValidString(got) {
|
||||
t.Fatalf("no-match snippet must stay a clean line: %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestSearchScanBoundTruncates(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
store, err := Open(filepath.Join(t.TempDir(), "scan.db"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer store.Close()
|
||||
if _, err := store.Submit(ctx, "scan", Template([]string{"one"}, true)); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
items, _ := store.List(ctx)
|
||||
definition, _ := store.Get(ctx, items[0].ID, 1)
|
||||
// An older step that does match, and a newer step whose body alone exceeds
|
||||
// the scan budget: the budget must stop the scan and be reported.
|
||||
older, _, err := store.CreateRun(ctx, definition, "scan-older", "/work", false, nil, nil)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := store.BeginStep(ctx, older, 1, "one", nil); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := store.FinishStep(ctx, older, protocol.RunStep{Sequence: 1, Status: "completed", Outcome: "success", Target: protocol.SuccessEnd, Output: map[string]any{"summary": "needle in older step"}}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
newer, _, err := store.CreateRun(ctx, definition, "scan-newer", "/work", false, nil, nil)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := store.BeginStep(ctx, newer, 1, "one", nil); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
blob := strings.Repeat("x", protocol.SearchMaxScanBytes+1024)
|
||||
if err := store.FinishStep(ctx, newer, protocol.RunStep{Sequence: 1, Status: "completed", Outcome: "success", Target: protocol.SuccessEnd, Output: map[string]any{"blob": blob}}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
response, err := store.Search(ctx, protocol.SearchRequest{Query: "needle"}, time.Now())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !response.Truncated {
|
||||
t.Fatalf("scan bound not reported: %+v", response)
|
||||
}
|
||||
for _, hit := range response.Results {
|
||||
if hit.Field != nil {
|
||||
t.Fatalf("scan passed the bound and matched: %+v", hit.Field)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestSearchScanBoundTruncatesByRowCount proves the step-count bound is
|
||||
// observable: more recorded steps than the budget sets Truncated even though
|
||||
// every body is tiny and fits the byte budget.
|
||||
func TestSearchScanBoundTruncatesByRowCount(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
store, err := Open(filepath.Join(t.TempDir(), "scanrows.db"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer store.Close()
|
||||
if _, err := store.Submit(ctx, "scanrows", Template([]string{"one"}, true)); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
items, _ := store.List(ctx)
|
||||
definition, _ := store.Get(ctx, items[0].ID, 1)
|
||||
run, _, err := store.CreateRun(ctx, definition, "scanrows", "/work", false, nil, nil)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
tx, err := store.db.Begin()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for sequence := 1; sequence <= protocol.SearchMaxScanSteps+1; sequence++ {
|
||||
if _, err := tx.Exec(`INSERT INTO run_steps(run_id,sequence,stage_key,input_json) VALUES(?,?,?,?)`, run, sequence, "one", "{}"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
if err := tx.Commit(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
response, err := store.Search(ctx, protocol.SearchRequest{Query: "needle"}, time.Now())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !response.Truncated {
|
||||
t.Fatalf("step-count bound not reported: scanned_steps=%d total=%d", response.ScannedSteps, protocol.SearchMaxScanSteps+1)
|
||||
}
|
||||
if response.ScannedSteps != protocol.SearchMaxScanSteps {
|
||||
t.Fatalf("scanned_steps=%d, want %d", response.ScannedSteps, protocol.SearchMaxScanSteps)
|
||||
}
|
||||
if response.ScannedBytes > protocol.SearchMaxScanBytes {
|
||||
t.Fatalf("scanned_bytes=%d exceeds bound %d", response.ScannedBytes, protocol.SearchMaxScanBytes)
|
||||
}
|
||||
}
|
||||
|
||||
func setCreated(t *testing.T, store *Store, runID int64, timestamp string) {
|
||||
t.Helper()
|
||||
if _, err := store.db.Exec("UPDATE pipeline_runs SET created_at=? WHERE id=?", timestamp, runID); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
func setFinishedNull(t *testing.T, store *Store, runID int64) {
|
||||
t.Helper()
|
||||
if _, err := store.db.Exec("UPDATE pipeline_runs SET finished_at=NULL,status='running' WHERE id=?", runID); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
@@ -8,6 +8,7 @@ import (
|
||||
"fmt"
|
||||
"git.a-wan.net/alan/pipelines/internal/protocol"
|
||||
"git.a-wan.net/alan/pipelines/internal/stage"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"sync"
|
||||
@@ -50,6 +51,26 @@ CREATE TABLE IF NOT EXISTS submissions (id INTEGER PRIMARY KEY, name TEXT NOT NU
|
||||
|
||||
func (s *Store) Close() error { return s.db.Close() }
|
||||
|
||||
// OpenReadOnly opens an existing database without running schema DDL or
|
||||
// changing journal mode, so a live database can be queried read-only while its
|
||||
// server keeps writing. It fails when the file does not exist. It is used by
|
||||
// tests that inspect a real database and is never part of server startup.
|
||||
func OpenReadOnly(path string) (*Store, error) {
|
||||
if _, err := os.Stat(path); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
db, err := sql.Open("sqlite", "file:"+path+"?mode=ro")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
db.SetMaxOpenConns(1)
|
||||
if _, err := db.Exec("PRAGMA busy_timeout=5000"); err != nil {
|
||||
db.Close()
|
||||
return nil, err
|
||||
}
|
||||
return &Store{db: db, dir: filepath.Dir(path)}, nil
|
||||
}
|
||||
|
||||
// Directory is the resolved directory holding the database, used to place
|
||||
// adjacent server state such as stage artifacts.
|
||||
func (s *Store) Directory() string { return s.dir }
|
||||
|
||||
@@ -133,6 +133,11 @@ type Message struct {
|
||||
WritePaths []string `json:"write_paths,omitempty"`
|
||||
RequestKey string `json:"request_key,omitempty"`
|
||||
Workspace string `json:"workspace,omitempty"`
|
||||
// Search is the broad, read-only search traffic over pipelines, versions,
|
||||
// runs, steps, field text and artifacts. See search.go for the per-type
|
||||
// payloads and the ranking/kind/bound vocabulary.
|
||||
Search *SearchRequest `json:"search,omitempty"`
|
||||
SearchResults *SearchResponse `json:"search_results,omitempty"`
|
||||
// Chat is the brokered OpenCode chat traffic (route B). See chat.go for the
|
||||
// per-type payload structs and the wire shapes they mirror.
|
||||
ChatSessions []ChatSession `json:"chat_sessions,omitempty"`
|
||||
|
||||
@@ -0,0 +1,158 @@
|
||||
package protocol
|
||||
|
||||
// Search brokered by the Pipelines server: one query can find a pipeline, a
|
||||
// pipeline version, a run, a recorded step/visit, the text of a step field
|
||||
// (input/output/evidence/transcript) or a recorded artifact. A client sends
|
||||
// Message{Type: Search, ID: <opaque>, Search: <SearchRequest>} and the server
|
||||
// echoes the ID with a SearchResults response, matching every other exchange.
|
||||
const (
|
||||
Search = "search"
|
||||
SearchResults = "search_results"
|
||||
)
|
||||
|
||||
// Search.Kind narrowing values. They are the only narrowing mechanism and are
|
||||
// enforced server-side; an empty Kind searches every kind.
|
||||
const (
|
||||
SearchKindPipeline = "pipeline"
|
||||
SearchKindVersion = "version"
|
||||
SearchKindRun = "run"
|
||||
SearchKindStep = "step"
|
||||
SearchKindField = "field"
|
||||
SearchKindArtifact = "artifact"
|
||||
)
|
||||
|
||||
// SearchKinds is the accepted Kind vocabulary in display order; the CLI help
|
||||
// and notes are kept in step with it.
|
||||
var SearchKinds = []string{
|
||||
SearchKindPipeline,
|
||||
SearchKindVersion,
|
||||
SearchKindRun,
|
||||
SearchKindStep,
|
||||
SearchKindField,
|
||||
SearchKindArtifact,
|
||||
}
|
||||
|
||||
// SearchStatuses is the accepted set of bare status words a query may name to
|
||||
// select runs. It mirrors the run status vocabulary; a bare word selects runs
|
||||
// rather than matching as free text.
|
||||
var SearchStatuses = []string{"queued", "running", "succeeded", "failed", "canceled", "interrupted"}
|
||||
|
||||
// Search budget and cap constants. They are fixed, enforced server-side and
|
||||
// reported on SearchResponse so a client can see the bound that applied.
|
||||
const (
|
||||
// SearchDefaultLimit is the result cap when a request omits Limit (or sends 0).
|
||||
SearchDefaultLimit = 50
|
||||
// SearchMaxLimit is the largest accepted result cap; larger requests clamp.
|
||||
SearchMaxLimit = 200
|
||||
// SearchMaxScanSteps bounds how many recorded steps one query inspects while
|
||||
// scanning input/output/transcript field text.
|
||||
SearchMaxScanSteps = 2000
|
||||
// SearchMaxScanBytes bounds the total field bytes (input_json + output_json +
|
||||
// evidence_json + transcript) one query decodes and searches before it stops
|
||||
// and reports Truncated.
|
||||
SearchMaxScanBytes = 1 << 20
|
||||
)
|
||||
|
||||
// SearchRequest is the search request body. Query is the free text, a bare
|
||||
// status word or a date expression; Kind optionally narrows to one SearchKind;
|
||||
// Limit caps the returned results (0 = server default).
|
||||
type SearchRequest struct {
|
||||
Query string `json:"query"`
|
||||
Kind string `json:"kind,omitempty"`
|
||||
Limit int `json:"limit,omitempty"`
|
||||
}
|
||||
|
||||
// SearchResponse is the ranked, bounded set of matches. Results are ordered by
|
||||
// score, then recency, then a stable kind/target tie-break. Truncated reports
|
||||
// that either the result cap or the field-scan budget dropped matches;
|
||||
// ScannedSteps/ScannedBytes and ScanBoundBytes make the body-scan bound
|
||||
// observable.
|
||||
type SearchResponse struct {
|
||||
Query string `json:"query"`
|
||||
Kind string `json:"kind,omitempty"`
|
||||
Results []SearchResult `json:"results"`
|
||||
Truncated bool `json:"truncated,omitempty"`
|
||||
ScannedSteps int `json:"scanned_steps"`
|
||||
ScannedBytes int `json:"scanned_bytes"`
|
||||
Cap int `json:"cap"`
|
||||
ScanBoundBytes int `json:"scan_bound_bytes"`
|
||||
}
|
||||
|
||||
// SearchResult is one typed, navigable match. Kind selects which payload is
|
||||
// set; Target is a stable token an existing CLI command can resolve
|
||||
// (pipeline:<id>, pipeline:<id>:v<version>, run:<id>, run:<id>:<sequence>,
|
||||
// run:<id>:<sequence>:<field>, or an artifact URL). Score and MatchedField
|
||||
// expose the ranking tier and why the hit matched so a client can re-rank;
|
||||
// Recency is the hit's sortable UTC timestamp.
|
||||
type SearchResult struct {
|
||||
Kind string `json:"kind"`
|
||||
Target string `json:"target"`
|
||||
Score int `json:"score"`
|
||||
MatchedField string `json:"matched_field"`
|
||||
Recency string `json:"recency,omitempty"`
|
||||
Pipeline *SearchPipeline `json:"pipeline,omitempty"`
|
||||
Version *SearchVersion `json:"version,omitempty"`
|
||||
Run *SearchRun `json:"run,omitempty"`
|
||||
Step *SearchStep `json:"step,omitempty"`
|
||||
Field *SearchField `json:"field,omitempty"`
|
||||
Artifact *SearchArtifact `json:"artifact,omitempty"`
|
||||
}
|
||||
|
||||
// SearchPipeline is a pipeline name/identity hit: its latest version and stage count.
|
||||
type SearchPipeline struct {
|
||||
ID int64 `json:"id"`
|
||||
Name string `json:"name"`
|
||||
Version int `json:"version"`
|
||||
Stages int `json:"stages"`
|
||||
}
|
||||
|
||||
// SearchVersion is a pipeline version/source hit.
|
||||
type SearchVersion struct {
|
||||
PipelineID int64 `json:"pipeline_id"`
|
||||
Version int `json:"version"`
|
||||
ContentHash string `json:"content_hash"`
|
||||
CreatedAt string `json:"created_at"`
|
||||
}
|
||||
|
||||
// SearchRun is a run identity/metadata hit. FinishedAt and DurationSeconds are
|
||||
// omitted for a run without a recorded finish.
|
||||
type SearchRun struct {
|
||||
ID int64 `json:"id"`
|
||||
PipelineName string `json:"pipeline_name"`
|
||||
Status string `json:"status"`
|
||||
Workspace string `json:"workspace,omitempty"`
|
||||
CreatedAt string `json:"created_at"`
|
||||
FinishedAt string `json:"finished_at,omitempty"`
|
||||
DurationSeconds float64 `json:"duration_seconds,omitempty"`
|
||||
}
|
||||
|
||||
// SearchStep is a recorded step/visit hit.
|
||||
type SearchStep struct {
|
||||
RunID int64 `json:"run_id"`
|
||||
Sequence int `json:"sequence"`
|
||||
Stage string `json:"stage"`
|
||||
Visit int `json:"visit,omitempty"`
|
||||
Outcome string `json:"outcome,omitempty"`
|
||||
Target string `json:"target,omitempty"`
|
||||
Diagnostic string `json:"diagnostic,omitempty"`
|
||||
}
|
||||
|
||||
// SearchField is a stage input/output/evidence/transcript text hit with a
|
||||
// snippet around the first match.
|
||||
type SearchField struct {
|
||||
RunID int64 `json:"run_id"`
|
||||
Sequence int `json:"sequence"`
|
||||
Field string `json:"field"`
|
||||
Snippet string `json:"snippet"`
|
||||
}
|
||||
|
||||
// SearchArtifact is a recorded artifact hit; contents are never read, only the
|
||||
// recorded metadata.
|
||||
type SearchArtifact struct {
|
||||
RunID int64 `json:"run_id"`
|
||||
Sequence int `json:"sequence"`
|
||||
Name string `json:"name"`
|
||||
ContentType string `json:"content_type"`
|
||||
SizeBytes int64 `json:"size_bytes"`
|
||||
URL string `json:"url"`
|
||||
}
|
||||
@@ -0,0 +1,82 @@
|
||||
package protocol
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestSearchRequestResponseRoundTrip(t *testing.T) {
|
||||
request := Message{Type: Search, ID: "req-1", Search: &SearchRequest{Query: "release", Kind: SearchKindRun, Limit: 5}}
|
||||
encoded, err := json.Marshal(request)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var decoded Message
|
||||
if err := json.Unmarshal(encoded, &decoded); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if decoded.Type != Search || decoded.ID != "req-1" || decoded.Search == nil {
|
||||
t.Fatalf("request envelope lost: %+v", decoded)
|
||||
}
|
||||
if decoded.Search.Query != "release" || decoded.Search.Kind != SearchKindRun || decoded.Search.Limit != 5 {
|
||||
t.Fatalf("search request lost: %+v", decoded.Search)
|
||||
}
|
||||
|
||||
response := Message{Type: SearchResults, ID: "req-1", SearchResults: &SearchResponse{
|
||||
Query: "release",
|
||||
Results: []SearchResult{{Kind: SearchKindPipeline, Target: "pipeline:1", Score: 3, MatchedField: "name", Pipeline: &SearchPipeline{ID: 1, Name: "release", Version: 2, Stages: 3}}},
|
||||
Cap: 50,
|
||||
ScanBoundBytes: SearchMaxScanBytes,
|
||||
}}
|
||||
encoded, err = json.Marshal(response)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var echoed Message
|
||||
if err := json.Unmarshal(encoded, &echoed); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if echoed.ID != "req-1" || echoed.Type != SearchResults || echoed.SearchResults == nil {
|
||||
t.Fatalf("response envelope or ID echo lost: %+v", echoed)
|
||||
}
|
||||
got := echoed.SearchResults
|
||||
if got.Query != "release" || got.Cap != 50 || got.ScanBoundBytes != SearchMaxScanBytes || len(got.Results) != 1 {
|
||||
t.Fatalf("search response lost: %+v", got)
|
||||
}
|
||||
hit := got.Results[0]
|
||||
if hit.Pipeline == nil || hit.Pipeline.Name != "release" || hit.Pipeline.Version != 2 || hit.Pipeline.Stages != 3 || hit.Score != 3 || hit.MatchedField != "name" || hit.Target != "pipeline:1" {
|
||||
t.Fatalf("pipeline payload lost: %+v", hit)
|
||||
}
|
||||
}
|
||||
|
||||
func TestSearchResultOmitsAbsentPayloadsAndEmptyKind(t *testing.T) {
|
||||
encoded, err := json.Marshal(SearchResult{Kind: SearchKindRun, Target: "run:1", Score: 1, MatchedField: "status"})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, key := range []string{"pipeline", "version", "run", "step", "field", "artifact", "recency"} {
|
||||
if strings.Contains(string(encoded), `"`+key+`":`) {
|
||||
t.Fatalf("absent %s payload must be omitted: %s", key, encoded)
|
||||
}
|
||||
}
|
||||
var decoded SearchResult
|
||||
if err := json.Unmarshal([]byte(`{"kind":"run","target":"run:1","score":1,"matched_field":"status"}`), &decoded); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if decoded.Pipeline != nil || decoded.Version != nil || decoded.Run != nil || decoded.Step != nil || decoded.Field != nil || decoded.Artifact != nil {
|
||||
t.Fatalf("absent payloads must decode nil: %+v", decoded)
|
||||
}
|
||||
|
||||
response := Message{Type: SearchResults, ID: "x", SearchResults: &SearchResponse{Query: "q", Results: []SearchResult{}}}
|
||||
encoded, err = json.Marshal(response)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if strings.Contains(string(encoded), `"kind"`) {
|
||||
t.Fatalf("empty response kind must be omitted: %s", encoded)
|
||||
}
|
||||
if !strings.Contains(string(encoded), `"results":[]`) {
|
||||
t.Fatalf("results must always be present: %s", encoded)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,105 @@
|
||||
package server
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"net/http/httptest"
|
||||
"path/filepath"
|
||||
"strconv"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"git.a-wan.net/alan/pipelines/internal/cli"
|
||||
"git.a-wan.net/alan/pipelines/internal/pipeline"
|
||||
"git.a-wan.net/alan/pipelines/internal/protocol"
|
||||
)
|
||||
|
||||
// TestCLISearchEndToEndEveryKind seeds a store whose pipeline, source, run,
|
||||
// stage, field text and artifact all contain one token, then runs the CLI
|
||||
// search command over a real Handler and asserts the server returned one match
|
||||
// of every kind and rejects an unknown --type rather than ignoring it.
|
||||
func TestCLISearchEndToEndEveryKind(t *testing.T) {
|
||||
t.Setenv("PIPELINES_HARNESS_CONFIG", filepath.Join(t.TempDir(), "harnesses.json"))
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
store, err := pipeline.Open(filepath.Join(t.TempDir(), "search-e2e.db"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer store.Close()
|
||||
if _, err := store.Submit(ctx, "Zephyr Build", pipeline.Template([]string{"zephyr"}, true)); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
items, err := store.List(ctx)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
definition, err := store.Get(ctx, items[0].ID, 1)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
run, _, err := store.CreateRun(ctx, definition, "zephyr-key", "/work/zephyr", false, nil, map[string]any{"prompt": "zephyr task"})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := store.BeginStep(ctx, run, 1, "zephyr", map[string]any{"prompt": "zephyr task"}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := store.FinishStep(ctx, run, protocol.RunStep{
|
||||
Sequence: 1, Status: "completed", Outcome: "success", Target: protocol.SuccessEnd,
|
||||
Output: map[string]any{"summary": "zephyr completed"},
|
||||
Artifacts: []protocol.StageArtifact{{Name: "zephyr.txt", SizeBytes: 12, ContentType: "text/plain", URL: "/artifacts/" + strconv.FormatInt(run, 10) + "/1/zephyr.txt"}},
|
||||
}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
handler := Handler(ctx, store)
|
||||
defer func() { cancel(); WaitRuns(handler) }()
|
||||
http := httptest.NewServer(handler)
|
||||
defer http.Close()
|
||||
url := "ws" + strings.TrimPrefix(http.URL, "http") + "/connect"
|
||||
|
||||
runCLI := func(args ...string) (string, error) {
|
||||
var out bytes.Buffer
|
||||
app := cli.App{Out: &out, Err: &out, In: strings.NewReader("")}
|
||||
err := app.Run(ctx, append([]string{"--url", url}, args...))
|
||||
return out.String(), err
|
||||
}
|
||||
output, err := runCLI("search", "zephyr", "--json")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var response protocol.SearchResponse
|
||||
if err := json.Unmarshal([]byte(output), &response); err != nil {
|
||||
t.Fatal(err, output)
|
||||
}
|
||||
kinds := map[string]bool{}
|
||||
for _, hit := range response.Results {
|
||||
kinds[hit.Kind] = true
|
||||
if hit.Target == "" || hit.MatchedField == "" || hit.Score == 0 {
|
||||
t.Fatalf("hit lacks ranking/navigation context: %+v", hit)
|
||||
}
|
||||
}
|
||||
for _, kind := range protocol.SearchKinds {
|
||||
if !kinds[kind] {
|
||||
t.Fatalf("kind %q missing from end-to-end search: %s", kind, output)
|
||||
}
|
||||
}
|
||||
|
||||
if output, err := runCLI("search", "zephyr", "--type", "run", "--json"); err != nil {
|
||||
t.Fatal(err)
|
||||
} else {
|
||||
var filtered protocol.SearchResponse
|
||||
if err := json.Unmarshal([]byte(output), &filtered); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, hit := range filtered.Results {
|
||||
if hit.Kind != protocol.SearchKindRun {
|
||||
t.Fatalf("kind filter leaked %q", hit.Kind)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if _, err := runCLI("search", "zephyr", "--type", "bogus"); err == nil || !strings.Contains(err.Error(), "unknown search kind") {
|
||||
t.Fatalf("invalid kind must error, got %v", err)
|
||||
}
|
||||
}
|
||||
@@ -192,6 +192,22 @@ func HandlerWithOptions(ctx context.Context, options Options, stores ...*pipelin
|
||||
} else {
|
||||
response.Pipelines = items
|
||||
}
|
||||
} else if request.Type == protocol.Search {
|
||||
if len(stores) == 0 {
|
||||
response.Type = protocol.Error
|
||||
response.Error = "search unavailable: no store"
|
||||
} else if request.Search == nil {
|
||||
response.Type = protocol.Error
|
||||
response.Error = "search requires a query"
|
||||
} else {
|
||||
response.Type = protocol.SearchResults
|
||||
var err error
|
||||
response.SearchResults, err = stores[0].Search(ctx, *request.Search, time.Now())
|
||||
if err != nil {
|
||||
response.Type = protocol.Error
|
||||
response.Error = err.Error()
|
||||
}
|
||||
}
|
||||
} else if runs != nil && (request.Type == protocol.RunPipeline || request.Type == protocol.RetryRun || request.Type == protocol.ListRuns || request.Type == protocol.GetRun || request.Type == protocol.CancelRun || request.Type == protocol.GetRunEvidence) {
|
||||
var err error
|
||||
switch request.Type {
|
||||
|
||||
Reference in New Issue
Block a user