retention: explicit run-history and artifact retention
Add `pipelines retention report` (strictly read-only) and `pipelines retention apply` (explicit, approval-gated removal) with an age and/or count policy, protected-run rules, atomic single-transaction removal and honest database space reclaim. Nothing prunes automatically.
This commit is contained in:
+106
@@ -269,6 +269,7 @@ Submitting activates included schedules, so review them before publishing.
|
||||
| Connection and phone settings | `connection --json`, `ping --json` |
|
||||
| Server/ntfy status popup | `status --json` |
|
||||
| Usage statistics/charts | `stats --json` |
|
||||
| Run-history retention | `retention report --keep N --json`, `retention apply --keep N --yes` |
|
||||
| Legacy operational diagnostics | managed `logs`, `doctor` |
|
||||
|
||||
The TUI does not spawn CLI processes for each request. Both front ends share
|
||||
@@ -607,6 +608,111 @@ for panning and `c` to center. Secondary progress/input/evidence/execution secti
|
||||
are collapsed initially. Small inactive graph previews collapse instead of showing
|
||||
severed boxes. See [UI/UX review](ui-ux-review.md) for screen-by-screen feedback.
|
||||
|
||||
## Run retention
|
||||
|
||||
Nothing prunes run history automatically. `pipelines retention report` and
|
||||
`pipelines retention apply` are the only commands that remove run history or run
|
||||
artifacts, they are local and operator-controlled, and they never run as a side
|
||||
effect of any other command, server startup, `newRunner`, the scheduler,
|
||||
`list_runs`, `stats`, `runs` or `inspect`. Pipeline definitions and their
|
||||
immutable versions are never touched: retention is about run history and the
|
||||
artifacts runs produced.
|
||||
|
||||
```sh
|
||||
# Read-only: show exactly what a policy would remove.
|
||||
pipelines retention report --db ~/.local/share/pipelines/data/pipelines.db --keep 200 --json
|
||||
|
||||
# Explicit removal of that same set (artifacts go with their run).
|
||||
pipelines retention apply --db ~/.local/share/pipelines/data/pipelines.db --keep 200 --yes --json
|
||||
```
|
||||
|
||||
Both commands accept `--older-than DURATION`, `--keep N`, `--db FILE`,
|
||||
`--artifacts DIR` and `--json`; `apply` also accepts `--yes`. At least one policy
|
||||
flag is required; an unknown subcommand, a negative count and a non-positive age
|
||||
are refused. `--db` must name an existing database and is never created. With only
|
||||
`--db`, the artifacts root defaults to `artifacts/` beside the database, matching
|
||||
`cmd/pipelines-server/main.go`; `--artifacts` overrides it. The commands are
|
||||
dispatched before server URL resolution, so they work without a running server
|
||||
(for example `--url not-a-server`) and never require, restart or redeploy the
|
||||
deployed server.
|
||||
|
||||
### Policy
|
||||
|
||||
- `--older-than` selects runs created **strictly before** `now - DURATION`. The
|
||||
age is a rolling UTC duration, not a calendar-day boundary. It accepts Go
|
||||
durations plus a count of days or weeks (`720h`, `30d`, `12w`).
|
||||
- `--keep N` retains the newest `N` runs by run id and selects ranks `N+1` and
|
||||
beyond. `--keep 0` selects every eligible run; a count larger than the history
|
||||
selects none.
|
||||
- Both together are the conservative **intersection**: a run is selected only
|
||||
when it is both older than the age bound and outside the newest `N`.
|
||||
|
||||
### Protected runs
|
||||
|
||||
A policy-selected run is nonetheless never removed when it:
|
||||
|
||||
- is `queued` or `running`;
|
||||
- is `waiting` on a person (a `waiting` run or a `waiting` step);
|
||||
- has any step with `agent_active=1` (cleanup still active); or
|
||||
- is referenced by a run that will remain through retry provenance
|
||||
(`run_retries.original_run_id`), propagated transitively along a retry chain.
|
||||
If both an original and its retry are selected, no retained link remains and
|
||||
both may be removed.
|
||||
|
||||
Protected runs are reported with their reason and their artifacts are never
|
||||
counted as reclaimable. Free-text run ids in prompts are not treated as
|
||||
dependencies; retry provenance is the only structured cross-run dependency the
|
||||
store records.
|
||||
|
||||
If a selected run becomes protected between the report and the delete — a new
|
||||
retry reference, a waiting/active state or new agent activity — `apply` aborts
|
||||
with the run ids and reasons, deletes no rows or artifacts, and un-stages
|
||||
anything already renamed. A removal never half-completes.
|
||||
|
||||
### What apply removes and reclaims
|
||||
|
||||
`apply` stages only the would-remove run directories into one
|
||||
`artifacts/.retention-staging-<timestamp>-<pid>/` directory (a same-filesystem
|
||||
rename), deletes the rows for those runs in one immediate write transaction
|
||||
(`step_notifications`, `step_file_changes`, `step_control`, `step_artifacts`,
|
||||
`step_verdicts`, `run_session_owners`, `run_steps`, `run_retries`, then
|
||||
`pipeline_runs`, batched at 500 ids), commits, and removes the staging directory.
|
||||
A symlink at `<root>/<id>` is removed as a link without touching its target; a
|
||||
regular file at `<root>/<id>` is refused. All regular files under a removed run
|
||||
directory are counted, including files beyond the server's 200-file/8-MiB/64-MiB
|
||||
inventory caps, which bound the recorded inventory, not disk ownership.
|
||||
|
||||
The database is then compacted with one `VACUUM` and one
|
||||
`PRAGMA wal_checkpoint(TRUNCATE)`, retried at most three short times on
|
||||
`SQLITE_BUSY`. `database_bytes_reclaimed` and `space_reclaimed_bytes` are
|
||||
measured, not estimated. If `VACUUM` fails the command returns an error, sets
|
||||
`vacuum_error` and claims no reclaimed database pages; a later `apply` with
|
||||
nothing left to remove can retry the `VACUUM`. If staging cleanup fails the
|
||||
report names the staging path and remaining bytes/files and does not count them
|
||||
as reclaimed.
|
||||
|
||||
### Measured example (this machine)
|
||||
|
||||
Reported before, applied, reported after, against the live database while its
|
||||
server kept running (server PID unchanged):
|
||||
|
||||
```text
|
||||
report --keep 200: total 279 · would remove 79 · protected 0 · retained 200
|
||||
artifacts 537 files · 499046205 bytes (476 MiB)
|
||||
database 41426944 bytes · free 4096 · reclaimable 499050301
|
||||
apply --keep 200: removed 79 runs · 537 files · 499046205 bytes
|
||||
vacuumed · database 41426944 -> 26898432 bytes
|
||||
database_bytes_reclaimed 14528512 · space_reclaimed_bytes 513574717
|
||||
report --keep 200: total 200 · would remove 0 · retained 200
|
||||
pipelines 18 · pipeline_versions 39 (unchanged)
|
||||
```
|
||||
|
||||
Independent observation: `du -sk` of the artifacts root fell from 3213324 to
|
||||
2724872 KiB, the database file from 41426944 to 26898432 bytes, and the logs
|
||||
stayed at 18404 KiB. A retained run's artifact still served (HTTP 200) while a
|
||||
removed run's URL returned 404. The measured numbers above come from that
|
||||
before/apply/after output, not an estimate.
|
||||
|
||||
## Verification scope
|
||||
|
||||
CLI integration tests exercise draft/validate/submit/history/conflict handling and
|
||||
|
||||
Reference in New Issue
Block a user