retention: report removed runs and follow a symlinked artifacts root

Fix two honesty gaps in run-history retention that the implementer stage
left uncommitted on top of 14175eb:

- Populate RemovedRuns, RemovedRunCount, RemovedArtifactFiles/Bytes and
  Applied right after the row transaction commits, before staging cleanup.
  A cleanup failure must not report a successful row/artifact removal as
  zero removed, and staging leftovers are excluded from SpaceReclaimedBytes.
  A failed VACUUM after removal now reports the artifact bytes actually
  removed while claiming no database pages.
- Stage artifacts through os.Stat instead of os.Lstat so a symlinked
  artifacts root is followed: the report sized that directory, so apply
  removes through it too instead of counting bytes it never deleted.

Tests cover post-commit staging-cleanup failure reporting, staging through a
symlinked artifacts root (including no staging directory left in the link
target), and an honest reclaim failure after removal.

Checked documentation surfaces:
- docs/cli.md: measured example refreshed with the current before/apply/after
  output and the symlink/staging/VACUUM reporting described accurately.
- README.md: retention prose (explicit, never automatic, protected runs,
  artifacts go with the run) is unchanged and still correct.
- internal/author/skill/references/operations.md: operator-only retention
  summary describes the same behaviour; no user-facing surface changed, so
  no reference update was needed.
- Packaged templates: retention is a CLI/store capability with no template
  change, so nothing to update. internal/author/install.go inventory
  unchanged (no reference added or renamed).

Verification: gofmt clean on the touched Go files, git diff --check clean,
go vet ./internal/pipeline clean, and go test ./internal/pipeline -run
'TestRetention' -count=1 passing.
This commit is contained in:
2026-10-17 16:19:39 +01:00
parent 14175eba70
commit 4908452a74
3 changed files with 240 additions and 30 deletions
+28 -19
View File
@@ -678,39 +678,48 @@ rename), deletes the rows for those runs in one immediate write transaction
`step_verdicts`, `run_session_owners`, `run_steps`, `run_retries`, then
`pipeline_runs`, batched at 500 ids), commits, and removes the staging directory.
A symlink at `<root>/<id>` is removed as a link without touching its target; a
regular file at `<root>/<id>` is refused. All regular files under a removed run
directory are counted, including files beyond the server's 200-file/8-MiB/64-MiB
inventory caps, which bound the recorded inventory, not disk ownership.
regular file at `<root>/<id>` is refused. A symlinked artifacts root is followed
consistently: the report sizes it and apply stages through the same directory, so
the artifacts really go. All regular files under a removed run directory are
counted, including files beyond the server's 200-file/8-MiB/64-MiB inventory
caps, which bound the recorded inventory, not disk ownership.
The database is then compacted with one `VACUUM` and one
`PRAGMA wal_checkpoint(TRUNCATE)`, retried at most three short times on
`SQLITE_BUSY`. `database_bytes_reclaimed` and `space_reclaimed_bytes` are
measured, not estimated. If `VACUUM` fails the command returns an error, sets
`vacuum_error` and claims no reclaimed database pages; a later `apply` with
nothing left to remove can retry the `VACUUM`. If staging cleanup fails the
report names the staging path and remaining bytes/files and does not count them
as reclaimed.
`vacuum_error` and claims no reclaimed database pages while still reporting the
artifact bytes it did remove; a later `apply` with nothing left to remove can
retry the `VACUUM`. If staging cleanup fails the report still gives the removed
run count and totals, names the staging path and the remaining bytes/files, and
does not count the leftovers as reclaimed.
### Measured example (this machine)
Reported before, applied, reported after, against the live database while its
server kept running (server PID unchanged):
server kept running (server PID 99492 unchanged throughout; the server was also
creating new runs concurrently, which is why the artifact `du` moved
independently):
```text
report --keep 200: total 279 · would remove 79 · protected 0 · retained 200
artifacts 537 files · 499046205 bytes (476 MiB)
database 41426944 bytes · free 4096 · reclaimable 499050301
apply --keep 200: removed 79 runs · 537 files · 499046205 bytes
vacuumed · database 41426944 -> 26898432 bytes
database_bytes_reclaimed 14528512 · space_reclaimed_bytes 513574717
report --keep 200: total 207 · would remove 7 · protected 0 · retained 200
artifacts 418 files · 279668627 bytes (266.7 MiB)
database 27897856 bytes · free 0 · reclaimable 279668627
apply --keep 200: removed 7 runs · 418 files · 266.7 MiB
vacuumed · database reclaimed 1.5 MiB · now 25.1 MiB · free 0
total space reclaimed 268.2 MiB
report --keep 200: total 200 · would remove 0 · retained 200
pipelines 18 · pipeline_versions 39 (unchanged)
artifacts 0 files · 0 bytes
database 26353664 bytes · free 0
pipelines 19 · pipeline_versions 45 (unchanged)
```
Independent observation: `du -sk` of the artifacts root fell from 3213324 to
2724872 KiB, the database file from 41426944 to 26898432 bytes, and the logs
stayed at 18404 KiB. A retained run's artifact still served (HTTP 200) while a
removed run's URL returned 404. The measured numbers above come from that
Independent observation: `du -sk` of the artifacts root fell from 2815316 to
2572456 KiB (the running server added artifacts during the same window), the
database file from 27897856 to 26353664 bytes, the WAL was truncated to zero and
the logs stayed at 18404 KiB. A retained run's artifact still served (HTTP 200,
`/artifacts/100/1/00_run94_artifacts.png`) while a removed run's URL returned 404
(`/artifacts/83/1/...`). The measured numbers above come from that
before/apply/after output, not an estimate.
## Verification scope