Merge memory-watched app verification gate
This commit is contained in:
@@ -0,0 +1,19 @@
|
||||
# Verification gate and app workflow documentation
|
||||
|
||||
`tool/check_full.sh` now calls the bounded analyzer and memory-watchdog wrappers,
|
||||
from the app directory regardless of the caller's working directory. It preserves
|
||||
their actual nonzero statuses instead of accepting success-looking text from a
|
||||
failed command. Wrapper summaries remain in `build/check/`; they name their own
|
||||
full logs. Analysis failure stops before tests run, including a missing wrapper.
|
||||
|
||||
`pipelines/README.md` now directs app tasks to the registered Mobile workflow and
|
||||
read-only inspection of its current version. The checked-in flutter-fix,
|
||||
flutter-design and flutter-preview definitions are labelled historical, and the
|
||||
README distinguishes code, checks, server landing, preview publication and the
|
||||
installed web root. No pipeline definition was edited or published.
|
||||
|
||||
Checks: five Python orchestration fixtures pass; they cover successful wrapper
|
||||
calls from the repository root, misleading success text on analysis/test failure,
|
||||
a watchdog kill status, and a missing wrapper. `sh -n tool/check_full.sh` and
|
||||
`git diff --check` pass. No real Flutter suite was run in this lane; the manager
|
||||
runs integrated analysis and the full memory-watched suite after merging.
|
||||
+53
-26
@@ -1,31 +1,58 @@
|
||||
# Flutter app pipelines
|
||||
# Flutter app workflows
|
||||
|
||||
The published Pipelines workflows for `app/`, kept here so their history lives
|
||||
with the code they work on. Publish a change with
|
||||
`pipelines validate pipelines/<file>.md` then
|
||||
`pipelines submit pipelines/<file>.md --name "<Name>"`.
|
||||
Use the registered **Mobile** pipeline for one app behaviour, bug or visual
|
||||
change. Its current flow prepares the worktree, implements the change, reviews
|
||||
the diff, reviews rendered UI when affected, and commits a candidate for the
|
||||
server's delivery checks and landing gate.
|
||||
|
||||
| Pipeline | File | One run does |
|
||||
| --- | --- | --- |
|
||||
| Flutter fix | `flutter-fix.md` | One behaviour or bug: Ground → Red → Code → Review → Verify → Commit |
|
||||
| Flutter design | `flutter-design.md` | One visual defect on one screen: Capture → Compare → Code → Recapture → UIReview → Verify → Commit |
|
||||
| Flutter preview | `flutter-preview.md` | Publish main to the preview: Gate → Publish → Smoke |
|
||||
Inspect the live definition before a run; the server's registered version is
|
||||
authoritative:
|
||||
|
||||
Running one item:
|
||||
```sh
|
||||
pipelines list
|
||||
pipelines show Mobile
|
||||
```
|
||||
|
||||
1. From the repository root, `git worktree add -b flutter/<slug> tmp/flutter-<slug> main`
|
||||
and `flutter pub get` in its `app/`.
|
||||
2. `pipelines run "Flutter fix" --workspace tmp/flutter-<slug>/app --prompt "<one behaviour, with what you already know>"`.
|
||||
Flutter design takes the screen and its routes; Flutter preview runs in the main
|
||||
checkout's `app/` and takes a short name for the rollback directory.
|
||||
3. A run ends with a local commit on its branch. The manager reads the diff,
|
||||
merges the branch into main with `--no-ff`, runs `sh tool/check_full.sh` on main,
|
||||
and publishes with Flutter preview.
|
||||
Run one item in its own repository-root worktree (not inside `app/`), keeping
|
||||
other worktrees available for independent work:
|
||||
|
||||
Every stage runs on `openai/gpt-6-luna` (app/AGENTS.md). Stages share the repository's
|
||||
own tools: `tool/check_full.sh` (analyze plus the full suite, passing only on
|
||||
`All tests passed!`), `tool/shoot_real_app.mjs` (phone and desktop captures of the
|
||||
real app against the loopback fixture) and `tool/publish_preview.sh` (build, then
|
||||
swap `build/web` by rename with a rollback directory). No stage restarts a server,
|
||||
uses the shared Pipelines server for captures, uses `git stash`, or adds AI
|
||||
attribution to commits.
|
||||
```sh
|
||||
git worktree add -b flutter/<slug> tmp/flutter-<slug> main
|
||||
cd tmp/flutter-<slug>
|
||||
(cd app && /Users/alan/development/flutter/bin/flutter pub get)
|
||||
pipelines run Mobile "<one observable change and its acceptance criteria>"
|
||||
```
|
||||
|
||||
Use Luna for pipeline stages unless the user requests another model. Verify
|
||||
the exact provider/model in `pipelines show Mobile`; do not copy model IDs from
|
||||
the old files here. Put task-specific requirements in the run prompt. Creating
|
||||
or publishing a workflow version requires an explicit pipeline-change request.
|
||||
|
||||
For evidence and recovery, use `pipelines inspect <run-id> --json`. A failed run
|
||||
keeps its branch and edits; inspect those before retrying or taking over. A
|
||||
successful model stage is not proof of landing: verify the delivery-check
|
||||
results and main's ancestry after the server's landing gate finishes. **App
|
||||
change** is the separate Caplens workflow; it is not the Flutter client workflow.
|
||||
|
||||
## Verification and delivery
|
||||
|
||||
From `app/`, use `tool/memwatch/full.sh test/<area>` and
|
||||
`tool/analyze.sh <changed paths>` while editing. For integrated acceptance,
|
||||
`sh tool/check_full.sh` runs bounded analysis followed by the full suite through
|
||||
the memory watchdog. It fails on the actual command exit status and records
|
||||
wrapper summaries under `build/check/`. Never run a bare `flutter test`.
|
||||
|
||||
Rendered changes also need interaction and inspected captures with
|
||||
`tool/capture/` (see its README); test-device requirements are in `app/AGENTS.md`.
|
||||
Code, checks, landing, publication and device installation are separate steps.
|
||||
Publish only when requested, preserve a rollback, and do not restart shared
|
||||
services over active work. `tool/publish_preview.sh` updates the checkout preview;
|
||||
the installed web root uses the installation's delivery path. APK publication
|
||||
uses `tool/publish_apk.sh` or the installation's `--apk` flow.
|
||||
|
||||
## Historical definitions
|
||||
|
||||
`flutter-fix.md`, `flutter-design.md` and `flutter-preview.md` preserve earlier
|
||||
workflow definitions. They are not the current registered Mobile definition and
|
||||
must not be submitted just to start or repair an ordinary run. In particular,
|
||||
their names, model IDs, prompts and manual verification steps may be stale.
|
||||
|
||||
+24
-21
@@ -1,25 +1,28 @@
|
||||
#!/bin/sh
|
||||
# Landing gate for app/ changes: analyze, then the full test suite.
|
||||
#
|
||||
# Usage (from app/): sh tool/check_full.sh
|
||||
# Exits 0 only when `flutter analyze` reports no issues and the full suite
|
||||
# prints `All tests passed!`. A piped `tail` cannot hide a failure here: the
|
||||
# verdict comes from the suite's own final line, which is echoed with the
|
||||
# totals. Logs go to build/check/ for the caller's evidence.
|
||||
set -u
|
||||
FLUTTER="${FLUTTER:-flutter}"
|
||||
# Usage: sh app/tool/check_full.sh (or sh tool/check_full.sh from app/).
|
||||
# The wrappers bound output and test memory. Trust their exit statuses, never
|
||||
# success-looking text in a failed command's log. Keep their summaries in
|
||||
# build/check/; each summary also names the underlying full log.
|
||||
set -eu
|
||||
cd "$(dirname "$0")/.."
|
||||
mkdir -p build/check
|
||||
"$FLUTTER" analyze > build/check/analyze.log 2>&1
|
||||
if ! grep -q "No issues found" build/check/analyze.log; then
|
||||
grep -E " • " build/check/analyze.log
|
||||
echo "CHECK FAILED: flutter analyze reported issues (build/check/analyze.log)"
|
||||
exit 1
|
||||
fi
|
||||
"$FLUTTER" test > build/check/test.log 2>&1
|
||||
verdict=$(grep -E "All tests passed!|Some tests failed" build/check/test.log | tail -1)
|
||||
if ! echo "$verdict" | grep -q "All tests passed!"; then
|
||||
grep -E "\[E\]" build/check/test.log | head -20
|
||||
echo "CHECK FAILED: $verdict (build/check/test.log)"
|
||||
exit 1
|
||||
fi
|
||||
echo "CHECK PASSED: analyze clean; $verdict"
|
||||
|
||||
check() {
|
||||
name=$1
|
||||
log=$2
|
||||
shift 2
|
||||
if "$@" > "$log" 2>&1; then
|
||||
cat "$log"
|
||||
else
|
||||
status=$?
|
||||
cat "$log"
|
||||
echo "CHECK FAILED: $name exited $status ($log)"
|
||||
return "$status"
|
||||
fi
|
||||
}
|
||||
|
||||
check analyze build/check/analyze.log tool/analyze.sh
|
||||
check test build/check/test.log tool/memwatch/full.sh test
|
||||
echo "CHECK PASSED: analyze and memory-watched test suite passed"
|
||||
|
||||
@@ -0,0 +1,80 @@
|
||||
"""Exercise the landing gate's orchestration without launching Flutter tests."""
|
||||
import os
|
||||
from pathlib import Path
|
||||
import shutil
|
||||
import subprocess
|
||||
import tempfile
|
||||
import unittest
|
||||
|
||||
|
||||
class CheckFullTests(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.temp = tempfile.TemporaryDirectory(prefix="check-full-")
|
||||
self.addCleanup(self.temp.cleanup)
|
||||
self.root = Path(self.temp.name)
|
||||
self.app = self.root / "app"
|
||||
self.tool = self.app / "tool"
|
||||
(self.tool / "memwatch").mkdir(parents=True)
|
||||
shutil.copyfile(Path(__file__).with_name("check_full.sh"),
|
||||
self.tool / "check_full.sh")
|
||||
self.wrapper("analyze.sh", "analyze", "ANALYZE")
|
||||
self.wrapper("memwatch/full.sh", "memwatch", "TEST")
|
||||
|
||||
def wrapper(self, path, name, prefix):
|
||||
script = self.tool / path
|
||||
script.write_text(
|
||||
"#!/bin/sh\n"
|
||||
f'printf "%s\\n" "{name}:$*:$PWD" >> "$CALL_LOG"\n'
|
||||
f'printf "%s\\n" "${{{prefix}_OUTPUT:-bounded summary}}"\n'
|
||||
f'exit "${{{prefix}_STATUS:-0}}"\n')
|
||||
script.chmod(0o755)
|
||||
|
||||
def run_gate(self, **overrides):
|
||||
env = dict(os.environ, CALL_LOG=str(self.root / "calls"), **overrides)
|
||||
result = subprocess.run(["sh", str(self.tool / "check_full.sh")],
|
||||
cwd=self.root, env=env, capture_output=True,
|
||||
text=True)
|
||||
calls = self.root / "calls"
|
||||
return result, calls.read_text().splitlines() if calls.exists() else []
|
||||
|
||||
def test_success_uses_both_wrappers_from_app_and_keeps_summaries(self):
|
||||
result, calls = self.run_gate()
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
self.assertEqual(calls, [f"analyze::{self.app}",
|
||||
f"memwatch:test:{self.app}"])
|
||||
self.assertIn("CHECK PASSED", result.stdout)
|
||||
for name in ("analyze", "test"):
|
||||
self.assertEqual((self.app / f"build/check/{name}.log").read_text(),
|
||||
"bounded summary\n")
|
||||
|
||||
def test_analysis_failure_cannot_be_hidden_by_success_text(self):
|
||||
result, calls = self.run_gate(ANALYZE_STATUS="14",
|
||||
ANALYZE_OUTPUT="No issues found")
|
||||
self.assertEqual(result.returncode, 14)
|
||||
self.assertEqual(len(calls), 1)
|
||||
self.assertNotIn("CHECK PASSED", result.stdout)
|
||||
self.assertFalse((self.app / "build/check/test.log").exists())
|
||||
|
||||
def test_test_failure_cannot_be_hidden_by_success_text(self):
|
||||
result, calls = self.run_gate(TEST_STATUS="7",
|
||||
TEST_OUTPUT="All tests passed!")
|
||||
self.assertEqual(result.returncode, 7)
|
||||
self.assertEqual(len(calls), 2)
|
||||
self.assertIn("CHECK FAILED: test exited 7", result.stdout)
|
||||
self.assertNotIn("CHECK PASSED", result.stdout)
|
||||
|
||||
def test_watchdog_kill_status_is_preserved(self):
|
||||
result, _ = self.run_gate(TEST_STATUS="137", TEST_OUTPUT="KILLED")
|
||||
self.assertEqual(result.returncode, 137)
|
||||
self.assertNotIn("CHECK PASSED", result.stdout)
|
||||
|
||||
def test_missing_wrapper_fails_instead_of_claiming_success(self):
|
||||
(self.tool / "memwatch/full.sh").unlink()
|
||||
result, calls = self.run_gate()
|
||||
self.assertNotEqual(result.returncode, 0)
|
||||
self.assertEqual(len(calls), 1)
|
||||
self.assertNotIn("CHECK PASSED", result.stdout)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
Reference in New Issue
Block a user