james.maesandClaude Opus 4.7
268878b29d
Ships the four shell scripts the sweep manifest references plus the R4-L7 contract test that 08-01's threat model deferred (T-08-05). Scripts: - scripts/shipping-gate.sh: invokes the debug-built jarvis-eval all binary; --live requires JARVIS_LIVE_EVAL=1 (D-04 dual-gate). Debug configuration is intentional: MCPCrashRunner consumes #if DEBUG-gated test seams on MCPClient/MCPServerHandle, so a release build of jarvis-eval would not compile. - scripts/promote-replay-session.sh: D-09 operator helper; copies recorded SQLite + writes sidecar JSON with temperature / model / replaySchemaVersion so DriftClassifier knows whether nondeterministicUnderSampling exclusions apply. - scripts/check-corpus-secrets.sh: D-07 pre-commit guard greps Corpora/ for sk-ant-, AKIA, ghp_, sk- patterns. Verified on positive (planted-secret) + negative (clean Corpora) inputs. - scripts/precommit-template.sh: committed template the operator symlinks to .git/hooks/pre-commit; updates flow through automatically. Worktree's pre-commit hook also installed at .git/hooks/pre-commit. ReplaySuppressionIntegrationTests (R4-L7): - 9 tests asserting the TurnSource suppression contract: .replay(sessionId:) and .evaluation(scenarioId:) both have dispatchesToHUD=false / speaks=false; .text and .voice (controls) have both true; .memoryExtraction shares the suppressed policy. - Asserted as a contract test (not a behavioral spy) because CONTEXT 5.1 forbids modifying production AgentOrchestrator/bus/TTS to inject spies. The properties are the architectural seam the first consumer reads; the test locks the policy table. jarvis-eval `all` cross-subcommand dispatch refactored from direct .init()+property assignment to .parse(argv) — the property-wrapper backing storage of @Flag/@Option requires the parser path; direct init trips the "Can't read a value from a parsable argument definition" guard. Shipping-gate.sh end-to-end smoke (recorded for SUMMARY): - checklist: PASS (with manual coverage on 06-voice + 1 on 01-foundations) - corpus-injection: PASS - corpus-sse: PASS - corpus-ndjson: PASS - cap-recovery: PASS - audio-rebuild: PASS (4/4 triggers automated) - wake-corpus: FAIL (empty corpus — operator-recorded WAVs deferred) - mcp-crash: FAIL (MockHelper fixture path resolution; pre-existing 08-03 issue with packages/packages/ duplication) - corpus-ndjson-live: SKIPPED (--live not set) - 8-hardening checklist: PASS (15/15 items) The infrastructure-failure-free run is the success criterion; the two downstream failures (wake-corpus + mcp-crash) trace back to 08-02 and 08-03 respectively and are not Task 3 bugs. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
49 lines
1.9 KiB
Bash
Executable File
49 lines
1.9 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# shipping-gate.sh — Phase 8 OBS-04 shipping gate.
|
|
#
|
|
# Invokes `jarvis-eval all` over the harness's eight pillars + the per-phase
|
|
# checklist manifests. Default mode: fixture-only (no Anthropic egress, no
|
|
# live Ollama daemon required). Pass `--live` to include the live-Anthropic
|
|
# + live-Ollama pillars; live mode requires `JARVIS_LIVE_EVAL=1` in the
|
|
# environment per D-04 dual-gate.
|
|
#
|
|
# Usage:
|
|
# scripts/shipping-gate.sh # fixture-only (default)
|
|
# JARVIS_LIVE_EVAL=1 scripts/shipping-gate.sh --live # full matrix incl. live
|
|
#
|
|
# Variable indirection per S-6 — a smoke-test harness can substitute:
|
|
# REPO=... override repo root detection
|
|
# HARNESS_PATH=... override packages/Harness path
|
|
# JARVIS_EVAL_BIN=... pre-built jarvis-eval binary (skip rebuild)
|
|
set -euo pipefail
|
|
|
|
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|
: "${REPO:=$(cd "$SCRIPT_DIR/.." && pwd)}"
|
|
: "${HARNESS_PATH:=$REPO/packages/Harness}"
|
|
|
|
LIVE_FLAG=""
|
|
if [[ "${1:-}" == "--live" ]]; then
|
|
LIVE_FLAG="--live"
|
|
if [[ "${JARVIS_LIVE_EVAL:-}" != "1" ]]; then
|
|
echo "shipping-gate: --live requires JARVIS_LIVE_EVAL=1 in environment (D-04 dual-gate)." >&2
|
|
exit 2
|
|
fi
|
|
fi
|
|
|
|
# Reuse pre-built binary if pointed at one; otherwise build from source.
|
|
# We build in debug configuration because MCPCrashRunner (pillar f) uses
|
|
# `#if DEBUG`-gated test seams on MCPClient/MCPServerHandle (per 08-03); a
|
|
# release build of jarvis-eval would fail to compile with those accessors
|
|
# absent. The shipping gate is for the developer's machine, not a release
|
|
# distribution — debug is the right configuration.
|
|
if [[ -n "${JARVIS_EVAL_BIN:-}" ]]; then
|
|
EVAL_BIN="$JARVIS_EVAL_BIN"
|
|
else
|
|
cd "$HARNESS_PATH"
|
|
swift build --product jarvis-eval
|
|
EVAL_BIN="$HARNESS_PATH/.build/debug/jarvis-eval"
|
|
fi
|
|
|
|
cd "$REPO"
|
|
"$EVAL_BIN" all $LIVE_FLAG
|