Slice 3 of dev-overlay-end-to-end. Adds a fourth DevOverlay tab that
tails every log record from every Jarvis subsystem in real time —
agent, voice, vision, memory, MCP, bus, replay, system — without
polling files or relying on Console.app.
How it works:
1. `LogBroadcaster` (new) — process-wide actor with a 500-line ring
buffer + subscriber set. Borrowed shape from
OrchestratorEventBroadcaster so the project's three broadcasters
share a mental model. New subscribers replay the buffer in order,
then go live.
2. `BroadcastLogHandler` (new) — swift-log LogHandler conformance.
Formats each log call into a `LogLine` and schedules an unstructured
`Task { await LogBroadcaster.shared.publish(...) }` so the logging
caller never blocks on actor isolation. Synchronous handler contract
preserved; ordering is best-effort under heavy load.
3. `LoggingBootstrap.make(label:)` now returns
`MultiplexLogHandler([file, os, broadcast])` — the new handler is
purely additive. File rotation + os.Logger streaming unchanged.
4. `DevOverlayLogsView` (new) — fourth tab in the overlay. Subscribes
to `LogBroadcaster.shared` on appear, caps at 1000 lines, renders
monospaced. Per-channel filter (Menu w/ checkboxes), per-level
filter (debug → critical), substring search, Pause/Resume button
(Cmd+P), auto-scroll-to-bottom while not paused. Color-coded by
level (warn=orange, error/critical=red).
Tests:
- 5 new LogBroadcasterTests covering publish/cap/replay/live/handler.
- swift test --package-path packages/Logging → 22/22 pass.
- swift test --package-path packages/DevOverlay → 14/14 pass.
- bash scripts/check-app-builds.sh → PASS.
- 18/18 boundary gates green.
Redaction note: file handler still routes through Redact.apply for
disk writes; the broadcaster sees raw messages because the DevOverlay
is a developer-only in-process surface. If this ever becomes
user-facing, route through Redact before publishing.
Replace `FileHandle(forWritingTo:)` with `Darwin.open(O_WRONLY | O_APPEND
| O_CREAT | O_CLOEXEC, 0o644)` wrapped in `FileHandle(fileDescriptor:
closeOnDealloc:)`. The previous code path opened the FD without
O_CLOEXEC, so JarvisMCP/ChildSpawnGate fatal-errored in Debug whenever
the runtime tried to spawn an MCP helper — the long-lived log FD leaks
across fork+exec by default on macOS.
`O_APPEND` makes every write atomically positioned at end-of-file, which
is more crash-resilient than the prior "seekToEnd then write" pair (a
concurrent rotator/write race could not truncate the file but could
interleave bytes). `O_CREAT` collapses the prior fileExists+createFile
roundtrip into a single syscall.
Together with the path-aware ChildSpawnGate change, this clears the
last hard-block on the native-launch path: AppDelegate now reaches
`MCPRuntime built — 3 tools` and the helper bundles spawn cleanly.
17/17 JarvisLoggingTests still pass (rotation, retention, redaction).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
ConfirmingToolDispatcher needs to emit a WARNING log line on broker timeout
(AGENT-11). The plan specifies the log lands in JarvisLogChannel.mcp, but
that case did not yet exist in the enum.
Adds .mcp (rawValue 'mcp') to the channel enum and updates the two
case-count tests (ChannelTests + LoggingTests) to expect 8 cases.
- ReplayLog actor: opens replay.db with WAL pragmas + DDL + meta seed; beginSession/startTurn/record/endTurn/close. Writes batched in 50ms window OR 64-event chunks. endTurn appends a turn_end event, flushes, UPDATEs ended_at + stop_reason, then PRAGMA wal_checkpoint(TRUNCATE) for the per-turn fsync. Failures logged via JarvisLogChannel.replay; never propagated.
- OrphanDetector actor: opens its own connection, increments meta.crash_count, runs the OBS-07 query (ended_at IS NULL AND turn_id NOT IN (turn_end events)), stamps each orphan with stop_reason='orphan_recovered' + recovery_marker='detected_at_launch:crash_count=N'.
- 12 new tests: L1-L8 (session/turn/buffering/window/chunk/large-blob/nothing-masked) + O1-O4 (orphan vs non-orphan, race window, crash_count increment, end-to-end SIGKILL parity).
- Adds JarvisLogChannel.replay + .devoverlay (additive — Logging test updated to assert 7 cases). The full Task 3 channel-tests file lands next commit.
Phase 1 REVIEW.md WR-08: `FileRotatingWriter` relied solely on wall-
clock `Date()` for day-boundary rotation and ran retention GC only as
a side effect of rotation. Two failure modes:
1. Clock jumping ±N days (NTP correction, manual change) would
rotate without signal — operators would see log files that
interleave pre/post-adjustment timestamps with no warning.
2. An idle app (no log write crossing midnight for >7 days) would
never GC stale files.
packages/Logging/Sources/JarvisLogging/FileRotatingWriter.swift
- `rotateIfNeeded()` computes the day-delta between the previous
window and the new day. When `|delta| > 2` emit an `.notice`-level
fault via os.Logger naming the skew magnitude. Rotation still
happens normally — the log just flags the unusual trigger.
- New `runRetentionGC()` dispatches `gcOldFiles` on the serial
queue, independent of rotation. Callers can invoke at shutdown or
on a periodic heartbeat.
packages/Logging/Sources/JarvisLogging/FileLogHandler.swift
- Public `runRetentionGC()` forwards to the writer so App-layer
callers can trigger GC without reaching into internals.
All 14 logging tests still pass.
Phase 1 REVIEW.md IN-02: `OSLogHandler.log` interpolated message text
as `\(message, privacy: .public)`. That override defeats os.Logger's
default `%{private}@` masking for interpolated values — a caller who
logs `logger.info("saving key \(userInput)")` produces a redacted line
in the file log (via `Redact.apply` in `FileLogHandler`) but a
plaintext line in `os.Logger`. The docstring already flagged the
discipline risk; the cure (caller redacts first) is on the honor
system with no enforcement.
packages/Logging/Sources/JarvisLogging/OSLogHandler.swift
- Drop the `.public` override. Interpolate as
`\(message, privacy: .private(mask: .hash))` so untrusted interpolated
values are hashed in Console.app captures by default.
- Keep `osLevel` via the `logger.log(level:)` API so severity signal
is preserved in Console.app filters (level isn't embedded in the
message body).
All 14 logging tests still pass — there's no explicit capture of the
os.Logger stream in-test, so the privacy override change is behaviour-
compatible at the test layer. Observable in Console.app only.
Phase 1 REVIEW.md WR-01 (OBS-06): `FileRotatingWriter` used `try?` on
both `FileHandle(forWritingTo:)` and `handle.write(contentsOf:)`, so a
disk-full, permission-revoked, or FD-evicted write silently dropped the
line with no signal. OBS-06 reads as "rotation must not lose data".
packages/Logging/Sources/JarvisLogging/FileRotatingWriter.swift
- Add `droppedLinesThisWindow` counter (serial-queue-local, no lock).
- Add `warnedAboutOpenFailureThisWindow` flag to emit a one-time
os.Logger fault per rotation window (prevents spam if the failure
persists for hours).
- `append()` now explicitly increments the counter on both the
nil-handle path and the `handle.write` throw path, then logs a
fault via `os.Logger` (subsystem `com.koftwentytwo.jarvis`,
category `logging.file`) so failures are visible in Console.app
even when the file log itself is the thing that's broken.
- `rotateIfNeeded()` emits an `.error`-level summary when the
just-closed window lost lines, then resets both flags for the new
window.
- Add `droppedLinesSnapshot()` test-only accessor that reads the
counter through the serial queue (no TSan race).
packages/Logging/Tests/JarvisLoggingTests/FileLogHandlerTests.swift
- New `test_dropsLinesWhenHandleUnavailable`: points the writer at a
path that is a regular file (so directory creation and file-open
both fail), appends three lines, asserts `droppedLinesSnapshot() == 3`.
Owner change: project will be owned by KofTwentyTwo. Bundle ID updates
propagate to three ${PRODUCT_BUNDLE_IDENTIFIER} targets, the Keychain
service name (SEC-01 / D-10), the OSLog subsystem identifier, and the
FileRotatingWriter DispatchQueue label. pbxproj regenerated via
xcodegen generate.
Forward-looking planning docs (STATE, REQUIREMENTS, PATTERNS, VALIDATION)
updated to match. Historical records (completed PLAN.md / SUMMARY.md /
RESEARCH.md / audits / CONTEXT / DISCUSSION-LOG) deliberately left as-is
to preserve the audit trail of the original naming decision.
Verified: 49 SPM tests pass (4 Keychain + 15 Config + 13 Logging + 17
Shell), xcodebuild Debug build succeeds with full codesign pipeline.
Decision logged in STATE.md Key Decisions (Locked 2026-04-22).
- JarvisLogChannel: four cases (agent, tools, ui, system)
- JarvisLogHandlerFactory.bootstrap wires MultiplexLogHandler that
fans out to FileLogHandler + OSLogHandler (D-17)
- FileLogHandler applies Redact.apply once (S-4 single call site)
- OSLogHandler does NOT redact (os.Logger handles %{private}@ at
system layer; callers redact untrusted input before logging)
- Redact covers exactly 5 patterns: sk-ant-, Authorization:Bearer,
sk- (OpenAI), AKIA, ghp_ (D-20, OBS-06); Anthropic branch ordered
before OpenAI so sk-ant- consumes the full prefix
- FileRotatingWriter: lazy midnight rotation via TestDateProvider-
injectable DateProvider; serial DispatchQueue; retain-7 GC on
rotate (D-18)
- LogPaths.channelDirectory = ~/Library/Logs/Jarvis/
- Swift 6 strict: ISO8601DateFormatter.jarvisShared uses
nonisolated(unsafe) since the formatter is documented thread-safe
after configuration but not Sendable-annotated by Apple
- OSLogHandler uses fully-qualified Logging.Logger to disambiguate
from os.Logger (both imported)
- Tests: 13/13 pass (7 Redact + 2 FileLogHandler rotate/retain +
4 bootstrap/factory/channels)
- Deletes Plan 01 placeholders
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- packages/Keychain — zero SPM deps (Security.framework at link time only)
- packages/Logging — depends on apple/swift-log 1.5.3+; library product is
named JarvisLogging to avoid collision with swift-log's own Logging module
- packages/Config — depends on Keychain + Logging (product JarvisLogging)
- packages/Shell — depends on Config + Logging; links AppKit,
ServiceManagement, IOKit, Carbon for hotkey/launch-at-login surfaces
Every target (main + test) opts into .swiftLanguageMode(.v6) per D-04 so
data-race errors are compile-time failures, not warnings.
Placeholder sources (one per package) plus one trivial XCTest per package
per D-03 so each package builds standalone and has a passing test. Real
sources land in Plan 02 (Keychain/Config/Logging) and Plan 04 (Shell).
Bus/LLM/MCP/Voice/Memory packages NOT pre-created per D-01 — they arrive
in their owning phases (P2/P4/P5/P6/P7) to keep rebuild scope tight.
james.maes
