Four parallel agents shipped Tier 1+2+3 of the punch list, coordinated through a fixed DetectionEventBus contract created upfront. ## Detection event bus (shared contract) - Sources/Process/DetectionEvents.swift — `DetectionEvent.Kind` enum covering objectAppeared/Refreshed/Disappeared, faceRecognized, faceSeenUnknown, textRecognized, gestureDetected, activityDetected, spatialRelation. - Sources/Process/DetectionEventBus.swift — thread-safe pub/sub, Token pattern with auto-unsubscribe on deinit, synchronous emit on caller's queue. Subscribers hop to their own queue as needed. ## ObjectDetector additions (Agent A) - Publishes events for every track-add, track-loss, face-match, unmatched-face, recognized-text — with per-event throttles (text 2s per string, face name 5s, etc.) so the bus doesn't drown in repeats. - New `DetectorStageTiming` struct + `lastStageTiming` accessor exposes the visionBundleMs / featurePrintMs / trackerMs / totalMs breakdown. Logged once a second at DEBUG level for visibility. - Track UUID persistence via Sources/Process/TrackStore.swift — label→UUID at ~/Library/Application Support/VisionPOC/tracks.json, MRU-pruned to 50 entries, debounced 500ms saves. Same object across launches gets the same trackId → consistent per-track color. - YOLO warmup at init (dispatched on detector queue): one dummy inference against a 416×416 mid-gray image so the first user-visible detection doesn't pay model-compile latency. ## Event consumers (Agent A) - Sources/Process/Greeter.swift — AVSpeechSynthesizer-driven "Hello, <name>" on faceRecognized events. Honors Theme.greeterMuted and the Privacy Mode toggle. Throttle: 30s per name on top of the bus-level 5s throttle. - Sources/Process/GestureRecognizer.swift — five gesture heuristics (thumbs_up, open_hand, fist, pointing, peace) over hand-pose PoseDetections. Throttled 2s per gesture name. - Sources/Process/ActivityRecognizer.swift — standing/sitting/ crouching/hand_raised over body-pose PoseDetections. Throttled 5s per activity. - Sources/Process/SpatialReasoner.swift — near/above/inside/holding relations between Detection pairs. Throttled 10s per triple. ## Video recording (Agent B) - Sources/Render/Recorder.swift — @MainActor AVAssetWriter pipeline writing H.264 / 12 Mbps / 60fps to ~/Desktop/VisionPOC-...mov. Captures the rendered MTKView output via a new frame-tap callback on Renderer. Synchronous blit→staging-texture→CVPixelBuffer→ AVAssetWriterInputPixelBufferAdaptor pattern; ~5-15ms per frame at 1080p, drops dynamically when active. Format-guarded for bgra8Unorm drawables and disabled when not recording (early return). - Renderer.swift gained `var frameTap` callback; invoked inside draw(in:) just before present, when set. ## History window (Agent C) - Sources/App/HistoryEntry.swift, HistoryStore.swift, HistoryView.swift, HistoryWindowController.swift — singleton ring buffer of 500 recent events translated into human-readable summaries with category tags (object/face-known/face-unknown/ocr/gesture/activity/spatial). - Open via ⌘H from the Faces menu. Polls the store at 1Hz. - Window position persists under VPOC.HistoryWindowFrame. ## Privacy mode (Agent C) - Theme.Performance.faceRecognitionDisabled — new live tunable. - Camera menu toggle (⇧⌘P) flips both faceRecognitionDisabled and greeterMuted. The History viewer also filters out face entries when the flag is set. - Documented limitation: bounding-box labels may still show enrolled names if the detector doesn't consult faceRecognitionDisabled (this round it doesn't; flag is for History + TTS only). ## Recording menu wiring (Agent C) - App menu gains "Start Recording" / "Stop Recording" (⇧⌘R). Title flips based on the Recorder's isRecording state. - Start: instantiates the recorder, calls startRecording at the current drawable size, hooks renderer.frameTap. - Stop: clears frameTap first (no more frames queue) then awaits the asset writer's finalize. On completion (MainActor hop), shows an NSAlert with the saved-path or failure. ## Settings panel additions (Agent C) - New "Faces" subsection: "Mute greeter (TTS)" toggle bound to TunableSettings.greeterMuted (writes through to Theme). - New "Diagnostics" section: per-stage detector timing readout polled at 2Hz. Decoupled from ObjectDetector via a closure-based timing provider so the App layer doesn't depend on the Process module's DetectorStageTiming type directly. ## Hygiene (Agent D) - Tests/IoUTests.swift, AspectFitTests.swift, LogStreamTests.swift, DetectionEventBusTests.swift, FaceRegistryDistanceTests.swift — 24 tests covering pure-logic algorithms. Private methods are mirrored verbatim into the test files with a "keep in sync" comment. - project.yml gains a `VisionPOCTests` target of type `bundle.unit-test`, depending on the VisionPOC application target. - .github/workflows/build.yml — macos-15 runner, brew install xcodegen, generate, build, test. LFS skipped to save bandwidth (the build tolerates a missing model since ObjectDetector falls back to Vision built-ins). - CLAUDE.md — onboarding doc covering project layout, theme tunables, pipelines, event vocabulary, observability, build idiosyncrasies, and "how to add a new detector / overlay" recipes. ## Integration fix - Recorder.swift's `stopRecording` completion path crossed actor isolation boundaries — captured a `(URL?) -> Void` closure and the AVAssetWriter instance into a @Sendable closure. Fixed by extracting `writer.status` / `writer.error?.localizedDescription` on the writer's queue before hopping, and wrapping the completion in a `nonisolated(unsafe)` shuttle (it's called from MainActor only, so the unsafe annotation is honest). ## Build status - `xcodebuild build` — clean - `xcodebuild test` — 24 / 24 passing Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
145 lines
4.8 KiB
Swift
145 lines
4.8 KiB
Swift
import Foundation
|
|
import os
|
|
|
|
/// Ring buffer of recent detection events for the History viewer.
|
|
///
|
|
/// Subscribes to `DetectionEventBus.shared` once at init and converts each
|
|
/// event into a `HistoryEntry`. The viewer polls `snapshot()` at 1 Hz; the
|
|
/// store itself is thread-safe so background-thread emitters can write
|
|
/// without hopping to MainActor.
|
|
///
|
|
/// Filtering rules:
|
|
/// - `.objectRefreshed` is dropped (too noisy for history).
|
|
/// - Face events are dropped when `Theme.Performance.faceRecognitionDisabled`
|
|
/// is set (Privacy Mode). This is the privacy guarantee the App layer can
|
|
/// make on its own, independent of whether the detector honors the flag.
|
|
final class HistoryStore: @unchecked Sendable {
|
|
static let shared = HistoryStore()
|
|
|
|
static let capacity = 500
|
|
|
|
private var lock = os_unfair_lock_s()
|
|
private var buffer: [HistoryEntry] = []
|
|
private var _version: UInt64 = 0
|
|
private var subscription: DetectionEventBus.Token?
|
|
|
|
private init() {
|
|
buffer.reserveCapacity(HistoryStore.capacity)
|
|
subscription = DetectionEventBus.shared.subscribe { [weak self] event in
|
|
self?.ingest(event)
|
|
}
|
|
}
|
|
|
|
/// Snapshot of current entries, oldest-first. Cheap copy because the
|
|
/// store is bounded to `capacity` rows.
|
|
func snapshot() -> [HistoryEntry] {
|
|
os_unfair_lock_lock(&lock)
|
|
let copy = buffer
|
|
os_unfair_lock_unlock(&lock)
|
|
return copy
|
|
}
|
|
|
|
/// Monotonically increasing counter that bumps on every append. The
|
|
/// viewer polls this to decide whether to re-render — same pattern as
|
|
/// `LogStream.shared.version`.
|
|
var version: UInt64 {
|
|
os_unfair_lock_lock(&lock)
|
|
let v = _version
|
|
os_unfair_lock_unlock(&lock)
|
|
return v
|
|
}
|
|
|
|
func clear() {
|
|
os_unfair_lock_lock(&lock)
|
|
buffer.removeAll(keepingCapacity: true)
|
|
_version &+= 1
|
|
os_unfair_lock_unlock(&lock)
|
|
}
|
|
|
|
// MARK: - Ingestion
|
|
|
|
private func ingest(_ event: DetectionEvent) {
|
|
guard let entry = HistoryStore.translate(event) else { return }
|
|
os_unfair_lock_lock(&lock)
|
|
if buffer.count >= HistoryStore.capacity {
|
|
buffer.removeFirst(buffer.count - HistoryStore.capacity + 1)
|
|
}
|
|
buffer.append(entry)
|
|
_version &+= 1
|
|
os_unfair_lock_unlock(&lock)
|
|
}
|
|
|
|
private static func translate(_ event: DetectionEvent) -> HistoryEntry? {
|
|
switch event.kind {
|
|
case .objectRefreshed:
|
|
return nil
|
|
|
|
case .objectAppeared(let label, _, let confidence, _):
|
|
return HistoryEntry(
|
|
id: UUID(),
|
|
timestamp: event.timestamp,
|
|
summary: "\(label.capitalized) appeared (conf \(String(format: "%.2f", confidence)))",
|
|
category: "object"
|
|
)
|
|
|
|
case .objectDisappeared(let label, _):
|
|
return HistoryEntry(
|
|
id: UUID(),
|
|
timestamp: event.timestamp,
|
|
summary: "\(label.capitalized) disappeared",
|
|
category: "object"
|
|
)
|
|
|
|
case .faceRecognized(let name, let distance, _):
|
|
if Theme.Performance.faceRecognitionDisabled { return nil }
|
|
return HistoryEntry(
|
|
id: UUID(),
|
|
timestamp: event.timestamp,
|
|
summary: "\(name) recognized (dist \(String(format: "%.1f", distance)))",
|
|
category: "face-known"
|
|
)
|
|
|
|
case .faceSeenUnknown:
|
|
if Theme.Performance.faceRecognitionDisabled { return nil }
|
|
return HistoryEntry(
|
|
id: UUID(),
|
|
timestamp: event.timestamp,
|
|
summary: "Unknown face seen",
|
|
category: "face-unknown"
|
|
)
|
|
|
|
case .textRecognized(let text, _):
|
|
return HistoryEntry(
|
|
id: UUID(),
|
|
timestamp: event.timestamp,
|
|
summary: "Read: \"\(text)\"",
|
|
category: "ocr"
|
|
)
|
|
|
|
case .gestureDetected(let gesture, _):
|
|
return HistoryEntry(
|
|
id: UUID(),
|
|
timestamp: event.timestamp,
|
|
summary: "Gesture: \(gesture)",
|
|
category: "gesture"
|
|
)
|
|
|
|
case .activityDetected(let activity, _):
|
|
return HistoryEntry(
|
|
id: UUID(),
|
|
timestamp: event.timestamp,
|
|
summary: "Activity: \(activity)",
|
|
category: "activity"
|
|
)
|
|
|
|
case .spatialRelation(let subject, let relation, let object):
|
|
return HistoryEntry(
|
|
id: UUID(),
|
|
timestamp: event.timestamp,
|
|
summary: "\(subject.capitalized) is \(relation) \(object)",
|
|
category: "spatial"
|
|
)
|
|
}
|
|
}
|
|
}
|