Files
james.maesandClaude Opus 4.7 958da3bb95 Event bus + greeter + gestures + activity + spatial + history + recording + tests + CI + CLAUDE.md
Four parallel agents shipped Tier 1+2+3 of the punch list, coordinated
through a fixed DetectionEventBus contract created upfront.

## Detection event bus (shared contract)
- Sources/Process/DetectionEvents.swift — `DetectionEvent.Kind` enum
  covering objectAppeared/Refreshed/Disappeared, faceRecognized,
  faceSeenUnknown, textRecognized, gestureDetected, activityDetected,
  spatialRelation.
- Sources/Process/DetectionEventBus.swift — thread-safe pub/sub, Token
  pattern with auto-unsubscribe on deinit, synchronous emit on caller's
  queue. Subscribers hop to their own queue as needed.

## ObjectDetector additions (Agent A)
- Publishes events for every track-add, track-loss, face-match,
  unmatched-face, recognized-text — with per-event throttles (text 2s
  per string, face name 5s, etc.) so the bus doesn't drown in repeats.
- New `DetectorStageTiming` struct + `lastStageTiming` accessor exposes
  the visionBundleMs / featurePrintMs / trackerMs / totalMs breakdown.
  Logged once a second at DEBUG level for visibility.
- Track UUID persistence via Sources/Process/TrackStore.swift
  — label→UUID at ~/Library/Application Support/VisionPOC/tracks.json,
  MRU-pruned to 50 entries, debounced 500ms saves. Same object across
  launches gets the same trackId → consistent per-track color.
- YOLO warmup at init (dispatched on detector queue): one dummy
  inference against a 416×416 mid-gray image so the first user-visible
  detection doesn't pay model-compile latency.

## Event consumers (Agent A)
- Sources/Process/Greeter.swift — AVSpeechSynthesizer-driven "Hello,
  <name>" on faceRecognized events. Honors Theme.greeterMuted and the
  Privacy Mode toggle. Throttle: 30s per name on top of the bus-level
  5s throttle.
- Sources/Process/GestureRecognizer.swift — five gesture heuristics
  (thumbs_up, open_hand, fist, pointing, peace) over hand-pose
  PoseDetections. Throttled 2s per gesture name.
- Sources/Process/ActivityRecognizer.swift — standing/sitting/
  crouching/hand_raised over body-pose PoseDetections. Throttled 5s
  per activity.
- Sources/Process/SpatialReasoner.swift — near/above/inside/holding
  relations between Detection pairs. Throttled 10s per triple.

## Video recording (Agent B)
- Sources/Render/Recorder.swift — @MainActor AVAssetWriter pipeline
  writing H.264 / 12 Mbps / 60fps to ~/Desktop/VisionPOC-...mov.
  Captures the rendered MTKView output via a new frame-tap callback
  on Renderer. Synchronous blit→staging-texture→CVPixelBuffer→
  AVAssetWriterInputPixelBufferAdaptor pattern; ~5-15ms per frame at
  1080p, drops dynamically when active. Format-guarded for bgra8Unorm
  drawables and disabled when not recording (early return).
- Renderer.swift gained `var frameTap` callback; invoked inside
  draw(in:) just before present, when set.

## History window (Agent C)
- Sources/App/HistoryEntry.swift, HistoryStore.swift, HistoryView.swift,
  HistoryWindowController.swift — singleton ring buffer of 500 recent
  events translated into human-readable summaries with category tags
  (object/face-known/face-unknown/ocr/gesture/activity/spatial).
- Open via ⌘H from the Faces menu. Polls the store at 1Hz.
- Window position persists under VPOC.HistoryWindowFrame.

## Privacy mode (Agent C)
- Theme.Performance.faceRecognitionDisabled — new live tunable.
- Camera menu toggle (⇧⌘P) flips both faceRecognitionDisabled and
  greeterMuted. The History viewer also filters out face entries when
  the flag is set.
- Documented limitation: bounding-box labels may still show enrolled
  names if the detector doesn't consult faceRecognitionDisabled (this
  round it doesn't; flag is for History + TTS only).

## Recording menu wiring (Agent C)
- App menu gains "Start Recording" / "Stop Recording" (⇧⌘R). Title
  flips based on the Recorder's isRecording state.
- Start: instantiates the recorder, calls startRecording at the
  current drawable size, hooks renderer.frameTap.
- Stop: clears frameTap first (no more frames queue) then awaits the
  asset writer's finalize. On completion (MainActor hop), shows an
  NSAlert with the saved-path or failure.

## Settings panel additions (Agent C)
- New "Faces" subsection: "Mute greeter (TTS)" toggle bound to
  TunableSettings.greeterMuted (writes through to Theme).
- New "Diagnostics" section: per-stage detector timing readout polled
  at 2Hz. Decoupled from ObjectDetector via a closure-based timing
  provider so the App layer doesn't depend on the Process module's
  DetectorStageTiming type directly.

## Hygiene (Agent D)
- Tests/IoUTests.swift, AspectFitTests.swift, LogStreamTests.swift,
  DetectionEventBusTests.swift, FaceRegistryDistanceTests.swift
  — 24 tests covering pure-logic algorithms. Private methods are
  mirrored verbatim into the test files with a "keep in sync" comment.
- project.yml gains a `VisionPOCTests` target of type
  `bundle.unit-test`, depending on the VisionPOC application target.
- .github/workflows/build.yml — macos-15 runner, brew install xcodegen,
  generate, build, test. LFS skipped to save bandwidth (the build
  tolerates a missing model since ObjectDetector falls back to Vision
  built-ins).
- CLAUDE.md — onboarding doc covering project layout, theme tunables,
  pipelines, event vocabulary, observability, build idiosyncrasies,
  and "how to add a new detector / overlay" recipes.

## Integration fix
- Recorder.swift's `stopRecording` completion path crossed actor
  isolation boundaries — captured a `(URL?) -> Void` closure and the
  AVAssetWriter instance into a @Sendable closure. Fixed by extracting
  `writer.status` / `writer.error?.localizedDescription` on the
  writer's queue before hopping, and wrapping the completion in a
  `nonisolated(unsafe)` shuttle (it's called from MainActor only, so
  the unsafe annotation is honest).

## Build status
- `xcodebuild build` — clean
- `xcodebuild test` — 24 / 24 passing

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-16 17:20:54 -05:00

128 lines
4.7 KiB
Swift

import Foundation
import os
/// Persists a small label → most-recent-track-UUID mapping to disk so that
/// when an object reappears in a new session, the same UUID is reused — which
/// in turn makes the renderer's per-track color stay stable across launches.
///
/// Storage: `~/Library/Application Support/VisionPOC/tracks.json`
/// Format: `[String: String]` (label → UUID string)
/// Size cap: last-50 MRU; oldest entries are pruned on save.
final class TrackStore: @unchecked Sendable {
static let shared = TrackStore()
/// MRU cap: keep the last N entries on disk so the file doesn't grow
/// unboundedly over months of use.
private static let maxEntries = 50
private var lock = os_unfair_lock_s()
/// In-memory map.
private var labelToUUID: [String: String] = [:]
/// MRU ordering — most recently touched label is at the end.
private var order: [String] = []
/// Debounce so back-to-back track bootstraps don't each hit the disk.
private var pendingSave: DispatchWorkItem?
private let saveQueue = DispatchQueue(label: "com.dmdbrands.VisionPOC.trackstore.save", qos: .utility)
private static var storageURL: URL {
let base = FileManager.default.urls(for: .applicationSupportDirectory, in: .userDomainMask)[0]
let dir = base.appendingPathComponent("VisionPOC", isDirectory: true)
try? FileManager.default.createDirectory(at: dir, withIntermediateDirectories: true)
return dir.appendingPathComponent("tracks.json")
}
init() {
loadFromDisk()
}
/// Returns the most recently persisted UUID for this label, if any.
/// Callers should treat this as a *preference* — if the persisted string
/// fails to parse as a UUID, this returns nil and a fresh UUID should be
/// minted.
func preferredUUID(for label: String) -> UUID? {
os_unfair_lock_lock(&lock)
defer { os_unfair_lock_unlock(&lock) }
guard let str = labelToUUID[label] else { return nil }
return UUID(uuidString: str)
}
/// Record that `label` is now associated with `id`. Moves the label to
/// the MRU end and schedules a debounced save.
func record(label: String, id: UUID) {
os_unfair_lock_lock(&lock)
labelToUUID[label] = id.uuidString
if let existing = order.firstIndex(of: label) {
order.remove(at: existing)
}
order.append(label)
prune()
os_unfair_lock_unlock(&lock)
scheduleSave()
}
/// Caller must hold the lock.
private func prune() {
while order.count > TrackStore.maxEntries {
let dropped = order.removeFirst()
labelToUUID.removeValue(forKey: dropped)
}
}
private func scheduleSave() {
os_unfair_lock_lock(&lock)
pendingSave?.cancel()
let work = DispatchWorkItem { [weak self] in
self?.saveToDisk()
}
pendingSave = work
os_unfair_lock_unlock(&lock)
// Coalesce bursts of track creations into a single disk write.
saveQueue.asyncAfter(deadline: .now() + 0.5, execute: work)
}
private func loadFromDisk() {
guard let data = try? Data(contentsOf: Self.storageURL) else { return }
struct Persisted: Codable {
var labelToUUID: [String: String]
var order: [String]
}
if let decoded = try? JSONDecoder().decode(Persisted.self, from: data) {
os_unfair_lock_lock(&lock)
labelToUUID = decoded.labelToUUID
order = decoded.order.filter { labelToUUID[$0] != nil }
// Defensive: ensure any keys missing from `order` get appended so
// pruning still works deterministically.
for k in labelToUUID.keys where !order.contains(k) {
order.append(k)
}
prune()
os_unfair_lock_unlock(&lock)
} else if let flat = try? JSONDecoder().decode([String: String].self, from: data) {
// Backward-compat: a previous version stored just the flat map.
os_unfair_lock_lock(&lock)
labelToUUID = flat
order = Array(flat.keys)
prune()
os_unfair_lock_unlock(&lock)
}
}
private func saveToDisk() {
os_unfair_lock_lock(&lock)
struct Persisted: Codable {
var labelToUUID: [String: String]
var order: [String]
}
let snapshot = Persisted(labelToUUID: labelToUUID, order: order)
os_unfair_lock_unlock(&lock)
do {
let data = try JSONEncoder().encode(snapshot)
try data.write(to: Self.storageURL, options: .atomic)
} catch {
NSLog("TrackStore: failed to write \(Self.storageURL.path): \(error)")
}
}
}