main
2
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  james.maesandClaude Opus 4.7
|
79ecd0184a |
Tier 1 + Tier 2 polish: 3 parallel agents — overlays, snapshots, menus
Coordinated three agents working in parallel on non-overlapping files,
with a shared `ProcessedFrame.swift` contract locking the API surface
between detector and renderer. All build clean on first integrate.
## Detector (ObjectDetector.swift)
- `VNRecognizeTextRequest` runs alongside YOLO at ~5 Hz; recognized
strings + boxes published via `currentTextDetections()`
- `VNDetectHumanBodyPoseRequest` produces a 14-segment skeleton (nose-
neck-shoulders-arms, neck-root-hips-legs); confidence-gated at 0.3
- `VNDetectHumanHandPoseRequest` produces 5 finger chains + palm ring
per hand, up to 2 hands; published as kind=.hand poses
- Body + hand published as `[PoseDetection]` via `currentPoses()`
- New `lastDetectMode: DetectMode { .yolo | .track }` so the footer can
distinguish the two latency regimes
- YOLO warmup runs one dummy inference at startup on the detector
queue — first user-visible detection no longer pays the ~200ms
model-compile latency
## Renderer (Renderer.swift + Render/ColorHash.swift)
- Box labels include 10%-bucketed confidence: "CUP 80%", "MOBILE PHONE
60%". Bucket keeps the label texture cache from exploding.
- Per-track stable color via UUID-byte → golden-ratio hash → HSV(h,
0.8, 1.0). Face labels stay cyan (calmer when JARVIS pane has
multiple labeled people in frame).
- Detection labels clamp to pane edges so boxes near the right margin
don't overflow into the next pane.
- Footer now reads "DET 28.4ms (yolo)" or "DET 2.1ms (track)" — the
flicker between the two values is informative, not a bug.
- LRU bound on `labelTextureCache` (128 entries, parallel access-order
array, drop-head-on-insert when at capacity).
- OCR text rendered as magenta boxes with truncated text labels (32
chars).
- Body pose drawn in mint, hand pose in amber; axis-aligned segment
rects with a 0.003 minimum thickness so horizontal/vertical lines
don't vanish at small viewport sizes. Joint dots at each unique
point.
- New `snapshotPNG() -> Data?` public method. Render loop piggybacks
one extra blit pass + waitUntilCompleted + CGImage encoding. <50ms
in practice, 1-second semaphore timeout if rendering stalls.
## App / Window / Capture
- Camera menu with a dynamic Devices submenu (rebuilt via
NSMenuDelegate.menuNeedsUpdate). Active device gets a checkmark,
Refresh List forces re-enumeration.
- CameraCapture.switchToDevice(_:) stops the session, swaps the input,
restarts. activeDeviceUniqueID exposed so the menu can match.
- Save Snapshot (⌘S) writes the 4-pane composite to
~/Desktop/VisionPOC-YYYY-MM-DD-HHMMSS.png
- Window-frame persistence for all three windows via UserDefaults
keys VPOC.MainWindowFrame / VPOC.SettingsWindowFrame /
VPOC.LogWindowFrame
- AVCaptureDevice.wasDisconnectedNotification → session stop + flag
set; permissionDenied returns true so the existing
"WAITING FOR CAMERA…" status path triggers without renderer changes
## Shared contract (new file)
- Sources/Process/ProcessedFrame.swift — TextDetection, PoseDetection,
PoseDetection.Segment, PoseKind (.body | .hand), DetectMode
(.yolo | .track). Sendable, value types, normalized [0..1] Vision
coords throughout.
## Theme
- Palette gains textStroke (magenta), bodyPose (mint), handPose (amber)
for the new overlays.
Note: Orbitron font and app icon (also Tier 2 items) deferred — no
clean source available for either without manual work.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
james.maesandClaude Opus 4.7
|
0d23d7ede9 |
Add a live Log Stream window (⌘L) and emit events from every subsystem
Built a Jarvis-style scrolling log viewer that surfaces what each subsystem is doing in real time. The buffer is a thread-safe ring of the last 1,000 entries; the SwiftUI viewer polls at 10 Hz, only re-renders when the buffer version bumps, and supports level / source filters, pause, auto-scroll, and clear. Components - Sources/App/LogStream.swift — singleton with Level (DBG/INF/WRN/ERR) and Source (APP / CAM / DETECT / TRACK / FACE / RENDER / VPOC) taxonomies, an os_unfair_lock-guarded buffer, and stdout mirroring so `log stream --process VisionPOC` shows the same feed in Terminal. - Sources/App/LogStreamView.swift — SwiftUI viewer with toolbar (level menu, source menu, auto-scroll, pause, clear, filtered/total counts) and a monospaced row-per-entry list that auto-scrolls to the bottom when new entries arrive. - Sources/App/LogStreamWindowController.swift — separate dark-aqua floating window so the log doesn't compete with the main 4-pane HUD for real estate. - AppDelegate gains a ⌘L "Log Stream…" menu item and a held reference to the controller (lazy-instantiated on first open). Wiring - AppDelegate: launch + ready - CameraCapture: device selection, session start, permission denied, enumeration failures - ObjectDetector: model load (success or fallback), 1 Hz detector stats (yolo vs track mode, ms, track count + label list), per-track add and per-track lose - ObjectDetector face match: when a face is recognized via FaceRegistry the distance is logged at DBG level - FaceRegistry: load on init, enroll (+prints, total), forget Implementation notes - The viewer is not @Observable-bound to LogStream so background- thread log emit calls don't pay a MainActor hop. Instead the view polls .version every 100ms and reloads on bump — bursts up to ~100 entries per tick render fine and the buffer absorbs writes from any rate the os_unfair_lock can sustain (~10 M/s on M-series). - LogStream.shared.log() can be called from any thread/queue. Stdout is also written from the same call so terminal tail works. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |