james.maesandClaude Opus 4.7
79ecd0184a
Coordinated three agents working in parallel on non-overlapping files,
with a shared `ProcessedFrame.swift` contract locking the API surface
between detector and renderer. All build clean on first integrate.
## Detector (ObjectDetector.swift)
- `VNRecognizeTextRequest` runs alongside YOLO at ~5 Hz; recognized
strings + boxes published via `currentTextDetections()`
- `VNDetectHumanBodyPoseRequest` produces a 14-segment skeleton (nose-
neck-shoulders-arms, neck-root-hips-legs); confidence-gated at 0.3
- `VNDetectHumanHandPoseRequest` produces 5 finger chains + palm ring
per hand, up to 2 hands; published as kind=.hand poses
- Body + hand published as `[PoseDetection]` via `currentPoses()`
- New `lastDetectMode: DetectMode { .yolo | .track }` so the footer can
distinguish the two latency regimes
- YOLO warmup runs one dummy inference at startup on the detector
queue — first user-visible detection no longer pays the ~200ms
model-compile latency
## Renderer (Renderer.swift + Render/ColorHash.swift)
- Box labels include 10%-bucketed confidence: "CUP 80%", "MOBILE PHONE
60%". Bucket keeps the label texture cache from exploding.
- Per-track stable color via UUID-byte → golden-ratio hash → HSV(h,
0.8, 1.0). Face labels stay cyan (calmer when JARVIS pane has
multiple labeled people in frame).
- Detection labels clamp to pane edges so boxes near the right margin
don't overflow into the next pane.
- Footer now reads "DET 28.4ms (yolo)" or "DET 2.1ms (track)" — the
flicker between the two values is informative, not a bug.
- LRU bound on `labelTextureCache` (128 entries, parallel access-order
array, drop-head-on-insert when at capacity).
- OCR text rendered as magenta boxes with truncated text labels (32
chars).
- Body pose drawn in mint, hand pose in amber; axis-aligned segment
rects with a 0.003 minimum thickness so horizontal/vertical lines
don't vanish at small viewport sizes. Joint dots at each unique
point.
- New `snapshotPNG() -> Data?` public method. Render loop piggybacks
one extra blit pass + waitUntilCompleted + CGImage encoding. <50ms
in practice, 1-second semaphore timeout if rendering stalls.
## App / Window / Capture
- Camera menu with a dynamic Devices submenu (rebuilt via
NSMenuDelegate.menuNeedsUpdate). Active device gets a checkmark,
Refresh List forces re-enumeration.
- CameraCapture.switchToDevice(_:) stops the session, swaps the input,
restarts. activeDeviceUniqueID exposed so the menu can match.
- Save Snapshot (⌘S) writes the 4-pane composite to
~/Desktop/VisionPOC-YYYY-MM-DD-HHMMSS.png
- Window-frame persistence for all three windows via UserDefaults
keys VPOC.MainWindowFrame / VPOC.SettingsWindowFrame /
VPOC.LogWindowFrame
- AVCaptureDevice.wasDisconnectedNotification → session stop + flag
set; permissionDenied returns true so the existing
"WAITING FOR CAMERA…" status path triggers without renderer changes
## Shared contract (new file)
- Sources/Process/ProcessedFrame.swift — TextDetection, PoseDetection,
PoseDetection.Segment, PoseKind (.body | .hand), DetectMode
(.yolo | .track). Sendable, value types, normalized [0..1] Vision
coords throughout.
## Theme
- Palette gains textStroke (magenta), bodyPose (mint), handPose (amber)
for the new overlays.
Note: Orbitron font and app icon (also Tier 2 items) deferred — no
clean source available for either without manual work.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
54 lines
2.3 KiB
Swift
54 lines
2.3 KiB
Swift
import Foundation
|
|
import simd
|
|
|
|
/// Deterministic per-trackId color generator. Hashes a UUID into a hue value
|
|
/// in [0, 1) and converts HSV(hue, 0.8, 1.0) to RGB. Used to give each tracked
|
|
/// object a stable, visually-distinct box + label color so it's clear which
|
|
/// label belongs to which box when several boxes overlap.
|
|
///
|
|
/// Uniqueness caveat: with hue alone covering ~32 buckets of human-discernible
|
|
/// difference, ~6 simultaneous tracks of the same hue family will start to look
|
|
/// similar. For the POC's typical 3-8 tracks this is fine; if track count
|
|
/// regularly exceeds ~12 a second axis (saturation or value) would help.
|
|
enum ColorHash {
|
|
/// Returns a stable RGBA color for the given track id.
|
|
/// Saturation 0.8, value 1.0, alpha 1.0.
|
|
static func colorFor(trackId: UUID) -> SIMD4<Float> {
|
|
// Spread the UUID's bytes across the hash so adjacent UUIDs (which the
|
|
// standard library doesn't actually generate adjacently, but be safe)
|
|
// map to far-apart hues. Mix several bytes from the uuid tuple.
|
|
let bytes = trackId.uuid
|
|
// Combine bytes 0, 3, 7, 11 with a multiplicative hash. The constant
|
|
// 2654435769 is Knuth's golden-ratio hash multiplier — distributes
|
|
// small input differences across the full 32-bit output range.
|
|
var h: UInt32 = UInt32(bytes.0)
|
|
h = h &* 2654435769 &+ UInt32(bytes.3)
|
|
h = h &* 2654435769 &+ UInt32(bytes.7)
|
|
h = h &* 2654435769 &+ UInt32(bytes.11)
|
|
let hue = Float(h % 360) / 360.0
|
|
return hsvToRGB(h: hue, s: 0.8, v: 1.0)
|
|
}
|
|
|
|
/// HSV → RGB, standard formula. Hue is in [0,1), s and v in [0,1].
|
|
private static func hsvToRGB(h: Float, s: Float, v: Float) -> SIMD4<Float> {
|
|
let hh = (h - floor(h)) * 6.0
|
|
let i = Int(floor(hh))
|
|
let f = hh - Float(i)
|
|
let p = v * (1 - s)
|
|
let q = v * (1 - s * f)
|
|
let t = v * (1 - s * (1 - f))
|
|
let r: Float
|
|
let g: Float
|
|
let b: Float
|
|
switch i % 6 {
|
|
case 0: r = v; g = t; b = p
|
|
case 1: r = q; g = v; b = p
|
|
case 2: r = p; g = v; b = t
|
|
case 3: r = p; g = q; b = v
|
|
case 4: r = t; g = p; b = v
|
|
default: r = v; g = p; b = q
|
|
}
|
|
return SIMD4<Float>(r, g, b, 1.0)
|
|
}
|
|
}
|