Three asks landing together. ## History view — now identical controls to Log Stream - Oldest-first order with auto-scroll-to-bottom on append (was newest- first, no auto-scroll). ScrollViewReader + onChange pattern lifted from LogStreamView for behavioral parity. - Toolbar now matches the Log Stream layout exactly: Category filter, Auto-scroll toggle, Pause toggle, filtered/total count, Clear button. - Poll rate bumped 1 Hz → 0.10 s so new events appear instantly. Pause toggle freezes refresh without dropping the running event collection. ## Proper line shader (much better hand + body rendering) - New `line_vertex` / `line_fragment` in Boxes.metal alongside the existing box shader. `LineUniforms` carries endpoints + thickness + soft-edge falloff + a dot-mode flag. - Line mode: emits a rotated thin quad oriented along (end - start) with width = thickness. Soft-edge antialiasing via smoothstep on the signed perpendicular UV makes diagonal skeleton segments read as actual strokes instead of axis-aligned bounding boxes. - Dot mode: emits a square around `start` and the fragment discards outside the unit-circle — produces filled disks for joint markers with sub-pixel-clean soft edges. - Pipelines gains a `lines` blended pipeline. - Renderer.drawPoses() rewritten to use the line pipeline — one draw per segment, one draw per joint, all visually consistent regardless of segment angle. Theme.Palette.bodyPose (mint) for body, handPose (amber) for hand. ## Status panel (⌘I) - New `CurrentStateStore` singleton subscribes to the event bus and retains the most recent value per concept: expression (smile / frown / neutral_face), activity (standing / sitting / etc.), per-chirality finger counts, plus a small ring of recent non-finger non-expression gestures. - New StatusView (SwiftUI) reads three sources at 4 Hz: the live ObjectDetector for FPS + DET timing + objects + faces, the CurrentStateStore for hands/expression/activity/recent-gestures, and the HistoryStore for an 8-entry tail of recent events. - New StatusWindowController hosts the SwiftUI view as a dark-aqua floating window with frame persistence (VPOC.StatusWindowFrame). - AppDelegate `openStatus(_:)` builds a DetectorSnapshot from the live windowController + detector each tick, partitioning detections into face entries (anything labeled FACE or matching an enrolled name) vs object entries. Object entries carry the same per-track color the JARVIS pane uses (via ColorHash.colorFor) so the panel's dots match the on-screen box colors. - Renderer exposes `smoothedFPSForReadout` so the panel can show the same FPS the footer does without reaching into private state. - FingerCounter now records its count per chirality via CurrentStateStore.recordFingers (in addition to the existing bus emit), using VNHumanHandPoseObservation.chirality. The panel displays "Left: 3 fingers / Right: 5 fingers" correctly. Menu structure now: - App menu: Settings (⌘,) · Log Stream (⌘L) · Status (⌘I) · Save Snapshot (⌘S) · Start/Stop Recording (⇧⌘R) · Quit - Faces menu: Enroll (⌘E) · Forget (⇧⌘E) · History (⌘H) - Camera menu: Devices ▶ · Refresh · Privacy Mode (⇧⌘P) Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
258 lines
8.9 KiB
Swift
258 lines
8.9 KiB
Swift
import SwiftUI
|
|
|
|
/// "What VisionPOC knows right now" panel. Reads three live sources at 4 Hz:
|
|
/// the ObjectDetector for current detections / FPS / inference timing, the
|
|
/// CurrentStateStore for hands / expression / activity / recent gestures,
|
|
/// and the HistoryStore for a small "recent events" tail.
|
|
///
|
|
/// Surfaced via ⌘I in the App menu.
|
|
struct StatusView: View {
|
|
/// Provided by the AppDelegate at window construction. Returns the
|
|
/// current state of the world or nil if the detector isn't reachable.
|
|
let detectorAccess: @MainActor () -> DetectorSnapshot?
|
|
|
|
@State private var snapshot: DetectorSnapshot = .empty
|
|
@State private var state: CurrentStateStore.Snapshot = CurrentStateStore.shared.snapshot()
|
|
@State private var recent: [HistoryEntry] = []
|
|
@State private var historyVersion: UInt64 = 0
|
|
@State private var stateVersion: UInt64 = 0
|
|
|
|
private let pollTimer = Timer.publish(every: 0.25, on: .main, in: .common).autoconnect()
|
|
|
|
var body: some View {
|
|
VStack(alignment: .leading, spacing: 0) {
|
|
scroll
|
|
}
|
|
.frame(minWidth: 360, idealWidth: 420, minHeight: 480, idealHeight: 640)
|
|
.background(Color(white: 0.07))
|
|
.onReceive(pollTimer) { _ in tick() }
|
|
.onAppear { tick() }
|
|
}
|
|
|
|
private func tick() {
|
|
if let s = detectorAccess() { snapshot = s }
|
|
let sv = CurrentStateStore.shared.version
|
|
if sv != stateVersion {
|
|
state = CurrentStateStore.shared.snapshot()
|
|
stateVersion = sv
|
|
}
|
|
let hv = HistoryStore.shared.version
|
|
if hv != historyVersion {
|
|
let entries = HistoryStore.shared.snapshot()
|
|
recent = Array(entries.reversed().prefix(8))
|
|
historyVersion = hv
|
|
}
|
|
}
|
|
|
|
@ViewBuilder
|
|
private var scroll: some View {
|
|
ScrollView {
|
|
VStack(alignment: .leading, spacing: 16) {
|
|
header
|
|
Divider().background(Color(white: 0.18))
|
|
objectsSection
|
|
facesSection
|
|
handsSection
|
|
expressionActivitySection
|
|
recentSection
|
|
}
|
|
.padding(16)
|
|
}
|
|
}
|
|
|
|
@ViewBuilder
|
|
private var header: some View {
|
|
VStack(alignment: .leading, spacing: 4) {
|
|
Text("VISIONPOC STATUS")
|
|
.font(.system(size: 11, weight: .semibold))
|
|
.tracking(1.5)
|
|
.foregroundStyle(Color(red: 0.20, green: 0.90, blue: 0.95))
|
|
HStack(spacing: 16) {
|
|
kv("FPS", String(format: "%.1f", snapshot.fps))
|
|
kv("DET", String(format: "%.1f ms (%@)", snapshot.lastInferenceMs, snapshot.detectMode))
|
|
}
|
|
}
|
|
}
|
|
|
|
@ViewBuilder
|
|
private var objectsSection: some View {
|
|
section(title: "OBJECTS (\(snapshot.objects.count))",
|
|
color: Color(red: 0.20, green: 0.90, blue: 0.95)) {
|
|
if snapshot.objects.isEmpty {
|
|
empty("nothing tracked")
|
|
} else {
|
|
ForEach(Array(snapshot.objects.enumerated()), id: \.offset) { _, item in
|
|
HStack(spacing: 8) {
|
|
Circle()
|
|
.fill(item.color)
|
|
.frame(width: 8, height: 8)
|
|
Text(item.label.lowercased())
|
|
.foregroundStyle(.primary)
|
|
Spacer()
|
|
Text("\(Int(item.confidence * 100))%")
|
|
.foregroundStyle(.secondary)
|
|
.font(.system(.caption, design: .monospaced))
|
|
}
|
|
.font(.system(.caption, design: .monospaced))
|
|
}
|
|
}
|
|
}
|
|
}
|
|
|
|
@ViewBuilder
|
|
private var facesSection: some View {
|
|
section(title: "FACES (\(snapshot.faces.count))",
|
|
color: Color(red: 0.30, green: 1.00, blue: 0.75)) {
|
|
if snapshot.faces.isEmpty {
|
|
empty("no face in frame")
|
|
} else {
|
|
ForEach(Array(snapshot.faces.enumerated()), id: \.offset) { _, face in
|
|
HStack {
|
|
Text(face.name)
|
|
.foregroundStyle(.primary)
|
|
Spacer()
|
|
if let d = face.distance {
|
|
Text(String(format: "dist %.1f", d))
|
|
.foregroundStyle(.secondary)
|
|
.font(.system(.caption, design: .monospaced))
|
|
}
|
|
}
|
|
.font(.system(.caption, design: .monospaced))
|
|
}
|
|
}
|
|
}
|
|
}
|
|
|
|
@ViewBuilder
|
|
private var handsSection: some View {
|
|
section(title: "HANDS",
|
|
color: Color(red: 1.00, green: 0.75, blue: 0.20)) {
|
|
if state.fingerCounts.isEmpty {
|
|
empty("no hands in frame")
|
|
} else {
|
|
let keys = state.fingerCounts.keys.sorted()
|
|
ForEach(keys, id: \.self) { key in
|
|
HStack {
|
|
Text(key.capitalized)
|
|
.foregroundStyle(.primary)
|
|
.frame(width: 80, alignment: .leading)
|
|
Text("\(state.fingerCounts[key] ?? 0) fingers")
|
|
.foregroundStyle(.secondary)
|
|
Spacer()
|
|
}
|
|
.font(.system(.caption, design: .monospaced))
|
|
}
|
|
}
|
|
}
|
|
}
|
|
|
|
@ViewBuilder
|
|
private var expressionActivitySection: some View {
|
|
section(title: "STATE",
|
|
color: Color(red: 0.80, green: 0.55, blue: 1.00)) {
|
|
HStack {
|
|
Text("Expression")
|
|
.frame(width: 100, alignment: .leading)
|
|
.foregroundStyle(.primary)
|
|
Text(state.expression ?? "—")
|
|
.foregroundStyle(.secondary)
|
|
Spacer()
|
|
}
|
|
.font(.system(.caption, design: .monospaced))
|
|
HStack {
|
|
Text("Activity")
|
|
.frame(width: 100, alignment: .leading)
|
|
.foregroundStyle(.primary)
|
|
Text(state.activity ?? "—")
|
|
.foregroundStyle(.secondary)
|
|
Spacer()
|
|
}
|
|
.font(.system(.caption, design: .monospaced))
|
|
}
|
|
}
|
|
|
|
@ViewBuilder
|
|
private var recentSection: some View {
|
|
section(title: "RECENT",
|
|
color: Color(white: 0.6)) {
|
|
if recent.isEmpty {
|
|
empty("waiting for events")
|
|
} else {
|
|
ForEach(recent) { entry in
|
|
HStack(alignment: .top, spacing: 8) {
|
|
Text(Self.timeFormatter.string(from: entry.timestamp))
|
|
.foregroundStyle(.secondary)
|
|
.frame(width: 70, alignment: .leading)
|
|
Text(entry.summary)
|
|
.foregroundStyle(.primary)
|
|
.lineLimit(1)
|
|
Spacer()
|
|
}
|
|
.font(.system(.caption, design: .monospaced))
|
|
}
|
|
}
|
|
}
|
|
}
|
|
|
|
// MARK: - View builders
|
|
|
|
@ViewBuilder
|
|
private func section<Content: View>(title: String, color: Color, @ViewBuilder content: () -> Content) -> some View {
|
|
VStack(alignment: .leading, spacing: 6) {
|
|
Text(title)
|
|
.font(.system(size: 10, weight: .semibold))
|
|
.tracking(1.5)
|
|
.foregroundStyle(color)
|
|
content()
|
|
}
|
|
}
|
|
|
|
@ViewBuilder
|
|
private func kv(_ label: String, _ value: String) -> some View {
|
|
HStack(spacing: 6) {
|
|
Text(label)
|
|
.foregroundStyle(.secondary)
|
|
.font(.system(size: 10, weight: .semibold))
|
|
.tracking(1)
|
|
Text(value)
|
|
.foregroundStyle(.primary)
|
|
.font(.system(.caption, design: .monospaced))
|
|
}
|
|
}
|
|
|
|
@ViewBuilder
|
|
private func empty(_ text: String) -> some View {
|
|
Text(text)
|
|
.foregroundStyle(.secondary)
|
|
.font(.system(.caption, design: .monospaced))
|
|
.italic()
|
|
}
|
|
|
|
private static let timeFormatter: DateFormatter = {
|
|
let df = DateFormatter()
|
|
df.dateFormat = "HH:mm:ss.SSS"
|
|
return df
|
|
}()
|
|
}
|
|
|
|
/// What StatusView needs from the detector each tick. Built on MainActor
|
|
/// from the live ObjectDetector + Renderer.
|
|
struct DetectorSnapshot: Sendable {
|
|
struct ObjectEntry: Sendable {
|
|
let label: String
|
|
let confidence: Float
|
|
let color: Color
|
|
}
|
|
struct FaceEntry: Sendable {
|
|
let name: String
|
|
let distance: Float?
|
|
}
|
|
var fps: Double = 0
|
|
var lastInferenceMs: Double = 0
|
|
var detectMode: String = "—"
|
|
var objects: [ObjectEntry] = []
|
|
var faces: [FaceEntry] = []
|
|
|
|
static let empty = DetectorSnapshot()
|
|
}
|