james.maesandClaude Opus 4.7
f6601fa593
End-to-end VisionPOC pipeline: Capture (Sources/Capture/CameraCapture.swift) - AVCaptureSession on a dedicated queue, BGRA @ 1080p - Zero-copy CVPixelBuffer -> MTLTexture via CVMetalTextureCache - os_unfair_lock around the latest-frame snapshot - Permission flow with denied state surfaced to the renderer Processing - ObjectDetector: VNGenerateObjectnessBasedSaliencyImageRequest is the primary all-objects path; face, human, and animal detectors layer in as labels. IOU-merge keeps labels and drops duplicate generic boxes. Throttled to Theme.Performance.detectorTargetHz on a serial queue so the renderer never blocks. Runs on the ANE via Vision's default routing. - EdgePass: MPSImageSobel into an intermediate texture, then an edge_threshold compute kernel binarizes to crisp white-on-black. - JarvisStylePass: builds the per-frame uniform struct (time, beam phase, scanline density, hex grid scale, tint). Render - Single MTKView render pass, four viewports, one HUD overlay - Live passthrough, Jarvis stylization, edge map, and live-plus-boxes - Corner brackets and pane labels composited with the boxes/text pipelines - Footer reports FPS and detector latency, rebuilt every 10 frames Shaders - quad_vertex emits a v-flipped fullscreen quad (origin-top-left camera textures display upright) - jarvis_fragment: luma -> cyan tint, scanlines, hex grid, scanning beam, vignette, light chromatic edge ghost - edges_fragment: cyan-tinted edge display with nearest sampling - box_fragment: dual-mode (hollow stroke for detections, solid fill for HUD chrome) selected by BoxUniforms.params.x - text_fragment: glyph texture multiplied by tint at fragment buffer 0 App - @main AppDelegate, regular activation policy, ⌘Q menu - Bundled-font registration via CTFontManagerRegisterFontsForURL - MainWindowController owns the MTKView and wires the pipeline Bumped deployment target to macOS 26 to match the installed SDK. Build verified: xcodegen generate && xcodebuild ... && open the .app — launches, render loop runs, camera permission prompt fires on first frame. Known cosmetic issue: Orbitron-Bold.ttf inherited from MetalPOC is a GitHub HTML page rather than a real font; Theme.Font falls back to Helvetica Neue. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
30 lines
859 B
Swift
30 lines
859 B
Swift
import Foundation
|
|
import CoreGraphics
|
|
import simd
|
|
|
|
struct JarvisUniforms {
|
|
var time: Float
|
|
var resolution: SIMD2<Float>
|
|
var scanlineDensity: Float
|
|
var hexGridScale: Float
|
|
var beamPhase: Float
|
|
var tintColor: SIMD4<Float>
|
|
}
|
|
|
|
final class JarvisStylePass {
|
|
init() {}
|
|
|
|
func uniforms(time: Float, viewportSize: CGSize) -> JarvisUniforms {
|
|
let period = Float(Theme.Tick.scanningBeamPeriod)
|
|
let phase = period > 0 ? (time / period).truncatingRemainder(dividingBy: 1.0) : 0
|
|
return JarvisUniforms(
|
|
time: time,
|
|
resolution: SIMD2<Float>(Float(viewportSize.width), Float(viewportSize.height)),
|
|
scanlineDensity: Theme.Tick.scanlineDensity,
|
|
hexGridScale: Theme.Tick.hexGridScale,
|
|
beamPhase: phase,
|
|
tintColor: Theme.Palette.cyan
|
|
)
|
|
}
|
|
}
|