End-to-end VisionPOC pipeline: Capture (Sources/Capture/CameraCapture.swift) - AVCaptureSession on a dedicated queue, BGRA @ 1080p - Zero-copy CVPixelBuffer -> MTLTexture via CVMetalTextureCache - os_unfair_lock around the latest-frame snapshot - Permission flow with denied state surfaced to the renderer Processing - ObjectDetector: VNGenerateObjectnessBasedSaliencyImageRequest is the primary all-objects path; face, human, and animal detectors layer in as labels. IOU-merge keeps labels and drops duplicate generic boxes. Throttled to Theme.Performance.detectorTargetHz on a serial queue so the renderer never blocks. Runs on the ANE via Vision's default routing. - EdgePass: MPSImageSobel into an intermediate texture, then an edge_threshold compute kernel binarizes to crisp white-on-black. - JarvisStylePass: builds the per-frame uniform struct (time, beam phase, scanline density, hex grid scale, tint). Render - Single MTKView render pass, four viewports, one HUD overlay - Live passthrough, Jarvis stylization, edge map, and live-plus-boxes - Corner brackets and pane labels composited with the boxes/text pipelines - Footer reports FPS and detector latency, rebuilt every 10 frames Shaders - quad_vertex emits a v-flipped fullscreen quad (origin-top-left camera textures display upright) - jarvis_fragment: luma -> cyan tint, scanlines, hex grid, scanning beam, vignette, light chromatic edge ghost - edges_fragment: cyan-tinted edge display with nearest sampling - box_fragment: dual-mode (hollow stroke for detections, solid fill for HUD chrome) selected by BoxUniforms.params.x - text_fragment: glyph texture multiplied by tint at fragment buffer 0 App - @main AppDelegate, regular activation policy, ⌘Q menu - Bundled-font registration via CTFontManagerRegisterFontsForURL - MainWindowController owns the MTKView and wires the pipeline Bumped deployment target to macOS 26 to match the installed SDK. Build verified: xcodegen generate && xcodebuild ... && open the .app — launches, render loop runs, camera permission prompt fires on first frame. Known cosmetic issue: Orbitron-Bold.ttf inherited from MetalPOC is a GitHub HTML page rather than a real font; Theme.Font falls back to Helvetica Neue. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
25 lines
967 B
Metal
25 lines
967 B
Metal
#include <metal_stdlib>
|
|
using namespace metal;
|
|
|
|
// QuadVaryings is also declared in Common.metal. Each .metal file compiles as
|
|
// its own translation unit, so redeclaring here is safe and keeps each shader
|
|
// readable in isolation.
|
|
struct QuadVaryings {
|
|
float4 position [[position]];
|
|
float2 uv;
|
|
};
|
|
|
|
// -----------------------------------------------------------------------------
|
|
// Live passthrough fragment.
|
|
//
|
|
// Samples the camera texture and returns it unmodified. Linear sampler with
|
|
// clamp-to-edge — the UVs come from quad_vertex (origin top-left) so the
|
|
// camera frame displays upright.
|
|
// -----------------------------------------------------------------------------
|
|
fragment float4 live_fragment(QuadVaryings in [[stage_in]],
|
|
texture2d<float> camTex [[texture(0)]]) {
|
|
constexpr sampler s(coord::normalized, filter::linear,
|
|
address::clamp_to_edge);
|
|
return camTex.sample(s, in.uv);
|
|
}
|