Both build on Vision's existing requests — no new model files, no
Python conversion. The FingerCounter reads named hand-joint positions
from the existing VNDetectHumanHandPoseRequest output; the
FacialExpressionAnalyzer reads outer-lip landmarks from a swapped-in
VNDetectFaceLandmarksRequest (a superset of the existing rectangles
request — same bbox, plus the landmarks).
## FingerCounter
For each detected hand:
- Index / middle / ring / pinky are "extended" when their tip is
farther from the wrist than the MCP base joint by >1.25× (tolerates
partial curl while rejecting fully-tucked fingers)
- Thumb is special — measures perpendicular distance from the wrist→
middleMCP axis; if the thumb tip is >0.045 normalized units lateral
to that line, it's extended
- Emits `gestureDetected("fingers_<N>", confidence)` on the event bus
where confidence is the average joint confidence
- Throttled to 1.5s per N — so showing 3 fingers, opening to 5,
closing to 3 will emit cleanly but holding 3 fingers up steady won't
flood the bus
## FacialExpressionAnalyzer
For each face with landmarks:
- Read `landmarks.outerLips.normalizedPoints` — closed contour around
the mouth
- Mouth corners = leftmost + rightmost x in the contour
- Centroid y = average of all contour y values
- Compare corner_avg_y vs centroid_y with a dead-band scaled to the
face's bounding-box height (default 1%):
- Corners above centroid → smile
- Corners below centroid → frown
- Within dead-band → neutral_face
- Emits `gestureDetected("smile" | "frown" | "neutral_face", confidence)`
- Throttled to 4s per expression name
## ObjectDetector wiring
- `VNDetectFaceRectanglesRequest` → `VNDetectFaceLandmarksRequest`
(still returns VNFaceObservation with .boundingBox for tracking +
face recognition, but now also populates .landmarks for expression)
- Two new private members: `fingerCounter` + `facialExpressionAnalyzer`
- After the face-result loop in `detectAndBootstrap`, both analyzers
receive the raw Vision observations (PoseDetection.points are
unnamed, so the analyzers go straight to the source)
History and Log Stream both display the new events automatically since
they reuse the existing `gestureDetected` case in DetectionEvent.Kind.
No event-vocabulary changes, no UI changes needed.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
