Replaces the bundled detector with Ultralytics' YOLOv8x trained on
Open Images V7 — 601 object classes including coffee cup, mug, wine
glass, mobile phone, computer monitor, sunglasses, laptop, plus body
parts (human face / hand / eye / arm / leg). Substantially better
"is that a coffee mug" detection than the 80-class COCO model.
Conversion
- Ultralytics' yolov8x-oiv7.pt → Core ML mlpackage via
ultralytics.YOLO(...).export(format="coreml", nms=True). Output is a
Vision-friendly pipeline (preprocessing + model + NMS), so
VNCoreMLRequest continues to return [VNRecognizedObjectObservation]
directly with no manual decoding.
- scripts/convert-yolo-oiv7.py is the conversion entry point.
- scripts/fetch-model.sh rewritten as a no-LFS fallback that
bootstraps a Python venv, installs coremltools + ultralytics, runs
the conversion, and drops the result into Sources/Resources/Models/.
Bundling
- yolov8x-oiv7.mlpackage is a directory (~131 MB weight.bin + small
model.mlmodel descriptor + Manifest.json).
- .gitattributes adds LFS tracking for **/weight.bin and
**/model.mlmodel inside Sources/Resources/Models so the contents go
to LFS without the surrounding directory metadata pretending to be
binary.
- Old YOLOv3.mlmodel removed from the repo (still in LFS history if
needed).
Theme
- Theme.Performance.detectorModelName updated to "yolov8x-oiv7" (the
detector falls back to mlpackage extension automatically via the
existing Bundle.main.url lookup).
README + scripts updated to reflect the new model and 601-class
vocabulary.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Drops Apple's full YOLOv3 (~248 MB) into Sources/Resources/Models/ as the
primary object detector. Xcode auto-compiles the .mlmodel into a .mlmodelc
inside the app bundle; the model is a Vision-friendly pipeline so
VNCoreMLRequest returns VNRecognizedObjectObservation directly — no manual
NMS or output decoding required.
Detection pipeline
- YOLOv3 is the primary detector, returning bounding boxes labeled with
COCO classes: person, bicycle, car, ..., bottle, wine glass, cup, fork,
knife, spoon, bowl, ... laptop, mouse, remote, keyboard, cell phone, ...
book, clock, vase. 80 classes total — covers the "is that a coffee mug"
question and many others.
- VNDetectFaceRectanglesRequest stays as a supplementary FACE labeler
(more precise than YOLO's "person" for the face region).
- VNRecognizeAnimalsRequest stays for finer-grained cat/dog labels.
- Saliency and human-rectangle requests removed — YOLO supersedes them.
- Per-label NMS with IoU>0.5 inside filterAndMerge drops same-label
overlaps. Cross-label overlaps (e.g. person + face) are kept since they
carry different information.
- Min-confidence filter (Theme.Performance.detectionMinConfidence, 0.30)
prunes weak guesses.
Distribution
- Model is tracked via Git LFS (.gitattributes) — it exceeds GitHub's
100 MB per-file limit.
- README documents the `git lfs install && git lfs pull` prerequisite.
- scripts/fetch-model.sh remains as a no-LFS fallback that downloads the
same model from Apple's Core ML catalog.
Theme knobs
- Performance.detectionMinConfidence (default 0.30)
- Performance.detectorModelName (default "YOLOv3")
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
james.maesandClaude Opus 4.7