From c4c32f7aaefa9e884c261aa474da7f1d2fb04930 Mon Sep 17 00:00:00 2001 From: James Maes Date: Tue, 12 May 2026 15:00:21 -0500 Subject: [PATCH 1/2] =?UTF-8?q?docs(readme):=20refresh=20=E2=80=94=20diagr?= =?UTF-8?q?ams=20+=20architecture=20+=20culture=20+=20roadmap?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Audit-and-rewrite of root README against current truth (2026-05-12). The previous README was accurate but list-shaped and unillustrated; the rewrite reorients around what a new reader (human or AI agent) needs in 60 seconds. New / changed sections: - Tagline + two-paragraph intro (what / who / why) - Status banner pointing at GitHub Issues + Milestones (canonical) - Screenshots placeholder + shot list under assets/screenshots/ - "Architecture at a glance" with a top-level mermaid flowchart (Swift host + LLM providers + WKWebView, with 14 nodes) - Subsystems table — eight first-class + supporting packages - "Install order" section with a mermaid DAG of the AppDelegate install sequence (vision → agent → voice, plus parallel memory/MCP/replay), matching check-install-order.sh - "Agent turn lifecycle" mermaid showing the orchestrator → single broadcaster fanout, with the strict-mode degradation preamble inject point called out - "Boot-health probes" mermaid for the eight NOT FAKED probes → rollup → menu-bar icon / banner / agent preamble - "Project culture" section codifying the in-code-docs philosophy, NOT FAKED probes, strict-mode escalation, 18 boundary gates, the three-name test-double taxonomy, audit-and-stabilize cadence, and the GitHub-Issues-is-canonical policy - "Roadmap" reorganized around the v0.1 → v1.1 milestones from GitHub Milestones (user-outcome-focused, not subsystem-focused) - Build & run cleaned up; boundary-gates one-liner; opt-in real-hardware test commands kept Also adds assets/screenshots/README.md with a prioritized shot list — eight screenshots in priority order — and a note on why diagrams stay as inline mermaid (GitHub-native, no re-export). Removed: the older "Status" subsystem-state table, which had become a mix of v0.12.0 milestone audit terms (Track A/B/C/D, B-8) that don't map cleanly to the new GitHub Issues taxonomy. Status now points at GitHub Issues + Milestones as the source of truth. Gates: 17/18 pass. check-app-builds.sh fails on the in-flight AppDelegate.swift rewrite (unstaged, owned by a parallel agent) — unrelated to this markdown-only change. Confirmed only README.md + assets/screenshots/README.md are staged here. Co-Authored-By: Claude Opus 4.7 --- README.md | 342 ++++++++++++++++++++++++++++++----- assets/screenshots/README.md | 50 +++++ 2 files changed, 345 insertions(+), 47 deletions(-) create mode 100644 assets/screenshots/README.md diff --git a/README.md b/README.md index 95d6adf..00120ae 100644 --- a/README.md +++ b/README.md @@ -1,83 +1,195 @@ # Jarvis -A personal, always-on macOS AI assistant styled after the Iron Man HUD. Native Swift host with an embedded React + React Three Fiber HUD, driven by an agent loop that streams Claude Opus 4.7 (primary) or local Ollama (`qwen2.5-coder:32b`) over an `LLMProvider` protocol, with MCP tool servers, fully-local voice (openWakeWord + SpeechAnalyzer + AVSpeechSynthesizer / Orpheus), local memory (SQLite + FTS5 + sqlite-vec), and on-device vision capture. Personal use only — single user, single machine. +> A personal, always-on macOS AI assistant in the shape of the Iron Man HUD — +> native Swift brain, R3F particle-ring face, voice-first, locally-grounded. -## Status +Jarvis is a hybrid native + embedded-web macOS app that runs as an ambient +menu-bar presence. A Swift host owns the operating system (mic, camera, +clipboard, AppleScript, hotkeys, persistent storage) and an agent loop that +streams **Claude Opus 4.7** with **Ollama** as a first-class local fallback. +A `WKWebView` mounts a React + **React Three Fiber** HUD — a single pulsing +particle ring plus a chat panel — purely as the rendering surface. Tools +ride on **MCP** (Model Context Protocol). Voice (wake-word, STT, TTS) and +memory (SQLite + FTS5 + `sqlite-vec`, mem0-style extraction) are **fully +local**: no cloud TTS/STT, no remote embeddings. -`develop` at app version **0.1.0** (Info.plist `CFBundleShortVersionString`). Nine planned phases shipped; Phase 10 (self-awareness diagnostics) mid-flight — Wave 1 landed at `3974a15` (four self-knowledge MCP tools). Post-milestone audit-and-stabilize closures (Tracks A, B-1..7, C-1..6, D-1..6) are in. Day-to-day state lives in [`docs/SESSION-STATE.md`](docs/SESSION-STATE.md) and [`docs/TODO.md`](docs/TODO.md); milestone-level state in [`.planning/STATE.md`](.planning/STATE.md). +It is built for one user on one machine — the author. There is no +multi-tenancy, no auth, no commercial framing. The repo is public-shaped +purely for portability across the author's machines and so AI coding agents +can read it. -What works today, against the live tree: +> **Status:** alpha / under active development. `develop` is `0.1.0`. The +> nine GSD phases shipped; the project is now in an audit-and-stabilize +> window plus a v1.0 typed-API migration. See [Roadmap](#roadmap) and +> [GitHub Issues](https://github.com/KofTwentyTwo/Jarvis/issues) for what +> is done versus in flight. -| Modality | State | -|----------|-------| -| Text turns (Anthropic + Ollama) | Working end-to-end; HUD chat panel + token streaming | -| Voice loop (wake word → VAD-gated STT → orchestrator → tier-1 TTS) | Wired end-to-end; HUMAN-UAT pending on real hardware | -| Tier-2 TTS (Orpheus) | Deferred — `B-8` requires ~6 GB MLX weight download; degrades to tier-1 | -| Vision (camera capture, frame-attach to a turn) | Wired; T2 vision provider explicitly missing pending sidecar | -| Memory (extraction + hybrid search) | Code-complete; functionally OFF until `vec0.dylib` ships (Track D-5/D-6) and `ollama pull nomic-embed-text` runs (D-7) | -| MCP tools | Helpers: `get_time`, `get_clipboard`, `run_applescript` (HUD-confirmation-gated). In-process: `search_memory`, `forget_fact` (memory; gated on store + vec availability), and the Phase 10 Wave 1 self-knowledge set — `list_audio_devices`, `get_active_audio_route`, `get_self_state`, `list_camera_devices` (`74b9ac2`). | +## Screenshots -Personal-use only; not multi-tenant, not auth'd, not a commercial product. See [`.planning/PROJECT.md`](.planning/PROJECT.md) for project context. +Screenshots are TBD — when there is something worth showing, it lands in +[`assets/screenshots/`](assets/screenshots/) per the shot list in that +directory's `README.md`. -## Architecture quick-take +- HUD particle ring across the four agent states (idle / listening / thinking / speaking) +- Status panel (menu-bar → *Status…*) with the live boot-health probe set +- Dev Overlay tabs (Turns / Tools / Bus / Logs) +- Non-dismissible degradation banner (critical-severity probe failure) -- **Native Swift / SwiftUI host** owns window management, OS integration (mic, camera, AppleScript, hotkeys, file I/O), the agent orchestrator, and persistent storage. -- **`WKWebView` hosts a React + React Three Fiber HUD** for visuals only — particle ring, chat panel, camera button. The webview is a pure rendering layer; Swift holds all truth. -- **Bidirectional typed JSON Bus** over `WKScriptMessageHandler` ([`packages/Bus`](packages/Bus)) carries state changes Swift→JS and user actions JS→Swift. -- **`LLMProvider` protocol** is provider-agnostic: `AnthropicProvider` (Opus 4.7 streaming) and `OllamaProvider` (`/api/chat` + `/v1/chat/completions`) both conform. Swap is a config toggle. -- **MCP for tools, separately codesigned helper apps under `Contents/Helpers/`.** Each helper has its own TCC identity; `mcp-applescript` is the only one with Apple Events entitlement. +## Architecture at a glance -Full detail in [`ARCHITECTURE.md`](ARCHITECTURE.md). +A two-process, one-bridge shape: Swift owns truth, the webview is a pure +view, the Bus is the only legal way they talk. Subsystems live in +single-purpose SPM packages; **18 boundary-gate scripts** keep the +isolation honest. -## Repository layout +```mermaid +flowchart TB + User[User
voice / text / hotkey] -| Path | Purpose | -|------|---------| -| [`App/`](App) | Swift host — `AppDelegate`, MenuBar, HUD window, Settings, Wizard, MCP runtime wiring | -| [`packages/`](packages) | 13 Swift packages under SPM (one per subsystem). See [`ARCHITECTURE.md`](ARCHITECTURE.md) for the dependency graph. | -| [`webview/`](webview) | pnpm workspace — `@jarvis/bus` (TS mirror of the Swift Bus protocol) and `@jarvis/hud` (React + R3F HUD bundle) | -| [`mcp-servers/`](mcp-servers) | Out-of-process MCP helpers — `mcp-time`, `mcp-clipboard`, `mcp-applescript` | -| [`scripts/`](scripts) | Build/codesign scripts and 18 boundary-gate `check-*.sh` invariant linters | -| [`Resources/`](Resources) | Bundled assets — voice ONNX models, HUD bundle target, app icon source | -| [`docs/`](docs) | Live cross-session state — `SESSION-STATE.md`, `TODO.md` | -| [`.planning/`](.planning) | GSD workflow artifacts — phase plans/summaries, audit reports, research syntheses, roadmap | -| [`CLAUDE.md`](CLAUDE.md) | Project conventions and architectural decisions for AI coding agents | + subgraph Swift["Swift host (AppKit + SwiftUI)"] + AppDel[AppDelegate
install DAG + lifecycle] + Orch[AgentOrchestrator
tool-loop + streaming] + Bridge[WebviewBridge
typed JSON Bus] + Voice[Voice
wake / STT / TTS] + Vision[Vision
camera + frame-attach] + Memory[Memory
SQLite + vec + mem0] + MCP[MCPRuntime
helpers + in-proc tools] + Replay[Replay
turn-by-turn audit log] + Boot[BootHealth
NOT FAKED probes] + end -## Build & test + LLM["LLMProvider
Anthropic Opus 4.7 / Ollama"] -Requires Xcode 16+ on macOS 26 Tahoe (Apple Silicon). + subgraph Web["WKWebView (React + R3F)"] + Ring[Particle ring] + Chat[Chat panel] + DevOv[Dev Overlay] + end -**SPM packages — fast, offline:** + User --> Voice + User --> Bridge + AppDel --> Vision + AppDel --> Orch + AppDel --> Voice + AppDel --> Boot + Voice --> Orch + Orch <--> LLM + Orch --> MCP + Orch --> Memory + Orch --> Vision + Orch --> Replay + Orch --> Bridge + Boot --> Bridge + Bridge <-->|Bus protocol| Web +``` + +The deep architecture doc is [`ARCHITECTURE.md`](ARCHITECTURE.md) (subsystem +deep-dives, anti-patterns, navigation cheat sheet). The v1.0 typed-API +contract that is currently being migrated to lives at +[`.planning/architecture/JARVIS-API-DESIGN-v0.2.md`](.planning/architecture/JARVIS-API-DESIGN-v0.2.md); +the rollout phases are in +[`.planning/architecture/JARVIS-API-MIGRATION-PLAN.md`](.planning/architecture/JARVIS-API-MIGRATION-PLAN.md). + +## Subsystems + +Every subsystem is one SPM package under [`packages/`](packages/). The Bus +sits in the middle as the only legal Swift↔JS channel. + +| Subsystem | What it does | Package | +|-----------|--------------|---------| +| Agent loop | Claude Opus 4.7 + Ollama via `LLMProvider`; tool-call loop; streaming events to a single broadcaster | [`packages/AgentCore`](packages/AgentCore) | +| Bus | Typed JSON message bus over `WKScriptMessageHandler`; protocol-versioned; harness-parity-checked | [`packages/Bus`](packages/Bus) | +| Voice | openWakeWord (`hey_jarvis`) + Silero VAD v6.2.1 + SpeechAnalyzer STT + AVSpeech / Orpheus TTS | [`packages/Voice`](packages/Voice) | +| Vision | AVCaptureSession camera capture + frame-attach to a turn; T2 presence pipeline deferred | [`packages/Vision`](packages/Vision) | +| Memory | SQLite with FTS5 + `sqlite-vec` (statically linked) + mem0 ADD/UPDATE/NOOP extractor on Qwen 2.5-Coder | [`packages/Memory`](packages/Memory) | +| MCP | Model Context Protocol runtime — separately codesigned helper apps under `Contents/Helpers/` plus in-process tools | [`packages/MCP`](packages/MCP) | +| Replay | Per-turn audit log to a local SQLite — user input, LLM output, tool calls, tool results | [`packages/Replay`](packages/Replay) | +| BootHealth | Eight NOT FAKED probes feeding a strict-mode severity rollup that drives HUD chrome + agent preamble | [`packages/AgentCore`](packages/AgentCore) (`BootHealth*`) | +| HUD | React + R3F particle ring + chat panel + Dev Overlay tabs | [`webview/`](webview) | + +Supporting packages: [`Config`](packages/Config), [`DevOverlay`](packages/DevOverlay), +[`Harness`](packages/Harness) (`Mock*`/`Stub*`/`Fake*` doubles per the test naming +convention in `CLAUDE.md`), [`Keychain`](packages/Keychain), [`Logging`](packages/Logging), +[`Shell`](packages/Shell), [`VoiceLog`](packages/VoiceLog). + +## Install order + +The Swift host installs subsystems in a fixed DAG in +`AppDelegate.applicationWillFinishLaunching`. Order is **load-bearing** — +`packages/AgentCore`'s orchestrator constructor takes the `VisionRouter` (so +Vision installs first), Voice's adapters require the orchestrator and the +turn-transcript store (so Agent installs before Voice). The +[`scripts/check-install-order.sh`](scripts/check-install-order.sh) gate +asserts the spawn-line ordering AND that `voiceInstallTask` awaits +`agentInstallTask?.value` (a strictly stronger guarantee than spawn-order +alone). + +```mermaid +flowchart LR + Boot([applicationWillFinishLaunching]) --> Vision[installVision] + Boot --> Memory[installMemory] + Boot --> MCP[installMCP] + Boot --> Replay[installReplay] + Vision --> Agent[installAgent] + Memory --> Agent + MCP --> Agent + Replay --> Agent + Agent --> Voice[installVoice] + Voice --> Probes[registerBootHealthProbes] + Probes --> Bridge[Webview handshake] + Bridge --> Ready([HUD ready]) +``` + +## Build & run + +Requires **Xcode 16+ on macOS 26 Tahoe (Apple Silicon)**. The Hardened +Runtime needs `com.apple.security.cs.allow-jit` for the WKWebView JIT; +`SpeechAnalyzer` needs `com.apple.developer.speech-recognition-assets`. +Both are wired in `App/Jarvis.entitlements` — do not strip them. + +**SPM packages (fast, offline, the default dev loop):** ```bash swift test --package-path packages/AgentCore +swift test --package-path packages/Bus swift test --package-path packages/Voice swift test --package-path packages/Memory -# … one per package; see CLAUDE.md > Build / test for the full list +# … one per package; full list in CLAUDE.md → Build / test. ``` -**App target** (`xcodebuild build -configuration Debug` wrapper — `swift test` doesn't compile the App target on Xcode 26): +Single-test filter: `swift test --package-path packages/ --filter /`. + +**App target** — `swift test` does not compile the App target under Xcode +26 (the xctest harness is upstream-broken on ad-hoc Debug bundles), so use +the wrapper that drives `xcodebuild build -configuration Debug`: ```bash bash scripts/check-app-builds.sh ``` -**Webview HUD bundle:** +**Webview (HUD bundle):** ```bash bash scripts/build-webview.sh -# or, in webview/ during dev: +# or, during dev: cd webview && pnpm install && pnpm --filter @jarvis/hud dev cd webview && pnpm --filter @jarvis/hud test # vitest ``` -**Boundary gates** (18 grep-based architectural invariant checks; run before pushing): +**Boundary gates (18 grep-based architectural invariants — run before pushing):** ```bash for s in scripts/check-*.sh; do bash "$s" || break; done ``` -**Real-hardware / real-network env-flag tests** (interactive only): +The gates encode invariants the type system can't — single-writer +`HUDState`, single-consumer orchestrator events, no `evaluateJavaScript` +outside the Bus, vision isolation, presence-bus disallowed in TTS path, +embedding-dim literal pinned, install order, applescript-confirmation, +Bus protocol version + harness parity, no leftover stubs, no modal +presentation, and a handful more. + +**Real-hardware / real-network env-flag gates (interactive, opt-in):** ```bash JARVIS_REAL_MODELS=1 swift test --package-path packages/Memory --filter MemoryRegressionCorpusTests @@ -85,14 +197,150 @@ JARVIS_REAL_CAMERA=1 swift test --package-path packages/Vision --filter CameraCa JARVIS_REAL_MODELS=1 swift test --package-path packages/Voice --filter OrpheusTTFATests ``` +## Agent turn lifecycle + +Every user turn flows through a single broadcaster and fans out to the +subscribers that need it. If a subsystem is degraded, the orchestrator +**injects a system-prompt preamble** so the agent doesn't promise to +remember things the memory store can't actually persist. + +```mermaid +flowchart LR + In[User input
text / voice transcript] --> Orch[AgentOrchestrator] + Health[BootHealth.rollup] -->|degradation preamble| Orch + Orch --> Provider[LLMProvider.stream] + Provider --> Events[(streaming events
token / thinking / tool-use / stop)] + Events --> Bcast[Broadcaster
single consumer] + Bcast --> Tx[Transcript] + Bcast --> Mem[Memory extractor] + Bcast --> Dev[DevOverlay] + Bcast --> Voice[Voice / TTS] + Bcast --> Bus[WebviewBridge → HUD] + Orch --> Tools[MCP tool calls] + Tools --> Provider +``` + +## Boot-health probes + +Eight probes run on a strict-mode escalation ladder: +`ok → soft → loud → critical`. `.unknown(reason:)` is honest absence; a +probe never returns `.ok` when the dependency is actually broken. Critical +failures turn the menu-bar icon **red**, raise a non-dismissible HUD +banner, and inject the degradation preamble into the agent's system +prompt. + +```mermaid +flowchart LR + M[MemoryBootHealthProbe
vec0 + FTS5 reachable] + A[AnthropicBootHealthProbe
keychain has API key] + O[OllamaBootHealthProbe
service reachable + model loaded] + Vc[VoiceBootHealthProbe
audio graph alive] + Vi[VisionBootHealthProbe
AVCapture authorized] + Mc[MCPBootHealthProbe
helper apps + in-proc tools] + R[ReplayBootHealthProbe
WAL writable] + W[WebviewBootHealthProbe
handshake completed] + Roll[BootHealthOrchestrator.rollup] + Icon[Menu-bar icon
red on critical] + Banner[Non-dismissible banner] + Pre[Agent system-prompt
degradation preamble] + M --> Roll + A --> Roll + O --> Roll + Vc --> Roll + Vi --> Roll + Mc --> Roll + R --> Roll + W --> Roll + Roll --> Icon + Roll --> Banner + Roll --> Pre +``` + +## Project culture + +A few opinions, encoded into the codebase as gates and conventions: + +- **In-code documentation only.** No sprawling `/docs` that goes stale the + day after it's written. The code teaches itself or it doesn't get + taught. The full philosophy lives in + [`.planning/code-commenting-guide.md`](.planning/code-commenting-guide.md). + This README is the one exception — it is documentation **of** the + documentation strategy, which is allowed. +- **NOT FAKED probes.** Every subsystem has a live-probe boot-health + check. `.unknown(reason:)` is the honest answer when we can't tell; + `.ok` requires evidence. The audit-and-stabilize discipline came from + finding too many "wired but dead" subsystems that *looked* installed + but had no observable signal. +- **Strict-mode escalation.** Soft failures are logged; loud failures + surface in the Status panel; critical failures take over the chrome and + the agent's system prompt. The user always knows when Jarvis is + degraded. +- **18 boundary gates** encode architectural invariants the type system + can't (single-writer state, isolated subsystems, no rogue + `evaluateJavaScript`, pinned embedding dim, install order, …). +- **Three-name test-double taxonomy.** `Mock*` records + scripts; + `Stub*` returns canned data; `Fake*` is a behaviorally-realistic + alternative implementation. Pick by role, not by sibling. +- **Audit-and-stabilize cadence.** Multi-agent audits land under + `.planning/audit-YYYY-MM-DD/` (most recent: + [`audit-2026-05-12`](.planning/audit-2026-05-12/)) and surface + wired-but-dead bugs as GitHub Issues with severity + area labels. +- **GitHub Issues is canonical.** All bugs, audit findings, and migration + work live as issues. `docs/TODO.md` is historical only. + +## Roadmap + +The roadmap is owned by [GitHub Milestones](https://github.com/KofTwentyTwo/Jarvis/milestones) +— user-outcome-focused, not subsystem-focused: + +| Milestone | Theme | +|-----------|-------| +| **v0.1** — *It actually works* | Closes the CRITICAL audit findings that break basic capabilities (voice, vision, facts, dialogs). Some get re-fixed during the API migration; v0.1 ships working NOW. | +| **v0.2** — *Observable + stable* | Everything CRITICAL/HIGH from the 2026-05-12 audits that isn't a basic-capability blocker; tightens the observability surface; closes wired-but-dead gaps. | +| **v0.5** — *API M-0..M-3* | Scaffold `JarvisAPI` package, dual-version Bus handshake, then land Self / Settings / Diagnostics typed surfaces. HUD state moves to Diagnostics. | +| **v0.8** — *API M-4..M-7* | Memory / Vision / Voice / Turn typed surfaces. Per-turn `OutboundBatcher` partitioning. Webview retires the `sessionHistory` push. Bus protocol bumps to 2.4.0 strict after a 3-day soak. | +| **v1.0** — *Shipping* | Code-commenting style applied to load-bearing files. Stale plan-id gate. Doc reconciliation. First-launch experience polish. | +| **v1.1** — *Backlog* | Orpheus TTS tier 2 activation. WhisperKit STT fallback. Voice Log menu. Vision T2 sidecar. Presence pipeline. Advanced polish. | + +Day-to-day session handoff still lives in +[`docs/SESSION-STATE.md`](docs/SESSION-STATE.md); milestone-level state in +[`.planning/STATE.md`](.planning/STATE.md). Per-phase artifacts under +[`.planning/phases//`](.planning/phases/). + +## Repository layout + +| Path | Purpose | +|------|---------| +| [`App/`](App) | Swift host — `AppDelegate`, MenuBar, HUD window, Settings, Wizard, MCP runtime wiring | +| [`packages/`](packages) | 14 Swift packages (SPM) — one per subsystem | +| [`webview/`](webview) | pnpm workspace — `@jarvis/bus` (TS mirror of the Swift Bus) + `@jarvis/hud` (R3F bundle) | +| [`mcp-servers/`](mcp-servers) | Out-of-process MCP helper apps — `mcp-time`, `mcp-clipboard`, `mcp-applescript` | +| [`scripts/`](scripts) | Build/codesign scripts and the 18 boundary-gate linters | +| [`Resources/`](Resources) | Bundled assets — voice ONNX models, HUD bundle target, app icon source | +| [`docs/`](docs) | Live cross-session state — `SESSION-STATE.md`, archived `TODO.md` | +| [`.planning/`](.planning) | GSD workflow artifacts — phase plans, audit reports, research syntheses, architecture | +| [`assets/`](assets) | Screenshots + diagrams (populated opportunistically) | + ## Key documents -- [`ARCHITECTURE.md`](ARCHITECTURE.md) — top-level diagram, subsystem deep-dives, anti-patterns, navigation cheat sheet -- [`CONTRIBUTING.md`](CONTRIBUTING.md) — dev environment setup, GSD workflow, commit conventions, anti-patterns -- [`CLAUDE.md`](CLAUDE.md) — settled architectural decisions and AI agent collaboration conventions -- [`docs/SESSION-STATE.md`](docs/SESSION-STATE.md) — current session state, recent commits, what's next -- [`docs/TODO.md`](docs/TODO.md) — live triage list +- [`ARCHITECTURE.md`](ARCHITECTURE.md) — deep architecture doc +- [`CONTRIBUTING.md`](CONTRIBUTING.md) — dev environment setup, GSD workflow, commit conventions +- [`CLAUDE.md`](CLAUDE.md) — settled architectural decisions, AI-agent collaboration conventions, issue-tracking policy +- [`.planning/code-commenting-guide.md`](.planning/code-commenting-guide.md) — the in-code documentation philosophy +- [`.planning/architecture/JARVIS-API-DESIGN-v0.2.md`](.planning/architecture/JARVIS-API-DESIGN-v0.2.md) — v1.0 typed-API contract (locked) +- [`.planning/architecture/JARVIS-API-MIGRATION-PLAN.md`](.planning/architecture/JARVIS-API-MIGRATION-PLAN.md) — M-0..M-7 rollout +- [`docs/SESSION-STATE.md`](docs/SESSION-STATE.md) — current session state + +## Contributing + +This is a solo project. If you've stumbled across it and want to discuss +something, the welcoming move is to +[open a GitHub issue](https://github.com/KofTwentyTwo/Jarvis/issues/new). +Style, commit conventions, and the GSD workflow live in +[`CONTRIBUTING.md`](CONTRIBUTING.md). ## License -Personal use only. No license is granted; no contributions are solicited from outside collaborators. The repository is public-shaped purely for portability across the author's machines. +Personal use only. No license is granted; no contributions are solicited +from outside collaborators. The repository is public-shaped purely for +portability across the author's machines. diff --git a/assets/screenshots/README.md b/assets/screenshots/README.md new file mode 100644 index 0000000..0558430 --- /dev/null +++ b/assets/screenshots/README.md @@ -0,0 +1,50 @@ +# Screenshots + +The root [`README.md`](../../README.md) references images from this +directory but currently ships zero of them — there's no point inventing +screenshots before there's something worth showing. + +When a screenshot lands here, name it after its referent (`hud-ring-idle.png`, +not `Screen Shot 2026-05-12 at 4.32.png`), keep it under 800 KB (PNG with +`pngquant --quality 70-85`), and reference it from the README with a +descriptive alt-text. + +## Shot list (priority order) + +When the user has cycles, the highest-signal shots to capture: + +1. **`hud-ring-idle.png`** — particle ring at rest, dim cyan. Establishes + the visual identity in the README's hero slot. +2. **`hud-ring-thinking.gif`** *(optional, animated)* — ring rotating during + a streamed Opus turn. Sells "this is alive." +3. **`hud-ring-states.png`** — 2×2 grid of the four agent states (idle / + listening / thinking / speaking) for the "Subsystems" section's HUD + row. +4. **`status-panel.png`** — menu-bar → *Status…* showing the eight + boot-health probes with mixed states (some `.ok`, one `.soft`, one + `.unknown`). Demonstrates the NOT-FAKED probe culture concretely. +5. **`degradation-banner.png`** — a non-dismissible HUD banner with a + critical-severity failure (e.g., "Memory store unreachable — Jarvis + will not remember new facts this session"). Shows the strict-mode + escalation in action. +6. **`devoverlay-turns.png`** — the Dev Overlay's Turns tab during a + live conversation, with one expanded turn showing the streamed events + timeline. Useful for the "Non-functional requirements" section. +7. **`devoverlay-bus.png`** — the Bus tab with a recent message trace. + Useful when documenting the Bus contract. +8. **`menu-bar.png`** — close-up of the menu-bar icon in normal state + beside its red degraded variant. Pairs with the boot-health probe + diagram. + +## Diagrams (different concern) + +Architecture / lifecycle / install-order diagrams live as inline +**mermaid** blocks inside `README.md` and `ARCHITECTURE.md` — they render +natively on GitHub and stay in sync with the code without anybody +re-exporting an SVG. Don't replace them with rasterized exports. + +If a diagram outgrows mermaid (≳ 20 nodes or needs precise layout), the +right move is to author it in [Excalidraw](https://excalidraw.com), export +both the `.excalidraw` source and a `.svg` into this directory, and +reference the SVG from the README. The `.excalidraw` source is what keeps +the diagram editable. -- 2.54.0 From bc17700253011641f2c8c427cec2fb9f43ec5c88 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue, 12 May 2026 20:00:33 +0000 Subject: [PATCH 2/2] chore(deps): bump @react-three/fiber from 9.6.0 to 9.6.1 in /webview Bumps [@react-three/fiber](https://github.com/pmndrs/react-three-fiber) from 9.6.0 to 9.6.1. - [Release notes](https://github.com/pmndrs/react-three-fiber/releases) - [Commits](https://github.com/pmndrs/react-three-fiber/compare/v9.6.0...v9.6.1) --- updated-dependencies: - dependency-name: "@react-three/fiber" dependency-version: 9.6.1 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] --- webview/packages/hud/package.json | 2 +- webview/pnpm-lock.yaml | 16 ++++++++-------- 2 files changed, 9 insertions(+), 9 deletions(-) diff --git a/webview/packages/hud/package.json b/webview/packages/hud/package.json index 34a6ab7..5890987 100644 --- a/webview/packages/hud/package.json +++ b/webview/packages/hud/package.json @@ -13,7 +13,7 @@ "@jarvis/bus": "workspace:*", "react": "^19.2.0", "react-dom": "^19.2.0", - "@react-three/fiber": "^9.6.0", + "@react-three/fiber": "^9.6.1", "@react-three/drei": "^10.7.0", "three": "^0.184.0", "zustand": "^5.0.0" diff --git a/webview/pnpm-lock.yaml b/webview/pnpm-lock.yaml index 500fd2f..5e2eb9c 100644 --- a/webview/pnpm-lock.yaml +++ b/webview/pnpm-lock.yaml @@ -34,10 +34,10 @@ importers: version: link:../bus '@react-three/drei': specifier: ^10.7.0 - version: 10.7.7(@react-three/fiber@9.6.0(@types/react@19.2.14)(react-dom@19.2.5(react@19.2.5))(react@19.2.5)(three@0.184.0))(@types/react@19.2.14)(@types/three@0.184.0)(react-dom@19.2.5(react@19.2.5))(react@19.2.5)(three@0.184.0) + version: 10.7.7(@react-three/fiber@9.6.1(@types/react@19.2.14)(react-dom@19.2.5(react@19.2.5))(react@19.2.5)(three@0.184.0))(@types/react@19.2.14)(@types/three@0.184.0)(react-dom@19.2.5(react@19.2.5))(react@19.2.5)(three@0.184.0) '@react-three/fiber': - specifier: ^9.6.0 - version: 9.6.0(@types/react@19.2.14)(react-dom@19.2.5(react@19.2.5))(react@19.2.5)(three@0.184.0) + specifier: ^9.6.1 + version: 9.6.1(@types/react@19.2.14)(react-dom@19.2.5(react@19.2.5))(react@19.2.5)(three@0.184.0) react: specifier: ^19.2.0 version: 19.2.5 @@ -399,8 +399,8 @@ packages: react-dom: optional: true - '@react-three/fiber@9.6.0': - resolution: {integrity: sha512-90abYK2q5/qDM+GACs9zRvc5KhEEpEWqWlHSd64zTPNxg+9wCJvTfyD9x2so7hlQhjRYO1Fa6flR3BC/kpTFkA==} + '@react-three/fiber@9.6.1': + resolution: {integrity: sha512-zF0rsKcVYpcJwbFEnv2HkHX9cvOEgsfQo/X8lwmR2dn13S4qEQJXir9fxf5js2LQFoXqxOY7MDkOkYx2uZ4gSg==} peerDependencies: expo: '>=43.0' expo-asset: '>=8.4' @@ -2015,12 +2015,12 @@ snapshots: '@oxc-project/types@0.127.0': {} - '@react-three/drei@10.7.7(@react-three/fiber@9.6.0(@types/react@19.2.14)(react-dom@19.2.5(react@19.2.5))(react@19.2.5)(three@0.184.0))(@types/react@19.2.14)(@types/three@0.184.0)(react-dom@19.2.5(react@19.2.5))(react@19.2.5)(three@0.184.0)': + '@react-three/drei@10.7.7(@react-three/fiber@9.6.1(@types/react@19.2.14)(react-dom@19.2.5(react@19.2.5))(react@19.2.5)(three@0.184.0))(@types/react@19.2.14)(@types/three@0.184.0)(react-dom@19.2.5(react@19.2.5))(react@19.2.5)(three@0.184.0)': dependencies: '@babel/runtime': 7.29.2 '@mediapipe/tasks-vision': 0.10.17 '@monogrid/gainmap-js': 3.4.0(three@0.184.0) - '@react-three/fiber': 9.6.0(@types/react@19.2.14)(react-dom@19.2.5(react@19.2.5))(react@19.2.5)(three@0.184.0) + '@react-three/fiber': 9.6.1(@types/react@19.2.14)(react-dom@19.2.5(react@19.2.5))(react@19.2.5)(three@0.184.0) '@use-gesture/react': 10.3.1(react@19.2.5) camera-controls: 3.1.2(three@0.184.0) cross-env: 7.0.3 @@ -2048,7 +2048,7 @@ snapshots: - '@types/three' - immer - '@react-three/fiber@9.6.0(@types/react@19.2.14)(react-dom@19.2.5(react@19.2.5))(react@19.2.5)(three@0.184.0)': + '@react-three/fiber@9.6.1(@types/react@19.2.14)(react-dom@19.2.5(react@19.2.5))(react@19.2.5)(three@0.184.0)': dependencies: '@babel/runtime': 7.29.2 '@types/webxr': 0.5.24 -- 2.54.0