KofTwentyTwo/Jarvis
2
0 Fork 0
Code Issues 111 Pull Requests 16 Actions Packages Projects Releases 4 Wiki Activity

feat(10-01): wire four self-knowledge MCP tools into live runtime

Browse Source 
App-side adapters bridge JarvisMCP's self-knowledge dispatcher protocols
to concrete runtime sources, mirroring the InProcessMemoryAdapters.swift
Track D-2 pattern (commit 707c45b). JarvisMCP stays unaware of
Voice/AVFoundation/CoreAudio.

- App/MCP/InProcessSelfStateAdapters.swift (NEW):
  - CoreAudioIntrospection.snapshotDevices() — full
    AudioObjectGetPropertyData enumeration, one entry per
    (device, direction) pair with name/uid/sampleRate/channels and
    isDefault/isActive flags resolved from
    kAudioHardwarePropertyDefaultInput/OutputDevice.
  - CoreAudioIntrospection.defaultDeviceUID(isInput:) — used by
    AudioGraphRouteAdapter to fill the route's input/output UID + name.
  - CoreAudioDeviceListAdapter — wraps snapshotDevices() (SELF-01).
  - AudioGraphRouteAdapter (actor) — hops into AudioGraphOwner for the
    probed sample rate + channels, combines with the CoreAudio default
    device UIDs for the active route (SELF-02).
  - SelfStateAdapter — closure-injected probes for provider, voice loop
    state, TTS tier, STT backend, wake-word mute. Uses captured launch
    Date for app uptime (NOT ProcessInfo.systemUptime which is host
    uptime per D-11). buildMode reads #if DEBUG. (SELF-03).
  - AVCaptureDeviceListAdapter — DiscoverySession enumeration; metadata-
    only, does NOT trigger camera TCC prompt (SELF-04).

- packages/Voice/Sources/Voice/AudioGraph/AudioGraph.swift: store the
  probedFormat on AudioGraph so AudioGraphOwner can expose it. Additive
  public let; no behavior change.

- packages/Voice/Sources/Voice/AudioGraph/AudioGraphOwner.swift:
  - new public ActiveRouteSnapshot Sendable struct (sampleRate, channels)
  - new public actor accessor activeRouteSnapshot() returns the probed
    input format. Hardware-identity (UID/name) intentionally lives in
    the App-side adapter so Voice doesn't take on CoreAudio-introspection
    concerns.

- App/AppDelegate.swift:
  - launchInstant: Date captured at AppDelegate construction time for
    SELF-03 app-uptime reporting (D-11).
  - selfKnowledgeInstallTask + installSelfKnowledgeTools() registers the
    four tools into the existing inProcessToolRegistry built by
    installMemory. Runs after voiceInstallTask so audioGraphOwner is
    available; degrades gracefully when audioGraphOwner is nil
    (registers the three tools that don't depend on it). All four tools
    declare requiresConfirmation:false (D-10 / AP-12); no caching
    (D-09 / AP-11). Provider identity reads from configStore.perTurn();
    voice loop state hops VoiceController.state; wake-word mute reads
    UserDefaults `features.voice.wakeWordMuted` (mirrors MuteWakeWord's
    persistence key).

Verification:
- swift test --package-path packages/MCP — all suites GREEN incl.
  registersFourSelfKnowledgeTools.
- swift test --package-path packages/Voice — 26 tests in 7 suites GREEN
  (AudioGraph probedFormat additive change, no regressions).
- bash scripts/check-app-builds.sh — PASS.
- All 18 boundary gates exit 0 (incl. check-applescript-confirmation,
  check-no-leftover-stubs, check-install-order, check-no-null-voice-
  adapters, check-no-evaluate-javascript).
This commit is contained in:
james.maes committed 2026-05-06 15:50:30 -05:00
1 parent b9ac76d07e
commit 74b9ac2ff5
5 files changed
+553
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

App/AppDelegate.swift
+146
View File
@@ -215,6 +215,17 @@ final class AppDelegate: NSObject, NSApplicationDelegate {
/// cancelled prematurely.
var voiceInstallTask: Task<Void, Never>?
/// Plan 10-01: registers the four self-knowledge MCP tools
/// (`list_audio_devices`, `get_active_audio_route`, `get_self_state`,
/// `list_camera_devices`) into the existing
/// `inProcessToolRegistry`. Spawned AFTER voiceInstallTask so the live
/// `audioGraphOwner` reference is available for `AudioGraphRouteAdapter`.
var selfKnowledgeInstallTask: Task<Void, Never>?
/// Plan 10-01 / D-11: captured at app start so `get_self_state` can
/// report APP uptime (NOT host uptime via `ProcessInfo.systemUptime`).
let launchInstant: Date = Date()
// MARK: - Memory subsystem (Plan 07-01..07-02 wired in 07-06)
/// MemoryStore opened against ~/Library/Application Support/Jarvis/jarvis.db.
@@ -648,6 +659,18 @@ final class AppDelegate: NSObject, NSApplicationDelegate {
await self?.agentInstallTask?.value
await self?.installVoice()
}
// 15. Plan 10-01: register the four self-knowledge MCP tools
// (list_audio_devices, get_active_audio_route, get_self_state,
// list_camera_devices) AFTER voice so the live audioGraphOwner
// reference is available for AudioGraphRouteAdapter. The
// CoreAudio + AVCaptureDevice queries are read-only metadata
// only — no TCC prompt. D-10: every tool registers with
// requiresConfirmation:false explicitly.
selfKnowledgeInstallTask = Task { @MainActor [weak self] in
await self?.voiceInstallTask?.value
await self?.installSelfKnowledgeTools()
}
}
/// CR-02 (REVIEW 05): on-disk replay log path. Lives next to
@@ -1442,6 +1465,129 @@ final class AppDelegate: NSObject, NSApplicationDelegate {
)
}
// MARK: - Self-knowledge install (Plan 10-01)
/// Plan 10-01 / SELF-01..04. Registers the four read-only self-knowledge
/// MCP tools into the in-process registry built by installMemory.
///
/// Runs AFTER installVoice so the live `audioGraphOwner` reference is
/// available for `AudioGraphRouteAdapter`. If `inProcessToolRegistry`
/// or `audioGraphOwner` are nil (memory short-circuited or voice
/// failed to construct the graph), this method degrades gracefully —
/// the tools that don't depend on the missing reference still
/// register, the rest are skipped with a single warning.
///
/// D-10: every tool registers with `requiresConfirmation = false`
/// explicitly (read-only, no side effects). The flag is on the tool
/// type itself; no defaulting at the registry boundary.
/// D-09: dispatchers do NOT cache — every call queries fresh state.
@MainActor
func installSelfKnowledgeTools() async {
guard let registry = self.inProcessToolRegistry else {
systemLogger?.warning(
"installSelfKnowledgeTools: inProcessToolRegistry nil — memory short-circuited; self-knowledge tools dormant"
)
return
}
// SELF-01 + SELF-04: pure CoreAudio / AVCaptureDevice queries —
// no live-actor refs needed.
await registry.register(
ListAudioDevicesTool(dispatcher: CoreAudioDeviceListAdapter())
)
await registry.register(
ListCameraDevicesTool(dispatcher: AVCaptureDeviceListAdapter())
)
// SELF-02: requires the live AudioGraphOwner. If voice didn't
// start (no mic, no models, etc.), skip this one so the tool
// doesn't appear in the registry returning a perpetually-nil
// route — better to omit than to silently lie.
if let owner = self.audioGraphOwner {
await registry.register(
GetActiveAudioRouteTool(
dispatcher: AudioGraphRouteAdapter(owner: owner)
)
)
} else {
systemLogger?.warning(
"installSelfKnowledgeTools: audioGraphOwner nil — get_active_audio_route not registered"
)
}
// SELF-03: closure-injected probes hop into the live actors.
// ProviderIdentity reads from the configStore's per-turn snapshot
// (the orchestrator's ground truth for which provider it'll route
// to next). Voice loop state hops into VoiceController. TTS tier
// and STT backend come from the per-turn config. Wake-word mute
// is persisted in UserDefaults by `MuteWakeWord`.
let configStore = self.configStore
let voiceController = self.voiceController
let launchInstant = self.launchInstant
let providerIdentity: @Sendable () async -> SelfStateAdapter.ProviderIdentity = {
guard let configStore else {
return (provider: "anthropic", model: "claude-opus-4-7")
}
let snap = await configStore.perTurn()
let providerName: String
let modelName: String
switch snap.provider {
case .anthropic:
providerName = "anthropic"
modelName = "claude-opus-4-7" // CLAUDE.md / RESEARCH-DELTAS D1
case .ollama:
providerName = "ollama"
modelName = "qwen2.5-coder:32b" // CLAUDE.md known-good local baseline
}
return (provider: providerName, model: modelName)
}
let voiceState: @Sendable () async -> String? = {
guard let voiceController else { return nil }
let s = await voiceController.state
switch s {
case .idle: return "idle"
case .listening(let source):
switch source {
case .wakeWord: return "listening:wakeWord"
case .ptt: return "listening:ptt"
}
case .thinking: return "thinking"
case .speaking: return "speaking"
case .reconfiguring(let reason): return "reconfiguring:\(reason)"
}
}
let ttsTier: @Sendable () async -> String = {
guard let configStore else { return "tier1" }
let snap = await configStore.perTurn()
return snap.tts.tier
}
let sttBackend: @Sendable () async -> String = {
guard let configStore else { return "speechAnalyzer" }
let snap = await configStore.perTurn()
return snap.stt.whisperKitFallback ? "whisperKit" : "speechAnalyzer"
}
let wakeWordMuted: @Sendable () async -> Bool = {
UserDefaults.standard.bool(forKey: "features.voice.wakeWordMuted")
}
let selfState = SelfStateAdapter(
bundle: .main,
launchInstant: launchInstant,
providerIdentity: providerIdentity,
voiceState: voiceState,
ttsTier: ttsTier,
sttBackend: sttBackend,
wakeWordMuted: wakeWordMuted
)
await registry.register(GetSelfStateTool(dispatcher: selfState))
let names = await registry.registered().map(\.name).sorted()
systemLogger?.info(
"installSelfKnowledgeTools: registered self-knowledge tools — registry now contains \(names)"
)
}
// (StopReason → TurnTerminator mapping moved to
// `BusForwarder.terminator(for:)` in AgentCore. The App-side translation
// from the protocol-agnostic `BusForwarder.Terminator` to the wire
App/MCP/InProcessSelfStateAdapters.swift
+361
View File
@@ -0,0 +1,361 @@
// InProcessSelfStateAdapters.swift
//
// Plan 10-01 / Track D-2 pattern. Bridges JarvisMCP's self-knowledge tool
// dispatcher protocols (`AudioDeviceListing`, `ActiveAudioRouteDispatching`,
// `SelfStateDispatching`, `CameraDeviceListing`) to concrete runtime
// sources. These adapters live in App/ rather than packages/MCP/ because
// they cross the module boundary — JarvisMCP intentionally does NOT
// import Voice/AVFoundation/CoreAudio (Track D-2 design: protocol seam
// keeps MCP unaware of subsystem internals; mirrors the
// `InProcessMemoryAdapters.swift` shape from commit 707c45b).
//
// All four adapters are read-only. None require user confirmation
// (D-10). None cache (D-09 / AP-11) — every call queries fresh state.
import Foundation
import AVFoundation
import AudioToolbox
import CoreAudio
import JarvisMCP
import Voice
// MARK: - CoreAudio device enumeration (SELF-01)
/// Pure synchronous CoreAudio enumeration. ~80 LOC of
/// `AudioObjectGetPropertyData` boilerplate per RESEARCH §Example 1
/// (template: argmax-oss-swift WhisperKit AudioProcessor.swift:818-879).
enum CoreAudioIntrospection {
enum CoreAudioError: Error {
case size(OSStatus)
case fetch(OSStatus)
}
/// Returns one entry per (device, direction) pair. A duplex device
/// emits TWO entries — one with `isInput=true`, one with `isInput=false`.
/// Devices with no streams in a given direction are skipped for that
/// direction.
static func snapshotDevices() throws -> [AudioDeviceEntry] {
let allDeviceIDs = try fetchAllDeviceIDs()
let defaultInputID = (try? fetchDefaultDeviceID(isInput: true)) ?? AudioDeviceID(0)
let defaultOutputID = (try? fetchDefaultDeviceID(isInput: false)) ?? AudioDeviceID(0)
var entries: [AudioDeviceEntry] = []
entries.reserveCapacity(allDeviceIDs.count * 2)
for deviceID in allDeviceIDs {
// Resolve name + UID once per device.
let name = (try? fetchString(
deviceID: deviceID,
selector: kAudioObjectPropertyName,
scope: kAudioObjectPropertyScopeGlobal
)) ?? "<unknown>"
let uid = (try? fetchString(
deviceID: deviceID,
selector: kAudioDevicePropertyDeviceUID,
scope: kAudioObjectPropertyScopeGlobal
)) ?? "<no-uid>"
let sampleRate = (try? fetchSampleRate(deviceID: deviceID)) ?? 0
// Probe each direction independently — most devices are
// unidirectional, but Aggregate / VPIO devices can be both.
for isInput in [true, false] {
let scope = isInput ? kAudioDevicePropertyScopeInput : kAudioDevicePropertyScopeOutput
let channels = (try? fetchChannelCount(deviceID: deviceID, scope: scope)) ?? 0
if channels == 0 { continue } // direction not supported
let isDefault = isInput
? (deviceID == defaultInputID)
: (deviceID == defaultOutputID)
let isActive = sampleRate > 0 && channels > 0
entries.append(AudioDeviceEntry(
name: name,
uid: uid,
isDefault: isDefault,
isActive: isActive,
isInput: isInput,
sampleRate: sampleRate,
channels: channels
))
}
}
return entries
}
/// Returns the UID of the system default input or output device, or
/// nil if the query fails (rare — usually means no audio hardware).
static func defaultDeviceUID(isInput: Bool) -> (uid: String, name: String)? {
guard let id = try? fetchDefaultDeviceID(isInput: isInput),
id != AudioDeviceID(0) else { return nil }
let uid = try? fetchString(
deviceID: id,
selector: kAudioDevicePropertyDeviceUID,
scope: kAudioObjectPropertyScopeGlobal
)
let name = try? fetchString(
deviceID: id,
selector: kAudioObjectPropertyName,
scope: kAudioObjectPropertyScopeGlobal
)
guard let uid else { return nil }
return (uid: uid, name: name ?? "<unknown>")
}
// MARK: - Private CoreAudio helpers
private static func fetchAllDeviceIDs() throws -> [AudioDeviceID] {
var address = AudioObjectPropertyAddress(
mSelector: kAudioHardwarePropertyDevices,
mScope: kAudioObjectPropertyScopeGlobal,
mElement: kAudioObjectPropertyElementMain
)
var size: UInt32 = 0
var status = AudioObjectGetPropertyDataSize(
AudioObjectID(kAudioObjectSystemObject), &address, 0, nil, &size
)
guard status == noErr else { throw CoreAudioError.size(status) }
let count = Int(size) / MemoryLayout<AudioDeviceID>.size
var ids = [AudioDeviceID](repeating: 0, count: count)
status = AudioObjectGetPropertyData(
AudioObjectID(kAudioObjectSystemObject), &address, 0, nil, &size, &ids
)
guard status == noErr else { throw CoreAudioError.fetch(status) }
return ids
}
private static func fetchDefaultDeviceID(isInput: Bool) throws -> AudioDeviceID {
var address = AudioObjectPropertyAddress(
mSelector: isInput
? kAudioHardwarePropertyDefaultInputDevice
: kAudioHardwarePropertyDefaultOutputDevice,
mScope: kAudioObjectPropertyScopeGlobal,
mElement: kAudioObjectPropertyElementMain
)
var deviceID: AudioDeviceID = 0
var size = UInt32(MemoryLayout<AudioDeviceID>.size)
let status = AudioObjectGetPropertyData(
AudioObjectID(kAudioObjectSystemObject), &address, 0, nil, &size, &deviceID
)
guard status == noErr else { throw CoreAudioError.fetch(status) }
return deviceID
}
private static func fetchString(
deviceID: AudioDeviceID,
selector: AudioObjectPropertySelector,
scope: AudioObjectPropertyScope
) throws -> String {
var address = AudioObjectPropertyAddress(
mSelector: selector,
mScope: scope,
mElement: kAudioObjectPropertyElementMain
)
var size = UInt32(MemoryLayout<CFString?>.size)
var cfString: CFString?
let status = withUnsafeMutablePointer(to: &cfString) { ptr in
AudioObjectGetPropertyData(deviceID, &address, 0, nil, &size, ptr)
}
guard status == noErr, let s = cfString else {
throw CoreAudioError.fetch(status)
}
return s as String
}
private static func fetchSampleRate(deviceID: AudioDeviceID) throws -> Double {
var address = AudioObjectPropertyAddress(
mSelector: kAudioDevicePropertyNominalSampleRate,
mScope: kAudioObjectPropertyScopeGlobal,
mElement: kAudioObjectPropertyElementMain
)
var rate: Double = 0
var size = UInt32(MemoryLayout<Double>.size)
let status = AudioObjectGetPropertyData(deviceID, &address, 0, nil, &size, &rate)
guard status == noErr else { throw CoreAudioError.fetch(status) }
return rate
}
private static func fetchChannelCount(
deviceID: AudioDeviceID,
scope: AudioObjectPropertyScope
) throws -> Int {
var address = AudioObjectPropertyAddress(
mSelector: kAudioDevicePropertyStreamConfiguration,
mScope: scope,
mElement: kAudioObjectPropertyElementMain
)
var size: UInt32 = 0
var status = AudioObjectGetPropertyDataSize(deviceID, &address, 0, nil, &size)
guard status == noErr else { throw CoreAudioError.size(status) }
guard size > 0 else { return 0 }
let bufferList = UnsafeMutablePointer<AudioBufferList>.allocate(capacity: 1)
// Allocate enough room for the variable-length AudioBufferList tail.
let raw = UnsafeMutableRawPointer.allocate(byteCount: Int(size), alignment: 8)
defer {
raw.deallocate()
bufferList.deallocate()
}
let typedRaw = raw.bindMemory(to: AudioBufferList.self, capacity: Int(size))
status = AudioObjectGetPropertyData(deviceID, &address, 0, nil, &size, typedRaw)
guard status == noErr else { throw CoreAudioError.fetch(status) }
let abl = UnsafeMutableAudioBufferListPointer(typedRaw)
var channels = 0
for buffer in abl {
channels += Int(buffer.mNumberChannels)
}
return channels
}
}
// MARK: - SELF-01 adapter
/// Bridges `ListAudioDevicesTool` to `CoreAudioIntrospection.snapshotDevices()`.
/// Stateless. No caching (D-09).
public struct CoreAudioDeviceListAdapter: AudioDeviceListing {
public init() {}
public func listAudioDevices() async throws -> [AudioDeviceEntry] {
try CoreAudioIntrospection.snapshotDevices()
}
}
// MARK: - SELF-02 adapter
/// Bridges `GetActiveAudioRouteTool` to `AudioGraphOwner.activeRouteSnapshot()`.
/// Hops into the actor on each call. Resolves the input/output device
/// UID + name via CoreAudio (the AudioGraph itself doesn't expose these;
/// `AudioGraphOwner.activeRouteSnapshot()` returns only sampleRate +
/// channels from the probed format — see Plan 10-01 design notes).
public actor AudioGraphRouteAdapter: ActiveAudioRouteDispatching {
private let owner: AudioGraphOwner
public init(owner: AudioGraphOwner) {
self.owner = owner
}
public func getActiveAudioRoute() async throws -> ActiveAudioRoute? {
guard let variant = await owner.currentVariant else { return nil }
guard let snap = await owner.activeRouteSnapshot() else { return nil }
let input = CoreAudioIntrospection.defaultDeviceUID(isInput: true)
let output = CoreAudioIntrospection.defaultDeviceUID(isInput: false)
let variantStr: String = {
switch variant {
case .aecOn: return "aecOn"
case .aecOff: return "aecOff"
}
}()
return ActiveAudioRoute(
inputDeviceUID: input?.uid,
inputDeviceName: input?.name,
outputDeviceUID: output?.uid,
outputDeviceName: output?.name,
sampleRate: snap.sampleRate,
channels: snap.channels,
variant: variantStr
)
}
}
// MARK: - SELF-03 adapter
/// Bridges `GetSelfStateTool` to `Bundle.main.infoDictionary` +
/// `ProcessInfo` + closure-injected probes for actor-isolated state
/// (provider, voice loop state, TTS tier, STT backend, wake-word mute).
///
/// Closures keep the adapter free of `Voice` / `Config` actor imports —
/// AppDelegate constructs the closures with live references at install
/// time. This avoids deadlock if the adapter is invoked from the agent
/// loop while one of those actors is mid-transition: the hops await
/// naturally rather than block.
public struct SelfStateAdapter: SelfStateDispatching {
public typealias ProviderIdentity = (provider: String, model: String)
private let bundle: Bundle
private let launchInstant: Date
private let providerIdentity: @Sendable () async -> ProviderIdentity
private let voiceState: @Sendable () async -> String?
private let ttsTier: @Sendable () async -> String
private let sttBackend: @Sendable () async -> String
private let wakeWordMuted: @Sendable () async -> Bool
public init(
bundle: Bundle = .main,
launchInstant: Date,
providerIdentity: @escaping @Sendable () async -> ProviderIdentity,
voiceState: @escaping @Sendable () async -> String?,
ttsTier: @escaping @Sendable () async -> String,
sttBackend: @escaping @Sendable () async -> String,
wakeWordMuted: @escaping @Sendable () async -> Bool
) {
self.bundle = bundle
self.launchInstant = launchInstant
self.providerIdentity = providerIdentity
self.voiceState = voiceState
self.ttsTier = ttsTier
self.sttBackend = sttBackend
self.wakeWordMuted = wakeWordMuted
}
public func getSelfState() async -> SelfState {
let info = bundle.infoDictionary ?? [:]
let appName = (info["CFBundleName"] as? String) ?? "Jarvis"
let appVersion = (info["CFBundleShortVersionString"] as? String) ?? "0.0.0"
let buildMode: String = {
#if DEBUG
return "debug"
#else
return "release"
#endif
}()
let pid = Int(ProcessInfo.processInfo.processIdentifier)
let uptime = max(0, Int(Date().timeIntervalSince(launchInstant)))
let provider = await providerIdentity()
return SelfState(
appName: appName,
appVersion: appVersion,
buildMode: buildMode,
pid: pid,
uptimeSeconds: uptime,
llmProvider: provider.provider,
llmModel: provider.model,
ttsTier: await ttsTier(),
sttBackend: await sttBackend(),
wakeWordMuted: await wakeWordMuted(),
voiceLoopState: await voiceState() ?? "idle"
)
}
}
// MARK: - SELF-04 adapter
/// Bridges `ListCameraDevicesTool` to `AVCaptureDevice.DiscoverySession`.
/// Read-only; does NOT trigger camera TCC prompt (RESEARCH §Example 2).
public struct AVCaptureDeviceListAdapter: CameraDeviceListing {
public init() {}
public func listCameraDevices() async throws -> [CameraDeviceEntry] {
let session = AVCaptureDevice.DiscoverySession(
deviceTypes: [.builtInWideAngleCamera, .external, .continuityCamera],
mediaType: .video,
position: .unspecified
)
return session.devices.map { dev in
CameraDeviceEntry(
localizedName: dev.localizedName,
uniqueID: dev.uniqueID,
isConnected: dev.isConnected,
position: Self.positionString(dev.position)
)
}
}
private static func positionString(_ p: AVCaptureDevice.Position) -> String {
switch p {
case .front: return "front"
case .back: return "back"
case .unspecified: return "external"
@unknown default: return "unknown"
}
}
}
Jarvis.xcodeproj/project.pbxproj
+4
View File
@@ -93,6 +93,7 @@
E218ADC35078C6F26F0D0041 /* HUDBanner.swift in Sources */ = {isa = PBXBuildFile; fileRef = 096AD64E7DB48B0F1EE47F89 /* HUDBanner.swift */; };
E39D7F3B185FA8D84C33B421 /* Config in Frameworks */ = {isa = PBXBuildFile; productRef = 1058DB02D52D27FA07A79025 /* Config */; };
E49FEEFE453A055F65DA31FF /* WizardStageHotkeyView.swift in Sources */ = {isa = PBXBuildFile; fileRef = 74FC22D7D34AB5E6061959D7 /* WizardStageHotkeyView.swift */; };
E7F6C8E910BA8B5F7629DEF7 /* InProcessSelfStateAdapters.swift in Sources */ = {isa = PBXBuildFile; fileRef = 906C8E54650D79A94FB3C669 /* InProcessSelfStateAdapters.swift */; };
E8CD13B014BB64CAB1ACA632 /* AppDelegateEntitlementTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = A231DEC7B592B6D527B0283C /* AppDelegateEntitlementTests.swift */; };
EB03A8906003E55271CAF133 /* JarvisVision in Frameworks */ = {isa = PBXBuildFile; productRef = C1168750998C14962FE14DEC /* JarvisVision */; };
EB07CD704A3568F27DD4D901 /* Shell in Frameworks */ = {isa = PBXBuildFile; productRef = 6FAD561608B0D1C402015E0D /* Shell */; };
@@ -194,6 +195,7 @@
8C27488AAB75A245B580A95B /* WizardTCCStageTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = WizardTCCStageTests.swift; sourceTree = "<group>"; };
8C8267C7BE6D4B16FEE6004E /* webview */ = {isa = PBXFileReference; lastKnownFileType = folder; name = webview; path = App/Resources/webview; sourceTree = SOURCE_ROOT; };
8FB47EBED48DF63272C6D77F /* AppDelegateFrameAttachAdapters.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = AppDelegateFrameAttachAdapters.swift; sourceTree = "<group>"; };
906C8E54650D79A94FB3C669 /* InProcessSelfStateAdapters.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = InProcessSelfStateAdapters.swift; sourceTree = "<group>"; };
909BFE54FAE9CA6B878D5625 /* AppDelegateWiringTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = AppDelegateWiringTests.swift; sourceTree = "<group>"; };
90D238B5CF4BD07EB47F84F0 /* JarvisApp.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = JarvisApp.swift; sourceTree = "<group>"; };
960E9FC636D6FF3DD68E5FE9 /* Replay */ = {isa = PBXFileReference; lastKnownFileType = folder; name = Replay; path = packages/Replay; sourceTree = SOURCE_ROOT; };
@@ -557,6 +559,7 @@
children = (
C4723FBC0F20AD2EFF146A24 /* AppDelegateContextBuilderAdapter.swift */,
252A86D40325F2186A86A505 /* InProcessMemoryAdapters.swift */,
906C8E54650D79A94FB3C669 /* InProcessSelfStateAdapters.swift */,
CBF3EF4CDCC653EF50D99AF7 /* MCPRuntimeWiring.swift */,
0358E0CAC1377752398FA3C4 /* ReplayingToolResultObserver.swift */,
);
@@ -1322,6 +1325,7 @@
5230995216EF1EE333B51529 /* HudStateCoordinator.swift in Sources */,
896984D88C412595C0F6F97E /* HudStateIntent.swift in Sources */,
98165CEB562FE32CD16BAA1D /* InProcessMemoryAdapters.swift in Sources */,
E7F6C8E910BA8B5F7629DEF7 /* InProcessSelfStateAdapters.swift in Sources */,
526BDF4D6BC5714809B2E937 /* JarvisApp.swift in Sources */,
BEFC509502C7940A21001F9F /* JarvisHUDPanel.swift in Sources */,
CD2C5779101FE0384611ECCC /* MCPBusGatewayAdapter.swift in Sources */,
packages/Voice/Sources/Voice/AudioGraph/AudioGraph.swift
+8
View File
@@ -127,6 +127,13 @@ public final class AudioGraph: Sendable {
/// `ringBuffer` for back-compat.
private let primarySubscription: BufferBroadcaster.Subscription
/// Probed input format — what AVAudioEngine reported BEFORE the
/// 16 kHz mono target. Plan 10-01 / SELF-02: exposed via
/// `AudioGraphOwner.activeRouteSnapshot()` so `get_active_audio_route`
/// can answer "what mic is the graph bound to?" with the actual
/// hardware-side sample rate, not the downstream contract value.
public let probedFormat: AVAudioFormat
// MARK: - Private AVFoundation objects
// `nonisolated(unsafe)` because they are created once in `init` and then
@@ -241,6 +248,7 @@ public final class AudioGraph: Sendable {
self.broadcaster = broadcaster
self.primarySubscription = primarySub
self.ringBuffer = ring
self.probedFormat = probedFormat
self.variant = aec
? .aecOn(targetFormat)
: .aecOff(targetFormat)
packages/Voice/Sources/Voice/AudioGraph/AudioGraphOwner.swift
+34
View File
@@ -3,6 +3,21 @@ import OSLog
private let logger = Logger(subsystem: "com.koftwentytwo.jarvis", category: "AudioGraphOwner")
/// Plan 10-01 / SELF-02. Minimal `Sendable` snapshot of the active
/// AudioGraph's probed input format. Returned by
/// `AudioGraphOwner.activeRouteSnapshot()`. Hardware-identity (UID/name)
/// is not part of this struct — the App-side `AudioGraphRouteAdapter`
/// resolves those via CoreAudio at snapshot time.
public struct ActiveRouteSnapshot: Sendable, Equatable {
public let sampleRate: Double
public let channels: Int
public init(sampleRate: Double, channels: Int) {
self.sampleRate = sampleRate
self.channels = channels
}
}
/// Actor that owns the live `AudioGraph` and manages its full lifecycle.
///
/// Responsibilities:
@@ -57,6 +72,25 @@ public actor AudioGraphOwner {
return graph.broadcaster.subscribe(capacityFrames: capacityFrames)
}
/// Plan 10-01 / SELF-02 additive accessor.
///
/// Returns the probed input-side sample rate + channel count the
/// AudioGraph reported at build time (matches the
/// `AudioGraph: probed format sampleRate=X channels=Y` log line).
/// Returns `nil` if the graph is not currently open.
///
/// Hardware-identity (input/output device UID + name) is intentionally
/// NOT included here — the App-side `AudioGraphRouteAdapter` queries
/// CoreAudio for those at snapshot time so Voice doesn't need to take
/// on the device-introspection concern. Read-only; no side effects.
public func activeRouteSnapshot() -> ActiveRouteSnapshot? {
guard let graph else { return nil }
return ActiveRouteSnapshot(
sampleRate: graph.probedFormat.sampleRate,
channels: Int(graph.probedFormat.channelCount)
)
}
// MARK: - Init
/// Creates the owner with stream continuations and an optional graph builder.