feat: always-on wake word with sherpa-onnx keyword spotting (v2.2.0)

- Catalog: 3.3 MB zipformer keyword-spotting model (kws-en).
- shared/keywords: BPE encoding of wake phrases (SentencePiece table for common words,
  greedy longest-match fallback over the model vocabulary) and keywords file builder.
- Worker: KeywordSpotter stream fed with 16-bit PCM, detections pushed as unsolicited
  messages; engine derives the keywords file, maps sensitivity to threshold/score,
  forwards detections to the renderer and re-arms after a worker restart.
- Renderer: WakeListener keeps one microphone stream, batches 256 ms frames to the
  spotter and captures the command on the same stream after detection (pre-roll, VAD),
  then resumes spotting; the wake word also interrupts speech. Settings: wake mode
  (off / always-on / transcript filter), keyword, sensitivity, status and one-click
  model download; HUD caption shows the active keyword.
- Validated: detection in the worker (fork) and through the real Electron IPC path on
  Kokoro audio, no false positive on an English recording; 23 unit tests.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Wn5VX9HNbJ7N54hR24u9Y
This commit is contained in:
Claude committed 2026-09-03 18:21:35 +00:00
1 parent 6d2f8f4ec9
commit 0d284fea9e
20 files changed
+760 -34

No files matched your search

+20 -3
View File
@@ -1,7 +1,7 @@
/** Local voice engine contract (sherpa-onnx in a utility process). Shared by main and renderer. */
export type VoiceModelKind = 'stt' | 'tts';
export type VoiceEngineKind = 'whisper' | 'sense-voice' | 'nemo-transducer' | 'kokoro' | 'piper';
export type VoiceModelKind = 'stt' | 'tts' | 'kws';
export type VoiceEngineKind = 'whisper' | 'sense-voice' | 'nemo-transducer' | 'kokoro' | 'piper' | 'kws-transducer';
export interface VoiceSpeaker {
id: number;
@@ -69,6 +69,19 @@ export interface SynthesizeResult {
audioSec: number;
}
export interface KwsStartRequest {
modelId: string;
/** Wake phrases in plain text (e.g. "jarvis", "hey jarvis"). */
keywords: string[];
/** 1 (strict) .. 5 (eager) */
sensitivity: number;
}
export interface KwsDetection {
keyword: string;
at: number;
}
export interface VoiceEngineStatus {
available: boolean;
error?: string;
@@ -86,5 +99,9 @@ export const VOICE_IPC = {
modelsProgress: 'voice:models:progress',
transcribe: 'voice:transcribe',
synthesize: 'voice:synthesize',
unload: 'voice:unload'
unload: 'voice:unload',
kwsStart: 'voice:kws:start',
kwsStop: 'voice:kws:stop',
kwsAudio: 'voice:kws:audio',
kwsDetected: 'voice:kws:detected'
} as const;