mirror of
https://github.com/R0m1k3/EveFlow.git
synced 2026-10-11 17:29:03 +02:00
Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
af8bdc5e9d | ||
|
|
4f6074183f | ||
|
|
0d284fea9e | ||
|
|
6d2f8f4ec9 | ||
|
|
5f0fa7a1e2 | ||
|
|
b1b8d0d5d4 | ||
|
|
eefc36aca8 | ||
|
|
af1e3d8aae | ||
|
|
2471add00b |
No files matched your search
@@ -43,7 +43,7 @@ jobs:
|
|||||||
id: ver
|
id: ver
|
||||||
shell: bash
|
shell: bash
|
||||||
run: |
|
run: |
|
||||||
version=$(node -p "require('./package.json').version")
|
version=$(node -p "const p=require('./package.json'); p.releaseVersion || p.version")
|
||||||
echo "version=$version" >> "$GITHUB_OUTPUT"
|
echo "version=$version" >> "$GITHUB_OUTPUT"
|
||||||
# Fourth component (hotfix number, e.g. 2.0.0.1) becomes the Windows file build number.
|
# Fourth component (hotfix number, e.g. 2.0.0.1) becomes the Windows file build number.
|
||||||
echo "build=$(echo "$version" | cut -d. -f4)" >> "$GITHUB_OUTPUT"
|
echo "build=$(echo "$version" | cut -d. -f4)" >> "$GITHUB_OUTPUT"
|
||||||
@@ -54,7 +54,7 @@ jobs:
|
|||||||
fi
|
fi
|
||||||
- run: npm run build
|
- run: npm run build
|
||||||
- name: Package Windows EXE
|
- name: Package Windows EXE
|
||||||
run: npx electron-builder --win --publish never
|
run: npm run dist -- --win --publish never
|
||||||
env:
|
env:
|
||||||
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||||
BUILD_NUMBER: ${{ steps.ver.outputs.build }}
|
BUILD_NUMBER: ${{ steps.ver.outputs.build }}
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
# EveFlow 2 — Interface vocale JARVIS pour Hermes Agent
|
# EveFlow 2 — Interface vocale JARVIS pour Hermes Agent
|
||||||
|
|
||||||
[](https://github.com/R0m1k3/EveFlow/actions)
|
[](https://github.com/R0m1k3/EveFlow/actions)
|
||||||
[](https://github.com/R0m1k3/EveFlow/releases)
|
[](https://github.com/R0m1k3/EveFlow/releases)
|
||||||
[](LICENSE)
|
[](LICENSE)
|
||||||
|
|
||||||
**EveFlow** est un compagnon de bureau Windows qui transforme [Hermes Agent](https://hermes-agent.nousresearch.com/) en assistant vocal à la JARVIS : un noyau holographique réactif au son, une conversation en streaming, les outils, sous-agents, approbations, crons, skills et sessions d'Hermes pilotés depuis un seul HUD.
|
**EveFlow** est un compagnon de bureau Windows qui transforme [Hermes Agent](https://hermes-agent.nousresearch.com/) en assistant vocal à la JARVIS : un noyau holographique réactif au son, une conversation en streaming, les outils, sous-agents, approbations, crons, skills et sessions d'Hermes pilotés depuis un seul HUD.
|
||||||
@@ -22,8 +22,13 @@ La version 2 est une réécriture complète : plus de robot 3D, un pipeline voca
|
|||||||
* **Capture micro** via AudioWorklet à 16 kHz, sans monitoring du micro dans les haut-parleurs, avec annulation d'écho et réduction de bruit.
|
* **Capture micro** via AudioWorklet à 16 kHz, sans monitoring du micro dans les haut-parleurs, avec annulation d'écho et réduction de bruit.
|
||||||
* **Détection d'activité vocale** (seuil adaptatif, sensibilité et silence de fin réglables) : l'enregistrement s'arrête tout seul quand vous avez fini de parler.
|
* **Détection d'activité vocale** (seuil adaptatif, sensibilité et silence de fin réglables) : l'enregistrement s'arrête tout seul quand vous avez fini de parler.
|
||||||
* **Mains libres** : le micro se réactive après chaque réponse.
|
* **Mains libres** : le micro se réactive après chaque réponse.
|
||||||
* **STT** : n'importe quelle API `/v1/audio/transcriptions` compatible OpenAI (Qwen3-ASR, Whisper, Speaches, faster-whisper-server, LocalAI, OpenAI). Repli sur la reconnaissance Chromium.
|
* **Modèles intégrés, hors ligne** (sherpa-onnx dans un processus séparé) : reconnaissance Whisper (base, small, large-v3 turbo) ou SenseVoice, synthèse Kokoro v1.0 (voix française Siwis et voix anglaises) ou Piper (Siwis, Tom, UPMC). Les modèles se téléchargent depuis **Paramètres → Modèles locaux** et tournent sur le processeur.
|
||||||
* **TTS** : API `/v1/audio/speech` compatible OpenAI (Kokoro, Piper, OpenAI…), voix système Windows ou Google Translate. Lecture phrase par phrase pendant le streaming, préchargement du segment suivant, coupure instantanée.
|
* **Fin de phrase neuronale** : en écoute permanente, Silero VAD (0,6 Mo, sherpa-onnx) décide du début et de la fin de la commande à la place du seuil d'énergie ; moins de faux départs sur le bruit, coupure plus nette. Repli automatique sur le VAD énergétique si le modèle n'est pas installé.
|
||||||
|
* **Vision d'écran** : « Jarvis, regarde mon écran » (ou le bouton de la barre de commande) joint une capture de l'écran principal à la question envoyée à Hermes.
|
||||||
|
* **Actions locales instantanées** : « verrouille la session », « monte le son », « coupe le son », « piste suivante », « ouvre Spotify », « ouvre github.com »… exécutées sur le PC sans passer par Hermes, résultat lu à voix haute. Liste blanche d'actions dans le processus principal, désactivable dans les paramètres.
|
||||||
|
* **Écoute permanente** : un détecteur de mot-clé de 3 Mo (sherpa-onnx, keyword spotting) tourne en continu sur le micro, quasi gratuit en CPU. « Jarvis » (ou n'importe quel mot-clé) ouvre l'écoute, « Jarvis, allume… » envoie directement la commande, et le mot coupe la voix en cours. Alternative : filtre du mot après transcription en mains libres.
|
||||||
|
* **STT externe** : n'importe quelle API `/v1/audio/transcriptions` compatible OpenAI (Qwen3-ASR, Whisper, Speaches, faster-whisper-server, LocalAI, OpenAI). Repli sur la reconnaissance Chromium.
|
||||||
|
* **TTS externe** : API `/v1/audio/speech` compatible OpenAI, voix système Windows ou Google Translate. Lecture phrase par phrase pendant le streaming, préchargement du segment suivant, coupure instantanée.
|
||||||
* Raccourcis globaux : `Ctrl+Shift+Espace` (micro), `Ctrl+Shift+J` (afficher/masquer), `Ctrl+Shift+Échap` (couper la voix).
|
* Raccourcis globaux : `Ctrl+Shift+Espace` (micro), `Ctrl+Shift+J` (afficher/masquer), `Ctrl+Shift+Échap` (couper la voix).
|
||||||
|
|
||||||
### Hermes, toute la puissance
|
### Hermes, toute la puissance
|
||||||
@@ -49,10 +54,12 @@ La version 2 est une réécriture complète : plus de robot 3D, un pipeline voca
|
|||||||
* Electron 44 (sandbox, contextIsolation, `webSecurity` actif, proxy HTTP en streaming dans le main process)
|
* Electron 44 (sandbox, contextIsolation, `webSecurity` actif, proxy HTTP en streaming dans le main process)
|
||||||
* React 19 + Vite 8 + TypeScript 5.9 + Zustand
|
* React 19 + Vite 8 + TypeScript 5.9 + Zustand
|
||||||
* Canvas 2D, Web Audio (AudioWorklet, AnalyserNode)
|
* Canvas 2D, Web Audio (AudioWorklet, AnalyserNode)
|
||||||
|
* sherpa-onnx (ONNX Runtime) pour la voix locale, exécuté dans un `utilityProcess` Electron
|
||||||
* Vitest pour les tests unitaires (SSE, VAD, WAV, normalisation d'événements Hermes)
|
* Vitest pour les tests unitaires (SSE, VAD, WAV, normalisation d'événements Hermes)
|
||||||
|
|
||||||
```
|
```
|
||||||
electron/ processus principal (fenêtre, tray, raccourcis, IPC, webhook, proxy HTTP)
|
electron/ processus principal (fenêtre, tray, raccourcis, IPC, webhook, proxy HTTP)
|
||||||
|
electron/voice catalogue de modèles, téléchargement, worker sherpa-onnx (STT/TTS)
|
||||||
shared/ contrat IPC + normalisation des pushs webhook (main + renderer)
|
shared/ contrat IPC + normalisation des pushs webhook (main + renderer)
|
||||||
src/lib transport, SSE, persistance, utilitaires texte
|
src/lib transport, SSE, persistance, utilitaires texte
|
||||||
src/services hermes/ (client, événements, outils locaux) voice/ (capture, VAD, STT, TTS)
|
src/services hermes/ (client, événements, outils locaux) voice/ (capture, VAD, STT, TTS)
|
||||||
@@ -96,13 +103,13 @@ npm run typecheck # renderer + main process
|
|||||||
|
|
||||||
Sans Electron, `npm run dev` puis `http://127.0.0.1:5173/` (ou `?mode=compact`) permet de travailler l'interface dans un navigateur ; les appels réseau passent alors directement par `fetch` (CORS requis côté serveur).
|
Sans Electron, `npm run dev` puis `http://127.0.0.1:5173/` (ou `?mode=compact`) permet de travailler l'interface dans un navigateur ; les appels réseau passent alors directement par `fetch` (CORS requis côté serveur).
|
||||||
|
|
||||||
## Packaging Windows
|
## Packaging Windows et versions
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
npm run dist
|
npm run dist
|
||||||
```
|
```
|
||||||
|
|
||||||
L'installateur NSIS est produit dans `out/`. Le workflow GitHub Actions construit l'exécutable sur chaque tag `v*`.
|
L'installateur NSIS est produit dans `out/`. Le numéro de version publié est `releaseVersion` dans `package.json` (quatre chiffres, ex. `2.0.0.1` : le quatrième chiffre sert aux petits correctifs). `version` reste la base à trois chiffres exigée par electron-builder. À chaque fusion sur master, le workflow GitHub Actions crée le tag et la release Windows si ce numéro n'a pas encore été publié.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,72 @@
|
|||||||
|
# Feuille de route : vers un vrai JARVIS
|
||||||
|
|
||||||
|
État au 3 septembre 2026, après la revue complète du code (trois audits : processus principal, services/état, interface) et le test de bout en bout de l'application Electron sous Xvfb avec les vrais modèles.
|
||||||
|
|
||||||
|
## Ce qui existe déjà
|
||||||
|
|
||||||
|
| Capacité | État | Notes |
|
||||||
|
|---|---|---|
|
||||||
|
| HUD arc-reactor réactif au son | Fait | Canvas 2D optimisé (pas d'ombres, couleurs en cache, 30 fps en veille, arrêt fenêtre masquée) |
|
||||||
|
| Reconnaissance vocale locale | Fait | Whisper base/small/turbo via sherpa-onnx dans un processus utilitaire |
|
||||||
|
| Synthèse vocale locale | Fait | Kokoro v1.0 (voix française Siwis) et Piper fr |
|
||||||
|
| Mot d'activation permanent | Fait (2.2.0) | Keyword spotting sherpa-onnx en continu ; mot-clé libre encodé en BPE ; validé sur audio réel (détection, zéro faux positif sur le test anglais) |
|
||||||
|
| Mot d'activation après transcription | Fait | Filtre « Jarvis … » en mains libres, tolérant aux erreurs de transcription |
|
||||||
|
| Détection de fin de phrase | Fait (2.3.0) | Silero VAD neuronal dans le worker (segment renvoyé au renderer), repli sur le VAD énergétique si le modèle manque |
|
||||||
|
| Vision d'écran | Fait (2.3.0) | Capture `desktopCapturer` jointe à la requête Hermes (bouton, ou « regarde mon écran ») |
|
||||||
|
| Actions système locales | Fait (2.3.0) | Verrouillage, volume et touches média, ouvrir une application ou une URL, presse-papiers, recherche de fichiers ; intentions courtes exécutées sans passer par Hermes |
|
||||||
|
| Hermes : runs, sessions, chat completions | Fait | Transport choisi selon `/v1/capabilities` |
|
||||||
|
| Approbations, steer, stop | Fait | Modales, injection de consigne en cours de run |
|
||||||
|
| Crons, skills, toolsets, sessions | Fait | Panneau Hermes Ops |
|
||||||
|
| Webhook local (Telegram, crons) | Fait | Loopback par défaut, secret pour le LAN |
|
||||||
|
| Télémétrie réelle | Fait | CPU, RAM, fréquence, FPS, uptime |
|
||||||
|
|
||||||
|
## Ce que les meilleurs projets « Jarvis » de 2026 font en plus
|
||||||
|
|
||||||
|
Sources : [jarvis-desktop-ai](https://github.com/ccarloshenri/jarvis-desktop-ai), [JarvisAi](https://github.com/PanPenek/JarvisAi), [bertrandmbanwi/Jarvis](https://github.com/bertrandmbanwi/Jarvis), [InterGenJLU/jarvis](https://github.com/InterGenJLU/jarvis), [livekit-wakeword](https://livekit.com/blog/livekit-wakeword), [sherpa-onnx keyword spotting](https://k2-fsa.github.io/sherpa/onnx/kws/index.html), [Hermes Agent features](https://hermes-agent.nousresearch.com/docs/user-guide/features/overview).
|
||||||
|
|
||||||
|
1. **Mot d'activation permanent, quasi gratuit en CPU.** Les projets de référence utilisent openWakeWord (« hey jarvis ») ou un modèle de keyword spotting qui écoute en continu, au lieu de transcrire chaque phrase. sherpa-onnx fournit un modèle KWS anglais de 3,3 Mo qui accepte n'importe quel mot-clé sans réentraînement ; « JARVIS » s'encode `▁JA R VI S` avec son modèle BPE (vérifié). Livré en 2.2.0 (voir l'étape 1 ci-dessous).
|
||||||
|
2. **VAD neuronal (Silero) au lieu du seuil d'énergie.** Fin de phrase plus nette (environ 500 ms gagnés) et beaucoup moins de faux départs sur le bruit ambiant. Silero est déjà livré dans sherpa-onnx (`silero_vad.onnx`, 0,6 Mo). Livré en 2.3.0.
|
||||||
|
3. **Latence perçue sous la seconde.** Les références visent 1 s entre la fin de parole et le premier mot prononcé : STT rapide, premier token en streaming, TTS phrase par phrase (déjà en place), et un modèle Hermes rapide pour la conversation courante.
|
||||||
|
4. **Vision d'écran.** Capture d'écran à la demande (« Jarvis, qu'est-ce que je regarde ? ») envoyée à Hermes comme image, ou lecture d'une fenêtre. Hermes accepte déjà les images inline. Livré en 2.3.0 (transport chat completions pour les images).
|
||||||
|
5. **Actions système locales.** Ouvrir une application, régler le volume, verrouiller la session, chercher un fichier. Livré en 2.3.0 sous forme d'intentions courtes exécutées localement avant Hermes ; reste à exposer les mêmes actions à Hermes via un serveur MCP local.
|
||||||
|
6. **Proactivité.** Notifications parlées à l'arrivée d'un cron, rappel, événement webhook, avec un résumé plutôt que la lecture intégrale ; c'est en partie fait via le webhook, à enrichir avec des règles (heures calmes, priorité).
|
||||||
|
7. **Mémoire et personnalisation.** Hermes gère la mémoire longue durée (`X-Hermes-Session-Key`) ; côté EveFlow, un profil (nom, préférences de voix, style de réponse) déjà transmis dans les instructions.
|
||||||
|
|
||||||
|
## Plan proposé
|
||||||
|
|
||||||
|
### Étape 1 : écoute permanente — livrée en 2.2.0
|
||||||
|
- Le renderer garde un seul flux micro (AudioWorklet 16 kHz) et envoie des blocs de 256 ms au processus principal, qui alimente le `KeywordSpotter` sherpa-onnx dans le worker.
|
||||||
|
- Mots-clés encodés en BPE (table SentencePiece pour les mots courants, repli glouton sur le vocabulaire du modèle), sensibilité réglable (seuil 0,45 → 0,12).
|
||||||
|
- À la détection : chime, capture de la commande sur le même flux (VAD énergétique, pré-roll 400 ms), transcription locale ou API, envoi à Hermes ; retour automatique à l'écoute.
|
||||||
|
- Fin de phrase Silero livrée en 2.3.0 : le renderer envoie des trames de 128 ms au worker pendant la commande, le worker renvoie le segment WAV complet ; VAD énergétique en repli.
|
||||||
|
|
||||||
|
### Étape 2 : vision et actions locales — livrée en 2.3.0
|
||||||
|
- Capture d'écran (`desktopCapturer`, JPEG 1600 px) jointe à la requête Hermes : bouton dans la barre de commande, ou phrase « regarde mon écran… » à l'oral comme à l'écrit.
|
||||||
|
- Actions locales (processus principal, liste blanche) : verrouiller, volume/mute/lecture/piste, ouvrir une application connue ou une URL http(s), presse-papiers, recherche de fichiers dans Documents/Bureau/Téléchargements/Images.
|
||||||
|
- Routeur d'intentions FR/EN (`src/services/localCommands.ts`) exécuté avant l'envoi à Hermes ; désactivable dans Paramètres → Micro. Les phrases composées (« ouvre X puis… ») partent à Hermes.
|
||||||
|
- Reste à faire : exposer ces actions à Hermes lui-même (serveur MCP local) pour qu'un run puisse les enchaîner.
|
||||||
|
|
||||||
|
### Étape 3 : conversation plus naturelle
|
||||||
|
- Barge-in réel : couper la voix dès que l'utilisateur parle (déjà préparé, à valider avec l'annulation d'écho Windows).
|
||||||
|
- Réponses courtes à l'oral, détails à l'écran : instruction Hermes dédiée déjà en place, à affiner avec des consignes de format (« deux phrases à l'oral, détails en Markdown »).
|
||||||
|
- Modèle rapide pour le bavardage, modèle puissant pour les missions (choix par transport ou par mot-clé).
|
||||||
|
|
||||||
|
### Étape 4 : présence
|
||||||
|
- Widget compact « glanceable » : dernière phrase, état, badge d'alertes.
|
||||||
|
- Heures calmes, priorité des notifications, résumé vocal des crons.
|
||||||
|
- Thèmes et voix par contexte (nuit, travail).
|
||||||
|
|
||||||
|
## Résultats du test de bout en bout (Linux, Xvfb, 4 cœurs lents)
|
||||||
|
|
||||||
|
| Étape | Résultat |
|
||||||
|
|---|---|
|
||||||
|
| Pont Electron, télémétrie, webhook | OK |
|
||||||
|
| Conversation via runs (streaming, outils, sous-agent) | OK |
|
||||||
|
| Kokoro (fr) → Whisper base, phrase courte de 2 s | synthèse 2,9 s, transcription 2,5 s, texte approximatif |
|
||||||
|
| Kokoro (fr) → Whisper base, phrase de 7 s | transcription correcte à un mot près |
|
||||||
|
| Piper (fr) → Whisper base | transcription exacte |
|
||||||
|
| Silero VAD sur phrase Kokoro de 2,3 s (2.3.0) | un seul segment, début et fin détectés, 6,7 s d'audio traités en 190 ms |
|
||||||
|
| Capture d'écran → Hermes (2.3.0) | JPEG de 107 ko reçu côté Hermes (mock chat completions) |
|
||||||
|
| Intention locale « coupe le son » (2.3.0) | traitée sans Hermes, résultat affiché dans le fil |
|
||||||
|
|
||||||
|
Sur un PC à 28 cœurs les temps sont nettement plus courts. Whisper small est maintenant recommandé pour le français.
|
||||||
+70
-14
@@ -22,12 +22,25 @@ const READABLE_EXT: Record<string, string> = {
|
|||||||
'.html': 'text/html'
|
'.html': 'text/html'
|
||||||
};
|
};
|
||||||
|
|
||||||
const FORBIDDEN_EXT = new Set([
|
/** Documents and media the assistant may open with the default application. */
|
||||||
'.exe', '.bat', '.cmd', '.ps1', '.vbs', '.msi', '.scr', '.pif', '.reg', '.sh',
|
const OPENABLE_EXT = new Set([
|
||||||
'.com', '.hta', '.vbe', '.wsf', '.lnk', '.jar', '.dll', '.js', '.jse'
|
...Object.keys(READABLE_EXT),
|
||||||
|
'.pdf', '.docx', '.doc', '.xlsx', '.xls', '.pptx', '.ppt', '.odt', '.ods', '.odp', '.rtf',
|
||||||
|
'.mp3', '.wav', '.ogg', '.flac', '.mp4', '.mkv', '.webm', '.zip', '.7z', '.py', '.ts', '.tsx', '.yaml', '.yml', '.toml', '.ini'
|
||||||
]);
|
]);
|
||||||
|
|
||||||
|
const FORBIDDEN_EXT = new Set([
|
||||||
|
'.exe', '.bat', '.cmd', '.ps1', '.vbs', '.msi', '.scr', '.pif', '.reg', '.sh', '.com', '.hta', '.vbe', '.wsf', '.lnk',
|
||||||
|
'.jar', '.dll', '.js', '.jse', '.pyw', '.msc', '.cpl', '.inf', '.url', '.scf', '.website', '.appref-ms', '.application',
|
||||||
|
'.gadget', '.psm1', '.psd1', '.ps1xml', '.wsh', '.wsc', '.sct', '.desktop', '.app', '.command', '.appimage', '.run'
|
||||||
|
]);
|
||||||
|
|
||||||
|
const WIN_RESERVED = /^(CON|PRN|AUX|NUL|COM[1-9]|LPT[1-9])(\..*)?$/i;
|
||||||
|
|
||||||
|
let sharedDir: string | null = null;
|
||||||
|
|
||||||
export function getSharedDirectory(): string {
|
export function getSharedDirectory(): string {
|
||||||
|
if (sharedDir && fs.existsSync(sharedDir)) return sharedDir;
|
||||||
const candidates = [
|
const candidates = [
|
||||||
path.join(os.homedir(), 'Documents', 'EveFlow_Shared'),
|
path.join(os.homedir(), 'Documents', 'EveFlow_Shared'),
|
||||||
path.join(app.getPath('userData'), 'shared')
|
path.join(app.getPath('userData'), 'shared')
|
||||||
@@ -35,32 +48,60 @@ export function getSharedDirectory(): string {
|
|||||||
for (const dir of candidates) {
|
for (const dir of candidates) {
|
||||||
try {
|
try {
|
||||||
fs.mkdirSync(dir, { recursive: true });
|
fs.mkdirSync(dir, { recursive: true });
|
||||||
|
sharedDir = dir;
|
||||||
return dir;
|
return dir;
|
||||||
} catch {
|
} catch {
|
||||||
/* try next candidate */
|
/* try next candidate */
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
return app.getPath('temp');
|
sharedDir = app.getPath('temp');
|
||||||
|
return sharedDir;
|
||||||
}
|
}
|
||||||
|
|
||||||
function normalizeSafe(filePath: string): string {
|
function normalizeSafe(filePath: string): string {
|
||||||
if (!filePath || typeof filePath !== 'string') throw new Error('Chemin invalide');
|
if (!filePath || typeof filePath !== 'string' || filePath.length > 4096) throw new Error('Chemin invalide');
|
||||||
const normalized = path.normalize(filePath);
|
const normalized = path.normalize(filePath);
|
||||||
if (normalized.split(/[\\/]/).includes('..')) throw new Error('Traversee de repertoire refusee');
|
if (normalized.split(/[\\/]/).includes('..')) throw new Error('Traversee de repertoire refusee');
|
||||||
return normalized;
|
return normalized;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
function allowedRoots(): string[] {
|
||||||
|
return [
|
||||||
|
getSharedDirectory(),
|
||||||
|
path.join(os.homedir(), 'Documents'),
|
||||||
|
path.join(os.homedir(), 'Pictures'),
|
||||||
|
path.join(os.homedir(), 'Downloads'),
|
||||||
|
path.join(os.homedir(), 'Desktop'),
|
||||||
|
app.getPath('temp')
|
||||||
|
].map((r) => path.resolve(r));
|
||||||
|
}
|
||||||
|
|
||||||
|
function isInside(root: string, target: string): boolean {
|
||||||
|
const rel = path.relative(root, target);
|
||||||
|
return rel === '' || (!rel.startsWith('..') && !path.isAbsolute(rel));
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Resolves symlinks and checks the real path against the allowed roots (prefix tricks like "bob-backup" are rejected). */
|
||||||
function isInsideAllowedRoots(target: string): boolean {
|
function isInsideAllowedRoots(target: string): boolean {
|
||||||
const roots = [getSharedDirectory(), os.homedir(), app.getPath('temp')];
|
let resolved: string;
|
||||||
const resolved = path.resolve(target);
|
try {
|
||||||
return roots.some((root) => resolved.startsWith(path.resolve(root)));
|
resolved = fs.realpathSync.native(target);
|
||||||
|
} catch {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
return allowedRoots().some((root) => isInside(root, resolved));
|
||||||
}
|
}
|
||||||
|
|
||||||
function stripFileScheme(value: string): string {
|
function stripFileScheme(value: string): string {
|
||||||
let out = value.replace(/^file:\/\//, '');
|
if (!/^file:\/\//i.test(value)) return value;
|
||||||
|
let out = value.replace(/^file:\/\//i, '');
|
||||||
// file:///C:/x -> /C:/x on Windows-style paths
|
// file:///C:/x -> /C:/x on Windows-style paths
|
||||||
if (/^\/[a-zA-Z]:/.test(out)) out = out.slice(1);
|
if (/^\/[a-zA-Z]:/.test(out)) out = out.slice(1);
|
||||||
return decodeURIComponent(out);
|
try {
|
||||||
|
return decodeURIComponent(out);
|
||||||
|
} catch {
|
||||||
|
return out;
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
export function registerFilesIpc(): void {
|
export function registerFilesIpc(): void {
|
||||||
@@ -78,8 +119,16 @@ export function registerFilesIpc(): void {
|
|||||||
|
|
||||||
ipcMain.handle(IPC.writeSharedFile, async (_e, filename: string, content: string, isBase64 = false) => {
|
ipcMain.handle(IPC.writeSharedFile, async (_e, filename: string, content: string, isBase64 = false) => {
|
||||||
if (!filename || typeof filename !== 'string') throw new Error('Nom de fichier invalide');
|
if (!filename || typeof filename !== 'string') throw new Error('Nom de fichier invalide');
|
||||||
|
if (typeof content !== 'string' || content.length > 50 * 1024 * 1024) throw new Error('Contenu invalide');
|
||||||
const safeName = path.basename(filename);
|
const safeName = path.basename(filename);
|
||||||
if (safeName !== filename || safeName.startsWith('.') || /[<>:"|?*]/.test(safeName)) {
|
if (
|
||||||
|
safeName !== filename ||
|
||||||
|
safeName.startsWith('.') ||
|
||||||
|
/[<>:"|?*\u0000-\u001f]/.test(safeName) ||
|
||||||
|
WIN_RESERVED.test(safeName) ||
|
||||||
|
/[. ]$/.test(safeName) ||
|
||||||
|
safeName.length > 200
|
||||||
|
) {
|
||||||
throw new Error('Nom de fichier non securise');
|
throw new Error('Nom de fichier non securise');
|
||||||
}
|
}
|
||||||
const ext = path.extname(safeName).toLowerCase();
|
const ext = path.extname(safeName).toLowerCase();
|
||||||
@@ -97,10 +146,17 @@ export function registerFilesIpc(): void {
|
|||||||
|
|
||||||
ipcMain.handle(IPC.openPath, async (_e, filePath: string) => {
|
ipcMain.handle(IPC.openPath, async (_e, filePath: string) => {
|
||||||
const target = normalizeSafe(stripFileScheme(filePath));
|
const target = normalizeSafe(stripFileScheme(filePath));
|
||||||
const ext = path.extname(target).toLowerCase();
|
|
||||||
if (FORBIDDEN_EXT.has(ext)) throw new Error(`Ouverture refusee pour ${ext}`);
|
|
||||||
if (!fs.existsSync(target)) throw new Error('Fichier introuvable');
|
if (!fs.existsSync(target)) throw new Error('Fichier introuvable');
|
||||||
const result = await shell.openPath(target);
|
const real = await fs.promises.realpath(target);
|
||||||
|
const stat = await fs.promises.stat(real);
|
||||||
|
const ext = path.extname(real).toLowerCase();
|
||||||
|
if (!stat.isDirectory() && (FORBIDDEN_EXT.has(ext) || !OPENABLE_EXT.has(ext))) throw new Error(`Ouverture refusee pour ${ext || 'ce fichier'}`);
|
||||||
|
// Outside the user's document folders we only reveal the file, never execute its handler.
|
||||||
|
if (!isInsideAllowedRoots(real)) {
|
||||||
|
shell.showItemInFolder(real);
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
const result = await shell.openPath(real);
|
||||||
if (result) throw new Error(result);
|
if (result) throw new Error(result);
|
||||||
return true;
|
return true;
|
||||||
});
|
});
|
||||||
|
|||||||
+11
-2
@@ -105,6 +105,8 @@ async function startStream(sender: WebContents, id: string, req: HttpProxyReques
|
|||||||
headers: headersToObject(response.headers)
|
headers: headersToObject(response.headers)
|
||||||
};
|
};
|
||||||
|
|
||||||
|
const onDestroyed = () => controller.abort();
|
||||||
|
sender.once('destroyed', onDestroyed);
|
||||||
const pump = async () => {
|
const pump = async () => {
|
||||||
try {
|
try {
|
||||||
if (!response.body) {
|
if (!response.body) {
|
||||||
@@ -133,7 +135,10 @@ async function startStream(sender: WebContents, id: string, req: HttpProxyReques
|
|||||||
if (e.name === 'AbortError') send({ id, type: 'end' });
|
if (e.name === 'AbortError') send({ id, type: 'end' });
|
||||||
else send({ id, type: 'error', message: e.message || String(err) });
|
else send({ id, type: 'error', message: e.message || String(err) });
|
||||||
} finally {
|
} finally {
|
||||||
activeStreams.delete(id);
|
// Releases the socket when we bailed out early (idle timeout, renderer gone).
|
||||||
|
controller.abort();
|
||||||
|
sender.removeListener('destroyed', onDestroyed);
|
||||||
|
if (activeStreams.get(id) === controller) activeStreams.delete(id);
|
||||||
}
|
}
|
||||||
};
|
};
|
||||||
void pump();
|
void pump();
|
||||||
@@ -147,7 +152,11 @@ export function abortAllStreams(): void {
|
|||||||
|
|
||||||
export function registerHttpIpc(): void {
|
export function registerHttpIpc(): void {
|
||||||
ipcMain.handle(IPC.httpFetch, (_e, req: HttpProxyRequest) => proxyFetch(req));
|
ipcMain.handle(IPC.httpFetch, (_e, req: HttpProxyRequest) => proxyFetch(req));
|
||||||
ipcMain.handle(IPC.httpStreamStart, (event, id: string, req: HttpProxyRequest) => startStream(event.sender, id, req));
|
ipcMain.handle(IPC.httpStreamStart, (event, id: unknown, req: HttpProxyRequest) => {
|
||||||
|
if (typeof id !== 'string' || !id || id.length > 64 || activeStreams.has(id)) throw new Error('Identifiant de flux invalide');
|
||||||
|
if (!req || typeof req.url !== 'string') throw new Error('Requête invalide');
|
||||||
|
return startStream(event.sender, id, req);
|
||||||
|
});
|
||||||
ipcMain.on(IPC.httpStreamAbort, (_e, id: string) => {
|
ipcMain.on(IPC.httpStreamAbort, (_e, id: string) => {
|
||||||
const controller = activeStreams.get(id);
|
const controller = activeStreams.get(id);
|
||||||
if (controller) {
|
if (controller) {
|
||||||
|
|||||||
+61
-12
@@ -2,10 +2,16 @@ import { app, ipcMain } from 'electron';
|
|||||||
import fs from 'node:fs';
|
import fs from 'node:fs';
|
||||||
import path from 'node:path';
|
import path from 'node:path';
|
||||||
import { IPC } from '../../shared/ipc';
|
import { IPC } from '../../shared/ipc';
|
||||||
|
import { log } from '../logger';
|
||||||
|
|
||||||
type StoreData = Record<string, unknown>;
|
type StoreData = Record<string, unknown>;
|
||||||
|
|
||||||
let cache: StoreData | null = null;
|
let cache: StoreData | null = null;
|
||||||
|
let dirty = false;
|
||||||
|
let timer: ReturnType<typeof setTimeout> | null = null;
|
||||||
|
let writing: Promise<void> = Promise.resolve();
|
||||||
|
|
||||||
|
const FORBIDDEN_KEYS = new Set(['__proto__', 'constructor', 'prototype']);
|
||||||
|
|
||||||
function storePath(): string {
|
function storePath(): string {
|
||||||
return path.join(app.getPath('userData'), 'userdata.json');
|
return path.join(app.getPath('userData'), 'userdata.json');
|
||||||
@@ -13,19 +19,55 @@ function storePath(): string {
|
|||||||
|
|
||||||
function load(): StoreData {
|
function load(): StoreData {
|
||||||
if (cache) return cache;
|
if (cache) return cache;
|
||||||
|
const target = storePath();
|
||||||
try {
|
try {
|
||||||
cache = JSON.parse(fs.readFileSync(storePath(), 'utf8')) as StoreData;
|
cache = JSON.parse(fs.readFileSync(target, 'utf8')) as StoreData;
|
||||||
} catch {
|
} catch (err) {
|
||||||
|
if ((err as NodeJS.ErrnoException).code !== 'ENOENT') {
|
||||||
|
// Keep a copy of an unreadable file instead of silently overwriting it.
|
||||||
|
try {
|
||||||
|
fs.copyFileSync(target, `${target}.corrupt-${Date.now()}`);
|
||||||
|
} catch {
|
||||||
|
/* ignore */
|
||||||
|
}
|
||||||
|
log('WARN', 'store', `userdata.json unreadable, starting fresh: ${(err as Error).message}`);
|
||||||
|
}
|
||||||
cache = {};
|
cache = {};
|
||||||
}
|
}
|
||||||
return cache;
|
return cache;
|
||||||
}
|
}
|
||||||
|
|
||||||
function persist(): void {
|
/** Debounced, asynchronous, atomic write with retries (Windows AV scanners lock files briefly). */
|
||||||
const target = storePath();
|
function schedulePersist(): void {
|
||||||
const tmp = target + '.tmp';
|
dirty = true;
|
||||||
fs.writeFileSync(tmp, JSON.stringify(cache ?? {}, null, 2), 'utf8');
|
if (!timer) timer = setTimeout(() => void flushStore(), 250);
|
||||||
fs.renameSync(tmp, target);
|
}
|
||||||
|
|
||||||
|
export function flushStore(): Promise<void> {
|
||||||
|
if (timer) {
|
||||||
|
clearTimeout(timer);
|
||||||
|
timer = null;
|
||||||
|
}
|
||||||
|
if (!dirty) return writing;
|
||||||
|
dirty = false;
|
||||||
|
const text = JSON.stringify(cache ?? {}, null, 2);
|
||||||
|
writing = writing
|
||||||
|
.then(async () => {
|
||||||
|
const target = storePath();
|
||||||
|
const tmp = `${target}.tmp`;
|
||||||
|
await fs.promises.writeFile(tmp, text, 'utf8');
|
||||||
|
for (let attempt = 0; ; attempt++) {
|
||||||
|
try {
|
||||||
|
await fs.promises.rename(tmp, target);
|
||||||
|
return;
|
||||||
|
} catch (err) {
|
||||||
|
if (attempt >= 5) throw err;
|
||||||
|
await new Promise((r) => setTimeout(r, 100 * (attempt + 1)));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
})
|
||||||
|
.catch((err) => log('ERROR', 'store', `cannot persist userdata.json: ${(err as Error).message}`));
|
||||||
|
return writing;
|
||||||
}
|
}
|
||||||
|
|
||||||
export function storeGet<T = unknown>(key: string): T | null {
|
export function storeGet<T = unknown>(key: string): T | null {
|
||||||
@@ -36,19 +78,26 @@ export function storeGet<T = unknown>(key: string): T | null {
|
|||||||
export function storeSet(key: string, value: unknown): void {
|
export function storeSet(key: string, value: unknown): void {
|
||||||
const data = load();
|
const data = load();
|
||||||
data[key] = value;
|
data[key] = value;
|
||||||
persist();
|
schedulePersist();
|
||||||
|
}
|
||||||
|
|
||||||
|
function validKey(key: unknown): key is string {
|
||||||
|
return typeof key === 'string' && key.length > 0 && key.length <= 128 && !FORBIDDEN_KEYS.has(key);
|
||||||
}
|
}
|
||||||
|
|
||||||
export function registerStoreIpc(): void {
|
export function registerStoreIpc(): void {
|
||||||
ipcMain.handle(IPC.storeGet, (_e, key: string) => storeGet(key));
|
ipcMain.handle(IPC.storeGet, (_e, key: unknown) => (validKey(key) ? storeGet(key) : null));
|
||||||
ipcMain.handle(IPC.storeSet, (_e, key: string, value: unknown) => {
|
ipcMain.handle(IPC.storeSet, (_e, key: unknown, value: unknown) => {
|
||||||
|
if (!validKey(key)) throw new Error('Clé de stockage invalide');
|
||||||
|
if (typeof value === 'function' || typeof value === 'symbol') throw new Error('Valeur non sérialisable');
|
||||||
storeSet(key, value);
|
storeSet(key, value);
|
||||||
return true;
|
return true;
|
||||||
});
|
});
|
||||||
ipcMain.handle(IPC.storeDelete, (_e, key: string) => {
|
ipcMain.handle(IPC.storeDelete, (_e, key: unknown) => {
|
||||||
|
if (!validKey(key)) return false;
|
||||||
const data = load();
|
const data = load();
|
||||||
delete data[key];
|
delete data[key];
|
||||||
persist();
|
schedulePersist();
|
||||||
return true;
|
return true;
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
@@ -0,0 +1,184 @@
|
|||||||
|
/**
|
||||||
|
* Local "JARVIS" actions: screenshot for vision, and an allow-list of system actions
|
||||||
|
* (lock, open an application or URL, media keys, clipboard, file search).
|
||||||
|
*/
|
||||||
|
import { app, clipboard, desktopCapturer, ipcMain, screen, shell } from 'electron';
|
||||||
|
import { execFile } from 'node:child_process';
|
||||||
|
import fs from 'node:fs';
|
||||||
|
import os from 'node:os';
|
||||||
|
import path from 'node:path';
|
||||||
|
import { IPC } from '../../shared/ipc';
|
||||||
|
import type { SystemAction, SystemActionResult } from '../../shared/bridge';
|
||||||
|
import { log } from '../logger';
|
||||||
|
|
||||||
|
const MEDIA_KEYS: Record<string, number> = {
|
||||||
|
'volume-up': 175,
|
||||||
|
'volume-down': 174,
|
||||||
|
mute: 173,
|
||||||
|
'play-pause': 179,
|
||||||
|
next: 176,
|
||||||
|
previous: 177
|
||||||
|
};
|
||||||
|
|
||||||
|
/** Applications the assistant may launch by name (Windows aliases + common Linux/macOS names). */
|
||||||
|
const APP_ALIASES: Record<string, string[]> = {
|
||||||
|
'bloc-notes': ['notepad.exe', 'gedit', 'TextEdit'],
|
||||||
|
notepad: ['notepad.exe', 'gedit', 'TextEdit'],
|
||||||
|
calculatrice: ['calc.exe', 'gnome-calculator', 'Calculator'],
|
||||||
|
calc: ['calc.exe', 'gnome-calculator', 'Calculator'],
|
||||||
|
explorateur: ['explorer.exe', 'nautilus', 'Finder'],
|
||||||
|
explorer: ['explorer.exe', 'nautilus', 'Finder'],
|
||||||
|
terminal: ['wt.exe', 'cmd.exe', 'gnome-terminal', 'Terminal'],
|
||||||
|
cmd: ['cmd.exe'],
|
||||||
|
powershell: ['powershell.exe'],
|
||||||
|
paint: ['mspaint.exe'],
|
||||||
|
chrome: ['chrome', 'google-chrome', 'Google Chrome'],
|
||||||
|
edge: ['msedge', 'microsoft-edge', 'Microsoft Edge'],
|
||||||
|
firefox: ['firefox', 'Firefox'],
|
||||||
|
vscode: ['code', 'Visual Studio Code'],
|
||||||
|
code: ['code', 'Visual Studio Code'],
|
||||||
|
spotify: ['spotify', 'Spotify'],
|
||||||
|
discord: ['discord', 'Discord'],
|
||||||
|
steam: ['steam', 'Steam'],
|
||||||
|
word: ['winword', 'Microsoft Word'],
|
||||||
|
excel: ['excel', 'Microsoft Excel'],
|
||||||
|
outlook: ['outlook', 'Microsoft Outlook'],
|
||||||
|
teams: ['ms-teams', 'Microsoft Teams'],
|
||||||
|
'task manager': ['taskmgr.exe'],
|
||||||
|
'gestionnaire des tâches': ['taskmgr.exe'],
|
||||||
|
paramètres: ['ms-settings:'],
|
||||||
|
settings: ['ms-settings:']
|
||||||
|
};
|
||||||
|
|
||||||
|
function run(cmd: string, args: string[], timeoutMs = 8000): Promise<{ code: number; out: string }> {
|
||||||
|
return new Promise((resolve) => {
|
||||||
|
execFile(cmd, args, { timeout: timeoutMs, windowsHide: true }, (err, stdout, stderr) => {
|
||||||
|
resolve({ code: err ? 1 : 0, out: `${stdout}${stderr}`.trim() });
|
||||||
|
});
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
async function openApp(name: string): Promise<SystemActionResult> {
|
||||||
|
const key = name.trim().toLowerCase();
|
||||||
|
if (!key || key.length > 40) return { ok: false, message: 'Nom d’application invalide' };
|
||||||
|
const candidates = APP_ALIASES[key] ?? [key.replace(/[^a-z0-9 ._-]/gi, '')];
|
||||||
|
if (process.platform === 'win32') {
|
||||||
|
for (const candidate of candidates) {
|
||||||
|
if (candidate.endsWith(':')) {
|
||||||
|
await shell.openExternal(candidate);
|
||||||
|
return { ok: true, message: `${name} ouvert` };
|
||||||
|
}
|
||||||
|
// `start` resolves App Paths, PATH and Start Menu names.
|
||||||
|
const result = await run('cmd.exe', ['/c', 'start', '', candidate]);
|
||||||
|
if (result.code === 0) return { ok: true, message: `${name} lancé` };
|
||||||
|
}
|
||||||
|
return { ok: false, message: `Impossible de lancer ${name}` };
|
||||||
|
}
|
||||||
|
for (const candidate of candidates) {
|
||||||
|
const result = process.platform === 'darwin' ? await run('open', ['-a', candidate]) : await run('sh', ['-c', `command -v ${JSON.stringify(candidate)} >/dev/null && (nohup ${JSON.stringify(candidate)} >/dev/null 2>&1 &)`]);
|
||||||
|
if (result.code === 0) return { ok: true, message: `${name} lancé` };
|
||||||
|
}
|
||||||
|
return { ok: false, message: `Application introuvable : ${name}` };
|
||||||
|
}
|
||||||
|
|
||||||
|
async function lockSession(): Promise<SystemActionResult> {
|
||||||
|
if (process.platform === 'win32') {
|
||||||
|
const r = await run('rundll32.exe', ['user32.dll,LockWorkStation']);
|
||||||
|
return { ok: r.code === 0, message: r.code === 0 ? 'Session verrouillée' : r.out };
|
||||||
|
}
|
||||||
|
if (process.platform === 'darwin') {
|
||||||
|
const r = await run('osascript', ['-e', 'tell application "System Events" to keystroke "q" using {command down, control down}']);
|
||||||
|
return { ok: r.code === 0, message: r.out };
|
||||||
|
}
|
||||||
|
const r = await run('sh', ['-c', 'loginctl lock-session || xdg-screensaver lock || gnome-screensaver-command -l']);
|
||||||
|
return { ok: r.code === 0, message: r.code === 0 ? 'Session verrouillée' : r.out };
|
||||||
|
}
|
||||||
|
|
||||||
|
async function mediaKey(key: string): Promise<SystemActionResult> {
|
||||||
|
const code = MEDIA_KEYS[key];
|
||||||
|
if (!code) return { ok: false, message: 'Touche inconnue' };
|
||||||
|
if (process.platform === 'win32') {
|
||||||
|
const script = `$s=Add-Type -MemberDefinition '[DllImport("user32.dll")] public static extern void keybd_event(byte b,byte s,uint f,UIntPtr e);' -Name K -Namespace W -PassThru; $s::keybd_event(${code},0,0,[UIntPtr]::Zero); $s::keybd_event(${code},0,2,[UIntPtr]::Zero)`;
|
||||||
|
const r = await run('powershell.exe', ['-NoProfile', '-NonInteractive', '-Command', script]);
|
||||||
|
return { ok: r.code === 0, message: r.code === 0 ? key : r.out };
|
||||||
|
}
|
||||||
|
const xdo: Record<string, string> = { 'volume-up': 'XF86AudioRaiseVolume', 'volume-down': 'XF86AudioLowerVolume', mute: 'XF86AudioMute', 'play-pause': 'XF86AudioPlay', next: 'XF86AudioNext', previous: 'XF86AudioPrev' };
|
||||||
|
const r = await run('sh', ['-c', `xdotool key ${xdo[key]}`]);
|
||||||
|
return { ok: r.code === 0, message: r.code === 0 ? key : 'xdotool indisponible' };
|
||||||
|
}
|
||||||
|
|
||||||
|
async function findFiles(query: string): Promise<SystemActionResult> {
|
||||||
|
const needle = query.trim().toLowerCase();
|
||||||
|
if (needle.length < 2) return { ok: false, message: 'Recherche trop courte' };
|
||||||
|
const roots = ['Documents', 'Desktop', 'Downloads', 'Pictures'].map((d) => path.join(os.homedir(), d));
|
||||||
|
const hits: string[] = [];
|
||||||
|
const deadline = Date.now() + 4000;
|
||||||
|
const walk = async (dir: string, depth: number) => {
|
||||||
|
if (depth > 4 || hits.length >= 25 || Date.now() > deadline) return;
|
||||||
|
let entries: fs.Dirent[] = [];
|
||||||
|
try {
|
||||||
|
entries = await fs.promises.readdir(dir, { withFileTypes: true });
|
||||||
|
} catch {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
for (const entry of entries) {
|
||||||
|
if (entry.name.startsWith('.') || entry.name === 'node_modules') continue;
|
||||||
|
const full = path.join(dir, entry.name);
|
||||||
|
if (entry.name.toLowerCase().includes(needle)) hits.push(full);
|
||||||
|
if (entry.isDirectory()) await walk(full, depth + 1);
|
||||||
|
if (hits.length >= 25) return;
|
||||||
|
}
|
||||||
|
};
|
||||||
|
for (const root of roots) await walk(root, 0);
|
||||||
|
return { ok: true, message: `${hits.length} résultat(s)`, data: hits };
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function captureScreen(maxWidth = 1600): Promise<string> {
|
||||||
|
const display = screen.getPrimaryDisplay();
|
||||||
|
const scale = Math.min(1, maxWidth / display.size.width);
|
||||||
|
const sources = await desktopCapturer.getSources({
|
||||||
|
types: ['screen'],
|
||||||
|
thumbnailSize: { width: Math.round(display.size.width * scale), height: Math.round(display.size.height * scale) }
|
||||||
|
});
|
||||||
|
const primary = sources.find((s) => s.display_id === String(display.id)) ?? sources[0];
|
||||||
|
if (!primary) throw new Error('Aucun écran capturable');
|
||||||
|
return primary.thumbnail.toJPEG(82).toString('base64').replace(/^/, 'data:image/jpeg;base64,');
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function runSystemAction(action: SystemAction): Promise<SystemActionResult> {
|
||||||
|
switch (action.type) {
|
||||||
|
case 'lock':
|
||||||
|
return lockSession();
|
||||||
|
case 'open-app':
|
||||||
|
return openApp(String(action.name ?? ''));
|
||||||
|
case 'open-url': {
|
||||||
|
const url = String(action.url ?? '');
|
||||||
|
if (!/^https?:\/\//i.test(url) || url.length > 2048) return { ok: false, message: 'URL refusée' };
|
||||||
|
await shell.openExternal(url);
|
||||||
|
return { ok: true, message: 'Ouvert dans le navigateur' };
|
||||||
|
}
|
||||||
|
case 'media':
|
||||||
|
return mediaKey(String(action.key));
|
||||||
|
case 'clipboard-read':
|
||||||
|
return { ok: true, data: (await clipboard.readText()).slice(0, 20_000) };
|
||||||
|
case 'clipboard-write':
|
||||||
|
await clipboard.writeText(String(action.text ?? '').slice(0, 100_000));
|
||||||
|
return { ok: true, message: 'Copié dans le presse-papiers' };
|
||||||
|
case 'find-files':
|
||||||
|
return findFiles(String(action.query ?? ''));
|
||||||
|
default:
|
||||||
|
return { ok: false, message: 'Action inconnue' };
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
export function registerSystemIpc(): void {
|
||||||
|
ipcMain.handle(IPC.screenCapture, (_e, maxWidth?: number) => captureScreen(Number(maxWidth) || 1600));
|
||||||
|
ipcMain.handle(IPC.systemAction, async (_e, action: SystemAction) => {
|
||||||
|
if (!action || typeof action.type !== 'string') throw new Error('Action invalide');
|
||||||
|
log('INFO', 'system', `action ${action.type}`);
|
||||||
|
const result = await runSystemAction(action);
|
||||||
|
if (!result.ok) log('WARN', 'system', `action ${action.type} failed: ${result.message}`);
|
||||||
|
return result;
|
||||||
|
});
|
||||||
|
void app;
|
||||||
|
}
|
||||||
+32
-8
@@ -5,22 +5,46 @@ import type { LogEntry } from '../shared/ipc';
|
|||||||
|
|
||||||
const MAX_LOG_BYTES = 5 * 1024 * 1024;
|
const MAX_LOG_BYTES = 5 * 1024 * 1024;
|
||||||
|
|
||||||
|
let stream: fs.WriteStream | null = null;
|
||||||
|
let written = 0;
|
||||||
|
|
||||||
export function getLogPath(): string {
|
export function getLogPath(): string {
|
||||||
return path.join(app.getPath('userData'), 'eveflow.log');
|
return path.join(app.getPath('userData'), 'eveflow.log');
|
||||||
}
|
}
|
||||||
|
|
||||||
|
function output(): fs.WriteStream {
|
||||||
|
if (!stream) {
|
||||||
|
const target = getLogPath();
|
||||||
|
try {
|
||||||
|
written = fs.statSync(target).size;
|
||||||
|
} catch {
|
||||||
|
written = 0;
|
||||||
|
}
|
||||||
|
stream = fs.createWriteStream(target, { flags: 'a' });
|
||||||
|
stream.on('error', (err) => console.error('[logger] write failed:', err.message));
|
||||||
|
}
|
||||||
|
return stream;
|
||||||
|
}
|
||||||
|
|
||||||
|
function rotate(): void {
|
||||||
|
const target = getLogPath();
|
||||||
|
stream?.end();
|
||||||
|
stream = null;
|
||||||
|
try {
|
||||||
|
fs.renameSync(target, `${target}.old`);
|
||||||
|
} catch {
|
||||||
|
/* ignore */
|
||||||
|
}
|
||||||
|
written = 0;
|
||||||
|
}
|
||||||
|
|
||||||
export function writeLog(entry: LogEntry): void {
|
export function writeLog(entry: LogEntry): void {
|
||||||
try {
|
try {
|
||||||
const logPath = getLogPath();
|
|
||||||
const data = entry.data !== undefined ? ' | ' + safeJson(entry.data) : '';
|
const data = entry.data !== undefined ? ' | ' + safeJson(entry.data) : '';
|
||||||
const line = `[${entry.ts}] [${entry.level}] [${entry.tag}] ${entry.message}${data}\n`;
|
const line = `[${entry.ts}] [${entry.level}] [${entry.tag}] ${entry.message}${data}\n`;
|
||||||
try {
|
written += Buffer.byteLength(line);
|
||||||
const stat = fs.statSync(logPath);
|
if (written > MAX_LOG_BYTES) rotate();
|
||||||
if (stat.size > MAX_LOG_BYTES) fs.renameSync(logPath, logPath + '.old');
|
output().write(line);
|
||||||
} catch {
|
|
||||||
/* first start: file does not exist yet */
|
|
||||||
}
|
|
||||||
fs.appendFileSync(logPath, line, 'utf8');
|
|
||||||
} catch (err) {
|
} catch (err) {
|
||||||
console.error('[logger] cannot write log file:', (err as Error).message);
|
console.error('[logger] cannot write log file:', (err as Error).message);
|
||||||
}
|
}
|
||||||
|
|||||||
+30
-11
@@ -2,15 +2,20 @@ import { app, BrowserWindow, globalShortcut, ipcMain, Menu, nativeImage, session
|
|||||||
import path from 'node:path';
|
import path from 'node:path';
|
||||||
import { IPC, type AppInfo, type HotkeyEvent, type LogEntry, type WindowMode } from '../shared/ipc';
|
import { IPC, type AppInfo, type HotkeyEvent, type LogEntry, type WindowMode } from '../shared/ipc';
|
||||||
import { getLogPath, log, writeLog } from './logger';
|
import { getLogPath, log, writeLog } from './logger';
|
||||||
import { registerStoreIpc, storeGet } from './ipc/store';
|
import { flushStore, registerStoreIpc, storeGet } from './ipc/store';
|
||||||
import { abortAllStreams, registerHttpIpc } from './ipc/http';
|
import { abortAllStreams, registerHttpIpc } from './ipc/http';
|
||||||
import { registerTelemetryIpc } from './ipc/telemetry';
|
import { registerTelemetryIpc } from './ipc/telemetry';
|
||||||
import { getSharedDirectory, registerFilesIpc } from './ipc/files';
|
import { getSharedDirectory, registerFilesIpc } from './ipc/files';
|
||||||
import { createMainWindow, getMainWindow, getWindowMode, setWindowMode, toggleWindowVisibility } from './window';
|
import { createMainWindow, getMainWindow, getWindowMode, setWindowMode, toggleWindowVisibility, windowEvents } from './window';
|
||||||
import { getWebhookStatus, startWebhookServer, stopWebhookServer } from './webhook';
|
import { getWebhookStatus, startWebhookServer, stopWebhookServer } from './webhook';
|
||||||
|
import { registerVoiceIpc } from './voice/ipc';
|
||||||
|
import { registerSystemIpc } from './ipc/system';
|
||||||
|
import { stopEngine } from './voice/engine';
|
||||||
|
|
||||||
// Audio playback must never be blocked behind a user gesture (TTS starts on incoming events).
|
// Audio playback must never be blocked behind a user gesture (TTS starts on incoming events).
|
||||||
app.commandLine.appendSwitch('autoplay-policy', 'no-user-gesture-required');
|
app.commandLine.appendSwitch('autoplay-policy', 'no-user-gesture-required');
|
||||||
|
// Test/dev aid: isolate settings, logs and models in a custom directory.
|
||||||
|
if (process.env.EVEFLOW_USER_DATA) app.setPath('userData', process.env.EVEFLOW_USER_DATA);
|
||||||
|
|
||||||
let tray: Tray | null = null;
|
let tray: Tray | null = null;
|
||||||
|
|
||||||
@@ -35,8 +40,9 @@ interface WebhookSettings {
|
|||||||
}
|
}
|
||||||
|
|
||||||
function readWebhookSettings(): WebhookSettings {
|
function readWebhookSettings(): WebhookSettings {
|
||||||
const saved = storeGet<WebhookSettings>('webhook');
|
const saved = storeGet<WebhookSettings>('webhook') ?? {};
|
||||||
return { enabled: true, port: 7842, secret: '', ...(saved ?? {}) };
|
const port = Number(saved.port);
|
||||||
|
return { enabled: saved.enabled !== false, port: Number.isInteger(port) && port > 0 && port < 65536 ? port : 7842, secret: typeof saved.secret === 'string' ? saved.secret : '' };
|
||||||
}
|
}
|
||||||
|
|
||||||
async function bootWebhook(): Promise<void> {
|
async function bootWebhook(): Promise<void> {
|
||||||
@@ -45,7 +51,7 @@ async function bootWebhook(): Promise<void> {
|
|||||||
await stopWebhookServer();
|
await stopWebhookServer();
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
await startWebhookServer(getMainWindow, { port: Number(settings.port) || 7842, secret: settings.secret || undefined });
|
await startWebhookServer(getMainWindow, { port: settings.port ?? 7842, secret: settings.secret || undefined });
|
||||||
}
|
}
|
||||||
|
|
||||||
function sendHotkey(event: HotkeyEvent): void {
|
function sendHotkey(event: HotkeyEvent): void {
|
||||||
@@ -61,7 +67,8 @@ function registerShortcuts(): void {
|
|||||||
sendHotkey('ptt-toggle');
|
sendHotkey('ptt-toggle');
|
||||||
}],
|
}],
|
||||||
['CommandOrControl+Shift+J', () => toggleWindowVisibility()],
|
['CommandOrControl+Shift+J', () => toggleWindowVisibility()],
|
||||||
['CommandOrControl+Shift+Escape', () => sendHotkey('stop-speaking')]
|
// Ctrl+Shift+Escape is the Windows Task Manager shortcut; use a free combination.
|
||||||
|
['CommandOrControl+Alt+Escape', () => sendHotkey('stop-speaking')]
|
||||||
];
|
];
|
||||||
for (const [accelerator, handler] of bindings) {
|
for (const [accelerator, handler] of bindings) {
|
||||||
try {
|
try {
|
||||||
@@ -94,7 +101,7 @@ function createTray(): void {
|
|||||||
};
|
};
|
||||||
rebuildMenu();
|
rebuildMenu();
|
||||||
tray.on('click', () => toggleWindowVisibility());
|
tray.on('click', () => toggleWindowVisibility());
|
||||||
ipcMain.on(IPC.windowSetMode, () => setTimeout(rebuildMenu, 0));
|
windowEvents.on('mode', rebuildMenu);
|
||||||
} catch (err) {
|
} catch (err) {
|
||||||
log('WARN', 'tray', `tray unavailable: ${(err as Error).message}`);
|
log('WARN', 'tray', `tray unavailable: ${(err as Error).message}`);
|
||||||
}
|
}
|
||||||
@@ -125,7 +132,10 @@ function registerCoreIpc(): void {
|
|||||||
}
|
}
|
||||||
});
|
});
|
||||||
ipcMain.on(IPC.windowSetMode, (_e, mode: WindowMode) => setWindowMode(mode === 'compact' ? 'compact' : 'hud'));
|
ipcMain.on(IPC.windowSetMode, (_e, mode: WindowMode) => setWindowMode(mode === 'compact' ? 'compact' : 'hud'));
|
||||||
ipcMain.on(IPC.log, (_e, entry: LogEntry) => writeLog(entry));
|
ipcMain.on(IPC.log, (_e, entry: LogEntry) => {
|
||||||
|
if (!entry || typeof entry.message !== 'string') return;
|
||||||
|
writeLog({ ...entry, message: entry.message.slice(0, 8000), tag: String(entry.tag).slice(0, 64) });
|
||||||
|
});
|
||||||
ipcMain.handle(IPC.appInfo, (): AppInfo => ({
|
ipcMain.handle(IPC.appInfo, (): AppInfo => ({
|
||||||
version: app.getVersion(),
|
version: app.getVersion(),
|
||||||
platform: process.platform,
|
platform: process.platform,
|
||||||
@@ -141,7 +151,7 @@ function registerCoreIpc(): void {
|
|||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
app.whenReady().then(async () => {
|
async function boot(): Promise<void> {
|
||||||
log('INFO', 'app', `EveFlow ${app.getVersion()} starting (electron ${process.versions.electron}, chrome ${process.versions.chrome})`);
|
log('INFO', 'app', `EveFlow ${app.getVersion()} starting (electron ${process.versions.electron}, chrome ${process.versions.chrome})`);
|
||||||
configureSession();
|
configureSession();
|
||||||
registerStoreIpc();
|
registerStoreIpc();
|
||||||
@@ -149,6 +159,8 @@ app.whenReady().then(async () => {
|
|||||||
registerTelemetryIpc();
|
registerTelemetryIpc();
|
||||||
registerFilesIpc();
|
registerFilesIpc();
|
||||||
registerCoreIpc();
|
registerCoreIpc();
|
||||||
|
registerVoiceIpc();
|
||||||
|
registerSystemIpc();
|
||||||
createMainWindow();
|
createMainWindow();
|
||||||
createTray();
|
createTray();
|
||||||
registerShortcuts();
|
registerShortcuts();
|
||||||
@@ -157,12 +169,19 @@ app.whenReady().then(async () => {
|
|||||||
app.on('activate', () => {
|
app.on('activate', () => {
|
||||||
if (BrowserWindow.getAllWindows().length === 0) createMainWindow();
|
if (BrowserWindow.getAllWindows().length === 0) createMainWindow();
|
||||||
});
|
});
|
||||||
});
|
}
|
||||||
|
|
||||||
app.on('will-quit', () => {
|
if (app.hasSingleInstanceLock()) {
|
||||||
|
app.whenReady().then(boot).catch((err) => log('ERROR', 'app', `boot failed: ${(err as Error).message}`));
|
||||||
|
}
|
||||||
|
|
||||||
|
app.on('will-quit', (event) => {
|
||||||
globalShortcut.unregisterAll();
|
globalShortcut.unregisterAll();
|
||||||
abortAllStreams();
|
abortAllStreams();
|
||||||
|
stopEngine();
|
||||||
void stopWebhookServer();
|
void stopWebhookServer();
|
||||||
|
event.preventDefault();
|
||||||
|
flushStore().finally(() => app.exit(0));
|
||||||
});
|
});
|
||||||
|
|
||||||
app.on('window-all-closed', () => {
|
app.on('window-all-closed', () => {
|
||||||
|
|||||||
+26
-3
@@ -13,7 +13,8 @@ import {
|
|||||||
type WebhookStatus,
|
type WebhookStatus,
|
||||||
type WindowMode
|
type WindowMode
|
||||||
} from '../shared/ipc';
|
} from '../shared/ipc';
|
||||||
import type { EveFlowBridge, Unsubscribe } from '../shared/bridge';
|
import type { EveFlowBridge, SystemAction, SystemActionResult, Unsubscribe } from '../shared/bridge';
|
||||||
|
import { VOICE_IPC, type KwsDetection, type KwsStartRequest, type VadEvent, type VadStartRequest, type SynthesizeRequest, type SynthesizeResult, type TranscribeRequest, type TranscribeResult, type VoiceDownloadProgress, type VoiceEngineStatus, type VoiceModelStatus } from '../shared/voice';
|
||||||
|
|
||||||
function subscribe<T>(channel: string, callback: (payload: T) => void): Unsubscribe {
|
function subscribe<T>(channel: string, callback: (payload: T) => void): Unsubscribe {
|
||||||
const listener = (_event: Electron.IpcRendererEvent, payload: T) => callback(payload);
|
const listener = (_event: Electron.IpcRendererEvent, payload: T) => callback(payload);
|
||||||
@@ -25,7 +26,8 @@ const api: EveFlowBridge = {
|
|||||||
window: {
|
window: {
|
||||||
control: (action: 'minimize' | 'maximize' | 'close' | 'hide') => ipcRenderer.send(IPC.windowControl, action),
|
control: (action: 'minimize' | 'maximize' | 'close' | 'hide') => ipcRenderer.send(IPC.windowControl, action),
|
||||||
setMode: (mode: WindowMode) => ipcRenderer.send(IPC.windowSetMode, mode),
|
setMode: (mode: WindowMode) => ipcRenderer.send(IPC.windowSetMode, mode),
|
||||||
onModeChanged: (cb: (mode: WindowMode) => void) => subscribe<WindowMode>(IPC.windowModeChanged, cb)
|
onModeChanged: (cb: (mode: WindowMode) => void) => subscribe<WindowMode>(IPC.windowModeChanged, cb),
|
||||||
|
onVisibility: (cb: (visible: boolean) => void) => subscribe<boolean>(IPC.windowVisibility, cb)
|
||||||
},
|
},
|
||||||
log: (entry: LogEntry) => ipcRenderer.send(IPC.log, entry),
|
log: (entry: LogEntry) => ipcRenderer.send(IPC.log, entry),
|
||||||
store: {
|
store: {
|
||||||
@@ -41,7 +43,9 @@ const api: EveFlowBridge = {
|
|||||||
},
|
},
|
||||||
system: {
|
system: {
|
||||||
metrics: () => ipcRenderer.invoke(IPC.metrics) as Promise<SystemMetrics>,
|
metrics: () => ipcRenderer.invoke(IPC.metrics) as Promise<SystemMetrics>,
|
||||||
appInfo: () => ipcRenderer.invoke(IPC.appInfo) as Promise<AppInfo>
|
appInfo: () => ipcRenderer.invoke(IPC.appInfo) as Promise<AppInfo>,
|
||||||
|
captureScreen: (maxWidth?: number) => ipcRenderer.invoke(IPC.screenCapture, maxWidth) as Promise<string>,
|
||||||
|
action: (action: SystemAction) => ipcRenderer.invoke(IPC.systemAction, action) as Promise<SystemActionResult>
|
||||||
},
|
},
|
||||||
files: {
|
files: {
|
||||||
readLocal: (filePath: string) => ipcRenderer.invoke(IPC.readLocalFile, filePath) as Promise<string>,
|
readLocal: (filePath: string) => ipcRenderer.invoke(IPC.readLocalFile, filePath) as Promise<string>,
|
||||||
@@ -57,6 +61,25 @@ const api: EveFlowBridge = {
|
|||||||
},
|
},
|
||||||
hotkeys: {
|
hotkeys: {
|
||||||
on: (cb: (event: HotkeyEvent) => void) => subscribe<HotkeyEvent>(IPC.hotkey, cb)
|
on: (cb: (event: HotkeyEvent) => void) => subscribe<HotkeyEvent>(IPC.hotkey, cb)
|
||||||
|
},
|
||||||
|
voice: {
|
||||||
|
status: () => ipcRenderer.invoke(VOICE_IPC.status) as Promise<VoiceEngineStatus>,
|
||||||
|
listModels: () => ipcRenderer.invoke(VOICE_IPC.modelsList) as Promise<VoiceModelStatus[]>,
|
||||||
|
downloadModel: (id: string) => ipcRenderer.invoke(VOICE_IPC.modelsDownload, id) as Promise<VoiceModelStatus>,
|
||||||
|
cancelDownload: (id: string) => ipcRenderer.invoke(VOICE_IPC.modelsCancel, id) as Promise<boolean>,
|
||||||
|
removeModel: (id: string) => ipcRenderer.invoke(VOICE_IPC.modelsRemove, id) as Promise<VoiceModelStatus[]>,
|
||||||
|
onProgress: (cb: (progress: VoiceDownloadProgress) => void) => subscribe<VoiceDownloadProgress>(VOICE_IPC.modelsProgress, cb),
|
||||||
|
transcribe: (req: TranscribeRequest) => ipcRenderer.invoke(VOICE_IPC.transcribe, req) as Promise<TranscribeResult>,
|
||||||
|
synthesize: (req: SynthesizeRequest) => ipcRenderer.invoke(VOICE_IPC.synthesize, req) as Promise<SynthesizeResult>,
|
||||||
|
unload: (id?: string) => ipcRenderer.invoke(VOICE_IPC.unload, id) as Promise<unknown>,
|
||||||
|
kwsStart: (req: KwsStartRequest) => ipcRenderer.invoke(VOICE_IPC.kwsStart, req) as Promise<{ accepted: string[]; rejected: string[] }>,
|
||||||
|
kwsStop: () => ipcRenderer.invoke(VOICE_IPC.kwsStop) as Promise<void>,
|
||||||
|
kwsAudio: (pcm: Uint8Array, sampleRate: number) => ipcRenderer.send(VOICE_IPC.kwsAudio, pcm, sampleRate),
|
||||||
|
onKwsDetected: (cb: (detection: KwsDetection) => void) => subscribe<KwsDetection>(VOICE_IPC.kwsDetected, cb),
|
||||||
|
vadStart: (req: VadStartRequest) => ipcRenderer.invoke(VOICE_IPC.vadStart, req) as Promise<void>,
|
||||||
|
vadStop: () => ipcRenderer.invoke(VOICE_IPC.vadStop) as Promise<void>,
|
||||||
|
vadAudio: (pcm: Uint8Array, sampleRate: number) => ipcRenderer.send(VOICE_IPC.vadAudio, pcm, sampleRate),
|
||||||
|
onVadEvent: (cb: (event: VadEvent) => void) => subscribe<VadEvent>(VOICE_IPC.vadEvent, cb)
|
||||||
}
|
}
|
||||||
};
|
};
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,166 @@
|
|||||||
|
import type { VoiceModelSpec, VoiceSpeaker } from '../../shared/voice';
|
||||||
|
|
||||||
|
const ASR = 'https://github.com/k2-fsa/sherpa-onnx/releases/download/asr-models';
|
||||||
|
const KWS = 'https://github.com/k2-fsa/sherpa-onnx/releases/download/kws-models';
|
||||||
|
const TTS = 'https://github.com/k2-fsa/sherpa-onnx/releases/download/tts-models';
|
||||||
|
|
||||||
|
const KOKORO_SPEAKERS: VoiceSpeaker[] = [
|
||||||
|
{ id: 30, name: 'Siwis (femme, français)', lang: 'fr' },
|
||||||
|
{ id: 3, name: 'Heart (femme, anglais US)', lang: 'en' },
|
||||||
|
{ id: 2, name: 'Bella (femme, anglais US)', lang: 'en' },
|
||||||
|
{ id: 7, name: 'Nova (femme, anglais US)', lang: 'en' },
|
||||||
|
{ id: 11, name: 'Adam (homme, anglais US)', lang: 'en' },
|
||||||
|
{ id: 16, name: 'Michael (homme, anglais US)', lang: 'en' },
|
||||||
|
{ id: 17, name: 'Onyx (homme, anglais US)', lang: 'en' },
|
||||||
|
{ id: 21, name: 'Emma (femme, anglais UK)', lang: 'en' },
|
||||||
|
{ id: 24, name: 'Daniel (homme, anglais UK)', lang: 'en' },
|
||||||
|
{ id: 26, name: 'George (homme, anglais UK)', lang: 'en' }
|
||||||
|
];
|
||||||
|
|
||||||
|
export const VOICE_CATALOG: VoiceModelSpec[] = [
|
||||||
|
{
|
||||||
|
id: 'whisper-base',
|
||||||
|
kind: 'stt',
|
||||||
|
engine: 'whisper',
|
||||||
|
name: 'Whisper base (multilingue)',
|
||||||
|
description: 'Le plus rapide sur CPU ; précision moyenne en français, suffisante pour des commandes courtes.',
|
||||||
|
languages: ['fr', 'en', 'multi'],
|
||||||
|
sizeMb: 208,
|
||||||
|
url: `${ASR}/sherpa-onnx-whisper-base.tar.bz2`,
|
||||||
|
dir: 'sherpa-onnx-whisper-base',
|
||||||
|
files: ['base-encoder.int8.onnx', 'base-decoder.int8.onnx', 'base-tokens.txt'],
|
||||||
|
recommended: true
|
||||||
|
},
|
||||||
|
{
|
||||||
|
id: 'whisper-small',
|
||||||
|
kind: 'stt',
|
||||||
|
engine: 'whisper',
|
||||||
|
name: 'Whisper small (multilingue)',
|
||||||
|
description: 'Recommandé pour le français : nettement plus précis que base, environ trois fois plus lent.',
|
||||||
|
languages: ['fr', 'en', 'multi'],
|
||||||
|
sizeMb: 640,
|
||||||
|
url: `${ASR}/sherpa-onnx-whisper-small.tar.bz2`,
|
||||||
|
dir: 'sherpa-onnx-whisper-small',
|
||||||
|
files: ['small-encoder.int8.onnx', 'small-decoder.int8.onnx', 'small-tokens.txt'],
|
||||||
|
recommended: true
|
||||||
|
},
|
||||||
|
{
|
||||||
|
id: 'whisper-turbo',
|
||||||
|
kind: 'stt',
|
||||||
|
engine: 'whisper',
|
||||||
|
name: 'Whisper large-v3 turbo (multilingue)',
|
||||||
|
description: 'La meilleure précision ; demande un processeur puissant.',
|
||||||
|
languages: ['fr', 'en', 'multi'],
|
||||||
|
sizeMb: 564,
|
||||||
|
url: `${ASR}/sherpa-onnx-whisper-turbo.tar.bz2`,
|
||||||
|
dir: 'sherpa-onnx-whisper-turbo',
|
||||||
|
files: ['turbo-encoder.int8.onnx', 'turbo-decoder.int8.onnx', 'turbo-tokens.txt']
|
||||||
|
},
|
||||||
|
{
|
||||||
|
id: 'sense-voice',
|
||||||
|
kind: 'stt',
|
||||||
|
engine: 'sense-voice',
|
||||||
|
name: 'SenseVoice small (zh / en / ja / ko)',
|
||||||
|
description: 'Très rapide, mais ne comprend pas le français.',
|
||||||
|
languages: ['en', 'zh', 'ja', 'ko'],
|
||||||
|
sizeMb: 163,
|
||||||
|
url: `${ASR}/sherpa-onnx-sense-voice-zh-en-ja-ko-yue-int8-2024-07-17.tar.bz2`,
|
||||||
|
dir: 'sherpa-onnx-sense-voice-zh-en-ja-ko-yue-int8-2024-07-17',
|
||||||
|
files: ['model.int8.onnx', 'tokens.txt']
|
||||||
|
},
|
||||||
|
{
|
||||||
|
id: 'kws-en',
|
||||||
|
kind: 'kws',
|
||||||
|
engine: 'kws-transducer',
|
||||||
|
name: 'Détecteur de mot-clé (zipformer, 3 Mo)',
|
||||||
|
description: 'Écoute permanente du mot d’activation, quasi gratuite en CPU. Mot-clé libre (« jarvis », « hey jarvis »…).',
|
||||||
|
languages: ['en', 'fr'],
|
||||||
|
sizeMb: 4,
|
||||||
|
url: `${KWS}/sherpa-onnx-kws-zipformer-gigaspeech-3.3M-2024-01-01.tar.bz2`,
|
||||||
|
dir: 'sherpa-onnx-kws-zipformer-gigaspeech-3.3M-2024-01-01',
|
||||||
|
files: [
|
||||||
|
'encoder-epoch-12-avg-2-chunk-16-left-64.int8.onnx',
|
||||||
|
'decoder-epoch-12-avg-2-chunk-16-left-64.int8.onnx',
|
||||||
|
'joiner-epoch-12-avg-2-chunk-16-left-64.int8.onnx',
|
||||||
|
'tokens.txt'
|
||||||
|
],
|
||||||
|
recommended: true
|
||||||
|
},
|
||||||
|
{
|
||||||
|
id: 'silero-vad',
|
||||||
|
kind: 'vad',
|
||||||
|
engine: 'silero',
|
||||||
|
name: 'Silero VAD (fin de phrase neuronale, 0,6 Mo)',
|
||||||
|
description: 'Détecte précisément le début et la fin de la parole pendant l’écoute permanente ; moins de faux départs sur le bruit.',
|
||||||
|
languages: ['multi'],
|
||||||
|
sizeMb: 1,
|
||||||
|
url: `${ASR}/silero_vad.onnx`,
|
||||||
|
dir: 'silero-vad',
|
||||||
|
files: ['silero_vad.onnx'],
|
||||||
|
recommended: true
|
||||||
|
},
|
||||||
|
{
|
||||||
|
id: 'kokoro-v1',
|
||||||
|
kind: 'tts',
|
||||||
|
engine: 'kokoro',
|
||||||
|
name: 'Kokoro v1.0 multilingue',
|
||||||
|
description: 'Voix très naturelle, une voix française (Siwis) et de nombreuses voix anglaises. 24 kHz.',
|
||||||
|
languages: ['fr', 'en', 'multi'],
|
||||||
|
sizeMb: 349,
|
||||||
|
url: `${TTS}/kokoro-multi-lang-v1_0.tar.bz2`,
|
||||||
|
dir: 'kokoro-multi-lang-v1_0',
|
||||||
|
files: ['model.onnx', 'voices.bin', 'tokens.txt', 'lexicon-us-en.txt', 'lexicon-zh.txt', 'espeak-ng-data/phontab'],
|
||||||
|
speakers: KOKORO_SPEAKERS,
|
||||||
|
sampleRate: 24000,
|
||||||
|
recommended: true
|
||||||
|
},
|
||||||
|
{
|
||||||
|
id: 'piper-fr-siwis',
|
||||||
|
kind: 'tts',
|
||||||
|
engine: 'piper',
|
||||||
|
name: 'Piper Siwis (femme, français)',
|
||||||
|
description: 'Léger et instantané, rendu un peu plus mécanique que Kokoro. 22 kHz.',
|
||||||
|
languages: ['fr'],
|
||||||
|
sizeMb: 67,
|
||||||
|
url: `${TTS}/vits-piper-fr_FR-siwis-medium.tar.bz2`,
|
||||||
|
dir: 'vits-piper-fr_FR-siwis-medium',
|
||||||
|
files: ['fr_FR-siwis-medium.onnx', 'tokens.txt', 'espeak-ng-data/phontab'],
|
||||||
|
speakers: [{ id: 0, name: 'Siwis', lang: 'fr' }],
|
||||||
|
sampleRate: 22050
|
||||||
|
},
|
||||||
|
{
|
||||||
|
id: 'piper-fr-tom',
|
||||||
|
kind: 'tts',
|
||||||
|
engine: 'piper',
|
||||||
|
name: 'Piper Tom (homme, français)',
|
||||||
|
description: 'Voix masculine légère et instantanée. 22 kHz.',
|
||||||
|
languages: ['fr'],
|
||||||
|
sizeMb: 67,
|
||||||
|
url: `${TTS}/vits-piper-fr_FR-tom-medium.tar.bz2`,
|
||||||
|
dir: 'vits-piper-fr_FR-tom-medium',
|
||||||
|
files: ['fr_FR-tom-medium.onnx', 'tokens.txt', 'espeak-ng-data/phontab'],
|
||||||
|
speakers: [{ id: 0, name: 'Tom', lang: 'fr' }],
|
||||||
|
sampleRate: 22050
|
||||||
|
},
|
||||||
|
{
|
||||||
|
id: 'piper-fr-upmc',
|
||||||
|
kind: 'tts',
|
||||||
|
engine: 'piper',
|
||||||
|
name: 'Piper UPMC (Jessica et Pierre, français)',
|
||||||
|
description: 'Deux voix françaises, une féminine et une masculine. 22 kHz.',
|
||||||
|
languages: ['fr'],
|
||||||
|
sizeMb: 80,
|
||||||
|
url: `${TTS}/vits-piper-fr_FR-upmc-medium.tar.bz2`,
|
||||||
|
dir: 'vits-piper-fr_FR-upmc-medium',
|
||||||
|
files: ['fr_FR-upmc-medium.onnx', 'tokens.txt', 'espeak-ng-data/phontab'],
|
||||||
|
speakers: [
|
||||||
|
{ id: 0, name: 'Jessica (femme)', lang: 'fr' },
|
||||||
|
{ id: 1, name: 'Pierre (homme)', lang: 'fr' }
|
||||||
|
],
|
||||||
|
sampleRate: 22050
|
||||||
|
}
|
||||||
|
];
|
||||||
|
|
||||||
|
export function findModel(id: string): VoiceModelSpec | undefined {
|
||||||
|
return VOICE_CATALOG.find((m) => m.id === id);
|
||||||
|
}
|
||||||
@@ -0,0 +1,240 @@
|
|||||||
|
/**
|
||||||
|
* Host side of the voice worker: spawns the utility process on demand, correlates
|
||||||
|
* requests and responses, restarts the worker if it crashes.
|
||||||
|
*/
|
||||||
|
import { utilityProcess, type UtilityProcess, type WebContents } from 'electron';
|
||||||
|
import { createHash } from 'node:crypto';
|
||||||
|
import fs from 'node:fs';
|
||||||
|
import path from 'node:path';
|
||||||
|
import { VOICE_IPC, type KwsDetection, type KwsStartRequest, type SynthesizeRequest, type SynthesizeResult, type TranscribeRequest, type TranscribeResult, type VadEvent, type VadStartRequest, type VoiceEngineStatus } from '../../shared/voice';
|
||||||
|
import { buildKeywordsFile, parseTokens } from '../../shared/keywords';
|
||||||
|
import { findModel } from './catalog';
|
||||||
|
import { isInstalled, modelDir, modelsDir } from './models';
|
||||||
|
import { log } from '../logger';
|
||||||
|
|
||||||
|
/** Renderer that receives keyword detections while spotting is active. */
|
||||||
|
let kwsSubscriber: WebContents | null = null;
|
||||||
|
let kwsActive = false;
|
||||||
|
let kwsRequest: KwsStartRequest | null = null;
|
||||||
|
let vadSubscriber: WebContents | null = null;
|
||||||
|
let vadActive = false;
|
||||||
|
|
||||||
|
interface Pending {
|
||||||
|
resolve: (value: unknown) => void;
|
||||||
|
reject: (err: Error) => void;
|
||||||
|
timer: ReturnType<typeof setTimeout>;
|
||||||
|
}
|
||||||
|
|
||||||
|
let child: UtilityProcess | null = null;
|
||||||
|
let nextId = 1;
|
||||||
|
const pending = new Map<number, Pending>();
|
||||||
|
|
||||||
|
function rejectAll(message: string): void {
|
||||||
|
for (const [id, p] of pending) {
|
||||||
|
clearTimeout(p.timer);
|
||||||
|
p.reject(new Error(message));
|
||||||
|
pending.delete(id);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function spawn(): UtilityProcess {
|
||||||
|
if (child) return child;
|
||||||
|
const script = path.join(__dirname, 'voice-worker.js');
|
||||||
|
const proc = utilityProcess.fork(script, [], { serviceName: 'eveflow-voice', stdio: 'pipe' });
|
||||||
|
child = proc;
|
||||||
|
proc.stdout?.on('data', (d: Buffer) => log('DEBUG', 'voice-worker', d.toString().trim()));
|
||||||
|
proc.stderr?.on('data', (d: Buffer) => log('WARN', 'voice-worker', d.toString().trim()));
|
||||||
|
proc.on('message', (msg: { id?: number; type?: string; ok?: boolean; result?: unknown; error?: string; keyword?: string; at?: number; event?: { type: string; wav?: string; durationSec?: number } }) => {
|
||||||
|
if (msg.type === 'vad.event' && msg.event) {
|
||||||
|
if (vadSubscriber && !vadSubscriber.isDestroyed()) {
|
||||||
|
const ev = msg.event;
|
||||||
|
const payload: VadEvent =
|
||||||
|
ev.type === 'segment' && ev.wav
|
||||||
|
? { type: 'segment', wav: new Uint8Array(Buffer.from(ev.wav, 'base64')), durationSec: ev.durationSec ?? 0 }
|
||||||
|
: ev.type === 'speech-start'
|
||||||
|
? { type: 'speech-start' }
|
||||||
|
: { type: 'error', message: 'événement VAD inconnu' };
|
||||||
|
vadSubscriber.send(VOICE_IPC.vadEvent, payload);
|
||||||
|
}
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if (msg.type === 'kws.detected') {
|
||||||
|
if (kwsSubscriber && !kwsSubscriber.isDestroyed()) {
|
||||||
|
kwsSubscriber.send(VOICE_IPC.kwsDetected, { keyword: msg.keyword ?? '', at: msg.at ?? Date.now() } satisfies KwsDetection);
|
||||||
|
}
|
||||||
|
log('INFO', 'voice', `wake word detected: ${msg.keyword}`);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if (typeof msg.id !== 'number') return;
|
||||||
|
const p = pending.get(msg.id);
|
||||||
|
if (!p) return;
|
||||||
|
pending.delete(msg.id);
|
||||||
|
clearTimeout(p.timer);
|
||||||
|
if (msg.ok) p.resolve(msg.result);
|
||||||
|
else p.reject(new Error(msg.error ?? 'erreur moteur vocal'));
|
||||||
|
});
|
||||||
|
proc.on('exit', (code) => {
|
||||||
|
log(code === 0 ? 'INFO' : 'ERROR', 'voice-worker', `exited with code ${code}`);
|
||||||
|
// A newer worker may already have replaced this one; only clean up if we are still current.
|
||||||
|
if (child !== proc) return;
|
||||||
|
child = null;
|
||||||
|
rejectAll('Le moteur vocal local s’est arrêté de façon inattendue.');
|
||||||
|
// Keyword spotting survives a worker restart: re-arm on the next audio frame.
|
||||||
|
if (kwsActive) kwsArmed = false;
|
||||||
|
});
|
||||||
|
log('INFO', 'voice', 'voice worker started');
|
||||||
|
return proc;
|
||||||
|
}
|
||||||
|
|
||||||
|
function request<T>(message: Record<string, unknown>, timeoutMs = 120_000): Promise<T> {
|
||||||
|
const proc = spawn();
|
||||||
|
const id = nextId++;
|
||||||
|
return new Promise<T>((resolve, reject) => {
|
||||||
|
const timer = setTimeout(() => {
|
||||||
|
pending.delete(id);
|
||||||
|
reject(new Error('Délai dépassé pour le moteur vocal local.'));
|
||||||
|
// The worker handles requests synchronously: a stuck request wedges everything behind it.
|
||||||
|
if (child === proc) {
|
||||||
|
log('WARN', 'voice', 'worker unresponsive, restarting');
|
||||||
|
stopEngine();
|
||||||
|
}
|
||||||
|
}, timeoutMs);
|
||||||
|
pending.set(id, { resolve: (v) => resolve(v as T), reject, timer });
|
||||||
|
try {
|
||||||
|
proc.postMessage({ id, ...message });
|
||||||
|
} catch (err) {
|
||||||
|
pending.delete(id);
|
||||||
|
clearTimeout(timer);
|
||||||
|
reject(err as Error);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
function modelRef(modelId: string) {
|
||||||
|
const spec = findModel(modelId);
|
||||||
|
if (!spec) throw new Error(`Modèle inconnu : ${modelId}`);
|
||||||
|
if (!isInstalled(spec)) throw new Error(`Modèle « ${spec.name} » non installé. Téléchargez-le dans Paramètres → Modèles.`);
|
||||||
|
return { id: spec.id, engine: spec.engine, dir: modelDir(spec), files: spec.files };
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function engineStatus(): Promise<VoiceEngineStatus> {
|
||||||
|
try {
|
||||||
|
const status = await request<Omit<VoiceEngineStatus, 'modelsDir'>>({ type: 'status' }, 20_000);
|
||||||
|
return { ...status, modelsDir: modelsDir() };
|
||||||
|
} catch (err) {
|
||||||
|
return { available: false, error: (err as Error).message, loaded: [], modelsDir: modelsDir() };
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
export function transcribe(req: TranscribeRequest): Promise<TranscribeResult> {
|
||||||
|
// Binary payloads are base64-encoded for the worker: the utility process channel rejects external buffers.
|
||||||
|
const wav = Buffer.from(req.wav.buffer, req.wav.byteOffset, req.wav.byteLength).toString('base64');
|
||||||
|
return request<TranscribeResult>({ type: 'transcribe', model: modelRef(req.modelId), wav, language: req.language }, 180_000);
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function synthesize(req: SynthesizeRequest): Promise<SynthesizeResult> {
|
||||||
|
const result = await request<Omit<SynthesizeResult, 'wav'> & { wav: string }>(
|
||||||
|
{ type: 'synthesize', model: modelRef(req.modelId), text: req.text, speaker: req.speaker, speed: req.speed },
|
||||||
|
180_000
|
||||||
|
);
|
||||||
|
const buffer = Buffer.from(result.wav, 'base64');
|
||||||
|
return { ...result, wav: new Uint8Array(buffer.buffer, buffer.byteOffset, buffer.byteLength) };
|
||||||
|
}
|
||||||
|
|
||||||
|
let kwsArmed = false;
|
||||||
|
const SENSITIVITY_THRESHOLD: Record<number, number> = { 1: 0.45, 2: 0.35, 3: 0.25, 4: 0.18, 5: 0.12 };
|
||||||
|
const SENSITIVITY_SCORE: Record<number, number> = { 1: 1.0, 2: 1.0, 3: 1.2, 4: 1.5, 5: 2.0 };
|
||||||
|
|
||||||
|
/** Start keyword spotting; the keywords file is derived from the phrases and the model vocabulary. */
|
||||||
|
export async function kwsStart(req: KwsStartRequest, sender: WebContents): Promise<{ accepted: string[]; rejected: string[] }> {
|
||||||
|
const spec = findModel(req.modelId);
|
||||||
|
if (!spec || spec.kind !== 'kws') throw new Error('Modèle de détection introuvable');
|
||||||
|
if (!isInstalled(spec)) throw new Error('Détecteur de mot-clé non installé (Paramètres → Modèles locaux).');
|
||||||
|
const dir = modelDir(spec);
|
||||||
|
const vocab = parseTokens(fs.readFileSync(path.join(dir, 'tokens.txt'), 'utf8'));
|
||||||
|
const phrases = req.keywords.map((k) => String(k).slice(0, 40)).filter(Boolean).slice(0, 8);
|
||||||
|
const file = buildKeywordsFile(phrases, vocab);
|
||||||
|
if (file.accepted.length === 0) throw new Error(`Aucun mot d’activation encodable : ${file.rejected.join(', ')}`);
|
||||||
|
const hash = createHash('sha1').update(file.content).digest('hex').slice(0, 10);
|
||||||
|
const keywordsFile = path.join(modelsDir(), `keywords-${hash}.txt`);
|
||||||
|
fs.writeFileSync(keywordsFile, file.content, 'utf8');
|
||||||
|
const sensitivity = Math.min(5, Math.max(1, Math.round(req.sensitivity))) as 1 | 2 | 3 | 4 | 5;
|
||||||
|
kwsSubscriber = sender;
|
||||||
|
kwsRequest = req;
|
||||||
|
await request({ type: 'kws.start', model: { id: spec.id, engine: spec.engine, dir, files: spec.files }, keywordsFile, threshold: SENSITIVITY_THRESHOLD[sensitivity], score: SENSITIVITY_SCORE[sensitivity] }, 60_000);
|
||||||
|
kwsActive = true;
|
||||||
|
kwsArmed = true;
|
||||||
|
log('INFO', 'voice', `keyword spotting on: ${file.accepted.join(', ')} (threshold ${SENSITIVITY_THRESHOLD[sensitivity]})`);
|
||||||
|
return { accepted: file.accepted, rejected: file.rejected };
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function kwsStop(): Promise<void> {
|
||||||
|
kwsActive = false;
|
||||||
|
kwsArmed = false;
|
||||||
|
kwsSubscriber = null;
|
||||||
|
if (child) await request({ type: 'kws.stop' }, 10_000).catch(() => undefined);
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Feed 16-bit PCM from the renderer (fire-and-forget). */
|
||||||
|
export function kwsFeed(pcm: Uint8Array, sampleRate: number): void {
|
||||||
|
if (!kwsActive) return;
|
||||||
|
const proc = spawn();
|
||||||
|
if (!kwsArmed) {
|
||||||
|
// Worker restarted: re-create the spotter before feeding audio.
|
||||||
|
if (kwsRequest && kwsSubscriber) {
|
||||||
|
kwsArmed = true;
|
||||||
|
kwsStart(kwsRequest, kwsSubscriber).catch((err) => log('WARN', 'voice', `kws re-arm failed: ${(err as Error).message}`));
|
||||||
|
}
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
const b64 = Buffer.from(pcm.buffer, pcm.byteOffset, pcm.byteLength).toString('base64');
|
||||||
|
try {
|
||||||
|
proc.postMessage({ id: -1, type: 'kws.audio', pcm: b64, sampleRate });
|
||||||
|
} catch (err) {
|
||||||
|
log('WARN', 'voice', `kws feed failed: ${(err as Error).message}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Neural end-of-speech detection (Silero) on frames streamed by the renderer. */
|
||||||
|
export async function vadStart(req: VadStartRequest, sender: WebContents): Promise<void> {
|
||||||
|
const spec = findModel(req.modelId);
|
||||||
|
if (!spec || spec.kind !== 'vad') throw new Error('Modèle VAD introuvable');
|
||||||
|
if (!isInstalled(spec)) throw new Error('Silero VAD non installé (Paramètres → Modèles locaux).');
|
||||||
|
vadSubscriber = sender;
|
||||||
|
await request(
|
||||||
|
{ type: 'vad.start', model: { id: spec.id, engine: spec.engine, dir: modelDir(spec), files: spec.files }, silenceMs: req.silenceMs, threshold: req.threshold, maxUtteranceSec: req.maxUtteranceSec },
|
||||||
|
30_000
|
||||||
|
);
|
||||||
|
vadActive = true;
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function vadStop(): Promise<void> {
|
||||||
|
vadActive = false;
|
||||||
|
vadSubscriber = null;
|
||||||
|
if (child) await request({ type: 'vad.stop' }, 10_000).catch(() => undefined);
|
||||||
|
}
|
||||||
|
|
||||||
|
export function vadFeed(pcm: Uint8Array, sampleRate: number): void {
|
||||||
|
if (!vadActive || !child) return;
|
||||||
|
const b64 = Buffer.from(pcm.buffer, pcm.byteOffset, pcm.byteLength).toString('base64');
|
||||||
|
try {
|
||||||
|
child.postMessage({ id: -1, type: 'vad.audio', pcm: b64, sampleRate });
|
||||||
|
} catch (err) {
|
||||||
|
log('WARN', 'voice', `vad feed failed: ${(err as Error).message}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Native model memory is only released deterministically by restarting the worker. */
|
||||||
|
export function unload(_modelId?: string): Promise<unknown> {
|
||||||
|
stopEngine();
|
||||||
|
return Promise.resolve({ ok: true });
|
||||||
|
}
|
||||||
|
|
||||||
|
export function stopEngine(): void {
|
||||||
|
const proc = child;
|
||||||
|
child = null;
|
||||||
|
if (proc) {
|
||||||
|
rejectAll('Moteur vocal redémarré.');
|
||||||
|
proc.kill();
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,48 @@
|
|||||||
|
import { ipcMain } from 'electron';
|
||||||
|
import { VOICE_IPC, type KwsStartRequest, type SynthesizeRequest, type TranscribeRequest, type VadStartRequest } from '../../shared/voice';
|
||||||
|
import { engineStatus, kwsFeed, kwsStart, kwsStop, synthesize, transcribe, unload, vadFeed, vadStart, vadStop } from './engine';
|
||||||
|
import { cancelDownload, downloadModel, listModels, removeModel } from './models';
|
||||||
|
|
||||||
|
export function registerVoiceIpc(): void {
|
||||||
|
ipcMain.handle(VOICE_IPC.status, () => engineStatus());
|
||||||
|
ipcMain.handle(VOICE_IPC.modelsList, () => listModels());
|
||||||
|
ipcMain.handle(VOICE_IPC.modelsDownload, (event, id: string) => downloadModel(id, event.sender));
|
||||||
|
ipcMain.handle(VOICE_IPC.modelsCancel, (_e, id: string) => {
|
||||||
|
cancelDownload(id);
|
||||||
|
return true;
|
||||||
|
});
|
||||||
|
ipcMain.handle(VOICE_IPC.modelsRemove, async (_e, id: string) => {
|
||||||
|
await unload(id).catch(() => undefined);
|
||||||
|
await removeModel(id);
|
||||||
|
return listModels();
|
||||||
|
});
|
||||||
|
ipcMain.handle(VOICE_IPC.transcribe, (_e, req: TranscribeRequest) => {
|
||||||
|
if (!req || !(req.wav instanceof Uint8Array) || req.wav.byteLength < 44 || req.wav.byteLength > 64 * 1024 * 1024) throw new Error('Audio invalide');
|
||||||
|
if (typeof req.modelId !== 'string') throw new Error('Modèle invalide');
|
||||||
|
return transcribe({ ...req, language: typeof req.language === 'string' ? req.language.slice(0, 8) : 'auto' });
|
||||||
|
});
|
||||||
|
ipcMain.handle(VOICE_IPC.synthesize, (_e, req: SynthesizeRequest) => {
|
||||||
|
if (!req || typeof req.text !== 'string' || !req.text.trim() || req.text.length > 5000) throw new Error('Texte invalide');
|
||||||
|
if (typeof req.modelId !== 'string') throw new Error('Modèle invalide');
|
||||||
|
return synthesize({ ...req, speaker: Number.isFinite(req.speaker) ? req.speaker : 0, speed: Number.isFinite(req.speed) ? req.speed : 1 });
|
||||||
|
});
|
||||||
|
ipcMain.handle(VOICE_IPC.unload, (_e, id?: string) => unload(id));
|
||||||
|
ipcMain.handle(VOICE_IPC.kwsStart, (event, req: KwsStartRequest) => {
|
||||||
|
if (!req || !Array.isArray(req.keywords) || typeof req.modelId !== 'string') throw new Error('Requête invalide');
|
||||||
|
return kwsStart({ ...req, keywords: req.keywords.filter((k) => typeof k === 'string'), sensitivity: Number(req.sensitivity) || 3 }, event.sender);
|
||||||
|
});
|
||||||
|
ipcMain.handle(VOICE_IPC.kwsStop, () => kwsStop());
|
||||||
|
ipcMain.handle(VOICE_IPC.vadStart, (event, req: VadStartRequest) => {
|
||||||
|
if (!req || typeof req.modelId !== 'string') throw new Error('Requête invalide');
|
||||||
|
return vadStart({ modelId: req.modelId, silenceMs: Number(req.silenceMs) || 700, threshold: Number(req.threshold) || 0.5, maxUtteranceSec: Number(req.maxUtteranceSec) || 30 }, event.sender);
|
||||||
|
});
|
||||||
|
ipcMain.handle(VOICE_IPC.vadStop, () => vadStop());
|
||||||
|
ipcMain.on(VOICE_IPC.vadAudio, (_e, pcm: unknown, sampleRate: unknown) => {
|
||||||
|
if (!(pcm instanceof Uint8Array) || pcm.byteLength === 0 || pcm.byteLength > 1024 * 1024) return;
|
||||||
|
vadFeed(pcm, typeof sampleRate === 'number' && sampleRate > 0 ? sampleRate : 16000);
|
||||||
|
});
|
||||||
|
ipcMain.on(VOICE_IPC.kwsAudio, (_e, pcm: unknown, sampleRate: unknown) => {
|
||||||
|
if (!(pcm instanceof Uint8Array) || pcm.byteLength === 0 || pcm.byteLength > 1024 * 1024) return;
|
||||||
|
kwsFeed(pcm, typeof sampleRate === 'number' && sampleRate > 0 ? sampleRate : 16000);
|
||||||
|
});
|
||||||
|
}
|
||||||
@@ -0,0 +1,159 @@
|
|||||||
|
/**
|
||||||
|
* Model manager: downloads sherpa-onnx model archives (tar.bz2) into userData/models,
|
||||||
|
* extracts them, verifies required files and reports progress to the renderer.
|
||||||
|
*/
|
||||||
|
import { app, type WebContents } from 'electron';
|
||||||
|
import fs from 'node:fs';
|
||||||
|
import path from 'node:path';
|
||||||
|
import { pipeline } from 'node:stream/promises';
|
||||||
|
import { Readable, Transform } from 'node:stream';
|
||||||
|
import * as tar from 'tar';
|
||||||
|
import unbzip2 from 'unbzip2-stream';
|
||||||
|
import { VOICE_IPC, type VoiceDownloadProgress, type VoiceModelSpec, type VoiceModelStatus } from '../../shared/voice';
|
||||||
|
import { VOICE_CATALOG, findModel } from './catalog';
|
||||||
|
import { stopEngine } from './engine';
|
||||||
|
import { log } from '../logger';
|
||||||
|
|
||||||
|
const active = new Map<string, AbortController>();
|
||||||
|
/** Windows keeps files locked briefly (AV scans, memory-mapped models): retry removals. */
|
||||||
|
const RM = { recursive: true, force: true, maxRetries: 10, retryDelay: 200 } as const;
|
||||||
|
const DOWNLOAD_IDLE_MS = 60_000;
|
||||||
|
|
||||||
|
export function modelsDir(): string {
|
||||||
|
const dir = path.join(app.getPath('userData'), 'models');
|
||||||
|
fs.mkdirSync(dir, { recursive: true });
|
||||||
|
return dir;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function modelDir(spec: VoiceModelSpec): string {
|
||||||
|
return path.join(modelsDir(), spec.dir);
|
||||||
|
}
|
||||||
|
|
||||||
|
export function isInstalled(spec: VoiceModelSpec): boolean {
|
||||||
|
const dir = modelDir(spec);
|
||||||
|
return spec.files.every((f) => fs.existsSync(path.join(dir, f)));
|
||||||
|
}
|
||||||
|
|
||||||
|
function dirSize(dir: string): number {
|
||||||
|
let total = 0;
|
||||||
|
try {
|
||||||
|
for (const entry of fs.readdirSync(dir, { withFileTypes: true })) {
|
||||||
|
const full = path.join(dir, entry.name);
|
||||||
|
if (entry.isDirectory()) total += dirSize(full);
|
||||||
|
else total += fs.statSync(full).size;
|
||||||
|
}
|
||||||
|
} catch {
|
||||||
|
/* missing */
|
||||||
|
}
|
||||||
|
return total;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function listModels(): VoiceModelStatus[] {
|
||||||
|
return VOICE_CATALOG.map((spec) => {
|
||||||
|
const installed = isInstalled(spec);
|
||||||
|
return {
|
||||||
|
...spec,
|
||||||
|
installed,
|
||||||
|
downloading: active.has(spec.id),
|
||||||
|
installedBytes: installed ? dirSize(modelDir(spec)) : 0
|
||||||
|
};
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function removeModel(id: string): Promise<void> {
|
||||||
|
const spec = findModel(id);
|
||||||
|
if (!spec) throw new Error(`Modèle inconnu : ${id}`);
|
||||||
|
stopEngine(); // releases memory-mapped model files before deleting them
|
||||||
|
await fs.promises.rm(modelDir(spec), RM);
|
||||||
|
log('INFO', 'voice', `model removed: ${id}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
export function cancelDownload(id: string): void {
|
||||||
|
active.get(id)?.abort();
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function downloadModel(id: string, sender: WebContents | null): Promise<VoiceModelStatus> {
|
||||||
|
const spec = findModel(id);
|
||||||
|
if (!spec) throw new Error(`Modèle inconnu : ${id}`);
|
||||||
|
if (active.has(id)) throw new Error('Téléchargement déjà en cours');
|
||||||
|
const controller = new AbortController();
|
||||||
|
active.set(id, controller);
|
||||||
|
const report = (progress: Omit<VoiceDownloadProgress, 'id'>) => {
|
||||||
|
if (sender && !sender.isDestroyed()) sender.send(VOICE_IPC.modelsProgress, { id, ...progress });
|
||||||
|
};
|
||||||
|
|
||||||
|
const tmpDir = path.join(modelsDir(), `.tmp-${spec.id}`);
|
||||||
|
await fs.promises.rm(tmpDir, RM);
|
||||||
|
await fs.promises.mkdir(tmpDir, { recursive: true });
|
||||||
|
// Abort the transfer when no byte arrives for a while (fetch itself has no inactivity timeout).
|
||||||
|
let idleTimer: ReturnType<typeof setTimeout> | null = null;
|
||||||
|
const touch = () => {
|
||||||
|
if (idleTimer) clearTimeout(idleTimer);
|
||||||
|
idleTimer = setTimeout(() => controller.abort(new Error('Téléchargement interrompu (aucune donnée reçue)')), DOWNLOAD_IDLE_MS);
|
||||||
|
};
|
||||||
|
|
||||||
|
try {
|
||||||
|
log('INFO', 'voice', `downloading ${spec.id} from ${spec.url}`);
|
||||||
|
const response = await fetch(spec.url, { signal: controller.signal, redirect: 'follow' });
|
||||||
|
if (!response.ok || !response.body) throw new Error(`HTTP ${response.status} lors du téléchargement`);
|
||||||
|
const total = Number(response.headers.get('content-length')) || Math.round(spec.sizeMb * 1024 * 1024);
|
||||||
|
let received = 0;
|
||||||
|
let lastReport = 0;
|
||||||
|
touch();
|
||||||
|
const counter = new Transform({
|
||||||
|
transform(chunk: Buffer, _enc, cb) {
|
||||||
|
received += chunk.length;
|
||||||
|
touch();
|
||||||
|
const now = Date.now();
|
||||||
|
if (now - lastReport > 250) {
|
||||||
|
lastReport = now;
|
||||||
|
report({ phase: 'download', received, total, percent: Math.min(99, Math.round((received / total) * 100)) });
|
||||||
|
}
|
||||||
|
cb(null, chunk);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
const source = Readable.fromWeb(response.body as import('node:stream/web').ReadableStream<Uint8Array>);
|
||||||
|
if (spec.url.endsWith('.tar.bz2')) {
|
||||||
|
report({ phase: 'download', received: 0, total, percent: 0 });
|
||||||
|
await pipeline(source, counter, unbzip2(), tar.x({ cwd: tmpDir }), { signal: controller.signal });
|
||||||
|
} else {
|
||||||
|
const target = path.join(tmpDir, spec.dir, path.basename(spec.url));
|
||||||
|
await fs.promises.mkdir(path.dirname(target), { recursive: true });
|
||||||
|
await pipeline(source, counter, fs.createWriteStream(target), { signal: controller.signal });
|
||||||
|
}
|
||||||
|
report({ phase: 'extract', received: total, total, percent: 99 });
|
||||||
|
|
||||||
|
const extracted = path.join(tmpDir, spec.dir);
|
||||||
|
const missing = spec.files.filter((f) => !fs.existsSync(path.join(extracted, f)));
|
||||||
|
if (missing.length) throw new Error(`Archive incomplète, fichiers manquants : ${missing.join(', ')}`);
|
||||||
|
|
||||||
|
const finalDir = modelDir(spec);
|
||||||
|
if (isInstalled(spec)) stopEngine();
|
||||||
|
await fs.promises.rm(finalDir, RM);
|
||||||
|
for (let attempt = 0; ; attempt++) {
|
||||||
|
try {
|
||||||
|
await fs.promises.rename(extracted, finalDir);
|
||||||
|
break;
|
||||||
|
} catch (err) {
|
||||||
|
const code = (err as NodeJS.ErrnoException).code ?? '';
|
||||||
|
if (attempt >= 10 || !['EPERM', 'EBUSY', 'ENOTEMPTY', 'EACCES'].includes(code)) throw err;
|
||||||
|
await new Promise((r) => setTimeout(r, 300));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
await fs.promises.rm(tmpDir, RM);
|
||||||
|
report({ phase: 'done', received: total, total, percent: 100 });
|
||||||
|
log('INFO', 'voice', `model installed: ${spec.id}`);
|
||||||
|
} catch (err) {
|
||||||
|
await fs.promises.rm(tmpDir, RM).catch(() => undefined);
|
||||||
|
const e = (controller.signal.reason instanceof Error ? controller.signal.reason : err) as Error;
|
||||||
|
const cancelled = (e.name === 'AbortError' || controller.signal.aborted) && !(controller.signal.reason instanceof Error);
|
||||||
|
report({ phase: cancelled ? 'cancelled' : 'error', received: 0, total: 0, percent: 0, message: cancelled ? 'annulé' : e.message });
|
||||||
|
log(cancelled ? 'INFO' : 'ERROR', 'voice', `download ${spec.id} ${cancelled ? 'cancelled' : 'failed: ' + e.message}`);
|
||||||
|
if (!cancelled) throw new Error(e.message);
|
||||||
|
} finally {
|
||||||
|
if (idleTimer) clearTimeout(idleTimer);
|
||||||
|
active.delete(id);
|
||||||
|
}
|
||||||
|
return listModels().find((m) => m.id === id)!;
|
||||||
|
}
|
||||||
@@ -0,0 +1,438 @@
|
|||||||
|
/**
|
||||||
|
* Voice worker: runs sherpa-onnx (speech recognition and synthesis) in an Electron utility
|
||||||
|
* process so heavy inference never blocks the main process. Also runnable with
|
||||||
|
* `child_process.fork` (advanced serialization) for local tests.
|
||||||
|
*/
|
||||||
|
import os from 'node:os';
|
||||||
|
import path from 'node:path';
|
||||||
|
import type { VoiceEngineKind } from '../../shared/voice';
|
||||||
|
|
||||||
|
interface ModelRef {
|
||||||
|
id: string;
|
||||||
|
engine: VoiceEngineKind;
|
||||||
|
dir: string;
|
||||||
|
files: string[];
|
||||||
|
}
|
||||||
|
|
||||||
|
type Request =
|
||||||
|
| { id: number; type: 'status' }
|
||||||
|
| { id: number; type: 'transcribe'; model: ModelRef; wav: Uint8Array | string; language: string }
|
||||||
|
| { id: number; type: 'synthesize'; model: ModelRef; text: string; speaker: number; speed: number }
|
||||||
|
| { id: number; type: 'unload'; modelId?: string }
|
||||||
|
| { id: number; type: 'kws.start'; model: ModelRef; keywordsFile: string; threshold: number; score: number }
|
||||||
|
| { id: number; type: 'kws.audio'; pcm: string; sampleRate: number }
|
||||||
|
| { id: number; type: 'kws.stop' }
|
||||||
|
| { id: number; type: 'vad.start'; model: ModelRef; silenceMs: number; threshold: number; maxUtteranceSec: number }
|
||||||
|
| { id: number; type: 'vad.audio'; pcm: string; sampleRate: number }
|
||||||
|
| { id: number; type: 'vad.stop' };
|
||||||
|
|
||||||
|
type Response = { id: number; ok: true; result: unknown } | { id: number; ok: false; error: string };
|
||||||
|
|
||||||
|
// ── sherpa-onnx loading (lazy, so a missing native package is reported, not fatal) ──
|
||||||
|
type Sherpa = {
|
||||||
|
OfflineRecognizer: new (config: unknown) => {
|
||||||
|
createStream: () => { acceptWaveform: (w: { sampleRate: number; samples: Float32Array }) => void };
|
||||||
|
decode: (s: unknown) => void;
|
||||||
|
getResult: (s: unknown) => { text: string; lang?: string };
|
||||||
|
};
|
||||||
|
Vad: new (config: unknown, bufferSizeInSeconds: number) => {
|
||||||
|
acceptWaveform: (samples: Float32Array) => void;
|
||||||
|
isEmpty: () => boolean;
|
||||||
|
isDetected: () => boolean;
|
||||||
|
pop: () => void;
|
||||||
|
clear: () => void;
|
||||||
|
front: (enableExternalBuffer?: boolean) => { start: number; samples: Float32Array };
|
||||||
|
reset: () => void;
|
||||||
|
flush: () => void;
|
||||||
|
};
|
||||||
|
KeywordSpotter: new (config: unknown) => {
|
||||||
|
createStream: () => KwsStream;
|
||||||
|
isReady: (s: KwsStream) => boolean;
|
||||||
|
decode: (s: KwsStream) => void;
|
||||||
|
reset: (s: KwsStream) => void;
|
||||||
|
getResult: (s: KwsStream) => { keyword?: string };
|
||||||
|
};
|
||||||
|
OfflineTts: new (config: unknown) => {
|
||||||
|
numSpeakers: number;
|
||||||
|
sampleRate: number;
|
||||||
|
generate: (req: { text: string; sid: number; speed: number; enableExternalBuffer?: boolean }) => { samples: Float32Array; sampleRate: number };
|
||||||
|
};
|
||||||
|
version: string;
|
||||||
|
};
|
||||||
|
|
||||||
|
type KwsStream = { acceptWaveform: (w: { sampleRate: number; samples: Float32Array }) => void };
|
||||||
|
let sherpa: Sherpa | null = null;
|
||||||
|
let kws: { spotter: InstanceType<Sherpa['KeywordSpotter']>; stream: KwsStream } | null = null;
|
||||||
|
let vad: { detector: InstanceType<Sherpa['Vad']>; speaking: boolean; windowSize: number; carry: Float32Array } | null = null;
|
||||||
|
let notify: ((message: unknown) => void) | null = null;
|
||||||
|
let loadError: string | null = null;
|
||||||
|
|
||||||
|
function loadSherpa(): Sherpa {
|
||||||
|
if (sherpa) return sherpa;
|
||||||
|
if (loadError) throw new Error(loadError);
|
||||||
|
try {
|
||||||
|
// eslint-disable-next-line @typescript-eslint/no-require-imports
|
||||||
|
sherpa = require('sherpa-onnx-node') as Sherpa;
|
||||||
|
return sherpa;
|
||||||
|
} catch (err) {
|
||||||
|
loadError = `Module natif sherpa-onnx indisponible : ${(err as Error).message}`;
|
||||||
|
throw new Error(loadError);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
const threads = Math.max(2, Math.min(6, Math.floor(os.cpus().length / 2)));
|
||||||
|
|
||||||
|
// ── caches ────────────────────────────────────────────────────────────────
|
||||||
|
const recognizers = new Map<string, InstanceType<Sherpa['OfflineRecognizer']>>();
|
||||||
|
const synthesizers = new Map<string, InstanceType<Sherpa['OfflineTts']>>();
|
||||||
|
|
||||||
|
function whisperPrefix(model: ModelRef): string {
|
||||||
|
const encoder = model.files.find((f) => f.includes('-encoder'));
|
||||||
|
return encoder ? encoder.slice(0, encoder.indexOf('-encoder')) : 'base';
|
||||||
|
}
|
||||||
|
|
||||||
|
function getRecognizer(model: ModelRef, language: string) {
|
||||||
|
const lang = language === 'auto' ? '' : language.split('-')[0].toLowerCase();
|
||||||
|
const key = `${model.id}:${lang}`;
|
||||||
|
const cached = recognizers.get(key);
|
||||||
|
if (cached) return cached;
|
||||||
|
const s = loadSherpa();
|
||||||
|
const p = (f: string) => path.join(model.dir, f);
|
||||||
|
let modelConfig: Record<string, unknown>;
|
||||||
|
switch (model.engine) {
|
||||||
|
case 'whisper': {
|
||||||
|
const prefix = whisperPrefix(model);
|
||||||
|
modelConfig = {
|
||||||
|
whisper: { encoder: p(`${prefix}-encoder.int8.onnx`), decoder: p(`${prefix}-decoder.int8.onnx`), language: lang, task: 'transcribe', tailPaddings: -1 },
|
||||||
|
tokens: p(`${prefix}-tokens.txt`)
|
||||||
|
};
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
case 'sense-voice':
|
||||||
|
modelConfig = { senseVoice: { model: p('model.int8.onnx'), language: lang || 'auto', useInverseTextNormalization: 1 }, tokens: p('tokens.txt') };
|
||||||
|
break;
|
||||||
|
case 'nemo-transducer':
|
||||||
|
modelConfig = {
|
||||||
|
transducer: { encoder: p('encoder.int8.onnx'), decoder: p('decoder.int8.onnx'), joiner: p('joiner.int8.onnx') },
|
||||||
|
tokens: p('tokens.txt'),
|
||||||
|
modelType: 'nemo_transducer'
|
||||||
|
};
|
||||||
|
break;
|
||||||
|
default:
|
||||||
|
throw new Error(`Moteur STT non supporté : ${model.engine}`);
|
||||||
|
}
|
||||||
|
for (const [k, v] of Object.entries({ numThreads: threads, provider: 'cpu', debug: 0 })) modelConfig[k] = v;
|
||||||
|
// Whisper keeps one recognizer per language; other engines ignore the language key.
|
||||||
|
for (const [k, r] of recognizers) if (k.startsWith(`${model.id}:`)) recognizers.delete(k) && void r;
|
||||||
|
const recognizer = new s.OfflineRecognizer({ featConfig: { sampleRate: 16000, featureDim: 80 }, modelConfig, decodingMethod: 'greedy_search' });
|
||||||
|
recognizers.set(key, recognizer);
|
||||||
|
return recognizer;
|
||||||
|
}
|
||||||
|
|
||||||
|
function getSynthesizer(model: ModelRef) {
|
||||||
|
const cached = synthesizers.get(model.id);
|
||||||
|
if (cached) return cached;
|
||||||
|
const s = loadSherpa();
|
||||||
|
const p = (f: string) => path.join(model.dir, f);
|
||||||
|
let ttsModel: Record<string, unknown>;
|
||||||
|
switch (model.engine) {
|
||||||
|
case 'kokoro':
|
||||||
|
ttsModel = {
|
||||||
|
kokoro: {
|
||||||
|
model: p('model.onnx'),
|
||||||
|
voices: p('voices.bin'),
|
||||||
|
tokens: p('tokens.txt'),
|
||||||
|
dataDir: p('espeak-ng-data'),
|
||||||
|
lexicon: [p('lexicon-us-en.txt'), p('lexicon-zh.txt')].join(',')
|
||||||
|
}
|
||||||
|
};
|
||||||
|
break;
|
||||||
|
case 'piper': {
|
||||||
|
const onnx = model.files.find((f) => f.endsWith('.onnx')) ?? 'model.onnx';
|
||||||
|
ttsModel = { vits: { model: p(onnx), tokens: p('tokens.txt'), dataDir: p('espeak-ng-data') } };
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
default:
|
||||||
|
throw new Error(`Moteur TTS non supporté : ${model.engine}`);
|
||||||
|
}
|
||||||
|
const tts = new s.OfflineTts({ model: { ...ttsModel, numThreads: threads, provider: 'cpu', debug: 0 }, maxNumSentences: 1 });
|
||||||
|
synthesizers.set(model.id, tts);
|
||||||
|
return tts;
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── audio helpers ──────────────────────────────────────────────────────────
|
||||||
|
function decodeWav(bytes: Uint8Array): { samples: Float32Array; sampleRate: number } {
|
||||||
|
const view = new DataView(bytes.buffer, bytes.byteOffset, bytes.byteLength);
|
||||||
|
if (String.fromCharCode(bytes[0], bytes[1], bytes[2], bytes[3]) !== 'RIFF' || String.fromCharCode(bytes[8], bytes[9], bytes[10], bytes[11]) !== 'WAVE') {
|
||||||
|
throw new Error('WAV invalide');
|
||||||
|
}
|
||||||
|
let offset = 12;
|
||||||
|
let sampleRate = 16000;
|
||||||
|
let channels = 1;
|
||||||
|
let bits = 16;
|
||||||
|
let format = 1; // 1 = PCM, 3 = IEEE float
|
||||||
|
let data: { start: number; length: number } | null = null;
|
||||||
|
while (offset + 8 <= bytes.byteLength) {
|
||||||
|
const id = String.fromCharCode(bytes[offset], bytes[offset + 1], bytes[offset + 2], bytes[offset + 3]);
|
||||||
|
const size = view.getUint32(offset + 4, true);
|
||||||
|
if (id === 'fmt ') {
|
||||||
|
format = view.getUint16(offset + 8, true);
|
||||||
|
channels = view.getUint16(offset + 10, true);
|
||||||
|
sampleRate = view.getUint32(offset + 12, true);
|
||||||
|
bits = view.getUint16(offset + 22, true);
|
||||||
|
} else if (id === 'data') {
|
||||||
|
data = { start: offset + 8, length: Math.min(size, bytes.byteLength - offset - 8) };
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
offset += 8 + size + (size % 2);
|
||||||
|
}
|
||||||
|
if (!data) throw new Error('WAV sans données');
|
||||||
|
const bytesPerSample = bits / 8;
|
||||||
|
const frames = Math.floor(data.length / bytesPerSample / channels);
|
||||||
|
const samples = new Float32Array(frames);
|
||||||
|
for (let i = 0; i < frames; i++) {
|
||||||
|
let sum = 0;
|
||||||
|
for (let c = 0; c < channels; c++) {
|
||||||
|
const pos = data.start + (i * channels + c) * bytesPerSample;
|
||||||
|
sum +=
|
||||||
|
format === 3 && bits === 32
|
||||||
|
? view.getFloat32(pos, true)
|
||||||
|
: bits === 16
|
||||||
|
? view.getInt16(pos, true) / 32768
|
||||||
|
: bits === 24
|
||||||
|
? (((bytes[pos] | (bytes[pos + 1] << 8) | (bytes[pos + 2] << 16)) << 8) >> 8) / 8388608
|
||||||
|
: bits === 32
|
||||||
|
? view.getInt32(pos, true) / 2147483648
|
||||||
|
: (bytes[pos] - 128) / 128;
|
||||||
|
}
|
||||||
|
samples[i] = sum / channels;
|
||||||
|
}
|
||||||
|
return { samples, sampleRate };
|
||||||
|
}
|
||||||
|
|
||||||
|
function resampleTo16k(samples: Float32Array, rate: number): Float32Array {
|
||||||
|
if (rate === 16000) return samples;
|
||||||
|
const ratio = rate / 16000;
|
||||||
|
const out = new Float32Array(Math.round(samples.length / ratio));
|
||||||
|
for (let i = 0; i < out.length; i++) {
|
||||||
|
const pos = i * ratio;
|
||||||
|
const idx = Math.floor(pos);
|
||||||
|
const frac = pos - idx;
|
||||||
|
const a = samples[Math.min(idx, samples.length - 1)];
|
||||||
|
const b = samples[Math.min(idx + 1, samples.length - 1)];
|
||||||
|
out[i] = a + (b - a) * frac;
|
||||||
|
}
|
||||||
|
return out;
|
||||||
|
}
|
||||||
|
|
||||||
|
function encodeWav(samples: Float32Array, sampleRate: number): Uint8Array {
|
||||||
|
const buffer = new ArrayBuffer(44 + samples.length * 2);
|
||||||
|
const view = new DataView(buffer);
|
||||||
|
const str = (o: number, s: string) => {
|
||||||
|
for (let i = 0; i < s.length; i++) view.setUint8(o + i, s.charCodeAt(i));
|
||||||
|
};
|
||||||
|
str(0, 'RIFF');
|
||||||
|
view.setUint32(4, 36 + samples.length * 2, true);
|
||||||
|
str(8, 'WAVE');
|
||||||
|
str(12, 'fmt ');
|
||||||
|
view.setUint32(16, 16, true);
|
||||||
|
view.setUint16(20, 1, true);
|
||||||
|
view.setUint16(22, 1, true);
|
||||||
|
view.setUint32(24, sampleRate, true);
|
||||||
|
view.setUint32(28, sampleRate * 2, true);
|
||||||
|
view.setUint16(32, 2, true);
|
||||||
|
view.setUint16(34, 16, true);
|
||||||
|
str(36, 'data');
|
||||||
|
view.setUint32(40, samples.length * 2, true);
|
||||||
|
let o = 44;
|
||||||
|
for (let i = 0; i < samples.length; i++, o += 2) {
|
||||||
|
const s = Math.max(-1, Math.min(1, samples[i]));
|
||||||
|
view.setInt16(o, s < 0 ? s * 0x8000 : s * 0x7fff, true);
|
||||||
|
}
|
||||||
|
return new Uint8Array(buffer);
|
||||||
|
}
|
||||||
|
|
||||||
|
function startKws(model: ModelRef, keywordsFile: string, threshold: number, score: number): void {
|
||||||
|
const s = loadSherpa();
|
||||||
|
kws = null;
|
||||||
|
const p = (f: string) => path.join(model.dir, f);
|
||||||
|
const enc = model.files.find((f) => f.startsWith('encoder')) ?? 'encoder.int8.onnx';
|
||||||
|
const dec = model.files.find((f) => f.startsWith('decoder')) ?? 'decoder.int8.onnx';
|
||||||
|
const join = model.files.find((f) => f.startsWith('joiner')) ?? 'joiner.int8.onnx';
|
||||||
|
const spotter = new s.KeywordSpotter({
|
||||||
|
featConfig: { sampleRate: 16000, featureDim: 80 },
|
||||||
|
modelConfig: { transducer: { encoder: p(enc), decoder: p(dec), joiner: p(join) }, tokens: p('tokens.txt'), numThreads: 1, provider: 'cpu', debug: 0 },
|
||||||
|
maxActivePaths: 4,
|
||||||
|
numTrailingBlanks: 1,
|
||||||
|
keywordsScore: score,
|
||||||
|
keywordsThreshold: threshold,
|
||||||
|
keywordsFile
|
||||||
|
});
|
||||||
|
kws = { spotter, stream: spotter.createStream() };
|
||||||
|
}
|
||||||
|
|
||||||
|
function feedKws(pcmBase64: string, sampleRate: number): void {
|
||||||
|
if (!kws) return;
|
||||||
|
const bytes = Buffer.from(pcmBase64, 'base64');
|
||||||
|
const int16 = new Int16Array(bytes.buffer, bytes.byteOffset, Math.floor(bytes.byteLength / 2));
|
||||||
|
let samples: Float32Array = new Float32Array(int16.length);
|
||||||
|
for (let i = 0; i < int16.length; i++) samples[i] = int16[i] / 32768;
|
||||||
|
if (sampleRate !== 16000) samples = resampleTo16k(samples, sampleRate);
|
||||||
|
kws.stream.acceptWaveform({ sampleRate: 16000, samples });
|
||||||
|
while (kws.spotter.isReady(kws.stream)) {
|
||||||
|
kws.spotter.decode(kws.stream);
|
||||||
|
const result = kws.spotter.getResult(kws.stream);
|
||||||
|
if (result.keyword) {
|
||||||
|
kws.spotter.reset(kws.stream);
|
||||||
|
notify?.({ type: 'kws.detected', keyword: result.keyword, at: Date.now() });
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function startVad(model: ModelRef, silenceMs: number, threshold: number, maxUtteranceSec: number): void {
|
||||||
|
const s = loadSherpa();
|
||||||
|
const windowSize = 512;
|
||||||
|
const detector = new s.Vad(
|
||||||
|
{
|
||||||
|
sileroVad: {
|
||||||
|
model: path.join(model.dir, 'silero_vad.onnx'),
|
||||||
|
threshold: Math.max(0.1, Math.min(0.95, threshold)),
|
||||||
|
minSilenceDuration: Math.max(0.15, silenceMs / 1000),
|
||||||
|
minSpeechDuration: 0.2,
|
||||||
|
windowSize,
|
||||||
|
maxSpeechDuration: Math.max(3, maxUtteranceSec)
|
||||||
|
},
|
||||||
|
sampleRate: 16000,
|
||||||
|
numThreads: 1,
|
||||||
|
provider: 'cpu',
|
||||||
|
debug: 0
|
||||||
|
},
|
||||||
|
Math.max(10, maxUtteranceSec + 5)
|
||||||
|
);
|
||||||
|
vad = { detector, speaking: false, windowSize, carry: new Float32Array(0) };
|
||||||
|
}
|
||||||
|
|
||||||
|
function feedVad(pcmBase64: string, sampleRate: number): void {
|
||||||
|
if (!vad) return;
|
||||||
|
const bytes = Buffer.from(pcmBase64, 'base64');
|
||||||
|
const int16 = new Int16Array(bytes.buffer, bytes.byteOffset, Math.floor(bytes.byteLength / 2));
|
||||||
|
let samples: Float32Array = new Float32Array(int16.length);
|
||||||
|
for (let i = 0; i < int16.length; i++) samples[i] = int16[i] / 32768;
|
||||||
|
if (sampleRate !== 16000) samples = resampleTo16k(samples, sampleRate);
|
||||||
|
// Silero expects fixed windows: keep the remainder for the next frame.
|
||||||
|
const merged = new Float32Array(vad.carry.length + samples.length);
|
||||||
|
merged.set(vad.carry, 0);
|
||||||
|
merged.set(samples, vad.carry.length);
|
||||||
|
const usable = merged.length - (merged.length % vad.windowSize);
|
||||||
|
for (let i = 0; i < usable; i += vad.windowSize) {
|
||||||
|
vad.detector.acceptWaveform(merged.subarray(i, i + vad.windowSize));
|
||||||
|
const detected = vad.detector.isDetected();
|
||||||
|
if (detected && !vad.speaking) {
|
||||||
|
vad.speaking = true;
|
||||||
|
notify?.({ type: 'vad.event', event: { type: 'speech-start' } });
|
||||||
|
}
|
||||||
|
while (!vad.detector.isEmpty()) {
|
||||||
|
const segment = vad.detector.front(false);
|
||||||
|
vad.detector.pop();
|
||||||
|
vad.speaking = false;
|
||||||
|
const wav = encodeWav(segment.samples, 16000);
|
||||||
|
notify?.({
|
||||||
|
type: 'vad.event',
|
||||||
|
event: { type: 'segment', wav: Buffer.from(wav.buffer, wav.byteOffset, wav.byteLength).toString('base64'), durationSec: segment.samples.length / 16000 }
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
vad.carry = merged.slice(usable);
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── request handling ───────────────────────────────────────────────────────
|
||||||
|
function handle(req: Request): unknown {
|
||||||
|
switch (req.type) {
|
||||||
|
case 'vad.start':
|
||||||
|
startVad(req.model, req.silenceMs, req.threshold, req.maxUtteranceSec);
|
||||||
|
return { ok: true };
|
||||||
|
case 'vad.audio':
|
||||||
|
feedVad(req.pcm, req.sampleRate);
|
||||||
|
return { ok: true };
|
||||||
|
case 'vad.stop':
|
||||||
|
vad = null;
|
||||||
|
return { ok: true };
|
||||||
|
case 'kws.start':
|
||||||
|
startKws(req.model, req.keywordsFile, req.threshold, req.score);
|
||||||
|
return { ok: true };
|
||||||
|
case 'kws.audio':
|
||||||
|
feedKws(req.pcm, req.sampleRate);
|
||||||
|
return { ok: true };
|
||||||
|
case 'kws.stop':
|
||||||
|
kws = null;
|
||||||
|
return { ok: true };
|
||||||
|
case 'status': {
|
||||||
|
try {
|
||||||
|
const s = loadSherpa();
|
||||||
|
return { available: true, version: s.version, loaded: [...recognizers.keys(), ...synthesizers.keys(), ...(kws ? ['kws'] : [])] };
|
||||||
|
} catch (err) {
|
||||||
|
return { available: false, error: (err as Error).message, loaded: [] };
|
||||||
|
}
|
||||||
|
}
|
||||||
|
case 'transcribe': {
|
||||||
|
const started = Date.now();
|
||||||
|
// Audio crosses the process boundary as base64: V8 refuses to serialize external buffers.
|
||||||
|
const bytes = typeof req.wav === 'string' ? new Uint8Array(Buffer.from(req.wav, 'base64')) : req.wav;
|
||||||
|
const { samples, sampleRate } = decodeWav(bytes);
|
||||||
|
const pcm = resampleTo16k(samples, sampleRate);
|
||||||
|
if (pcm.length < 1600) throw new Error('Audio trop court');
|
||||||
|
const recognizer = getRecognizer(req.model, req.language);
|
||||||
|
const stream = recognizer.createStream();
|
||||||
|
stream.acceptWaveform({ sampleRate: 16000, samples: pcm });
|
||||||
|
recognizer.decode(stream);
|
||||||
|
const result = recognizer.getResult(stream);
|
||||||
|
return { text: (result.text ?? '').trim(), language: result.lang, durationMs: Date.now() - started, audioSec: pcm.length / 16000 };
|
||||||
|
}
|
||||||
|
case 'synthesize': {
|
||||||
|
const started = Date.now();
|
||||||
|
const tts = getSynthesizer(req.model);
|
||||||
|
const sid = Math.max(0, Math.min(tts.numSpeakers - 1, Math.floor(req.speaker)));
|
||||||
|
// Electron forbids N-API external buffers: ask sherpa-onnx to copy the samples into a V8 buffer.
|
||||||
|
const audio = tts.generate({ text: req.text, sid, speed: Math.max(0.5, Math.min(2, req.speed || 1)), enableExternalBuffer: false });
|
||||||
|
const wav = encodeWav(audio.samples, audio.sampleRate);
|
||||||
|
return {
|
||||||
|
wav: Buffer.from(wav.buffer, wav.byteOffset, wav.byteLength).toString('base64'),
|
||||||
|
sampleRate: audio.sampleRate,
|
||||||
|
durationMs: Date.now() - started,
|
||||||
|
audioSec: audio.samples.length / audio.sampleRate
|
||||||
|
};
|
||||||
|
}
|
||||||
|
case 'unload': {
|
||||||
|
if (req.modelId) {
|
||||||
|
for (const k of [...recognizers.keys()]) if (k.startsWith(`${req.modelId}:`)) recognizers.delete(k);
|
||||||
|
synthesizers.delete(req.modelId);
|
||||||
|
} else {
|
||||||
|
recognizers.clear();
|
||||||
|
synthesizers.clear();
|
||||||
|
kws = null;
|
||||||
|
vad = null;
|
||||||
|
}
|
||||||
|
return { ok: true };
|
||||||
|
}
|
||||||
|
default:
|
||||||
|
throw new Error('requête inconnue');
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function respond(req: Request): Response {
|
||||||
|
try {
|
||||||
|
return { id: req.id, ok: true, result: handle(req) };
|
||||||
|
} catch (err) {
|
||||||
|
return { id: req.id, ok: false, error: (err as Error).message || String(err) };
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Electron utility process transport, with a child_process fallback for tests.
|
||||||
|
const parentPort = (process as unknown as { parentPort?: { on: (ev: 'message', cb: (e: { data: Request }) => void) => void; postMessage: (m: unknown) => void } }).parentPort;
|
||||||
|
if (parentPort) {
|
||||||
|
notify = (m) => parentPort.postMessage(m);
|
||||||
|
parentPort.on('message', (event) => parentPort.postMessage(respond(event.data)));
|
||||||
|
} else if (process.send) {
|
||||||
|
notify = (m) => process.send!(m);
|
||||||
|
process.on('message', (msg: Request) => process.send!(respond(msg)));
|
||||||
|
}
|
||||||
+24
-10
@@ -35,16 +35,20 @@ export function getWebhookStatus(): WebhookStatus {
|
|||||||
export function stopWebhookServer(): Promise<void> {
|
export function stopWebhookServer(): Promise<void> {
|
||||||
return new Promise((resolve) => {
|
return new Promise((resolve) => {
|
||||||
if (!server) return resolve();
|
if (!server) return resolve();
|
||||||
server.close(() => resolve());
|
const current = server;
|
||||||
server = null;
|
server = null;
|
||||||
status = { ...status, listening: false };
|
status = { ...status, listening: false };
|
||||||
|
current.closeAllConnections();
|
||||||
|
current.close(() => resolve());
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
export async function startWebhookServer(getWindow: () => BrowserWindow | null, options: WebhookOptions): Promise<WebhookStatus> {
|
export async function startWebhookServer(getWindow: () => BrowserWindow | null, options: WebhookOptions): Promise<WebhookStatus> {
|
||||||
await stopWebhookServer();
|
await stopWebhookServer();
|
||||||
const { port, secret } = options;
|
const { port, secret } = options;
|
||||||
const host = options.host ?? '0.0.0.0';
|
// Without a shared secret the hook only listens on loopback: anyone on the LAN could otherwise inject messages.
|
||||||
|
const host = options.host ?? (secret ? '0.0.0.0' : '127.0.0.1');
|
||||||
|
if (host !== '127.0.0.1' && !secret) log('WARN', 'webhook', `listening on ${host} without a secret`);
|
||||||
|
|
||||||
server = http.createServer((req, res) => {
|
server = http.createServer((req, res) => {
|
||||||
const json = (code: number, payload: unknown) => {
|
const json = (code: number, payload: unknown) => {
|
||||||
@@ -69,19 +73,24 @@ export async function startWebhookServer(getWindow: () => BrowserWindow | null,
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
let body = '';
|
const chunks: Buffer[] = [];
|
||||||
|
let bytes = 0;
|
||||||
let tooLarge = false;
|
let tooLarge = false;
|
||||||
req.on('data', (chunk: Buffer) => {
|
req.on('data', (chunk: Buffer) => {
|
||||||
if (tooLarge) return;
|
if (tooLarge) return;
|
||||||
body += chunk.toString('utf8');
|
bytes += chunk.length;
|
||||||
if (body.length > MAX_BODY_BYTES) {
|
if (bytes > MAX_BODY_BYTES) {
|
||||||
tooLarge = true;
|
tooLarge = true;
|
||||||
json(413, { error: 'Payload too large' });
|
res.writeHead(413, { 'Content-Type': 'application/json', Connection: 'close' });
|
||||||
req.destroy();
|
res.end(JSON.stringify({ error: 'Payload too large' }), () => req.socket.destroy());
|
||||||
|
return;
|
||||||
}
|
}
|
||||||
|
chunks.push(chunk);
|
||||||
});
|
});
|
||||||
req.on('end', () => {
|
req.on('end', () => {
|
||||||
if (tooLarge) return;
|
if (tooLarge) return;
|
||||||
|
// Concatenate before decoding so multibyte UTF-8 (accents, emoji) split across chunks stays intact.
|
||||||
|
const body = Buffer.concat(chunks).toString('utf8');
|
||||||
try {
|
try {
|
||||||
const raw: unknown = body.trim() ? JSON.parse(body) : {};
|
const raw: unknown = body.trim() ? JSON.parse(body) : {};
|
||||||
const events = normalizeHermesPush(raw);
|
const events = normalizeHermesPush(raw);
|
||||||
@@ -99,13 +108,18 @@ export async function startWebhookServer(getWindow: () => BrowserWindow | null,
|
|||||||
});
|
});
|
||||||
|
|
||||||
return new Promise((resolve) => {
|
return new Promise((resolve) => {
|
||||||
server!.once('error', (err: NodeJS.ErrnoException) => {
|
const instance = server!;
|
||||||
|
instance.on('error', (err: NodeJS.ErrnoException) => {
|
||||||
log('ERROR', 'webhook', `server error: ${err.message}`);
|
log('ERROR', 'webhook', `server error: ${err.message}`);
|
||||||
status = { listening: false, port, path: WEBHOOK_PATH, secretConfigured: !!secret, error: err.message };
|
status = { listening: false, port, path: WEBHOOK_PATH, secretConfigured: !!secret, error: err.message };
|
||||||
server = null;
|
if (server === instance) server = null;
|
||||||
resolve(status);
|
resolve(status);
|
||||||
});
|
});
|
||||||
server!.listen(port, host, () => {
|
if (!Number.isInteger(port) || port < 1 || port > 65535) {
|
||||||
|
instance.emit('error', new Error(`port invalide : ${port}`));
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
instance.listen(port, host, () => {
|
||||||
log('INFO', 'webhook', `listening on http://${host}:${port}${WEBHOOK_PATH}`);
|
log('INFO', 'webhook', `listening on http://${host}:${port}${WEBHOOK_PATH}`);
|
||||||
status = { listening: true, port, path: WEBHOOK_PATH, secretConfigured: !!secret };
|
status = { listening: true, port, path: WEBHOOK_PATH, secretConfigured: !!secret };
|
||||||
resolve(status);
|
resolve(status);
|
||||||
|
|||||||
+19
-6
@@ -1,4 +1,5 @@
|
|||||||
import { app, BrowserWindow, screen, shell } from 'electron';
|
import { app, BrowserWindow, screen, shell } from 'electron';
|
||||||
|
import { EventEmitter } from 'node:events';
|
||||||
import fs from 'node:fs';
|
import fs from 'node:fs';
|
||||||
import path from 'node:path';
|
import path from 'node:path';
|
||||||
import { IPC, type WindowMode } from '../shared/ipc';
|
import { IPC, type WindowMode } from '../shared/ipc';
|
||||||
@@ -9,6 +10,9 @@ const COMPACT_SIZE = { width: 380, height: 620, minWidth: 320, minHeight: 420 };
|
|||||||
|
|
||||||
let mainWindow: BrowserWindow | null = null;
|
let mainWindow: BrowserWindow | null = null;
|
||||||
let currentMode: WindowMode = 'hud';
|
let currentMode: WindowMode = 'hud';
|
||||||
|
export const windowEvents = new EventEmitter();
|
||||||
|
// Transparent frameless windows cannot be resized reliably on Windows; use an opaque, rounded window there.
|
||||||
|
const TRANSPARENT = process.platform !== 'win32';
|
||||||
let hudBounds: Electron.Rectangle | null = null;
|
let hudBounds: Electron.Rectangle | null = null;
|
||||||
|
|
||||||
export function getMainWindow(): BrowserWindow | null {
|
export function getMainWindow(): BrowserWindow | null {
|
||||||
@@ -49,8 +53,9 @@ export function createMainWindow(): BrowserWindow {
|
|||||||
const win = new BrowserWindow({
|
const win = new BrowserWindow({
|
||||||
...HUD_SIZE,
|
...HUD_SIZE,
|
||||||
frame: false,
|
frame: false,
|
||||||
transparent: true,
|
transparent: TRANSPARENT,
|
||||||
backgroundColor: '#00000000',
|
backgroundColor: TRANSPARENT ? '#00000000' : '#03070d',
|
||||||
|
roundedCorners: true,
|
||||||
hasShadow: true,
|
hasShadow: true,
|
||||||
show: false,
|
show: false,
|
||||||
title: 'EveFlow',
|
title: 'EveFlow',
|
||||||
@@ -78,11 +83,18 @@ export function createMainWindow(): BrowserWindow {
|
|||||||
return { action: 'deny' };
|
return { action: 'deny' };
|
||||||
});
|
});
|
||||||
win.webContents.on('will-navigate', (event, url) => {
|
win.webContents.on('will-navigate', (event, url) => {
|
||||||
if (!url.startsWith('file://') && !url.startsWith(process.env.VITE_DEV_SERVER_URL ?? 'http://127.0.0.1:5173')) {
|
if (url === win.webContents.getURL()) return;
|
||||||
event.preventDefault();
|
event.preventDefault();
|
||||||
void shell.openExternal(url);
|
if (/^https?:\/\//i.test(url)) void shell.openExternal(url);
|
||||||
}
|
|
||||||
});
|
});
|
||||||
|
win.webContents.on('will-redirect', (event) => event.preventDefault());
|
||||||
|
const notifyVisibility = () => {
|
||||||
|
if (!win.isDestroyed()) win.webContents.send(IPC.windowVisibility, win.isVisible() && !win.isMinimized());
|
||||||
|
};
|
||||||
|
win.on('show', notifyVisibility);
|
||||||
|
win.on('hide', notifyVisibility);
|
||||||
|
win.on('minimize', notifyVisibility);
|
||||||
|
win.on('restore', notifyVisibility);
|
||||||
|
|
||||||
win.on('closed', () => {
|
win.on('closed', () => {
|
||||||
mainWindow = null;
|
mainWindow = null;
|
||||||
@@ -129,6 +141,7 @@ export function setWindowMode(mode: WindowMode): void {
|
|||||||
}
|
}
|
||||||
currentMode = mode;
|
currentMode = mode;
|
||||||
win.webContents.send(IPC.windowModeChanged, mode);
|
win.webContents.send(IPC.windowModeChanged, mode);
|
||||||
|
windowEvents.emit('mode', mode);
|
||||||
}
|
}
|
||||||
|
|
||||||
export function toggleWindowVisibility(): void {
|
export function toggleWindowVisibility(): void {
|
||||||
|
|||||||
+1
-1
@@ -3,7 +3,7 @@
|
|||||||
<head>
|
<head>
|
||||||
<meta charset="UTF-8" />
|
<meta charset="UTF-8" />
|
||||||
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
|
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
|
||||||
<meta http-equiv="Content-Security-Policy" content="default-src 'self'; script-src 'self'; style-src 'self' 'unsafe-inline' https://fonts.googleapis.com; font-src 'self' https://fonts.gstatic.com data:; img-src 'self' data: blob: http: https:; media-src 'self' blob: data:; connect-src 'self' http: https: ws: wss: blob: data:; worker-src 'self' blob:;" />
|
<meta http-equiv="Content-Security-Policy" content="default-src 'self'; script-src 'self'; style-src 'self' 'unsafe-inline'; font-src 'self' data:; img-src 'self' data: blob: http: https:; media-src 'self' blob: data:; connect-src 'self' http: https: ws: wss: blob: data:; worker-src 'self' blob:;" />
|
||||||
<meta name="description" content="EveFlow — interface vocale JARVIS pour Hermes Agent." />
|
<meta name="description" content="EveFlow — interface vocale JARVIS pour Hermes Agent." />
|
||||||
<link rel="icon" type="image/svg+xml" href="data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 64 64'%3E%3Ccircle cx='32' cy='32' r='26' fill='none' stroke='%2334e4ff' stroke-width='4'/%3E%3Ccircle cx='32' cy='32' r='10' fill='%2334e4ff'/%3E%3C/svg%3E" />
|
<link rel="icon" type="image/svg+xml" href="data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 64 64'%3E%3Ccircle cx='32' cy='32' r='26' fill='none' stroke='%2334e4ff' stroke-width='4'/%3E%3Ccircle cx='32' cy='32' r='10' fill='%2334e4ff'/%3E%3C/svg%3E" />
|
||||||
<title>EveFlow</title>
|
<title>EveFlow</title>
|
||||||
|
|||||||
Generated
+212
-2
@@ -1,25 +1,30 @@
|
|||||||
{
|
{
|
||||||
"name": "eveflow",
|
"name": "eveflow",
|
||||||
"version": "2.0.0",
|
"version": "2.1.0",
|
||||||
"lockfileVersion": 3,
|
"lockfileVersion": 3,
|
||||||
"requires": true,
|
"requires": true,
|
||||||
"packages": {
|
"packages": {
|
||||||
"": {
|
"": {
|
||||||
"name": "eveflow",
|
"name": "eveflow",
|
||||||
"version": "2.0.0",
|
"version": "2.1.0",
|
||||||
"license": "MIT",
|
"license": "MIT",
|
||||||
"dependencies": {
|
"dependencies": {
|
||||||
|
"@fontsource/orbitron": "^5.3.0",
|
||||||
|
"@fontsource/rajdhani": "^5.3.0",
|
||||||
|
"@fontsource/share-tech-mono": "^5.3.0",
|
||||||
"lucide-react": "^1.40.0",
|
"lucide-react": "^1.40.0",
|
||||||
"react": "^19.2.8",
|
"react": "^19.2.8",
|
||||||
"react-dom": "^19.2.8",
|
"react-dom": "^19.2.8",
|
||||||
"react-markdown": "^10.1.0",
|
"react-markdown": "^10.1.0",
|
||||||
"remark-gfm": "^4.0.1",
|
"remark-gfm": "^4.0.1",
|
||||||
|
"sherpa-onnx-node": "^1.13.7",
|
||||||
"zustand": "^5.0.15"
|
"zustand": "^5.0.15"
|
||||||
},
|
},
|
||||||
"devDependencies": {
|
"devDependencies": {
|
||||||
"@types/node": "^24.0.0",
|
"@types/node": "^24.0.0",
|
||||||
"@types/react": "^19.2.18",
|
"@types/react": "^19.2.18",
|
||||||
"@types/react-dom": "^19.2.0",
|
"@types/react-dom": "^19.2.0",
|
||||||
|
"@types/unbzip2-stream": "^1.4.3",
|
||||||
"@vitejs/plugin-react": "^6.1.1",
|
"@vitejs/plugin-react": "^6.1.1",
|
||||||
"concurrently": "^10.0.5",
|
"concurrently": "^10.0.5",
|
||||||
"cross-env": "^10.0.0",
|
"cross-env": "^10.0.0",
|
||||||
@@ -27,7 +32,9 @@
|
|||||||
"electron-builder": "^26.15.3",
|
"electron-builder": "^26.15.3",
|
||||||
"esbuild": "^0.28.2",
|
"esbuild": "^0.28.2",
|
||||||
"jsdom": "^30.0.1",
|
"jsdom": "^30.0.1",
|
||||||
|
"tar": "^7.5.22",
|
||||||
"typescript": "^5.9.3",
|
"typescript": "^5.9.3",
|
||||||
|
"unbzip2-stream": "^1.4.3",
|
||||||
"vite": "^8.2.2",
|
"vite": "^8.2.2",
|
||||||
"vitest": "^5.0.0",
|
"vitest": "^5.0.0",
|
||||||
"wait-on": "^9.1.0"
|
"wait-on": "^9.1.0"
|
||||||
@@ -1056,6 +1063,33 @@
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
|
"node_modules/@fontsource/orbitron": {
|
||||||
|
"version": "5.3.0",
|
||||||
|
"resolved": "https://registry.npmjs.org/@fontsource/orbitron/-/orbitron-5.3.0.tgz",
|
||||||
|
"integrity": "sha512-+BhPsbUAo/AF7sRcf+TEexm+EWWkgTiR9Zwb7AhYIPkc5oplfvw0C45Oc2xlKPhwVEmxG8hHuA6/Ansbl+Bc6w==",
|
||||||
|
"license": "OFL-1.1",
|
||||||
|
"funding": {
|
||||||
|
"url": "https://github.com/sponsors/ayuhito"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"node_modules/@fontsource/rajdhani": {
|
||||||
|
"version": "5.3.0",
|
||||||
|
"resolved": "https://registry.npmjs.org/@fontsource/rajdhani/-/rajdhani-5.3.0.tgz",
|
||||||
|
"integrity": "sha512-oBJgSIibWJvVBKseme1JczqPLoDHN97LHti4uh0LjfcJqQoVmnQQlRKIpEjTJV559rmVQ6qbVZZwKwk2wkBTtw==",
|
||||||
|
"license": "OFL-1.1",
|
||||||
|
"funding": {
|
||||||
|
"url": "https://github.com/sponsors/ayuhito"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"node_modules/@fontsource/share-tech-mono": {
|
||||||
|
"version": "5.3.0",
|
||||||
|
"resolved": "https://registry.npmjs.org/@fontsource/share-tech-mono/-/share-tech-mono-5.3.0.tgz",
|
||||||
|
"integrity": "sha512-hVEzVCzEid/fBfhggMyvkfDzn9ODrMihxC0EiOeQK3BuQNvdjmith2ERnoDNUkbW8zhufbKNuCgaSoT8LUNpuQ==",
|
||||||
|
"license": "OFL-1.1",
|
||||||
|
"funding": {
|
||||||
|
"url": "https://github.com/sponsors/ayuhito"
|
||||||
|
}
|
||||||
|
},
|
||||||
"node_modules/@hapi/address": {
|
"node_modules/@hapi/address": {
|
||||||
"version": "5.1.1",
|
"version": "5.1.1",
|
||||||
"resolved": "https://registry.npmjs.org/@hapi/address/-/address-5.1.1.tgz",
|
"resolved": "https://registry.npmjs.org/@hapi/address/-/address-5.1.1.tgz",
|
||||||
@@ -1724,6 +1758,26 @@
|
|||||||
"@types/node": "*"
|
"@types/node": "*"
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
|
"node_modules/@types/through": {
|
||||||
|
"version": "0.0.33",
|
||||||
|
"resolved": "https://registry.npmjs.org/@types/through/-/through-0.0.33.tgz",
|
||||||
|
"integrity": "sha512-HsJ+z3QuETzP3cswwtzt2vEIiHBk/dCcHGhbmG5X3ecnwFD/lPrMpliGXxSCg03L9AhrdwA4Oz/qfspkDW+xGQ==",
|
||||||
|
"dev": true,
|
||||||
|
"license": "MIT",
|
||||||
|
"dependencies": {
|
||||||
|
"@types/node": "*"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"node_modules/@types/unbzip2-stream": {
|
||||||
|
"version": "1.4.3",
|
||||||
|
"resolved": "https://registry.npmjs.org/@types/unbzip2-stream/-/unbzip2-stream-1.4.3.tgz",
|
||||||
|
"integrity": "sha512-D8X5uuJRISqc8YtwL8jNW2FpPdUOCYXbfD6zNROCTbVXK9nawucxh10tVXE3MPjnHdRA1LvB0zDxVya/lBsnYw==",
|
||||||
|
"dev": true,
|
||||||
|
"license": "MIT",
|
||||||
|
"dependencies": {
|
||||||
|
"@types/through": "*"
|
||||||
|
}
|
||||||
|
},
|
||||||
"node_modules/@types/unist": {
|
"node_modules/@types/unist": {
|
||||||
"version": "3.0.3",
|
"version": "3.0.3",
|
||||||
"resolved": "https://registry.npmjs.org/@types/unist/-/unist-3.0.3.tgz",
|
"resolved": "https://registry.npmjs.org/@types/unist/-/unist-3.0.3.tgz",
|
||||||
@@ -2259,6 +2313,31 @@
|
|||||||
"node": "20 || >=22"
|
"node": "20 || >=22"
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
|
"node_modules/buffer": {
|
||||||
|
"version": "5.7.1",
|
||||||
|
"resolved": "https://registry.npmjs.org/buffer/-/buffer-5.7.1.tgz",
|
||||||
|
"integrity": "sha512-EHcyIPBQ4BSGlvjB16k5KgAJ27CIsHY/2JBmCRReo48y9rQ3MaUzWX3KVlBa4U7MyX02HdVj0K7C3WaB3ju7FQ==",
|
||||||
|
"dev": true,
|
||||||
|
"funding": [
|
||||||
|
{
|
||||||
|
"type": "github",
|
||||||
|
"url": "https://github.com/sponsors/feross"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"type": "patreon",
|
||||||
|
"url": "https://www.patreon.com/feross"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"type": "consulting",
|
||||||
|
"url": "https://feross.org/support"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"license": "MIT",
|
||||||
|
"dependencies": {
|
||||||
|
"base64-js": "^1.3.1",
|
||||||
|
"ieee754": "^1.1.13"
|
||||||
|
}
|
||||||
|
},
|
||||||
"node_modules/buffer-from": {
|
"node_modules/buffer-from": {
|
||||||
"version": "1.1.2",
|
"version": "1.1.2",
|
||||||
"resolved": "https://registry.npmjs.org/buffer-from/-/buffer-from-1.1.2.tgz",
|
"resolved": "https://registry.npmjs.org/buffer-from/-/buffer-from-1.1.2.tgz",
|
||||||
@@ -4149,6 +4228,27 @@
|
|||||||
"node": ">= 14"
|
"node": ">= 14"
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
|
"node_modules/ieee754": {
|
||||||
|
"version": "1.2.1",
|
||||||
|
"resolved": "https://registry.npmjs.org/ieee754/-/ieee754-1.2.1.tgz",
|
||||||
|
"integrity": "sha512-dcyqhDvX1C46lXZcVqCpK+FtMRQVdIMN6/Df5js2zouUsqG7I6sFxitIC+7KYK29KdXOLHdu9zL4sFnoVQnqaA==",
|
||||||
|
"dev": true,
|
||||||
|
"funding": [
|
||||||
|
{
|
||||||
|
"type": "github",
|
||||||
|
"url": "https://github.com/sponsors/feross"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"type": "patreon",
|
||||||
|
"url": "https://www.patreon.com/feross"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"type": "consulting",
|
||||||
|
"url": "https://feross.org/support"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"license": "BSD-3-Clause"
|
||||||
|
},
|
||||||
"node_modules/inflight": {
|
"node_modules/inflight": {
|
||||||
"version": "1.0.6",
|
"version": "1.0.6",
|
||||||
"resolved": "https://registry.npmjs.org/inflight/-/inflight-1.0.6.tgz",
|
"resolved": "https://registry.npmjs.org/inflight/-/inflight-1.0.6.tgz",
|
||||||
@@ -6737,6 +6837,98 @@
|
|||||||
"url": "https://github.com/sponsors/ljharb"
|
"url": "https://github.com/sponsors/ljharb"
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
|
"node_modules/sherpa-onnx-darwin-arm64": {
|
||||||
|
"version": "1.13.7",
|
||||||
|
"resolved": "https://registry.npmjs.org/sherpa-onnx-darwin-arm64/-/sherpa-onnx-darwin-arm64-1.13.7.tgz",
|
||||||
|
"integrity": "sha512-5NCE50hAvr3n2pdett0SgfPBJXaFZE0bqHwbHyiq+IKZ8Ids0l4M0VrG+ImGYIafCwie+oC3uAJ+pKj9xg/k+w==",
|
||||||
|
"cpu": [
|
||||||
|
"arm64"
|
||||||
|
],
|
||||||
|
"license": "Apache-2.0",
|
||||||
|
"optional": true,
|
||||||
|
"os": [
|
||||||
|
"darwin"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"node_modules/sherpa-onnx-darwin-x64": {
|
||||||
|
"version": "1.13.7",
|
||||||
|
"resolved": "https://registry.npmjs.org/sherpa-onnx-darwin-x64/-/sherpa-onnx-darwin-x64-1.13.7.tgz",
|
||||||
|
"integrity": "sha512-N3o+T+wn9WaQmsKV5DD8bTHdo+WN2+sXwmZcGJZiDjtOMR2zFz7uVCZnYCmEAMgvChC+oHcF5RvEEKcRCAu6Pw==",
|
||||||
|
"cpu": [
|
||||||
|
"x64"
|
||||||
|
],
|
||||||
|
"license": "Apache-2.0",
|
||||||
|
"optional": true,
|
||||||
|
"os": [
|
||||||
|
"darwin"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"node_modules/sherpa-onnx-linux-arm64": {
|
||||||
|
"version": "1.13.7",
|
||||||
|
"resolved": "https://registry.npmjs.org/sherpa-onnx-linux-arm64/-/sherpa-onnx-linux-arm64-1.13.7.tgz",
|
||||||
|
"integrity": "sha512-TFCVpXyTh69buhOtTS8KIfkRXOVKY4Y1qjAktSItrKS4A0chnnrlXO5bKWoNAPeI6fMxTF/uvMYbYgcvjEMfNg==",
|
||||||
|
"cpu": [
|
||||||
|
"arm64"
|
||||||
|
],
|
||||||
|
"license": "Apache-2.0",
|
||||||
|
"optional": true,
|
||||||
|
"os": [
|
||||||
|
"linux"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"node_modules/sherpa-onnx-linux-x64": {
|
||||||
|
"version": "1.13.7",
|
||||||
|
"resolved": "https://registry.npmjs.org/sherpa-onnx-linux-x64/-/sherpa-onnx-linux-x64-1.13.7.tgz",
|
||||||
|
"integrity": "sha512-npmxn5WwmAmlthgBhmbZ33t3i2j4mJwQt46dMEb3j7d41y1/uJrjrVAfa/DkvV+vn49ZWfcQ2UEWDipaZBVhuw==",
|
||||||
|
"cpu": [
|
||||||
|
"x64"
|
||||||
|
],
|
||||||
|
"license": "Apache-2.0",
|
||||||
|
"optional": true,
|
||||||
|
"os": [
|
||||||
|
"linux"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"node_modules/sherpa-onnx-node": {
|
||||||
|
"version": "1.13.7",
|
||||||
|
"resolved": "https://registry.npmjs.org/sherpa-onnx-node/-/sherpa-onnx-node-1.13.7.tgz",
|
||||||
|
"integrity": "sha512-0XGV7arGngBCnol0m8OLyqlnaUm19Q1KmetVj1DDBdymXa1upmAHZDwNdN47gjsEhqE5hXUEyc1vRQoXrNhNVg==",
|
||||||
|
"license": "Apache-2.0",
|
||||||
|
"optionalDependencies": {
|
||||||
|
"sherpa-onnx-darwin-arm64": "^1.13.7",
|
||||||
|
"sherpa-onnx-darwin-x64": "^1.13.7",
|
||||||
|
"sherpa-onnx-linux-arm64": "^1.13.7",
|
||||||
|
"sherpa-onnx-linux-x64": "^1.13.7",
|
||||||
|
"sherpa-onnx-win-ia32": "^1.13.7",
|
||||||
|
"sherpa-onnx-win-x64": "^1.13.7"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"node_modules/sherpa-onnx-win-ia32": {
|
||||||
|
"version": "1.13.7",
|
||||||
|
"resolved": "https://registry.npmjs.org/sherpa-onnx-win-ia32/-/sherpa-onnx-win-ia32-1.13.7.tgz",
|
||||||
|
"integrity": "sha512-sTwtpxPQ76XLn0giAbvknIDEDKD3XXi2mo2AVROEucf1pIK1DjQl+LjLkalTeFoQqbC4J3xGx/g+xgcHQD1dsw==",
|
||||||
|
"cpu": [
|
||||||
|
"ia32"
|
||||||
|
],
|
||||||
|
"license": "Apache-2.0",
|
||||||
|
"optional": true,
|
||||||
|
"os": [
|
||||||
|
"win32"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"node_modules/sherpa-onnx-win-x64": {
|
||||||
|
"version": "1.13.7",
|
||||||
|
"resolved": "https://registry.npmjs.org/sherpa-onnx-win-x64/-/sherpa-onnx-win-x64-1.13.7.tgz",
|
||||||
|
"integrity": "sha512-wBV1o+/zgsMrOjfCFIgGrH6S28xq6CqRCLSavCOjTZ6cqr80yGc07DUHxqsHFPZvfoJU+2JF5L2l3gyWFWoWdQ==",
|
||||||
|
"cpu": [
|
||||||
|
"x64"
|
||||||
|
],
|
||||||
|
"license": "Apache-2.0",
|
||||||
|
"optional": true,
|
||||||
|
"os": [
|
||||||
|
"win32"
|
||||||
|
]
|
||||||
|
},
|
||||||
"node_modules/siginfo": {
|
"node_modules/siginfo": {
|
||||||
"version": "2.0.0",
|
"version": "2.0.0",
|
||||||
"resolved": "https://registry.npmjs.org/siginfo/-/siginfo-2.0.0.tgz",
|
"resolved": "https://registry.npmjs.org/siginfo/-/siginfo-2.0.0.tgz",
|
||||||
@@ -6999,6 +7191,13 @@
|
|||||||
"fs-extra": "^10.0.0"
|
"fs-extra": "^10.0.0"
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
|
"node_modules/through": {
|
||||||
|
"version": "2.3.8",
|
||||||
|
"resolved": "https://registry.npmjs.org/through/-/through-2.3.8.tgz",
|
||||||
|
"integrity": "sha512-w89qg7PI8wAdvX60bMDP+bFoD5Dvhm9oLheFp5O4a2QF0cSBGsBX4qZmadPMvVqlLJBBci+WqGGOAPvcDeNSVg==",
|
||||||
|
"dev": true,
|
||||||
|
"license": "MIT"
|
||||||
|
},
|
||||||
"node_modules/tiny-async-pool": {
|
"node_modules/tiny-async-pool": {
|
||||||
"version": "1.3.0",
|
"version": "1.3.0",
|
||||||
"resolved": "https://registry.npmjs.org/tiny-async-pool/-/tiny-async-pool-1.3.0.tgz",
|
"resolved": "https://registry.npmjs.org/tiny-async-pool/-/tiny-async-pool-1.3.0.tgz",
|
||||||
@@ -7197,6 +7396,17 @@
|
|||||||
"node": ">=14.17"
|
"node": ">=14.17"
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
|
"node_modules/unbzip2-stream": {
|
||||||
|
"version": "1.4.3",
|
||||||
|
"resolved": "https://registry.npmjs.org/unbzip2-stream/-/unbzip2-stream-1.4.3.tgz",
|
||||||
|
"integrity": "sha512-mlExGW4w71ebDJviH16lQLtZS32VKqsSfk80GCfUlwT/4/hNRFsoscrF/c++9xinkMzECL1uL9DDwXqFWkruPg==",
|
||||||
|
"dev": true,
|
||||||
|
"license": "MIT",
|
||||||
|
"dependencies": {
|
||||||
|
"buffer": "^5.2.1",
|
||||||
|
"through": "^2.3.8"
|
||||||
|
}
|
||||||
|
},
|
||||||
"node_modules/undici": {
|
"node_modules/undici": {
|
||||||
"version": "7.29.0",
|
"version": "7.29.0",
|
||||||
"resolved": "https://registry.npmjs.org/undici/-/undici-7.29.0.tgz",
|
"resolved": "https://registry.npmjs.org/undici/-/undici-7.29.0.tgz",
|
||||||
|
|||||||
+20
-3
@@ -1,6 +1,7 @@
|
|||||||
{
|
{
|
||||||
"name": "eveflow",
|
"name": "eveflow",
|
||||||
"version": "2.0.0.1",
|
"version": "2.3.0",
|
||||||
|
"releaseVersion": "2.3.0",
|
||||||
"description": "JARVIS-style desktop HUD for Hermes Agent: voice, streaming runs, scheduled jobs, skills and telemetry",
|
"description": "JARVIS-style desktop HUD for Hermes Agent: voice, streaming runs, scheduled jobs, skills and telemetry",
|
||||||
"main": "dist-electron/main.js",
|
"main": "dist-electron/main.js",
|
||||||
"private": true,
|
"private": true,
|
||||||
@@ -16,7 +17,7 @@
|
|||||||
"electron": "electron .",
|
"electron": "electron .",
|
||||||
"start": "npm run build:electron && concurrently -k -s first \"npm run dev\" \"wait-on http://127.0.0.1:5173 && cross-env NODE_ENV=development VITE_DEV_SERVER_URL=http://127.0.0.1:5173 electron .\"",
|
"start": "npm run build:electron && concurrently -k -s first \"npm run dev\" \"wait-on http://127.0.0.1:5173 && cross-env NODE_ENV=development VITE_DEV_SERVER_URL=http://127.0.0.1:5173 electron .\"",
|
||||||
"preview": "vite preview",
|
"preview": "vite preview",
|
||||||
"dist": "npm run build && electron-builder --win --publish never",
|
"dist": "node scripts/dist.mjs",
|
||||||
"typecheck:electron": "tsc -p tsconfig.electron.json --noEmit"
|
"typecheck:electron": "tsc -p tsconfig.electron.json --noEmit"
|
||||||
},
|
},
|
||||||
"build": {
|
"build": {
|
||||||
@@ -52,20 +53,34 @@
|
|||||||
"from": "build/icon.png",
|
"from": "build/icon.png",
|
||||||
"to": "icon.png"
|
"to": "icon.png"
|
||||||
}
|
}
|
||||||
]
|
],
|
||||||
|
"asarUnpack": [
|
||||||
|
"dist-electron/voice-worker.js",
|
||||||
|
"node_modules/sherpa-onnx-node/**",
|
||||||
|
"node_modules/sherpa-onnx-win-x64/**",
|
||||||
|
"node_modules/sherpa-onnx-linux-x64/**",
|
||||||
|
"node_modules/sherpa-onnx-darwin-arm64/**",
|
||||||
|
"node_modules/sherpa-onnx-darwin-x64/**"
|
||||||
|
],
|
||||||
|
"npmRebuild": false
|
||||||
},
|
},
|
||||||
"dependencies": {
|
"dependencies": {
|
||||||
|
"@fontsource/orbitron": "^5.3.0",
|
||||||
|
"@fontsource/rajdhani": "^5.3.0",
|
||||||
|
"@fontsource/share-tech-mono": "^5.3.0",
|
||||||
"lucide-react": "^1.40.0",
|
"lucide-react": "^1.40.0",
|
||||||
"react": "^19.2.8",
|
"react": "^19.2.8",
|
||||||
"react-dom": "^19.2.8",
|
"react-dom": "^19.2.8",
|
||||||
"react-markdown": "^10.1.0",
|
"react-markdown": "^10.1.0",
|
||||||
"remark-gfm": "^4.0.1",
|
"remark-gfm": "^4.0.1",
|
||||||
|
"sherpa-onnx-node": "^1.13.7",
|
||||||
"zustand": "^5.0.15"
|
"zustand": "^5.0.15"
|
||||||
},
|
},
|
||||||
"devDependencies": {
|
"devDependencies": {
|
||||||
"@types/node": "^24.0.0",
|
"@types/node": "^24.0.0",
|
||||||
"@types/react": "^19.2.18",
|
"@types/react": "^19.2.18",
|
||||||
"@types/react-dom": "^19.2.0",
|
"@types/react-dom": "^19.2.0",
|
||||||
|
"@types/unbzip2-stream": "^1.4.3",
|
||||||
"@vitejs/plugin-react": "^6.1.1",
|
"@vitejs/plugin-react": "^6.1.1",
|
||||||
"concurrently": "^10.0.5",
|
"concurrently": "^10.0.5",
|
||||||
"cross-env": "^10.0.0",
|
"cross-env": "^10.0.0",
|
||||||
@@ -73,7 +88,9 @@
|
|||||||
"electron-builder": "^26.15.3",
|
"electron-builder": "^26.15.3",
|
||||||
"esbuild": "^0.28.2",
|
"esbuild": "^0.28.2",
|
||||||
"jsdom": "^30.0.1",
|
"jsdom": "^30.0.1",
|
||||||
|
"tar": "^7.5.22",
|
||||||
"typescript": "^5.9.3",
|
"typescript": "^5.9.3",
|
||||||
|
"unbzip2-stream": "^1.4.3",
|
||||||
"vite": "^8.2.2",
|
"vite": "^8.2.2",
|
||||||
"vitest": "^5.0.0",
|
"vitest": "^5.0.0",
|
||||||
"wait-on": "^9.1.0"
|
"wait-on": "^9.1.0"
|
||||||
|
|||||||
@@ -7,13 +7,18 @@ import { rmSync } from 'node:fs';
|
|||||||
rmSync('dist-electron', { recursive: true, force: true });
|
rmSync('dist-electron', { recursive: true, force: true });
|
||||||
|
|
||||||
await build({
|
await build({
|
||||||
entryPoints: ['electron/main.ts', 'electron/preload.ts'],
|
entryPoints: {
|
||||||
|
main: 'electron/main.ts',
|
||||||
|
preload: 'electron/preload.ts',
|
||||||
|
'voice-worker': 'electron/voice/worker.ts'
|
||||||
|
},
|
||||||
outdir: 'dist-electron',
|
outdir: 'dist-electron',
|
||||||
bundle: true,
|
bundle: true,
|
||||||
platform: 'node',
|
platform: 'node',
|
||||||
format: 'cjs',
|
format: 'cjs',
|
||||||
target: 'node22',
|
target: 'node22',
|
||||||
external: ['electron'],
|
// sherpa-onnx-node stays external: it is a native addon shipped unpacked from the asar.
|
||||||
|
external: ['electron', 'sherpa-onnx-node'],
|
||||||
sourcemap: false,
|
sourcemap: false,
|
||||||
logLevel: 'info'
|
logLevel: 'info'
|
||||||
});
|
});
|
||||||
@@ -0,0 +1,21 @@
|
|||||||
|
// Packages the app with electron-builder using the four-part release version
|
||||||
|
// (package.json "releaseVersion", e.g. 2.0.0.1). package.json "version" must stay semver.
|
||||||
|
import { spawnSync } from 'node:child_process';
|
||||||
|
import { existsSync, readFileSync } from 'node:fs';
|
||||||
|
|
||||||
|
const pkg = JSON.parse(readFileSync('package.json', 'utf8'));
|
||||||
|
const release = pkg.releaseVersion || pkg.version;
|
||||||
|
const build = release.split('.')[3] ?? '0';
|
||||||
|
const args = process.argv.slice(2);
|
||||||
|
if (!args.some((a) => a.startsWith('--win') || a.startsWith('--linux') || a.startsWith('--mac'))) args.unshift('--win');
|
||||||
|
// The Windows build must embed the Windows native addon (an optional dependency installed only on win32).
|
||||||
|
if (args.some((a) => a.startsWith('--win')) && !existsSync('node_modules/sherpa-onnx-win-x64/sherpa-onnx.node')) {
|
||||||
|
console.error('sherpa-onnx-win-x64 is not installed: build the Windows installer on Windows (npm ci) or install it with\n npm i --no-save --os=win32 --cpu=x64 sherpa-onnx-win-x64');
|
||||||
|
process.exit(1);
|
||||||
|
}
|
||||||
|
const result = spawnSync('npx', ['electron-builder', ...args, `--config.extraMetadata.version=${release}`], {
|
||||||
|
stdio: 'inherit',
|
||||||
|
shell: process.platform === 'win32',
|
||||||
|
env: { ...process.env, BUILD_NUMBER: process.env.BUILD_NUMBER ?? build }
|
||||||
|
});
|
||||||
|
process.exit(result.status ?? 1);
|
||||||
+58
-4
@@ -11,6 +11,19 @@ import type {
|
|||||||
WebhookStatus,
|
WebhookStatus,
|
||||||
WindowMode
|
WindowMode
|
||||||
} from './ipc';
|
} from './ipc';
|
||||||
|
import type {
|
||||||
|
KwsDetection,
|
||||||
|
KwsStartRequest,
|
||||||
|
VadEvent,
|
||||||
|
VadStartRequest,
|
||||||
|
SynthesizeRequest,
|
||||||
|
SynthesizeResult,
|
||||||
|
TranscribeRequest,
|
||||||
|
TranscribeResult,
|
||||||
|
VoiceDownloadProgress,
|
||||||
|
VoiceEngineStatus,
|
||||||
|
VoiceModelStatus
|
||||||
|
} from './voice';
|
||||||
|
|
||||||
export type Unsubscribe = () => void;
|
export type Unsubscribe = () => void;
|
||||||
|
|
||||||
@@ -20,6 +33,8 @@ export interface EveFlowBridge {
|
|||||||
control: (action: 'minimize' | 'maximize' | 'close' | 'hide') => void;
|
control: (action: 'minimize' | 'maximize' | 'close' | 'hide') => void;
|
||||||
setMode: (mode: WindowMode) => void;
|
setMode: (mode: WindowMode) => void;
|
||||||
onModeChanged: (cb: (mode: WindowMode) => void) => Unsubscribe;
|
onModeChanged: (cb: (mode: WindowMode) => void) => Unsubscribe;
|
||||||
|
/** Fires when the window is shown/hidden/minimised (the Page Visibility API is disabled by backgroundThrottling:false). */
|
||||||
|
onVisibility: (cb: (visible: boolean) => void) => Unsubscribe;
|
||||||
};
|
};
|
||||||
log: (entry: LogEntry) => void;
|
log: (entry: LogEntry) => void;
|
||||||
store: {
|
store: {
|
||||||
@@ -33,10 +48,6 @@ export interface EveFlowBridge {
|
|||||||
streamAbort: (id: string) => void;
|
streamAbort: (id: string) => void;
|
||||||
onStreamEvent: (cb: (event: HttpStreamEvent) => void) => Unsubscribe;
|
onStreamEvent: (cb: (event: HttpStreamEvent) => void) => Unsubscribe;
|
||||||
};
|
};
|
||||||
system: {
|
|
||||||
metrics: () => Promise<SystemMetrics>;
|
|
||||||
appInfo: () => Promise<AppInfo>;
|
|
||||||
};
|
|
||||||
files: {
|
files: {
|
||||||
readLocal: (filePath: string) => Promise<string>;
|
readLocal: (filePath: string) => Promise<string>;
|
||||||
writeShared: (filename: string, content: string, isBase64?: boolean) => Promise<{ path: string; url: string }>;
|
writeShared: (filename: string, content: string, isBase64?: boolean) => Promise<{ path: string; url: string }>;
|
||||||
@@ -51,4 +62,47 @@ export interface EveFlowBridge {
|
|||||||
hotkeys: {
|
hotkeys: {
|
||||||
on: (cb: (event: HotkeyEvent) => void) => Unsubscribe;
|
on: (cb: (event: HotkeyEvent) => void) => Unsubscribe;
|
||||||
};
|
};
|
||||||
|
voice: {
|
||||||
|
status: () => Promise<VoiceEngineStatus>;
|
||||||
|
listModels: () => Promise<VoiceModelStatus[]>;
|
||||||
|
downloadModel: (id: string) => Promise<VoiceModelStatus>;
|
||||||
|
cancelDownload: (id: string) => Promise<boolean>;
|
||||||
|
removeModel: (id: string) => Promise<VoiceModelStatus[]>;
|
||||||
|
onProgress: (cb: (progress: VoiceDownloadProgress) => void) => Unsubscribe;
|
||||||
|
transcribe: (req: TranscribeRequest) => Promise<TranscribeResult>;
|
||||||
|
synthesize: (req: SynthesizeRequest) => Promise<SynthesizeResult>;
|
||||||
|
unload: (id?: string) => Promise<unknown>;
|
||||||
|
kwsStart: (req: KwsStartRequest) => Promise<{ accepted: string[]; rejected: string[] }>;
|
||||||
|
kwsStop: () => Promise<void>;
|
||||||
|
/** Fire-and-forget 16-bit PCM frames for the keyword spotter. */
|
||||||
|
kwsAudio: (pcm: Uint8Array, sampleRate: number) => void;
|
||||||
|
onKwsDetected: (cb: (detection: KwsDetection) => void) => Unsubscribe;
|
||||||
|
vadStart: (req: VadStartRequest) => Promise<void>;
|
||||||
|
vadStop: () => Promise<void>;
|
||||||
|
vadAudio: (pcm: Uint8Array, sampleRate: number) => void;
|
||||||
|
onVadEvent: (cb: (event: VadEvent) => void) => Unsubscribe;
|
||||||
|
};
|
||||||
|
system: {
|
||||||
|
metrics: () => Promise<SystemMetrics>;
|
||||||
|
appInfo: () => Promise<AppInfo>;
|
||||||
|
/** Screenshot of the primary display as a JPEG data URL. */
|
||||||
|
captureScreen: (maxWidth?: number) => Promise<string>;
|
||||||
|
/** Allow-listed local actions (lock session, open app/url, media keys, clipboard). */
|
||||||
|
action: (action: SystemAction) => Promise<SystemActionResult>;
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
export type SystemAction =
|
||||||
|
| { type: 'lock' }
|
||||||
|
| { type: 'open-app'; name: string }
|
||||||
|
| { type: 'open-url'; url: string }
|
||||||
|
| { type: 'media'; key: 'volume-up' | 'volume-down' | 'mute' | 'play-pause' | 'next' | 'previous' }
|
||||||
|
| { type: 'clipboard-read' }
|
||||||
|
| { type: 'clipboard-write'; text: string }
|
||||||
|
| { type: 'find-files'; query: string };
|
||||||
|
|
||||||
|
export interface SystemActionResult {
|
||||||
|
ok: boolean;
|
||||||
|
message?: string;
|
||||||
|
data?: unknown;
|
||||||
}
|
}
|
||||||
@@ -100,6 +100,7 @@ export const IPC = {
|
|||||||
windowControl: 'window:control',
|
windowControl: 'window:control',
|
||||||
windowSetMode: 'window:set-mode',
|
windowSetMode: 'window:set-mode',
|
||||||
windowModeChanged: 'window:mode-changed',
|
windowModeChanged: 'window:mode-changed',
|
||||||
|
windowVisibility: 'window:visibility',
|
||||||
log: 'log:write',
|
log: 'log:write',
|
||||||
storeGet: 'store:get',
|
storeGet: 'store:get',
|
||||||
storeSet: 'store:set',
|
storeSet: 'store:set',
|
||||||
@@ -110,6 +111,8 @@ export const IPC = {
|
|||||||
httpStreamEvent: 'http:stream:event',
|
httpStreamEvent: 'http:stream:event',
|
||||||
metrics: 'system:metrics',
|
metrics: 'system:metrics',
|
||||||
appInfo: 'app:info',
|
appInfo: 'app:info',
|
||||||
|
screenCapture: 'system:screen-capture',
|
||||||
|
systemAction: 'system:action',
|
||||||
readLocalFile: 'files:read-local',
|
readLocalFile: 'files:read-local',
|
||||||
writeSharedFile: 'files:write-shared',
|
writeSharedFile: 'files:write-shared',
|
||||||
openPath: 'files:open-path',
|
openPath: 'files:open-path',
|
||||||
|
|||||||
@@ -0,0 +1,108 @@
|
|||||||
|
/**
|
||||||
|
* Encodes wake phrases into the BPE token sequences expected by the sherpa-onnx keyword
|
||||||
|
* spotter (gigaspeech BPE-500 model). Common phrases use sequences produced by the real
|
||||||
|
* SentencePiece model; anything else falls back to a greedy longest-match over the vocabulary,
|
||||||
|
* which is a close approximation for short words.
|
||||||
|
*/
|
||||||
|
|
||||||
|
const KNOWN: Record<string, string> = {
|
||||||
|
'JARVIS': '▁JA R VI S',
|
||||||
|
'HEY JARVIS': '▁HE Y ▁JA R VI S',
|
||||||
|
'OK JARVIS': '▁O K ▁JA R VI S',
|
||||||
|
'EVE': '▁E VE',
|
||||||
|
'HEY EVE': '▁HE Y ▁E VE',
|
||||||
|
'COMPUTER': '▁COMP U TER',
|
||||||
|
'HEY COMPUTER': '▁HE Y ▁COMP U TER',
|
||||||
|
'FRIDAY': '▁F RI DAY',
|
||||||
|
'HEY FRIDAY': '▁HE Y ▁F RI DAY',
|
||||||
|
'ALFRED': '▁A L F RE D',
|
||||||
|
'HERMES': '▁HER ME S',
|
||||||
|
'HEY HERMES': '▁HE Y ▁HER ME S',
|
||||||
|
'ASSISTANT': '▁AS S IST ANT',
|
||||||
|
'OK GOOGLE': '▁O K ▁GO O G LE',
|
||||||
|
'ALEXA': '▁A LE X A',
|
||||||
|
'SIRI': '▁S I RI',
|
||||||
|
'HEY SIRI': '▁HE Y ▁S I RI',
|
||||||
|
'NOVA': '▁NO V A',
|
||||||
|
'ATLAS': '▁AT LA S',
|
||||||
|
'HAL': '▁HA L'
|
||||||
|
};
|
||||||
|
|
||||||
|
export function normalizeKeyword(phrase: string): string {
|
||||||
|
return phrase
|
||||||
|
.normalize('NFD')
|
||||||
|
.replace(/[̀-ͯ]/g, '')
|
||||||
|
.toUpperCase()
|
||||||
|
.replace(/[^A-Z ]+/g, ' ')
|
||||||
|
.replace(/\s+/g, ' ')
|
||||||
|
.trim();
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Parse a sherpa tokens.txt ("piece id" per line) into the set of pieces. */
|
||||||
|
export function parseTokens(tokensFile: string): Set<string> {
|
||||||
|
const pieces = new Set<string>();
|
||||||
|
for (const line of tokensFile.split(/\r?\n/)) {
|
||||||
|
const piece = line.trim().split(/\s+/)[0];
|
||||||
|
if (piece && !piece.startsWith('<')) pieces.add(piece);
|
||||||
|
}
|
||||||
|
return pieces;
|
||||||
|
}
|
||||||
|
|
||||||
|
function greedyWord(word: string, vocab: Set<string>): string[] | null {
|
||||||
|
const out: string[] = [];
|
||||||
|
let i = 0;
|
||||||
|
let first = true;
|
||||||
|
while (i < word.length) {
|
||||||
|
let matched = '';
|
||||||
|
for (let len = word.length - i; len >= 1; len--) {
|
||||||
|
const candidate = (first ? '▁' : '') + word.slice(i, i + len);
|
||||||
|
if (vocab.has(candidate)) {
|
||||||
|
matched = candidate;
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (!matched && first) {
|
||||||
|
// No word-initial piece: use a bare "▁" if available, then continue without the prefix.
|
||||||
|
if (vocab.has('▁')) out.push('▁');
|
||||||
|
first = false;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
if (!matched) return null;
|
||||||
|
out.push(matched);
|
||||||
|
i += matched.length - (first ? 1 : 0);
|
||||||
|
first = false;
|
||||||
|
}
|
||||||
|
return out;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Encode a phrase into space-separated BPE pieces, or null when it cannot be represented. */
|
||||||
|
export function encodeKeyword(phrase: string, vocab: Set<string>): string | null {
|
||||||
|
const normalized = normalizeKeyword(phrase);
|
||||||
|
if (!normalized) return null;
|
||||||
|
if (KNOWN[normalized]) return KNOWN[normalized];
|
||||||
|
const pieces: string[] = [];
|
||||||
|
for (const word of normalized.split(' ')) {
|
||||||
|
const encoded = greedyWord(word, vocab);
|
||||||
|
if (!encoded) return null;
|
||||||
|
pieces.push(...encoded);
|
||||||
|
}
|
||||||
|
return pieces.join(' ');
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Build the keywords file content: one line per phrase, with the readable label. */
|
||||||
|
export function buildKeywordsFile(phrases: string[], vocab: Set<string>): { content: string; accepted: string[]; rejected: string[] } {
|
||||||
|
const lines: string[] = [];
|
||||||
|
const accepted: string[] = [];
|
||||||
|
const rejected: string[] = [];
|
||||||
|
for (const phrase of phrases) {
|
||||||
|
const encoded = encodeKeyword(phrase, vocab);
|
||||||
|
const label = normalizeKeyword(phrase).toLowerCase().replace(/ /g, '_');
|
||||||
|
if (!encoded || !label) {
|
||||||
|
rejected.push(phrase);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
lines.push(`${encoded} @${label}`);
|
||||||
|
accepted.push(label);
|
||||||
|
}
|
||||||
|
return { content: lines.join('\n') + '\n', accepted, rejected };
|
||||||
|
}
|
||||||
+125
@@ -0,0 +1,125 @@
|
|||||||
|
/** Local voice engine contract (sherpa-onnx in a utility process). Shared by main and renderer. */
|
||||||
|
|
||||||
|
export type VoiceModelKind = 'stt' | 'tts' | 'kws' | 'vad';
|
||||||
|
export type VoiceEngineKind = 'whisper' | 'sense-voice' | 'nemo-transducer' | 'kokoro' | 'piper' | 'kws-transducer' | 'silero';
|
||||||
|
|
||||||
|
export interface VoiceSpeaker {
|
||||||
|
id: number;
|
||||||
|
name: string;
|
||||||
|
lang: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface VoiceModelSpec {
|
||||||
|
id: string;
|
||||||
|
kind: VoiceModelKind;
|
||||||
|
engine: VoiceEngineKind;
|
||||||
|
name: string;
|
||||||
|
description: string;
|
||||||
|
languages: string[];
|
||||||
|
sizeMb: number;
|
||||||
|
url: string;
|
||||||
|
/** Folder created by the archive (files are referenced relative to it). */
|
||||||
|
dir: string;
|
||||||
|
/** Files that must exist once installed. */
|
||||||
|
files: string[];
|
||||||
|
speakers?: VoiceSpeaker[];
|
||||||
|
sampleRate?: number;
|
||||||
|
recommended?: boolean;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface VoiceModelStatus extends VoiceModelSpec {
|
||||||
|
installed: boolean;
|
||||||
|
downloading: boolean;
|
||||||
|
installedBytes: number;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface VoiceDownloadProgress {
|
||||||
|
id: string;
|
||||||
|
phase: 'download' | 'extract' | 'done' | 'error' | 'cancelled';
|
||||||
|
received: number;
|
||||||
|
total: number;
|
||||||
|
percent: number;
|
||||||
|
message?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface TranscribeRequest {
|
||||||
|
modelId: string;
|
||||||
|
wav: Uint8Array;
|
||||||
|
language: string; // 'fr', 'en', 'auto'
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface TranscribeResult {
|
||||||
|
text: string;
|
||||||
|
language?: string;
|
||||||
|
durationMs: number;
|
||||||
|
audioSec: number;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface SynthesizeRequest {
|
||||||
|
modelId: string;
|
||||||
|
text: string;
|
||||||
|
speaker: number;
|
||||||
|
speed: number;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface SynthesizeResult {
|
||||||
|
wav: Uint8Array;
|
||||||
|
sampleRate: number;
|
||||||
|
durationMs: number;
|
||||||
|
audioSec: number;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface KwsStartRequest {
|
||||||
|
modelId: string;
|
||||||
|
/** Wake phrases in plain text (e.g. "jarvis", "hey jarvis"). */
|
||||||
|
keywords: string[];
|
||||||
|
/** 1 (strict) .. 5 (eager) */
|
||||||
|
sensitivity: number;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface KwsDetection {
|
||||||
|
keyword: string;
|
||||||
|
at: number;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface VadStartRequest {
|
||||||
|
modelId: string;
|
||||||
|
/** Silence that ends an utterance, in ms. */
|
||||||
|
silenceMs: number;
|
||||||
|
/** Detection threshold 0..1 (0.5 default). */
|
||||||
|
threshold: number;
|
||||||
|
maxUtteranceSec: number;
|
||||||
|
}
|
||||||
|
|
||||||
|
export type VadEvent =
|
||||||
|
| { type: 'speech-start' }
|
||||||
|
| { type: 'segment'; wav: Uint8Array; durationSec: number }
|
||||||
|
| { type: 'error'; message: string };
|
||||||
|
|
||||||
|
export interface VoiceEngineStatus {
|
||||||
|
available: boolean;
|
||||||
|
error?: string;
|
||||||
|
version?: string;
|
||||||
|
loaded: string[];
|
||||||
|
modelsDir: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
export const VOICE_IPC = {
|
||||||
|
status: 'voice:status',
|
||||||
|
modelsList: 'voice:models:list',
|
||||||
|
modelsDownload: 'voice:models:download',
|
||||||
|
modelsCancel: 'voice:models:cancel',
|
||||||
|
modelsRemove: 'voice:models:remove',
|
||||||
|
modelsProgress: 'voice:models:progress',
|
||||||
|
transcribe: 'voice:transcribe',
|
||||||
|
synthesize: 'voice:synthesize',
|
||||||
|
unload: 'voice:unload',
|
||||||
|
kwsStart: 'voice:kws:start',
|
||||||
|
kwsStop: 'voice:kws:stop',
|
||||||
|
kwsAudio: 'voice:kws:audio',
|
||||||
|
kwsDetected: 'voice:kws:detected',
|
||||||
|
vadStart: 'voice:vad:start',
|
||||||
|
vadStop: 'voice:vad:stop',
|
||||||
|
vadAudio: 'voice:vad:audio',
|
||||||
|
vadEvent: 'voice:vad:event'
|
||||||
|
} as const;
|
||||||
+44
-11
@@ -1,4 +1,4 @@
|
|||||||
import { useEffect, useState } from 'react';
|
import { useEffect, useRef, useState } from 'react';
|
||||||
import { AlertTriangle, X } from 'lucide-react';
|
import { AlertTriangle, X } from 'lucide-react';
|
||||||
import type { WindowMode } from '../shared/ipc';
|
import type { WindowMode } from '../shared/ipc';
|
||||||
import { bridge } from './lib/bridge';
|
import { bridge } from './lib/bridge';
|
||||||
@@ -10,6 +10,7 @@ import { useChat } from './state/chat';
|
|||||||
import { useHermes } from './state/hermes';
|
import { useHermes } from './state/hermes';
|
||||||
import { useSettings } from './state/settings';
|
import { useSettings } from './state/settings';
|
||||||
import { useVoice } from './state/voice';
|
import { useVoice } from './state/voice';
|
||||||
|
import { useVoiceModels } from './state/voiceModels';
|
||||||
import { ChatPanel } from './components/chat/ChatPanel';
|
import { ChatPanel } from './components/chat/ChatPanel';
|
||||||
import { PendingRequests } from './components/chat/PendingRequests';
|
import { PendingRequests } from './components/chat/PendingRequests';
|
||||||
import { CompactWidget } from './components/compact/CompactWidget';
|
import { CompactWidget } from './components/compact/CompactWidget';
|
||||||
@@ -41,6 +42,20 @@ function useBoot(): boolean {
|
|||||||
Log.error('app', 'window.eveflow missing inside Electron');
|
Log.error('app', 'window.eveflow missing inside Electron');
|
||||||
}
|
}
|
||||||
if (api) {
|
if (api) {
|
||||||
|
disposers.push(useVoiceModels.getState().subscribe());
|
||||||
|
void useVoiceModels.getState().refresh();
|
||||||
|
if (settings.voice.wakeMode === 'kws') void voiceController.startWakeMode();
|
||||||
|
let lastWake = JSON.stringify([settings.voice.wakeMode, settings.voice.wakeWord, settings.voice.kwsSensitivity, settings.voice.micDeviceId]);
|
||||||
|
disposers.push(
|
||||||
|
useSettings.subscribe((s) => {
|
||||||
|
const v = s.settings.voice;
|
||||||
|
const key = JSON.stringify([v.wakeMode, v.wakeWord, v.kwsSensitivity, v.micDeviceId]);
|
||||||
|
if (key === lastWake) return;
|
||||||
|
lastWake = key;
|
||||||
|
if (v.wakeMode === 'kws') void voiceController.restartWakeMode();
|
||||||
|
else void voiceController.stopWakeMode();
|
||||||
|
})
|
||||||
|
);
|
||||||
disposers.push(api.hermes.onPush(handlePush));
|
disposers.push(api.hermes.onPush(handlePush));
|
||||||
disposers.push(
|
disposers.push(
|
||||||
api.hotkeys.on((event) => {
|
api.hotkeys.on((event) => {
|
||||||
@@ -96,15 +111,23 @@ function useWindowMode(): WindowMode {
|
|||||||
return mode;
|
return mode;
|
||||||
}
|
}
|
||||||
|
|
||||||
function ErrorBanner() {
|
/** Errors surface as a transient toast over the core instead of shifting the layout. */
|
||||||
|
function ErrorToast() {
|
||||||
const error = useChat((s) => s.error);
|
const error = useChat((s) => s.error);
|
||||||
const setError = useChat((s) => s.setError);
|
const setError = useChat((s) => s.setError);
|
||||||
|
useEffect(() => {
|
||||||
|
if (!error) return;
|
||||||
|
const timer = setTimeout(() => setError(null), 9000);
|
||||||
|
return () => clearTimeout(timer);
|
||||||
|
}, [error, setError]);
|
||||||
if (!error) return null;
|
if (!error) return null;
|
||||||
return (
|
return (
|
||||||
<div className="error-banner">
|
<div className="toast-stack">
|
||||||
<AlertTriangle size={16} />
|
<div className="error-banner" role="alert">
|
||||||
<span>{error}</span>
|
<AlertTriangle size={16} />
|
||||||
<button className="icon-btn" onClick={() => setError(null)}><X size={14} /></button>
|
<span>{error}</span>
|
||||||
|
<button className="icon-btn" aria-label="Fermer" onClick={() => setError(null)}><X size={14} /></button>
|
||||||
|
</div>
|
||||||
</div>
|
</div>
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
@@ -115,15 +138,25 @@ export default function App() {
|
|||||||
const mode = useWindowMode();
|
const mode = useWindowMode();
|
||||||
const showTelemetry = useSettings((s) => s.settings.ui.showTelemetry);
|
const showTelemetry = useSettings((s) => s.settings.ui.showTelemetry);
|
||||||
const [showSettings, setShowSettings] = useState(false);
|
const [showSettings, setShowSettings] = useState(false);
|
||||||
|
const [showOps, setShowOps] = useState(false);
|
||||||
|
const opsBadge = useChat((s) => s.activity.filter((a) => a.status === 'running').length) + useHermes((s) => s.jobRuns.filter((r) => !r.read).length);
|
||||||
|
const showSettingsRef = useRef(false);
|
||||||
|
const showOpsRef = useRef(false);
|
||||||
|
showSettingsRef.current = showSettings;
|
||||||
|
showOpsRef.current = showOps;
|
||||||
|
|
||||||
useEffect(() => {
|
useEffect(() => {
|
||||||
const onKey = (e: KeyboardEvent) => {
|
const onKey = (e: KeyboardEvent) => {
|
||||||
if (e.key === 'Escape' && showSettings) setShowSettings(false);
|
if (e.key === 'Escape') {
|
||||||
|
if (showSettingsRef.current) setShowSettings(false);
|
||||||
|
else if (showOpsRef.current) setShowOps(false);
|
||||||
|
}
|
||||||
if (e.key === ',' && (e.ctrlKey || e.metaKey)) setShowSettings((v) => !v);
|
if (e.key === ',' && (e.ctrlKey || e.metaKey)) setShowSettings((v) => !v);
|
||||||
|
if (e.key.toLowerCase() === 'o' && (e.ctrlKey || e.metaKey) && e.shiftKey) setShowOps((v) => !v);
|
||||||
};
|
};
|
||||||
window.addEventListener('keydown', onKey);
|
window.addEventListener('keydown', onKey);
|
||||||
return () => window.removeEventListener('keydown', onKey);
|
return () => window.removeEventListener('keydown', onKey);
|
||||||
}, [showSettings]);
|
}, []);
|
||||||
|
|
||||||
if (!loaded) {
|
if (!loaded) {
|
||||||
return (
|
return (
|
||||||
@@ -144,22 +177,22 @@ export default function App() {
|
|||||||
|
|
||||||
return (
|
return (
|
||||||
<div className="hud-root">
|
<div className="hud-root">
|
||||||
<TopBar onOpenSettings={() => setShowSettings(true)} />
|
<TopBar onOpenSettings={() => setShowSettings(true)} onToggleOps={() => setShowOps((v) => !v)} opsBadge={opsBadge} />
|
||||||
<div className="hud-body">
|
<div className="hud-body">
|
||||||
<div className="hud-column">
|
<div className="hud-column">
|
||||||
<ChatPanel />
|
<ChatPanel />
|
||||||
</div>
|
</div>
|
||||||
<div className="hud-column">
|
<div className="hud-column">
|
||||||
<ErrorBanner />
|
|
||||||
<section className="panel bracket" style={{ flex: 1 }}>
|
<section className="panel bracket" style={{ flex: 1 }}>
|
||||||
<CoreStage />
|
<CoreStage />
|
||||||
</section>
|
</section>
|
||||||
</div>
|
</div>
|
||||||
<div className="hud-column ops-column">
|
<div className={`hud-column ops-column${showOps ? ' open' : ''}`}>
|
||||||
<OpsPanel />
|
<OpsPanel />
|
||||||
{showTelemetry && <Telemetry />}
|
{showTelemetry && <Telemetry />}
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
|
<ErrorToast />
|
||||||
{showSettings && <SettingsDrawer onClose={() => setShowSettings(false)} />}
|
{showSettings && <SettingsDrawer onClose={() => setShowSettings(false)} />}
|
||||||
<PendingRequests />
|
<PendingRequests />
|
||||||
</div>
|
</div>
|
||||||
|
|||||||
@@ -12,18 +12,36 @@ interface Props {
|
|||||||
export function ChatPanel({ compact }: Props) {
|
export function ChatPanel({ compact }: Props) {
|
||||||
const messages = useChat((s) => s.messages);
|
const messages = useChat((s) => s.messages);
|
||||||
const clear = useChat((s) => s.clear);
|
const clear = useChat((s) => s.clear);
|
||||||
const settings = useSettings((s) => s.settings);
|
const assistantName = useSettings((s) => s.settings.assistantName);
|
||||||
|
const userName = useSettings((s) => s.settings.userName);
|
||||||
|
const showReasoning = useSettings((s) => s.settings.ui.showReasoning);
|
||||||
const listRef = useRef<HTMLDivElement>(null);
|
const listRef = useRef<HTMLDivElement>(null);
|
||||||
|
const programmatic = useRef(false);
|
||||||
const [zoom, setZoom] = useState<string | null>(null);
|
const [zoom, setZoom] = useState<string | null>(null);
|
||||||
const [stick, setStick] = useState(true);
|
const [stick, setStick] = useState(true);
|
||||||
|
|
||||||
useEffect(() => {
|
useEffect(() => {
|
||||||
if (!stick) return;
|
if (!stick) return;
|
||||||
const el = listRef.current;
|
const el = listRef.current;
|
||||||
if (el) el.scrollTop = el.scrollHeight;
|
if (!el) return;
|
||||||
|
programmatic.current = true;
|
||||||
|
el.scrollTop = el.scrollHeight;
|
||||||
}, [messages, stick]);
|
}, [messages, stick]);
|
||||||
|
|
||||||
|
useEffect(() => {
|
||||||
|
if (!zoom) return;
|
||||||
|
const onKey = (e: KeyboardEvent) => {
|
||||||
|
if (e.key === 'Escape') setZoom(null);
|
||||||
|
};
|
||||||
|
window.addEventListener('keydown', onKey);
|
||||||
|
return () => window.removeEventListener('keydown', onKey);
|
||||||
|
}, [zoom]);
|
||||||
|
|
||||||
const onScroll = () => {
|
const onScroll = () => {
|
||||||
|
if (programmatic.current) {
|
||||||
|
programmatic.current = false;
|
||||||
|
return;
|
||||||
|
}
|
||||||
const el = listRef.current;
|
const el = listRef.current;
|
||||||
if (!el) return;
|
if (!el) return;
|
||||||
setStick(el.scrollHeight - el.scrollTop - el.clientHeight < 80);
|
setStick(el.scrollHeight - el.scrollTop - el.clientHeight < 80);
|
||||||
@@ -47,7 +65,7 @@ export function ChatPanel({ compact }: Props) {
|
|||||||
<div ref={listRef} className={`chat-list${compact ? ' compact' : ''}`} onScroll={onScroll}>
|
<div ref={listRef} className={`chat-list${compact ? ' compact' : ''}`} onScroll={onScroll}>
|
||||||
{visible.length === 0 && (
|
{visible.length === 0 && (
|
||||||
<div className="empty">
|
<div className="empty">
|
||||||
{compact ? 'En attente…' : `Canal ouvert. Parlez ou écrivez à ${settings.assistantName}.`}
|
{compact ? 'En attente…' : `Canal ouvert. Parlez ou écrivez à ${assistantName}.`}
|
||||||
{!compact && (
|
{!compact && (
|
||||||
<div className="muted" style={{ marginTop: 8, fontSize: 12 }}>
|
<div className="muted" style={{ marginTop: 8, fontSize: 12 }}>
|
||||||
<span className="kbd">Ctrl</span> + <span className="kbd">Shift</span> + <span className="kbd">Espace</span> active le micro depuis n’importe où.
|
<span className="kbd">Ctrl</span> + <span className="kbd">Shift</span> + <span className="kbd">Espace</span> active le micro depuis n’importe où.
|
||||||
@@ -59,9 +77,9 @@ export function ChatPanel({ compact }: Props) {
|
|||||||
<MessageBubble
|
<MessageBubble
|
||||||
key={m.id}
|
key={m.id}
|
||||||
message={m}
|
message={m}
|
||||||
assistantName={settings.assistantName}
|
assistantName={assistantName}
|
||||||
userName={settings.userName}
|
userName={userName}
|
||||||
showReasoning={settings.ui.showReasoning}
|
showReasoning={showReasoning}
|
||||||
compact={compact}
|
compact={compact}
|
||||||
onOpenImage={setZoom}
|
onOpenImage={setZoom}
|
||||||
/>
|
/>
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
import { useEffect, useRef, useState } from 'react';
|
import { useEffect, useRef, useState } from 'react';
|
||||||
import { Mic, Paperclip, Send, Square, X, Radio, Loader2, Volume2, VolumeX, Navigation } from 'lucide-react';
|
import { Mic, Paperclip, Send, Square, X, Radio, Loader2, Volume2, VolumeX, Navigation, ScanEye } from 'lucide-react';
|
||||||
import { sendMessage, steer, stopGeneration } from '../../services/conversation';
|
import { sendMessage, steer, stopGeneration , captureScreen } from '../../services/conversation';
|
||||||
import { speech } from '../../services/voice/speech';
|
import { speech } from '../../services/voice/speech';
|
||||||
import { voiceController } from '../../services/voice/voiceController';
|
import { voiceController } from '../../services/voice/voiceController';
|
||||||
import { useChat } from '../../state/chat';
|
import { useChat } from '../../state/chat';
|
||||||
@@ -67,17 +67,34 @@ export function CommandBar({ compact }: Props) {
|
|||||||
|
|
||||||
const submit = async () => {
|
const submit = async () => {
|
||||||
const text = draft.trim();
|
const text = draft.trim();
|
||||||
if (steerMode && currentRunId) {
|
if (steerMode) {
|
||||||
|
if (!currentRunId) {
|
||||||
|
setSteerMode(false);
|
||||||
|
return;
|
||||||
|
}
|
||||||
if (await steer(text)) setDraft('');
|
if (await steer(text)) setDraft('');
|
||||||
|
inputRef.current?.focus();
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
if (!text && images.length === 0) return;
|
if (!text && images.length === 0) return;
|
||||||
const attached = images;
|
const attached = images;
|
||||||
setDraft('');
|
setDraft('');
|
||||||
setImages([]);
|
setImages([]);
|
||||||
|
inputRef.current?.focus();
|
||||||
await sendMessage(text, attached);
|
await sendMessage(text, attached);
|
||||||
};
|
};
|
||||||
|
|
||||||
|
useEffect(() => {
|
||||||
|
const onKey = (e: KeyboardEvent) => {
|
||||||
|
if ((e.ctrlKey || e.metaKey) && e.key.toLowerCase() === 'k') {
|
||||||
|
e.preventDefault();
|
||||||
|
inputRef.current?.focus();
|
||||||
|
}
|
||||||
|
};
|
||||||
|
window.addEventListener('keydown', onKey);
|
||||||
|
return () => window.removeEventListener('keydown', onKey);
|
||||||
|
}, []);
|
||||||
|
|
||||||
const onKeyDown = (e: React.KeyboardEvent<HTMLTextAreaElement>) => {
|
const onKeyDown = (e: React.KeyboardEvent<HTMLTextAreaElement>) => {
|
||||||
if (e.key === 'Enter' && !e.shiftKey) {
|
if (e.key === 'Enter' && !e.shiftKey) {
|
||||||
e.preventDefault();
|
e.preventDefault();
|
||||||
@@ -131,6 +148,8 @@ export function CommandBar({ compact }: Props) {
|
|||||||
className={`mic-btn ${micClass}`}
|
className={`mic-btn ${micClass}`}
|
||||||
style={{ ['--level' as string]: level }}
|
style={{ ['--level' as string]: level }}
|
||||||
title={micTitle}
|
title={micTitle}
|
||||||
|
aria-label={micTitle}
|
||||||
|
aria-pressed={listening}
|
||||||
onClick={() => voiceController.toggle()}
|
onClick={() => voiceController.toggle()}
|
||||||
disabled={phase === 'transcribing'}
|
disabled={phase === 'transcribing'}
|
||||||
>
|
>
|
||||||
@@ -148,17 +167,28 @@ export function CommandBar({ compact }: Props) {
|
|||||||
spellCheck={false}
|
spellCheck={false}
|
||||||
/>
|
/>
|
||||||
{!compact && (
|
{!compact && (
|
||||||
<button className="icon-btn" title="Joindre une image" onClick={() => fileRef.current?.click()} style={{ height: 46, width: 40 }}>
|
<>
|
||||||
<Paperclip size={18} />
|
<button
|
||||||
</button>
|
className="icon-btn"
|
||||||
|
title="Joindre une capture de l’écran (Hermes la voit)"
|
||||||
|
aria-label="Capturer l’écran"
|
||||||
|
onClick={() => void captureScreen().then((shot) => { if (shot) setImages((prev) => [...prev, shot].slice(-4)); })}
|
||||||
|
style={{ height: 46, width: 40 }}
|
||||||
|
>
|
||||||
|
<ScanEye size={18} />
|
||||||
|
</button>
|
||||||
|
<button className="icon-btn" title="Joindre une image" onClick={() => fileRef.current?.click()} style={{ height: 46, width: 40 }}>
|
||||||
|
<Paperclip size={18} />
|
||||||
|
</button>
|
||||||
|
</>
|
||||||
)}
|
)}
|
||||||
<input ref={fileRef} type="file" accept="image/*" multiple hidden onChange={(e) => void onFiles(e.target.files)} />
|
<input ref={fileRef} type="file" accept="image/*" multiple hidden onChange={(e) => void onFiles(e.target.files)} />
|
||||||
{isSending && !steerMode ? (
|
{isSending && !steerMode ? (
|
||||||
<button className="send-btn stop" title="Interrompre (Échap)" onClick={stopGeneration}>
|
<button className="send-btn stop" title="Interrompre (Échap)" aria-label="Interrompre" onClick={stopGeneration}>
|
||||||
<Square size={18} />
|
<Square size={18} />
|
||||||
</button>
|
</button>
|
||||||
) : (
|
) : (
|
||||||
<button className="send-btn primary" title={steerMode ? 'Envoyer la consigne' : 'Envoyer (Entrée)'} onClick={() => void submit()} disabled={!draft.trim() && images.length === 0}>
|
<button className="send-btn primary" title={steerMode ? 'Envoyer la consigne' : 'Envoyer (Entrée)'} aria-label={steerMode ? 'Envoyer la consigne' : 'Envoyer'} onClick={() => void submit()} disabled={!draft.trim() && images.length === 0}>
|
||||||
{steerMode ? <Navigation size={18} /> : <Send size={18} />}
|
{steerMode ? <Navigation size={18} /> : <Send size={18} />}
|
||||||
</button>
|
</button>
|
||||||
)}
|
)}
|
||||||
|
|||||||
@@ -1,4 +1,4 @@
|
|||||||
import { memo, useState } from 'react';
|
import { memo, useEffect, useRef, useState } from 'react';
|
||||||
import ReactMarkdown from 'react-markdown';
|
import ReactMarkdown from 'react-markdown';
|
||||||
import remarkGfm from 'remark-gfm';
|
import remarkGfm from 'remark-gfm';
|
||||||
import { Copy, Volume2, Check } from 'lucide-react';
|
import { Copy, Volume2, Check } from 'lucide-react';
|
||||||
@@ -28,6 +28,10 @@ function openLink(href: string): void {
|
|||||||
|
|
||||||
export const MessageBubble = memo(function MessageBubble({ message, assistantName, userName, showReasoning, compact, onOpenImage }: Props) {
|
export const MessageBubble = memo(function MessageBubble({ message, assistantName, userName, showReasoning, compact, onOpenImage }: Props) {
|
||||||
const [copied, setCopied] = useState(false);
|
const [copied, setCopied] = useState(false);
|
||||||
|
const copyTimer = useRef<number | null>(null);
|
||||||
|
useEffect(() => () => {
|
||||||
|
if (copyTimer.current) window.clearTimeout(copyTimer.current);
|
||||||
|
}, []);
|
||||||
const isUser = message.role === 'user';
|
const isUser = message.role === 'user';
|
||||||
const who = isUser ? userName || 'VOUS' : message.role === 'system' ? 'SYSTÈME' : assistantName;
|
const who = isUser ? userName || 'VOUS' : message.role === 'system' ? 'SYSTÈME' : assistantName;
|
||||||
const status = message.status ?? 'done';
|
const status = message.status ?? 'done';
|
||||||
@@ -35,7 +39,8 @@ export const MessageBubble = memo(function MessageBubble({ message, assistantNam
|
|||||||
const copy = () => {
|
const copy = () => {
|
||||||
navigator.clipboard.writeText(message.content).then(() => {
|
navigator.clipboard.writeText(message.content).then(() => {
|
||||||
setCopied(true);
|
setCopied(true);
|
||||||
setTimeout(() => setCopied(false), 1200);
|
if (copyTimer.current) window.clearTimeout(copyTimer.current);
|
||||||
|
copyTimer.current = window.setTimeout(() => setCopied(false), 1200);
|
||||||
}).catch(() => undefined);
|
}).catch(() => undefined);
|
||||||
};
|
};
|
||||||
|
|
||||||
@@ -50,8 +55,8 @@ export const MessageBubble = memo(function MessageBubble({ message, assistantNam
|
|||||||
{message.usage?.total_tokens ? <span>{message.usage.total_tokens} tok</span> : null}
|
{message.usage?.total_tokens ? <span>{message.usage.total_tokens} tok</span> : null}
|
||||||
{message.role === 'assistant' && status === 'done' && (
|
{message.role === 'assistant' && status === 'done' && (
|
||||||
<span className="msg-actions">
|
<span className="msg-actions">
|
||||||
<button className="icon-btn" title="Copier" onClick={copy}>{copied ? <Check size={13} /> : <Copy size={13} />}</button>
|
<button className="icon-btn" title="Copier" aria-label="Copier le message" onClick={copy}>{copied ? <Check size={13} /> : <Copy size={13} />}</button>
|
||||||
<button className="icon-btn" title="Lire à voix haute" onClick={() => speech.say(message.content)}><Volume2 size={13} /></button>
|
<button className="icon-btn" title="Lire à voix haute" aria-label="Lire à voix haute" onClick={() => speech.say(message.content)}><Volume2 size={13} /></button>
|
||||||
</span>
|
</span>
|
||||||
)}
|
)}
|
||||||
</div>
|
</div>
|
||||||
|
|||||||
@@ -1,4 +1,4 @@
|
|||||||
import { useState } from 'react';
|
import { useEffect, useState } from 'react';
|
||||||
import { ShieldAlert, HelpCircle, KeyRound } from 'lucide-react';
|
import { ShieldAlert, HelpCircle, KeyRound } from 'lucide-react';
|
||||||
import { resolvePending } from '../../services/conversation';
|
import { resolvePending } from '../../services/conversation';
|
||||||
import { useChat, type PendingRequest } from '../../state/chat';
|
import { useChat, type PendingRequest } from '../../state/chat';
|
||||||
@@ -7,10 +7,22 @@ function RequestCard({ request }: { request: PendingRequest }) {
|
|||||||
const [answer, setAnswer] = useState('');
|
const [answer, setAnswer] = useState('');
|
||||||
const icon = request.kind === 'approval' ? <ShieldAlert size={16} /> : request.kind === 'clarify' ? <HelpCircle size={16} /> : <KeyRound size={16} />;
|
const icon = request.kind === 'approval' ? <ShieldAlert size={16} /> : request.kind === 'clarify' ? <HelpCircle size={16} /> : <KeyRound size={16} />;
|
||||||
|
|
||||||
|
useEffect(() => {
|
||||||
|
const onKey = (e: KeyboardEvent) => {
|
||||||
|
if (e.key === 'Escape') {
|
||||||
|
e.stopPropagation();
|
||||||
|
if (request.kind === 'approval') void resolvePending(request, 'deny');
|
||||||
|
else useChat.getState().removePending(request.id);
|
||||||
|
}
|
||||||
|
};
|
||||||
|
window.addEventListener('keydown', onKey, true);
|
||||||
|
return () => window.removeEventListener('keydown', onKey, true);
|
||||||
|
}, [request]);
|
||||||
|
|
||||||
return (
|
return (
|
||||||
<div className="modal">
|
<div className="modal" role="dialog" aria-modal="true" aria-labelledby={`req-${request.id}`}>
|
||||||
<div className="request-card">
|
<div className="request-card">
|
||||||
<div className="title">
|
<div className="title" id={`req-${request.id}`}>
|
||||||
{icon}
|
{icon}
|
||||||
{request.title}
|
{request.title}
|
||||||
{request.tool && <span className="chip warn">{request.tool}</span>}
|
{request.tool && <span className="chip warn">{request.tool}</span>}
|
||||||
@@ -20,7 +32,7 @@ function RequestCard({ request }: { request: PendingRequest }) {
|
|||||||
{request.kind === 'approval' ? (
|
{request.kind === 'approval' ? (
|
||||||
<div className="actions">
|
<div className="actions">
|
||||||
<button className="btn danger" onClick={() => void resolvePending(request, 'deny')}>Refuser</button>
|
<button className="btn danger" onClick={() => void resolvePending(request, 'deny')}>Refuser</button>
|
||||||
<button className="btn" onClick={() => void resolvePending(request, 'once')}>Autoriser une fois</button>
|
<button className="btn" autoFocus onClick={() => void resolvePending(request, 'once')}>Autoriser une fois</button>
|
||||||
<button className="btn" onClick={() => void resolvePending(request, 'session')}>Pour la session</button>
|
<button className="btn" onClick={() => void resolvePending(request, 'session')}>Pour la session</button>
|
||||||
<button className="btn primary" onClick={() => void resolvePending(request, 'always')}>Toujours</button>
|
<button className="btn primary" onClick={() => void resolvePending(request, 'always')}>Toujours</button>
|
||||||
</div>
|
</div>
|
||||||
|
|||||||
@@ -40,6 +40,10 @@ export function ResilientImage({ src = '', alt, onOpen }: Props) {
|
|||||||
setResolved(cache.get(src)!);
|
setResolved(cache.get(src)!);
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
if (failures.has(src)) {
|
||||||
|
setError(failures.get(src)!);
|
||||||
|
return;
|
||||||
|
}
|
||||||
if (src.startsWith('data:') || src.startsWith('blob:') || /^https?:\/\//i.test(src)) {
|
if (src.startsWith('data:') || src.startsWith('blob:') || /^https?:\/\//i.test(src)) {
|
||||||
ok(src);
|
ok(src);
|
||||||
return;
|
return;
|
||||||
|
|||||||
@@ -5,12 +5,13 @@ import { CoreStage } from '../hud/CoreStage';
|
|||||||
import { ChatPanel } from '../chat/ChatPanel';
|
import { ChatPanel } from '../chat/ChatPanel';
|
||||||
|
|
||||||
export function CompactWidget() {
|
export function CompactWidget() {
|
||||||
const settings = useSettings((s) => s.settings);
|
const assistantName = useSettings((s) => s.settings.assistantName);
|
||||||
|
const compactOpacity = useSettings((s) => s.settings.ui.compactOpacity);
|
||||||
const api = bridge();
|
const api = bridge();
|
||||||
return (
|
return (
|
||||||
<div className="compact-root" style={{ ['--compact-opacity' as string]: settings.ui.compactOpacity }}>
|
<div className="compact-root" style={{ ['--compact-opacity' as string]: compactOpacity }}>
|
||||||
<div className="compact-head">
|
<div className="compact-head">
|
||||||
<span className="title">{settings.assistantName}</span>
|
<span className="title">{assistantName}</span>
|
||||||
{api && (
|
{api && (
|
||||||
<>
|
<>
|
||||||
<button className="icon-btn" title="Mode HUD complet" onClick={() => api.window.setMode('hud')}><Maximize2 size={14} /></button>
|
<button className="icon-btn" title="Mode HUD complet" onClick={() => api.window.setMode('hud')}><Maximize2 size={14} /></button>
|
||||||
|
|||||||
@@ -1,3 +1,4 @@
|
|||||||
|
import { useEffect, useState } from 'react';
|
||||||
import { useChat } from '../../state/chat';
|
import { useChat } from '../../state/chat';
|
||||||
import { useHermes } from '../../state/hermes';
|
import { useHermes } from '../../state/hermes';
|
||||||
import { useSettings } from '../../state/settings';
|
import { useSettings } from '../../state/settings';
|
||||||
@@ -21,40 +22,62 @@ interface Props {
|
|||||||
|
|
||||||
export function CoreStage({ compact }: Props) {
|
export function CoreStage({ compact }: Props) {
|
||||||
const hud = useChat((s) => s.hudOverride ?? s.hud);
|
const hud = useChat((s) => s.hudOverride ?? s.hud);
|
||||||
const activity = useChat((s) => s.activity);
|
const running = useChat((s) => {
|
||||||
|
for (let i = s.activity.length - 1; i >= 0; i--) if (s.activity[i].status === 'running') return s.activity[i];
|
||||||
|
return undefined;
|
||||||
|
});
|
||||||
const isSending = useChat((s) => s.isSending);
|
const isSending = useChat((s) => s.isSending);
|
||||||
const error = useChat((s) => s.error);
|
const error = useChat((s) => s.error);
|
||||||
|
const pulse = useChat((s) => s.pingCount);
|
||||||
|
const latencyMs = useChat((s) => s.latencyMs);
|
||||||
const phase = useVoice((s) => s.phase);
|
const phase = useVoice((s) => s.phase);
|
||||||
const level = useVoice((s) => s.inputLevel);
|
const level = useVoice((s) => s.inputLevel);
|
||||||
const transcript = useVoice((s) => s.lastTranscript);
|
const transcript = useVoice((s) => s.lastTranscript);
|
||||||
|
const interim = useVoice((s) => s.interim);
|
||||||
const voiceError = useVoice((s) => s.error);
|
const voiceError = useVoice((s) => s.error);
|
||||||
const settings = useSettings((s) => s.settings);
|
const wake = useVoice((s) => s.wake);
|
||||||
|
const wakeKeywords = useVoice((s) => s.wakeKeywords);
|
||||||
|
const assistantName = useSettings((s) => s.settings.assistantName);
|
||||||
|
const reduceMotion = useSettings((s) => s.settings.ui.reduceMotion);
|
||||||
const transport = useHermes((s) => s.transport);
|
const transport = useHermes((s) => s.transport);
|
||||||
const link = useHermes((s) => s.link);
|
const link = useHermes((s) => s.link);
|
||||||
|
const [elapsed, setElapsed] = useState(0);
|
||||||
|
|
||||||
|
useEffect(() => {
|
||||||
|
if (!isSending) {
|
||||||
|
setElapsed(0);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
const started = Date.now();
|
||||||
|
const timer = setInterval(() => setElapsed(Date.now() - started), 250);
|
||||||
|
return () => clearInterval(timer);
|
||||||
|
}, [isSending]);
|
||||||
|
|
||||||
const running = activity.filter((a) => a.status === 'running').slice(-1)[0];
|
|
||||||
let sub = '';
|
let sub = '';
|
||||||
if (hud === 'listening') sub = phase === 'speech' ? 'parole détectée…' : phase === 'transcribing' ? 'transcription…' : 'parlez maintenant';
|
if (hud === 'listening') sub = phase === 'speech' ? 'parole détectée…' : phase === 'transcribing' ? 'transcription…' : 'parlez maintenant';
|
||||||
else if (hud === 'thinking') sub = running ? `${running.name}${running.detail ? ` · ${previewText(running.detail, 60)}` : ''}` : isSending ? `Hermes · ${transport}` : '';
|
else if (hud === 'thinking') sub = running ? `${running.name}${running.detail ? ` · ${previewText(running.detail, 60)}` : ''}` : isSending ? `Hermes · ${transport} · ${(elapsed / 1000).toFixed(1)} s` : '';
|
||||||
else if (hud === 'error') sub = previewText(error ?? voiceError ?? 'erreur', 90);
|
else if (hud === 'error') sub = previewText(error ?? voiceError ?? 'erreur', 90);
|
||||||
else if (hud === 'speaking') sub = 'synthèse vocale';
|
else if (hud === 'speaking') sub = 'synthèse vocale';
|
||||||
else if (transcript) sub = `« ${previewText(transcript, 80)} »`;
|
else if (transcript) sub = `« ${previewText(transcript, 80)} »`;
|
||||||
|
else if (wake === 'spotting') sub = `dites « ${wakeKeywords[0] ?? 'jarvis'} »${link === 'online' ? ` · ${transport}` : link === 'offline' ? ' · Hermes hors ligne' : ''}`;
|
||||||
else sub = link === 'online' ? `liaison Hermes · ${transport}` : link === 'offline' ? 'Hermes hors ligne' : '';
|
else sub = link === 'online' ? `liaison Hermes · ${transport}` : link === 'offline' ? 'Hermes hors ligne' : '';
|
||||||
|
|
||||||
return (
|
return (
|
||||||
<div className={compact ? 'compact-core' : 'core-stage'}>
|
<div className={compact ? 'compact-core' : 'core-stage'}>
|
||||||
{!compact && (
|
{!compact && (
|
||||||
<div className="core-name">
|
<div className="core-name">
|
||||||
{settings.assistantName}
|
{assistantName}
|
||||||
<small>EVEFLOW · HERMES INTERFACE</small>
|
<small>EVEFLOW · HERMES INTERFACE</small>
|
||||||
</div>
|
</div>
|
||||||
)}
|
)}
|
||||||
<div className={compact ? undefined : 'core-canvas-wrap'} style={compact ? { position: 'absolute', inset: '0 0 36px 0' } : undefined}>
|
<div className={compact ? undefined : 'core-canvas-wrap'} style={compact ? { position: 'absolute', inset: '0 0 36px 0' } : undefined}>
|
||||||
<JarvisCore state={hud} inputLevel={level} reduceMotion={settings.ui.reduceMotion} />
|
<JarvisCore state={hud} inputLevel={level} reduceMotion={reduceMotion} pulse={pulse} />
|
||||||
</div>
|
</div>
|
||||||
<div className="core-caption">
|
<div className="core-caption" aria-live="polite">
|
||||||
|
{!compact && (phase === 'speech' || phase === 'transcribing') && <div className="core-interim">{interim || (phase === 'transcribing' ? '…' : '')}</div>}
|
||||||
<div className={`core-state ${hud}`}>{STATE_LABEL[hud] ?? hud}</div>
|
<div className={`core-state ${hud}`}>{STATE_LABEL[hud] ?? hud}</div>
|
||||||
{sub && <div className="core-sub">{sub}</div>}
|
{sub && <div className="core-sub">{sub}</div>}
|
||||||
|
{!compact && latencyMs !== null && <div className="core-latency">premier mot · {latencyMs} ms</div>}
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
);
|
);
|
||||||
|
|||||||
@@ -2,8 +2,13 @@
|
|||||||
* JARVIS-style arc reactor core rendered on a 2D canvas.
|
* JARVIS-style arc reactor core rendered on a 2D canvas.
|
||||||
* Rotating tick rings, segmented arcs and an audio-reactive radial spectrum that reads the
|
* Rotating tick rings, segmented arcs and an audio-reactive radial spectrum that reads the
|
||||||
* real TTS output (or the microphone while listening).
|
* real TTS output (or the microphone while listening).
|
||||||
|
*
|
||||||
|
* Performance notes: no canvas shadows (glow is a second wide low-alpha pass), colour strings
|
||||||
|
* are cached, the spectrum buffer is reused, the theme accent is read once per theme change,
|
||||||
|
* idle runs at 30 fps and the loop stops entirely while the window is hidden.
|
||||||
*/
|
*/
|
||||||
import { useEffect, useRef } from 'react';
|
import { useEffect, useRef } from 'react';
|
||||||
|
import { bridge } from '../../lib/bridge';
|
||||||
import { audioBus } from '../../services/voice/audioBus';
|
import { audioBus } from '../../services/voice/audioBus';
|
||||||
import type { HudState } from '../../state/chat';
|
import type { HudState } from '../../state/chat';
|
||||||
|
|
||||||
@@ -12,12 +17,15 @@ interface Props {
|
|||||||
/** 0..1 override for the microphone level (VAD), used while listening. */
|
/** 0..1 override for the microphone level (VAD), used while listening. */
|
||||||
inputLevel?: number;
|
inputLevel?: number;
|
||||||
reduceMotion?: boolean;
|
reduceMotion?: boolean;
|
||||||
|
/** Bumped by the parent to trigger a short "ping" flash (send, tool start, speech start). */
|
||||||
|
pulse?: number;
|
||||||
className?: string;
|
className?: string;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
type Rgb = [number, number, number];
|
||||||
interface Palette {
|
interface Palette {
|
||||||
main: [number, number, number];
|
main: Rgb;
|
||||||
glow: [number, number, number];
|
glow: Rgb;
|
||||||
}
|
}
|
||||||
|
|
||||||
const PALETTES: Record<HudState, Palette> = {
|
const PALETTES: Record<HudState, Palette> = {
|
||||||
@@ -31,23 +39,30 @@ const PALETTES: Record<HudState, Palette> = {
|
|||||||
};
|
};
|
||||||
|
|
||||||
const BARS = 72;
|
const BARS = 72;
|
||||||
|
const TICKS = 120;
|
||||||
|
const SEGMENTS = 6;
|
||||||
|
|
||||||
function readThemeAccent(): [number, number, number] | null {
|
function readThemeAccent(): Rgb | null {
|
||||||
const raw = getComputedStyle(document.documentElement).getPropertyValue('--accent-rgb').trim();
|
const raw = getComputedStyle(document.documentElement).getPropertyValue('--accent-rgb').trim();
|
||||||
if (!raw) return null;
|
if (!raw) return null;
|
||||||
const parts = raw.split(',').map((p) => Number(p.trim()));
|
const parts = raw.split(',').map((p) => Number(p.trim()));
|
||||||
return parts.length === 3 && parts.every((n) => Number.isFinite(n)) ? (parts as [number, number, number]) : null;
|
return parts.length === 3 && parts.every((n) => Number.isFinite(n)) ? (parts as Rgb) : null;
|
||||||
}
|
}
|
||||||
|
|
||||||
export function JarvisCore({ state, inputLevel = 0, reduceMotion = false, className }: Props) {
|
export function JarvisCore({ state, inputLevel = 0, reduceMotion = false, pulse = 0, className }: Props) {
|
||||||
const canvasRef = useRef<HTMLCanvasElement>(null);
|
const canvasRef = useRef<HTMLCanvasElement>(null);
|
||||||
const stateRef = useRef(state);
|
const stateRef = useRef(state);
|
||||||
const inputRef = useRef(inputLevel);
|
const inputRef = useRef(inputLevel);
|
||||||
const motionRef = useRef(reduceMotion);
|
const motionRef = useRef(reduceMotion);
|
||||||
|
const pulseRef = useRef(0);
|
||||||
stateRef.current = state;
|
stateRef.current = state;
|
||||||
inputRef.current = inputLevel;
|
inputRef.current = inputLevel;
|
||||||
motionRef.current = reduceMotion;
|
motionRef.current = reduceMotion;
|
||||||
|
|
||||||
|
useEffect(() => {
|
||||||
|
if (pulse > 0) pulseRef.current = 1;
|
||||||
|
}, [pulse]);
|
||||||
|
|
||||||
useEffect(() => {
|
useEffect(() => {
|
||||||
const canvas = canvasRef.current;
|
const canvas = canvasRef.current;
|
||||||
if (!canvas) return;
|
if (!canvas) return;
|
||||||
@@ -59,13 +74,33 @@ export function JarvisCore({ state, inputLevel = 0, reduceMotion = false, classN
|
|||||||
let height = 0;
|
let height = 0;
|
||||||
let dpr = 1;
|
let dpr = 1;
|
||||||
const bars = new Float32Array(BARS);
|
const bars = new Float32Array(BARS);
|
||||||
const color = { main: [52, 228, 255] as [number, number, number], glow: [52, 228, 255] as [number, number, number] };
|
const spectrumBuffer = new Float32Array(BARS);
|
||||||
|
const color = { main: [52, 228, 255] as Rgb, glow: [52, 228, 255] as Rgb };
|
||||||
let energy = 0;
|
let energy = 0;
|
||||||
let spin = 0;
|
let spin = 0;
|
||||||
let spin2 = 0;
|
let spin2 = 0;
|
||||||
let spin3 = 0;
|
let spin3 = 0;
|
||||||
let last = performance.now();
|
let last = performance.now();
|
||||||
let visible = true;
|
let visible = document.visibilityState === 'visible';
|
||||||
|
let frame = 0;
|
||||||
|
let accent = readThemeAccent();
|
||||||
|
|
||||||
|
const colorCache = new Map<number, string>();
|
||||||
|
// Quantised rgba cache: colours are quantised to integers and alpha to 1/64 steps.
|
||||||
|
const rgba = (c: Rgb, a: number): string => {
|
||||||
|
const r = c[0] | 0;
|
||||||
|
const g = c[1] | 0;
|
||||||
|
const b = c[2] | 0;
|
||||||
|
const q = Math.max(0, Math.min(64, Math.round(a * 64)));
|
||||||
|
const key = (((r << 8) | g) << 8 | b) * 65 + q;
|
||||||
|
let s = colorCache.get(key);
|
||||||
|
if (!s) {
|
||||||
|
s = `rgba(${r},${g},${b},${(q / 64).toFixed(3)})`;
|
||||||
|
if (colorCache.size > 4000) colorCache.clear();
|
||||||
|
colorCache.set(key, s);
|
||||||
|
}
|
||||||
|
return s;
|
||||||
|
};
|
||||||
|
|
||||||
const resize = () => {
|
const resize = () => {
|
||||||
const rect = canvas.getBoundingClientRect();
|
const rect = canvas.getBoundingClientRect();
|
||||||
@@ -79,41 +114,70 @@ export function JarvisCore({ state, inputLevel = 0, reduceMotion = false, classN
|
|||||||
observer.observe(canvas);
|
observer.observe(canvas);
|
||||||
resize();
|
resize();
|
||||||
|
|
||||||
|
const themeObserver = new MutationObserver(() => {
|
||||||
|
accent = readThemeAccent();
|
||||||
|
});
|
||||||
|
themeObserver.observe(document.documentElement, { attributes: true, attributeFilter: ['data-theme'] });
|
||||||
|
|
||||||
|
const start = () => {
|
||||||
|
if (!raf) {
|
||||||
|
last = performance.now();
|
||||||
|
raf = requestAnimationFrame(draw);
|
||||||
|
}
|
||||||
|
};
|
||||||
const onVisibility = () => {
|
const onVisibility = () => {
|
||||||
visible = document.visibilityState === 'visible';
|
visible = document.visibilityState === 'visible';
|
||||||
|
if (visible) start();
|
||||||
};
|
};
|
||||||
document.addEventListener('visibilitychange', onVisibility);
|
document.addEventListener('visibilitychange', onVisibility);
|
||||||
|
// backgroundThrottling is off, so the Page Visibility API never fires: main tells us instead.
|
||||||
|
const offWindow = bridge()?.window.onVisibility?.((shown) => {
|
||||||
|
visible = shown;
|
||||||
|
if (visible) start();
|
||||||
|
});
|
||||||
|
|
||||||
const lerp = (a: number, b: number, t: number) => a + (b - a) * t;
|
const lerp = (a: number, b: number, t: number) => a + (b - a) * t;
|
||||||
const rgba = (c: [number, number, number], a: number) => `rgba(${c[0]},${c[1]},${c[2]},${a})`;
|
|
||||||
|
|
||||||
const draw = (now: number) => {
|
const draw = (now: number) => {
|
||||||
|
if (!visible) {
|
||||||
|
raf = 0;
|
||||||
|
return;
|
||||||
|
}
|
||||||
raf = requestAnimationFrame(draw);
|
raf = requestAnimationFrame(draw);
|
||||||
if (!visible) return;
|
const st = stateRef.current;
|
||||||
|
frame++;
|
||||||
|
// Idle economy: 30 fps is plenty for the breathing animation.
|
||||||
|
if (st === 'idle' && pulseRef.current < 0.01 && frame % 2 === 1) return;
|
||||||
|
|
||||||
const dt = Math.min(0.05, (now - last) / 1000);
|
const dt = Math.min(0.05, (now - last) / 1000);
|
||||||
last = now;
|
last = now;
|
||||||
const st = stateRef.current;
|
|
||||||
const motion = motionRef.current ? 0.25 : 1;
|
const motion = motionRef.current ? 0.25 : 1;
|
||||||
|
|
||||||
// ── target colours & energy
|
|
||||||
const target = PALETTES[st];
|
const target = PALETTES[st];
|
||||||
const themeAccent = st === 'idle' || st === 'listening' || st === 'speaking' ? readThemeAccent() : null;
|
const themeAccent = st === 'idle' || st === 'listening' || st === 'speaking' ? accent : null;
|
||||||
const mainTarget = themeAccent ?? target.main;
|
const mainTarget = themeAccent ?? target.main;
|
||||||
|
const glowTarget = themeAccent ?? target.glow;
|
||||||
for (let i = 0; i < 3; i++) {
|
for (let i = 0; i < 3; i++) {
|
||||||
color.main[i] = lerp(color.main[i], mainTarget[i], 0.08);
|
color.main[i] = lerp(color.main[i], mainTarget[i], 0.08);
|
||||||
color.glow[i] = lerp(color.glow[i], (themeAccent ?? target.glow)[i], 0.08);
|
color.glow[i] = lerp(color.glow[i], glowTarget[i], 0.08);
|
||||||
}
|
}
|
||||||
const speaking = st === 'speaking';
|
const speaking = st === 'speaking';
|
||||||
const listening = st === 'listening';
|
const listening = st === 'listening';
|
||||||
const spectrum = speaking ? audioBus.spectrum(BARS, 'output') : listening ? audioBus.spectrum(BARS, 'input') : null;
|
const spectrum = speaking
|
||||||
|
? audioBus.spectrum(BARS, 'output', spectrumBuffer)
|
||||||
|
: listening
|
||||||
|
? audioBus.spectrum(BARS, 'input', spectrumBuffer)
|
||||||
|
: null;
|
||||||
const level = speaking ? audioBus.outputLevel() : listening ? Math.max(inputRef.current, audioBus.inputLevel()) : 0;
|
const level = speaking ? audioBus.outputLevel() : listening ? Math.max(inputRef.current, audioBus.inputLevel()) : 0;
|
||||||
const targetEnergy = st === 'thinking' ? 0.45 : st === 'alert' || st === 'error' ? 0.6 : level;
|
pulseRef.current = Math.max(0, pulseRef.current - dt * 2.2);
|
||||||
energy = lerp(energy, targetEnergy, speaking || listening ? 0.35 : 0.08);
|
const ping = pulseRef.current;
|
||||||
|
const targetEnergy = Math.max(ping, st === 'thinking' ? 0.45 : st === 'alert' || st === 'error' ? 0.6 : level);
|
||||||
|
energy = lerp(energy, targetEnergy, speaking || listening || ping > 0 ? 0.35 : 0.08);
|
||||||
for (let i = 0; i < BARS; i++) {
|
for (let i = 0; i < BARS; i++) {
|
||||||
let t = 0;
|
let t = 0;
|
||||||
if (spectrum) {
|
if (spectrum) {
|
||||||
const v = spectrum[i];
|
const v = spectrum[i];
|
||||||
t = Math.pow(v, 1.4);
|
t = v * Math.sqrt(v);
|
||||||
if (listening) t = Math.max(t, inputRef.current * (0.5 + 0.5 * Math.sin(now / 90 + i)));
|
if (listening) t = Math.max(t, inputRef.current * (0.5 + 0.5 * Math.sin(now / 90 + i)));
|
||||||
} else if (st === 'thinking') {
|
} else if (st === 'thinking') {
|
||||||
t = 0.15 + 0.12 * Math.sin(now / 260 + i * 0.35);
|
t = 0.15 + 0.12 * Math.sin(now / 260 + i * 0.35);
|
||||||
@@ -122,10 +186,10 @@ export function JarvisCore({ state, inputLevel = 0, reduceMotion = false, classN
|
|||||||
} else {
|
} else {
|
||||||
t = 0.12 + 0.08 * Math.sin(now / 140 + i * 0.7);
|
t = 0.12 + 0.08 * Math.sin(now / 140 + i * 0.7);
|
||||||
}
|
}
|
||||||
|
t = Math.max(t, ping * 0.6);
|
||||||
bars[i] = lerp(bars[i], t, t > bars[i] ? 0.5 : 0.12);
|
bars[i] = lerp(bars[i], t, t > bars[i] ? 0.5 : 0.12);
|
||||||
}
|
}
|
||||||
|
|
||||||
// ── rotation
|
|
||||||
const speed = (st === 'thinking' ? 1.6 : st === 'alert' || st === 'error' ? 1.2 : speaking ? 0.7 : 0.35) * motion;
|
const speed = (st === 'thinking' ? 1.6 : st === 'alert' || st === 'error' ? 1.2 : speaking ? 0.7 : 0.35) * motion;
|
||||||
spin += dt * 0.35 * speed;
|
spin += dt * 0.35 * speed;
|
||||||
spin2 -= dt * 0.55 * speed;
|
spin2 -= dt * 0.55 * speed;
|
||||||
@@ -139,7 +203,7 @@ export function JarvisCore({ state, inputLevel = 0, reduceMotion = false, classN
|
|||||||
const main = color.main;
|
const main = color.main;
|
||||||
const glow = color.glow;
|
const glow = color.glow;
|
||||||
|
|
||||||
// ── ambient glow
|
// ambient glow
|
||||||
const halo = ctx.createRadialGradient(cx, cy, R * 0.1, cx, cy, R * 1.05);
|
const halo = ctx.createRadialGradient(cx, cy, R * 0.1, cx, cy, R * 1.05);
|
||||||
halo.addColorStop(0, rgba(glow, 0.18 + energy * 0.25));
|
halo.addColorStop(0, rgba(glow, 0.18 + energy * 0.25));
|
||||||
halo.addColorStop(0.5, rgba(glow, 0.05));
|
halo.addColorStop(0.5, rgba(glow, 0.05));
|
||||||
@@ -147,53 +211,51 @@ export function JarvisCore({ state, inputLevel = 0, reduceMotion = false, classN
|
|||||||
ctx.fillStyle = halo;
|
ctx.fillStyle = halo;
|
||||||
ctx.fillRect(0, 0, width, height);
|
ctx.fillRect(0, 0, width, height);
|
||||||
|
|
||||||
// ── outer tick ring
|
// outer tick ring: three batched paths (long / mid / short ticks)
|
||||||
ctx.save();
|
ctx.save();
|
||||||
ctx.translate(cx, cy);
|
ctx.translate(cx, cy);
|
||||||
ctx.rotate(spin);
|
ctx.rotate(spin);
|
||||||
ctx.strokeStyle = rgba(main, 0.55);
|
|
||||||
ctx.lineWidth = 1;
|
ctx.lineWidth = 1;
|
||||||
for (let i = 0; i < 120; i++) {
|
for (let group = 0; group < 3; group++) {
|
||||||
const a = (i / 120) * Math.PI * 2;
|
|
||||||
const long = i % 10 === 0;
|
|
||||||
const mid = i % 5 === 0;
|
|
||||||
const r0 = R * (long ? 0.93 : mid ? 0.955 : 0.97);
|
|
||||||
ctx.globalAlpha = long ? 0.9 : mid ? 0.6 : 0.3;
|
|
||||||
ctx.beginPath();
|
ctx.beginPath();
|
||||||
ctx.moveTo(Math.cos(a) * r0, Math.sin(a) * r0);
|
for (let i = 0; i < TICKS; i++) {
|
||||||
ctx.lineTo(Math.cos(a) * R, Math.sin(a) * R);
|
const kind = i % 10 === 0 ? 0 : i % 5 === 0 ? 1 : 2;
|
||||||
|
if (kind !== group) continue;
|
||||||
|
const a = (i / TICKS) * Math.PI * 2;
|
||||||
|
const r0 = R * (kind === 0 ? 0.93 : kind === 1 ? 0.955 : 0.97);
|
||||||
|
ctx.moveTo(Math.cos(a) * r0, Math.sin(a) * r0);
|
||||||
|
ctx.lineTo(Math.cos(a) * R, Math.sin(a) * R);
|
||||||
|
}
|
||||||
|
ctx.strokeStyle = rgba(main, group === 0 ? 0.5 : group === 1 ? 0.33 : 0.17);
|
||||||
ctx.stroke();
|
ctx.stroke();
|
||||||
}
|
}
|
||||||
ctx.globalAlpha = 1;
|
|
||||||
ctx.beginPath();
|
ctx.beginPath();
|
||||||
ctx.arc(0, 0, R, 0, Math.PI * 2);
|
ctx.arc(0, 0, R, 0, Math.PI * 2);
|
||||||
ctx.strokeStyle = rgba(main, 0.35);
|
ctx.strokeStyle = rgba(main, 0.35);
|
||||||
ctx.stroke();
|
ctx.stroke();
|
||||||
ctx.restore();
|
ctx.restore();
|
||||||
|
|
||||||
// ── segmented arc ring
|
// segmented arc ring
|
||||||
ctx.save();
|
ctx.save();
|
||||||
ctx.translate(cx, cy);
|
ctx.translate(cx, cy);
|
||||||
ctx.rotate(spin2);
|
ctx.rotate(spin2);
|
||||||
ctx.lineCap = 'butt';
|
for (let i = 0; i < SEGMENTS; i++) {
|
||||||
const segments = 6;
|
const startAngle = (i / SEGMENTS) * Math.PI * 2;
|
||||||
for (let i = 0; i < segments; i++) {
|
const span = (Math.PI * 2) / SEGMENTS - 0.22;
|
||||||
const start = (i / segments) * Math.PI * 2;
|
|
||||||
const span = (Math.PI * 2) / segments - 0.22;
|
|
||||||
ctx.beginPath();
|
ctx.beginPath();
|
||||||
ctx.arc(0, 0, R * 0.86, start, start + span);
|
ctx.arc(0, 0, R * 0.86, startAngle, startAngle + span);
|
||||||
ctx.lineWidth = 6;
|
ctx.lineWidth = 6;
|
||||||
ctx.strokeStyle = rgba(main, 0.16 + (i % 2 === 0 ? 0.18 : 0.05) + energy * 0.2);
|
ctx.strokeStyle = rgba(main, 0.16 + (i % 2 === 0 ? 0.18 : 0.05) + energy * 0.2);
|
||||||
ctx.stroke();
|
ctx.stroke();
|
||||||
ctx.beginPath();
|
ctx.beginPath();
|
||||||
ctx.arc(0, 0, R * 0.82, start + 0.08, start + span * 0.45);
|
ctx.arc(0, 0, R * 0.82, startAngle + 0.08, startAngle + span * 0.45);
|
||||||
ctx.lineWidth = 2;
|
ctx.lineWidth = 2;
|
||||||
ctx.strokeStyle = rgba(main, 0.75);
|
ctx.strokeStyle = rgba(main, 0.75);
|
||||||
ctx.stroke();
|
ctx.stroke();
|
||||||
}
|
}
|
||||||
ctx.restore();
|
ctx.restore();
|
||||||
|
|
||||||
// ── dashed ring
|
// dashed ring
|
||||||
ctx.save();
|
ctx.save();
|
||||||
ctx.translate(cx, cy);
|
ctx.translate(cx, cy);
|
||||||
ctx.rotate(spin3);
|
ctx.rotate(spin3);
|
||||||
@@ -206,36 +268,46 @@ export function JarvisCore({ state, inputLevel = 0, reduceMotion = false, classN
|
|||||||
ctx.setLineDash([]);
|
ctx.setLineDash([]);
|
||||||
ctx.restore();
|
ctx.restore();
|
||||||
|
|
||||||
// ── audio spectrum bars
|
// audio spectrum bars: one wide low-alpha glow pass, then one crisp pass per intensity bucket
|
||||||
ctx.save();
|
ctx.save();
|
||||||
ctx.translate(cx, cy);
|
ctx.translate(cx, cy);
|
||||||
ctx.lineCap = 'round';
|
ctx.lineCap = 'round';
|
||||||
const inner = R * 0.5;
|
const inner = R * 0.5;
|
||||||
const maxLen = R * 0.16;
|
const maxLen = R * 0.16;
|
||||||
|
ctx.lineWidth = 7;
|
||||||
|
ctx.strokeStyle = rgba(glow, 0.1 + energy * 0.12);
|
||||||
|
ctx.beginPath();
|
||||||
for (let i = 0; i < BARS; i++) {
|
for (let i = 0; i < BARS; i++) {
|
||||||
const a = (i / BARS) * Math.PI * 2 - Math.PI / 2;
|
const a = (i / BARS) * Math.PI * 2 - Math.PI / 2;
|
||||||
const len = 2 + bars[i] * maxLen;
|
const len = 2 + bars[i] * maxLen;
|
||||||
const x0 = Math.cos(a) * inner;
|
ctx.moveTo(Math.cos(a) * inner, Math.sin(a) * inner);
|
||||||
const y0 = Math.sin(a) * inner;
|
ctx.lineTo(Math.cos(a) * (inner + len), Math.sin(a) * (inner + len));
|
||||||
const x1 = Math.cos(a) * (inner + len);
|
}
|
||||||
const y1 = Math.sin(a) * (inner + len);
|
ctx.stroke();
|
||||||
ctx.lineWidth = 2.4;
|
ctx.lineWidth = 2.4;
|
||||||
ctx.strokeStyle = rgba(main, 0.35 + bars[i] * 0.65);
|
for (let bucket = 0; bucket < 4; bucket++) {
|
||||||
ctx.shadowBlur = 8 * bars[i];
|
|
||||||
ctx.shadowColor = rgba(glow, 0.8);
|
|
||||||
ctx.beginPath();
|
ctx.beginPath();
|
||||||
ctx.moveTo(x0, y0);
|
let any = false;
|
||||||
ctx.lineTo(x1, y1);
|
for (let i = 0; i < BARS; i++) {
|
||||||
|
const b = Math.min(3, Math.floor(bars[i] * 4));
|
||||||
|
if (b !== bucket) continue;
|
||||||
|
any = true;
|
||||||
|
const a = (i / BARS) * Math.PI * 2 - Math.PI / 2;
|
||||||
|
const len = 2 + bars[i] * maxLen;
|
||||||
|
ctx.moveTo(Math.cos(a) * inner, Math.sin(a) * inner);
|
||||||
|
ctx.lineTo(Math.cos(a) * (inner + len), Math.sin(a) * (inner + len));
|
||||||
|
}
|
||||||
|
if (!any) continue;
|
||||||
|
ctx.strokeStyle = rgba(main, 0.35 + (bucket + 0.5) * 0.16);
|
||||||
ctx.stroke();
|
ctx.stroke();
|
||||||
}
|
}
|
||||||
ctx.shadowBlur = 0;
|
|
||||||
ctx.restore();
|
ctx.restore();
|
||||||
|
|
||||||
// ── reactor core
|
// reactor core
|
||||||
const coreR = R * 0.36;
|
const coreR = R * 0.36;
|
||||||
const breath = 1 + Math.sin(now / 700) * 0.02 + energy * 0.08;
|
const breath = 1 + Math.sin(now / 700) * 0.02 + energy * 0.08;
|
||||||
const core = ctx.createRadialGradient(cx, cy, 0, cx, cy, coreR * breath);
|
const core = ctx.createRadialGradient(cx, cy, 0, cx, cy, coreR * breath);
|
||||||
core.addColorStop(0, `rgba(255,255,255,${0.75 + energy * 0.25})`);
|
core.addColorStop(0, `rgba(255,255,255,${(0.75 + energy * 0.25).toFixed(3)})`);
|
||||||
core.addColorStop(0.25, rgba(main, 0.55 + energy * 0.3));
|
core.addColorStop(0.25, rgba(main, 0.55 + energy * 0.3));
|
||||||
core.addColorStop(0.7, rgba(glow, 0.12));
|
core.addColorStop(0.7, rgba(glow, 0.12));
|
||||||
core.addColorStop(1, 'rgba(0,0,0,0)');
|
core.addColorStop(1, 'rgba(0,0,0,0)');
|
||||||
@@ -253,7 +325,6 @@ export function JarvisCore({ state, inputLevel = 0, reduceMotion = false, classN
|
|||||||
ctx.strokeStyle = rgba(main, 0.7 - i * 0.2);
|
ctx.strokeStyle = rgba(main, 0.7 - i * 0.2);
|
||||||
ctx.stroke();
|
ctx.stroke();
|
||||||
}
|
}
|
||||||
// reactor triangle
|
|
||||||
ctx.rotate(-spin2 * 0.5);
|
ctx.rotate(-spin2 * 0.5);
|
||||||
ctx.beginPath();
|
ctx.beginPath();
|
||||||
for (let i = 0; i < 3; i++) {
|
for (let i = 0; i < 3; i++) {
|
||||||
@@ -268,24 +339,28 @@ export function JarvisCore({ state, inputLevel = 0, reduceMotion = false, classN
|
|||||||
ctx.stroke();
|
ctx.stroke();
|
||||||
ctx.restore();
|
ctx.restore();
|
||||||
|
|
||||||
// ── thinking orbiters
|
// thinking orbiters (radial gradient dots instead of shadow blur)
|
||||||
if (st === 'thinking' || st === 'alert') {
|
if (st === 'thinking' || st === 'alert') {
|
||||||
ctx.save();
|
ctx.save();
|
||||||
ctx.translate(cx, cy);
|
ctx.translate(cx, cy);
|
||||||
for (let i = 0; i < 3; i++) {
|
for (let i = 0; i < 3; i++) {
|
||||||
const a = spin3 * 2 + (i / 3) * Math.PI * 2;
|
const a = spin3 * 2 + (i / 3) * Math.PI * 2;
|
||||||
const r = R * 0.78;
|
const r = R * 0.78;
|
||||||
|
const x = Math.cos(a) * r;
|
||||||
|
const y = Math.sin(a) * r;
|
||||||
|
const dot = ctx.createRadialGradient(x, y, 0, x, y, 11);
|
||||||
|
dot.addColorStop(0, rgba(main, 1));
|
||||||
|
dot.addColorStop(0.35, rgba(glow, 0.6));
|
||||||
|
dot.addColorStop(1, 'rgba(0,0,0,0)');
|
||||||
|
ctx.fillStyle = dot;
|
||||||
ctx.beginPath();
|
ctx.beginPath();
|
||||||
ctx.arc(Math.cos(a) * r, Math.sin(a) * r, 3.5, 0, Math.PI * 2);
|
ctx.arc(x, y, 11, 0, Math.PI * 2);
|
||||||
ctx.fillStyle = rgba(main, 0.95);
|
|
||||||
ctx.shadowBlur = 14;
|
|
||||||
ctx.shadowColor = rgba(glow, 1);
|
|
||||||
ctx.fill();
|
ctx.fill();
|
||||||
}
|
}
|
||||||
ctx.restore();
|
ctx.restore();
|
||||||
}
|
}
|
||||||
|
|
||||||
// ── listening ring (input level)
|
// listening ring (input level)
|
||||||
if (listening) {
|
if (listening) {
|
||||||
ctx.save();
|
ctx.save();
|
||||||
ctx.translate(cx, cy);
|
ctx.translate(cx, cy);
|
||||||
@@ -298,11 +373,14 @@ export function JarvisCore({ state, inputLevel = 0, reduceMotion = false, classN
|
|||||||
}
|
}
|
||||||
};
|
};
|
||||||
|
|
||||||
raf = requestAnimationFrame(draw);
|
start();
|
||||||
return () => {
|
return () => {
|
||||||
cancelAnimationFrame(raf);
|
if (raf) cancelAnimationFrame(raf);
|
||||||
|
raf = 0;
|
||||||
observer.disconnect();
|
observer.disconnect();
|
||||||
|
themeObserver.disconnect();
|
||||||
document.removeEventListener('visibilitychange', onVisibility);
|
document.removeEventListener('visibilitychange', onVisibility);
|
||||||
|
offWindow?.();
|
||||||
};
|
};
|
||||||
}, []);
|
}, []);
|
||||||
|
|
||||||
|
|||||||
@@ -1,4 +1,4 @@
|
|||||||
import { Minus, X, Settings, PictureInPicture2, RefreshCw, Webhook } from 'lucide-react';
|
import { Minus, X, Settings, PictureInPicture2, RefreshCw, Webhook, PanelRight } from 'lucide-react';
|
||||||
import { bridge } from '../../lib/bridge';
|
import { bridge } from '../../lib/bridge';
|
||||||
import { useClock } from '../../hooks/useMetrics';
|
import { useClock } from '../../hooks/useMetrics';
|
||||||
import { useHermes } from '../../state/hermes';
|
import { useHermes } from '../../state/hermes';
|
||||||
@@ -6,9 +6,11 @@ import { useSettings } from '../../state/settings';
|
|||||||
|
|
||||||
interface Props {
|
interface Props {
|
||||||
onOpenSettings: () => void;
|
onOpenSettings: () => void;
|
||||||
|
onToggleOps?: () => void;
|
||||||
|
opsBadge?: number;
|
||||||
}
|
}
|
||||||
|
|
||||||
export function TopBar({ onOpenSettings }: Props) {
|
export function TopBar({ onOpenSettings, onToggleOps, opsBadge = 0 }: Props) {
|
||||||
const now = useClock();
|
const now = useClock();
|
||||||
const link = useHermes((s) => s.link);
|
const link = useHermes((s) => s.link);
|
||||||
const linkDetail = useHermes((s) => s.linkDetail);
|
const linkDetail = useHermes((s) => s.linkDetail);
|
||||||
@@ -55,9 +57,15 @@ export function TopBar({ onOpenSettings }: Props) {
|
|||||||
<RefreshCw size={14} className={link === 'checking' ? 'spin' : undefined} />
|
<RefreshCw size={14} className={link === 'checking' ? 'spin' : undefined} />
|
||||||
</button>
|
</button>
|
||||||
</div>
|
</div>
|
||||||
<span className="clock">{now.toLocaleTimeString([], { hour: '2-digit', minute: '2-digit', second: '2-digit', hour12: false })}</span>
|
<span className="clock">{now.toLocaleTimeString([], { hour: '2-digit', minute: '2-digit', hour12: false })}</span>
|
||||||
<div className="controls">
|
<div className="controls">
|
||||||
<button className="icon-btn" title="Paramètres" onClick={onOpenSettings}><Settings size={16} /></button>
|
{onToggleOps && (
|
||||||
|
<button className="icon-btn ops-toggle" title="Panneau Hermes Ops" aria-label="Panneau Hermes Ops" onClick={onToggleOps} style={{ position: 'relative' }}>
|
||||||
|
<PanelRight size={16} />
|
||||||
|
{opsBadge > 0 && <span className="badge" style={{ position: 'absolute', top: 2, right: 2, minWidth: 14, height: 14, fontSize: 9, borderRadius: 7, background: 'var(--danger)', color: '#fff', display: 'inline-flex', alignItems: 'center', justifyContent: 'center', padding: '0 3px' }}>{opsBadge}</span>}
|
||||||
|
</button>
|
||||||
|
)}
|
||||||
|
<button className="icon-btn" title="Paramètres (Ctrl+,)" aria-label="Paramètres" onClick={onOpenSettings}><Settings size={16} /></button>
|
||||||
{api && (
|
{api && (
|
||||||
<>
|
<>
|
||||||
<button className="icon-btn" title="Mode compact flottant" onClick={() => api.window.setMode('compact')}><PictureInPicture2 size={16} /></button>
|
<button className="icon-btn" title="Mode compact flottant" onClick={() => api.window.setMode('compact')}><PictureInPicture2 size={16} /></button>
|
||||||
|
|||||||
@@ -1,4 +1,5 @@
|
|||||||
import { useEffect, useState } from 'react';
|
import { Fragment, useEffect, useState } from 'react';
|
||||||
|
import { useShallow } from 'zustand/react/shallow';
|
||||||
import {
|
import {
|
||||||
Wrench, CalendarClock, Sparkles, Layers, History, Play, Pause, Trash2, Pencil, Plus, Save, X, RefreshCw,
|
Wrench, CalendarClock, Sparkles, Layers, History, Play, Pause, Trash2, Pencil, Plus, Save, X, RefreshCw,
|
||||||
CheckCircle2, XCircle, Loader2, Bot, Cpu, GitBranch, Webhook, FolderOpen
|
CheckCircle2, XCircle, Loader2, Bot, Cpu, GitBranch, Webhook, FolderOpen
|
||||||
@@ -142,7 +143,10 @@ function HistoryList() {
|
|||||||
<div
|
<div
|
||||||
key={r.id}
|
key={r.id}
|
||||||
className="row-item clickable"
|
className="row-item clickable"
|
||||||
|
role="button"
|
||||||
|
tabIndex={0}
|
||||||
style={{ opacity: r.read ? 0.7 : 1 }}
|
style={{ opacity: r.read ? 0.7 : 1 }}
|
||||||
|
onKeyDown={(e) => { if (e.key === 'Enter') (e.currentTarget as HTMLDivElement).click(); }}
|
||||||
onClick={() => {
|
onClick={() => {
|
||||||
markRead(r.id);
|
markRead(r.id);
|
||||||
addMessage({ role: 'assistant', content: r.output, source: 'cron', jobName: r.jobName, status: 'done', timestamp: new Date(r.at).getTime() || Date.now() });
|
addMessage({ role: 'assistant', content: r.output, source: 'cron', jobName: r.jobName, status: 'done', timestamp: new Date(r.at).getTime() || Date.now() });
|
||||||
@@ -222,7 +226,7 @@ function SessionsTab() {
|
|||||||
const transport = useHermes((s) => s.transport);
|
const transport = useHermes((s) => s.transport);
|
||||||
const sessionId = useSettings((s) => s.settings.hermesSessionId);
|
const sessionId = useSettings((s) => s.settings.hermesSessionId);
|
||||||
const setSessionId = useSettings((s) => s.setHermesSessionId);
|
const setSessionId = useSettings((s) => s.setHermesSessionId);
|
||||||
const chat = useChat();
|
const chat = useChat.getState();
|
||||||
|
|
||||||
const activeId = sessionId.replace(/^hs:/, '');
|
const activeId = sessionId.replace(/^hs:/, '');
|
||||||
const load = async (id: string) => {
|
const load = async (id: string) => {
|
||||||
@@ -255,7 +259,7 @@ function SessionsTab() {
|
|||||||
{sessions.length === 0 ? <div className="empty">Aucune session listée par Hermes.</div> : (
|
{sessions.length === 0 ? <div className="empty">Aucune session listée par Hermes.</div> : (
|
||||||
<div className="list">
|
<div className="list">
|
||||||
{sessions.map((s) => (
|
{sessions.map((s) => (
|
||||||
<div key={s.id} className={`row-item clickable`} style={s.id === activeId ? { borderColor: 'var(--accent)' } : undefined} onClick={() => void load(s.id)}>
|
<div key={s.id} className="row-item clickable" role="button" tabIndex={0} style={s.id === activeId ? { borderColor: 'var(--accent)' } : undefined} onClick={() => void load(s.id)} onKeyDown={(e) => { if (e.key === 'Enter') void load(s.id); }}>
|
||||||
<div className="main">
|
<div className="main">
|
||||||
<span className="name">{s.title || s.id}</span>
|
<span className="name">{s.title || s.id}</span>
|
||||||
<span className="meta">{[s.platform ?? s.source, s.message_count ? `${s.message_count} msg` : null, formatDateTime(s.updated_at ?? s.created_at)].filter(Boolean).join(' · ')}</span>
|
<span className="meta">{[s.platform ?? s.source, s.message_count ? `${s.message_count} msg` : null, formatDateTime(s.updated_at ?? s.created_at)].filter(Boolean).join(' · ')}</span>
|
||||||
@@ -273,7 +277,9 @@ function SessionsTab() {
|
|||||||
}
|
}
|
||||||
|
|
||||||
function LinkTab() {
|
function LinkTab() {
|
||||||
const { link, linkDetail, health, capabilities, transport, models, webhook } = useHermes();
|
const { link, linkDetail, health, capabilities, transport, models, webhook } = useHermes(
|
||||||
|
useShallow((s) => ({ link: s.link, linkDetail: s.linkDetail, health: s.health, capabilities: s.capabilities, transport: s.transport, models: s.models, webhook: s.webhook }))
|
||||||
|
);
|
||||||
const connect = useHermes((s) => s.connect);
|
const connect = useHermes((s) => s.connect);
|
||||||
const hermes = useSettings((s) => s.settings.hermes);
|
const hermes = useSettings((s) => s.settings.hermes);
|
||||||
const [info, setInfo] = useState<{ sharedDir: string; logPath: string; version: string } | null>(null);
|
const [info, setInfo] = useState<{ sharedDir: string; logPath: string; version: string } | null>(null);
|
||||||
@@ -311,7 +317,7 @@ function LinkTab() {
|
|||||||
<div className="section-title"><CheckCircle2 size={12} /> Santé</div>
|
<div className="section-title"><CheckCircle2 size={12} /> Santé</div>
|
||||||
<dl className="kv">
|
<dl className="kv">
|
||||||
{Object.entries(checks).map(([k, v]) => (
|
{Object.entries(checks).map(([k, v]) => (
|
||||||
<><dt key={`${k}-t`}>{k}</dt><dd key={`${k}-d`}>{typeof v === 'object' ? JSON.stringify(v) : String(v)}</dd></>
|
<Fragment key={k}><dt>{k}</dt><dd>{typeof v === 'object' ? JSON.stringify(v) : String(v)}</dd></Fragment>
|
||||||
))}
|
))}
|
||||||
</dl>
|
</dl>
|
||||||
</>
|
</>
|
||||||
|
|||||||
@@ -0,0 +1,120 @@
|
|||||||
|
import { useEffect } from 'react';
|
||||||
|
import { Download, Trash2, X, CheckCircle2, Cpu, Mic, Volume2, AlertTriangle, RefreshCw, Ear, Activity } from 'lucide-react';
|
||||||
|
import type { VoiceModelStatus } from '../../../shared/voice';
|
||||||
|
import { bridge } from '../../lib/bridge';
|
||||||
|
import { useShallow } from 'zustand/react/shallow';
|
||||||
|
import { useVoiceModels } from '../../state/voiceModels';
|
||||||
|
import { useSettings } from '../../state/settings';
|
||||||
|
|
||||||
|
function formatMb(bytes: number): string {
|
||||||
|
return bytes >= 1024 * 1024 * 1024 ? `${(bytes / 1024 ** 3).toFixed(2)} Go` : `${Math.round(bytes / 1024 / 1024)} Mo`;
|
||||||
|
}
|
||||||
|
|
||||||
|
function ModelRow({ model }: { model: VoiceModelStatus }) {
|
||||||
|
const progress = useVoiceModels((s) => s.progress[model.id]);
|
||||||
|
const { download, cancel, remove } = useVoiceModels.getState();
|
||||||
|
const activeStt = useSettings((s) => s.settings.voice.localModel);
|
||||||
|
const activeTts = useSettings((s) => s.settings.speech.localModel);
|
||||||
|
const update = useSettings((s) => s.update);
|
||||||
|
const isActive = model.kind === 'stt' ? activeStt === model.id : activeTts === model.id;
|
||||||
|
const busy = !!progress && (progress.phase === 'download' || progress.phase === 'extract');
|
||||||
|
|
||||||
|
const wakeMode = useSettings((s) => s.settings.voice.wakeMode);
|
||||||
|
const activate = () => {
|
||||||
|
if (model.kind === 'stt') update({ voice: { localModel: model.id, provider: 'local' } });
|
||||||
|
else if (model.kind === 'kws') update({ voice: { wakeMode: 'kws' } });
|
||||||
|
else if (model.kind === 'vad') update({ voice: { neuralVad: true } });
|
||||||
|
else update({ speech: { localModel: model.id, provider: 'local', localSpeaker: model.speakers?.[0]?.id ?? 0 } });
|
||||||
|
};
|
||||||
|
const neuralVad = useSettings((s) => s.settings.voice.neuralVad);
|
||||||
|
const activeNow = model.kind === 'kws' ? wakeMode === 'kws' : model.kind === 'vad' ? neuralVad : isActive;
|
||||||
|
|
||||||
|
return (
|
||||||
|
<div className="row-item" style={activeNow && model.installed ? { borderColor: 'var(--accent)' } : undefined}>
|
||||||
|
<div className="main">
|
||||||
|
<div className="row wrap" style={{ gap: 6 }}>
|
||||||
|
<span className="name">{model.name}</span>
|
||||||
|
{model.recommended && <span className="status-pill ok">recommandé</span>}
|
||||||
|
{model.installed && <span className="status-pill online">installé · {formatMb(model.installedBytes)}</span>}
|
||||||
|
{!model.installed && !busy && <span className="status-pill">{model.sizeMb} Mo</span>}
|
||||||
|
{activeNow && model.installed && <span className="status-pill active">actif</span>}
|
||||||
|
</div>
|
||||||
|
<span className="desc">{model.description}</span>
|
||||||
|
<span className="meta">langues : {model.languages.join(', ')}{model.speakers ? ` · ${model.speakers.length} voix` : ''}</span>
|
||||||
|
{busy && (
|
||||||
|
<div style={{ marginTop: 6 }}>
|
||||||
|
<div className="level-meter"><div style={{ width: `${progress.percent}%` }} /></div>
|
||||||
|
<span className="meta">{progress.phase === 'extract' ? 'extraction…' : `${progress.percent}% · ${formatMb(progress.received)} / ${formatMb(progress.total)}`}</span>
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
{progress?.phase === 'error' && <span className="meta" style={{ color: 'var(--danger)' }}>échec : {progress.message}</span>}
|
||||||
|
</div>
|
||||||
|
<div className="actions">
|
||||||
|
{busy ? (
|
||||||
|
<button className="icon-btn danger" title="Annuler" onClick={() => void cancel(model.id)}><X size={13} /></button>
|
||||||
|
) : model.installed ? (
|
||||||
|
<>
|
||||||
|
{!activeNow && <button className="btn small" onClick={activate}>Utiliser</button>}
|
||||||
|
<button className="icon-btn danger" title="Supprimer du disque" onClick={() => { if (confirm(`Supprimer « ${model.name} » ?`)) void remove(model.id); }}><Trash2 size={13} /></button>
|
||||||
|
</>
|
||||||
|
) : (
|
||||||
|
<button className="btn small primary" onClick={() => void download(model.id)}><Download size={13} /> Télécharger</button>
|
||||||
|
)}
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
export function ModelsSection() {
|
||||||
|
const { models, engine, error } = useVoiceModels(useShallow((s) => ({ models: s.models, engine: s.engine, error: s.error })));
|
||||||
|
const { refresh, checkEngine } = useVoiceModels.getState();
|
||||||
|
useEffect(() => {
|
||||||
|
void refresh();
|
||||||
|
void checkEngine();
|
||||||
|
}, [refresh, checkEngine]);
|
||||||
|
|
||||||
|
if (!bridge()) return <div className="card"><div className="empty">Les modèles locaux ne sont disponibles que dans l’application Electron.</div></div>;
|
||||||
|
|
||||||
|
const stt = models.filter((m) => m.kind === 'stt');
|
||||||
|
const tts = models.filter((m) => m.kind === 'tts');
|
||||||
|
const kws = models.filter((m) => m.kind === 'kws');
|
||||||
|
const vad = models.filter((m) => m.kind === 'vad');
|
||||||
|
return (
|
||||||
|
<>
|
||||||
|
<div className="card">
|
||||||
|
<div className="section-title"><Cpu size={12} /> Moteur local (sherpa-onnx)</div>
|
||||||
|
{engine === null ? (
|
||||||
|
<span className="muted">vérification…</span>
|
||||||
|
) : engine.available ? (
|
||||||
|
<div className="test-result ok"><CheckCircle2 size={13} /> moteur prêt · version {engine.version}{engine.loaded.length ? ` · chargé : ${engine.loaded.join(', ')}` : ''}</div>
|
||||||
|
) : (
|
||||||
|
<div className="test-result fail"><AlertTriangle size={13} /> {engine.error}</div>
|
||||||
|
)}
|
||||||
|
<span className="hint" style={{ display: 'block', marginTop: 8 }}>
|
||||||
|
Les modèles tournent sur le processeur, sans connexion, dans un processus séparé. Dossier : <code>{engine?.modelsDir}</code>
|
||||||
|
</span>
|
||||||
|
{error && <div className="test-result fail" style={{ marginTop: 8 }}><AlertTriangle size={13} /> {error}</div>}
|
||||||
|
<div className="row" style={{ marginTop: 8 }}>
|
||||||
|
<button className="btn small ghost" onClick={() => { void refresh(); void checkEngine(); }}><RefreshCw size={13} /> Actualiser</button>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
<div className="card">
|
||||||
|
<div className="section-title"><Ear size={12} /> Mot d’activation</div>
|
||||||
|
<div className="list">{kws.map((m) => <ModelRow key={m.id} model={m} />)}</div>
|
||||||
|
</div>
|
||||||
|
<div className="card">
|
||||||
|
<div className="section-title"><Activity size={12} /> Fin de phrase</div>
|
||||||
|
<div className="list">{vad.map((m) => <ModelRow key={m.id} model={m} />)}</div>
|
||||||
|
<span className="hint" style={{ display: 'block', marginTop: 6 }}>Utilisé automatiquement en écoute permanente dès qu’il est installé ; sinon le VAD énergétique prend le relais.</span>
|
||||||
|
</div>
|
||||||
|
<div className="card">
|
||||||
|
<div className="section-title"><Mic size={12} /> Reconnaissance vocale</div>
|
||||||
|
<div className="list">{stt.map((m) => <ModelRow key={m.id} model={m} />)}</div>
|
||||||
|
</div>
|
||||||
|
<div className="card">
|
||||||
|
<div className="section-title"><Volume2 size={12} /> Synthèse vocale</div>
|
||||||
|
<div className="list">{tts.map((m) => <ModelRow key={m.id} model={m} />)}</div>
|
||||||
|
</div>
|
||||||
|
</>
|
||||||
|
);
|
||||||
|
}
|
||||||
@@ -10,8 +10,10 @@ import { encodeWav } from '../../services/voice/wav';
|
|||||||
import { useHermes } from '../../state/hermes';
|
import { useHermes } from '../../state/hermes';
|
||||||
import { DEFAULT_SETTINGS, useSettings, type HudTheme } from '../../state/settings';
|
import { DEFAULT_SETTINGS, useSettings, type HudTheme } from '../../state/settings';
|
||||||
import { useVoice } from '../../state/voice';
|
import { useVoice } from '../../state/voice';
|
||||||
|
import { installedModels, useVoiceModels } from '../../state/voiceModels';
|
||||||
|
import { ModelsSection } from './ModelsSection';
|
||||||
|
|
||||||
type Section = 'general' | 'hermes' | 'voice' | 'speech' | 'webhook' | 'ui';
|
type Section = 'general' | 'hermes' | 'voice' | 'speech' | 'models' | 'webhook' | 'ui';
|
||||||
|
|
||||||
interface Props {
|
interface Props {
|
||||||
onClose: () => void;
|
onClose: () => void;
|
||||||
@@ -19,24 +21,65 @@ interface Props {
|
|||||||
|
|
||||||
function Toggle({ on, onChange, label, hint }: { on: boolean; onChange: (v: boolean) => void; label: string; hint?: string }) {
|
function Toggle({ on, onChange, label, hint }: { on: boolean; onChange: (v: boolean) => void; label: string; hint?: string }) {
|
||||||
return (
|
return (
|
||||||
<div className="switch" onClick={() => onChange(!on)}>
|
<button type="button" role="switch" aria-checked={on} className="switch" onClick={() => onChange(!on)}>
|
||||||
<div className="text">
|
<div className="text">
|
||||||
<span>{label}</span>
|
<span>{label}</span>
|
||||||
{hint && <small>{hint}</small>}
|
{hint && <small>{hint}</small>}
|
||||||
</div>
|
</div>
|
||||||
<span className={`toggle${on ? ' on' : ''}`} />
|
<span className={`toggle${on ? ' on' : ''}`} />
|
||||||
</div>
|
</button>
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
type TestState = { status: 'idle' | 'running' | 'ok' | 'fail'; message: string };
|
type TestState = { status: 'idle' | 'running' | 'ok' | 'fail'; message: string };
|
||||||
|
|
||||||
|
function WakeStatus() {
|
||||||
|
const wake = useVoice((s) => s.wake);
|
||||||
|
const keywords = useVoice((s) => s.wakeKeywords);
|
||||||
|
const error = useVoice((s) => s.error);
|
||||||
|
const models = useVoiceModels((s) => s.models);
|
||||||
|
const download = useVoiceModels((s) => s.download);
|
||||||
|
const progress = useVoiceModels((s) => s.progress['kws-en']);
|
||||||
|
const installed = models.find((m) => m.id === 'kws-en')?.installed;
|
||||||
|
if (installed === false) {
|
||||||
|
return (
|
||||||
|
<div className="test-result fail" style={{ marginTop: 8 }}>
|
||||||
|
<XCircle size={13} />
|
||||||
|
<span>Détecteur non installé.</span>
|
||||||
|
<button className="btn small primary" style={{ marginLeft: 'auto' }} disabled={!!progress} onClick={() => void download('kws-en')}>
|
||||||
|
{progress ? `${progress.percent}%` : 'Télécharger (4 Mo)'}
|
||||||
|
</button>
|
||||||
|
</div>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
return (
|
||||||
|
<div className={`test-result ${wake === 'spotting' ? 'ok' : wake === 'error' ? 'fail' : ''}`} style={{ marginTop: 8 }}>
|
||||||
|
{wake === 'spotting' ? <CheckCircle2 size={13} /> : wake === 'error' ? <XCircle size={13} /> : <Loader2 size={13} className="spin" />}
|
||||||
|
<span>{wake === 'spotting' ? `à l’écoute de « ${keywords.join(' », « ')} »` : wake === 'error' ? error ?? 'erreur' : 'démarrage…'}</span>
|
||||||
|
</div>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
function TestResult({ t }: { t: TestState }) {
|
||||||
|
if (t.status === 'idle') return null;
|
||||||
|
return (
|
||||||
|
<div className={`test-result ${t.status === 'ok' ? 'ok' : t.status === 'fail' ? 'fail' : ''}`}>
|
||||||
|
{t.status === 'running' ? <Loader2 size={13} className="spin" /> : t.status === 'ok' ? <CheckCircle2 size={13} /> : <XCircle size={13} />}
|
||||||
|
<span>{t.message}</span>
|
||||||
|
</div>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
export function SettingsDrawer({ onClose }: Props) {
|
export function SettingsDrawer({ onClose }: Props) {
|
||||||
const settings = useSettings((s) => s.settings);
|
const settings = useSettings((s) => s.settings);
|
||||||
const update = useSettings((s) => s.update);
|
const update = useSettings((s) => s.update);
|
||||||
const reset = useSettings((s) => s.reset);
|
const reset = useSettings((s) => s.reset);
|
||||||
const hermesStore = useHermes();
|
const hermesModels = useHermes((s) => s.models);
|
||||||
|
const hermesWebhook = useHermes((s) => s.webhook);
|
||||||
|
const hermesConnect = useHermes((s) => s.connect);
|
||||||
const micDevices = useVoice((s) => s.micDevices);
|
const micDevices = useVoice((s) => s.micDevices);
|
||||||
|
const voiceModels = useVoiceModels((s) => s.models);
|
||||||
|
const refreshModels = useVoiceModels((s) => s.refresh);
|
||||||
const [section, setSection] = useState<Section>('hermes');
|
const [section, setSection] = useState<Section>('hermes');
|
||||||
const [hermesTest, setHermesTest] = useState<TestState>({ status: 'idle', message: '' });
|
const [hermesTest, setHermesTest] = useState<TestState>({ status: 'idle', message: '' });
|
||||||
const [sttTest, setSttTest] = useState<TestState>({ status: 'idle', message: '' });
|
const [sttTest, setSttTest] = useState<TestState>({ status: 'idle', message: '' });
|
||||||
@@ -49,8 +92,9 @@ export function SettingsDrawer({ onClose }: Props) {
|
|||||||
refresh();
|
refresh();
|
||||||
speechSynthesis.addEventListener?.('voiceschanged', refresh);
|
speechSynthesis.addEventListener?.('voiceschanged', refresh);
|
||||||
void listMicrophones().then((d) => useVoice.getState().setMicDevices(d));
|
void listMicrophones().then((d) => useVoice.getState().setMicDevices(d));
|
||||||
|
void refreshModels();
|
||||||
return () => speechSynthesis.removeEventListener?.('voiceschanged', refresh);
|
return () => speechSynthesis.removeEventListener?.('voiceschanged', refresh);
|
||||||
}, []);
|
}, [refreshModels]);
|
||||||
|
|
||||||
const testHermes = async () => {
|
const testHermes = async () => {
|
||||||
setHermesTest({ status: 'running', message: 'connexion…' });
|
setHermesTest({ status: 'running', message: 'connexion…' });
|
||||||
@@ -69,7 +113,7 @@ export function SettingsDrawer({ onClose }: Props) {
|
|||||||
}
|
}
|
||||||
const via = bridge() ? 'via Electron' : 'via navigateur';
|
const via = bridge() ? 'via Electron' : 'via navigateur';
|
||||||
setHermesTest({ status: 'ok', message: `${health.status} (${via})${caps}` });
|
setHermesTest({ status: 'ok', message: `${health.status} (${via})${caps}` });
|
||||||
void hermesStore.connect();
|
void hermesConnect();
|
||||||
} catch (err) {
|
} catch (err) {
|
||||||
setHermesTest({ status: 'fail', message: (err as Error).message });
|
setHermesTest({ status: 'fail', message: (err as Error).message });
|
||||||
}
|
}
|
||||||
@@ -101,18 +145,11 @@ export function SettingsDrawer({ onClose }: Props) {
|
|||||||
useHermes.setState({ webhook: status });
|
useHermes.setState({ webhook: status });
|
||||||
};
|
};
|
||||||
|
|
||||||
const Result = ({ t }: { t: TestState }) =>
|
|
||||||
t.status === 'idle' ? null : (
|
|
||||||
<div className={`test-result ${t.status === 'ok' ? 'ok' : t.status === 'fail' ? 'fail' : ''}`}>
|
|
||||||
{t.status === 'running' ? <Loader2 size={13} className="spin" /> : t.status === 'ok' ? <CheckCircle2 size={13} /> : <XCircle size={13} />}
|
|
||||||
<span>{t.message}</span>
|
|
||||||
</div>
|
|
||||||
);
|
|
||||||
|
|
||||||
const nav: Array<[Section, string]> = [
|
const nav: Array<[Section, string]> = [
|
||||||
['hermes', 'Hermes'],
|
['hermes', 'Hermes'],
|
||||||
['voice', 'Micro / STT'],
|
['voice', 'Micro / STT'],
|
||||||
['speech', 'Voix / TTS'],
|
['speech', 'Voix / TTS'],
|
||||||
|
['models', 'Modèles locaux'],
|
||||||
['webhook', 'Webhook'],
|
['webhook', 'Webhook'],
|
||||||
['general', 'Général'],
|
['general', 'Général'],
|
||||||
['ui', 'Interface']
|
['ui', 'Interface']
|
||||||
@@ -149,7 +186,7 @@ export function SettingsDrawer({ onClose }: Props) {
|
|||||||
<label>Modèle</label>
|
<label>Modèle</label>
|
||||||
<input className="input" list="hermes-models" value={settings.hermes.model} placeholder="(défaut du serveur)" onChange={(e) => update({ hermes: { model: e.target.value } })} />
|
<input className="input" list="hermes-models" value={settings.hermes.model} placeholder="(défaut du serveur)" onChange={(e) => update({ hermes: { model: e.target.value } })} />
|
||||||
<datalist id="hermes-models">
|
<datalist id="hermes-models">
|
||||||
{hermesStore.models.map((m) => <option key={m.id} value={m.id} />)}
|
{hermesModels.map((m) => <option key={m.id} value={m.id} />)}
|
||||||
</datalist>
|
</datalist>
|
||||||
</div>
|
</div>
|
||||||
<div className="field">
|
<div className="field">
|
||||||
@@ -185,7 +222,7 @@ export function SettingsDrawer({ onClose }: Props) {
|
|||||||
<div className="row" style={{ marginTop: 10 }}>
|
<div className="row" style={{ marginTop: 10 }}>
|
||||||
<button className="btn primary small" onClick={() => void testHermes()}><Play size={13} /> Tester la liaison</button>
|
<button className="btn primary small" onClick={() => void testHermes()}><Play size={13} /> Tester la liaison</button>
|
||||||
</div>
|
</div>
|
||||||
<div style={{ marginTop: 8 }}><Result t={hermesTest} /></div>
|
<div style={{ marginTop: 8 }}><TestResult t={hermesTest} /></div>
|
||||||
</div>
|
</div>
|
||||||
</>
|
</>
|
||||||
)}
|
)}
|
||||||
@@ -195,10 +232,23 @@ export function SettingsDrawer({ onClose }: Props) {
|
|||||||
<div className="field">
|
<div className="field">
|
||||||
<label>Moteur de reconnaissance</label>
|
<label>Moteur de reconnaissance</label>
|
||||||
<select className="select" value={settings.voice.provider} onChange={(e) => update({ voice: { provider: e.target.value as typeof settings.voice.provider } })}>
|
<select className="select" value={settings.voice.provider} onChange={(e) => update({ voice: { provider: e.target.value as typeof settings.voice.provider } })}>
|
||||||
|
<option value="local">Local dans l’application (Whisper via sherpa-onnx, hors ligne)</option>
|
||||||
<option value="openai-compatible">API compatible OpenAI (Qwen3-ASR, Whisper, Speaches, LocalAI…)</option>
|
<option value="openai-compatible">API compatible OpenAI (Qwen3-ASR, Whisper, Speaches, LocalAI…)</option>
|
||||||
<option value="browser">Reconnaissance Chromium (en ligne, secours)</option>
|
<option value="browser">Reconnaissance Chromium (en ligne, secours)</option>
|
||||||
</select>
|
</select>
|
||||||
</div>
|
</div>
|
||||||
|
{settings.voice.provider === 'local' && (
|
||||||
|
<div className="field">
|
||||||
|
<label>Modèle local</label>
|
||||||
|
{installedModels(voiceModels, 'stt').length === 0 ? (
|
||||||
|
<span className="hint">Aucun modèle installé. <a href="#" onClick={(e) => { e.preventDefault(); setSection('models'); }}>Téléchargez Whisper base dans « Modèles locaux »</a>.</span>
|
||||||
|
) : (
|
||||||
|
<select className="select" value={settings.voice.localModel} onChange={(e) => update({ voice: { localModel: e.target.value } })}>
|
||||||
|
{installedModels(voiceModels, 'stt').map((m) => <option key={m.id} value={m.id}>{m.name}</option>)}
|
||||||
|
</select>
|
||||||
|
)}
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
{settings.voice.provider === 'openai-compatible' && (
|
{settings.voice.provider === 'openai-compatible' && (
|
||||||
<>
|
<>
|
||||||
<div className="field">
|
<div className="field">
|
||||||
@@ -255,10 +305,47 @@ export function SettingsDrawer({ onClose }: Props) {
|
|||||||
</div>
|
</div>
|
||||||
<Toggle on={settings.voice.handsFree} onChange={(v) => { update({ voice: { handsFree: v } }); useVoice.getState().setHandsFree(v); }} label="Mains libres au démarrage" hint="Le micro se réactive automatiquement après chaque réponse." />
|
<Toggle on={settings.voice.handsFree} onChange={(v) => { update({ voice: { handsFree: v } }); useVoice.getState().setHandsFree(v); }} label="Mains libres au démarrage" hint="Le micro se réactive automatiquement après chaque réponse." />
|
||||||
<Toggle on={settings.voice.wakeChime} onChange={(v) => update({ voice: { wakeChime: v } })} label="Signal sonore d’écoute" />
|
<Toggle on={settings.voice.wakeChime} onChange={(v) => update({ voice: { wakeChime: v } })} label="Signal sonore d’écoute" />
|
||||||
<div className="row" style={{ marginTop: 10 }}>
|
<div className="field" style={{ marginTop: 10 }}>
|
||||||
<button className="btn small" onClick={() => void testStt()} disabled={settings.voice.provider !== 'openai-compatible'}><Mic size={13} /> Tester l’API STT</button>
|
<label>Mot d’activation</label>
|
||||||
|
<select className="select" value={settings.voice.wakeMode} onChange={(e) => update({ voice: { wakeMode: e.target.value as typeof settings.voice.wakeMode, wakeWordEnabled: e.target.value === 'transcript' } })}>
|
||||||
|
<option value="off">Désactivé (bouton, raccourci ou mains libres)</option>
|
||||||
|
<option value="kws">Écoute permanente : le micro reste ouvert et réagit au mot-clé (recommandé)</option>
|
||||||
|
<option value="transcript">Filtre après transcription (mains libres) : les phrases sans le mot sont ignorées</option>
|
||||||
|
</select>
|
||||||
|
<span className="hint">
|
||||||
|
{settings.voice.wakeMode === 'kws'
|
||||||
|
? 'Détection locale par un modèle de 3 Mo, quasi gratuite en CPU. Dire le mot seul ouvre l’écoute ; dire le mot puis la commande envoie directement. Le mot coupe aussi la voix en cours.'
|
||||||
|
: settings.voice.wakeMode === 'transcript'
|
||||||
|
? 'Chaque phrase est transcrite puis filtrée : plus coûteux, à réserver à la reconnaissance locale.'
|
||||||
|
: 'Le micro s’active avec le bouton, Ctrl+Shift+Espace ou la boucle mains libres.'}
|
||||||
|
</span>
|
||||||
</div>
|
</div>
|
||||||
<div style={{ marginTop: 8 }}><Result t={sttTest} /></div>
|
{settings.voice.wakeMode !== 'off' && (
|
||||||
|
<div className="grid-2">
|
||||||
|
<div className="field">
|
||||||
|
<label>Mot-clé</label>
|
||||||
|
<input className="input" value={settings.voice.wakeWord} placeholder="jarvis" onChange={(e) => update({ voice: { wakeWord: e.target.value.toLowerCase().slice(0, 40) } })} />
|
||||||
|
<span className="hint">Prononciation anglaise conseillée (« jarvis », « hey jarvis », « computer », « friday »…).</span>
|
||||||
|
</div>
|
||||||
|
{settings.voice.wakeMode === 'kws' && (
|
||||||
|
<div className="field">
|
||||||
|
<label>Sensibilité du mot-clé : {settings.voice.kwsSensitivity}/5</label>
|
||||||
|
<input className="range" type="range" min={1} max={5} step={1} value={settings.voice.kwsSensitivity} onChange={(e) => update({ voice: { kwsSensitivity: Number(e.target.value) } })} />
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
{settings.voice.wakeMode === 'kws' && (
|
||||||
|
<>
|
||||||
|
<WakeStatus />
|
||||||
|
<Toggle on={settings.voice.neuralVad} onChange={(v) => update({ voice: { neuralVad: v } })} label="Fin de phrase neuronale (Silero)" hint="Coupe l’écoute au bon moment, même avec du bruit de fond. Nécessite le modèle Silero VAD (0,6 Mo) dans Modèles locaux." />
|
||||||
|
</>
|
||||||
|
)}
|
||||||
|
<Toggle on={settings.voice.localCommands} onChange={(v) => update({ voice: { localCommands: v } })} label="Commandes locales instantanées" hint="« Verrouille la session », « monte le son », « ouvre Spotify », « regarde mon écran »… exécutées sur ce PC sans passer par Hermes." />
|
||||||
|
<div className="row" style={{ marginTop: 10 }}>
|
||||||
|
<button className="btn small" onClick={() => void testStt()} disabled={settings.voice.provider === 'browser'}><Mic size={13} /> Tester la reconnaissance</button>
|
||||||
|
</div>
|
||||||
|
<div style={{ marginTop: 8 }}><TestResult t={sttTest} /></div>
|
||||||
</div>
|
</div>
|
||||||
)}
|
)}
|
||||||
|
|
||||||
@@ -267,6 +354,7 @@ export function SettingsDrawer({ onClose }: Props) {
|
|||||||
<div className="field">
|
<div className="field">
|
||||||
<label>Moteur de synthèse</label>
|
<label>Moteur de synthèse</label>
|
||||||
<select className="select" value={settings.speech.provider} onChange={(e) => update({ speech: { provider: e.target.value as typeof settings.speech.provider } })}>
|
<select className="select" value={settings.speech.provider} onChange={(e) => update({ speech: { provider: e.target.value as typeof settings.speech.provider } })}>
|
||||||
|
<option value="local">Local dans l’application (Kokoro / Piper via sherpa-onnx, hors ligne)</option>
|
||||||
<option value="openai-compatible">API compatible OpenAI /v1/audio/speech (Kokoro, Piper, OpenAI, LocalAI…)</option>
|
<option value="openai-compatible">API compatible OpenAI /v1/audio/speech (Kokoro, Piper, OpenAI, LocalAI…)</option>
|
||||||
<option value="system">Voix système Windows</option>
|
<option value="system">Voix système Windows</option>
|
||||||
<option value="google-free">Google Translate (gratuit, en ligne)</option>
|
<option value="google-free">Google Translate (gratuit, en ligne)</option>
|
||||||
@@ -303,6 +391,33 @@ export function SettingsDrawer({ onClose }: Props) {
|
|||||||
</div>
|
</div>
|
||||||
</>
|
</>
|
||||||
)}
|
)}
|
||||||
|
{settings.speech.provider === 'local' && (
|
||||||
|
installedModels(voiceModels, 'tts').length === 0 ? (
|
||||||
|
<div className="field">
|
||||||
|
<span className="hint">Aucun modèle de voix installé. <a href="#" onClick={(e) => { e.preventDefault(); setSection('models'); }}>Téléchargez Kokoro dans « Modèles locaux »</a>.</span>
|
||||||
|
</div>
|
||||||
|
) : (
|
||||||
|
<div className="grid-2">
|
||||||
|
<div className="field">
|
||||||
|
<label>Modèle local</label>
|
||||||
|
<select className="select" value={settings.speech.localModel} onChange={(e) => {
|
||||||
|
const m = voiceModels.find((x) => x.id === e.target.value);
|
||||||
|
update({ speech: { localModel: e.target.value, localSpeaker: m?.speakers?.[0]?.id ?? 0 } });
|
||||||
|
}}>
|
||||||
|
{installedModels(voiceModels, 'tts').map((m) => <option key={m.id} value={m.id}>{m.name}</option>)}
|
||||||
|
</select>
|
||||||
|
</div>
|
||||||
|
<div className="field">
|
||||||
|
<label>Voix</label>
|
||||||
|
<select className="select" value={settings.speech.localSpeaker} onChange={(e) => update({ speech: { localSpeaker: Number(e.target.value) } })}>
|
||||||
|
{(voiceModels.find((m) => m.id === settings.speech.localModel)?.speakers ?? [{ id: 0, name: 'Voix 0', lang: '' }]).map((sp) => (
|
||||||
|
<option key={sp.id} value={sp.id}>{sp.name}</option>
|
||||||
|
))}
|
||||||
|
</select>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
)
|
||||||
|
)}
|
||||||
{settings.speech.provider === 'system' && (
|
{settings.speech.provider === 'system' && (
|
||||||
<div className="field">
|
<div className="field">
|
||||||
<label>Voix système</label>
|
<label>Voix système</label>
|
||||||
@@ -327,10 +442,12 @@ export function SettingsDrawer({ onClose }: Props) {
|
|||||||
<div className="row" style={{ marginTop: 10 }}>
|
<div className="row" style={{ marginTop: 10 }}>
|
||||||
<button className="btn small" onClick={testTts} disabled={settings.speech.provider === 'off'}><Volume2 size={13} /> Tester la voix</button>
|
<button className="btn small" onClick={testTts} disabled={settings.speech.provider === 'off'}><Volume2 size={13} /> Tester la voix</button>
|
||||||
</div>
|
</div>
|
||||||
<div style={{ marginTop: 8 }}><Result t={ttsTest} /></div>
|
<div style={{ marginTop: 8 }}><TestResult t={ttsTest} /></div>
|
||||||
</div>
|
</div>
|
||||||
)}
|
)}
|
||||||
|
|
||||||
|
{section === 'models' && <ModelsSection />}
|
||||||
|
|
||||||
{section === 'webhook' && (
|
{section === 'webhook' && (
|
||||||
<div className="card">
|
<div className="card">
|
||||||
<Toggle on={settings.webhook.enabled} onChange={(v) => update({ webhook: { enabled: v } })} label="Serveur webhook local" hint="Hermes (crons, gateway, scripts) peut pousser des messages vers EveFlow." />
|
<Toggle on={settings.webhook.enabled} onChange={(v) => update({ webhook: { enabled: v } })} label="Serveur webhook local" hint="Hermes (crons, gateway, scripts) peut pousser des messages vers EveFlow." />
|
||||||
@@ -356,7 +473,7 @@ export function SettingsDrawer({ onClose }: Props) {
|
|||||||
</div>
|
</div>
|
||||||
<div className="row">
|
<div className="row">
|
||||||
<button className="btn primary small" onClick={() => void applyWebhook()} disabled={!bridge()}>Appliquer et redémarrer</button>
|
<button className="btn primary small" onClick={() => void applyWebhook()} disabled={!bridge()}>Appliquer et redémarrer</button>
|
||||||
{hermesStore.webhook && <span className={`status-pill ${hermesStore.webhook.listening ? 'online' : 'offline'}`}>{hermesStore.webhook.listening ? `port ${hermesStore.webhook.port}` : hermesStore.webhook.error ?? 'inactif'}</span>}
|
{hermesWebhook && <span className={`status-pill ${hermesWebhook.listening ? 'online' : 'offline'}`}>{hermesWebhook.listening ? `port ${hermesWebhook.port}` : hermesWebhook.error ?? 'inactif'}</span>}
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
)}
|
)}
|
||||||
@@ -378,13 +495,13 @@ export function SettingsDrawer({ onClose }: Props) {
|
|||||||
<div className="theme-swatches">
|
<div className="theme-swatches">
|
||||||
{(['arc', 'gold', 'crimson', 'emerald'] as HudTheme[]).map((t) => {
|
{(['arc', 'gold', 'crimson', 'emerald'] as HudTheme[]).map((t) => {
|
||||||
const colors: Record<HudTheme, string> = { arc: '#34e4ff', gold: '#ffc35a', crimson: '#ff5c6c', emerald: '#4ef0a8' };
|
const colors: Record<HudTheme, string> = { arc: '#34e4ff', gold: '#ffc35a', crimson: '#ff5c6c', emerald: '#4ef0a8' };
|
||||||
return <button key={t} className={`swatch${settings.theme === t ? ' active' : ''}`} style={{ background: colors[t] }} title={t} onClick={() => update({ theme: t })} />;
|
return <button key={t} className={`swatch${settings.theme === t ? ' active' : ''}`} style={{ background: colors[t] }} title={t} aria-label={`Thème ${t}`} aria-pressed={settings.theme === t} onClick={() => update({ theme: t })} />;
|
||||||
})}
|
})}
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
<div className="field">
|
<div className="field">
|
||||||
<label>Raccourcis globaux</label>
|
<label>Raccourcis globaux</label>
|
||||||
<span className="hint"><span className="kbd">Ctrl+Shift+Espace</span> micro · <span className="kbd">Ctrl+Shift+J</span> afficher/masquer · <span className="kbd">Ctrl+Shift+Échap</span> couper la voix</span>
|
<span className="hint"><span className="kbd">Ctrl+Shift+Espace</span> micro · <span className="kbd">Ctrl+Shift+J</span> afficher/masquer · <span className="kbd">Ctrl+Alt+Échap</span> couper la voix · <span className="kbd">Ctrl+K</span> saisie</span>
|
||||||
</div>
|
</div>
|
||||||
<div className="divider" />
|
<div className="divider" />
|
||||||
<button className="btn danger small" onClick={() => { if (confirm('Réinitialiser tous les paramètres ?')) reset(); }}><RotateCcw size={13} /> Réinitialiser</button>
|
<button className="btn danger small" onClick={() => { if (confirm('Réinitialiser tous les paramètres ?')) reset(); }}><RotateCcw size={13} /> Réinitialiser</button>
|
||||||
|
|||||||
@@ -62,7 +62,7 @@ export function useMetrics(enabled = true, intervalMs = 2000): LiveMetrics {
|
|||||||
export function useClock(): Date {
|
export function useClock(): Date {
|
||||||
const [now, setNow] = useState(() => new Date());
|
const [now, setNow] = useState(() => new Date());
|
||||||
useEffect(() => {
|
useEffect(() => {
|
||||||
const timer = setInterval(() => setNow(new Date()), 1000);
|
const timer = setInterval(() => setNow(new Date()), 15_000);
|
||||||
return () => clearInterval(timer);
|
return () => clearInterval(timer);
|
||||||
}, []);
|
}, []);
|
||||||
return now;
|
return now;
|
||||||
|
|||||||
@@ -20,6 +20,8 @@ export class SseParser {
|
|||||||
this.buffer += chunk;
|
this.buffer += chunk;
|
||||||
let index: number;
|
let index: number;
|
||||||
while ((index = this.buffer.search(/\r\n|\n|\r/)) !== -1) {
|
while ((index = this.buffer.search(/\r\n|\n|\r/)) !== -1) {
|
||||||
|
// A trailing "\r" may be the first half of a "\r\n" split across chunks: wait for more data.
|
||||||
|
if (this.buffer[index] === '\r' && index === this.buffer.length - 1) break;
|
||||||
const line = this.buffer.slice(0, index);
|
const line = this.buffer.slice(0, index);
|
||||||
const sepLength = this.buffer.startsWith('\r\n', index) ? 2 : 1;
|
const sepLength = this.buffer.startsWith('\r\n', index) ? 2 : 1;
|
||||||
this.buffer = this.buffer.slice(index + sepLength);
|
this.buffer = this.buffer.slice(index + sepLength);
|
||||||
|
|||||||
+6
-4
@@ -65,12 +65,14 @@ export function cleanForSpeech(raw: string): string {
|
|||||||
return applyPhonetics(text);
|
return applyPhonetics(text);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
const PHONETIC_RULES: Array<[RegExp, string]> = Object.entries(PHONETIC).map(([key, value]) => {
|
||||||
|
const escaped = key.replace(/[-/\\^$*+?.()|[\]{}]/g, '\\$&');
|
||||||
|
return [new RegExp(`(?<=^|\\s|\\p{P})${escaped}(?=$|\\s|\\p{P})`, 'giu'), value];
|
||||||
|
});
|
||||||
|
|
||||||
function applyPhonetics(text: string): string {
|
function applyPhonetics(text: string): string {
|
||||||
let out = text;
|
let out = text;
|
||||||
for (const [key, value] of Object.entries(PHONETIC)) {
|
for (const [regex, value] of PHONETIC_RULES) out = out.replace(regex, value);
|
||||||
const escaped = key.replace(/[-/\\^$*+?.()|[\]{}]/g, '\\$&');
|
|
||||||
out = out.replace(new RegExp(`(?<=^|\\s|\\p{P})${escaped}(?=$|\\s|\\p{P})`, 'giu'), value);
|
|
||||||
}
|
|
||||||
return out;
|
return out;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -130,8 +130,3 @@ export async function httpStream(req: HttpProxyRequest, handlers: StreamHandlers
|
|||||||
return { start, done, abort: () => controller.abort() };
|
return { start, done, abort: () => controller.abort() };
|
||||||
}
|
}
|
||||||
|
|
||||||
/** Collects the whole body of a streaming response (used to read error payloads). */
|
|
||||||
export async function collectStreamBody(handle: StreamHandle, first: string[]): Promise<string> {
|
|
||||||
await handle.done.catch(() => undefined);
|
|
||||||
return first.join('');
|
|
||||||
}
|
|
||||||
@@ -1,6 +1,14 @@
|
|||||||
import React from 'react';
|
import React from 'react';
|
||||||
import ReactDOM from 'react-dom/client';
|
import ReactDOM from 'react-dom/client';
|
||||||
import App from './App';
|
import App from './App';
|
||||||
|
import '@fontsource/orbitron/500.css';
|
||||||
|
import '@fontsource/orbitron/700.css';
|
||||||
|
import '@fontsource/orbitron/900.css';
|
||||||
|
import '@fontsource/rajdhani/400.css';
|
||||||
|
import '@fontsource/rajdhani/500.css';
|
||||||
|
import '@fontsource/rajdhani/600.css';
|
||||||
|
import '@fontsource/rajdhani/700.css';
|
||||||
|
import '@fontsource/share-tech-mono/400.css';
|
||||||
import './styles/tokens.css';
|
import './styles/tokens.css';
|
||||||
import './styles/app.css';
|
import './styles/app.css';
|
||||||
import './styles/chat.css';
|
import './styles/chat.css';
|
||||||
|
|||||||
+112
-22
@@ -1,7 +1,7 @@
|
|||||||
/**
|
/**
|
||||||
* Conversation orchestrator: sends user input to Hermes through the resolved transport,
|
* Conversation orchestrator: sends user input to Hermes through the resolved transport,
|
||||||
* streams the answer into the chat store, drives the HUD state, speaks the reply and
|
* streams the answer into the chat store (coalesced per animation frame), drives the HUD
|
||||||
* surfaces approvals / clarifications as pending requests.
|
* state, speaks the reply and surfaces approvals / clarifications as pending requests.
|
||||||
*/
|
*/
|
||||||
import type { HermesPushEvent } from '../../shared/ipc';
|
import type { HermesPushEvent } from '../../shared/ipc';
|
||||||
import { Log } from '../lib/log';
|
import { Log } from '../lib/log';
|
||||||
@@ -12,12 +12,42 @@ import { useSettings } from '../state/settings';
|
|||||||
import { executeLocalTool, LOCAL_TOOL_DEFINITIONS } from './hermes/localTools';
|
import { executeLocalTool, LOCAL_TOOL_DEFINITIONS } from './hermes/localTools';
|
||||||
import type { HermesStreamEvent, SendHandle } from './hermes/types';
|
import type { HermesStreamEvent, SendHandle } from './hermes/types';
|
||||||
import { speech } from './voice/speech';
|
import { speech } from './voice/speech';
|
||||||
|
import { bridge } from '../lib/bridge';
|
||||||
|
import { parseLocalIntent, runLocalIntent } from './localCommands';
|
||||||
|
|
||||||
let active: SendHandle | null = null;
|
let active: SendHandle | null = null;
|
||||||
let activeMessageId: string | null = null;
|
let activeMessageId: string | null = null;
|
||||||
|
|
||||||
const toolLabel = (name: string) => name.replace(/_/g, ' ');
|
const toolLabel = (name: string) => name.replace(/_/g, ' ');
|
||||||
|
|
||||||
|
// ── delta coalescing: one store update per animation frame ─────────────────
|
||||||
|
const pendingDelta = { id: '', content: '', reasoning: '', raf: 0 };
|
||||||
|
|
||||||
|
function flushDelta(): void {
|
||||||
|
if (pendingDelta.raf) cancelAnimationFrame(pendingDelta.raf);
|
||||||
|
const { id, content, reasoning } = pendingDelta;
|
||||||
|
pendingDelta.id = '';
|
||||||
|
pendingDelta.content = '';
|
||||||
|
pendingDelta.reasoning = '';
|
||||||
|
pendingDelta.raf = 0;
|
||||||
|
if (!id) return;
|
||||||
|
const chat = useChat.getState();
|
||||||
|
if (content) chat.appendToMessage(id, content);
|
||||||
|
if (reasoning) chat.appendToMessage(id, reasoning, 'reasoning');
|
||||||
|
}
|
||||||
|
|
||||||
|
function queueDelta(id: string, text: string, field: 'content' | 'reasoning'): void {
|
||||||
|
if (pendingDelta.id && pendingDelta.id !== id) flushDelta();
|
||||||
|
pendingDelta.id = id;
|
||||||
|
pendingDelta[field] += text;
|
||||||
|
if (!pendingDelta.raf) pendingDelta.raf = requestAnimationFrame(flushDelta);
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Bumps a counter the core reads to flash briefly (send, tool start, speech start). */
|
||||||
|
export function pingCore(): void {
|
||||||
|
useChat.getState().ping();
|
||||||
|
}
|
||||||
|
|
||||||
function localToolContext() {
|
function localToolContext() {
|
||||||
const chat = useChat.getState();
|
const chat = useChat.getState();
|
||||||
const settings = useSettings.getState().settings;
|
const settings = useSettings.getState().settings;
|
||||||
@@ -49,7 +79,7 @@ function localToolContext() {
|
|||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
function handleEvent(event: HermesStreamEvent, messageId: string, runIdRef: { id: string | null }): void {
|
function handleEvent(event: HermesStreamEvent, messageId: string, runIdRef: { id: string | null }, timing: { firstToken: number | null; startedAt: number }): void {
|
||||||
const chat = useChat.getState();
|
const chat = useChat.getState();
|
||||||
switch (event.kind) {
|
switch (event.kind) {
|
||||||
case 'run.started':
|
case 'run.started':
|
||||||
@@ -57,15 +87,20 @@ function handleEvent(event: HermesStreamEvent, messageId: string, runIdRef: { id
|
|||||||
chat.setSending(true, event.runId);
|
chat.setSending(true, event.runId);
|
||||||
break;
|
break;
|
||||||
case 'delta':
|
case 'delta':
|
||||||
chat.appendToMessage(messageId, event.text);
|
if (timing.firstToken === null) {
|
||||||
|
timing.firstToken = Date.now() - timing.startedAt;
|
||||||
|
chat.setLatency(timing.firstToken);
|
||||||
|
}
|
||||||
|
queueDelta(messageId, event.text, 'content');
|
||||||
speech.pushStream(event.text);
|
speech.pushStream(event.text);
|
||||||
break;
|
break;
|
||||||
case 'reasoning':
|
case 'reasoning':
|
||||||
chat.appendToMessage(messageId, event.text, 'reasoning');
|
queueDelta(messageId, event.text, 'reasoning');
|
||||||
break;
|
break;
|
||||||
case 'tool.start':
|
case 'tool.start':
|
||||||
chat.pushActivity({ id: event.id, kind: 'tool', name: event.name, status: 'running', detail: event.args ? previewText(event.args, 160) : undefined });
|
chat.pushActivity({ id: event.id, kind: 'tool', name: event.name, status: 'running', detail: event.args ? previewText(event.args, 160) : undefined });
|
||||||
chat.setHud('thinking');
|
chat.setHud('thinking');
|
||||||
|
pingCore();
|
||||||
break;
|
break;
|
||||||
case 'tool.progress':
|
case 'tool.progress':
|
||||||
if (!chat.activity.some((a) => (event.id && a.id === event.id) || (a.name === event.name && a.status === 'running'))) {
|
if (!chat.activity.some((a) => (event.id && a.id === event.id) || (a.name === event.name && a.status === 'running'))) {
|
||||||
@@ -113,6 +148,7 @@ function handleEvent(event: HermesStreamEvent, messageId: string, runIdRef: { id
|
|||||||
break;
|
break;
|
||||||
case 'completed':
|
case 'completed':
|
||||||
if (event.usage) chat.updateMessage(messageId, { usage: event.usage });
|
if (event.usage) chat.updateMessage(messageId, { usage: event.usage });
|
||||||
|
if (event.status === 'length') chat.updateMessage(messageId, { status: 'cancelled' });
|
||||||
break;
|
break;
|
||||||
case 'error':
|
case 'error':
|
||||||
chat.setError(event.message);
|
chat.setError(event.message);
|
||||||
@@ -127,18 +163,60 @@ function addPending(request: Omit<PendingRequest, 'id' | 'createdAt'>): void {
|
|||||||
const chat = useChat.getState();
|
const chat = useChat.getState();
|
||||||
chat.addPending(request);
|
chat.addPending(request);
|
||||||
chat.setHud('alert');
|
chat.setHud('alert');
|
||||||
speech.say(request.kind === 'approval' ? 'Autorisation requise.' : request.kind === 'clarify' ? request.description : 'Saisie requise.');
|
// Spoken without interrupting the streamed answer.
|
||||||
|
speech.say(request.kind === 'approval' ? 'Autorisation requise.' : request.kind === 'clarify' ? request.description : 'Saisie requise.', { interrupt: false });
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Screenshot of the primary display as a data URL (Electron only). */
|
||||||
|
export async function captureScreen(): Promise<string | null> {
|
||||||
|
const api = bridge();
|
||||||
|
if (!api) return null;
|
||||||
|
try {
|
||||||
|
return await api.system.captureScreen(1600);
|
||||||
|
} catch (err) {
|
||||||
|
useChat.getState().setError(`Capture d’écran : ${(err as Error).message}`);
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Short system intents handled on the machine without a Hermes round trip. Returns true when consumed. */
|
||||||
|
async function tryLocalIntent(text: string, source: string): Promise<{ handled: boolean; images?: string[]; text?: string }> {
|
||||||
|
const settings = useSettings.getState().settings;
|
||||||
|
if (!settings.voice.localCommands || !bridge()) return { handled: false };
|
||||||
|
const intent = parseLocalIntent(text);
|
||||||
|
if (!intent) return { handled: false };
|
||||||
|
if (intent.kind === 'screenshot') {
|
||||||
|
const shot = await captureScreen();
|
||||||
|
if (!shot) return { handled: false };
|
||||||
|
return { handled: false, images: [shot], text: intent.question || text };
|
||||||
|
}
|
||||||
|
const chat = useChat.getState();
|
||||||
|
chat.addMessage({ role: 'user', content: text, source, status: 'done' });
|
||||||
|
const result = await runLocalIntent(intent);
|
||||||
|
chat.addMessage({ role: 'assistant', content: result.message, source: 'local', status: result.ok ? 'done' : 'error' });
|
||||||
|
chat.setHud(result.ok ? 'idle' : 'error');
|
||||||
|
if (settings.speech.autoSpeak) speech.say(result.message, { interrupt: true });
|
||||||
|
return { handled: true };
|
||||||
}
|
}
|
||||||
|
|
||||||
export async function sendMessage(text: string, images: string[] = [], source = 'eveflow'): Promise<void> {
|
export async function sendMessage(text: string, images: string[] = [], source = 'eveflow'): Promise<void> {
|
||||||
const trimmed = text.trim();
|
let trimmed = text.trim();
|
||||||
if ((!trimmed && images.length === 0) || active) return;
|
if ((!trimmed && images.length === 0) || active) return;
|
||||||
|
if (trimmed && images.length === 0) {
|
||||||
|
const local = await tryLocalIntent(trimmed, source);
|
||||||
|
if (local.handled) return;
|
||||||
|
if (local.images) {
|
||||||
|
images = local.images;
|
||||||
|
trimmed = (local.text ?? trimmed).trim();
|
||||||
|
}
|
||||||
|
}
|
||||||
const chat = useChat.getState();
|
const chat = useChat.getState();
|
||||||
const hermes = useHermes.getState();
|
const hermes = useHermes.getState();
|
||||||
const settings = useSettings.getState().settings;
|
const settings = useSettings.getState().settings;
|
||||||
|
|
||||||
speech.stop();
|
speech.stop();
|
||||||
chat.setError(null);
|
chat.setError(null);
|
||||||
|
chat.setLatency(null);
|
||||||
chat.addMessage({ role: 'user', content: trimmed || '(image)', images: images.length ? images : undefined, source, status: 'done' });
|
chat.addMessage({ role: 'user', content: trimmed || '(image)', images: images.length ? images : undefined, source, status: 'done' });
|
||||||
const history = chat.messages
|
const history = chat.messages
|
||||||
.filter((m) => m.role !== 'system' && m.status !== 'error')
|
.filter((m) => m.role !== 'system' && m.status !== 'error')
|
||||||
@@ -148,7 +226,9 @@ export async function sendMessage(text: string, images: string[] = [], source =
|
|||||||
activeMessageId = messageId;
|
activeMessageId = messageId;
|
||||||
chat.setSending(true);
|
chat.setSending(true);
|
||||||
chat.setHud('thinking');
|
chat.setHud('thinking');
|
||||||
|
pingCore();
|
||||||
const startedAt = Date.now();
|
const startedAt = Date.now();
|
||||||
|
const timing = { firstToken: null as number | null, startedAt };
|
||||||
const runIdRef = { id: null as string | null };
|
const runIdRef = { id: null as string | null };
|
||||||
|
|
||||||
const client = hermes.client();
|
const client = hermes.client();
|
||||||
@@ -158,7 +238,7 @@ export async function sendMessage(text: string, images: string[] = [], source =
|
|||||||
images,
|
images,
|
||||||
history,
|
history,
|
||||||
sessionId: settings.hermesSessionId,
|
sessionId: settings.hermesSessionId,
|
||||||
onEvent: (event) => handleEvent(event, messageId, runIdRef),
|
onEvent: (event) => handleEvent(event, messageId, runIdRef, timing),
|
||||||
localToolDefinitions: LOCAL_TOOL_DEFINITIONS,
|
localToolDefinitions: LOCAL_TOOL_DEFINITIONS,
|
||||||
localToolExecutor: (name, args) => executeLocalTool(name, args, localToolContext())
|
localToolExecutor: (name, args) => executeLocalTool(name, args, localToolContext())
|
||||||
},
|
},
|
||||||
@@ -169,20 +249,26 @@ export async function sendMessage(text: string, images: string[] = [], source =
|
|||||||
|
|
||||||
try {
|
try {
|
||||||
const reply = await handle.result;
|
const reply = await handle.result;
|
||||||
const current = useChat.getState().messages.find((m) => m.id === messageId);
|
flushDelta();
|
||||||
const finalText = reply || current?.content || '';
|
const state = useChat.getState();
|
||||||
useChat.getState().updateMessage(messageId, { content: finalText, status: 'done' });
|
const current = state.messages.find((m) => m.id === messageId);
|
||||||
// Anything not streamed (non-delta transports) is spoken now.
|
const streamedText = current?.content ?? '';
|
||||||
if (settings.speech.autoSpeak) {
|
const finalText = handle.aborted ? streamedText : reply || streamedText;
|
||||||
if (!current?.content && finalText) speech.say(finalText);
|
const truncated = current?.status === 'cancelled';
|
||||||
|
state.updateMessage(messageId, { content: finalText || (handle.aborted ? '(interrompu)' : ''), status: handle.aborted || truncated ? 'cancelled' : 'done' });
|
||||||
|
if (handle.aborted) {
|
||||||
|
speech.discardStream();
|
||||||
|
} else if (settings.speech.autoSpeak) {
|
||||||
|
if (!streamedText && finalText) speech.say(finalText);
|
||||||
else speech.endStream();
|
else speech.endStream();
|
||||||
} else {
|
} else {
|
||||||
speech.discardStream();
|
speech.discardStream();
|
||||||
}
|
}
|
||||||
useChat.getState().setHud(speech.isSpeaking() ? 'speaking' : 'idle');
|
state.setHud(speech.isSpeaking() ? 'speaking' : 'idle');
|
||||||
if (useHermes.getState().link === 'offline') useHermes.getState().connect().catch(() => undefined);
|
if (!handle.aborted && useHermes.getState().link === 'offline') useHermes.getState().connect().catch(() => undefined);
|
||||||
Log.info('hermes', `reply in ${Date.now() - startedAt} ms via ${handle.transport}`, { chars: finalText.length });
|
Log.info('hermes', `${handle.aborted ? 'aborted' : 'reply'} in ${Date.now() - startedAt} ms via ${handle.transport}`, { chars: finalText.length, firstToken: timing.firstToken });
|
||||||
} catch (err) {
|
} catch (err) {
|
||||||
|
flushDelta();
|
||||||
const message = (err as Error).message || String(err);
|
const message = (err as Error).message || String(err);
|
||||||
const partial = useChat.getState().messages.find((m) => m.id === messageId)?.content ?? '';
|
const partial = useChat.getState().messages.find((m) => m.id === messageId)?.content ?? '';
|
||||||
useChat.getState().updateMessage(messageId, {
|
useChat.getState().updateMessage(messageId, {
|
||||||
@@ -200,10 +286,14 @@ export async function sendMessage(text: string, images: string[] = [], source =
|
|||||||
} finally {
|
} finally {
|
||||||
active = null;
|
active = null;
|
||||||
activeMessageId = null;
|
activeMessageId = null;
|
||||||
useChat.getState().setSending(false);
|
const state = useChat.getState();
|
||||||
for (const a of useChat.getState().activity) {
|
state.setSending(false);
|
||||||
if (a.status === 'running') useChat.getState().finishActivity({ id: a.id }, { status: 'done' });
|
state.setLatency(null);
|
||||||
|
for (const a of state.activity) {
|
||||||
|
if (a.status === 'running') state.finishActivity({ id: a.id }, { status: 'done' });
|
||||||
}
|
}
|
||||||
|
// Requests bound to this run cannot be answered once it is over.
|
||||||
|
for (const p of state.pending) if (!runIdRef.id || p.runId === runIdRef.id) state.removePending(p.id);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -235,14 +325,13 @@ export async function resolvePending(request: PendingRequest, decision: string):
|
|||||||
chat.removePending(request.id);
|
chat.removePending(request.id);
|
||||||
const client = useHermes.getState().client();
|
const client = useHermes.getState().client();
|
||||||
try {
|
try {
|
||||||
|
if (!request.runId) throw new Error('run inconnu');
|
||||||
if (request.kind === 'approval') {
|
if (request.kind === 'approval') {
|
||||||
const choice = (['once', 'session', 'always', 'deny'].includes(decision) ? decision : decision === 'approve' ? 'once' : 'deny') as 'once' | 'session' | 'always' | 'deny';
|
const choice = (['once', 'session', 'always', 'deny'].includes(decision) ? decision : decision === 'approve' ? 'once' : 'deny') as 'once' | 'session' | 'always' | 'deny';
|
||||||
if (!request.runId) throw new Error('run inconnu');
|
|
||||||
await client.approveRun(request.runId, choice, request.requestId || undefined);
|
await client.approveRun(request.runId, choice, request.requestId || undefined);
|
||||||
chat.addMessage({ role: 'system', content: `Autorisation ${choice === 'deny' ? 'refusée' : 'accordée'}${request.tool ? ` (${toolLabel(request.tool)})` : ''}.`, status: 'done' });
|
chat.addMessage({ role: 'system', content: `Autorisation ${choice === 'deny' ? 'refusée' : 'accordée'}${request.tool ? ` (${toolLabel(request.tool)})` : ''}.`, status: 'done' });
|
||||||
} else {
|
} else {
|
||||||
// Clarifications and secrets are answered by steering the run with the reply.
|
// Clarifications and secrets are answered by steering the run with the reply.
|
||||||
if (!request.runId) throw new Error('run inconnu');
|
|
||||||
await client.steerRun(request.runId, decision);
|
await client.steerRun(request.runId, decision);
|
||||||
chat.addMessage({ role: 'system', content: request.secret ? 'Saisie transmise à Hermes.' : `Réponse transmise : ${decision}`, status: 'done' });
|
chat.addMessage({ role: 'system', content: request.secret ? 'Saisie transmise à Hermes.' : `Réponse transmise : ${decision}`, status: 'done' });
|
||||||
}
|
}
|
||||||
@@ -278,6 +367,7 @@ export function handlePush(event: HermesPushEvent): void {
|
|||||||
speech.say(event.jobName ? `Résultat de ${event.jobName}. ${event.text}` : event.text);
|
speech.say(event.jobName ? `Résultat de ${event.jobName}. ${event.text}` : event.text);
|
||||||
}
|
}
|
||||||
chat.setHud(event.status?.includes('fail') ? 'alert' : 'success');
|
chat.setHud(event.status?.includes('fail') ? 'alert' : 'success');
|
||||||
|
pingCore();
|
||||||
setTimeout(() => {
|
setTimeout(() => {
|
||||||
const s = useChat.getState();
|
const s = useChat.getState();
|
||||||
if (s.hud === 'success' || s.hud === 'alert') s.setHud(speech.isSpeaking() ? 'speaking' : 'idle');
|
if (s.hud === 'success' || s.hud === 'alert') s.setHud(speech.isSpeaking() ? 'speaking' : 'idle');
|
||||||
|
|||||||
@@ -261,6 +261,7 @@ export class HermesClient {
|
|||||||
for (const event of normalizeLifecycleEvent(eventName, data)) onEvent(event);
|
for (const event of normalizeLifecycleEvent(eventName, data)) onEvent(event);
|
||||||
});
|
});
|
||||||
const errorChunks: string[] = [];
|
const errorChunks: string[] = [];
|
||||||
|
let handleReady = false;
|
||||||
const handle = await httpStream(
|
const handle = await httpStream(
|
||||||
{
|
{
|
||||||
url: `${this.base}${path}`,
|
url: `${this.base}${path}`,
|
||||||
@@ -271,7 +272,6 @@ export class HermesClient {
|
|||||||
},
|
},
|
||||||
{ onChunk: (text) => (handleReady ? parser.feed(text) : errorChunks.push(text)) }
|
{ onChunk: (text) => (handleReady ? parser.feed(text) : errorChunks.push(text)) }
|
||||||
);
|
);
|
||||||
let handleReady = handle.start.ok;
|
|
||||||
if (!handle.start.ok) {
|
if (!handle.start.ok) {
|
||||||
await handle.done.catch(() => undefined);
|
await handle.done.catch(() => undefined);
|
||||||
throw new HttpError(handle.start.status, errorMessage(handle.start.status, errorChunks.join('')));
|
throw new HttpError(handle.start.status, errorMessage(handle.start.status, errorChunks.join('')));
|
||||||
@@ -285,25 +285,45 @@ export class HermesClient {
|
|||||||
|
|
||||||
send(options: SendOptions, transport: ResolvedTransport): SendHandle {
|
send(options: SendOptions, transport: ResolvedTransport): SendHandle {
|
||||||
const effective: ResolvedTransport = options.images?.length && transport === 'runs' ? 'completions' : transport;
|
const effective: ResolvedTransport = options.images?.length && transport === 'runs' ? 'completions' : transport;
|
||||||
let abort: () => void = () => undefined;
|
let aborted = false;
|
||||||
|
let abortImpl: () => void = () => undefined;
|
||||||
|
const setAbort = (fn: () => void) => {
|
||||||
|
abortImpl = fn;
|
||||||
|
if (aborted) fn();
|
||||||
|
};
|
||||||
|
const isAborted = () => aborted;
|
||||||
const result =
|
const result =
|
||||||
effective === 'runs'
|
effective === 'runs'
|
||||||
? this.sendViaRuns(options, (fn) => (abort = fn))
|
? this.sendViaRuns(options, setAbort, isAborted)
|
||||||
: effective === 'sessions'
|
: effective === 'sessions'
|
||||||
? this.sendViaSessions(options, (fn) => (abort = fn))
|
? this.sendViaSessions(options, setAbort, isAborted)
|
||||||
: this.sendViaCompletions(options, (fn) => (abort = fn));
|
: this.sendViaCompletions(options, setAbort, isAborted);
|
||||||
return { result, abort: () => abort(), transport: effective };
|
return {
|
||||||
|
result,
|
||||||
|
transport: effective,
|
||||||
|
get aborted() {
|
||||||
|
return aborted;
|
||||||
|
},
|
||||||
|
abort: () => {
|
||||||
|
aborted = true;
|
||||||
|
abortImpl();
|
||||||
|
}
|
||||||
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
private async sendViaRuns(options: SendOptions, setAbort: (fn: () => void) => void): Promise<string> {
|
private async sendViaRuns(options: SendOptions, setAbort: (fn: () => void) => void, isAborted: () => boolean): Promise<string> {
|
||||||
const { onEvent } = options;
|
const { onEvent } = options;
|
||||||
const run = await this.startRun({
|
const run = await this.startRun({
|
||||||
input: options.text,
|
input: options.text,
|
||||||
session_id: options.sessionId || undefined,
|
session_id: plainSession(options.sessionId) || undefined,
|
||||||
instructions: this.config.instructions || undefined,
|
instructions: this.config.instructions || undefined,
|
||||||
model: this.config.model || undefined
|
model: this.config.model || undefined
|
||||||
});
|
});
|
||||||
const runId = run.run_id;
|
const runId = run.run_id;
|
||||||
|
if (isAborted()) {
|
||||||
|
this.stopRun(runId).catch(() => undefined);
|
||||||
|
return '';
|
||||||
|
}
|
||||||
onEvent({ kind: 'run.started', runId });
|
onEvent({ kind: 'run.started', runId });
|
||||||
Log.info('hermes', `run ${runId} started`);
|
Log.info('hermes', `run ${runId} started`);
|
||||||
|
|
||||||
@@ -329,7 +349,7 @@ export class HermesClient {
|
|||||||
});
|
});
|
||||||
|
|
||||||
await handle.done;
|
await handle.done;
|
||||||
if (stopped) return streamedText;
|
if (stopped || isAborted()) return streamedText;
|
||||||
if (failure) throw new Error(failure);
|
if (failure) throw new Error(failure);
|
||||||
|
|
||||||
if (!completed) {
|
if (!completed) {
|
||||||
@@ -349,7 +369,7 @@ export class HermesClient {
|
|||||||
return finalText || streamedText;
|
return finalText || streamedText;
|
||||||
}
|
}
|
||||||
|
|
||||||
private async sendViaSessions(options: SendOptions, setAbort: (fn: () => void) => void): Promise<string> {
|
private async sendViaSessions(options: SendOptions, setAbort: (fn: () => void) => void, isAborted: () => boolean): Promise<string> {
|
||||||
const { onEvent } = options;
|
const { onEvent } = options;
|
||||||
let sessionId = options.sessionId;
|
let sessionId = options.sessionId;
|
||||||
if (!sessionId || !sessionId.startsWith('hs:')) {
|
if (!sessionId || !sessionId.startsWith('hs:')) {
|
||||||
@@ -357,6 +377,7 @@ export class HermesClient {
|
|||||||
sessionId = `hs:${session.id}`;
|
sessionId = `hs:${session.id}`;
|
||||||
onEvent({ kind: 'session', sessionId });
|
onEvent({ kind: 'session', sessionId });
|
||||||
}
|
}
|
||||||
|
if (isAborted()) return '';
|
||||||
const realId = sessionId.slice(3);
|
const realId = sessionId.slice(3);
|
||||||
const body: Rec = { input: options.text };
|
const body: Rec = { input: options.text };
|
||||||
if (this.config.instructions) body.instructions = this.config.instructions;
|
if (this.config.instructions) body.instructions = this.config.instructions;
|
||||||
@@ -374,11 +395,12 @@ export class HermesClient {
|
|||||||
});
|
});
|
||||||
setAbort(() => handle.abort());
|
setAbort(() => handle.abort());
|
||||||
await handle.done;
|
await handle.done;
|
||||||
|
if (isAborted()) return streamed;
|
||||||
if (failure) throw new Error(failure);
|
if (failure) throw new Error(failure);
|
||||||
return finalText || streamed;
|
return finalText || streamed;
|
||||||
}
|
}
|
||||||
|
|
||||||
private async sendViaCompletions(options: SendOptions, setAbort: (fn: () => void) => void): Promise<string> {
|
private async sendViaCompletions(options: SendOptions, setAbort: (fn: () => void) => void, isAborted: () => boolean): Promise<string> {
|
||||||
const { onEvent } = options;
|
const { onEvent } = options;
|
||||||
const useTools = this.config.localTools && !!options.localToolExecutor && !!options.localToolDefinitions?.length;
|
const useTools = this.config.localTools && !!options.localToolExecutor && !!options.localToolDefinitions?.length;
|
||||||
const messages: HermesChatMessage[] = [];
|
const messages: HermesChatMessage[] = [];
|
||||||
@@ -389,16 +411,13 @@ export class HermesClient {
|
|||||||
: options.text;
|
: options.text;
|
||||||
messages.push({ role: 'user', content: userContent });
|
messages.push({ role: 'user', content: userContent });
|
||||||
|
|
||||||
let aborted = false;
|
|
||||||
let currentHandle: StreamHandle | null = null;
|
let currentHandle: StreamHandle | null = null;
|
||||||
setAbort(() => {
|
setAbort(() => currentHandle?.abort());
|
||||||
aborted = true;
|
const aborted = () => isAborted();
|
||||||
currentHandle?.abort();
|
|
||||||
});
|
|
||||||
|
|
||||||
let fullText = '';
|
let fullText = '';
|
||||||
let toolsAllowed = useTools;
|
let toolsAllowed = useTools;
|
||||||
for (let iteration = 0; iteration < 6 && !aborted; iteration++) {
|
for (let iteration = 0; iteration < 6 && !aborted(); iteration++) {
|
||||||
const payload: Rec = { model: this.config.model || 'hermes-agent', messages, stream: true };
|
const payload: Rec = { model: this.config.model || 'hermes-agent', messages, stream: true };
|
||||||
if (toolsAllowed) {
|
if (toolsAllowed) {
|
||||||
payload.tools = options.localToolDefinitions;
|
payload.tools = options.localToolDefinitions;
|
||||||
@@ -428,7 +447,8 @@ export class HermesClient {
|
|||||||
});
|
});
|
||||||
|
|
||||||
const headers = this.headers({ Accept: 'text/event-stream' });
|
const headers = this.headers({ Accept: 'text/event-stream' });
|
||||||
if (options.sessionId) headers['X-Hermes-Session-Id'] = options.sessionId;
|
const sessionId = plainSession(options.sessionId);
|
||||||
|
if (sessionId) headers['X-Hermes-Session-Id'] = sessionId;
|
||||||
const handle = await httpStream(
|
const handle = await httpStream(
|
||||||
{ url: `${this.base}/v1/chat/completions`, method: 'POST', headers, body: JSON.stringify(payload), timeoutMs: 10 * 60_000 },
|
{ url: `${this.base}/v1/chat/completions`, method: 'POST', headers, body: JSON.stringify(payload), timeoutMs: 10 * 60_000 },
|
||||||
{ onChunk: (text) => (ready ? parser.feed(text) : errorChunks.push(text)) }
|
{ onChunk: (text) => (ready ? parser.feed(text) : errorChunks.push(text)) }
|
||||||
@@ -447,6 +467,8 @@ export class HermesClient {
|
|||||||
throw new HttpError(handle.start.status, detail);
|
throw new HttpError(handle.start.status, detail);
|
||||||
}
|
}
|
||||||
ready = true;
|
ready = true;
|
||||||
|
// Chunks that raced ahead of the start event were buffered as potential error bodies: replay them.
|
||||||
|
for (const chunk of errorChunks.splice(0)) parser.feed(chunk);
|
||||||
const rotated = handle.start.headers['x-hermes-session-id'];
|
const rotated = handle.start.headers['x-hermes-session-id'];
|
||||||
if (rotated && rotated !== options.sessionId) onEvent({ kind: 'session', sessionId: rotated });
|
if (rotated && rotated !== options.sessionId) onEvent({ kind: 'session', sessionId: rotated });
|
||||||
|
|
||||||
@@ -454,7 +476,7 @@ export class HermesClient {
|
|||||||
parser.end();
|
parser.end();
|
||||||
fullText += iterationText;
|
fullText += iterationText;
|
||||||
|
|
||||||
if (finishReason === 'tool_calls' && toolsAllowed && toolCalls.size > 0 && !aborted) {
|
if (finishReason === 'tool_calls' && toolsAllowed && toolCalls.size > 0 && !aborted()) {
|
||||||
const calls = [...toolCalls.values()];
|
const calls = [...toolCalls.values()];
|
||||||
messages.push({
|
messages.push({
|
||||||
role: 'assistant',
|
role: 'assistant',
|
||||||
@@ -471,11 +493,16 @@ export class HermesClient {
|
|||||||
}
|
}
|
||||||
break;
|
break;
|
||||||
}
|
}
|
||||||
onEvent({ kind: 'completed', status: aborted ? 'cancelled' : 'completed', text: fullText });
|
onEvent({ kind: 'completed', status: aborted() ? 'cancelled' : 'completed', text: fullText });
|
||||||
return fullText;
|
return fullText;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/** Session ids created by the sessions transport carry an `hs:` prefix; other endpoints get the bare id. */
|
||||||
|
export function plainSession(id: string): string {
|
||||||
|
return id.startsWith('hs:') ? id.slice(3) : id;
|
||||||
|
}
|
||||||
|
|
||||||
export function normalizeJob(job: HermesJob): HermesJob {
|
export function normalizeJob(job: HermesJob): HermesJob {
|
||||||
const raw = job as Rec;
|
const raw = job as Rec;
|
||||||
const id = String(raw.id ?? raw.job_id ?? raw.name ?? '');
|
const id = String(raw.id ?? raw.job_id ?? raw.name ?? '');
|
||||||
|
|||||||
@@ -212,6 +212,7 @@ export function normalizeCompletionChunk(
|
|||||||
): { events: HermesStreamEvent[]; finishReason: string | null } {
|
): { events: HermesStreamEvent[]; finishReason: string | null } {
|
||||||
const events: HermesStreamEvent[] = [];
|
const events: HermesStreamEvent[] = [];
|
||||||
if (!isRec(chunk)) return { events, finishReason: null };
|
if (!isRec(chunk)) return { events, finishReason: null };
|
||||||
|
if (isRec(chunk.usage)) events.push({ kind: 'completed', status: 'usage', usage: chunk.usage as HermesUsage });
|
||||||
const choice = Array.isArray(chunk.choices) && isRec(chunk.choices[0]) ? (chunk.choices[0] as Rec) : null;
|
const choice = Array.isArray(chunk.choices) && isRec(chunk.choices[0]) ? (chunk.choices[0] as Rec) : null;
|
||||||
if (!choice) return { events, finishReason: null };
|
if (!choice) return { events, finishReason: null };
|
||||||
const delta = isRec(choice.delta) ? (choice.delta as Rec) : isRec(choice.message) ? (choice.message as Rec) : null;
|
const delta = isRec(choice.delta) ? (choice.delta as Rec) : isRec(choice.message) ? (choice.message as Rec) : null;
|
||||||
|
|||||||
@@ -167,8 +167,10 @@ export interface SendOptions {
|
|||||||
}
|
}
|
||||||
|
|
||||||
export interface SendHandle {
|
export interface SendHandle {
|
||||||
/** Resolves with the final assistant text. */
|
/** Resolves with the final assistant text (partial text when aborted). */
|
||||||
result: Promise<string>;
|
result: Promise<string>;
|
||||||
abort: () => void;
|
abort: () => void;
|
||||||
|
/** True once abort() was called; the caller must not treat the result as a completed reply. */
|
||||||
|
readonly aborted: boolean;
|
||||||
transport: Exclude<HermesTransport, 'auto'>;
|
transport: Exclude<HermesTransport, 'auto'>;
|
||||||
}
|
}
|
||||||
@@ -0,0 +1,75 @@
|
|||||||
|
/**
|
||||||
|
* Instant local commands: short French/English intents handled on the machine without a
|
||||||
|
* round trip to Hermes (lock, open app/url, volume, media, screenshot to Hermes).
|
||||||
|
* Anything unmatched goes to Hermes as usual.
|
||||||
|
*/
|
||||||
|
import type { SystemAction } from '../../shared/bridge';
|
||||||
|
import { bridge } from '../lib/bridge';
|
||||||
|
import { Log } from '../lib/log';
|
||||||
|
|
||||||
|
export interface LocalIntent {
|
||||||
|
kind: 'action' | 'screenshot';
|
||||||
|
action?: SystemAction;
|
||||||
|
/** Spoken confirmation. */
|
||||||
|
reply: string;
|
||||||
|
/** For screenshot: the question to send to Hermes with the image. */
|
||||||
|
question?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const norm = (t: string) =>
|
||||||
|
t
|
||||||
|
.normalize('NFD')
|
||||||
|
.replace(/[̀-ͯ]/g, '')
|
||||||
|
.toLowerCase()
|
||||||
|
.replace(/[’']/g, ' ')
|
||||||
|
.replace(/[^a-z0-9 :/._-]+/g, ' ')
|
||||||
|
.replace(/\s+/g, ' ')
|
||||||
|
.trim();
|
||||||
|
|
||||||
|
const LOCK = /^(verrouille|verrouiller|bloque|lock)( (la |ma )?(session|le pc|l ordinateur|the (pc|computer|screen)))?$/;
|
||||||
|
const VOL_UP = /^(monte|augmente|hausse|plus fort|volume plus|turn up|raise)( (le|the)? ?(son|volume))?( de \d+)?$/;
|
||||||
|
const VOL_DOWN = /^(baisse|diminue|moins fort|volume moins|turn down|lower)( (le|the)? ?(son|volume))?( de \d+)?$/;
|
||||||
|
const MUTE = /^(coupe|couper|mute|silence|desactive)( (le|the)? ?(son|volume|audio))?$/;
|
||||||
|
const PLAY = /^(pause|play|lecture|reprends|reprendre|mets en pause|met en pause|stop la musique|arrete la musique)$/;
|
||||||
|
const NEXT = /^(suivant|suivante|piste suivante|musique suivante|next|skip)$/;
|
||||||
|
const PREV = /^(precedent|precedente|piste precedente|previous)$/;
|
||||||
|
const OPEN_APP = /^(ouvre|ouvrir|lance|lancer|demarre|open|launch|start) ((l application|l appli|le logiciel|le programme|the app|le|la|les|l|un|une|the|moi) )*(.+)$/;
|
||||||
|
const OPEN_URL = /^(ouvre|ouvrir|va sur|open|go to) (https?:\/\/\S+|(www\.)?[a-z0-9-]+\.[a-z]{2,}(\/\S*)?)$/;
|
||||||
|
const SCREEN = /(regarde|regardes|analyse|decris|decrit|lis|explique|qu est ce qu il y a sur|que vois tu sur|what is on|look at|read) (mon |l |the )?(ecran|screen)|capture (d )?ecran|screenshot/;
|
||||||
|
|
||||||
|
export function parseLocalIntent(text: string): LocalIntent | null {
|
||||||
|
const t = norm(text).replace(/^(jarvis|hey jarvis|ok jarvis)[ ,]*/, '');
|
||||||
|
if (!t) return null;
|
||||||
|
if (LOCK.test(t)) return { kind: 'action', action: { type: 'lock' }, reply: 'Session verrouillée.' };
|
||||||
|
if (MUTE.test(t)) return { kind: 'action', action: { type: 'media', key: 'mute' }, reply: 'Son coupé.' };
|
||||||
|
if (VOL_UP.test(t)) return { kind: 'action', action: { type: 'media', key: 'volume-up' }, reply: 'Volume augmenté.' };
|
||||||
|
if (VOL_DOWN.test(t)) return { kind: 'action', action: { type: 'media', key: 'volume-down' }, reply: 'Volume baissé.' };
|
||||||
|
if (PLAY.test(t)) return { kind: 'action', action: { type: 'media', key: 'play-pause' }, reply: 'Lecture.' };
|
||||||
|
if (NEXT.test(t)) return { kind: 'action', action: { type: 'media', key: 'next' }, reply: 'Piste suivante.' };
|
||||||
|
if (PREV.test(t)) return { kind: 'action', action: { type: 'media', key: 'previous' }, reply: 'Piste précédente.' };
|
||||||
|
if (SCREEN.test(t)) return { kind: 'screenshot', reply: 'Je regarde votre écran.', question: text.trim() };
|
||||||
|
const url = OPEN_URL.exec(t);
|
||||||
|
if (url) {
|
||||||
|
const target = url[2].startsWith('http') ? url[2] : `https://${url[2]}`;
|
||||||
|
return { kind: 'action', action: { type: 'open-url', url: target }, reply: `J’ouvre ${url[2]}.` };
|
||||||
|
}
|
||||||
|
const app = OPEN_APP.exec(t);
|
||||||
|
if (app) {
|
||||||
|
const name = app[app.length - 1].trim();
|
||||||
|
if (name.length <= 40 && !/\s(et|puis|and)\s/.test(name)) return { kind: 'action', action: { type: 'open-app', name }, reply: `J’ouvre ${name}.` };
|
||||||
|
}
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Execute an action intent through the bridge. Returns the spoken outcome. */
|
||||||
|
export async function runLocalIntent(intent: LocalIntent): Promise<{ ok: boolean; message: string }> {
|
||||||
|
const api = bridge();
|
||||||
|
if (!api || !intent.action) return { ok: false, message: 'Actions locales indisponibles hors Electron.' };
|
||||||
|
try {
|
||||||
|
const result = await api.system.action(intent.action);
|
||||||
|
Log.info('local', `${intent.action.type}: ${result.ok ? 'ok' : result.message}`);
|
||||||
|
return { ok: result.ok, message: result.ok ? intent.reply : result.message || 'Échec de la commande locale.' };
|
||||||
|
} catch (err) {
|
||||||
|
return { ok: false, message: (err as Error).message };
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -1,70 +0,0 @@
|
|||||||
/**
|
|
||||||
* LogService — Centralized structured logger for EveFlow.
|
|
||||||
*
|
|
||||||
* In Electron (production/dev): sends log entries via window.electronAPI.log()
|
|
||||||
* which the main process writes to %APPDATA%/EveFlow/eveflow.log
|
|
||||||
*
|
|
||||||
* Fallback (browser only): console only.
|
|
||||||
*/
|
|
||||||
|
|
||||||
export type LogLevel = 'DEBUG' | 'INFO' | 'WARN' | 'ERROR';
|
|
||||||
|
|
||||||
export interface LogEntry {
|
|
||||||
ts: string; // ISO timestamp
|
|
||||||
level: LogLevel;
|
|
||||||
tag: string; // Component / service tag (e.g. "HermesAgent", "Minimax")
|
|
||||||
message: string;
|
|
||||||
data?: unknown; // Any extra serializable payload
|
|
||||||
}
|
|
||||||
|
|
||||||
export class LogService {
|
|
||||||
private static readonly MAX_CONSOLE_DATA_LEN = 2000;
|
|
||||||
|
|
||||||
// Core write method
|
|
||||||
public static log(level: LogLevel, tag: string, message: string, data?: unknown): void {
|
|
||||||
const entry: LogEntry = {
|
|
||||||
ts: new Date().toISOString(),
|
|
||||||
level,
|
|
||||||
tag,
|
|
||||||
message,
|
|
||||||
data
|
|
||||||
};
|
|
||||||
|
|
||||||
// Always write to console for DevTools visibility
|
|
||||||
const consoleMsg = `[${entry.ts}] [${level}] [${tag}] ${message}`;
|
|
||||||
const dataStr = data !== undefined ? JSON.stringify(data, null, 2) : '';
|
|
||||||
|
|
||||||
switch (level) {
|
|
||||||
case 'ERROR': console.error(consoleMsg, dataStr ? dataStr.slice(0, this.MAX_CONSOLE_DATA_LEN) : ''); break;
|
|
||||||
case 'WARN': console.warn(consoleMsg, dataStr ? dataStr.slice(0, this.MAX_CONSOLE_DATA_LEN) : ''); break;
|
|
||||||
case 'DEBUG': console.debug(consoleMsg, dataStr ? dataStr.slice(0, this.MAX_CONSOLE_DATA_LEN) : ''); break;
|
|
||||||
default: console.info(consoleMsg, dataStr ? dataStr.slice(0, this.MAX_CONSOLE_DATA_LEN) : ''); break;
|
|
||||||
}
|
|
||||||
|
|
||||||
// Send to Electron main process for file persistence
|
|
||||||
try {
|
|
||||||
const electronAPI = (window as any).electronAPI;
|
|
||||||
if (electronAPI?.writeLog) {
|
|
||||||
electronAPI.writeLog(entry);
|
|
||||||
}
|
|
||||||
} catch {
|
|
||||||
// Silently ignore: browser context has no electronAPI
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
public static debug(tag: string, message: string, data?: unknown): void {
|
|
||||||
this.log('DEBUG', tag, message, data);
|
|
||||||
}
|
|
||||||
|
|
||||||
public static info(tag: string, message: string, data?: unknown): void {
|
|
||||||
this.log('INFO', tag, message, data);
|
|
||||||
}
|
|
||||||
|
|
||||||
public static warn(tag: string, message: string, data?: unknown): void {
|
|
||||||
this.log('WARN', tag, message, data);
|
|
||||||
}
|
|
||||||
|
|
||||||
public static error(tag: string, message: string, data?: unknown): void {
|
|
||||||
this.log('ERROR', tag, message, data);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -71,9 +71,9 @@ class AudioBus {
|
|||||||
}
|
}
|
||||||
|
|
||||||
/** Normalised output spectrum (0..1) with `bins` values, low to high frequency. */
|
/** Normalised output spectrum (0..1) with `bins` values, low to high frequency. */
|
||||||
spectrum(bins: number, source: 'output' | 'input' = 'output'): Float32Array {
|
spectrum(bins: number, source: 'output' | 'input' = 'output', out: Float32Array = new Float32Array(bins)): Float32Array {
|
||||||
const analyser = source === 'output' ? this.outputAnalyser : this.inputAnalyser;
|
const analyser = source === 'output' ? this.outputAnalyser : this.inputAnalyser;
|
||||||
const out = new Float32Array(bins);
|
out.fill(0);
|
||||||
if (!analyser) {
|
if (!analyser) {
|
||||||
if (this.synthetic > 0) {
|
if (this.synthetic > 0) {
|
||||||
const t = performance.now() / 1000;
|
const t = performance.now() / 1000;
|
||||||
|
|||||||
@@ -88,6 +88,7 @@ export class MicCapture {
|
|||||||
private active = false;
|
private active = false;
|
||||||
private options: CaptureOptions | null = null;
|
private options: CaptureOptions | null = null;
|
||||||
private sampleRate = 16_000;
|
private sampleRate = 16_000;
|
||||||
|
private hadSpeech = false;
|
||||||
|
|
||||||
get isActive(): boolean {
|
get isActive(): boolean {
|
||||||
return this.active;
|
return this.active;
|
||||||
@@ -98,6 +99,7 @@ export class MicCapture {
|
|||||||
this.options = options;
|
this.options = options;
|
||||||
this.chunks = [];
|
this.chunks = [];
|
||||||
this.totalSamples = 0;
|
this.totalSamples = 0;
|
||||||
|
this.hadSpeech = false;
|
||||||
this.vad = options.mode === 'auto' ? new EnergyVad(options.vad ?? DEFAULT_VAD) : null;
|
this.vad = options.mode === 'auto' ? new EnergyVad(options.vad ?? DEFAULT_VAD) : null;
|
||||||
|
|
||||||
if (!navigator.mediaDevices?.getUserMedia) {
|
if (!navigator.mediaDevices?.getUserMedia) {
|
||||||
@@ -169,6 +171,12 @@ export class MicCapture {
|
|||||||
if (!this.active || !this.options) return;
|
if (!this.active || !this.options) return;
|
||||||
this.chunks.push(samples);
|
this.chunks.push(samples);
|
||||||
this.totalSamples += samples.length;
|
this.totalSamples += samples.length;
|
||||||
|
// Before speech starts only a short pre-roll (~400 ms) is kept, so silence is never shipped to STT.
|
||||||
|
if (this.vad && !this.hadSpeech) {
|
||||||
|
while (this.chunks.length > 1 && this.totalSamples - this.chunks[0].length > this.sampleRate * 0.4) {
|
||||||
|
this.totalSamples -= this.chunks.shift()!.length;
|
||||||
|
}
|
||||||
|
}
|
||||||
const level = rms(samples);
|
const level = rms(samples);
|
||||||
this.options.callbacks.onLevel?.(Math.min(1, level * 6));
|
this.options.callbacks.onLevel?.(Math.min(1, level * 6));
|
||||||
|
|
||||||
@@ -184,6 +192,7 @@ export class MicCapture {
|
|||||||
private handleVad(signal: VadSignal): void {
|
private handleVad(signal: VadSignal): void {
|
||||||
switch (signal) {
|
switch (signal) {
|
||||||
case 'speech-start':
|
case 'speech-start':
|
||||||
|
this.hadSpeech = true;
|
||||||
this.options?.callbacks.onSpeechStart?.();
|
this.options?.callbacks.onSpeechStart?.();
|
||||||
break;
|
break;
|
||||||
case 'speech-end':
|
case 'speech-end':
|
||||||
@@ -222,10 +231,12 @@ export class MicCapture {
|
|||||||
/** Stop capture. In manual mode this finalises the utterance; otherwise reports the reason. */
|
/** Stop capture. In manual mode this finalises the utterance; otherwise reports the reason. */
|
||||||
stop(reason: 'stopped' | 'no-speech' | 'too-short' = 'stopped'): void {
|
stop(reason: 'stopped' | 'no-speech' | 'too-short' = 'stopped'): void {
|
||||||
if (!this.active) return;
|
if (!this.active) return;
|
||||||
if (reason === 'stopped') {
|
// Manual stop: transcribe only if something was actually said (or no VAD is running).
|
||||||
|
if (reason === 'stopped' && (!this.vad || this.hadSpeech)) {
|
||||||
this.finishUtterance('stopped');
|
this.finishUtterance('stopped');
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
if (reason === 'stopped') reason = 'no-speech';
|
||||||
const opts = this.options;
|
const opts = this.options;
|
||||||
this.teardown();
|
this.teardown();
|
||||||
opts?.callbacks.onEnd(reason);
|
opts?.callbacks.onEnd(reason);
|
||||||
|
|||||||
@@ -35,9 +35,10 @@ class SpeechFacade {
|
|||||||
return this.engine?.isActive ?? false;
|
return this.engine?.isActive ?? false;
|
||||||
}
|
}
|
||||||
|
|
||||||
say(text: string): void {
|
say(text: string, options: { interrupt?: boolean } = {}): void {
|
||||||
if (useSettings.getState().settings.speech.provider === 'off') return;
|
if (useSettings.getState().settings.speech.provider === 'off') return;
|
||||||
this.tts.speak(text);
|
// While an answer streams, spoken notices are inserted without discarding the rest.
|
||||||
|
this.tts.speak(text, { interrupt: options.interrupt ?? !this.streaming });
|
||||||
}
|
}
|
||||||
|
|
||||||
pushStream(delta: string): void {
|
pushStream(delta: string): void {
|
||||||
|
|||||||
@@ -1,7 +1,8 @@
|
|||||||
import { Log } from '../../lib/log';
|
import { Log } from '../../lib/log';
|
||||||
|
import { bridge } from '../../lib/bridge';
|
||||||
import { httpFetch } from '../../lib/transport';
|
import { httpFetch } from '../../lib/transport';
|
||||||
|
|
||||||
export type SttProvider = 'openai-compatible' | 'browser';
|
export type SttProvider = 'openai-compatible' | 'browser' | 'local';
|
||||||
|
|
||||||
export interface SttConfig {
|
export interface SttConfig {
|
||||||
provider: SttProvider;
|
provider: SttProvider;
|
||||||
@@ -10,6 +11,8 @@ export interface SttConfig {
|
|||||||
model: string;
|
model: string;
|
||||||
language: string; // BCP-47, e.g. fr-FR
|
language: string; // BCP-47, e.g. fr-FR
|
||||||
prompt?: string;
|
prompt?: string;
|
||||||
|
/** Catalog id of the local sherpa-onnx model (provider 'local'). */
|
||||||
|
localModel: string;
|
||||||
}
|
}
|
||||||
|
|
||||||
export function sttEndpoint(apiUrl: string): string {
|
export function sttEndpoint(apiUrl: string): string {
|
||||||
@@ -20,8 +23,20 @@ export function sttEndpoint(apiUrl: string): string {
|
|||||||
return `${base}/v1/audio/transcriptions`;
|
return `${base}/v1/audio/transcriptions`;
|
||||||
}
|
}
|
||||||
|
|
||||||
/** Transcribe a WAV buffer through any OpenAI-compatible `/v1/audio/transcriptions` endpoint. */
|
/** Transcribe with the in-app sherpa-onnx engine (runs in the Electron main process). */
|
||||||
|
export async function transcribeLocal(wav: Uint8Array, config: SttConfig): Promise<string> {
|
||||||
|
const api = bridge();
|
||||||
|
if (!api) throw new Error('La reconnaissance locale nécessite l’application Electron.');
|
||||||
|
if (!config.localModel) throw new Error('Aucun modèle de reconnaissance local sélectionné.');
|
||||||
|
const started = Date.now();
|
||||||
|
const result = await api.voice.transcribe({ modelId: config.localModel, wav, language: config.language.split('-')[0] || 'auto' });
|
||||||
|
Log.info('stt', `local transcription in ${Date.now() - started} ms (engine ${result.durationMs} ms, audio ${result.audioSec.toFixed(1)}s)`);
|
||||||
|
return result.text.trim();
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Transcribe a WAV buffer with the configured provider. */
|
||||||
export async function transcribeWav(wav: Uint8Array, config: SttConfig): Promise<string> {
|
export async function transcribeWav(wav: Uint8Array, config: SttConfig): Promise<string> {
|
||||||
|
if (config.provider === 'local') return transcribeLocal(wav, config);
|
||||||
if (!config.apiUrl.trim()) throw new Error("URL de l'API de transcription non configurée.");
|
if (!config.apiUrl.trim()) throw new Error("URL de l'API de transcription non configurée.");
|
||||||
const endpoint = sttEndpoint(config.apiUrl);
|
const endpoint = sttEndpoint(config.apiUrl);
|
||||||
const fields: Record<string, string> = {
|
const fields: Record<string, string> = {
|
||||||
|
|||||||
+84
-38
@@ -1,14 +1,15 @@
|
|||||||
/**
|
/**
|
||||||
* Text-to-speech engine with a sentence queue, prefetching and Web Audio playback so the HUD
|
* Text-to-speech engine with a sentence queue, bounded prefetching and Web Audio playback so
|
||||||
* reacts to the actual waveform. Providers: OpenAI-compatible /v1/audio/speech, system voices,
|
* the HUD reacts to the actual waveform. Providers: in-app sherpa-onnx (Kokoro / Piper),
|
||||||
* and the legacy Google Translate endpoint (no key, online only).
|
* OpenAI-compatible /v1/audio/speech, system voices, and the legacy Google Translate endpoint.
|
||||||
*/
|
*/
|
||||||
import { Log } from '../../lib/log';
|
import { Log } from '../../lib/log';
|
||||||
|
import { bridge } from '../../lib/bridge';
|
||||||
import { chunkForSpeech, cleanForSpeech, extractSentences } from '../../lib/text';
|
import { chunkForSpeech, cleanForSpeech, extractSentences } from '../../lib/text';
|
||||||
import { httpFetch } from '../../lib/transport';
|
import { httpFetch } from '../../lib/transport';
|
||||||
import { audioBus } from './audioBus';
|
import { audioBus } from './audioBus';
|
||||||
|
|
||||||
export type TtsProvider = 'openai-compatible' | 'system' | 'google-free' | 'off';
|
export type TtsProvider = 'openai-compatible' | 'system' | 'google-free' | 'local' | 'off';
|
||||||
|
|
||||||
export interface TtsConfig {
|
export interface TtsConfig {
|
||||||
provider: TtsProvider;
|
provider: TtsProvider;
|
||||||
@@ -21,6 +22,10 @@ export interface TtsConfig {
|
|||||||
systemVoice: string;
|
systemVoice: string;
|
||||||
language: string;
|
language: string;
|
||||||
volume: number;
|
volume: number;
|
||||||
|
/** Catalog id of the local sherpa-onnx voice model (provider 'local'). */
|
||||||
|
localModel: string;
|
||||||
|
/** Speaker id inside the local model. */
|
||||||
|
localSpeaker: number;
|
||||||
}
|
}
|
||||||
|
|
||||||
export type TtsState = 'idle' | 'loading' | 'speaking';
|
export type TtsState = 'idle' | 'loading' | 'speaking';
|
||||||
@@ -30,6 +35,8 @@ interface QueueItem {
|
|||||||
audio?: Promise<AudioBuffer | null>;
|
audio?: Promise<AudioBuffer | null>;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
const LOOKAHEAD = 2;
|
||||||
|
|
||||||
export function ttsEndpoint(apiUrl: string): string {
|
export function ttsEndpoint(apiUrl: string): string {
|
||||||
const base = apiUrl.trim().replace(/\/+$/, '');
|
const base = apiUrl.trim().replace(/\/+$/, '');
|
||||||
if (base.endsWith('/speech')) return base;
|
if (base.endsWith('/speech')) return base;
|
||||||
@@ -41,8 +48,7 @@ export function ttsEndpoint(apiUrl: string): string {
|
|||||||
export class TtsEngine {
|
export class TtsEngine {
|
||||||
private queue: QueueItem[] = [];
|
private queue: QueueItem[] = [];
|
||||||
private playing = false;
|
private playing = false;
|
||||||
private currentSource: AudioBufferSourceNode | null = null;
|
private cancelCurrent: (() => void) | null = null;
|
||||||
private currentUtterance: SpeechSynthesisUtterance | null = null;
|
|
||||||
private streamBuffer = '';
|
private streamBuffer = '';
|
||||||
private generation = 0;
|
private generation = 0;
|
||||||
private config: TtsConfig;
|
private config: TtsConfig;
|
||||||
@@ -55,7 +61,6 @@ export class TtsEngine {
|
|||||||
|
|
||||||
updateConfig(config: TtsConfig): void {
|
updateConfig(config: TtsConfig): void {
|
||||||
this.config = config;
|
this.config = config;
|
||||||
audioBus.setVolume(config.volume);
|
|
||||||
}
|
}
|
||||||
|
|
||||||
onState(listener: (state: TtsState) => void): () => void {
|
onState(listener: (state: TtsState) => void): () => void {
|
||||||
@@ -73,10 +78,18 @@ export class TtsEngine {
|
|||||||
for (const l of this.listeners) l(state);
|
for (const l of this.listeners) l(state);
|
||||||
}
|
}
|
||||||
|
|
||||||
/** Speak a finished text (clears any pending speech first). */
|
/**
|
||||||
speak(text: string): void {
|
* Speak a finished text. With `interrupt` (default) pending speech is discarded first;
|
||||||
this.stop();
|
* without it the text is inserted ahead of the queue and streamed speech resumes afterwards.
|
||||||
for (const chunk of chunkForSpeech(text)) this.enqueue(chunk);
|
*/
|
||||||
|
speak(text: string, options: { interrupt?: boolean } = {}): void {
|
||||||
|
const interrupt = options.interrupt ?? true;
|
||||||
|
if (interrupt) this.stop();
|
||||||
|
const items = chunkForSpeech(text).map((chunk) => this.makeItem(chunk)).filter((i): i is QueueItem => !!i);
|
||||||
|
if (interrupt) this.queue.push(...items);
|
||||||
|
else this.queue.unshift(...items);
|
||||||
|
this.fillPrefetch();
|
||||||
|
void this.drain();
|
||||||
}
|
}
|
||||||
|
|
||||||
/** Feed streamed tokens; complete sentences are spoken as soon as they are available. */
|
/** Feed streamed tokens; complete sentences are spoken as soon as they are available. */
|
||||||
@@ -95,32 +108,32 @@ export class TtsEngine {
|
|||||||
}
|
}
|
||||||
|
|
||||||
enqueue(text: string): void {
|
enqueue(text: string): void {
|
||||||
if (this.config.provider === 'off') return;
|
const item = this.makeItem(text);
|
||||||
const clean = cleanForSpeech(text);
|
if (!item) return;
|
||||||
if (!clean || !/[\p{L}\p{N}]/u.test(clean)) return;
|
|
||||||
const item: QueueItem = { text: clean };
|
|
||||||
if (this.config.provider !== 'system') item.audio = this.prefetch(clean, this.generation);
|
|
||||||
this.queue.push(item);
|
this.queue.push(item);
|
||||||
|
this.fillPrefetch();
|
||||||
void this.drain();
|
void this.drain();
|
||||||
}
|
}
|
||||||
|
|
||||||
|
private makeItem(text: string): QueueItem | null {
|
||||||
|
if (this.config.provider === 'off') return null;
|
||||||
|
const clean = cleanForSpeech(text);
|
||||||
|
if (!clean || !/[\p{L}\p{N}]/u.test(clean)) return null;
|
||||||
|
return { text: clean };
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Keep at most LOOKAHEAD synthesis requests in flight ahead of playback. */
|
||||||
|
private fillPrefetch(): void {
|
||||||
|
if (this.config.provider === 'system') return;
|
||||||
|
for (const item of this.queue.slice(0, LOOKAHEAD)) item.audio ??= this.prefetch(item.text, this.generation);
|
||||||
|
}
|
||||||
|
|
||||||
stop(): void {
|
stop(): void {
|
||||||
this.generation++;
|
this.generation++;
|
||||||
this.queue = [];
|
this.queue = [];
|
||||||
this.streamBuffer = '';
|
this.streamBuffer = '';
|
||||||
if (this.currentSource) {
|
this.cancelCurrent?.();
|
||||||
try {
|
this.cancelCurrent = null;
|
||||||
this.currentSource.onended = null;
|
|
||||||
this.currentSource.stop();
|
|
||||||
} catch {
|
|
||||||
/* already stopped */
|
|
||||||
}
|
|
||||||
this.currentSource = null;
|
|
||||||
}
|
|
||||||
if (this.currentUtterance) {
|
|
||||||
this.currentUtterance.onend = null;
|
|
||||||
this.currentUtterance = null;
|
|
||||||
}
|
|
||||||
if (typeof speechSynthesis !== 'undefined') speechSynthesis.cancel();
|
if (typeof speechSynthesis !== 'undefined') speechSynthesis.cancel();
|
||||||
audioBus.setSyntheticLevel(0);
|
audioBus.setSyntheticLevel(0);
|
||||||
this.playing = false;
|
this.playing = false;
|
||||||
@@ -133,15 +146,17 @@ export class TtsEngine {
|
|||||||
const gen = this.generation;
|
const gen = this.generation;
|
||||||
while (this.queue.length > 0 && gen === this.generation) {
|
while (this.queue.length > 0 && gen === this.generation) {
|
||||||
const item = this.queue.shift()!;
|
const item = this.queue.shift()!;
|
||||||
|
this.fillPrefetch();
|
||||||
try {
|
try {
|
||||||
if (item.audio) {
|
if (this.config.provider !== 'system') {
|
||||||
this.setState('loading');
|
this.setState('loading');
|
||||||
const buffer = await item.audio;
|
const buffer = await (item.audio ?? this.prefetch(item.text, gen));
|
||||||
if (gen !== this.generation) break;
|
if (gen !== this.generation) break;
|
||||||
if (buffer) {
|
if (buffer) {
|
||||||
this.setState('speaking');
|
this.setState('speaking');
|
||||||
await this.playBuffer(buffer, gen);
|
await this.playBuffer(buffer, gen);
|
||||||
} else {
|
} else {
|
||||||
|
this.setState('speaking');
|
||||||
await this.speakSystem(item.text, gen);
|
await this.speakSystem(item.text, gen);
|
||||||
}
|
}
|
||||||
} else {
|
} else {
|
||||||
@@ -159,7 +174,8 @@ export class TtsEngine {
|
|||||||
}
|
}
|
||||||
|
|
||||||
private prefetch(text: string, gen: number): Promise<AudioBuffer | null> {
|
private prefetch(text: string, gen: number): Promise<AudioBuffer | null> {
|
||||||
const task = this.config.provider === 'google-free' ? this.fetchGoogle(text) : this.fetchOpenAi(text);
|
const task =
|
||||||
|
this.config.provider === 'google-free' ? this.fetchGoogle(text) : this.config.provider === 'local' ? this.fetchLocal(text) : this.fetchOpenAi(text);
|
||||||
return task
|
return task
|
||||||
.then(async (bytes) => {
|
.then(async (bytes) => {
|
||||||
if (gen !== this.generation || !bytes) return null;
|
if (gen !== this.generation || !bytes) return null;
|
||||||
@@ -168,7 +184,7 @@ export class TtsEngine {
|
|||||||
return audioBus.context.decodeAudioData(copy);
|
return audioBus.context.decodeAudioData(copy);
|
||||||
})
|
})
|
||||||
.catch((err) => {
|
.catch((err) => {
|
||||||
Log.warn('tts', `synthesis failed, falling back to system voice: ${(err as Error).message}`);
|
if (gen === this.generation) Log.warn('tts', `synthesis failed, falling back to system voice: ${(err as Error).message}`);
|
||||||
return null;
|
return null;
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
@@ -199,6 +215,20 @@ export class TtsEngine {
|
|||||||
return res.binary;
|
return res.binary;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
private async fetchLocal(text: string): Promise<Uint8Array> {
|
||||||
|
const api = bridge();
|
||||||
|
if (!api) throw new Error('La synthèse locale nécessite l’application Electron.');
|
||||||
|
if (!this.config.localModel) throw new Error('Aucun modèle de voix local sélectionné.');
|
||||||
|
const result = await api.voice.synthesize({
|
||||||
|
modelId: this.config.localModel,
|
||||||
|
text,
|
||||||
|
speaker: this.config.localSpeaker,
|
||||||
|
speed: Math.max(0.5, Math.min(2, this.config.speed || 1))
|
||||||
|
});
|
||||||
|
Log.debug('tts', `local synthesis ${result.durationMs} ms for ${result.audioSec.toFixed(1)}s of audio`);
|
||||||
|
return result.wav;
|
||||||
|
}
|
||||||
|
|
||||||
private async fetchGoogle(text: string): Promise<Uint8Array> {
|
private async fetchGoogle(text: string): Promise<Uint8Array> {
|
||||||
const url = `https://translate.google.com/translate_tts?ie=UTF-8&tl=${encodeURIComponent(this.config.language.split('-')[0] || 'fr')}&client=tw-ob&q=${encodeURIComponent(text.slice(0, 200))}`;
|
const url = `https://translate.google.com/translate_tts?ie=UTF-8&tl=${encodeURIComponent(this.config.language.split('-')[0] || 'fr')}&client=tw-ob&q=${encodeURIComponent(text.slice(0, 200))}`;
|
||||||
const res = await httpFetch({
|
const res = await httpFetch({
|
||||||
@@ -218,15 +248,26 @@ export class TtsEngine {
|
|||||||
return new Promise((resolve) => {
|
return new Promise((resolve) => {
|
||||||
if (gen !== this.generation) return resolve();
|
if (gen !== this.generation) return resolve();
|
||||||
const ctx = audioBus.context;
|
const ctx = audioBus.context;
|
||||||
|
audioBus.setVolume(this.config.volume);
|
||||||
const source = ctx.createBufferSource();
|
const source = ctx.createBufferSource();
|
||||||
source.buffer = buffer;
|
source.buffer = buffer;
|
||||||
if (this.config.provider === 'google-free') source.playbackRate.value = Math.max(0.5, Math.min(2, this.config.speed || 1));
|
if (this.config.provider === 'google-free') source.playbackRate.value = Math.max(0.5, Math.min(2, this.config.speed || 1));
|
||||||
source.connect(audioBus.output);
|
source.connect(audioBus.output);
|
||||||
source.onended = () => {
|
const finish = () => {
|
||||||
if (this.currentSource === source) this.currentSource = null;
|
if (this.cancelCurrent === cancel) this.cancelCurrent = null;
|
||||||
resolve();
|
resolve();
|
||||||
};
|
};
|
||||||
this.currentSource = source;
|
const cancel = () => {
|
||||||
|
source.onended = null;
|
||||||
|
try {
|
||||||
|
source.stop();
|
||||||
|
} catch {
|
||||||
|
/* already stopped */
|
||||||
|
}
|
||||||
|
finish();
|
||||||
|
};
|
||||||
|
source.onended = finish;
|
||||||
|
this.cancelCurrent = cancel;
|
||||||
source.start();
|
source.start();
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
@@ -248,12 +289,17 @@ export class TtsEngine {
|
|||||||
if (pulse) clearInterval(pulse);
|
if (pulse) clearInterval(pulse);
|
||||||
pulse = null;
|
pulse = null;
|
||||||
audioBus.setSyntheticLevel(0);
|
audioBus.setSyntheticLevel(0);
|
||||||
if (this.currentUtterance === utterance) this.currentUtterance = null;
|
if (this.cancelCurrent === cancel) this.cancelCurrent = null;
|
||||||
resolve();
|
resolve();
|
||||||
};
|
};
|
||||||
|
const cancel = () => {
|
||||||
|
utterance.onend = null;
|
||||||
|
utterance.onerror = null;
|
||||||
|
finish();
|
||||||
|
};
|
||||||
utterance.onend = finish;
|
utterance.onend = finish;
|
||||||
utterance.onerror = finish;
|
utterance.onerror = finish;
|
||||||
this.currentUtterance = utterance;
|
this.cancelCurrent = cancel;
|
||||||
speechSynthesis.speak(utterance);
|
speechSynthesis.speak(utterance);
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -10,16 +10,52 @@ import { audioBus } from './audioBus';
|
|||||||
import { listMicrophones, MicCapture } from './capture';
|
import { listMicrophones, MicCapture } from './capture';
|
||||||
import { speech } from './speech';
|
import { speech } from './speech';
|
||||||
import { BrowserRecognizer, transcribeWav } from './stt';
|
import { BrowserRecognizer, transcribeWav } from './stt';
|
||||||
|
import { WakeListener } from './wakeListener';
|
||||||
import type { WavResult } from './wav';
|
import type { WavResult } from './wav';
|
||||||
|
import { bridge } from '../../lib/bridge';
|
||||||
|
|
||||||
const SENSITIVITY_RATIO: Record<number, number> = { 1: 4.5, 2: 3.4, 3: 2.6, 4: 2.0, 5: 1.6 };
|
const SENSITIVITY_RATIO: Record<number, number> = { 1: 4.5, 2: 3.4, 3: 2.6, 4: 2.0, 5: 1.6 };
|
||||||
const SENSITIVITY_MIN_RMS: Record<number, number> = { 1: 0.03, 2: 0.02, 3: 0.012, 4: 0.008, 5: 0.005 };
|
const SENSITIVITY_MIN_RMS: Record<number, number> = { 1: 0.03, 2: 0.02, 3: 0.012, 4: 0.008, 5: 0.005 };
|
||||||
|
|
||||||
|
/** Levenshtein distance, used to tolerate transcription slips in the wake word ("javis" for "jarvis"). */
|
||||||
|
function editDistance(a: string, b: string): number {
|
||||||
|
const dp = Array.from({ length: a.length + 1 }, (_, i) => [i, ...new Array<number>(b.length).fill(0)]);
|
||||||
|
for (let j = 1; j <= b.length; j++) dp[0][j] = j;
|
||||||
|
for (let i = 1; i <= a.length; i++) {
|
||||||
|
for (let j = 1; j <= b.length; j++) {
|
||||||
|
dp[i][j] = Math.min(dp[i - 1][j] + 1, dp[i][j - 1] + 1, dp[i - 1][j - 1] + (a[i - 1] === b[j - 1] ? 0 : 1));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return dp[a.length][b.length];
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Compare the start of an utterance with the wake word, tolerant to accents, case, punctuation and small slips. */
|
||||||
|
export function matchWakeWord(text: string, wakeWord: string): { matched: boolean; rest: string } {
|
||||||
|
const norm = (v: string) => v.normalize('NFD').replace(/[\u0300-\u036f]/g, '').toLowerCase();
|
||||||
|
const target = norm(wakeWord).split(/[\s\p{P}]+/u).filter(Boolean);
|
||||||
|
if (target.length === 0) return { matched: true, rest: text.trim() };
|
||||||
|
const original = text.trim().replace(/^[\s\p{P}]+/u, '');
|
||||||
|
const words = original.split(/\s+/);
|
||||||
|
const head = words.slice(0, target.length).map((w) => norm(w).replace(/[\p{P}]+/gu, ''));
|
||||||
|
if (head.length < target.length) return { matched: false, rest: '' };
|
||||||
|
const joinedHead = head.join(' ');
|
||||||
|
const joinedTarget = target.join(' ');
|
||||||
|
const tolerance = joinedTarget.length >= 6 ? 2 : joinedTarget.length >= 4 ? 1 : 0;
|
||||||
|
if (editDistance(joinedHead, joinedTarget) > tolerance) return { matched: false, rest: '' };
|
||||||
|
const rest = words.slice(target.length).join(' ').replace(/^[\s\p{P}]+/u, '');
|
||||||
|
return { matched: true, rest };
|
||||||
|
}
|
||||||
|
|
||||||
class VoiceController {
|
class VoiceController {
|
||||||
private capture = new MicCapture();
|
private capture = new MicCapture();
|
||||||
|
private wake = new WakeListener();
|
||||||
|
private unsubscribeKws: (() => void) | null = null;
|
||||||
|
/** After a bare wake word, the next utterance is accepted without the wake word. */
|
||||||
|
private attentionUntil = 0;
|
||||||
private browser = new BrowserRecognizer();
|
private browser = new BrowserRecognizer();
|
||||||
private handsFreeTimer: ReturnType<typeof setTimeout> | null = null;
|
private handsFreeTimer: ReturnType<typeof setTimeout> | null = null;
|
||||||
private stopping = false;
|
private stopping = false;
|
||||||
|
private startSeq = 0;
|
||||||
private unsubscribeVoice: (() => void) | null = null;
|
private unsubscribeVoice: (() => void) | null = null;
|
||||||
|
|
||||||
init(): void {
|
init(): void {
|
||||||
@@ -30,28 +66,141 @@ class VoiceController {
|
|||||||
this.scheduleHandsFree(350);
|
this.scheduleHandsFree(350);
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
// Hands-free also re-arms when a reply ends without speech (voice off, empty answer, error).
|
||||||
|
useChat.subscribe((state, prev) => {
|
||||||
|
if (prev.isSending && !state.isSending) {
|
||||||
|
const v = useVoice.getState();
|
||||||
|
if (v.handsFree && v.phase === 'off' && !speech.isSpeaking()) this.scheduleHandsFree(400);
|
||||||
|
}
|
||||||
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
dispose(): void {
|
dispose(): void {
|
||||||
this.unsubscribeVoice?.();
|
this.unsubscribeVoice?.();
|
||||||
this.stop();
|
this.stop();
|
||||||
|
void this.stopWakeMode();
|
||||||
}
|
}
|
||||||
|
|
||||||
get isListening(): boolean {
|
get isListening(): boolean {
|
||||||
return useVoice.getState().phase !== 'off';
|
return useVoice.getState().phase !== 'off';
|
||||||
}
|
}
|
||||||
|
|
||||||
|
get wakeActive(): boolean {
|
||||||
|
return this.wake.isActive;
|
||||||
|
}
|
||||||
|
|
||||||
toggle(): void {
|
toggle(): void {
|
||||||
|
if (this.wake.isActive) {
|
||||||
|
// Always-on mode: the button starts/cancels a command capture on the shared microphone.
|
||||||
|
if (this.wake.currentPhase === 'command' || this.wake.currentPhase === 'speech') this.wake.cancelCommand();
|
||||||
|
else this.onWakeDetected('manuel');
|
||||||
|
return;
|
||||||
|
}
|
||||||
if (this.isListening) this.stop();
|
if (this.isListening) this.stop();
|
||||||
else void this.start();
|
else void this.start();
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/** Always-on keyword spotting: the microphone stays open and the main process spots the wake word. */
|
||||||
|
async startWakeMode(): Promise<void> {
|
||||||
|
const api = bridge();
|
||||||
|
const voice = useVoice.getState();
|
||||||
|
if (!api || this.wake.isActive) return;
|
||||||
|
const settings = useSettings.getState().settings.voice;
|
||||||
|
voice.setWake('starting');
|
||||||
|
try {
|
||||||
|
const phrases = [settings.wakeWord || 'jarvis', `hey ${settings.wakeWord || 'jarvis'}`];
|
||||||
|
const result = await api.voice.kwsStart({ modelId: 'kws-en', keywords: phrases, sensitivity: settings.kwsSensitivity });
|
||||||
|
this.unsubscribeKws?.();
|
||||||
|
this.unsubscribeKws = api.voice.onKwsDetected((d) => this.onWakeDetected(d.keyword));
|
||||||
|
const sensitivity = Math.min(5, Math.max(1, Math.round(settings.sensitivity))) as 1 | 2 | 3 | 4 | 5;
|
||||||
|
let neuralVad = false;
|
||||||
|
if (settings.neuralVad) {
|
||||||
|
try {
|
||||||
|
const installed = (await api.voice.listModels()).some((m) => m.id === 'silero-vad' && m.installed);
|
||||||
|
if (installed) {
|
||||||
|
await api.voice.vadStart({ modelId: 'silero-vad', silenceMs: Math.max(300, Math.min(1500, settings.silenceMs - 200)), threshold: 0.5, maxUtteranceSec: 25 });
|
||||||
|
neuralVad = true;
|
||||||
|
}
|
||||||
|
} catch (err) {
|
||||||
|
Log.warn('voice', `silero unavailable, energy VAD fallback: ${(err as Error).message}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
await this.wake.start({
|
||||||
|
deviceId: settings.micDeviceId || undefined,
|
||||||
|
neuralVad,
|
||||||
|
vad: { silenceMs: settings.silenceMs, speechRatio: SENSITIVITY_RATIO[sensitivity], minRms: SENSITIVITY_MIN_RMS[sensitivity] },
|
||||||
|
callbacks: {
|
||||||
|
onPhase: (phase) => {
|
||||||
|
const v = useVoice.getState();
|
||||||
|
if (phase === 'spotting') {
|
||||||
|
v.setPhase('off');
|
||||||
|
const chat = useChat.getState();
|
||||||
|
if (chat.hud === 'listening') chat.setHud(chat.isSending ? 'thinking' : 'idle');
|
||||||
|
} else if (phase === 'command') {
|
||||||
|
v.setPhase('listening');
|
||||||
|
useChat.getState().setHud('listening');
|
||||||
|
} else if (phase === 'speech') {
|
||||||
|
v.setPhase('speech');
|
||||||
|
useChat.getState().ping();
|
||||||
|
}
|
||||||
|
},
|
||||||
|
onLevel: (level) => {
|
||||||
|
const q = Math.round(level * 20) / 20;
|
||||||
|
if (q !== useVoice.getState().inputLevel) useVoice.getState().setInputLevel(q);
|
||||||
|
},
|
||||||
|
onUtterance: (wav) => void this.transcribe(wav, true),
|
||||||
|
onNoSpeech: () => useVoice.getState().setInputLevel(0),
|
||||||
|
onError: (message) => useVoice.getState().setError(message)
|
||||||
|
}
|
||||||
|
});
|
||||||
|
voice.setWake('spotting', result.accepted.map((k) => k.replace(/_/g, ' ')));
|
||||||
|
voice.setNeuralVad(neuralVad);
|
||||||
|
Log.info('voice', `wake mode on: ${result.accepted.join(', ')}${result.rejected.length ? ` (rejetés : ${result.rejected.join(', ')})` : ''}${neuralVad ? ' · Silero' : ''}`);
|
||||||
|
} catch (err) {
|
||||||
|
const message = (err as Error).message;
|
||||||
|
voice.setWake('error');
|
||||||
|
voice.setError(message);
|
||||||
|
useChat.getState().setError(`Mot d’activation : ${message}`);
|
||||||
|
Log.error('voice', `wake mode failed: ${message}`);
|
||||||
|
await api.voice.kwsStop().catch(() => undefined);
|
||||||
|
await api.voice.vadStop().catch(() => undefined);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async stopWakeMode(): Promise<void> {
|
||||||
|
this.unsubscribeKws?.();
|
||||||
|
this.unsubscribeKws = null;
|
||||||
|
this.wake.stop();
|
||||||
|
useVoice.getState().setWake('off');
|
||||||
|
useVoice.getState().setNeuralVad(false);
|
||||||
|
const api = bridge();
|
||||||
|
await api?.voice.kwsStop().catch(() => undefined);
|
||||||
|
await api?.voice.vadStop().catch(() => undefined);
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Restart spotting with the current settings (wake word, sensitivity, microphone). */
|
||||||
|
async restartWakeMode(): Promise<void> {
|
||||||
|
await this.stopWakeMode();
|
||||||
|
if (useSettings.getState().settings.voice.wakeMode === 'kws') await this.startWakeMode();
|
||||||
|
}
|
||||||
|
|
||||||
|
private onWakeDetected(keyword: string): void {
|
||||||
|
if (!this.wake.isActive || this.wake.currentPhase !== 'spotting') return;
|
||||||
|
if (useChat.getState().isSending) return;
|
||||||
|
Log.info('voice', `wake: ${keyword}`);
|
||||||
|
speech.stop();
|
||||||
|
useChat.getState().ping();
|
||||||
|
this.playChime(true);
|
||||||
|
this.wake.beginCommand();
|
||||||
|
}
|
||||||
|
|
||||||
setHandsFree(on: boolean): void {
|
setHandsFree(on: boolean): void {
|
||||||
useVoice.getState().setHandsFree(on);
|
useVoice.getState().setHandsFree(on);
|
||||||
useSettings.getState().update({ voice: { handsFree: on } });
|
useSettings.getState().update({ voice: { handsFree: on } });
|
||||||
if (on && !this.isListening && !speech.isSpeaking()) void this.start();
|
if (on && !this.isListening && !speech.isSpeaking()) void this.start();
|
||||||
if (!on) {
|
if (!on) {
|
||||||
this.clearHandsFreeTimer();
|
this.clearHandsFreeTimer();
|
||||||
|
if (this.isListening) this.cancel();
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -59,7 +208,8 @@ class VoiceController {
|
|||||||
this.clearHandsFreeTimer();
|
this.clearHandsFreeTimer();
|
||||||
this.handsFreeTimer = setTimeout(() => {
|
this.handsFreeTimer = setTimeout(() => {
|
||||||
const v = useVoice.getState();
|
const v = useVoice.getState();
|
||||||
if (v.handsFree && v.phase === 'off' && !useChat.getState().isSending && !speech.isSpeaking()) void this.start();
|
const bargeIn = useSettings.getState().settings.voice.bargeIn;
|
||||||
|
if (v.handsFree && v.phase === 'off' && !useChat.getState().isSending && (bargeIn || !speech.isSpeaking())) void this.start(true);
|
||||||
}, delay);
|
}, delay);
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -68,14 +218,21 @@ class VoiceController {
|
|||||||
this.handsFreeTimer = null;
|
this.handsFreeTimer = null;
|
||||||
}
|
}
|
||||||
|
|
||||||
async start(): Promise<void> {
|
/** @param auto true when re-armed by the hands-free loop (no chime, patient timeout). */
|
||||||
|
async start(auto = false): Promise<void> {
|
||||||
const voice = useVoice.getState();
|
const voice = useVoice.getState();
|
||||||
if (voice.phase !== 'off') return;
|
if (voice.phase !== 'off') return;
|
||||||
|
if (this.wake.isActive) {
|
||||||
|
// The always-on listener owns the microphone: open a command capture on it instead.
|
||||||
|
if (!auto) this.onWakeDetected('manuel');
|
||||||
|
return;
|
||||||
|
}
|
||||||
const settings = useSettings.getState().settings.voice;
|
const settings = useSettings.getState().settings.voice;
|
||||||
|
const seq = ++this.startSeq;
|
||||||
voice.setError(null);
|
voice.setError(null);
|
||||||
voice.setInterim('');
|
voice.setInterim('');
|
||||||
if (!settings.bargeIn) speech.stop();
|
// Barge-in keeps the assistant talking while the mic opens (echo cancellation handles the overlap).
|
||||||
else speech.stop();
|
if (!(auto && settings.bargeIn)) speech.stop();
|
||||||
useChat.getState().setHud('listening');
|
useChat.getState().setHud('listening');
|
||||||
voice.setPhase('arming');
|
voice.setPhase('arming');
|
||||||
|
|
||||||
@@ -94,18 +251,29 @@ class VoiceController {
|
|||||||
silenceMs: settings.silenceMs,
|
silenceMs: settings.silenceMs,
|
||||||
speechRatio: SENSITIVITY_RATIO[sensitivity],
|
speechRatio: SENSITIVITY_RATIO[sensitivity],
|
||||||
minRms: SENSITIVITY_MIN_RMS[sensitivity],
|
minRms: SENSITIVITY_MIN_RMS[sensitivity],
|
||||||
noSpeechTimeoutMs: useVoice.getState().handsFree ? 20_000 : 9_000
|
noSpeechTimeoutMs: useVoice.getState().handsFree ? Number.POSITIVE_INFINITY : 9_000
|
||||||
},
|
},
|
||||||
callbacks: {
|
callbacks: {
|
||||||
onLevel: (level) => useVoice.getState().setInputLevel(level),
|
onLevel: (level) => {
|
||||||
onSpeechStart: () => useVoice.getState().setPhase('speech'),
|
const q = Math.round(level * 20) / 20;
|
||||||
|
if (q !== useVoice.getState().inputLevel) useVoice.getState().setInputLevel(q);
|
||||||
|
},
|
||||||
|
onSpeechStart: () => {
|
||||||
|
useVoice.getState().setPhase('speech');
|
||||||
|
useChat.getState().ping();
|
||||||
|
},
|
||||||
onUtterance: (wav) => void this.transcribe(wav),
|
onUtterance: (wav) => void this.transcribe(wav),
|
||||||
onEnd: (reason) => this.onCaptureEnd(reason),
|
onEnd: (reason) => this.onCaptureEnd(reason),
|
||||||
onError: (message) => useVoice.getState().setError(message)
|
onError: (message) => useVoice.getState().setError(message)
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
if (seq !== this.startSeq || useVoice.getState().phase !== 'arming') {
|
||||||
|
// stop() was called while the microphone was opening.
|
||||||
|
this.capture.cancel();
|
||||||
|
return;
|
||||||
|
}
|
||||||
useVoice.getState().setPhase('listening');
|
useVoice.getState().setPhase('listening');
|
||||||
this.playChime(true);
|
if (!auto) this.playChime(true);
|
||||||
} catch (err) {
|
} catch (err) {
|
||||||
const message = (err as Error).message;
|
const message = (err as Error).message;
|
||||||
useVoice.getState().setError(message);
|
useVoice.getState().setError(message);
|
||||||
@@ -135,6 +303,7 @@ class VoiceController {
|
|||||||
|
|
||||||
stop(): void {
|
stop(): void {
|
||||||
this.stopping = true;
|
this.stopping = true;
|
||||||
|
this.startSeq++;
|
||||||
this.clearHandsFreeTimer();
|
this.clearHandsFreeTimer();
|
||||||
if (useSettings.getState().settings.voice.provider === 'browser') this.browser.stop();
|
if (useSettings.getState().settings.voice.provider === 'browser') this.browser.stop();
|
||||||
else if (this.capture.isActive) this.capture.stop('stopped');
|
else if (this.capture.isActive) this.capture.stop('stopped');
|
||||||
@@ -143,6 +312,7 @@ class VoiceController {
|
|||||||
}
|
}
|
||||||
|
|
||||||
cancel(): void {
|
cancel(): void {
|
||||||
|
this.startSeq++;
|
||||||
this.clearHandsFreeTimer();
|
this.clearHandsFreeTimer();
|
||||||
this.browser.stop();
|
this.browser.stop();
|
||||||
this.capture.cancel();
|
this.capture.cancel();
|
||||||
@@ -162,19 +332,39 @@ class VoiceController {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
private async transcribe(wav: WavResult): Promise<void> {
|
private async transcribe(wav: WavResult, fromWake = false): Promise<void> {
|
||||||
const voice = useVoice.getState();
|
const voice = useVoice.getState();
|
||||||
voice.setPhase('transcribing');
|
voice.setPhase('transcribing');
|
||||||
useChat.getState().setHud('thinking');
|
useChat.getState().setHud('thinking');
|
||||||
this.playChime(false);
|
this.playChime(false);
|
||||||
const settings = useSettings.getState().settings.voice;
|
const settings = useSettings.getState().settings.voice;
|
||||||
try {
|
try {
|
||||||
const text = await transcribeWav(wav.bytes, settings);
|
let text = await transcribeWav(wav.bytes, settings);
|
||||||
voice.setTranscript(text);
|
voice.setTranscript(text);
|
||||||
voice.setPhase('off');
|
voice.setPhase('off');
|
||||||
Log.info('voice', `transcript (${wav.durationSec.toFixed(1)}s): ${text}`);
|
Log.info('voice', `transcript (${wav.durationSec.toFixed(1)}s): ${text}`);
|
||||||
|
const filterByTranscript = settings.wakeMode === 'transcript' || (settings.wakeMode === 'off' && settings.wakeWordEnabled);
|
||||||
|
if (!fromWake && voice.handsFree && filterByTranscript && Date.now() > this.attentionUntil) {
|
||||||
|
const { matched, rest } = matchWakeWord(text, settings.wakeWord || 'jarvis');
|
||||||
|
if (!matched) {
|
||||||
|
Log.debug('voice', 'utterance ignored: no wake word');
|
||||||
|
voice.setTranscript('');
|
||||||
|
useChat.getState().setHud('idle');
|
||||||
|
this.scheduleHandsFree(200);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if (!rest.trim()) {
|
||||||
|
this.attentionUntil = Date.now() + 10_000;
|
||||||
|
useChat.getState().setHud('listening');
|
||||||
|
speech.say('Oui ?');
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
text = rest;
|
||||||
|
}
|
||||||
|
this.attentionUntil = 0;
|
||||||
if (text.trim()) {
|
if (text.trim()) {
|
||||||
await sendMessage(text, [], 'voice');
|
await sendMessage(text, [], 'voice');
|
||||||
|
if (useVoice.getState().handsFree && !speech.isSpeaking()) this.scheduleHandsFree(350);
|
||||||
} else {
|
} else {
|
||||||
voice.setError('Transcription vide.');
|
voice.setError('Transcription vide.');
|
||||||
useChat.getState().setHud('idle');
|
useChat.getState().setHud('idle');
|
||||||
@@ -188,6 +378,7 @@ class VoiceController {
|
|||||||
useChat.getState().setHud('error');
|
useChat.getState().setHud('error');
|
||||||
Log.error('voice', `transcription failed: ${message}`);
|
Log.error('voice', `transcription failed: ${message}`);
|
||||||
setTimeout(() => useChat.getState().setHud('idle'), 3000);
|
setTimeout(() => useChat.getState().setHud('idle'), 3000);
|
||||||
|
if (useVoice.getState().handsFree) this.scheduleHandsFree(3000);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,273 @@
|
|||||||
|
/**
|
||||||
|
* Always-on microphone for the keyword spotter. Streams 16 kHz PCM to the main process
|
||||||
|
* (which feeds the sherpa-onnx spotter); once the wake word is detected it captures the
|
||||||
|
* following utterance with the energy VAD and hands back a WAV, then resumes spotting.
|
||||||
|
* One microphone stream, no re-opening between phrases.
|
||||||
|
*/
|
||||||
|
import { bridge } from '../../lib/bridge';
|
||||||
|
import { Log } from '../../lib/log';
|
||||||
|
import { audioBus } from './audioBus';
|
||||||
|
import { DEFAULT_VAD, EnergyVad, type VadOptions } from './vad';
|
||||||
|
import { buildWav16k, rms, type WavResult } from './wav';
|
||||||
|
import type { VadEvent } from '../../../shared/voice';
|
||||||
|
|
||||||
|
const WORKLET_SOURCE = `
|
||||||
|
class EveFlowWakeProcessor extends AudioWorkletProcessor {
|
||||||
|
constructor() { super(); this.buffer = new Float32Array(2048); this.offset = 0; }
|
||||||
|
process(inputs) {
|
||||||
|
const channel = inputs[0] && inputs[0][0];
|
||||||
|
if (!channel) return true;
|
||||||
|
let i = 0;
|
||||||
|
while (i < channel.length) {
|
||||||
|
const n = Math.min(channel.length - i, this.buffer.length - this.offset);
|
||||||
|
this.buffer.set(channel.subarray(i, i + n), this.offset);
|
||||||
|
this.offset += n; i += n;
|
||||||
|
if (this.offset === this.buffer.length) {
|
||||||
|
this.port.postMessage(this.buffer, [this.buffer.buffer]);
|
||||||
|
this.buffer = new Float32Array(2048); this.offset = 0;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
registerProcessor('eveflow-wake', EveFlowWakeProcessor);
|
||||||
|
`;
|
||||||
|
|
||||||
|
export type WakePhase = 'off' | 'spotting' | 'command' | 'speech';
|
||||||
|
|
||||||
|
export interface WakeCallbacks {
|
||||||
|
onPhase: (phase: WakePhase) => void;
|
||||||
|
onLevel?: (level: number) => void;
|
||||||
|
onUtterance: (wav: WavResult) => void;
|
||||||
|
onNoSpeech: () => void;
|
||||||
|
onError: (message: string) => void;
|
||||||
|
}
|
||||||
|
|
||||||
|
let workletUrl: string | null = null;
|
||||||
|
|
||||||
|
export class WakeListener {
|
||||||
|
private ctx: AudioContext | null = null;
|
||||||
|
private stream: MediaStream | null = null;
|
||||||
|
private node: AudioWorkletNode | null = null;
|
||||||
|
private phase: WakePhase = 'off';
|
||||||
|
private vad: EnergyVad | null = null;
|
||||||
|
private chunks: Float32Array[] = [];
|
||||||
|
private totalSamples = 0;
|
||||||
|
private sampleRate = 16_000;
|
||||||
|
private pending: Float32Array[] = [];
|
||||||
|
private pendingSamples = 0;
|
||||||
|
private callbacks: WakeCallbacks | null = null;
|
||||||
|
private vadOptions: Partial<VadOptions> = {};
|
||||||
|
private noSpeechTimer: ReturnType<typeof setTimeout> | null = null;
|
||||||
|
/** Neural end-of-speech (Silero in the main process) instead of the energy VAD. */
|
||||||
|
private neural = false;
|
||||||
|
private unsubscribeVad: (() => void) | null = null;
|
||||||
|
private vadPending: Float32Array[] = [];
|
||||||
|
private vadPendingSamples = 0;
|
||||||
|
|
||||||
|
get isActive(): boolean {
|
||||||
|
return this.phase !== 'off';
|
||||||
|
}
|
||||||
|
|
||||||
|
get currentPhase(): WakePhase {
|
||||||
|
return this.phase;
|
||||||
|
}
|
||||||
|
|
||||||
|
async start(options: { deviceId?: string; vad?: Partial<VadOptions>; neuralVad?: boolean; callbacks: WakeCallbacks }): Promise<void> {
|
||||||
|
if (this.phase !== 'off') return;
|
||||||
|
this.callbacks = options.callbacks;
|
||||||
|
this.vadOptions = options.vad ?? {};
|
||||||
|
this.neural = !!options.neuralVad;
|
||||||
|
if (this.neural) {
|
||||||
|
this.unsubscribeVad?.();
|
||||||
|
this.unsubscribeVad = bridge()?.voice.onVadEvent((event) => this.onVadEvent(event)) ?? null;
|
||||||
|
}
|
||||||
|
this.stream = await navigator.mediaDevices.getUserMedia({
|
||||||
|
audio: {
|
||||||
|
deviceId: options.deviceId ? { exact: options.deviceId } : undefined,
|
||||||
|
echoCancellation: true,
|
||||||
|
noiseSuppression: true,
|
||||||
|
autoGainControl: true,
|
||||||
|
channelCount: 1
|
||||||
|
}
|
||||||
|
});
|
||||||
|
try {
|
||||||
|
this.ctx = new AudioContext({ sampleRate: 16_000, latencyHint: 'playback' });
|
||||||
|
} catch {
|
||||||
|
this.ctx = new AudioContext({ latencyHint: 'playback' });
|
||||||
|
}
|
||||||
|
if (this.ctx.state === 'suspended') await this.ctx.resume().catch(() => undefined);
|
||||||
|
this.sampleRate = this.ctx.sampleRate;
|
||||||
|
const source = this.ctx.createMediaStreamSource(this.stream);
|
||||||
|
const analyser = this.ctx.createAnalyser();
|
||||||
|
analyser.fftSize = 512;
|
||||||
|
source.connect(analyser);
|
||||||
|
audioBus.setInputAnalyser(analyser);
|
||||||
|
if (!workletUrl) workletUrl = URL.createObjectURL(new Blob([WORKLET_SOURCE], { type: 'application/javascript' }));
|
||||||
|
await this.ctx.audioWorklet.addModule(workletUrl);
|
||||||
|
this.node = new AudioWorkletNode(this.ctx, 'eveflow-wake', { numberOfInputs: 1, numberOfOutputs: 0, channelCount: 1 });
|
||||||
|
this.node.port.onmessage = (event: MessageEvent<Float32Array>) => this.onSamples(event.data);
|
||||||
|
source.connect(this.node);
|
||||||
|
this.setPhase('spotting');
|
||||||
|
Log.info('wake', `listener started (${this.sampleRate} Hz, fin de phrase ${this.neural ? 'Silero' : 'énergie'})`);
|
||||||
|
}
|
||||||
|
|
||||||
|
private onVadEvent(event: VadEvent): void {
|
||||||
|
if (!this.neural || (this.phase !== 'command' && this.phase !== 'speech')) return;
|
||||||
|
if (event.type === 'speech-start') {
|
||||||
|
if (this.phase === 'command') {
|
||||||
|
this.setPhase('speech');
|
||||||
|
if (this.noSpeechTimer) clearTimeout(this.noSpeechTimer);
|
||||||
|
this.noSpeechTimer = null;
|
||||||
|
}
|
||||||
|
} else if (event.type === 'segment') {
|
||||||
|
const wav: WavResult = { bytes: event.wav, sampleRate: 16_000, durationSec: event.durationSec };
|
||||||
|
this.resumeSpotting();
|
||||||
|
if (wav.durationSec >= 0.25) this.callbacks?.onUtterance(wav);
|
||||||
|
else this.callbacks?.onNoSpeech();
|
||||||
|
} else if (event.type === 'error') {
|
||||||
|
this.callbacks?.onError(event.message);
|
||||||
|
this.resumeSpotting();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Called by the controller when the main process reports the wake word (or on manual trigger). */
|
||||||
|
beginCommand(): void {
|
||||||
|
if (this.phase === 'off' || this.phase === 'command' || this.phase === 'speech') return;
|
||||||
|
this.vad = this.neural ? null : new EnergyVad({ ...DEFAULT_VAD, ...this.vadOptions, noSpeechTimeoutMs: Number.POSITIVE_INFINITY });
|
||||||
|
this.vadPending = [];
|
||||||
|
this.vadPendingSamples = 0;
|
||||||
|
this.chunks = [];
|
||||||
|
this.totalSamples = 0;
|
||||||
|
this.setPhase('command');
|
||||||
|
if (this.noSpeechTimer) clearTimeout(this.noSpeechTimer);
|
||||||
|
this.noSpeechTimer = setTimeout(() => {
|
||||||
|
if (this.phase === 'command') {
|
||||||
|
this.resumeSpotting();
|
||||||
|
this.callbacks?.onNoSpeech();
|
||||||
|
}
|
||||||
|
}, 8000);
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Abort a command capture and go back to spotting. */
|
||||||
|
cancelCommand(): void {
|
||||||
|
if (this.phase === 'command' || this.phase === 'speech') this.resumeSpotting();
|
||||||
|
}
|
||||||
|
|
||||||
|
stop(): void {
|
||||||
|
if (this.phase === 'off') return;
|
||||||
|
if (this.noSpeechTimer) clearTimeout(this.noSpeechTimer);
|
||||||
|
this.noSpeechTimer = null;
|
||||||
|
this.unsubscribeVad?.();
|
||||||
|
this.unsubscribeVad = null;
|
||||||
|
audioBus.setInputAnalyser(null);
|
||||||
|
if (this.node) {
|
||||||
|
this.node.port.onmessage = null;
|
||||||
|
this.node.disconnect();
|
||||||
|
this.node = null;
|
||||||
|
}
|
||||||
|
this.stream?.getTracks().forEach((t) => t.stop());
|
||||||
|
this.stream = null;
|
||||||
|
void this.ctx?.close().catch(() => undefined);
|
||||||
|
this.ctx = null;
|
||||||
|
this.chunks = [];
|
||||||
|
this.pending = [];
|
||||||
|
this.pendingSamples = 0;
|
||||||
|
this.vadPending = [];
|
||||||
|
this.vadPendingSamples = 0;
|
||||||
|
this.setPhase('off');
|
||||||
|
Log.info('wake', 'listener stopped');
|
||||||
|
}
|
||||||
|
|
||||||
|
private setPhase(phase: WakePhase): void {
|
||||||
|
this.phase = phase;
|
||||||
|
this.callbacks?.onPhase(phase);
|
||||||
|
}
|
||||||
|
|
||||||
|
private resumeSpotting(): void {
|
||||||
|
if (this.noSpeechTimer) clearTimeout(this.noSpeechTimer);
|
||||||
|
this.noSpeechTimer = null;
|
||||||
|
this.vad = null;
|
||||||
|
this.chunks = [];
|
||||||
|
this.totalSamples = 0;
|
||||||
|
this.setPhase('spotting');
|
||||||
|
}
|
||||||
|
|
||||||
|
private onSamples(samples: Float32Array): void {
|
||||||
|
if (this.phase === 'off') return;
|
||||||
|
const level = rms(samples);
|
||||||
|
this.callbacks?.onLevel?.(Math.min(1, level * 6));
|
||||||
|
|
||||||
|
if (this.phase === 'spotting') {
|
||||||
|
// Batch ~256 ms of audio per IPC message for the keyword spotter.
|
||||||
|
this.pending.push(samples);
|
||||||
|
this.pendingSamples += samples.length;
|
||||||
|
if (this.pendingSamples >= this.sampleRate * 0.25) this.flushToSpotter();
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (this.neural) {
|
||||||
|
// command / speech with Silero: stream ~128 ms frames to the main process, which returns the segment.
|
||||||
|
this.vadPending.push(samples);
|
||||||
|
this.vadPendingSamples += samples.length;
|
||||||
|
if (this.vadPendingSamples >= this.sampleRate * 0.128) this.flushToVad();
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
// command / speech: collect the utterance
|
||||||
|
this.chunks.push(samples);
|
||||||
|
this.totalSamples += samples.length;
|
||||||
|
if (this.vad && this.phase === 'command') {
|
||||||
|
// keep only a short pre-roll before speech starts
|
||||||
|
while (this.chunks.length > 1 && this.totalSamples - this.chunks[0].length > this.sampleRate * 0.4) {
|
||||||
|
this.totalSamples -= this.chunks.shift()!.length;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (!this.vad) return;
|
||||||
|
const signal = this.vad.feed(level, (samples.length / this.sampleRate) * 1000);
|
||||||
|
if (signal === 'speech-start') {
|
||||||
|
this.setPhase('speech');
|
||||||
|
if (this.noSpeechTimer) clearTimeout(this.noSpeechTimer);
|
||||||
|
this.noSpeechTimer = null;
|
||||||
|
} else if (signal === 'speech-end' || signal === 'max-length') {
|
||||||
|
const wav = buildWav16k(this.chunks, this.sampleRate);
|
||||||
|
this.resumeSpotting();
|
||||||
|
if (wav.durationSec >= 0.25) this.callbacks?.onUtterance(wav);
|
||||||
|
else this.callbacks?.onNoSpeech();
|
||||||
|
} else if (signal === 'too-short') {
|
||||||
|
this.resumeSpotting();
|
||||||
|
this.callbacks?.onNoSpeech();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private static toInt16(chunks: Float32Array[], total: number): Uint8Array {
|
||||||
|
const merged = new Int16Array(total);
|
||||||
|
let offset = 0;
|
||||||
|
for (const chunk of chunks) {
|
||||||
|
for (let i = 0; i < chunk.length; i++) {
|
||||||
|
const s = Math.max(-1, Math.min(1, chunk[i]));
|
||||||
|
merged[offset + i] = s < 0 ? s * 0x8000 : s * 0x7fff;
|
||||||
|
}
|
||||||
|
offset += chunk.length;
|
||||||
|
}
|
||||||
|
return new Uint8Array(merged.buffer);
|
||||||
|
}
|
||||||
|
|
||||||
|
private flushToSpotter(): void {
|
||||||
|
const api = bridge();
|
||||||
|
if (!api) return;
|
||||||
|
const bytes = WakeListener.toInt16(this.pending, this.pendingSamples);
|
||||||
|
this.pending = [];
|
||||||
|
this.pendingSamples = 0;
|
||||||
|
api.voice.kwsAudio(bytes, this.sampleRate);
|
||||||
|
}
|
||||||
|
|
||||||
|
private flushToVad(): void {
|
||||||
|
const api = bridge();
|
||||||
|
if (!api) return;
|
||||||
|
const bytes = WakeListener.toInt16(this.vadPending, this.vadPendingSamples);
|
||||||
|
this.vadPending = [];
|
||||||
|
this.vadPendingSamples = 0;
|
||||||
|
api.voice.vadAudio(bytes, this.sampleRate);
|
||||||
|
}
|
||||||
|
}
|
||||||
+11
-1
@@ -53,6 +53,10 @@ interface ChatStore {
|
|||||||
hudOverride: HudState | null;
|
hudOverride: HudState | null;
|
||||||
error: string | null;
|
error: string | null;
|
||||||
draft: string;
|
draft: string;
|
||||||
|
/** Incremented to make the core flash briefly. */
|
||||||
|
pingCount: number;
|
||||||
|
/** Time to first token of the current reply, in ms. */
|
||||||
|
latencyMs: number | null;
|
||||||
|
|
||||||
addMessage: (message: Omit<ChatMessage, 'id' | 'timestamp'> & Partial<Pick<ChatMessage, 'id' | 'timestamp'>>) => string;
|
addMessage: (message: Omit<ChatMessage, 'id' | 'timestamp'> & Partial<Pick<ChatMessage, 'id' | 'timestamp'>>) => string;
|
||||||
updateMessage: (id: string, patch: Partial<ChatMessage> | ((m: ChatMessage) => Partial<ChatMessage>)) => void;
|
updateMessage: (id: string, patch: Partial<ChatMessage> | ((m: ChatMessage) => Partial<ChatMessage>)) => void;
|
||||||
@@ -68,6 +72,8 @@ interface ChatStore {
|
|||||||
setHudOverride: (state: HudState | null) => void;
|
setHudOverride: (state: HudState | null) => void;
|
||||||
setError: (error: string | null) => void;
|
setError: (error: string | null) => void;
|
||||||
setDraft: (draft: string) => void;
|
setDraft: (draft: string) => void;
|
||||||
|
ping: () => void;
|
||||||
|
setLatency: (ms: number | null) => void;
|
||||||
}
|
}
|
||||||
|
|
||||||
const MAX_MESSAGES = 400;
|
const MAX_MESSAGES = 400;
|
||||||
@@ -83,6 +89,8 @@ export const useChat = create<ChatStore>((set, get) => ({
|
|||||||
hudOverride: null,
|
hudOverride: null,
|
||||||
error: null,
|
error: null,
|
||||||
draft: '',
|
draft: '',
|
||||||
|
pingCount: 0,
|
||||||
|
latencyMs: null,
|
||||||
|
|
||||||
addMessage: (message) => {
|
addMessage: (message) => {
|
||||||
const id = message.id ?? uid('msg');
|
const id = message.id ?? uid('msg');
|
||||||
@@ -137,5 +145,7 @@ export const useChat = create<ChatStore>((set, get) => ({
|
|||||||
},
|
},
|
||||||
setHudOverride: (hudOverride) => set({ hudOverride }),
|
setHudOverride: (hudOverride) => set({ hudOverride }),
|
||||||
setError: (error) => set({ error }),
|
setError: (error) => set({ error }),
|
||||||
setDraft: (draft) => set({ draft })
|
setDraft: (draft) => set({ draft }),
|
||||||
|
ping: () => set((s) => ({ pingCount: s.pingCount + 1 })),
|
||||||
|
setLatency: (latencyMs) => set({ latencyMs })
|
||||||
}));
|
}));
|
||||||
+17
-4
@@ -57,6 +57,8 @@ interface HermesStore {
|
|||||||
}
|
}
|
||||||
|
|
||||||
const CACHE_KEY = 'eveflow.hermes.cache.v2';
|
const CACHE_KEY = 'eveflow.hermes.cache.v2';
|
||||||
|
let connectInflight: Promise<void> | null = null;
|
||||||
|
let lastCacheSnapshot = '';
|
||||||
const isTerminal = (status: string) => ['ok', 'failed', 'delivery_failed', 'completed', 'error'].includes(status.toLowerCase());
|
const isTerminal = (status: string) => ['ok', 'failed', 'delivery_failed', 'completed', 'error'].includes(status.toLowerCase());
|
||||||
|
|
||||||
function runFromJob(job: HermesJob): JobRun | null {
|
function runFromJob(job: HermesJob): JobRun | null {
|
||||||
@@ -98,7 +100,9 @@ export const useHermes = create<HermesStore>((set, get) => ({
|
|||||||
return new HermesClient({ ...config, model });
|
return new HermesClient({ ...config, model });
|
||||||
},
|
},
|
||||||
|
|
||||||
connect: async () => {
|
connect: () => {
|
||||||
|
if (connectInflight) return connectInflight;
|
||||||
|
connectInflight = (async () => {
|
||||||
const config = useSettings.getState().settings.hermes;
|
const config = useSettings.getState().settings.hermes;
|
||||||
if (!config.url.trim()) {
|
if (!config.url.trim()) {
|
||||||
set({ link: 'offline', linkDetail: 'URL Hermes non configurée' });
|
set({ link: 'offline', linkDetail: 'URL Hermes non configurée' });
|
||||||
@@ -126,6 +130,10 @@ export const useHermes = create<HermesStore>((set, get) => ({
|
|||||||
set({ link: 'offline', linkDetail: message, transport: resolveTransport(config, null) });
|
set({ link: 'offline', linkDetail: message, transport: resolveTransport(config, null) });
|
||||||
Log.warn('hermes', `connection failed: ${message}`);
|
Log.warn('hermes', `connection failed: ${message}`);
|
||||||
}
|
}
|
||||||
|
})().finally(() => {
|
||||||
|
connectInflight = null;
|
||||||
|
});
|
||||||
|
return connectInflight;
|
||||||
},
|
},
|
||||||
|
|
||||||
refreshCatalog: async () => {
|
refreshCatalog: async () => {
|
||||||
@@ -157,13 +165,18 @@ export const useHermes = create<HermesStore>((set, get) => ({
|
|||||||
const merged = [...incoming, ...previous.filter((r) => !incoming.some((i) => i.id === r.id))]
|
const merged = [...incoming, ...previous.filter((r) => !incoming.some((i) => i.id === r.id))]
|
||||||
.sort((a, b) => new Date(b.at).getTime() - new Date(a.at).getTime())
|
.sort((a, b) => new Date(b.at).getTime() - new Date(a.at).getTime())
|
||||||
.slice(0, 100);
|
.slice(0, 100);
|
||||||
set({ jobs, jobRuns: merged, lastSyncAt: Date.now(), link: get().link === 'offline' ? 'online' : get().link });
|
set({ jobs, jobRuns: merged, lastSyncAt: Date.now(), link: get().link === 'offline' || get().link === 'degraded' ? 'online' : get().link });
|
||||||
persistSet(CACHE_KEY, { jobs, jobRuns: merged, syncedAt: Date.now() });
|
const snapshot = JSON.stringify({ jobs, jobRuns: merged });
|
||||||
|
if (snapshot !== lastCacheSnapshot) {
|
||||||
|
lastCacheSnapshot = snapshot;
|
||||||
|
persistSet(CACHE_KEY, { jobs, jobRuns: merged, syncedAt: Date.now() });
|
||||||
|
}
|
||||||
} catch (err) {
|
} catch (err) {
|
||||||
const message = (err as Error).message;
|
const message = (err as Error).message;
|
||||||
Log.warn('hermes', `jobs sync failed: ${message}`);
|
Log.warn('hermes', `jobs sync failed: ${message}`);
|
||||||
if (/HTTP 404/.test(message)) return; // jobs API disabled on this server
|
if (/HTTP 404/.test(message)) return; // jobs API disabled on this server
|
||||||
set({ link: 'offline', linkDetail: message });
|
// A failing jobs poll does not mean chat is down: degrade, and let the next health probe decide.
|
||||||
|
set({ linkDetail: message, link: get().link === 'online' ? 'degraded' : get().link });
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
|
|
||||||
|
|||||||
+21
-1
@@ -16,6 +16,16 @@ export interface VoiceSettings extends SttConfig {
|
|||||||
silenceMs: number;
|
silenceMs: number;
|
||||||
sensitivity: number; // 1 (low) .. 5 (high)
|
sensitivity: number; // 1 (low) .. 5 (high)
|
||||||
wakeChime: boolean;
|
wakeChime: boolean;
|
||||||
|
/** Hands-free: only react to utterances starting with this word (local STT recommended). */
|
||||||
|
wakeWordEnabled: boolean;
|
||||||
|
wakeWord: string;
|
||||||
|
/** off = push-to-talk / hands-free; transcript = filter after transcription; kws = always-on keyword spotting. */
|
||||||
|
wakeMode: 'off' | 'transcript' | 'kws';
|
||||||
|
kwsSensitivity: number; // 1..5
|
||||||
|
/** Use Silero VAD for end-of-speech in always-on mode when the model is installed. */
|
||||||
|
neuralVad: boolean;
|
||||||
|
/** Execute short system intents locally (lock, volume, open app…) instead of asking Hermes. */
|
||||||
|
localCommands: boolean;
|
||||||
}
|
}
|
||||||
|
|
||||||
export interface SpeechSettings extends TtsConfig {
|
export interface SpeechSettings extends TtsConfig {
|
||||||
@@ -77,7 +87,14 @@ export const DEFAULT_SETTINGS: Settings = {
|
|||||||
micDeviceId: '',
|
micDeviceId: '',
|
||||||
silenceMs: 900,
|
silenceMs: 900,
|
||||||
sensitivity: 3,
|
sensitivity: 3,
|
||||||
wakeChime: true
|
wakeChime: true,
|
||||||
|
wakeWordEnabled: false,
|
||||||
|
wakeWord: 'jarvis',
|
||||||
|
wakeMode: 'off',
|
||||||
|
kwsSensitivity: 3,
|
||||||
|
neuralVad: true,
|
||||||
|
localCommands: true,
|
||||||
|
localModel: 'whisper-base'
|
||||||
},
|
},
|
||||||
speech: {
|
speech: {
|
||||||
provider: 'openai-compatible',
|
provider: 'openai-compatible',
|
||||||
@@ -90,6 +107,8 @@ export const DEFAULT_SETTINGS: Settings = {
|
|||||||
systemVoice: '',
|
systemVoice: '',
|
||||||
language: 'fr-FR',
|
language: 'fr-FR',
|
||||||
volume: 1,
|
volume: 1,
|
||||||
|
localModel: 'kokoro-v1',
|
||||||
|
localSpeaker: 30,
|
||||||
autoSpeak: true,
|
autoSpeak: true,
|
||||||
speakIncoming: true
|
speakIncoming: true
|
||||||
},
|
},
|
||||||
@@ -173,6 +192,7 @@ export const useSettings = create<SettingsStore>((set, get) => ({
|
|||||||
}
|
}
|
||||||
const merged = merge(DEFAULT_SETTINGS, saved);
|
const merged = merge(DEFAULT_SETTINGS, saved);
|
||||||
if (!merged.hermesSessionId) merged.hermesSessionId = uid('eveflow');
|
if (!merged.hermesSessionId) merged.hermesSessionId = uid('eveflow');
|
||||||
|
if (saved && saved.voice && saved.voice.wakeMode === undefined && saved.voice.wakeWordEnabled) merged.voice.wakeMode = 'transcript';
|
||||||
set({ settings: merged, loaded: true });
|
set({ settings: merged, loaded: true });
|
||||||
persistSet(STORAGE_KEY, merged);
|
persistSet(STORAGE_KEY, merged);
|
||||||
},
|
},
|
||||||
|
|||||||
@@ -2,6 +2,7 @@ import { create } from 'zustand';
|
|||||||
import type { TtsState } from '../services/voice/tts';
|
import type { TtsState } from '../services/voice/tts';
|
||||||
|
|
||||||
export type ListenPhase = 'off' | 'arming' | 'listening' | 'speech' | 'transcribing';
|
export type ListenPhase = 'off' | 'arming' | 'listening' | 'speech' | 'transcribing';
|
||||||
|
export type WakeState = 'off' | 'starting' | 'spotting' | 'error';
|
||||||
|
|
||||||
interface VoiceStore {
|
interface VoiceStore {
|
||||||
phase: ListenPhase;
|
phase: ListenPhase;
|
||||||
@@ -12,6 +13,11 @@ interface VoiceStore {
|
|||||||
interim: string;
|
interim: string;
|
||||||
error: string | null;
|
error: string | null;
|
||||||
micDevices: Array<{ deviceId: string; label: string }>;
|
micDevices: Array<{ deviceId: string; label: string }>;
|
||||||
|
wake: WakeState;
|
||||||
|
wakeKeywords: string[];
|
||||||
|
neuralVad: boolean;
|
||||||
|
setNeuralVad: (on: boolean) => void;
|
||||||
|
setWake: (state: WakeState, keywords?: string[]) => void;
|
||||||
setPhase: (phase: ListenPhase) => void;
|
setPhase: (phase: ListenPhase) => void;
|
||||||
setInputLevel: (level: number) => void;
|
setInputLevel: (level: number) => void;
|
||||||
setTts: (state: TtsState) => void;
|
setTts: (state: TtsState) => void;
|
||||||
@@ -31,6 +37,11 @@ export const useVoice = create<VoiceStore>((set) => ({
|
|||||||
interim: '',
|
interim: '',
|
||||||
error: null,
|
error: null,
|
||||||
micDevices: [],
|
micDevices: [],
|
||||||
|
wake: 'off',
|
||||||
|
wakeKeywords: [],
|
||||||
|
neuralVad: false,
|
||||||
|
setNeuralVad: (neuralVad) => set({ neuralVad }),
|
||||||
|
setWake: (wake, wakeKeywords) => set(wakeKeywords ? { wake, wakeKeywords } : { wake }),
|
||||||
setPhase: (phase) => set({ phase }),
|
setPhase: (phase) => set({ phase }),
|
||||||
setInputLevel: (inputLevel) => set({ inputLevel }),
|
setInputLevel: (inputLevel) => set({ inputLevel }),
|
||||||
setTts: (tts) => set({ tts }),
|
setTts: (tts) => set({ tts }),
|
||||||
|
|||||||
@@ -0,0 +1,91 @@
|
|||||||
|
import { create } from 'zustand';
|
||||||
|
import type { VoiceDownloadProgress, VoiceEngineStatus, VoiceModelStatus } from '../../shared/voice';
|
||||||
|
import { bridge } from '../lib/bridge';
|
||||||
|
import { Log } from '../lib/log';
|
||||||
|
|
||||||
|
interface VoiceModelsStore {
|
||||||
|
models: VoiceModelStatus[];
|
||||||
|
progress: Record<string, VoiceDownloadProgress>;
|
||||||
|
engine: VoiceEngineStatus | null;
|
||||||
|
error: string | null;
|
||||||
|
refresh: () => Promise<void>;
|
||||||
|
checkEngine: () => Promise<void>;
|
||||||
|
download: (id: string) => Promise<void>;
|
||||||
|
cancel: (id: string) => Promise<void>;
|
||||||
|
remove: (id: string) => Promise<void>;
|
||||||
|
subscribe: () => () => void;
|
||||||
|
}
|
||||||
|
|
||||||
|
export const useVoiceModels = create<VoiceModelsStore>((set, get) => ({
|
||||||
|
models: [],
|
||||||
|
progress: {},
|
||||||
|
engine: null,
|
||||||
|
error: null,
|
||||||
|
|
||||||
|
refresh: async () => {
|
||||||
|
const api = bridge();
|
||||||
|
if (!api) return;
|
||||||
|
try {
|
||||||
|
set({ models: await api.voice.listModels(), error: null });
|
||||||
|
} catch (err) {
|
||||||
|
set({ error: (err as Error).message });
|
||||||
|
}
|
||||||
|
},
|
||||||
|
checkEngine: async () => {
|
||||||
|
const api = bridge();
|
||||||
|
if (!api) return;
|
||||||
|
try {
|
||||||
|
set({ engine: await api.voice.status() });
|
||||||
|
} catch (err) {
|
||||||
|
set({ engine: { available: false, error: (err as Error).message, loaded: [], modelsDir: '' } });
|
||||||
|
}
|
||||||
|
},
|
||||||
|
download: async (id) => {
|
||||||
|
const api = bridge();
|
||||||
|
if (!api) return;
|
||||||
|
set((s) => ({ progress: { ...s.progress, [id]: { id, phase: 'download', received: 0, total: 0, percent: 0 } }, error: null }));
|
||||||
|
try {
|
||||||
|
await api.voice.downloadModel(id);
|
||||||
|
} catch (err) {
|
||||||
|
const message = (err as Error).message;
|
||||||
|
Log.error('voice', `download ${id} failed: ${message}`);
|
||||||
|
set({ error: message });
|
||||||
|
} finally {
|
||||||
|
await get().refresh();
|
||||||
|
}
|
||||||
|
},
|
||||||
|
cancel: async (id) => {
|
||||||
|
await bridge()?.voice.cancelDownload(id);
|
||||||
|
},
|
||||||
|
remove: async (id) => {
|
||||||
|
const api = bridge();
|
||||||
|
if (!api) return;
|
||||||
|
try {
|
||||||
|
set({ models: await api.voice.removeModel(id) });
|
||||||
|
} catch (err) {
|
||||||
|
set({ error: (err as Error).message });
|
||||||
|
}
|
||||||
|
},
|
||||||
|
subscribe: () => {
|
||||||
|
const api = bridge();
|
||||||
|
if (!api) return () => undefined;
|
||||||
|
return api.voice.onProgress((p) => {
|
||||||
|
set((s) => {
|
||||||
|
const progress = { ...s.progress, [p.id]: p };
|
||||||
|
if (p.phase === 'done' || p.phase === 'error' || p.phase === 'cancelled') {
|
||||||
|
setTimeout(() => {
|
||||||
|
set((s2) => {
|
||||||
|
const next = { ...s2.progress };
|
||||||
|
delete next[p.id];
|
||||||
|
return { progress: next };
|
||||||
|
});
|
||||||
|
void get().refresh();
|
||||||
|
}, 1500);
|
||||||
|
}
|
||||||
|
return { progress };
|
||||||
|
});
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}));
|
||||||
|
|
||||||
|
export const installedModels = (models: VoiceModelStatus[], kind: 'stt' | 'tts') => models.filter((m) => m.kind === kind && m.installed);
|
||||||
+54
-8
@@ -7,6 +7,7 @@
|
|||||||
grid-template-rows: 46px 1fr;
|
grid-template-rows: 46px 1fr;
|
||||||
border-radius: 14px;
|
border-radius: 14px;
|
||||||
overflow: hidden;
|
overflow: hidden;
|
||||||
|
background-clip: padding-box;
|
||||||
background:
|
background:
|
||||||
radial-gradient(1200px 700px at 50% 40%, rgba(var(--accent-rgb), 0.08), transparent 60%),
|
radial-gradient(1200px 700px at 50% 40%, rgba(var(--accent-rgb), 0.08), transparent 60%),
|
||||||
linear-gradient(180deg, var(--bg-1) 0%, var(--bg-0) 100%);
|
linear-gradient(180deg, var(--bg-1) 0%, var(--bg-0) 100%);
|
||||||
@@ -51,6 +52,17 @@
|
|||||||
grid-template-columns: minmax(340px, 1fr) minmax(380px, 1fr);
|
grid-template-columns: minmax(340px, 1fr) minmax(380px, 1fr);
|
||||||
}
|
}
|
||||||
.hud-body > .ops-column {
|
.hud-body > .ops-column {
|
||||||
|
position: absolute;
|
||||||
|
right: 14px;
|
||||||
|
top: 12px;
|
||||||
|
bottom: 14px;
|
||||||
|
width: min(360px, 60%);
|
||||||
|
z-index: 6;
|
||||||
|
background: var(--bg-1);
|
||||||
|
border-radius: var(--radius);
|
||||||
|
box-shadow: -20px 0 60px rgba(0, 0, 0, 0.5);
|
||||||
|
}
|
||||||
|
.hud-body > .ops-column:not(.open) {
|
||||||
display: none;
|
display: none;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -68,7 +80,6 @@
|
|||||||
background: var(--panel);
|
background: var(--panel);
|
||||||
border: 1px solid var(--line);
|
border: 1px solid var(--line);
|
||||||
border-radius: var(--radius);
|
border-radius: var(--radius);
|
||||||
backdrop-filter: blur(14px);
|
|
||||||
display: flex;
|
display: flex;
|
||||||
flex-direction: column;
|
flex-direction: column;
|
||||||
min-height: 0;
|
min-height: 0;
|
||||||
@@ -310,7 +321,8 @@
|
|||||||
background: linear-gradient(180deg, rgba(var(--accent-rgb), 0.06), transparent);
|
background: linear-gradient(180deg, rgba(var(--accent-rgb), 0.06), transparent);
|
||||||
-webkit-app-region: drag;
|
-webkit-app-region: drag;
|
||||||
}
|
}
|
||||||
.topbar > * {
|
.topbar button,
|
||||||
|
.topbar .chip {
|
||||||
-webkit-app-region: no-drag;
|
-webkit-app-region: no-drag;
|
||||||
}
|
}
|
||||||
.topbar .brand {
|
.topbar .brand {
|
||||||
@@ -346,9 +358,6 @@
|
|||||||
overflow: hidden;
|
overflow: hidden;
|
||||||
-webkit-app-region: drag;
|
-webkit-app-region: drag;
|
||||||
}
|
}
|
||||||
.topbar .status-strip > * {
|
|
||||||
-webkit-app-region: no-drag;
|
|
||||||
}
|
|
||||||
.topbar .clock {
|
.topbar .clock {
|
||||||
font-family: var(--font-mono);
|
font-family: var(--font-mono);
|
||||||
font-size: 13px;
|
font-size: 13px;
|
||||||
@@ -421,6 +430,21 @@
|
|||||||
.core-state.success {
|
.core-state.success {
|
||||||
color: var(--success);
|
color: var(--success);
|
||||||
}
|
}
|
||||||
|
.core-interim {
|
||||||
|
font-family: var(--font-ui);
|
||||||
|
font-size: 15px;
|
||||||
|
color: var(--ink-1);
|
||||||
|
max-width: 82%;
|
||||||
|
text-align: center;
|
||||||
|
opacity: 0.85;
|
||||||
|
min-height: 1.4em;
|
||||||
|
}
|
||||||
|
.core-latency {
|
||||||
|
font-family: var(--font-mono);
|
||||||
|
font-size: 10px;
|
||||||
|
letter-spacing: 0.2em;
|
||||||
|
color: var(--ink-3);
|
||||||
|
}
|
||||||
.core-sub {
|
.core-sub {
|
||||||
font-family: var(--font-mono);
|
font-family: var(--font-mono);
|
||||||
font-size: 12px;
|
font-size: 12px;
|
||||||
@@ -438,10 +462,13 @@
|
|||||||
right: 0;
|
right: 0;
|
||||||
text-align: center;
|
text-align: center;
|
||||||
font-family: var(--font-display);
|
font-family: var(--font-display);
|
||||||
font-size: 22px;
|
font-size: clamp(13px, 2.2vw, 22px);
|
||||||
font-weight: 900;
|
font-weight: 900;
|
||||||
letter-spacing: 0.5em;
|
letter-spacing: clamp(0.15em, 0.8vw, 0.5em);
|
||||||
padding-left: 0.5em;
|
padding: 0 12px 0 calc(12px + 0.4em);
|
||||||
|
white-space: nowrap;
|
||||||
|
overflow: hidden;
|
||||||
|
text-overflow: ellipsis;
|
||||||
color: var(--ink-0);
|
color: var(--ink-0);
|
||||||
text-shadow: 0 0 22px var(--accent-glow);
|
text-shadow: 0 0 22px var(--accent-glow);
|
||||||
pointer-events: none;
|
pointer-events: none;
|
||||||
@@ -474,6 +501,25 @@
|
|||||||
margin-left: auto;
|
margin-left: auto;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/* ── Toasts ────────────────────────────────────────────────────────────── */
|
||||||
|
.toast-stack {
|
||||||
|
position: absolute;
|
||||||
|
left: 50%;
|
||||||
|
bottom: 22px;
|
||||||
|
transform: translateX(-50%);
|
||||||
|
z-index: 18;
|
||||||
|
display: flex;
|
||||||
|
flex-direction: column;
|
||||||
|
gap: 8px;
|
||||||
|
width: min(560px, 80%);
|
||||||
|
pointer-events: none;
|
||||||
|
}
|
||||||
|
.toast-stack .error-banner {
|
||||||
|
pointer-events: auto;
|
||||||
|
box-shadow: 0 20px 50px rgba(0, 0, 0, 0.5);
|
||||||
|
animation: rise-in 200ms ease-out;
|
||||||
|
}
|
||||||
|
|
||||||
/* ── Overlay / modal ───────────────────────────────────────────────────── */
|
/* ── Overlay / modal ───────────────────────────────────────────────────── */
|
||||||
.overlay {
|
.overlay {
|
||||||
position: absolute;
|
position: absolute;
|
||||||
|
|||||||
@@ -11,7 +11,6 @@
|
|||||||
display: flex;
|
display: flex;
|
||||||
flex-direction: column;
|
flex-direction: column;
|
||||||
gap: 12px;
|
gap: 12px;
|
||||||
scroll-behavior: smooth;
|
|
||||||
}
|
}
|
||||||
.chat-list.compact {
|
.chat-list.compact {
|
||||||
padding: 8px 10px;
|
padding: 8px 10px;
|
||||||
|
|||||||
+10
-2
@@ -7,7 +7,15 @@
|
|||||||
background: linear-gradient(180deg, rgba(5, 12, 22, var(--compact-opacity, 0.92)), rgba(2, 6, 12, var(--compact-opacity, 0.92)));
|
background: linear-gradient(180deg, rgba(5, 12, 22, var(--compact-opacity, 0.92)), rgba(2, 6, 12, var(--compact-opacity, 0.92)));
|
||||||
border: 1px solid rgba(var(--accent-rgb), 0.3);
|
border: 1px solid rgba(var(--accent-rgb), 0.3);
|
||||||
box-shadow: 0 20px 60px rgba(0, 0, 0, 0.6), 0 0 30px rgba(var(--accent-rgb), 0.12);
|
box-shadow: 0 20px 60px rgba(0, 0, 0, 0.6), 0 0 30px rgba(var(--accent-rgb), 0.12);
|
||||||
backdrop-filter: blur(16px);
|
}
|
||||||
|
.compact-body .panel {
|
||||||
|
background: transparent;
|
||||||
|
border: none;
|
||||||
|
border-radius: 0;
|
||||||
|
}
|
||||||
|
.compact-body .panel.bracket::before,
|
||||||
|
.compact-body .panel.bracket::after {
|
||||||
|
display: none;
|
||||||
}
|
}
|
||||||
.compact-head {
|
.compact-head {
|
||||||
display: flex;
|
display: flex;
|
||||||
@@ -29,7 +37,7 @@
|
|||||||
}
|
}
|
||||||
.compact-core {
|
.compact-core {
|
||||||
position: relative;
|
position: relative;
|
||||||
height: 220px;
|
height: clamp(140px, 38vh, 220px);
|
||||||
flex-shrink: 0;
|
flex-shrink: 0;
|
||||||
}
|
}
|
||||||
.compact-core canvas {
|
.compact-core canvas {
|
||||||
|
|||||||
@@ -104,3 +104,14 @@
|
|||||||
background: linear-gradient(90deg, var(--accent), var(--success));
|
background: linear-gradient(90deg, var(--accent), var(--success));
|
||||||
transition: width 60ms linear;
|
transition: width 60ms linear;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
.switch {
|
||||||
|
width: 100%;
|
||||||
|
background: transparent;
|
||||||
|
border-left: none;
|
||||||
|
border-right: none;
|
||||||
|
border-top: none;
|
||||||
|
color: inherit;
|
||||||
|
text-align: left;
|
||||||
|
font: inherit;
|
||||||
|
}
|
||||||
+21
-3
@@ -1,5 +1,3 @@
|
|||||||
@import url('https://fonts.googleapis.com/css2?family=Orbitron:wght@500;700;900&family=Rajdhani:wght@400;500;600;700&family=Share+Tech+Mono&display=swap');
|
|
||||||
|
|
||||||
:root {
|
:root {
|
||||||
--font-display: 'Orbitron', 'Rajdhani', 'Segoe UI', sans-serif;
|
--font-display: 'Orbitron', 'Rajdhani', 'Segoe UI', sans-serif;
|
||||||
--font-ui: 'Rajdhani', 'Segoe UI', 'Helvetica Neue', sans-serif;
|
--font-ui: 'Rajdhani', 'Segoe UI', 'Helvetica Neue', sans-serif;
|
||||||
@@ -137,7 +135,27 @@ textarea {
|
|||||||
background: rgba(var(--accent-rgb), 0.45);
|
background: rgba(var(--accent-rgb), 0.45);
|
||||||
}
|
}
|
||||||
|
|
||||||
:root[data-motion='reduce'] * {
|
:root[data-motion='reduce'] *,
|
||||||
|
:root[data-motion='reduce'] *::before,
|
||||||
|
:root[data-motion='reduce'] *::after {
|
||||||
|
animation-iteration-count: 1 !important;
|
||||||
animation-duration: 0.001s !important;
|
animation-duration: 0.001s !important;
|
||||||
transition-duration: 0.001s !important;
|
transition-duration: 0.001s !important;
|
||||||
}
|
}
|
||||||
|
@media (prefers-reduced-motion: reduce) {
|
||||||
|
:root:not([data-motion='full']) *,
|
||||||
|
:root:not([data-motion='full']) *::before,
|
||||||
|
:root:not([data-motion='full']) *::after {
|
||||||
|
animation-iteration-count: 1 !important;
|
||||||
|
animation-duration: 0.001s !important;
|
||||||
|
transition-duration: 0.001s !important;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
.error-banner span,
|
||||||
|
.activity-item .detail,
|
||||||
|
.row-item .desc,
|
||||||
|
.row-item .meta,
|
||||||
|
.core-sub,
|
||||||
|
.kv dd {
|
||||||
|
user-select: text;
|
||||||
|
}
|
||||||
@@ -0,0 +1,25 @@
|
|||||||
|
import { describe, expect, it } from 'vitest';
|
||||||
|
import { buildKeywordsFile, encodeKeyword, normalizeKeyword, parseTokens } from '../shared/keywords';
|
||||||
|
|
||||||
|
const vocab = parseTokens(['<blk> 0', '▁ 3', '▁JA 4', 'R 5', 'VI 6', 'S 7', '▁HE 8', 'Y 9', '▁NO 10', 'V 11', 'A 12', '▁MA 13', 'X 14', '▁T 15', 'ON 16', 'Y 17'].join('\n'));
|
||||||
|
|
||||||
|
describe('keywords', () => {
|
||||||
|
it('normalises accents, case and punctuation', () => {
|
||||||
|
expect(normalizeKeyword(' Hé, Jarvis ! ')).toBe('HE JARVIS');
|
||||||
|
});
|
||||||
|
it('uses the known SentencePiece encodings', () => {
|
||||||
|
expect(encodeKeyword('jarvis', vocab)).toBe('▁JA R VI S');
|
||||||
|
expect(encodeKeyword('Hey Jarvis', vocab)).toBe('▁HE Y ▁JA R VI S');
|
||||||
|
});
|
||||||
|
it('falls back to greedy longest match', () => {
|
||||||
|
expect(encodeKeyword('max', vocab)).toBe('▁MA X');
|
||||||
|
expect(encodeKeyword('tony', vocab)).toBe('▁T ON Y');
|
||||||
|
expect(encodeKeyword('zzz', vocab)).toBeNull();
|
||||||
|
});
|
||||||
|
it('builds a keywords file with labels', () => {
|
||||||
|
const file = buildKeywordsFile(['jarvis', 'hey jarvis', 'zzz'], vocab);
|
||||||
|
expect(file.content).toBe('▁JA R VI S @jarvis\n▁HE Y ▁JA R VI S @hey_jarvis\n');
|
||||||
|
expect(file.accepted).toEqual(['jarvis', 'hey_jarvis']);
|
||||||
|
expect(file.rejected).toEqual(['zzz']);
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -0,0 +1,24 @@
|
|||||||
|
import { describe, expect, it } from 'vitest';
|
||||||
|
import { parseLocalIntent } from '../src/services/localCommands';
|
||||||
|
|
||||||
|
describe('parseLocalIntent', () => {
|
||||||
|
it('recognises system actions in French and English', () => {
|
||||||
|
expect(parseLocalIntent('Jarvis, verrouille la session')?.action).toEqual({ type: 'lock' });
|
||||||
|
expect(parseLocalIntent('monte le son')?.action).toEqual({ type: 'media', key: 'volume-up' });
|
||||||
|
expect(parseLocalIntent('Baisse le volume')?.action).toEqual({ type: 'media', key: 'volume-down' });
|
||||||
|
expect(parseLocalIntent('coupe le son')?.action).toEqual({ type: 'media', key: 'mute' });
|
||||||
|
expect(parseLocalIntent('piste suivante')?.action).toEqual({ type: 'media', key: 'next' });
|
||||||
|
expect(parseLocalIntent('ouvre le bloc-notes')?.action).toEqual({ type: 'open-app', name: 'bloc-notes' });
|
||||||
|
expect(parseLocalIntent('open spotify')?.action).toEqual({ type: 'open-app', name: 'spotify' });
|
||||||
|
expect(parseLocalIntent('ouvre github.com')?.action).toEqual({ type: 'open-url', url: 'https://github.com' });
|
||||||
|
});
|
||||||
|
it('detects screen questions', () => {
|
||||||
|
expect(parseLocalIntent('Jarvis, regarde mon écran et dis-moi ce que tu vois')?.kind).toBe('screenshot');
|
||||||
|
expect(parseLocalIntent('fais une capture d’écran')?.kind).toBe('screenshot');
|
||||||
|
});
|
||||||
|
it('leaves everything else to Hermes', () => {
|
||||||
|
expect(parseLocalIntent('Quelle est la météo à Paris demain ?')).toBeNull();
|
||||||
|
expect(parseLocalIntent('ouvre le fichier puis envoie-le à Marc')).toBeNull();
|
||||||
|
expect(parseLocalIntent('')).toBeNull();
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -0,0 +1,16 @@
|
|||||||
|
import { describe, expect, it } from 'vitest';
|
||||||
|
import { matchWakeWord } from '../src/services/voice/voiceController';
|
||||||
|
|
||||||
|
describe('matchWakeWord', () => {
|
||||||
|
it('accepts the wake word with punctuation, accents and small slips', () => {
|
||||||
|
expect(matchWakeWord('Jarvis, quelle heure est-il ?', 'jarvis')).toEqual({ matched: true, rest: 'quelle heure est-il ?' });
|
||||||
|
expect(matchWakeWord('Javis allume la lumière', 'jarvis')).toEqual({ matched: true, rest: 'allume la lumière' });
|
||||||
|
expect(matchWakeWord('JARVIS.', 'jarvis')).toEqual({ matched: true, rest: '' });
|
||||||
|
expect(matchWakeWord('Hé Jarvis, bonjour', 'hé jarvis')).toEqual({ matched: true, rest: 'bonjour' });
|
||||||
|
});
|
||||||
|
it('rejects unrelated speech', () => {
|
||||||
|
expect(matchWakeWord('Il fait beau aujourd’hui', 'jarvis').matched).toBe(false);
|
||||||
|
expect(matchWakeWord('Service client', 'jarvis').matched).toBe(false);
|
||||||
|
expect(matchWakeWord('', 'jarvis').matched).toBe(false);
|
||||||
|
});
|
||||||
|
});
|
||||||
Reference in new issue
Block a user