Compare commits

...
22 Commits
Author SHA1 Message Date
LogiFlowandClaude Fable 5.1 5ef440e8cf feat: voix JARVIS, Parakeet v3 pour le français, worklets audio sous CSP stricte (v2.4.1) (#19)
* feat: voix JARVIS, Parakeet v3 pour le français, worklets audio sous CSP stricte (v2.4.1)

Voix
- Timbre « JARVIS » (Web Audio) : hauteur légèrement abaissée, chaleur dans
  les basses, présence, compression douce, courte réverbération d'intercom.
  Activé par défaut, réglable dans Paramètres → Voix.
- Préréglage « Voix JARVIS » en un clic : moteur local, voix masculine
  française (Piper Tom téléchargé automatiquement, Kokoro n'ayant pas de voix
  française masculine), débit calme. Affichage de la voix active.
- Boutons Masculine / Féminine désormais lisibles (style du sélecteur ajouté).

Reconnaissance
- Parakeet TDT 0.6B v3 (NVIDIA NeMo, int8, 25 langues européennes dont le
  français) ajouté au catalogue et recommandé : plus précis et plus rapide que
  Whisper sur processeur, ponctuation incluse.
- 0,4 s de silence ajoutées avant et après chaque énoncé avant la
  reconnaissance (syllabes coupées, hallucinations de Whisper sur les clips
  courts).

Écoute permanente
- Les modules AudioWorklet sont livrés en fichiers statiques (public/worklets)
  chargés depuis l'application : en version installée, la CSP stricte
  (script-src 'self') refusait les URL blob et l'écoute permanente échouait
  avec « Unable to load a worklet's module ». Repli blob conservé pour le dev.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Wn5VX9HNbJ7N54hR24u9Y

* docs: mesures Parakeet vs Whisper et worklets dans la feuille de route

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Wn5VX9HNbJ7N54hR24u9Y

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-09-04 13:02:32 +02:00
LogiFlowandClaude d1b12c16c5 fix: portail laissant passer /health mais bloquant /v1 traité comme liaison en échec (v2.4.0.3) (#18)
- Une page web reçue sur /v1/capabilities fait échouer la connexion et le test
  de liaison même si /health répond, ce qui déclenche la recherche de l'API.
- La découverte sonde /v1/capabilities puis /v1/models (JSON exigé, 401/403
  accepté) avant /health, pour ne pas prendre un portail pour l'API.


Claude-Session: https://claude.ai/code/session_017Wn5VX9HNbJ7N54hR24u9Y

Co-authored-by: Claude <noreply@anthropic.com>
2026-09-04 08:22:19 +02:00
LogiFlowandClaude df25dd591c fix: page web reçue à la place de l'API Hermes détectée, URL découverte automatiquement (v2.4.0.2) (#17)
- Toute réponse HTML (portail de connexion, tableau de bord, erreur de proxy)
  est reconnue et expliquée avec le titre de la page, au lieu d'être prise
  pour un état « ok » ou de produire une réponse vide.
- Découverte automatique : quand la liaison échoue, EveFlow essaie l'API sur le
  même hôte (port 8642, /api, /v1, sous-domaines api. et hermes.) et corrige
  l'URL si un serveur Hermes répond. Le bouton « Tester la liaison » fait de
  même et affiche les essais.

Tests : 47 tests unitaires ; e2e Electron vert.


Claude-Session: https://claude.ai/code/session_017Wn5VX9HNbJ7N54hR24u9Y

Co-authored-by: Claude <noreply@anthropic.com>
2026-09-04 08:06:05 +02:00
LogiFlow c75c1d674c Merge pull request #16 from R0m1k3/claude/refonte-app-vocale-v91uz7
fix: liaison « dégradé », réponses vides, transcriptions parasites, v…
2026-09-04 07:53:40 +02:00
LogiFlowandClaude 6d3073b083 fix: liaison « dégradé », réponses vides, transcriptions parasites, voix masculine (v2.4.0.1) (#15)
- Liaison Hermes : l'échec de l'API des crons (/api/jobs absente ou refusée)
  ne fait plus passer la liaison en « dégradé » ; l'onglet Crons affiche la
  raison. Les états « healthy / ready / ok » sont reconnus et un état
  « degraded » détaille les contrôles en échec.
- Chat completions : une réponse sans fragment n'affiche plus « … » ; le corps
  est relu (JSON non streamé, erreur renvoyée en HTTP 200) et sinon l'erreur
  explicite « Réponse vide de Hermes » est affichée avec le début du corps.
- Voix : transcriptions parasites de Whisper (« (cliquant) », « *Claire* »,
  « [Musique] », génériques de sous-titres) ignorées au lieu d'être envoyées.
- Préférence de voix masculine / féminine (Paramètres → Voix), appliquée à
  tous les moteurs : Piper Tom / Pierre en local (téléchargement automatique),
  onyx / nova en API OpenAI, Paul / Hortense en voix Windows. Genre des voix
  du catalogue déduit des libellés.

Tests : 42 tests unitaires (santé, récupération de réponse, filtre de bruit,
préférence de voix) ; e2e Electron inchangé et vert.


Claude-Session: https://claude.ai/code/session_017Wn5VX9HNbJ7N54hR24u9Y

Co-authored-by: Claude <noreply@anthropic.com>
2026-09-04 07:44:07 +02:00
Claude 31659500ab fix: liaison « dégradé », réponses vides, transcriptions parasites, voix masculine (v2.4.0.1)
- Liaison Hermes : l'échec de l'API des crons (/api/jobs absente ou refusée)
  ne fait plus passer la liaison en « dégradé » ; l'onglet Crons affiche la
  raison. Les états « healthy / ready / ok » sont reconnus et un état
  « degraded » détaille les contrôles en échec.
- Chat completions : une réponse sans fragment n'affiche plus « … » ; le corps
  est relu (JSON non streamé, erreur renvoyée en HTTP 200) et sinon l'erreur
  explicite « Réponse vide de Hermes » est affichée avec le début du corps.
- Voix : transcriptions parasites de Whisper (« (cliquant) », « *Claire* »,
  « [Musique] », génériques de sous-titres) ignorées au lieu d'être envoyées.
- Préférence de voix masculine / féminine (Paramètres → Voix), appliquée à
  tous les moteurs : Piper Tom / Pierre en local (téléchargement automatique),
  onyx / nova en API OpenAI, Paul / Hortense en voix Windows. Genre des voix
  du catalogue déduit des libellés.

Tests : 42 tests unitaires (santé, récupération de réponse, filtre de bruit,
préférence de voix) ; e2e Electron inchangé et vert.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Wn5VX9HNbJ7N54hR24u9Y
2026-09-04 05:43:41 +00:00
LogiFlowandClaude 6fc38d335b feat: serveur MCP, heures calmes, mode mission, widget glanceable, barge-in (v2.4.0) (#14)
Serveur MCP local
- Endpoint /mcp (JSON-RPC 2.0, Streamable HTTP, réponses JSON) sur le serveur
  webhook : initialize, tools/list, tools/call, ping. Même secret que le
  webhook. 14 outils : capture_screen (image MCP), lock_session, open_app,
  open_url, media_key, clipboard_get/set, find_files, speak_text,
  notify_user, set_hud_state, get_app_status, get_conversation_history,
  show_message. Les outils UI transitent par IPC vers le renderer.
- Les mêmes actions système sont proposées au modèle en chat completions.

Notifications
- Heures calmes (plage horaire, franchissement de minuit), mots prioritaires,
  échecs de crons toujours lus, résumé vocal des messages entrants (n premières
  phrases), thème nuit automatique, compteur de non-lus.

Conversation
- Mode mission : bouton dans la barre de commande, second modèle Hermes
  (hermes.missionModel) pour les tâches longues.
- Barge-in : la parole de l'utilisateur coupe la voix ; seuil d'énergie et
  ratio relevés pendant la synthèse pour ignorer l'écho.

Widget compact
- Bandeau glanceable : état, dernière phrase, non-lus, indicateurs heures
  calmes et mission.

Tests : 28 tests unitaires (heures calmes, priorité, résumé), e2e Electron
sous Xvfb : MCP initialize/tools/list/tools/call (presse-papiers, capture en
image, état et message via le renderer), bandeau compact, aucune erreur.


Claude-Session: https://claude.ai/code/session_017Wn5VX9HNbJ7N54hR24u9Y

Co-authored-by: Claude <noreply@anthropic.com>
2026-09-04 00:47:34 +02:00
LogiFlowandClaude af8bdc5e9d feat: fin de phrase Silero, vision d'écran et actions locales (v2.3.0) (#13)
Écoute permanente
- Silero VAD dans le worker sherpa-onnx (vad.start/audio/stop) ; le renderer
  envoie des trames de 128 ms pendant la commande et reçoit le segment WAV
  complet (speech-start / segment). Repli sur le VAD énergétique si le modèle
  n'est pas installé ou si l'option est désactivée.
- Modèle « silero-vad » (0,6 Mo) dans le catalogue, section « Fin de phrase »
  dans Modèles locaux, réglage dans Paramètres → Micro.

Vision d'écran
- IPC system:screen-capture (desktopCapturer, JPEG 1600 px) ; bouton dans la
  barre de commande ; « regarde mon écran… » joint la capture à la question
  envoyée à Hermes (transport chat completions pour les images).

Actions locales
- IPC system:action à liste blanche : verrouiller la session, touches média
  (volume, mute, lecture, piste), ouvrir une application connue ou une URL
  http(s), presse-papiers, recherche de fichiers dans les dossiers utilisateur.
- Routeur d'intentions FR/EN exécuté avant Hermes (« coupe le son », « ouvre
  Spotify », « verrouille la session »…), résultat affiché et lu ; désactivable.

Correctif
- Chat completions : les premiers fragments SSE arrivés avant l'événement de
  démarrage étaient perdus (premier mot manquant) ; ils sont maintenant rejoués.

Tests : 26 tests unitaires (intentions locales ajoutées), e2e Electron sous
Xvfb avec les vrais modèles (Silero : un segment par phrase, 6,7 s d'audio
traités en 190 ms ; capture 109 ko reçue par Hermes ; intention locale
traitée sans Hermes).


Claude-Session: https://claude.ai/code/session_017Wn5VX9HNbJ7N54hR24u9Y

Co-authored-by: Claude <noreply@anthropic.com>
2026-09-03 20:46:26 +02:00
LogiFlow 4f6074183f feat: écoute permanente par détection de mot-clé (v2.2.0) (#12)
feat: écoute permanente par détection de mot-clé (v2.2.0)
2026-09-03 20:22:00 +02:00
Claude 0d284fea9e feat: always-on wake word with sherpa-onnx keyword spotting (v2.2.0)
- Catalog: 3.3 MB zipformer keyword-spotting model (kws-en).
- shared/keywords: BPE encoding of wake phrases (SentencePiece table for common words,
  greedy longest-match fallback over the model vocabulary) and keywords file builder.
- Worker: KeywordSpotter stream fed with 16-bit PCM, detections pushed as unsolicited
  messages; engine derives the keywords file, maps sensitivity to threshold/score,
  forwards detections to the renderer and re-arms after a worker restart.
- Renderer: WakeListener keeps one microphone stream, batches 256 ms frames to the
  spotter and captures the command on the same stream after detection (pre-roll, VAD),
  then resumes spotting; the wake word also interrupts speech. Settings: wake mode
  (off / always-on / transcript filter), keyword, sensitivity, status and one-click
  model download; HUD caption shows the active keyword.
- Validated: detection in the worker (fork) and through the real Electron IPC path on
  Kokoro audio, no false positive on an English recording; 23 unit tests.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Wn5VX9HNbJ7N54hR24u9Y
2026-09-03 18:21:35 +00:00
LogiFlow 6d2f8f4ec9 fix: passe de revue complète, test Electron réel et feuille de route JARVIS (v2.1.0.1) (#11)
fix: passe de revue complète, test Electron réel et feuille de route JARVIS (v2.1.0.1)
2026-09-03 20:08:36 +02:00
Claude 5f0fa7a1e2 fix: full review pass (main process, voice pipeline, Hermes client, HUD) and e2e-validated local voice (v2.1.0.1)
Electron main
- utility-process channel: sherpa output copied into V8 buffers (Electron rejects external
  buffers), audio exchanged as base64; worker restart on timeout, identity-safe exit handling,
  deterministic native unload by restart
- webhook: loopback by default without secret, UTF-8-safe body assembly, clean restart, port validation
- files IPC: realpath-based root check, openPath allow-list (reveal-only outside document folders),
  Windows reserved names; store: debounced async atomic writes with backup of corrupt files;
  logger: streaming writes with rotation; log message size cap
- HTTP stream proxy: socket released on idle timeout / renderer destroyed, id validation
- model manager: inactivity timeout, retrying rm/rename (Windows locks), engine stopped before
  replacing a model; WAV decoder handles float/24-bit; IPC payload validation
- window: opaque rounded window on Windows, navigation lock-down, visibility events;
  Ctrl+Alt+Escape instead of the Task Manager shortcut; single-instance guard; EVEFLOW_USER_DATA

Voice pipeline
- abort semantics (SendHandle.aborted), abort before the stream opens, session id prefixes per transport
- hands-free re-arm after replies without speech, start/stop race, no chime on auto re-arm,
  no silence shipped to STT (400 ms pre-roll), no transcription of empty manual stops
- TTS: bounded prefetch, cancellable segments, non-interrupting notices, volume applied at play time
- SSE CRLF split, usage in chat completions, finish_reason length, phonetic regex hoisted

HUD
- core renderer: no canvas shadows, cached colours, reusable spectrum buffer, theme read on change,
  30 fps idle, stops when the window is hidden; ping flashes on send / tool / speech
- deltas coalesced per animation frame; stable auto-scroll; narrow selectors everywhere
- bundled fonts (offline), reduce-motion fix, error toasts, ops drawer below 1180 px,
  interim transcript and first-token latency in the core caption, Ctrl+K, dialog semantics,
  switch/aria roles, compact widget cleanup, Whisper small recommended for French
- docs/ROADMAP.md: audit results, e2e results and the plan towards a real JARVIS

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Wn5VX9HNbJ7N54hR24u9Y
2026-09-03 18:08:09 +00:00
LogiFlow b1b8d0d5d4 feat: modèles vocaux intégrés hors ligne (sherpa-onnx) et mot d'activation (v2.1.0) (#10)
feat: modèles vocaux intégrés hors ligne (sherpa-onnx), gestionnaire de modèles et mot d'activation (v2.1.0)
2026-09-03 19:44:22 +02:00
Claude eefc36aca8 feat: local offline voice models (sherpa-onnx) with model manager and wake word (v2.1.0)
- electron/voice: model catalog (Whisper base/small/turbo, SenseVoice, Kokoro v1.0,
  Piper fr), streaming tar.bz2 downloader with progress, sherpa-onnx worker running in
  an Electron utilityProcess (transcribe / synthesize / status / unload), IPC + bridge.
- Renderer: 'local' providers for STT and TTS, Settings → Modèles locaux (download,
  progress, delete, activate), speaker selection, hands-free wake word with a tolerant
  matcher (accents, punctuation, edit distance) and attention window after a bare wake word.
- Packaging: native addon and worker unpacked from the asar; release 2.1.0.

Validated on Linux: Kokoro (fr) → Whisper base round trip, Piper (fr) → Whisper round trip.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Wn5VX9HNbJ7N54hR24u9Y
2026-09-03 17:43:52 +00:00
LogiFlow af1e3d8aae build: numéro de release 2.0.0.1 injecté au packaging (#9)
build: numéro de release 2.0.0.1 injecté au packaging (electron-builder exige un semver)
2026-09-03 17:47:50 +02:00
Claude 2471add00b build: four-part releaseVersion (2.0.0.1) injected at packaging time
electron-builder refuses a non-semver package.json version, so the published
number lives in package.json releaseVersion and scripts/dist.mjs passes it as
extraMetadata.version (plus the hotfix digit as Windows build number). The
workflow tags and names the release from the same field.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Wn5VX9HNbJ7N54hR24u9Y
2026-09-03 15:47:32 +00:00
LogiFlow 35437e2d51 chore: version 2.0.0.1 et télémétrie sans fausses valeurs (#8)
chore: version 2.0.0.1 (schéma correctif à quatre chiffres) et télémétrie sans fausses valeurs
2026-09-03 17:43:37 +02:00
Claude e2c67a5770 chore: hotfix versioning 2.0.0.1 and honest telemetry placeholders when the bridge is missing
- Version numbers gain a fourth hotfix component (2.0.0.1); the workflow maps it to the
  Windows file build number.
- The system panel shows dashes instead of 0% until real measurements arrive from the
  main process.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Wn5VX9HNbJ7N54hR24u9Y
2026-09-03 15:43:19 +00:00
LogiFlow 050ae20773 fix: modèle Hermes par défaut et repli sans outils client (v2.0.2) (#7)
fix: modèle Hermes par défaut et repli sans outils client (v2.0.2)
2026-09-03 17:40:44 +02:00
Claude 2a471e11a4 fix: default Hermes model from /v1/models and retry chat completions without client tools on HTTP 400 (v2.0.2)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Wn5VX9HNbJ7N54hR24u9Y
2026-09-03 15:40:21 +00:00
LogiFlow b5357c5abd fix: charger le pont Electron dans l'application installée (v2.0.1) (#6)
fix: charger le pont Electron dans l'application installée (v2.0.1)
2026-09-03 17:38:58 +02:00
Claude c73114eb0d fix: bundle the Electron preload so the sandboxed bridge loads in packaged builds (v2.0.1)
The preload script imported ../shared/ipc at runtime; sandboxed preloads cannot
require local modules, so window.eveflow was never exposed in the installed app
and every request fell back to browser fetch (Failed to fetch, LOCALHOST metrics).
Main and preload are now bundled with esbuild into single files in dist-electron/.

Also: explicit error banner when the bridge is missing inside Electron, and a more
precise connection test message (transport path, capabilities status).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Wn5VX9HNbJ7N54hR24u9Y
2026-09-03 15:38:36 +00:00
84 changed files with 5805 additions and 465 deletions

No files matched your search

+5 -2
View File
@@ -43,8 +43,10 @@ jobs:
id: ver id: ver
shell: bash shell: bash
run: | run: |
version=$(node -p "require('./package.json').version") version=$(node -p "const p=require('./package.json'); p.releaseVersion || p.version")
echo "version=$version" >> "$GITHUB_OUTPUT" echo "version=$version" >> "$GITHUB_OUTPUT"
# Fourth component (hotfix number, e.g. 2.0.0.1) becomes the Windows file build number.
echo "build=$(echo "$version" | cut -d. -f4)" >> "$GITHUB_OUTPUT"
if git ls-remote --exit-code --tags origin "refs/tags/v$version" >/dev/null 2>&1; then if git ls-remote --exit-code --tags origin "refs/tags/v$version" >/dev/null 2>&1; then
echo "released=true" >> "$GITHUB_OUTPUT" echo "released=true" >> "$GITHUB_OUTPUT"
else else
@@ -52,9 +54,10 @@ jobs:
fi fi
- run: npm run build - run: npm run build
- name: Package Windows EXE - name: Package Windows EXE
run: npx electron-builder --win --publish never run: npm run dist -- --win --publish never
env: env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
BUILD_NUMBER: ${{ steps.ver.outputs.build }}
- uses: actions/upload-artifact@v4 - uses: actions/upload-artifact@v4
with: with:
name: EveFlow-Windows-v${{ steps.ver.outputs.version }} name: EveFlow-Windows-v${{ steps.ver.outputs.version }}
+38 -5
View File
@@ -1,7 +1,7 @@
# EveFlow 2 — Interface vocale JARVIS pour Hermes Agent # EveFlow 2 — Interface vocale JARVIS pour Hermes Agent
[![Build](https://img.shields.io/github/actions/workflow/status/R0m1k3/EveFlow/windows-release.yml?style=flat-square)](https://github.com/R0m1k3/EveFlow/actions) [![Build](https://img.shields.io/github/actions/workflow/status/R0m1k3/EveFlow/windows-release.yml?style=flat-square)](https://github.com/R0m1k3/EveFlow/actions)
[![Version](https://img.shields.io/badge/version-2.0.0-brightgreen.svg?style=flat-square)](https://github.com/R0m1k3/EveFlow/releases) [![Version](https://img.shields.io/badge/version-2.4.1-brightgreen.svg?style=flat-square)](https://github.com/R0m1k3/EveFlow/releases)
[![License](https://img.shields.io/badge/license-MIT-lightgrey.svg?style=flat-square)](LICENSE) [![License](https://img.shields.io/badge/license-MIT-lightgrey.svg?style=flat-square)](LICENSE)
**EveFlow** est un compagnon de bureau Windows qui transforme [Hermes Agent](https://hermes-agent.nousresearch.com/) en assistant vocal à la JARVIS : un noyau holographique réactif au son, une conversation en streaming, les outils, sous-agents, approbations, crons, skills et sessions d'Hermes pilotés depuis un seul HUD. **EveFlow** est un compagnon de bureau Windows qui transforme [Hermes Agent](https://hermes-agent.nousresearch.com/) en assistant vocal à la JARVIS : un noyau holographique réactif au son, une conversation en streaming, les outils, sous-agents, approbations, crons, skills et sessions d'Hermes pilotés depuis un seul HUD.
@@ -22,8 +22,21 @@ La version 2 est une réécriture complète : plus de robot 3D, un pipeline voca
* **Capture micro** via AudioWorklet à 16 kHz, sans monitoring du micro dans les haut-parleurs, avec annulation d'écho et réduction de bruit. * **Capture micro** via AudioWorklet à 16 kHz, sans monitoring du micro dans les haut-parleurs, avec annulation d'écho et réduction de bruit.
* **Détection d'activité vocale** (seuil adaptatif, sensibilité et silence de fin réglables) : l'enregistrement s'arrête tout seul quand vous avez fini de parler. * **Détection d'activité vocale** (seuil adaptatif, sensibilité et silence de fin réglables) : l'enregistrement s'arrête tout seul quand vous avez fini de parler.
* **Mains libres** : le micro se réactive après chaque réponse. * **Mains libres** : le micro se réactive après chaque réponse.
* **STT** : n'importe quelle API `/v1/audio/transcriptions` compatible OpenAI (Qwen3-ASR, Whisper, Speaches, faster-whisper-server, LocalAI, OpenAI). Repli sur la reconnaissance Chromium. * **Modèles intégrés, hors ligne** (sherpa-onnx dans un processus séparé) : reconnaissance Whisper (base, small, large-v3 turbo) ou SenseVoice, synthèse Kokoro v1.0 (voix française Siwis et voix anglaises) ou Piper (Siwis, Tom, UPMC). Les modèles se téléchargent depuis **Paramètres → Modèles locaux** et tournent sur le processeur.
* **TTS** : API `/v1/audio/speech` compatible OpenAI (Kokoro, Piper, OpenAI…), voix système Windows ou Google Translate. Lecture phrase par phrase pendant le streaming, préchargement du segment suivant, coupure instantanée. * **Fin de phrase neuronale** : en écoute permanente, Silero VAD (0,6 Mo, sherpa-onnx) décide du début et de la fin de la commande à la place du seuil d'énergie ; moins de faux départs sur le bruit, coupure plus nette. Repli automatique sur le VAD énergétique si le modèle n'est pas installé.
* **Vision d'écran** : « Jarvis, regarde mon écran » (ou le bouton de la barre de commande) joint une capture de l'écran principal à la question envoyée à Hermes.
* **Actions locales instantanées** : « verrouille la session », « monte le son », « coupe le son », « piste suivante », « ouvre Spotify », « ouvre github.com »… exécutées sur le PC sans passer par Hermes, résultat lu à voix haute. Liste blanche d'actions dans le processus principal, désactivable dans les paramètres.
* **Serveur MCP intégré** : Hermes se connecte à `http://<pc>:7842/mcp` et obtient les outils du PC (capture d'écran renvoyée en image, verrouillage, applications, URL, touches média, presse-papiers, recherche de fichiers, voix, notifications, état du HUD, affichage dans le fil). Même port et même secret que le webhook ; en mode chat completions, les mêmes outils sont proposés directement au modèle.
* **Heures calmes et priorités** : plage horaire pendant laquelle les messages poussés s'affichent sans être lus ni faire clignoter le noyau (badge « non lus » à la place), thème nuit automatique, mots prioritaires lus quand même, résumé vocal des rapports longs (les premières phrases seulement).
* **Mode mission** : un bouton dans la barre de commande bascule sur un second modèle Hermes (plus puissant) pour les tâches longues ; le modèle rapide reste utilisé pour la conversation courante.
* **Widget compact « glanceable »** : état (veille, écoute, réflexion, parle), dernière phrase de l'assistant, badge de non-lus, indicateurs heures calmes et mission.
* **Voix JARVIS** : préréglage en un clic (Paramètres → Voix) : voix française masculine locale (Piper Tom, téléchargée automatiquement), timbre « JARVIS » (légèrement plus grave et posé, chaleur, présence, courte réverbération d'intercom), débit calme. Kokoro n'a pas de voix française masculine.
* **Reconnaissance française de référence** : Parakeet TDT 0.6B v3 (NVIDIA NeMo, 25 langues européennes) dans le catalogue, plus précis et bien plus rapide que Whisper sur processeur, avec ponctuation. Whisper base/small/turbo restent disponibles.
* **Voix masculine ou féminine** : un réglage unique (Paramètres → Voix) appliqué à tous les moteurs. En local, Piper Tom ou Pierre (UPMC) pour le masculin, téléchargé automatiquement si aucune voix masculine n'est installée ; onyx / nova pour les API compatibles OpenAI ; Paul / Hortense pour les voix Windows.
* **Barge-in** : en mains libres, parler par-dessus l'assistant coupe sa voix ; le seuil est relevé pendant qu'il parle pour ignorer l'écho du haut-parleur.
* **Écoute permanente** : un détecteur de mot-clé de 3 Mo (sherpa-onnx, keyword spotting) tourne en continu sur le micro, quasi gratuit en CPU. « Jarvis » (ou n'importe quel mot-clé) ouvre l'écoute, « Jarvis, allume… » envoie directement la commande, et le mot coupe la voix en cours. Alternative : filtre du mot après transcription en mains libres.
* **STT externe** : n'importe quelle API `/v1/audio/transcriptions` compatible OpenAI (Qwen3-ASR, Whisper, Speaches, faster-whisper-server, LocalAI, OpenAI). Repli sur la reconnaissance Chromium.
* **TTS externe** : API `/v1/audio/speech` compatible OpenAI, voix système Windows ou Google Translate. Lecture phrase par phrase pendant le streaming, préchargement du segment suivant, coupure instantanée.
* Raccourcis globaux : `Ctrl+Shift+Espace` (micro), `Ctrl+Shift+J` (afficher/masquer), `Ctrl+Shift+Échap` (couper la voix). * Raccourcis globaux : `Ctrl+Shift+Espace` (micro), `Ctrl+Shift+J` (afficher/masquer), `Ctrl+Shift+Échap` (couper la voix).
### Hermes, toute la puissance ### Hermes, toute la puissance
@@ -49,10 +62,12 @@ La version 2 est une réécriture complète : plus de robot 3D, un pipeline voca
* Electron 44 (sandbox, contextIsolation, `webSecurity` actif, proxy HTTP en streaming dans le main process) * Electron 44 (sandbox, contextIsolation, `webSecurity` actif, proxy HTTP en streaming dans le main process)
* React 19 + Vite 8 + TypeScript 5.9 + Zustand * React 19 + Vite 8 + TypeScript 5.9 + Zustand
* Canvas 2D, Web Audio (AudioWorklet, AnalyserNode) * Canvas 2D, Web Audio (AudioWorklet, AnalyserNode)
* sherpa-onnx (ONNX Runtime) pour la voix locale, exécuté dans un `utilityProcess` Electron
* Vitest pour les tests unitaires (SSE, VAD, WAV, normalisation d'événements Hermes) * Vitest pour les tests unitaires (SSE, VAD, WAV, normalisation d'événements Hermes)
``` ```
electron/ processus principal (fenêtre, tray, raccourcis, IPC, webhook, proxy HTTP) electron/ processus principal (fenêtre, tray, raccourcis, IPC, webhook, proxy HTTP)
electron/voice catalogue de modèles, téléchargement, worker sherpa-onnx (STT/TTS)
shared/ contrat IPC + normalisation des pushs webhook (main + renderer) shared/ contrat IPC + normalisation des pushs webhook (main + renderer)
src/lib transport, SSE, persistance, utilitaires texte src/lib transport, SSE, persistance, utilitaires texte
src/services hermes/ (client, événements, outils locaux) voice/ (capture, VAD, STT, TTS) src/services hermes/ (client, événements, outils locaux) voice/ (capture, VAD, STT, TTS)
@@ -85,6 +100,24 @@ Pour recevoir les résultats de crons ou le miroir d'autres canaux dans EveFlow,
--- ---
### Donner à Hermes les outils du PC (MCP)
Dans `~/.hermes/config.yaml` côté Hermes :
```yaml
mcp_servers:
eveflow:
url: "http://<ip-du-pc>:7842/mcp"
headers:
Authorization: "Bearer <secret du webhook EveFlow>"
```
Sans secret, EveFlow n'écoute qu'en local (`127.0.0.1`) ; définissez un secret dans Paramètres → Webhook pour un Hermes distant. Outils exposés : `capture_screen`, `lock_session`, `open_app`, `open_url`, `media_key`, `clipboard_get`, `clipboard_set`, `find_files`, `speak_text`, `notify_user`, `set_hud_state`, `get_app_status`, `get_conversation_history`, `show_message`.
### L'URL répond par une page web ?
Si « Tester la liaison » signale « Le serveur renvoie une page web … au lieu de l'API Hermes », l'URL saisie pointe vers un portail (page de connexion, tableau de bord) et non vers le serveur API. EveFlow cherche alors automatiquement l'API sur le même hôte (port 8642, chemins `/api` et `/v1`, sous-domaines `api.` ou `hermes.`) et corrige l'URL s'il la trouve. Sinon, ouvrez le port 8642 du serveur Hermes ou exposez-le sur un chemin dédié de votre proxy, sans authentification web devant lui (la clé `API_SERVER_KEY` suffit).
## Développement ## Développement
```bash ```bash
@@ -96,13 +129,13 @@ npm run typecheck # renderer + main process
Sans Electron, `npm run dev` puis `http://127.0.0.1:5173/` (ou `?mode=compact`) permet de travailler l'interface dans un navigateur ; les appels réseau passent alors directement par `fetch` (CORS requis côté serveur). Sans Electron, `npm run dev` puis `http://127.0.0.1:5173/` (ou `?mode=compact`) permet de travailler l'interface dans un navigateur ; les appels réseau passent alors directement par `fetch` (CORS requis côté serveur).
## Packaging Windows ## Packaging Windows et versions
```bash ```bash
npm run dist npm run dist
``` ```
L'installateur NSIS est produit dans `out/`. Le workflow GitHub Actions construit l'exécutable sur chaque tag `v*`. L'installateur NSIS est produit dans `out/`. Le numéro de version publié est `releaseVersion` dans `package.json` (quatre chiffres, ex. `2.0.0.1` : le quatrième chiffre sert aux petits correctifs). `version` reste la base à trois chiffres exigée par electron-builder. À chaque fusion sur master, le workflow GitHub Actions crée le tag et la release Windows si ce numéro n'a pas encore été publié.
--- ---
+88
View File
@@ -0,0 +1,88 @@
# Feuille de route : vers un vrai JARVIS
État au 3 septembre 2026, après la revue complète du code (trois audits : processus principal, services/état, interface) et le test de bout en bout de l'application Electron sous Xvfb avec les vrais modèles.
## Ce qui existe déjà
| Capacité | État | Notes |
|---|---|---|
| HUD arc-reactor réactif au son | Fait | Canvas 2D optimisé (pas d'ombres, couleurs en cache, 30 fps en veille, arrêt fenêtre masquée) |
| Reconnaissance vocale locale | Fait | Whisper base/small/turbo via sherpa-onnx dans un processus utilitaire |
| Synthèse vocale locale | Fait | Kokoro v1.0 (voix française Siwis) et Piper fr |
| Mot d'activation permanent | Fait (2.2.0) | Keyword spotting sherpa-onnx en continu ; mot-clé libre encodé en BPE ; validé sur audio réel (détection, zéro faux positif sur le test anglais) |
| Mot d'activation après transcription | Fait | Filtre « Jarvis … » en mains libres, tolérant aux erreurs de transcription |
| Détection de fin de phrase | Fait (2.3.0) | Silero VAD neuronal dans le worker (segment renvoyé au renderer), repli sur le VAD énergétique si le modèle manque |
| Vision d'écran | Fait (2.3.0) | Capture `desktopCapturer` jointe à la requête Hermes (bouton, ou « regarde mon écran ») |
| Serveur MCP local (outils du PC pour Hermes) | Fait (2.4.0) | `/mcp` sur le serveur webhook, JSON-RPC Streamable HTTP, 14 outils dont la capture d'écran renvoyée en image |
| Heures calmes, priorités, résumé vocal | Fait (2.4.0) | Paramètres → Notifications ; thème nuit automatique ; badge non-lus |
| Mode mission (second modèle) | Fait (2.4.0) | Bouton dans la barre de commande ; modèle dédié aux tâches longues |
| Widget compact glanceable | Fait (2.4.0) | État, dernière phrase, non-lus, indicateurs |
| Barge-in | Fait (2.4.0) | Coupe la voix dès que l'utilisateur parle ; seuil relevé pendant la synthèse (à valider avec l'annulation d'écho Windows) |
| Actions système locales | Fait (2.3.0) | Verrouillage, volume et touches média, ouvrir une application ou une URL, presse-papiers, recherche de fichiers ; intentions courtes exécutées sans passer par Hermes |
| Hermes : runs, sessions, chat completions | Fait | Transport choisi selon `/v1/capabilities` |
| Approbations, steer, stop | Fait | Modales, injection de consigne en cours de run |
| Crons, skills, toolsets, sessions | Fait | Panneau Hermes Ops |
| Webhook local (Telegram, crons) | Fait | Loopback par défaut, secret pour le LAN |
| Télémétrie réelle | Fait | CPU, RAM, fréquence, FPS, uptime |
## Ce que les meilleurs projets « Jarvis » de 2026 font en plus
Sources : [jarvis-desktop-ai](https://github.com/ccarloshenri/jarvis-desktop-ai), [JarvisAi](https://github.com/PanPenek/JarvisAi), [bertrandmbanwi/Jarvis](https://github.com/bertrandmbanwi/Jarvis), [InterGenJLU/jarvis](https://github.com/InterGenJLU/jarvis), [livekit-wakeword](https://livekit.com/blog/livekit-wakeword), [sherpa-onnx keyword spotting](https://k2-fsa.github.io/sherpa/onnx/kws/index.html), [Hermes Agent features](https://hermes-agent.nousresearch.com/docs/user-guide/features/overview).
1. **Mot d'activation permanent, quasi gratuit en CPU.** Les projets de référence utilisent openWakeWord (« hey jarvis ») ou un modèle de keyword spotting qui écoute en continu, au lieu de transcrire chaque phrase. sherpa-onnx fournit un modèle KWS anglais de 3,3 Mo qui accepte n'importe quel mot-clé sans réentraînement ; « JARVIS » s'encode `▁JA R VI S` avec son modèle BPE (vérifié). Livré en 2.2.0 (voir l'étape 1 ci-dessous).
2. **VAD neuronal (Silero) au lieu du seuil d'énergie.** Fin de phrase plus nette (environ 500 ms gagnés) et beaucoup moins de faux départs sur le bruit ambiant. Silero est déjà livré dans sherpa-onnx (`silero_vad.onnx`, 0,6 Mo). Livré en 2.3.0.
3. **Latence perçue sous la seconde.** Les références visent 1 s entre la fin de parole et le premier mot prononcé : STT rapide, premier token en streaming, TTS phrase par phrase (déjà en place), et un modèle Hermes rapide pour la conversation courante.
4. **Vision d'écran.** Capture d'écran à la demande (« Jarvis, qu'est-ce que je regarde ? ») envoyée à Hermes comme image, ou lecture d'une fenêtre. Hermes accepte déjà les images inline. Livré en 2.3.0 (transport chat completions pour les images).
5. **Actions système locales.** Ouvrir une application, régler le volume, verrouiller la session, chercher un fichier. Livré en 2.3.0 sous forme d'intentions courtes exécutées localement avant Hermes ; reste à exposer les mêmes actions à Hermes via un serveur MCP local.
6. **Proactivité.** Notifications parlées à l'arrivée d'un cron, rappel, événement webhook, avec un résumé plutôt que la lecture intégrale ; c'est en partie fait via le webhook, à enrichir avec des règles (heures calmes, priorité).
7. **Mémoire et personnalisation.** Hermes gère la mémoire longue durée (`X-Hermes-Session-Key`) ; côté EveFlow, un profil (nom, préférences de voix, style de réponse) déjà transmis dans les instructions.
## Plan proposé
### Étape 1 : écoute permanente — livrée en 2.2.0
- Le renderer garde un seul flux micro (AudioWorklet 16 kHz) et envoie des blocs de 256 ms au processus principal, qui alimente le `KeywordSpotter` sherpa-onnx dans le worker.
- Mots-clés encodés en BPE (table SentencePiece pour les mots courants, repli glouton sur le vocabulaire du modèle), sensibilité réglable (seuil 0,45 → 0,12).
- À la détection : chime, capture de la commande sur le même flux (VAD énergétique, pré-roll 400 ms), transcription locale ou API, envoi à Hermes ; retour automatique à l'écoute.
- Fin de phrase Silero livrée en 2.3.0 : le renderer envoie des trames de 128 ms au worker pendant la commande, le worker renvoie le segment WAV complet ; VAD énergétique en repli.
### Étape 2 : vision et actions locales — livrée en 2.3.0
- Capture d'écran (`desktopCapturer`, JPEG 1600 px) jointe à la requête Hermes : bouton dans la barre de commande, ou phrase « regarde mon écran… » à l'oral comme à l'écrit.
- Actions locales (processus principal, liste blanche) : verrouiller, volume/mute/lecture/piste, ouvrir une application connue ou une URL http(s), presse-papiers, recherche de fichiers dans Documents/Bureau/Téléchargements/Images.
- Routeur d'intentions FR/EN (`src/services/localCommands.ts`) exécuté avant l'envoi à Hermes ; désactivable dans Paramètres → Micro. Les phrases composées (« ouvre X puis… ») partent à Hermes.
- Exposition à Hermes via le serveur MCP local livrée en 2.4.0.
### Étape 3 : conversation plus naturelle — livrée en 2.4.0
- Barge-in : l'utilisateur qui parle par-dessus coupe la voix ; seuil d'énergie relevé pendant la synthèse pour ignorer l'écho (à valider sur Windows avec l'annulation d'écho du micro).
- Mode mission : second modèle Hermes choisi d'un clic pour les tâches longues.
- Serveur MCP local : Hermes enchaîne lui-même les actions du PC dans ses runs.
### Étape 4 : présence — livrée en 2.4.0
- Widget compact glanceable : état, dernière phrase, non-lus, indicateurs heures calmes / mission.
- Heures calmes, mots prioritaires, résumé vocal des messages entrants, thème nuit.
### Pistes suivantes
- Voix différente par contexte (nuit, travail) et profils de réponse.
- Mémoire locale des préférences transmise à Hermes (`X-Hermes-Session-Key` déjà en place).
- Validation du barge-in et des touches média sur Windows réel (retours utilisateurs).
## Résultats du test de bout en bout (Linux, Xvfb, 4 cœurs lents)
| Étape | Résultat |
|---|---|
| Pont Electron, télémétrie, webhook | OK |
| Conversation via runs (streaming, outils, sous-agent) | OK |
| Kokoro (fr) → Whisper base, phrase courte de 2 s | synthèse 2,9 s, transcription 2,5 s, texte approximatif |
| Kokoro (fr) → Whisper base, phrase de 7 s | transcription correcte à un mot près |
| Piper (fr) → Whisper base | transcription exacte |
| Silero VAD sur phrase Kokoro de 2,3 s (2.3.0) | un seul segment, début et fin détectés, 6,7 s d'audio traités en 190 ms |
| Capture d'écran → Hermes (2.3.0) | JPEG de 107 ko reçu côté Hermes (mock chat completions) |
| Intention locale « coupe le son » (2.3.0) | traitée sans Hermes, résultat affiché dans le fil |
| Voix JARVIS et Parakeet (2.4.1) | timbre JARVIS (Web Audio : pitch, EQ, compression, réverbération courte), préréglage voix masculine française, Parakeet TDT v3 pour le français, worklets audio livrés en fichiers statiques (l'écoute permanente échouait sous la CSP stricte en version installée), rembourrage de silence avant la reconnaissance |
| Correctif 2.4.0.3 | un portail qui laisse passer /health mais renvoie une page web sur /v1 est traité comme une liaison en échec et déclenche la recherche de l'API (sondes /v1/capabilities, /v1/models) |
| Correctif 2.4.0.2 | une page web (portail de connexion) reçue à la place de l'API Hermes est reconnue et expliquée ; l'API est recherchée automatiquement sur le même hôte (port 8642, /api, hermes.…) et l'URL corrigée |
| Correctifs 2.4.0.1 | réponse vide en chat completions désormais expliquée (JSON non streamé ou erreur HTTP 200), la liaison ne passe plus en « dégradé » quand seule l'API des crons échoue, transcriptions parasites (« (cliquant) », « *Claire* ») ignorées, préférence de voix masculine/féminine |
| Parakeet v3 vs Whisper base sur trois phrases Piper (fr) (2.4.1) | Parakeet : 3/3 exactes avec ponctuation, 0,4 à 0,5 s à chaud (6,7 s au premier appel) ; Whisper base : erreurs sur « Jarvis », « Peux-tu », 0,7 à 0,9 s |
| Worklets audio sous CSP stricte (2.4.1) | chargement des modules statiques OK dans l'application empaquetée |
| Serveur MCP (2.4.0) | initialize, tools/list (14 outils), tools/call côté principal (presse-papiers, capture image) et côté renderer (état, message dans le fil) |
Sur un PC à 28 cœurs les temps sont nettement plus courts. Whisper small est maintenant recommandé pour le français.
+70 -14
View File
@@ -22,12 +22,25 @@ const READABLE_EXT: Record<string, string> = {
'.html': 'text/html' '.html': 'text/html'
}; };
const FORBIDDEN_EXT = new Set([ /** Documents and media the assistant may open with the default application. */
'.exe', '.bat', '.cmd', '.ps1', '.vbs', '.msi', '.scr', '.pif', '.reg', '.sh', const OPENABLE_EXT = new Set([
'.com', '.hta', '.vbe', '.wsf', '.lnk', '.jar', '.dll', '.js', '.jse' ...Object.keys(READABLE_EXT),
'.pdf', '.docx', '.doc', '.xlsx', '.xls', '.pptx', '.ppt', '.odt', '.ods', '.odp', '.rtf',
'.mp3', '.wav', '.ogg', '.flac', '.mp4', '.mkv', '.webm', '.zip', '.7z', '.py', '.ts', '.tsx', '.yaml', '.yml', '.toml', '.ini'
]); ]);
const FORBIDDEN_EXT = new Set([
'.exe', '.bat', '.cmd', '.ps1', '.vbs', '.msi', '.scr', '.pif', '.reg', '.sh', '.com', '.hta', '.vbe', '.wsf', '.lnk',
'.jar', '.dll', '.js', '.jse', '.pyw', '.msc', '.cpl', '.inf', '.url', '.scf', '.website', '.appref-ms', '.application',
'.gadget', '.psm1', '.psd1', '.ps1xml', '.wsh', '.wsc', '.sct', '.desktop', '.app', '.command', '.appimage', '.run'
]);
const WIN_RESERVED = /^(CON|PRN|AUX|NUL|COM[1-9]|LPT[1-9])(\..*)?$/i;
let sharedDir: string | null = null;
export function getSharedDirectory(): string { export function getSharedDirectory(): string {
if (sharedDir && fs.existsSync(sharedDir)) return sharedDir;
const candidates = [ const candidates = [
path.join(os.homedir(), 'Documents', 'EveFlow_Shared'), path.join(os.homedir(), 'Documents', 'EveFlow_Shared'),
path.join(app.getPath('userData'), 'shared') path.join(app.getPath('userData'), 'shared')
@@ -35,32 +48,60 @@ export function getSharedDirectory(): string {
for (const dir of candidates) { for (const dir of candidates) {
try { try {
fs.mkdirSync(dir, { recursive: true }); fs.mkdirSync(dir, { recursive: true });
sharedDir = dir;
return dir; return dir;
} catch { } catch {
/* try next candidate */ /* try next candidate */
} }
} }
return app.getPath('temp'); sharedDir = app.getPath('temp');
return sharedDir;
} }
function normalizeSafe(filePath: string): string { function normalizeSafe(filePath: string): string {
if (!filePath || typeof filePath !== 'string') throw new Error('Chemin invalide'); if (!filePath || typeof filePath !== 'string' || filePath.length > 4096) throw new Error('Chemin invalide');
const normalized = path.normalize(filePath); const normalized = path.normalize(filePath);
if (normalized.split(/[\\/]/).includes('..')) throw new Error('Traversee de repertoire refusee'); if (normalized.split(/[\\/]/).includes('..')) throw new Error('Traversee de repertoire refusee');
return normalized; return normalized;
} }
function allowedRoots(): string[] {
return [
getSharedDirectory(),
path.join(os.homedir(), 'Documents'),
path.join(os.homedir(), 'Pictures'),
path.join(os.homedir(), 'Downloads'),
path.join(os.homedir(), 'Desktop'),
app.getPath('temp')
].map((r) => path.resolve(r));
}
function isInside(root: string, target: string): boolean {
const rel = path.relative(root, target);
return rel === '' || (!rel.startsWith('..') && !path.isAbsolute(rel));
}
/** Resolves symlinks and checks the real path against the allowed roots (prefix tricks like "bob-backup" are rejected). */
function isInsideAllowedRoots(target: string): boolean { function isInsideAllowedRoots(target: string): boolean {
const roots = [getSharedDirectory(), os.homedir(), app.getPath('temp')]; let resolved: string;
const resolved = path.resolve(target); try {
return roots.some((root) => resolved.startsWith(path.resolve(root))); resolved = fs.realpathSync.native(target);
} catch {
return false;
}
return allowedRoots().some((root) => isInside(root, resolved));
} }
function stripFileScheme(value: string): string { function stripFileScheme(value: string): string {
let out = value.replace(/^file:\/\//, ''); if (!/^file:\/\//i.test(value)) return value;
let out = value.replace(/^file:\/\//i, '');
// file:///C:/x -> /C:/x on Windows-style paths // file:///C:/x -> /C:/x on Windows-style paths
if (/^\/[a-zA-Z]:/.test(out)) out = out.slice(1); if (/^\/[a-zA-Z]:/.test(out)) out = out.slice(1);
return decodeURIComponent(out); try {
return decodeURIComponent(out);
} catch {
return out;
}
} }
export function registerFilesIpc(): void { export function registerFilesIpc(): void {
@@ -78,8 +119,16 @@ export function registerFilesIpc(): void {
ipcMain.handle(IPC.writeSharedFile, async (_e, filename: string, content: string, isBase64 = false) => { ipcMain.handle(IPC.writeSharedFile, async (_e, filename: string, content: string, isBase64 = false) => {
if (!filename || typeof filename !== 'string') throw new Error('Nom de fichier invalide'); if (!filename || typeof filename !== 'string') throw new Error('Nom de fichier invalide');
if (typeof content !== 'string' || content.length > 50 * 1024 * 1024) throw new Error('Contenu invalide');
const safeName = path.basename(filename); const safeName = path.basename(filename);
if (safeName !== filename || safeName.startsWith('.') || /[<>:"|?*]/.test(safeName)) { if (
safeName !== filename ||
safeName.startsWith('.') ||
/[<>:"|?*\u0000-\u001f]/.test(safeName) ||
WIN_RESERVED.test(safeName) ||
/[. ]$/.test(safeName) ||
safeName.length > 200
) {
throw new Error('Nom de fichier non securise'); throw new Error('Nom de fichier non securise');
} }
const ext = path.extname(safeName).toLowerCase(); const ext = path.extname(safeName).toLowerCase();
@@ -97,10 +146,17 @@ export function registerFilesIpc(): void {
ipcMain.handle(IPC.openPath, async (_e, filePath: string) => { ipcMain.handle(IPC.openPath, async (_e, filePath: string) => {
const target = normalizeSafe(stripFileScheme(filePath)); const target = normalizeSafe(stripFileScheme(filePath));
const ext = path.extname(target).toLowerCase();
if (FORBIDDEN_EXT.has(ext)) throw new Error(`Ouverture refusee pour ${ext}`);
if (!fs.existsSync(target)) throw new Error('Fichier introuvable'); if (!fs.existsSync(target)) throw new Error('Fichier introuvable');
const result = await shell.openPath(target); const real = await fs.promises.realpath(target);
const stat = await fs.promises.stat(real);
const ext = path.extname(real).toLowerCase();
if (!stat.isDirectory() && (FORBIDDEN_EXT.has(ext) || !OPENABLE_EXT.has(ext))) throw new Error(`Ouverture refusee pour ${ext || 'ce fichier'}`);
// Outside the user's document folders we only reveal the file, never execute its handler.
if (!isInsideAllowedRoots(real)) {
shell.showItemInFolder(real);
return true;
}
const result = await shell.openPath(real);
if (result) throw new Error(result); if (result) throw new Error(result);
return true; return true;
}); });
+11 -2
View File
@@ -105,6 +105,8 @@ async function startStream(sender: WebContents, id: string, req: HttpProxyReques
headers: headersToObject(response.headers) headers: headersToObject(response.headers)
}; };
const onDestroyed = () => controller.abort();
sender.once('destroyed', onDestroyed);
const pump = async () => { const pump = async () => {
try { try {
if (!response.body) { if (!response.body) {
@@ -133,7 +135,10 @@ async function startStream(sender: WebContents, id: string, req: HttpProxyReques
if (e.name === 'AbortError') send({ id, type: 'end' }); if (e.name === 'AbortError') send({ id, type: 'end' });
else send({ id, type: 'error', message: e.message || String(err) }); else send({ id, type: 'error', message: e.message || String(err) });
} finally { } finally {
activeStreams.delete(id); // Releases the socket when we bailed out early (idle timeout, renderer gone).
controller.abort();
sender.removeListener('destroyed', onDestroyed);
if (activeStreams.get(id) === controller) activeStreams.delete(id);
} }
}; };
void pump(); void pump();
@@ -147,7 +152,11 @@ export function abortAllStreams(): void {
export function registerHttpIpc(): void { export function registerHttpIpc(): void {
ipcMain.handle(IPC.httpFetch, (_e, req: HttpProxyRequest) => proxyFetch(req)); ipcMain.handle(IPC.httpFetch, (_e, req: HttpProxyRequest) => proxyFetch(req));
ipcMain.handle(IPC.httpStreamStart, (event, id: string, req: HttpProxyRequest) => startStream(event.sender, id, req)); ipcMain.handle(IPC.httpStreamStart, (event, id: unknown, req: HttpProxyRequest) => {
if (typeof id !== 'string' || !id || id.length > 64 || activeStreams.has(id)) throw new Error('Identifiant de flux invalide');
if (!req || typeof req.url !== 'string') throw new Error('Requête invalide');
return startStream(event.sender, id, req);
});
ipcMain.on(IPC.httpStreamAbort, (_e, id: string) => { ipcMain.on(IPC.httpStreamAbort, (_e, id: string) => {
const controller = activeStreams.get(id); const controller = activeStreams.get(id);
if (controller) { if (controller) {
+61 -12
View File
@@ -2,10 +2,16 @@ import { app, ipcMain } from 'electron';
import fs from 'node:fs'; import fs from 'node:fs';
import path from 'node:path'; import path from 'node:path';
import { IPC } from '../../shared/ipc'; import { IPC } from '../../shared/ipc';
import { log } from '../logger';
type StoreData = Record<string, unknown>; type StoreData = Record<string, unknown>;
let cache: StoreData | null = null; let cache: StoreData | null = null;
let dirty = false;
let timer: ReturnType<typeof setTimeout> | null = null;
let writing: Promise<void> = Promise.resolve();
const FORBIDDEN_KEYS = new Set(['__proto__', 'constructor', 'prototype']);
function storePath(): string { function storePath(): string {
return path.join(app.getPath('userData'), 'userdata.json'); return path.join(app.getPath('userData'), 'userdata.json');
@@ -13,19 +19,55 @@ function storePath(): string {
function load(): StoreData { function load(): StoreData {
if (cache) return cache; if (cache) return cache;
const target = storePath();
try { try {
cache = JSON.parse(fs.readFileSync(storePath(), 'utf8')) as StoreData; cache = JSON.parse(fs.readFileSync(target, 'utf8')) as StoreData;
} catch { } catch (err) {
if ((err as NodeJS.ErrnoException).code !== 'ENOENT') {
// Keep a copy of an unreadable file instead of silently overwriting it.
try {
fs.copyFileSync(target, `${target}.corrupt-${Date.now()}`);
} catch {
/* ignore */
}
log('WARN', 'store', `userdata.json unreadable, starting fresh: ${(err as Error).message}`);
}
cache = {}; cache = {};
} }
return cache; return cache;
} }
function persist(): void { /** Debounced, asynchronous, atomic write with retries (Windows AV scanners lock files briefly). */
const target = storePath(); function schedulePersist(): void {
const tmp = target + '.tmp'; dirty = true;
fs.writeFileSync(tmp, JSON.stringify(cache ?? {}, null, 2), 'utf8'); if (!timer) timer = setTimeout(() => void flushStore(), 250);
fs.renameSync(tmp, target); }
export function flushStore(): Promise<void> {
if (timer) {
clearTimeout(timer);
timer = null;
}
if (!dirty) return writing;
dirty = false;
const text = JSON.stringify(cache ?? {}, null, 2);
writing = writing
.then(async () => {
const target = storePath();
const tmp = `${target}.tmp`;
await fs.promises.writeFile(tmp, text, 'utf8');
for (let attempt = 0; ; attempt++) {
try {
await fs.promises.rename(tmp, target);
return;
} catch (err) {
if (attempt >= 5) throw err;
await new Promise((r) => setTimeout(r, 100 * (attempt + 1)));
}
}
})
.catch((err) => log('ERROR', 'store', `cannot persist userdata.json: ${(err as Error).message}`));
return writing;
} }
export function storeGet<T = unknown>(key: string): T | null { export function storeGet<T = unknown>(key: string): T | null {
@@ -36,19 +78,26 @@ export function storeGet<T = unknown>(key: string): T | null {
export function storeSet(key: string, value: unknown): void { export function storeSet(key: string, value: unknown): void {
const data = load(); const data = load();
data[key] = value; data[key] = value;
persist(); schedulePersist();
}
function validKey(key: unknown): key is string {
return typeof key === 'string' && key.length > 0 && key.length <= 128 && !FORBIDDEN_KEYS.has(key);
} }
export function registerStoreIpc(): void { export function registerStoreIpc(): void {
ipcMain.handle(IPC.storeGet, (_e, key: string) => storeGet(key)); ipcMain.handle(IPC.storeGet, (_e, key: unknown) => (validKey(key) ? storeGet(key) : null));
ipcMain.handle(IPC.storeSet, (_e, key: string, value: unknown) => { ipcMain.handle(IPC.storeSet, (_e, key: unknown, value: unknown) => {
if (!validKey(key)) throw new Error('Clé de stockage invalide');
if (typeof value === 'function' || typeof value === 'symbol') throw new Error('Valeur non sérialisable');
storeSet(key, value); storeSet(key, value);
return true; return true;
}); });
ipcMain.handle(IPC.storeDelete, (_e, key: string) => { ipcMain.handle(IPC.storeDelete, (_e, key: unknown) => {
if (!validKey(key)) return false;
const data = load(); const data = load();
delete data[key]; delete data[key];
persist(); schedulePersist();
return true; return true;
}); });
} }
+184
View File
@@ -0,0 +1,184 @@
/**
* Local "JARVIS" actions: screenshot for vision, and an allow-list of system actions
* (lock, open an application or URL, media keys, clipboard, file search).
*/
import { app, clipboard, desktopCapturer, ipcMain, screen, shell } from 'electron';
import { execFile } from 'node:child_process';
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import { IPC } from '../../shared/ipc';
import type { SystemAction, SystemActionResult } from '../../shared/bridge';
import { log } from '../logger';
const MEDIA_KEYS: Record<string, number> = {
'volume-up': 175,
'volume-down': 174,
mute: 173,
'play-pause': 179,
next: 176,
previous: 177
};
/** Applications the assistant may launch by name (Windows aliases + common Linux/macOS names). */
const APP_ALIASES: Record<string, string[]> = {
'bloc-notes': ['notepad.exe', 'gedit', 'TextEdit'],
notepad: ['notepad.exe', 'gedit', 'TextEdit'],
calculatrice: ['calc.exe', 'gnome-calculator', 'Calculator'],
calc: ['calc.exe', 'gnome-calculator', 'Calculator'],
explorateur: ['explorer.exe', 'nautilus', 'Finder'],
explorer: ['explorer.exe', 'nautilus', 'Finder'],
terminal: ['wt.exe', 'cmd.exe', 'gnome-terminal', 'Terminal'],
cmd: ['cmd.exe'],
powershell: ['powershell.exe'],
paint: ['mspaint.exe'],
chrome: ['chrome', 'google-chrome', 'Google Chrome'],
edge: ['msedge', 'microsoft-edge', 'Microsoft Edge'],
firefox: ['firefox', 'Firefox'],
vscode: ['code', 'Visual Studio Code'],
code: ['code', 'Visual Studio Code'],
spotify: ['spotify', 'Spotify'],
discord: ['discord', 'Discord'],
steam: ['steam', 'Steam'],
word: ['winword', 'Microsoft Word'],
excel: ['excel', 'Microsoft Excel'],
outlook: ['outlook', 'Microsoft Outlook'],
teams: ['ms-teams', 'Microsoft Teams'],
'task manager': ['taskmgr.exe'],
'gestionnaire des tâches': ['taskmgr.exe'],
paramètres: ['ms-settings:'],
settings: ['ms-settings:']
};
function run(cmd: string, args: string[], timeoutMs = 8000): Promise<{ code: number; out: string }> {
return new Promise((resolve) => {
execFile(cmd, args, { timeout: timeoutMs, windowsHide: true }, (err, stdout, stderr) => {
resolve({ code: err ? 1 : 0, out: `${stdout}${stderr}`.trim() });
});
});
}
async function openApp(name: string): Promise<SystemActionResult> {
const key = name.trim().toLowerCase();
if (!key || key.length > 40) return { ok: false, message: 'Nom d’application invalide' };
const candidates = APP_ALIASES[key] ?? [key.replace(/[^a-z0-9 ._-]/gi, '')];
if (process.platform === 'win32') {
for (const candidate of candidates) {
if (candidate.endsWith(':')) {
await shell.openExternal(candidate);
return { ok: true, message: `${name} ouvert` };
}
// `start` resolves App Paths, PATH and Start Menu names.
const result = await run('cmd.exe', ['/c', 'start', '', candidate]);
if (result.code === 0) return { ok: true, message: `${name} lancé` };
}
return { ok: false, message: `Impossible de lancer ${name}` };
}
for (const candidate of candidates) {
const result = process.platform === 'darwin' ? await run('open', ['-a', candidate]) : await run('sh', ['-c', `command -v ${JSON.stringify(candidate)} >/dev/null && (nohup ${JSON.stringify(candidate)} >/dev/null 2>&1 &)`]);
if (result.code === 0) return { ok: true, message: `${name} lancé` };
}
return { ok: false, message: `Application introuvable : ${name}` };
}
async function lockSession(): Promise<SystemActionResult> {
if (process.platform === 'win32') {
const r = await run('rundll32.exe', ['user32.dll,LockWorkStation']);
return { ok: r.code === 0, message: r.code === 0 ? 'Session verrouillée' : r.out };
}
if (process.platform === 'darwin') {
const r = await run('osascript', ['-e', 'tell application "System Events" to keystroke "q" using {command down, control down}']);
return { ok: r.code === 0, message: r.out };
}
const r = await run('sh', ['-c', 'loginctl lock-session || xdg-screensaver lock || gnome-screensaver-command -l']);
return { ok: r.code === 0, message: r.code === 0 ? 'Session verrouillée' : r.out };
}
async function mediaKey(key: string): Promise<SystemActionResult> {
const code = MEDIA_KEYS[key];
if (!code) return { ok: false, message: 'Touche inconnue' };
if (process.platform === 'win32') {
const script = `$s=Add-Type -MemberDefinition '[DllImport("user32.dll")] public static extern void keybd_event(byte b,byte s,uint f,UIntPtr e);' -Name K -Namespace W -PassThru; $s::keybd_event(${code},0,0,[UIntPtr]::Zero); $s::keybd_event(${code},0,2,[UIntPtr]::Zero)`;
const r = await run('powershell.exe', ['-NoProfile', '-NonInteractive', '-Command', script]);
return { ok: r.code === 0, message: r.code === 0 ? key : r.out };
}
const xdo: Record<string, string> = { 'volume-up': 'XF86AudioRaiseVolume', 'volume-down': 'XF86AudioLowerVolume', mute: 'XF86AudioMute', 'play-pause': 'XF86AudioPlay', next: 'XF86AudioNext', previous: 'XF86AudioPrev' };
const r = await run('sh', ['-c', `xdotool key ${xdo[key]}`]);
return { ok: r.code === 0, message: r.code === 0 ? key : 'xdotool indisponible' };
}
async function findFiles(query: string): Promise<SystemActionResult> {
const needle = query.trim().toLowerCase();
if (needle.length < 2) return { ok: false, message: 'Recherche trop courte' };
const roots = ['Documents', 'Desktop', 'Downloads', 'Pictures'].map((d) => path.join(os.homedir(), d));
const hits: string[] = [];
const deadline = Date.now() + 4000;
const walk = async (dir: string, depth: number) => {
if (depth > 4 || hits.length >= 25 || Date.now() > deadline) return;
let entries: fs.Dirent[] = [];
try {
entries = await fs.promises.readdir(dir, { withFileTypes: true });
} catch {
return;
}
for (const entry of entries) {
if (entry.name.startsWith('.') || entry.name === 'node_modules') continue;
const full = path.join(dir, entry.name);
if (entry.name.toLowerCase().includes(needle)) hits.push(full);
if (entry.isDirectory()) await walk(full, depth + 1);
if (hits.length >= 25) return;
}
};
for (const root of roots) await walk(root, 0);
return { ok: true, message: `${hits.length} résultat(s)`, data: hits };
}
export async function captureScreen(maxWidth = 1600): Promise<string> {
const display = screen.getPrimaryDisplay();
const scale = Math.min(1, maxWidth / display.size.width);
const sources = await desktopCapturer.getSources({
types: ['screen'],
thumbnailSize: { width: Math.round(display.size.width * scale), height: Math.round(display.size.height * scale) }
});
const primary = sources.find((s) => s.display_id === String(display.id)) ?? sources[0];
if (!primary) throw new Error('Aucun écran capturable');
return primary.thumbnail.toJPEG(82).toString('base64').replace(/^/, 'data:image/jpeg;base64,');
}
export async function runSystemAction(action: SystemAction): Promise<SystemActionResult> {
switch (action.type) {
case 'lock':
return lockSession();
case 'open-app':
return openApp(String(action.name ?? ''));
case 'open-url': {
const url = String(action.url ?? '');
if (!/^https?:\/\//i.test(url) || url.length > 2048) return { ok: false, message: 'URL refusée' };
await shell.openExternal(url);
return { ok: true, message: 'Ouvert dans le navigateur' };
}
case 'media':
return mediaKey(String(action.key));
case 'clipboard-read':
return { ok: true, data: (await clipboard.readText()).slice(0, 20_000) };
case 'clipboard-write':
await clipboard.writeText(String(action.text ?? '').slice(0, 100_000));
return { ok: true, message: 'Copié dans le presse-papiers' };
case 'find-files':
return findFiles(String(action.query ?? ''));
default:
return { ok: false, message: 'Action inconnue' };
}
}
export function registerSystemIpc(): void {
ipcMain.handle(IPC.screenCapture, (_e, maxWidth?: number) => captureScreen(Number(maxWidth) || 1600));
ipcMain.handle(IPC.systemAction, async (_e, action: SystemAction) => {
if (!action || typeof action.type !== 'string') throw new Error('Action invalide');
log('INFO', 'system', `action ${action.type}`);
const result = await runSystemAction(action);
if (!result.ok) log('WARN', 'system', `action ${action.type} failed: ${result.message}`);
return result;
});
void app;
}
+32 -8
View File
@@ -5,22 +5,46 @@ import type { LogEntry } from '../shared/ipc';
const MAX_LOG_BYTES = 5 * 1024 * 1024; const MAX_LOG_BYTES = 5 * 1024 * 1024;
let stream: fs.WriteStream | null = null;
let written = 0;
export function getLogPath(): string { export function getLogPath(): string {
return path.join(app.getPath('userData'), 'eveflow.log'); return path.join(app.getPath('userData'), 'eveflow.log');
} }
function output(): fs.WriteStream {
if (!stream) {
const target = getLogPath();
try {
written = fs.statSync(target).size;
} catch {
written = 0;
}
stream = fs.createWriteStream(target, { flags: 'a' });
stream.on('error', (err) => console.error('[logger] write failed:', err.message));
}
return stream;
}
function rotate(): void {
const target = getLogPath();
stream?.end();
stream = null;
try {
fs.renameSync(target, `${target}.old`);
} catch {
/* ignore */
}
written = 0;
}
export function writeLog(entry: LogEntry): void { export function writeLog(entry: LogEntry): void {
try { try {
const logPath = getLogPath();
const data = entry.data !== undefined ? ' | ' + safeJson(entry.data) : ''; const data = entry.data !== undefined ? ' | ' + safeJson(entry.data) : '';
const line = `[${entry.ts}] [${entry.level}] [${entry.tag}] ${entry.message}${data}\n`; const line = `[${entry.ts}] [${entry.level}] [${entry.tag}] ${entry.message}${data}\n`;
try { written += Buffer.byteLength(line);
const stat = fs.statSync(logPath); if (written > MAX_LOG_BYTES) rotate();
if (stat.size > MAX_LOG_BYTES) fs.renameSync(logPath, logPath + '.old'); output().write(line);
} catch {
/* first start: file does not exist yet */
}
fs.appendFileSync(logPath, line, 'utf8');
} catch (err) { } catch (err) {
console.error('[logger] cannot write log file:', (err as Error).message); console.error('[logger] cannot write log file:', (err as Error).message);
} }
+31 -12
View File
@@ -2,15 +2,20 @@ import { app, BrowserWindow, globalShortcut, ipcMain, Menu, nativeImage, session
import path from 'node:path'; import path from 'node:path';
import { IPC, type AppInfo, type HotkeyEvent, type LogEntry, type WindowMode } from '../shared/ipc'; import { IPC, type AppInfo, type HotkeyEvent, type LogEntry, type WindowMode } from '../shared/ipc';
import { getLogPath, log, writeLog } from './logger'; import { getLogPath, log, writeLog } from './logger';
import { registerStoreIpc, storeGet } from './ipc/store'; import { flushStore, registerStoreIpc, storeGet } from './ipc/store';
import { abortAllStreams, registerHttpIpc } from './ipc/http'; import { abortAllStreams, registerHttpIpc } from './ipc/http';
import { registerTelemetryIpc } from './ipc/telemetry'; import { registerTelemetryIpc } from './ipc/telemetry';
import { getSharedDirectory, registerFilesIpc } from './ipc/files'; import { getSharedDirectory, registerFilesIpc } from './ipc/files';
import { createMainWindow, getMainWindow, getWindowMode, setWindowMode, toggleWindowVisibility } from './window'; import { createMainWindow, getMainWindow, getWindowMode, setWindowMode, toggleWindowVisibility, windowEvents } from './window';
import { getWebhookStatus, startWebhookServer, stopWebhookServer } from './webhook'; import { getWebhookStatus, startWebhookServer, stopWebhookServer } from './webhook';
import { registerVoiceIpc } from './voice/ipc';
import { registerSystemIpc } from './ipc/system';
import { stopEngine } from './voice/engine';
// Audio playback must never be blocked behind a user gesture (TTS starts on incoming events). // Audio playback must never be blocked behind a user gesture (TTS starts on incoming events).
app.commandLine.appendSwitch('autoplay-policy', 'no-user-gesture-required'); app.commandLine.appendSwitch('autoplay-policy', 'no-user-gesture-required');
// Test/dev aid: isolate settings, logs and models in a custom directory.
if (process.env.EVEFLOW_USER_DATA) app.setPath('userData', process.env.EVEFLOW_USER_DATA);
let tray: Tray | null = null; let tray: Tray | null = null;
@@ -35,8 +40,9 @@ interface WebhookSettings {
} }
function readWebhookSettings(): WebhookSettings { function readWebhookSettings(): WebhookSettings {
const saved = storeGet<WebhookSettings>('webhook'); const saved = storeGet<WebhookSettings>('webhook') ?? {};
return { enabled: true, port: 7842, secret: '', ...(saved ?? {}) }; const port = Number(saved.port);
return { enabled: saved.enabled !== false, port: Number.isInteger(port) && port > 0 && port < 65536 ? port : 7842, secret: typeof saved.secret === 'string' ? saved.secret : '' };
} }
async function bootWebhook(): Promise<void> { async function bootWebhook(): Promise<void> {
@@ -45,7 +51,7 @@ async function bootWebhook(): Promise<void> {
await stopWebhookServer(); await stopWebhookServer();
return; return;
} }
await startWebhookServer(getMainWindow, { port: Number(settings.port) || 7842, secret: settings.secret || undefined }); await startWebhookServer(getMainWindow, { port: settings.port ?? 7842, secret: settings.secret || undefined });
} }
function sendHotkey(event: HotkeyEvent): void { function sendHotkey(event: HotkeyEvent): void {
@@ -61,7 +67,8 @@ function registerShortcuts(): void {
sendHotkey('ptt-toggle'); sendHotkey('ptt-toggle');
}], }],
['CommandOrControl+Shift+J', () => toggleWindowVisibility()], ['CommandOrControl+Shift+J', () => toggleWindowVisibility()],
['CommandOrControl+Shift+Escape', () => sendHotkey('stop-speaking')] // Ctrl+Shift+Escape is the Windows Task Manager shortcut; use a free combination.
['CommandOrControl+Alt+Escape', () => sendHotkey('stop-speaking')]
]; ];
for (const [accelerator, handler] of bindings) { for (const [accelerator, handler] of bindings) {
try { try {
@@ -76,7 +83,7 @@ function createTray(): void {
try { try {
const iconPath = app.isPackaged const iconPath = app.isPackaged
? path.join(process.resourcesPath, 'icon.png') ? path.join(process.resourcesPath, 'icon.png')
: path.join(__dirname, '..', '..', 'build', 'icon.png'); : path.join(__dirname, '..', 'build', 'icon.png');
let image = nativeImage.createFromPath(iconPath); let image = nativeImage.createFromPath(iconPath);
if (image.isEmpty()) image = nativeImage.createEmpty(); if (image.isEmpty()) image = nativeImage.createEmpty();
else image = image.resize({ width: 16, height: 16 }); else image = image.resize({ width: 16, height: 16 });
@@ -94,7 +101,7 @@ function createTray(): void {
}; };
rebuildMenu(); rebuildMenu();
tray.on('click', () => toggleWindowVisibility()); tray.on('click', () => toggleWindowVisibility());
ipcMain.on(IPC.windowSetMode, () => setTimeout(rebuildMenu, 0)); windowEvents.on('mode', rebuildMenu);
} catch (err) { } catch (err) {
log('WARN', 'tray', `tray unavailable: ${(err as Error).message}`); log('WARN', 'tray', `tray unavailable: ${(err as Error).message}`);
} }
@@ -125,7 +132,10 @@ function registerCoreIpc(): void {
} }
}); });
ipcMain.on(IPC.windowSetMode, (_e, mode: WindowMode) => setWindowMode(mode === 'compact' ? 'compact' : 'hud')); ipcMain.on(IPC.windowSetMode, (_e, mode: WindowMode) => setWindowMode(mode === 'compact' ? 'compact' : 'hud'));
ipcMain.on(IPC.log, (_e, entry: LogEntry) => writeLog(entry)); ipcMain.on(IPC.log, (_e, entry: LogEntry) => {
if (!entry || typeof entry.message !== 'string') return;
writeLog({ ...entry, message: entry.message.slice(0, 8000), tag: String(entry.tag).slice(0, 64) });
});
ipcMain.handle(IPC.appInfo, (): AppInfo => ({ ipcMain.handle(IPC.appInfo, (): AppInfo => ({
version: app.getVersion(), version: app.getVersion(),
platform: process.platform, platform: process.platform,
@@ -141,7 +151,7 @@ function registerCoreIpc(): void {
}); });
} }
app.whenReady().then(async () => { async function boot(): Promise<void> {
log('INFO', 'app', `EveFlow ${app.getVersion()} starting (electron ${process.versions.electron}, chrome ${process.versions.chrome})`); log('INFO', 'app', `EveFlow ${app.getVersion()} starting (electron ${process.versions.electron}, chrome ${process.versions.chrome})`);
configureSession(); configureSession();
registerStoreIpc(); registerStoreIpc();
@@ -149,6 +159,8 @@ app.whenReady().then(async () => {
registerTelemetryIpc(); registerTelemetryIpc();
registerFilesIpc(); registerFilesIpc();
registerCoreIpc(); registerCoreIpc();
registerVoiceIpc();
registerSystemIpc();
createMainWindow(); createMainWindow();
createTray(); createTray();
registerShortcuts(); registerShortcuts();
@@ -157,12 +169,19 @@ app.whenReady().then(async () => {
app.on('activate', () => { app.on('activate', () => {
if (BrowserWindow.getAllWindows().length === 0) createMainWindow(); if (BrowserWindow.getAllWindows().length === 0) createMainWindow();
}); });
}); }
app.on('will-quit', () => { if (app.hasSingleInstanceLock()) {
app.whenReady().then(boot).catch((err) => log('ERROR', 'app', `boot failed: ${(err as Error).message}`));
}
app.on('will-quit', (event) => {
globalShortcut.unregisterAll(); globalShortcut.unregisterAll();
abortAllStreams(); abortAllStreams();
stopEngine();
void stopWebhookServer(); void stopWebhookServer();
event.preventDefault();
flushStore().finally(() => app.exit(0));
}); });
app.on('window-all-closed', () => { app.on('window-all-closed', () => {
+191
View File
@@ -0,0 +1,191 @@
/**
* Minimal MCP server (Streamable HTTP, JSON-RPC 2.0) mounted on the webhook HTTP server at /mcp.
* Hermes Agent connects to it as a remote MCP server and gains the PC-side tools: screen capture,
* system actions, voice, notifications and HUD state. No SDK: initialize / tools/list / tools/call
* are the only methods a client needs, and every reply is a plain JSON body.
*/
import { ipcMain, type BrowserWindow } from 'electron';
import type { IncomingMessage, ServerResponse } from 'node:http';
import { IPC, type McpToolRequest, type McpToolResponse } from '../shared/ipc';
import type { SystemAction } from '../shared/bridge';
import { log } from './logger';
import { captureScreen, runSystemAction } from './ipc/system';
export const MCP_PATH = '/mcp';
const PROTOCOL = '2025-03-26';
const RENDERER_TIMEOUT_MS = 20_000;
interface ToolSpec {
name: string;
description: string;
inputSchema: Record<string, unknown>;
/** 'main' = executed here; 'renderer' = forwarded to the UI process. */
where: 'main' | 'renderer';
}
const obj = (properties: Record<string, unknown>, required: string[] = []) => ({ type: 'object', properties, required, additionalProperties: false });
export const MCP_TOOLS: ToolSpec[] = [
{ name: 'capture_screen', description: "Capture l'écran principal de l'utilisateur et renvoie l'image (JPEG). À utiliser quand l'utilisateur parle de ce qu'il voit ou demande de l'aide sur son écran.", inputSchema: obj({ max_width: { type: 'integer', description: 'Largeur maximale en pixels (défaut 1600)' } }), where: 'main' },
{ name: 'lock_session', description: "Verrouille la session Windows/Linux/macOS de l'utilisateur.", inputSchema: obj({}), where: 'main' },
{ name: 'open_app', description: "Lance une application sur le PC de l'utilisateur (bloc-notes, calculatrice, chrome, spotify, vscode, terminal, explorateur…).", inputSchema: obj({ name: { type: 'string' } }, ['name']), where: 'main' },
{ name: 'open_url', description: "Ouvre une URL http(s) dans le navigateur par défaut de l'utilisateur.", inputSchema: obj({ url: { type: 'string' } }, ['url']), where: 'main' },
{ name: 'media_key', description: 'Envoie une touche média : volume-up, volume-down, mute, play-pause, next, previous.', inputSchema: obj({ key: { type: 'string', enum: ['volume-up', 'volume-down', 'mute', 'play-pause', 'next', 'previous'] } }, ['key']), where: 'main' },
{ name: 'clipboard_get', description: 'Lit le texte du presse-papiers.', inputSchema: obj({}), where: 'main' },
{ name: 'clipboard_set', description: 'Place un texte dans le presse-papiers.', inputSchema: obj({ text: { type: 'string' } }, ['text']), where: 'main' },
{ name: 'find_files', description: "Cherche des fichiers par nom dans Documents, Bureau, Téléchargements et Images de l'utilisateur (25 résultats max).", inputSchema: obj({ query: { type: 'string' } }, ['query']), where: 'main' },
{ name: 'speak_text', description: "Fait prononcer un texte par la voix d'EveFlow, immédiatement.", inputSchema: obj({ text: { type: 'string' } }, ['text']), where: 'renderer' },
{ name: 'notify_user', description: 'Affiche une notification système (titre + corps).', inputSchema: obj({ title: { type: 'string' }, body: { type: 'string' } }, ['body']), where: 'renderer' },
{ name: 'set_hud_state', description: 'Change momentanément l’état visuel du HUD : neutral, happy, thinking, alert, error.', inputSchema: obj({ state: { type: 'string', enum: ['neutral', 'happy', 'thinking', 'alert', 'error'] } }, ['state']), where: 'renderer' },
{ name: 'get_app_status', description: "État d'EveFlow : transport, liaison, nom de l'assistant, mains libres, voix en cours, heures calmes.", inputSchema: obj({}), where: 'renderer' },
{ name: 'get_conversation_history', description: 'Derniers messages de la conversation EveFlow (n ≤ 30).', inputSchema: obj({ n: { type: 'integer' } }), where: 'renderer' },
{ name: 'show_message', description: "Affiche un message dans le fil EveFlow (sans le prononcer) — pour les rapports longs ou les résultats de crons.", inputSchema: obj({ text: { type: 'string' }, title: { type: 'string' } }, ['text']), where: 'renderer' }
];
type Rec = Record<string, unknown>;
interface RpcRequest {
jsonrpc?: string;
id?: number | string | null;
method?: string;
params?: Rec;
}
const pending = new Map<string, { resolve: (r: McpToolResponse) => void; timer: ReturnType<typeof setTimeout> }>();
let seq = 0;
let ipcRegistered = false;
function ensureIpc(): void {
if (ipcRegistered) return;
ipcRegistered = true;
ipcMain.on(IPC.mcpResponse, (_e, res: McpToolResponse) => {
if (!res || typeof res.id !== 'string') return;
const entry = pending.get(res.id);
if (!entry) return;
clearTimeout(entry.timer);
pending.delete(res.id);
entry.resolve(res);
});
}
function askRenderer(win: BrowserWindow | null, name: string, args: Rec): Promise<McpToolResponse> {
ensureIpc();
if (!win || win.isDestroyed()) return Promise.resolve({ id: '', ok: false, error: 'EveFlow window unavailable' });
const id = `mcp-${++seq}-${Date.now()}`;
const req: McpToolRequest = { id, name, args };
return new Promise((resolve) => {
const timer = setTimeout(() => {
pending.delete(id);
resolve({ id, ok: false, error: 'renderer timeout' });
}, RENDERER_TIMEOUT_MS);
pending.set(id, { resolve, timer });
win.webContents.send(IPC.mcpRequest, req);
});
}
function text(value: unknown): { content: Array<Rec>; isError?: boolean } {
return { content: [{ type: 'text', text: typeof value === 'string' ? value : JSON.stringify(value) }] };
}
function failure(message: string): { content: Array<Rec>; isError: boolean } {
return { content: [{ type: 'text', text: message }], isError: true };
}
async function callTool(win: BrowserWindow | null, name: string, args: Rec): Promise<{ content: Array<Rec>; isError?: boolean }> {
const spec = MCP_TOOLS.find((t) => t.name === name);
if (!spec) return failure(`Outil inconnu : ${name}`);
if (spec.where === 'renderer') {
const res = await askRenderer(win, name, args);
return res.ok ? text(res.result ?? { ok: true }) : failure(res.error ?? 'échec');
}
const sys = async (action: SystemAction) => {
const r = await runSystemAction(action);
return r.ok ? text(r.data !== undefined ? { ok: true, message: r.message, data: r.data } : { ok: true, message: r.message }) : failure(r.message ?? 'échec');
};
switch (name) {
case 'capture_screen': {
const dataUrl = await captureScreen(Number(args.max_width) || 1600);
const base64 = dataUrl.slice(dataUrl.indexOf(',') + 1);
return { content: [{ type: 'image', data: base64, mimeType: 'image/jpeg' }, { type: 'text', text: `Capture de l'écran principal (${Math.round(base64.length * 0.75 / 1024)} ko).` }] };
}
case 'lock_session':
return sys({ type: 'lock' });
case 'open_app':
return sys({ type: 'open-app', name: String(args.name ?? '') });
case 'open_url':
return sys({ type: 'open-url', url: String(args.url ?? '') });
case 'media_key':
return sys({ type: 'media', key: String(args.key ?? '') as 'mute' });
case 'clipboard_get':
return sys({ type: 'clipboard-read' });
case 'clipboard_set':
return sys({ type: 'clipboard-write', text: String(args.text ?? '') });
case 'find_files':
return sys({ type: 'find-files', query: String(args.query ?? '') });
default:
return failure(`Outil non implémenté : ${name}`);
}
}
/** Handle one JSON-RPC message; returns the response object (or null for notifications). */
export async function handleMcpMessage(win: BrowserWindow | null, msg: RpcRequest): Promise<Rec | null> {
const id = msg.id ?? null;
const reply = (result: unknown) => ({ jsonrpc: '2.0', id, result });
const error = (code: number, message: string) => ({ jsonrpc: '2.0', id, error: { code, message } });
if (!msg.method) return error(-32600, 'Invalid Request');
if (msg.method.startsWith('notifications/')) return null;
switch (msg.method) {
case 'initialize':
return reply({ protocolVersion: PROTOCOL, capabilities: { tools: { listChanged: false } }, serverInfo: { name: 'eveflow', version: process.env.npm_package_version ?? '2.4.0' }, instructions: "Outils du PC de l'utilisateur via EveFlow : écran, applications, volume, presse-papiers, voix et notifications." });
case 'ping':
return reply({});
case 'tools/list':
return reply({ tools: MCP_TOOLS.map(({ name, description, inputSchema }) => ({ name, description, inputSchema })) });
case 'tools/call': {
const params = (msg.params ?? {}) as Rec;
const name = String(params.name ?? '');
const args = (params.arguments && typeof params.arguments === 'object' ? params.arguments : {}) as Rec;
log('INFO', 'mcp', `tools/call ${name}`);
try {
return reply(await callTool(win, name, args));
} catch (err) {
return reply(failure((err as Error).message));
}
}
case 'resources/list':
return reply({ resources: [] });
case 'prompts/list':
return reply({ prompts: [] });
default:
return error(-32601, `Method not found: ${msg.method}`);
}
}
/** HTTP entry point: POST /mcp with one JSON-RPC message or a batch. */
export async function handleMcpHttp(win: BrowserWindow | null, req: IncomingMessage, res: ServerResponse, body: string): Promise<void> {
const send = (code: number, payload: unknown) => {
if (payload === undefined) {
res.writeHead(code, { 'Cache-Control': 'no-store' });
res.end();
return;
}
res.writeHead(code, { 'Content-Type': 'application/json', 'Cache-Control': 'no-store' });
res.end(JSON.stringify(payload));
};
if (req.method === 'GET') return send(405, { error: 'SSE stream not supported; use POST' });
if (req.method === 'DELETE') return send(204, undefined);
if (req.method !== 'POST') return send(405, { error: 'Method Not Allowed' });
let parsed: unknown;
try {
parsed = body.trim() ? JSON.parse(body) : {};
} catch {
return send(400, { jsonrpc: '2.0', id: null, error: { code: -32700, message: 'Parse error' } });
}
const messages = Array.isArray(parsed) ? (parsed as RpcRequest[]) : [parsed as RpcRequest];
const replies: Rec[] = [];
for (const msg of messages) {
const out = await handleMcpMessage(win, msg && typeof msg === 'object' ? msg : {});
if (out) replies.push(out);
}
if (replies.length === 0) return send(202, undefined);
return send(200, Array.isArray(parsed) ? replies : replies[0]);
}
+29 -3
View File
@@ -13,7 +13,9 @@ import {
type WebhookStatus, type WebhookStatus,
type WindowMode type WindowMode
} from '../shared/ipc'; } from '../shared/ipc';
import type { EveFlowBridge, Unsubscribe } from '../shared/bridge'; import type { EveFlowBridge, SystemAction, SystemActionResult, Unsubscribe } from '../shared/bridge';
import type { McpToolRequest, McpToolResponse } from '../shared/ipc';
import { VOICE_IPC, type KwsDetection, type KwsStartRequest, type VadEvent, type VadStartRequest, type SynthesizeRequest, type SynthesizeResult, type TranscribeRequest, type TranscribeResult, type VoiceDownloadProgress, type VoiceEngineStatus, type VoiceModelStatus } from '../shared/voice';
function subscribe<T>(channel: string, callback: (payload: T) => void): Unsubscribe { function subscribe<T>(channel: string, callback: (payload: T) => void): Unsubscribe {
const listener = (_event: Electron.IpcRendererEvent, payload: T) => callback(payload); const listener = (_event: Electron.IpcRendererEvent, payload: T) => callback(payload);
@@ -25,7 +27,8 @@ const api: EveFlowBridge = {
window: { window: {
control: (action: 'minimize' | 'maximize' | 'close' | 'hide') => ipcRenderer.send(IPC.windowControl, action), control: (action: 'minimize' | 'maximize' | 'close' | 'hide') => ipcRenderer.send(IPC.windowControl, action),
setMode: (mode: WindowMode) => ipcRenderer.send(IPC.windowSetMode, mode), setMode: (mode: WindowMode) => ipcRenderer.send(IPC.windowSetMode, mode),
onModeChanged: (cb: (mode: WindowMode) => void) => subscribe<WindowMode>(IPC.windowModeChanged, cb) onModeChanged: (cb: (mode: WindowMode) => void) => subscribe<WindowMode>(IPC.windowModeChanged, cb),
onVisibility: (cb: (visible: boolean) => void) => subscribe<boolean>(IPC.windowVisibility, cb)
}, },
log: (entry: LogEntry) => ipcRenderer.send(IPC.log, entry), log: (entry: LogEntry) => ipcRenderer.send(IPC.log, entry),
store: { store: {
@@ -41,7 +44,9 @@ const api: EveFlowBridge = {
}, },
system: { system: {
metrics: () => ipcRenderer.invoke(IPC.metrics) as Promise<SystemMetrics>, metrics: () => ipcRenderer.invoke(IPC.metrics) as Promise<SystemMetrics>,
appInfo: () => ipcRenderer.invoke(IPC.appInfo) as Promise<AppInfo> appInfo: () => ipcRenderer.invoke(IPC.appInfo) as Promise<AppInfo>,
captureScreen: (maxWidth?: number) => ipcRenderer.invoke(IPC.screenCapture, maxWidth) as Promise<string>,
action: (action: SystemAction) => ipcRenderer.invoke(IPC.systemAction, action) as Promise<SystemActionResult>
}, },
files: { files: {
readLocal: (filePath: string) => ipcRenderer.invoke(IPC.readLocalFile, filePath) as Promise<string>, readLocal: (filePath: string) => ipcRenderer.invoke(IPC.readLocalFile, filePath) as Promise<string>,
@@ -53,10 +58,31 @@ const api: EveFlowBridge = {
hermes: { hermes: {
onPush: (cb: (event: HermesPushEvent) => void) => subscribe<HermesPushEvent>(IPC.hermesPush, cb), onPush: (cb: (event: HermesPushEvent) => void) => subscribe<HermesPushEvent>(IPC.hermesPush, cb),
webhookStatus: () => ipcRenderer.invoke(IPC.webhookStatus) as Promise<WebhookStatus>, webhookStatus: () => ipcRenderer.invoke(IPC.webhookStatus) as Promise<WebhookStatus>,
onToolRequest: (cb: (req: McpToolRequest) => void) => subscribe<McpToolRequest>(IPC.mcpRequest, cb),
toolResponse: (res: McpToolResponse) => ipcRenderer.send(IPC.mcpResponse, res),
webhookRestart: () => ipcRenderer.invoke(IPC.webhookRestart) as Promise<WebhookStatus> webhookRestart: () => ipcRenderer.invoke(IPC.webhookRestart) as Promise<WebhookStatus>
}, },
hotkeys: { hotkeys: {
on: (cb: (event: HotkeyEvent) => void) => subscribe<HotkeyEvent>(IPC.hotkey, cb) on: (cb: (event: HotkeyEvent) => void) => subscribe<HotkeyEvent>(IPC.hotkey, cb)
},
voice: {
status: () => ipcRenderer.invoke(VOICE_IPC.status) as Promise<VoiceEngineStatus>,
listModels: () => ipcRenderer.invoke(VOICE_IPC.modelsList) as Promise<VoiceModelStatus[]>,
downloadModel: (id: string) => ipcRenderer.invoke(VOICE_IPC.modelsDownload, id) as Promise<VoiceModelStatus>,
cancelDownload: (id: string) => ipcRenderer.invoke(VOICE_IPC.modelsCancel, id) as Promise<boolean>,
removeModel: (id: string) => ipcRenderer.invoke(VOICE_IPC.modelsRemove, id) as Promise<VoiceModelStatus[]>,
onProgress: (cb: (progress: VoiceDownloadProgress) => void) => subscribe<VoiceDownloadProgress>(VOICE_IPC.modelsProgress, cb),
transcribe: (req: TranscribeRequest) => ipcRenderer.invoke(VOICE_IPC.transcribe, req) as Promise<TranscribeResult>,
synthesize: (req: SynthesizeRequest) => ipcRenderer.invoke(VOICE_IPC.synthesize, req) as Promise<SynthesizeResult>,
unload: (id?: string) => ipcRenderer.invoke(VOICE_IPC.unload, id) as Promise<unknown>,
kwsStart: (req: KwsStartRequest) => ipcRenderer.invoke(VOICE_IPC.kwsStart, req) as Promise<{ accepted: string[]; rejected: string[] }>,
kwsStop: () => ipcRenderer.invoke(VOICE_IPC.kwsStop) as Promise<void>,
kwsAudio: (pcm: Uint8Array, sampleRate: number) => ipcRenderer.send(VOICE_IPC.kwsAudio, pcm, sampleRate),
onKwsDetected: (cb: (detection: KwsDetection) => void) => subscribe<KwsDetection>(VOICE_IPC.kwsDetected, cb),
vadStart: (req: VadStartRequest) => ipcRenderer.invoke(VOICE_IPC.vadStart, req) as Promise<void>,
vadStop: () => ipcRenderer.invoke(VOICE_IPC.vadStop) as Promise<void>,
vadAudio: (pcm: Uint8Array, sampleRate: number) => ipcRenderer.send(VOICE_IPC.vadAudio, pcm, sampleRate),
onVadEvent: (cb: (event: VadEvent) => void) => subscribe<VadEvent>(VOICE_IPC.vadEvent, cb)
} }
}; };
+188
View File
@@ -0,0 +1,188 @@
import type { VoiceModelSpec, VoiceSpeaker } from '../../shared/voice';
const ASR = 'https://github.com/k2-fsa/sherpa-onnx/releases/download/asr-models';
const KWS = 'https://github.com/k2-fsa/sherpa-onnx/releases/download/kws-models';
const TTS = 'https://github.com/k2-fsa/sherpa-onnx/releases/download/tts-models';
const KOKORO_SPEAKERS: VoiceSpeaker[] = [
{ id: 30, name: 'Siwis (femme, français)', lang: 'fr' },
{ id: 3, name: 'Heart (femme, anglais US)', lang: 'en' },
{ id: 2, name: 'Bella (femme, anglais US)', lang: 'en' },
{ id: 7, name: 'Nova (femme, anglais US)', lang: 'en' },
{ id: 11, name: 'Adam (homme, anglais US)', lang: 'en' },
{ id: 16, name: 'Michael (homme, anglais US)', lang: 'en' },
{ id: 17, name: 'Onyx (homme, anglais US)', lang: 'en' },
{ id: 21, name: 'Emma (femme, anglais UK)', lang: 'en' },
{ id: 24, name: 'Daniel (homme, anglais UK)', lang: 'en' },
{ id: 26, name: 'George (homme, anglais UK)', lang: 'en' }
];
export const VOICE_CATALOG: VoiceModelSpec[] = [
{
id: 'whisper-base',
kind: 'stt',
engine: 'whisper',
name: 'Whisper base (multilingue)',
description: 'Le plus rapide sur CPU ; précision moyenne en français, suffisante pour des commandes courtes.',
languages: ['fr', 'en', 'multi'],
sizeMb: 208,
url: `${ASR}/sherpa-onnx-whisper-base.tar.bz2`,
dir: 'sherpa-onnx-whisper-base',
files: ['base-encoder.int8.onnx', 'base-decoder.int8.onnx', 'base-tokens.txt'],
recommended: true
},
{
id: 'whisper-small',
kind: 'stt',
engine: 'whisper',
name: 'Whisper small (multilingue)',
description: 'Recommandé pour le français : nettement plus précis que base, environ trois fois plus lent.',
languages: ['fr', 'en', 'multi'],
sizeMb: 640,
url: `${ASR}/sherpa-onnx-whisper-small.tar.bz2`,
dir: 'sherpa-onnx-whisper-small',
files: ['small-encoder.int8.onnx', 'small-decoder.int8.onnx', 'small-tokens.txt'],
recommended: true
},
{
id: 'parakeet-v3',
kind: 'stt',
engine: 'nemo-transducer',
name: 'Parakeet TDT 0.6B v3 (25 langues européennes, français)',
description: 'Le plus précis en français et nettement plus rapide que Whisper sur processeur (NVIDIA NeMo, int8). Ponctuation et majuscules incluses.',
languages: ['fr', 'en', 'de', 'es', 'it', 'multi'],
sizeMb: 640,
url: `${ASR}/sherpa-onnx-nemo-parakeet-tdt-0.6b-v3-int8.tar.bz2`,
dir: 'sherpa-onnx-nemo-parakeet-tdt-0.6b-v3-int8',
files: ['encoder.int8.onnx', 'decoder.int8.onnx', 'joiner.int8.onnx', 'tokens.txt'],
recommended: true
},
{
id: 'whisper-turbo',
kind: 'stt',
engine: 'whisper',
name: 'Whisper large-v3 turbo (multilingue)',
description: 'Très bonne précision multilingue ; 2 à 4 s par phrase sur un processeur récent. Parakeet est plus rapide en français.',
languages: ['fr', 'en', 'multi'],
sizeMb: 564,
url: `${ASR}/sherpa-onnx-whisper-turbo.tar.bz2`,
dir: 'sherpa-onnx-whisper-turbo',
files: ['turbo-encoder.int8.onnx', 'turbo-decoder.int8.onnx', 'turbo-tokens.txt']
},
{
id: 'sense-voice',
kind: 'stt',
engine: 'sense-voice',
name: 'SenseVoice small (zh / en / ja / ko)',
description: 'Très rapide, mais ne comprend pas le français.',
languages: ['en', 'zh', 'ja', 'ko'],
sizeMb: 163,
url: `${ASR}/sherpa-onnx-sense-voice-zh-en-ja-ko-yue-int8-2024-07-17.tar.bz2`,
dir: 'sherpa-onnx-sense-voice-zh-en-ja-ko-yue-int8-2024-07-17',
files: ['model.int8.onnx', 'tokens.txt']
},
{
id: 'kws-en',
kind: 'kws',
engine: 'kws-transducer',
name: 'Détecteur de mot-clé (zipformer, 3 Mo)',
description: 'Écoute permanente du mot d’activation, quasi gratuite en CPU. Mot-clé libre (« jarvis », « hey jarvis »…).',
languages: ['en', 'fr'],
sizeMb: 4,
url: `${KWS}/sherpa-onnx-kws-zipformer-gigaspeech-3.3M-2024-01-01.tar.bz2`,
dir: 'sherpa-onnx-kws-zipformer-gigaspeech-3.3M-2024-01-01',
files: [
'encoder-epoch-12-avg-2-chunk-16-left-64.int8.onnx',
'decoder-epoch-12-avg-2-chunk-16-left-64.int8.onnx',
'joiner-epoch-12-avg-2-chunk-16-left-64.int8.onnx',
'tokens.txt'
],
recommended: true
},
{
id: 'silero-vad',
kind: 'vad',
engine: 'silero',
name: 'Silero VAD (fin de phrase neuronale, 0,6 Mo)',
description: 'Détecte précisément le début et la fin de la parole pendant l’écoute permanente ; moins de faux départs sur le bruit.',
languages: ['multi'],
sizeMb: 1,
url: `${ASR}/silero_vad.onnx`,
dir: 'silero-vad',
files: ['silero_vad.onnx'],
recommended: true
},
{
id: 'kokoro-v1',
kind: 'tts',
engine: 'kokoro',
name: 'Kokoro v1.0 multilingue',
description: 'Voix très naturelle, une voix française (Siwis) et de nombreuses voix anglaises. 24 kHz.',
languages: ['fr', 'en', 'multi'],
sizeMb: 349,
url: `${TTS}/kokoro-multi-lang-v1_0.tar.bz2`,
dir: 'kokoro-multi-lang-v1_0',
files: ['model.onnx', 'voices.bin', 'tokens.txt', 'lexicon-us-en.txt', 'lexicon-zh.txt', 'espeak-ng-data/phontab'],
speakers: KOKORO_SPEAKERS,
sampleRate: 24000,
recommended: true
},
{
id: 'piper-fr-siwis',
kind: 'tts',
engine: 'piper',
name: 'Piper Siwis (femme, français)',
description: 'Léger et instantané, rendu un peu plus mécanique que Kokoro. 22 kHz.',
languages: ['fr'],
sizeMb: 67,
url: `${TTS}/vits-piper-fr_FR-siwis-medium.tar.bz2`,
dir: 'vits-piper-fr_FR-siwis-medium',
files: ['fr_FR-siwis-medium.onnx', 'tokens.txt', 'espeak-ng-data/phontab'],
speakers: [{ id: 0, name: 'Siwis', lang: 'fr' }],
sampleRate: 22050
},
{
id: 'piper-fr-tom',
kind: 'tts',
engine: 'piper',
name: 'Piper Tom (homme, français)',
description: 'Voix masculine légère et instantanée. 22 kHz.',
languages: ['fr'],
sizeMb: 67,
url: `${TTS}/vits-piper-fr_FR-tom-medium.tar.bz2`,
dir: 'vits-piper-fr_FR-tom-medium',
files: ['fr_FR-tom-medium.onnx', 'tokens.txt', 'espeak-ng-data/phontab'],
speakers: [{ id: 0, name: 'Tom', lang: 'fr' }],
sampleRate: 22050
},
{
id: 'piper-fr-upmc',
kind: 'tts',
engine: 'piper',
name: 'Piper UPMC (Jessica et Pierre, français)',
description: 'Deux voix françaises, une féminine et une masculine. 22 kHz.',
languages: ['fr'],
sizeMb: 80,
url: `${TTS}/vits-piper-fr_FR-upmc-medium.tar.bz2`,
dir: 'vits-piper-fr_FR-upmc-medium',
files: ['fr_FR-upmc-medium.onnx', 'tokens.txt', 'espeak-ng-data/phontab'],
speakers: [
{ id: 0, name: 'Jessica (femme)', lang: 'fr' },
{ id: 1, name: 'Pierre (homme)', lang: 'fr' }
],
sampleRate: 22050
}
];
export function findModel(id: string): VoiceModelSpec | undefined {
return VOICE_CATALOG.find((m) => m.id === id);
}
// Speaker gender from the label ("(homme, …)", "(femme, …)", known first names) so the UI and the
// voice preference can pick a masculine or feminine voice without a lookup table per model.
const MALE = /\b(homme|tom|pierre|adam|michael|eric|liam|george|lewis|daniel|fenrir|puck|onyx|echo|santa)\b/i;
for (const spec of VOICE_CATALOG) {
for (const sp of spec.speakers ?? []) {
if (!sp.gender) sp.gender = /\b(femme|female)\b/i.test(sp.name) ? 'f' : MALE.test(sp.name) ? 'm' : /\bfemme\b/i.test(spec.name) ? 'f' : /\bhomme\b/i.test(spec.name) ? 'm' : undefined;
}
}
+240
View File
@@ -0,0 +1,240 @@
/**
* Host side of the voice worker: spawns the utility process on demand, correlates
* requests and responses, restarts the worker if it crashes.
*/
import { utilityProcess, type UtilityProcess, type WebContents } from 'electron';
import { createHash } from 'node:crypto';
import fs from 'node:fs';
import path from 'node:path';
import { VOICE_IPC, type KwsDetection, type KwsStartRequest, type SynthesizeRequest, type SynthesizeResult, type TranscribeRequest, type TranscribeResult, type VadEvent, type VadStartRequest, type VoiceEngineStatus } from '../../shared/voice';
import { buildKeywordsFile, parseTokens } from '../../shared/keywords';
import { findModel } from './catalog';
import { isInstalled, modelDir, modelsDir } from './models';
import { log } from '../logger';
/** Renderer that receives keyword detections while spotting is active. */
let kwsSubscriber: WebContents | null = null;
let kwsActive = false;
let kwsRequest: KwsStartRequest | null = null;
let vadSubscriber: WebContents | null = null;
let vadActive = false;
interface Pending {
resolve: (value: unknown) => void;
reject: (err: Error) => void;
timer: ReturnType<typeof setTimeout>;
}
let child: UtilityProcess | null = null;
let nextId = 1;
const pending = new Map<number, Pending>();
function rejectAll(message: string): void {
for (const [id, p] of pending) {
clearTimeout(p.timer);
p.reject(new Error(message));
pending.delete(id);
}
}
function spawn(): UtilityProcess {
if (child) return child;
const script = path.join(__dirname, 'voice-worker.js');
const proc = utilityProcess.fork(script, [], { serviceName: 'eveflow-voice', stdio: 'pipe' });
child = proc;
proc.stdout?.on('data', (d: Buffer) => log('DEBUG', 'voice-worker', d.toString().trim()));
proc.stderr?.on('data', (d: Buffer) => log('WARN', 'voice-worker', d.toString().trim()));
proc.on('message', (msg: { id?: number; type?: string; ok?: boolean; result?: unknown; error?: string; keyword?: string; at?: number; event?: { type: string; wav?: string; durationSec?: number } }) => {
if (msg.type === 'vad.event' && msg.event) {
if (vadSubscriber && !vadSubscriber.isDestroyed()) {
const ev = msg.event;
const payload: VadEvent =
ev.type === 'segment' && ev.wav
? { type: 'segment', wav: new Uint8Array(Buffer.from(ev.wav, 'base64')), durationSec: ev.durationSec ?? 0 }
: ev.type === 'speech-start'
? { type: 'speech-start' }
: { type: 'error', message: 'événement VAD inconnu' };
vadSubscriber.send(VOICE_IPC.vadEvent, payload);
}
return;
}
if (msg.type === 'kws.detected') {
if (kwsSubscriber && !kwsSubscriber.isDestroyed()) {
kwsSubscriber.send(VOICE_IPC.kwsDetected, { keyword: msg.keyword ?? '', at: msg.at ?? Date.now() } satisfies KwsDetection);
}
log('INFO', 'voice', `wake word detected: ${msg.keyword}`);
return;
}
if (typeof msg.id !== 'number') return;
const p = pending.get(msg.id);
if (!p) return;
pending.delete(msg.id);
clearTimeout(p.timer);
if (msg.ok) p.resolve(msg.result);
else p.reject(new Error(msg.error ?? 'erreur moteur vocal'));
});
proc.on('exit', (code) => {
log(code === 0 ? 'INFO' : 'ERROR', 'voice-worker', `exited with code ${code}`);
// A newer worker may already have replaced this one; only clean up if we are still current.
if (child !== proc) return;
child = null;
rejectAll('Le moteur vocal local s’est arrêté de façon inattendue.');
// Keyword spotting survives a worker restart: re-arm on the next audio frame.
if (kwsActive) kwsArmed = false;
});
log('INFO', 'voice', 'voice worker started');
return proc;
}
function request<T>(message: Record<string, unknown>, timeoutMs = 120_000): Promise<T> {
const proc = spawn();
const id = nextId++;
return new Promise<T>((resolve, reject) => {
const timer = setTimeout(() => {
pending.delete(id);
reject(new Error('Délai dépassé pour le moteur vocal local.'));
// The worker handles requests synchronously: a stuck request wedges everything behind it.
if (child === proc) {
log('WARN', 'voice', 'worker unresponsive, restarting');
stopEngine();
}
}, timeoutMs);
pending.set(id, { resolve: (v) => resolve(v as T), reject, timer });
try {
proc.postMessage({ id, ...message });
} catch (err) {
pending.delete(id);
clearTimeout(timer);
reject(err as Error);
}
});
}
function modelRef(modelId: string) {
const spec = findModel(modelId);
if (!spec) throw new Error(`Modèle inconnu : ${modelId}`);
if (!isInstalled(spec)) throw new Error(`Modèle « ${spec.name} » non installé. Téléchargez-le dans Paramètres → Modèles.`);
return { id: spec.id, engine: spec.engine, dir: modelDir(spec), files: spec.files };
}
export async function engineStatus(): Promise<VoiceEngineStatus> {
try {
const status = await request<Omit<VoiceEngineStatus, 'modelsDir'>>({ type: 'status' }, 20_000);
return { ...status, modelsDir: modelsDir() };
} catch (err) {
return { available: false, error: (err as Error).message, loaded: [], modelsDir: modelsDir() };
}
}
export function transcribe(req: TranscribeRequest): Promise<TranscribeResult> {
// Binary payloads are base64-encoded for the worker: the utility process channel rejects external buffers.
const wav = Buffer.from(req.wav.buffer, req.wav.byteOffset, req.wav.byteLength).toString('base64');
return request<TranscribeResult>({ type: 'transcribe', model: modelRef(req.modelId), wav, language: req.language }, 180_000);
}
export async function synthesize(req: SynthesizeRequest): Promise<SynthesizeResult> {
const result = await request<Omit<SynthesizeResult, 'wav'> & { wav: string }>(
{ type: 'synthesize', model: modelRef(req.modelId), text: req.text, speaker: req.speaker, speed: req.speed },
180_000
);
const buffer = Buffer.from(result.wav, 'base64');
return { ...result, wav: new Uint8Array(buffer.buffer, buffer.byteOffset, buffer.byteLength) };
}
let kwsArmed = false;
const SENSITIVITY_THRESHOLD: Record<number, number> = { 1: 0.45, 2: 0.35, 3: 0.25, 4: 0.18, 5: 0.12 };
const SENSITIVITY_SCORE: Record<number, number> = { 1: 1.0, 2: 1.0, 3: 1.2, 4: 1.5, 5: 2.0 };
/** Start keyword spotting; the keywords file is derived from the phrases and the model vocabulary. */
export async function kwsStart(req: KwsStartRequest, sender: WebContents): Promise<{ accepted: string[]; rejected: string[] }> {
const spec = findModel(req.modelId);
if (!spec || spec.kind !== 'kws') throw new Error('Modèle de détection introuvable');
if (!isInstalled(spec)) throw new Error('Détecteur de mot-clé non installé (Paramètres → Modèles locaux).');
const dir = modelDir(spec);
const vocab = parseTokens(fs.readFileSync(path.join(dir, 'tokens.txt'), 'utf8'));
const phrases = req.keywords.map((k) => String(k).slice(0, 40)).filter(Boolean).slice(0, 8);
const file = buildKeywordsFile(phrases, vocab);
if (file.accepted.length === 0) throw new Error(`Aucun mot d’activation encodable : ${file.rejected.join(', ')}`);
const hash = createHash('sha1').update(file.content).digest('hex').slice(0, 10);
const keywordsFile = path.join(modelsDir(), `keywords-${hash}.txt`);
fs.writeFileSync(keywordsFile, file.content, 'utf8');
const sensitivity = Math.min(5, Math.max(1, Math.round(req.sensitivity))) as 1 | 2 | 3 | 4 | 5;
kwsSubscriber = sender;
kwsRequest = req;
await request({ type: 'kws.start', model: { id: spec.id, engine: spec.engine, dir, files: spec.files }, keywordsFile, threshold: SENSITIVITY_THRESHOLD[sensitivity], score: SENSITIVITY_SCORE[sensitivity] }, 60_000);
kwsActive = true;
kwsArmed = true;
log('INFO', 'voice', `keyword spotting on: ${file.accepted.join(', ')} (threshold ${SENSITIVITY_THRESHOLD[sensitivity]})`);
return { accepted: file.accepted, rejected: file.rejected };
}
export async function kwsStop(): Promise<void> {
kwsActive = false;
kwsArmed = false;
kwsSubscriber = null;
if (child) await request({ type: 'kws.stop' }, 10_000).catch(() => undefined);
}
/** Feed 16-bit PCM from the renderer (fire-and-forget). */
export function kwsFeed(pcm: Uint8Array, sampleRate: number): void {
if (!kwsActive) return;
const proc = spawn();
if (!kwsArmed) {
// Worker restarted: re-create the spotter before feeding audio.
if (kwsRequest && kwsSubscriber) {
kwsArmed = true;
kwsStart(kwsRequest, kwsSubscriber).catch((err) => log('WARN', 'voice', `kws re-arm failed: ${(err as Error).message}`));
}
return;
}
const b64 = Buffer.from(pcm.buffer, pcm.byteOffset, pcm.byteLength).toString('base64');
try {
proc.postMessage({ id: -1, type: 'kws.audio', pcm: b64, sampleRate });
} catch (err) {
log('WARN', 'voice', `kws feed failed: ${(err as Error).message}`);
}
}
/** Neural end-of-speech detection (Silero) on frames streamed by the renderer. */
export async function vadStart(req: VadStartRequest, sender: WebContents): Promise<void> {
const spec = findModel(req.modelId);
if (!spec || spec.kind !== 'vad') throw new Error('Modèle VAD introuvable');
if (!isInstalled(spec)) throw new Error('Silero VAD non installé (Paramètres → Modèles locaux).');
vadSubscriber = sender;
await request(
{ type: 'vad.start', model: { id: spec.id, engine: spec.engine, dir: modelDir(spec), files: spec.files }, silenceMs: req.silenceMs, threshold: req.threshold, maxUtteranceSec: req.maxUtteranceSec },
30_000
);
vadActive = true;
}
export async function vadStop(): Promise<void> {
vadActive = false;
vadSubscriber = null;
if (child) await request({ type: 'vad.stop' }, 10_000).catch(() => undefined);
}
export function vadFeed(pcm: Uint8Array, sampleRate: number): void {
if (!vadActive || !child) return;
const b64 = Buffer.from(pcm.buffer, pcm.byteOffset, pcm.byteLength).toString('base64');
try {
child.postMessage({ id: -1, type: 'vad.audio', pcm: b64, sampleRate });
} catch (err) {
log('WARN', 'voice', `vad feed failed: ${(err as Error).message}`);
}
}
/** Native model memory is only released deterministically by restarting the worker. */
export function unload(_modelId?: string): Promise<unknown> {
stopEngine();
return Promise.resolve({ ok: true });
}
export function stopEngine(): void {
const proc = child;
child = null;
if (proc) {
rejectAll('Moteur vocal redémarré.');
proc.kill();
}
}
+48
View File
@@ -0,0 +1,48 @@
import { ipcMain } from 'electron';
import { VOICE_IPC, type KwsStartRequest, type SynthesizeRequest, type TranscribeRequest, type VadStartRequest } from '../../shared/voice';
import { engineStatus, kwsFeed, kwsStart, kwsStop, synthesize, transcribe, unload, vadFeed, vadStart, vadStop } from './engine';
import { cancelDownload, downloadModel, listModels, removeModel } from './models';
export function registerVoiceIpc(): void {
ipcMain.handle(VOICE_IPC.status, () => engineStatus());
ipcMain.handle(VOICE_IPC.modelsList, () => listModels());
ipcMain.handle(VOICE_IPC.modelsDownload, (event, id: string) => downloadModel(id, event.sender));
ipcMain.handle(VOICE_IPC.modelsCancel, (_e, id: string) => {
cancelDownload(id);
return true;
});
ipcMain.handle(VOICE_IPC.modelsRemove, async (_e, id: string) => {
await unload(id).catch(() => undefined);
await removeModel(id);
return listModels();
});
ipcMain.handle(VOICE_IPC.transcribe, (_e, req: TranscribeRequest) => {
if (!req || !(req.wav instanceof Uint8Array) || req.wav.byteLength < 44 || req.wav.byteLength > 64 * 1024 * 1024) throw new Error('Audio invalide');
if (typeof req.modelId !== 'string') throw new Error('Modèle invalide');
return transcribe({ ...req, language: typeof req.language === 'string' ? req.language.slice(0, 8) : 'auto' });
});
ipcMain.handle(VOICE_IPC.synthesize, (_e, req: SynthesizeRequest) => {
if (!req || typeof req.text !== 'string' || !req.text.trim() || req.text.length > 5000) throw new Error('Texte invalide');
if (typeof req.modelId !== 'string') throw new Error('Modèle invalide');
return synthesize({ ...req, speaker: Number.isFinite(req.speaker) ? req.speaker : 0, speed: Number.isFinite(req.speed) ? req.speed : 1 });
});
ipcMain.handle(VOICE_IPC.unload, (_e, id?: string) => unload(id));
ipcMain.handle(VOICE_IPC.kwsStart, (event, req: KwsStartRequest) => {
if (!req || !Array.isArray(req.keywords) || typeof req.modelId !== 'string') throw new Error('Requête invalide');
return kwsStart({ ...req, keywords: req.keywords.filter((k) => typeof k === 'string'), sensitivity: Number(req.sensitivity) || 3 }, event.sender);
});
ipcMain.handle(VOICE_IPC.kwsStop, () => kwsStop());
ipcMain.handle(VOICE_IPC.vadStart, (event, req: VadStartRequest) => {
if (!req || typeof req.modelId !== 'string') throw new Error('Requête invalide');
return vadStart({ modelId: req.modelId, silenceMs: Number(req.silenceMs) || 700, threshold: Number(req.threshold) || 0.5, maxUtteranceSec: Number(req.maxUtteranceSec) || 30 }, event.sender);
});
ipcMain.handle(VOICE_IPC.vadStop, () => vadStop());
ipcMain.on(VOICE_IPC.vadAudio, (_e, pcm: unknown, sampleRate: unknown) => {
if (!(pcm instanceof Uint8Array) || pcm.byteLength === 0 || pcm.byteLength > 1024 * 1024) return;
vadFeed(pcm, typeof sampleRate === 'number' && sampleRate > 0 ? sampleRate : 16000);
});
ipcMain.on(VOICE_IPC.kwsAudio, (_e, pcm: unknown, sampleRate: unknown) => {
if (!(pcm instanceof Uint8Array) || pcm.byteLength === 0 || pcm.byteLength > 1024 * 1024) return;
kwsFeed(pcm, typeof sampleRate === 'number' && sampleRate > 0 ? sampleRate : 16000);
});
}
+159
View File
@@ -0,0 +1,159 @@
/**
* Model manager: downloads sherpa-onnx model archives (tar.bz2) into userData/models,
* extracts them, verifies required files and reports progress to the renderer.
*/
import { app, type WebContents } from 'electron';
import fs from 'node:fs';
import path from 'node:path';
import { pipeline } from 'node:stream/promises';
import { Readable, Transform } from 'node:stream';
import * as tar from 'tar';
import unbzip2 from 'unbzip2-stream';
import { VOICE_IPC, type VoiceDownloadProgress, type VoiceModelSpec, type VoiceModelStatus } from '../../shared/voice';
import { VOICE_CATALOG, findModel } from './catalog';
import { stopEngine } from './engine';
import { log } from '../logger';
const active = new Map<string, AbortController>();
/** Windows keeps files locked briefly (AV scans, memory-mapped models): retry removals. */
const RM = { recursive: true, force: true, maxRetries: 10, retryDelay: 200 } as const;
const DOWNLOAD_IDLE_MS = 60_000;
export function modelsDir(): string {
const dir = path.join(app.getPath('userData'), 'models');
fs.mkdirSync(dir, { recursive: true });
return dir;
}
export function modelDir(spec: VoiceModelSpec): string {
return path.join(modelsDir(), spec.dir);
}
export function isInstalled(spec: VoiceModelSpec): boolean {
const dir = modelDir(spec);
return spec.files.every((f) => fs.existsSync(path.join(dir, f)));
}
function dirSize(dir: string): number {
let total = 0;
try {
for (const entry of fs.readdirSync(dir, { withFileTypes: true })) {
const full = path.join(dir, entry.name);
if (entry.isDirectory()) total += dirSize(full);
else total += fs.statSync(full).size;
}
} catch {
/* missing */
}
return total;
}
export function listModels(): VoiceModelStatus[] {
return VOICE_CATALOG.map((spec) => {
const installed = isInstalled(spec);
return {
...spec,
installed,
downloading: active.has(spec.id),
installedBytes: installed ? dirSize(modelDir(spec)) : 0
};
});
}
export async function removeModel(id: string): Promise<void> {
const spec = findModel(id);
if (!spec) throw new Error(`Modèle inconnu : ${id}`);
stopEngine(); // releases memory-mapped model files before deleting them
await fs.promises.rm(modelDir(spec), RM);
log('INFO', 'voice', `model removed: ${id}`);
}
export function cancelDownload(id: string): void {
active.get(id)?.abort();
}
export async function downloadModel(id: string, sender: WebContents | null): Promise<VoiceModelStatus> {
const spec = findModel(id);
if (!spec) throw new Error(`Modèle inconnu : ${id}`);
if (active.has(id)) throw new Error('Téléchargement déjà en cours');
const controller = new AbortController();
active.set(id, controller);
const report = (progress: Omit<VoiceDownloadProgress, 'id'>) => {
if (sender && !sender.isDestroyed()) sender.send(VOICE_IPC.modelsProgress, { id, ...progress });
};
const tmpDir = path.join(modelsDir(), `.tmp-${spec.id}`);
await fs.promises.rm(tmpDir, RM);
await fs.promises.mkdir(tmpDir, { recursive: true });
// Abort the transfer when no byte arrives for a while (fetch itself has no inactivity timeout).
let idleTimer: ReturnType<typeof setTimeout> | null = null;
const touch = () => {
if (idleTimer) clearTimeout(idleTimer);
idleTimer = setTimeout(() => controller.abort(new Error('Téléchargement interrompu (aucune donnée reçue)')), DOWNLOAD_IDLE_MS);
};
try {
log('INFO', 'voice', `downloading ${spec.id} from ${spec.url}`);
const response = await fetch(spec.url, { signal: controller.signal, redirect: 'follow' });
if (!response.ok || !response.body) throw new Error(`HTTP ${response.status} lors du téléchargement`);
const total = Number(response.headers.get('content-length')) || Math.round(spec.sizeMb * 1024 * 1024);
let received = 0;
let lastReport = 0;
touch();
const counter = new Transform({
transform(chunk: Buffer, _enc, cb) {
received += chunk.length;
touch();
const now = Date.now();
if (now - lastReport > 250) {
lastReport = now;
report({ phase: 'download', received, total, percent: Math.min(99, Math.round((received / total) * 100)) });
}
cb(null, chunk);
}
});
const source = Readable.fromWeb(response.body as import('node:stream/web').ReadableStream<Uint8Array>);
if (spec.url.endsWith('.tar.bz2')) {
report({ phase: 'download', received: 0, total, percent: 0 });
await pipeline(source, counter, unbzip2(), tar.x({ cwd: tmpDir }), { signal: controller.signal });
} else {
const target = path.join(tmpDir, spec.dir, path.basename(spec.url));
await fs.promises.mkdir(path.dirname(target), { recursive: true });
await pipeline(source, counter, fs.createWriteStream(target), { signal: controller.signal });
}
report({ phase: 'extract', received: total, total, percent: 99 });
const extracted = path.join(tmpDir, spec.dir);
const missing = spec.files.filter((f) => !fs.existsSync(path.join(extracted, f)));
if (missing.length) throw new Error(`Archive incomplète, fichiers manquants : ${missing.join(', ')}`);
const finalDir = modelDir(spec);
if (isInstalled(spec)) stopEngine();
await fs.promises.rm(finalDir, RM);
for (let attempt = 0; ; attempt++) {
try {
await fs.promises.rename(extracted, finalDir);
break;
} catch (err) {
const code = (err as NodeJS.ErrnoException).code ?? '';
if (attempt >= 10 || !['EPERM', 'EBUSY', 'ENOTEMPTY', 'EACCES'].includes(code)) throw err;
await new Promise((r) => setTimeout(r, 300));
}
}
await fs.promises.rm(tmpDir, RM);
report({ phase: 'done', received: total, total, percent: 100 });
log('INFO', 'voice', `model installed: ${spec.id}`);
} catch (err) {
await fs.promises.rm(tmpDir, RM).catch(() => undefined);
const e = (controller.signal.reason instanceof Error ? controller.signal.reason : err) as Error;
const cancelled = (e.name === 'AbortError' || controller.signal.aborted) && !(controller.signal.reason instanceof Error);
report({ phase: cancelled ? 'cancelled' : 'error', received: 0, total: 0, percent: 0, message: cancelled ? 'annulé' : e.message });
log(cancelled ? 'INFO' : 'ERROR', 'voice', `download ${spec.id} ${cancelled ? 'cancelled' : 'failed: ' + e.message}`);
if (!cancelled) throw new Error(e.message);
} finally {
if (idleTimer) clearTimeout(idleTimer);
active.delete(id);
}
return listModels().find((m) => m.id === id)!;
}
+443
View File
@@ -0,0 +1,443 @@
/**
* Voice worker: runs sherpa-onnx (speech recognition and synthesis) in an Electron utility
* process so heavy inference never blocks the main process. Also runnable with
* `child_process.fork` (advanced serialization) for local tests.
*/
import os from 'node:os';
import path from 'node:path';
import type { VoiceEngineKind } from '../../shared/voice';
interface ModelRef {
id: string;
engine: VoiceEngineKind;
dir: string;
files: string[];
}
type Request =
| { id: number; type: 'status' }
| { id: number; type: 'transcribe'; model: ModelRef; wav: Uint8Array | string; language: string }
| { id: number; type: 'synthesize'; model: ModelRef; text: string; speaker: number; speed: number }
| { id: number; type: 'unload'; modelId?: string }
| { id: number; type: 'kws.start'; model: ModelRef; keywordsFile: string; threshold: number; score: number }
| { id: number; type: 'kws.audio'; pcm: string; sampleRate: number }
| { id: number; type: 'kws.stop' }
| { id: number; type: 'vad.start'; model: ModelRef; silenceMs: number; threshold: number; maxUtteranceSec: number }
| { id: number; type: 'vad.audio'; pcm: string; sampleRate: number }
| { id: number; type: 'vad.stop' };
type Response = { id: number; ok: true; result: unknown } | { id: number; ok: false; error: string };
// ── sherpa-onnx loading (lazy, so a missing native package is reported, not fatal) ──
type Sherpa = {
OfflineRecognizer: new (config: unknown) => {
createStream: () => { acceptWaveform: (w: { sampleRate: number; samples: Float32Array }) => void };
decode: (s: unknown) => void;
getResult: (s: unknown) => { text: string; lang?: string };
};
Vad: new (config: unknown, bufferSizeInSeconds: number) => {
acceptWaveform: (samples: Float32Array) => void;
isEmpty: () => boolean;
isDetected: () => boolean;
pop: () => void;
clear: () => void;
front: (enableExternalBuffer?: boolean) => { start: number; samples: Float32Array };
reset: () => void;
flush: () => void;
};
KeywordSpotter: new (config: unknown) => {
createStream: () => KwsStream;
isReady: (s: KwsStream) => boolean;
decode: (s: KwsStream) => void;
reset: (s: KwsStream) => void;
getResult: (s: KwsStream) => { keyword?: string };
};
OfflineTts: new (config: unknown) => {
numSpeakers: number;
sampleRate: number;
generate: (req: { text: string; sid: number; speed: number; enableExternalBuffer?: boolean }) => { samples: Float32Array; sampleRate: number };
};
version: string;
};
type KwsStream = { acceptWaveform: (w: { sampleRate: number; samples: Float32Array }) => void };
let sherpa: Sherpa | null = null;
let kws: { spotter: InstanceType<Sherpa['KeywordSpotter']>; stream: KwsStream } | null = null;
let vad: { detector: InstanceType<Sherpa['Vad']>; speaking: boolean; windowSize: number; carry: Float32Array } | null = null;
let notify: ((message: unknown) => void) | null = null;
let loadError: string | null = null;
function loadSherpa(): Sherpa {
if (sherpa) return sherpa;
if (loadError) throw new Error(loadError);
try {
// eslint-disable-next-line @typescript-eslint/no-require-imports
sherpa = require('sherpa-onnx-node') as Sherpa;
return sherpa;
} catch (err) {
loadError = `Module natif sherpa-onnx indisponible : ${(err as Error).message}`;
throw new Error(loadError);
}
}
const threads = Math.max(2, Math.min(6, Math.floor(os.cpus().length / 2)));
// ── caches ────────────────────────────────────────────────────────────────
const recognizers = new Map<string, InstanceType<Sherpa['OfflineRecognizer']>>();
const synthesizers = new Map<string, InstanceType<Sherpa['OfflineTts']>>();
function whisperPrefix(model: ModelRef): string {
const encoder = model.files.find((f) => f.includes('-encoder'));
return encoder ? encoder.slice(0, encoder.indexOf('-encoder')) : 'base';
}
function getRecognizer(model: ModelRef, language: string) {
const lang = language === 'auto' ? '' : language.split('-')[0].toLowerCase();
const key = `${model.id}:${lang}`;
const cached = recognizers.get(key);
if (cached) return cached;
const s = loadSherpa();
const p = (f: string) => path.join(model.dir, f);
let modelConfig: Record<string, unknown>;
switch (model.engine) {
case 'whisper': {
const prefix = whisperPrefix(model);
modelConfig = {
whisper: { encoder: p(`${prefix}-encoder.int8.onnx`), decoder: p(`${prefix}-decoder.int8.onnx`), language: lang, task: 'transcribe', tailPaddings: -1 },
tokens: p(`${prefix}-tokens.txt`)
};
break;
}
case 'sense-voice':
modelConfig = { senseVoice: { model: p('model.int8.onnx'), language: lang || 'auto', useInverseTextNormalization: 1 }, tokens: p('tokens.txt') };
break;
case 'nemo-transducer':
modelConfig = {
transducer: { encoder: p('encoder.int8.onnx'), decoder: p('decoder.int8.onnx'), joiner: p('joiner.int8.onnx') },
tokens: p('tokens.txt'),
modelType: 'nemo_transducer'
};
break;
default:
throw new Error(`Moteur STT non supporté : ${model.engine}`);
}
for (const [k, v] of Object.entries({ numThreads: threads, provider: 'cpu', debug: 0 })) modelConfig[k] = v;
// Whisper keeps one recognizer per language; other engines ignore the language key.
for (const [k, r] of recognizers) if (k.startsWith(`${model.id}:`)) recognizers.delete(k) && void r;
const recognizer = new s.OfflineRecognizer({ featConfig: { sampleRate: 16000, featureDim: 80 }, modelConfig, decodingMethod: 'greedy_search' });
recognizers.set(key, recognizer);
return recognizer;
}
function getSynthesizer(model: ModelRef) {
const cached = synthesizers.get(model.id);
if (cached) return cached;
const s = loadSherpa();
const p = (f: string) => path.join(model.dir, f);
let ttsModel: Record<string, unknown>;
switch (model.engine) {
case 'kokoro':
ttsModel = {
kokoro: {
model: p('model.onnx'),
voices: p('voices.bin'),
tokens: p('tokens.txt'),
dataDir: p('espeak-ng-data'),
lexicon: [p('lexicon-us-en.txt'), p('lexicon-zh.txt')].join(',')
}
};
break;
case 'piper': {
const onnx = model.files.find((f) => f.endsWith('.onnx')) ?? 'model.onnx';
ttsModel = { vits: { model: p(onnx), tokens: p('tokens.txt'), dataDir: p('espeak-ng-data') } };
break;
}
default:
throw new Error(`Moteur TTS non supporté : ${model.engine}`);
}
const tts = new s.OfflineTts({ model: { ...ttsModel, numThreads: threads, provider: 'cpu', debug: 0 }, maxNumSentences: 1 });
synthesizers.set(model.id, tts);
return tts;
}
// ── audio helpers ──────────────────────────────────────────────────────────
function decodeWav(bytes: Uint8Array): { samples: Float32Array; sampleRate: number } {
const view = new DataView(bytes.buffer, bytes.byteOffset, bytes.byteLength);
if (String.fromCharCode(bytes[0], bytes[1], bytes[2], bytes[3]) !== 'RIFF' || String.fromCharCode(bytes[8], bytes[9], bytes[10], bytes[11]) !== 'WAVE') {
throw new Error('WAV invalide');
}
let offset = 12;
let sampleRate = 16000;
let channels = 1;
let bits = 16;
let format = 1; // 1 = PCM, 3 = IEEE float
let data: { start: number; length: number } | null = null;
while (offset + 8 <= bytes.byteLength) {
const id = String.fromCharCode(bytes[offset], bytes[offset + 1], bytes[offset + 2], bytes[offset + 3]);
const size = view.getUint32(offset + 4, true);
if (id === 'fmt ') {
format = view.getUint16(offset + 8, true);
channels = view.getUint16(offset + 10, true);
sampleRate = view.getUint32(offset + 12, true);
bits = view.getUint16(offset + 22, true);
} else if (id === 'data') {
data = { start: offset + 8, length: Math.min(size, bytes.byteLength - offset - 8) };
break;
}
offset += 8 + size + (size % 2);
}
if (!data) throw new Error('WAV sans données');
const bytesPerSample = bits / 8;
const frames = Math.floor(data.length / bytesPerSample / channels);
const samples = new Float32Array(frames);
for (let i = 0; i < frames; i++) {
let sum = 0;
for (let c = 0; c < channels; c++) {
const pos = data.start + (i * channels + c) * bytesPerSample;
sum +=
format === 3 && bits === 32
? view.getFloat32(pos, true)
: bits === 16
? view.getInt16(pos, true) / 32768
: bits === 24
? (((bytes[pos] | (bytes[pos + 1] << 8) | (bytes[pos + 2] << 16)) << 8) >> 8) / 8388608
: bits === 32
? view.getInt32(pos, true) / 2147483648
: (bytes[pos] - 128) / 128;
}
samples[i] = sum / channels;
}
return { samples, sampleRate };
}
function resampleTo16k(samples: Float32Array, rate: number): Float32Array {
if (rate === 16000) return samples;
const ratio = rate / 16000;
const out = new Float32Array(Math.round(samples.length / ratio));
for (let i = 0; i < out.length; i++) {
const pos = i * ratio;
const idx = Math.floor(pos);
const frac = pos - idx;
const a = samples[Math.min(idx, samples.length - 1)];
const b = samples[Math.min(idx + 1, samples.length - 1)];
out[i] = a + (b - a) * frac;
}
return out;
}
function encodeWav(samples: Float32Array, sampleRate: number): Uint8Array {
const buffer = new ArrayBuffer(44 + samples.length * 2);
const view = new DataView(buffer);
const str = (o: number, s: string) => {
for (let i = 0; i < s.length; i++) view.setUint8(o + i, s.charCodeAt(i));
};
str(0, 'RIFF');
view.setUint32(4, 36 + samples.length * 2, true);
str(8, 'WAVE');
str(12, 'fmt ');
view.setUint32(16, 16, true);
view.setUint16(20, 1, true);
view.setUint16(22, 1, true);
view.setUint32(24, sampleRate, true);
view.setUint32(28, sampleRate * 2, true);
view.setUint16(32, 2, true);
view.setUint16(34, 16, true);
str(36, 'data');
view.setUint32(40, samples.length * 2, true);
let o = 44;
for (let i = 0; i < samples.length; i++, o += 2) {
const s = Math.max(-1, Math.min(1, samples[i]));
view.setInt16(o, s < 0 ? s * 0x8000 : s * 0x7fff, true);
}
return new Uint8Array(buffer);
}
function startKws(model: ModelRef, keywordsFile: string, threshold: number, score: number): void {
const s = loadSherpa();
kws = null;
const p = (f: string) => path.join(model.dir, f);
const enc = model.files.find((f) => f.startsWith('encoder')) ?? 'encoder.int8.onnx';
const dec = model.files.find((f) => f.startsWith('decoder')) ?? 'decoder.int8.onnx';
const join = model.files.find((f) => f.startsWith('joiner')) ?? 'joiner.int8.onnx';
const spotter = new s.KeywordSpotter({
featConfig: { sampleRate: 16000, featureDim: 80 },
modelConfig: { transducer: { encoder: p(enc), decoder: p(dec), joiner: p(join) }, tokens: p('tokens.txt'), numThreads: 1, provider: 'cpu', debug: 0 },
maxActivePaths: 4,
numTrailingBlanks: 1,
keywordsScore: score,
keywordsThreshold: threshold,
keywordsFile
});
kws = { spotter, stream: spotter.createStream() };
}
function feedKws(pcmBase64: string, sampleRate: number): void {
if (!kws) return;
const bytes = Buffer.from(pcmBase64, 'base64');
const int16 = new Int16Array(bytes.buffer, bytes.byteOffset, Math.floor(bytes.byteLength / 2));
let samples: Float32Array = new Float32Array(int16.length);
for (let i = 0; i < int16.length; i++) samples[i] = int16[i] / 32768;
if (sampleRate !== 16000) samples = resampleTo16k(samples, sampleRate);
kws.stream.acceptWaveform({ sampleRate: 16000, samples });
while (kws.spotter.isReady(kws.stream)) {
kws.spotter.decode(kws.stream);
const result = kws.spotter.getResult(kws.stream);
if (result.keyword) {
kws.spotter.reset(kws.stream);
notify?.({ type: 'kws.detected', keyword: result.keyword, at: Date.now() });
}
}
}
function startVad(model: ModelRef, silenceMs: number, threshold: number, maxUtteranceSec: number): void {
const s = loadSherpa();
const windowSize = 512;
const detector = new s.Vad(
{
sileroVad: {
model: path.join(model.dir, 'silero_vad.onnx'),
threshold: Math.max(0.1, Math.min(0.95, threshold)),
minSilenceDuration: Math.max(0.15, silenceMs / 1000),
minSpeechDuration: 0.2,
windowSize,
maxSpeechDuration: Math.max(3, maxUtteranceSec)
},
sampleRate: 16000,
numThreads: 1,
provider: 'cpu',
debug: 0
},
Math.max(10, maxUtteranceSec + 5)
);
vad = { detector, speaking: false, windowSize, carry: new Float32Array(0) };
}
function feedVad(pcmBase64: string, sampleRate: number): void {
if (!vad) return;
const bytes = Buffer.from(pcmBase64, 'base64');
const int16 = new Int16Array(bytes.buffer, bytes.byteOffset, Math.floor(bytes.byteLength / 2));
let samples: Float32Array = new Float32Array(int16.length);
for (let i = 0; i < int16.length; i++) samples[i] = int16[i] / 32768;
if (sampleRate !== 16000) samples = resampleTo16k(samples, sampleRate);
// Silero expects fixed windows: keep the remainder for the next frame.
const merged = new Float32Array(vad.carry.length + samples.length);
merged.set(vad.carry, 0);
merged.set(samples, vad.carry.length);
const usable = merged.length - (merged.length % vad.windowSize);
for (let i = 0; i < usable; i += vad.windowSize) {
vad.detector.acceptWaveform(merged.subarray(i, i + vad.windowSize));
const detected = vad.detector.isDetected();
if (detected && !vad.speaking) {
vad.speaking = true;
notify?.({ type: 'vad.event', event: { type: 'speech-start' } });
}
while (!vad.detector.isEmpty()) {
const segment = vad.detector.front(false);
vad.detector.pop();
vad.speaking = false;
const wav = encodeWav(segment.samples, 16000);
notify?.({
type: 'vad.event',
event: { type: 'segment', wav: Buffer.from(wav.buffer, wav.byteOffset, wav.byteLength).toString('base64'), durationSec: segment.samples.length / 16000 }
});
}
}
vad.carry = merged.slice(usable);
}
// ── request handling ───────────────────────────────────────────────────────
function handle(req: Request): unknown {
switch (req.type) {
case 'vad.start':
startVad(req.model, req.silenceMs, req.threshold, req.maxUtteranceSec);
return { ok: true };
case 'vad.audio':
feedVad(req.pcm, req.sampleRate);
return { ok: true };
case 'vad.stop':
vad = null;
return { ok: true };
case 'kws.start':
startKws(req.model, req.keywordsFile, req.threshold, req.score);
return { ok: true };
case 'kws.audio':
feedKws(req.pcm, req.sampleRate);
return { ok: true };
case 'kws.stop':
kws = null;
return { ok: true };
case 'status': {
try {
const s = loadSherpa();
return { available: true, version: s.version, loaded: [...recognizers.keys(), ...synthesizers.keys(), ...(kws ? ['kws'] : [])] };
} catch (err) {
return { available: false, error: (err as Error).message, loaded: [] };
}
}
case 'transcribe': {
const started = Date.now();
// Audio crosses the process boundary as base64: V8 refuses to serialize external buffers.
const bytes = typeof req.wav === 'string' ? new Uint8Array(Buffer.from(req.wav, 'base64')) : req.wav;
const { samples, sampleRate } = decodeWav(bytes);
const raw = resampleTo16k(samples, sampleRate);
if (raw.length < 1600) throw new Error('Audio trop court');
// 0.4 s of silence on both sides: utterances cut close to the words lose the first/last syllable
// and short clips make Whisper hallucinate.
const pad = 6400;
const pcm = new Float32Array(raw.length + 2 * pad);
pcm.set(raw, pad);
const recognizer = getRecognizer(req.model, req.language);
const stream = recognizer.createStream();
stream.acceptWaveform({ sampleRate: 16000, samples: pcm });
recognizer.decode(stream);
const result = recognizer.getResult(stream);
return { text: (result.text ?? '').trim(), language: result.lang, durationMs: Date.now() - started, audioSec: pcm.length / 16000 };
}
case 'synthesize': {
const started = Date.now();
const tts = getSynthesizer(req.model);
const sid = Math.max(0, Math.min(tts.numSpeakers - 1, Math.floor(req.speaker)));
// Electron forbids N-API external buffers: ask sherpa-onnx to copy the samples into a V8 buffer.
const audio = tts.generate({ text: req.text, sid, speed: Math.max(0.5, Math.min(2, req.speed || 1)), enableExternalBuffer: false });
const wav = encodeWav(audio.samples, audio.sampleRate);
return {
wav: Buffer.from(wav.buffer, wav.byteOffset, wav.byteLength).toString('base64'),
sampleRate: audio.sampleRate,
durationMs: Date.now() - started,
audioSec: audio.samples.length / audio.sampleRate
};
}
case 'unload': {
if (req.modelId) {
for (const k of [...recognizers.keys()]) if (k.startsWith(`${req.modelId}:`)) recognizers.delete(k);
synthesizers.delete(req.modelId);
} else {
recognizers.clear();
synthesizers.clear();
kws = null;
vad = null;
}
return { ok: true };
}
default:
throw new Error('requête inconnue');
}
}
function respond(req: Request): Response {
try {
return { id: req.id, ok: true, result: handle(req) };
} catch (err) {
return { id: req.id, ok: false, error: (err as Error).message || String(err) };
}
}
// Electron utility process transport, with a child_process fallback for tests.
const parentPort = (process as unknown as { parentPort?: { on: (ev: 'message', cb: (e: { data: Request }) => void) => void; postMessage: (m: unknown) => void } }).parentPort;
if (parentPort) {
notify = (m) => parentPort.postMessage(m);
parentPort.on('message', (event) => parentPort.postMessage(respond(event.data)));
} else if (process.send) {
notify = (m) => process.send!(m);
process.on('message', (msg: Request) => process.send!(respond(msg)));
}
+35 -12
View File
@@ -8,6 +8,7 @@ import type { BrowserWindow } from 'electron';
import { IPC, type WebhookStatus } from '../shared/ipc'; import { IPC, type WebhookStatus } from '../shared/ipc';
import { normalizeHermesPush } from '../shared/hermesPush'; import { normalizeHermesPush } from '../shared/hermesPush';
import { log } from './logger'; import { log } from './logger';
import { handleMcpHttp, MCP_PATH } from './mcp';
export const WEBHOOK_PATH = '/eveflow/hook'; export const WEBHOOK_PATH = '/eveflow/hook';
const MAX_BODY_BYTES = 2 * 1024 * 1024; const MAX_BODY_BYTES = 2 * 1024 * 1024;
@@ -35,16 +36,20 @@ export function getWebhookStatus(): WebhookStatus {
export function stopWebhookServer(): Promise<void> { export function stopWebhookServer(): Promise<void> {
return new Promise((resolve) => { return new Promise((resolve) => {
if (!server) return resolve(); if (!server) return resolve();
server.close(() => resolve()); const current = server;
server = null; server = null;
status = { ...status, listening: false }; status = { ...status, listening: false };
current.closeAllConnections();
current.close(() => resolve());
}); });
} }
export async function startWebhookServer(getWindow: () => BrowserWindow | null, options: WebhookOptions): Promise<WebhookStatus> { export async function startWebhookServer(getWindow: () => BrowserWindow | null, options: WebhookOptions): Promise<WebhookStatus> {
await stopWebhookServer(); await stopWebhookServer();
const { port, secret } = options; const { port, secret } = options;
const host = options.host ?? '0.0.0.0'; // Without a shared secret the hook only listens on loopback: anyone on the LAN could otherwise inject messages.
const host = options.host ?? (secret ? '0.0.0.0' : '127.0.0.1');
if (host !== '127.0.0.1' && !secret) log('WARN', 'webhook', `listening on ${host} without a secret`);
server = http.createServer((req, res) => { server = http.createServer((req, res) => {
const json = (code: number, payload: unknown) => { const json = (code: number, payload: unknown) => {
@@ -53,9 +58,10 @@ export async function startWebhookServer(getWindow: () => BrowserWindow | null,
}; };
if (req.method === 'GET' && (req.url === '/health' || req.url === '/')) { if (req.method === 'GET' && (req.url === '/health' || req.url === '/')) {
return json(200, { ok: true, app: 'eveflow', path: WEBHOOK_PATH }); return json(200, { ok: true, app: 'eveflow', path: WEBHOOK_PATH, mcp: MCP_PATH });
} }
if (req.method !== 'POST' || !req.url?.startsWith(WEBHOOK_PATH)) { const isMcp = !!req.url && (req.url === MCP_PATH || req.url.startsWith(`${MCP_PATH}?`));
if (!isMcp && (req.method !== 'POST' || !req.url?.startsWith(WEBHOOK_PATH))) {
return json(404, { error: 'Not Found' }); return json(404, { error: 'Not Found' });
} }
@@ -69,19 +75,31 @@ export async function startWebhookServer(getWindow: () => BrowserWindow | null,
} }
} }
let body = ''; const chunks: Buffer[] = [];
let bytes = 0;
let tooLarge = false; let tooLarge = false;
req.on('data', (chunk: Buffer) => { req.on('data', (chunk: Buffer) => {
if (tooLarge) return; if (tooLarge) return;
body += chunk.toString('utf8'); bytes += chunk.length;
if (body.length > MAX_BODY_BYTES) { if (bytes > MAX_BODY_BYTES) {
tooLarge = true; tooLarge = true;
json(413, { error: 'Payload too large' }); res.writeHead(413, { 'Content-Type': 'application/json', Connection: 'close' });
req.destroy(); res.end(JSON.stringify({ error: 'Payload too large' }), () => req.socket.destroy());
return;
} }
chunks.push(chunk);
}); });
req.on('end', () => { req.on('end', () => {
if (tooLarge) return; if (tooLarge) return;
// Concatenate before decoding so multibyte UTF-8 (accents, emoji) split across chunks stays intact.
const body = Buffer.concat(chunks).toString('utf8');
if (isMcp) {
handleMcpHttp(getWindow(), req, res, body).catch((err: Error) => {
log('ERROR', 'mcp', err.message);
if (!res.headersSent) json(500, { error: err.message });
});
return;
}
try { try {
const raw: unknown = body.trim() ? JSON.parse(body) : {}; const raw: unknown = body.trim() ? JSON.parse(body) : {};
const events = normalizeHermesPush(raw); const events = normalizeHermesPush(raw);
@@ -99,13 +117,18 @@ export async function startWebhookServer(getWindow: () => BrowserWindow | null,
}); });
return new Promise((resolve) => { return new Promise((resolve) => {
server!.once('error', (err: NodeJS.ErrnoException) => { const instance = server!;
instance.on('error', (err: NodeJS.ErrnoException) => {
log('ERROR', 'webhook', `server error: ${err.message}`); log('ERROR', 'webhook', `server error: ${err.message}`);
status = { listening: false, port, path: WEBHOOK_PATH, secretConfigured: !!secret, error: err.message }; status = { listening: false, port, path: WEBHOOK_PATH, secretConfigured: !!secret, error: err.message };
server = null; if (server === instance) server = null;
resolve(status); resolve(status);
}); });
server!.listen(port, host, () => { if (!Number.isInteger(port) || port < 1 || port > 65535) {
instance.emit('error', new Error(`port invalide : ${port}`));
return;
}
instance.listen(port, host, () => {
log('INFO', 'webhook', `listening on http://${host}:${port}${WEBHOOK_PATH}`); log('INFO', 'webhook', `listening on http://${host}:${port}${WEBHOOK_PATH}`);
status = { listening: true, port, path: WEBHOOK_PATH, secretConfigured: !!secret }; status = { listening: true, port, path: WEBHOOK_PATH, secretConfigured: !!secret };
resolve(status); resolve(status);
+20 -7
View File
@@ -1,4 +1,5 @@
import { app, BrowserWindow, screen, shell } from 'electron'; import { app, BrowserWindow, screen, shell } from 'electron';
import { EventEmitter } from 'node:events';
import fs from 'node:fs'; import fs from 'node:fs';
import path from 'node:path'; import path from 'node:path';
import { IPC, type WindowMode } from '../shared/ipc'; import { IPC, type WindowMode } from '../shared/ipc';
@@ -9,6 +10,9 @@ const COMPACT_SIZE = { width: 380, height: 620, minWidth: 320, minHeight: 420 };
let mainWindow: BrowserWindow | null = null; let mainWindow: BrowserWindow | null = null;
let currentMode: WindowMode = 'hud'; let currentMode: WindowMode = 'hud';
export const windowEvents = new EventEmitter();
// Transparent frameless windows cannot be resized reliably on Windows; use an opaque, rounded window there.
const TRANSPARENT = process.platform !== 'win32';
let hudBounds: Electron.Rectangle | null = null; let hudBounds: Electron.Rectangle | null = null;
export function getMainWindow(): BrowserWindow | null { export function getMainWindow(): BrowserWindow | null {
@@ -37,7 +41,7 @@ async function loadRenderer(win: BrowserWindow): Promise<void> {
log('WARN', 'window', `dev server unreachable (${(err as Error).message}), loading built bundle`); log('WARN', 'window', `dev server unreachable (${(err as Error).message}), loading built bundle`);
} }
} }
const builtIndex = path.join(__dirname, '..', '..', 'dist', 'index.html'); const builtIndex = path.join(__dirname, '..', 'dist', 'index.html');
if (fs.existsSync(builtIndex)) { if (fs.existsSync(builtIndex)) {
await win.loadFile(builtIndex); await win.loadFile(builtIndex);
return; return;
@@ -49,8 +53,9 @@ export function createMainWindow(): BrowserWindow {
const win = new BrowserWindow({ const win = new BrowserWindow({
...HUD_SIZE, ...HUD_SIZE,
frame: false, frame: false,
transparent: true, transparent: TRANSPARENT,
backgroundColor: '#00000000', backgroundColor: TRANSPARENT ? '#00000000' : '#03070d',
roundedCorners: true,
hasShadow: true, hasShadow: true,
show: false, show: false,
title: 'EveFlow', title: 'EveFlow',
@@ -78,11 +83,18 @@ export function createMainWindow(): BrowserWindow {
return { action: 'deny' }; return { action: 'deny' };
}); });
win.webContents.on('will-navigate', (event, url) => { win.webContents.on('will-navigate', (event, url) => {
if (!url.startsWith('file://') && !url.startsWith(process.env.VITE_DEV_SERVER_URL ?? 'http://127.0.0.1:5173')) { if (url === win.webContents.getURL()) return;
event.preventDefault(); event.preventDefault();
void shell.openExternal(url); if (/^https?:\/\//i.test(url)) void shell.openExternal(url);
}
}); });
win.webContents.on('will-redirect', (event) => event.preventDefault());
const notifyVisibility = () => {
if (!win.isDestroyed()) win.webContents.send(IPC.windowVisibility, win.isVisible() && !win.isMinimized());
};
win.on('show', notifyVisibility);
win.on('hide', notifyVisibility);
win.on('minimize', notifyVisibility);
win.on('restore', notifyVisibility);
win.on('closed', () => { win.on('closed', () => {
mainWindow = null; mainWindow = null;
@@ -129,6 +141,7 @@ export function setWindowMode(mode: WindowMode): void {
} }
currentMode = mode; currentMode = mode;
win.webContents.send(IPC.windowModeChanged, mode); win.webContents.send(IPC.windowModeChanged, mode);
windowEvents.emit('mode', mode);
} }
export function toggleWindowVisibility(): void { export function toggleWindowVisibility(): void {
+1 -1
View File
@@ -3,7 +3,7 @@
<head> <head>
<meta charset="UTF-8" /> <meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" /> <meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta http-equiv="Content-Security-Policy" content="default-src 'self'; script-src 'self'; style-src 'self' 'unsafe-inline' https://fonts.googleapis.com; font-src 'self' https://fonts.gstatic.com data:; img-src 'self' data: blob: http: https:; media-src 'self' blob: data:; connect-src 'self' http: https: ws: wss: blob: data:; worker-src 'self' blob:;" /> <meta http-equiv="Content-Security-Policy" content="default-src 'self'; script-src 'self'; style-src 'self' 'unsafe-inline'; font-src 'self' data:; img-src 'self' data: blob: http: https:; media-src 'self' blob: data:; connect-src 'self' http: https: ws: wss: blob: data:; worker-src 'self' blob:;" />
<meta name="description" content="EveFlow — interface vocale JARVIS pour Hermes Agent." /> <meta name="description" content="EveFlow — interface vocale JARVIS pour Hermes Agent." />
<link rel="icon" type="image/svg+xml" href="data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 64 64'%3E%3Ccircle cx='32' cy='32' r='26' fill='none' stroke='%2334e4ff' stroke-width='4'/%3E%3Ccircle cx='32' cy='32' r='10' fill='%2334e4ff'/%3E%3C/svg%3E" /> <link rel="icon" type="image/svg+xml" href="data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 64 64'%3E%3Ccircle cx='32' cy='32' r='26' fill='none' stroke='%2334e4ff' stroke-width='4'/%3E%3Ccircle cx='32' cy='32' r='10' fill='%2334e4ff'/%3E%3C/svg%3E" />
<title>EveFlow</title> <title>EveFlow</title>
+697 -2
View File
@@ -1,32 +1,40 @@
{ {
"name": "eveflow", "name": "eveflow",
"version": "2.0.0", "version": "2.1.0",
"lockfileVersion": 3, "lockfileVersion": 3,
"requires": true, "requires": true,
"packages": { "packages": {
"": { "": {
"name": "eveflow", "name": "eveflow",
"version": "2.0.0", "version": "2.1.0",
"license": "MIT", "license": "MIT",
"dependencies": { "dependencies": {
"@fontsource/orbitron": "^5.3.0",
"@fontsource/rajdhani": "^5.3.0",
"@fontsource/share-tech-mono": "^5.3.0",
"lucide-react": "^1.40.0", "lucide-react": "^1.40.0",
"react": "^19.2.8", "react": "^19.2.8",
"react-dom": "^19.2.8", "react-dom": "^19.2.8",
"react-markdown": "^10.1.0", "react-markdown": "^10.1.0",
"remark-gfm": "^4.0.1", "remark-gfm": "^4.0.1",
"sherpa-onnx-node": "^1.13.7",
"zustand": "^5.0.15" "zustand": "^5.0.15"
}, },
"devDependencies": { "devDependencies": {
"@types/node": "^24.0.0", "@types/node": "^24.0.0",
"@types/react": "^19.2.18", "@types/react": "^19.2.18",
"@types/react-dom": "^19.2.0", "@types/react-dom": "^19.2.0",
"@types/unbzip2-stream": "^1.4.3",
"@vitejs/plugin-react": "^6.1.1", "@vitejs/plugin-react": "^6.1.1",
"concurrently": "^10.0.5", "concurrently": "^10.0.5",
"cross-env": "^10.0.0", "cross-env": "^10.0.0",
"electron": "^44.1.1", "electron": "^44.1.1",
"electron-builder": "^26.15.3", "electron-builder": "^26.15.3",
"esbuild": "^0.28.2",
"jsdom": "^30.0.1", "jsdom": "^30.0.1",
"tar": "^7.5.22",
"typescript": "^5.9.3", "typescript": "^5.9.3",
"unbzip2-stream": "^1.4.3",
"vite": "^8.2.2", "vite": "^8.2.2",
"vitest": "^5.0.0", "vitest": "^5.0.0",
"wait-on": "^9.1.0" "wait-on": "^9.1.0"
@@ -595,6 +603,448 @@
"dev": true, "dev": true,
"license": "MIT" "license": "MIT"
}, },
"node_modules/@esbuild/aix-ppc64": {
"version": "0.28.2",
"resolved": "https://registry.npmjs.org/@esbuild/aix-ppc64/-/aix-ppc64-0.28.2.tgz",
"integrity": "sha512-XExcO+dvLKvVtNTibSTBej1NCAbaGhWn9Ww1ZPx80qsahhPFe/8jgWP0IchNe0F3HwkU7n8ejhH8bjonqht8mQ==",
"cpu": [
"ppc64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"aix"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/android-arm": {
"version": "0.28.2",
"resolved": "https://registry.npmjs.org/@esbuild/android-arm/-/android-arm-0.28.2.tgz",
"integrity": "sha512-kXXoiPVVGQcnIYGOeaovwOURpniDBpSq4A03qkQ+BMQqtGG6HYap3xne9C1O1yo4TR3qxlCX5IqqmX6fFo2Lqg==",
"cpu": [
"arm"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"android"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/android-arm64": {
"version": "0.28.2",
"resolved": "https://registry.npmjs.org/@esbuild/android-arm64/-/android-arm64-0.28.2.tgz",
"integrity": "sha512-5YfKeeI8qWfBZIX+u2xZC3Zlb3Os/gLS2sbEKM+I4ZOcsWmHS2WLysCcQZDAFRslDUU5Oiq44gf6PYN1vGwG5A==",
"cpu": [
"arm64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"android"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/android-x64": {
"version": "0.28.2",
"resolved": "https://registry.npmjs.org/@esbuild/android-x64/-/android-x64-0.28.2.tgz",
"integrity": "sha512-O387ite7SzUyCcy3JQX4P4bLtEA7bLLkx+esve5JHnyYfNTxcVpXZo9jhdB0lTKN44gztELTdU7nS8Nr16Fs1Q==",
"cpu": [
"x64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"android"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/darwin-arm64": {
"version": "0.28.2",
"resolved": "https://registry.npmjs.org/@esbuild/darwin-arm64/-/darwin-arm64-0.28.2.tgz",
"integrity": "sha512-n4KqkOQrraxHJcgjM1RvwbigfQKIKJVpM7xp+KsxiyUSrRdIXnt73VhrPAx0fV44hgfmIVKjxMN9J1t5jySVkw==",
"cpu": [
"arm64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"darwin"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/darwin-x64": {
"version": "0.28.2",
"resolved": "https://registry.npmjs.org/@esbuild/darwin-x64/-/darwin-x64-0.28.2.tgz",
"integrity": "sha512-uq6suIWYP37qzGddBKPw5QEQPi6HiLGsO7UmkpfyaYNQ3D+rN6w6WfwH+nuqcGXWvawGwxOEroO4YGnFh95azw==",
"cpu": [
"x64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"darwin"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/freebsd-arm64": {
"version": "0.28.2",
"resolved": "https://registry.npmjs.org/@esbuild/freebsd-arm64/-/freebsd-arm64-0.28.2.tgz",
"integrity": "sha512-n+I0BTSRIoy+d6RPKnEVwql5UwBJolytvY4mAOIEJorKlqgPII8ix6slVVrfZ5Tnj7glIZvloylbB/EJPMWEXw==",
"cpu": [
"arm64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"freebsd"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/freebsd-x64": {
"version": "0.28.2",
"resolved": "https://registry.npmjs.org/@esbuild/freebsd-x64/-/freebsd-x64-0.28.2.tgz",
"integrity": "sha512-78XJTJkvPs0kz2w61301PJjXl4g7q3JqiYMZ/M/yVI73EHBrCRTgkhu9oqG7vPqq+a/yadEW8aD+agKlk5xrmg==",
"cpu": [
"x64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"freebsd"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/linux-arm": {
"version": "0.28.2",
"resolved": "https://registry.npmjs.org/@esbuild/linux-arm/-/linux-arm-0.28.2.tgz",
"integrity": "sha512-XlDnu2q5yoqems+xay6wSAcg9DDD7K9RLKZEBOMZm3ckNpJBvOX20tSfby8KfrrhINDyv9V2YVZKY/SpoGJI8w==",
"cpu": [
"arm"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/linux-arm64": {
"version": "0.28.2",
"resolved": "https://registry.npmjs.org/@esbuild/linux-arm64/-/linux-arm64-0.28.2.tgz",
"integrity": "sha512-pW4AC0P3it8c7do9MVM4p51FzHzdM/TZrerurgRcHJ2WTa1VQ1CIq18xncfpBJw4ojkiZZrKW2yIBWBP92j6Ug==",
"cpu": [
"arm64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/linux-ia32": {
"version": "0.28.2",
"resolved": "https://registry.npmjs.org/@esbuild/linux-ia32/-/linux-ia32-0.28.2.tgz",
"integrity": "sha512-CYbnj78HsIeA+DhgUKgFCfvNsTHFhMMrinUrMZpDXJXKN8T3XViTZ/+wtHeVxEWY8ewSzTFN+nRmSwO2tZaLUQ==",
"cpu": [
"ia32"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/linux-loong64": {
"version": "0.28.2",
"resolved": "https://registry.npmjs.org/@esbuild/linux-loong64/-/linux-loong64-0.28.2.tgz",
"integrity": "sha512-buwkd8nsph4R+ajRvw0qM5Hja/TXQow3ptzWO2EbG/cqcIkHloRrdlBtQlshyYGTNFvfkfJ5tpPLVkY4DtsPfQ==",
"cpu": [
"loong64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/linux-mips64el": {
"version": "0.28.2",
"resolved": "https://registry.npmjs.org/@esbuild/linux-mips64el/-/linux-mips64el-0.28.2.tgz",
"integrity": "sha512-ZVykbDyk7519VwiNb9Lcj9m8XM6v5V9uKPvrEMkkEedVewf+0itkhahp4HDpgERXhwLRpWFypsGbG/J8s0QjJA==",
"cpu": [
"mips64el"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/linux-ppc64": {
"version": "0.28.2",
"resolved": "https://registry.npmjs.org/@esbuild/linux-ppc64/-/linux-ppc64-0.28.2.tgz",
"integrity": "sha512-CAXl+Dtd9UUuJd8pKKdwh6MLm3MUMiqMPmhZ3tTSXPqfyQ3vDl6R5hZdZ/kYojK4ofXtdfSv1tFq8XzWx3heNQ==",
"cpu": [
"ppc64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/linux-riscv64": {
"version": "0.28.2",
"resolved": "https://registry.npmjs.org/@esbuild/linux-riscv64/-/linux-riscv64-0.28.2.tgz",
"integrity": "sha512-GeXCej4IQtU1B+QlDV8W/RRvbzI3O/Stss+/bCXv4lZls5WGRtu2a+3JkA3i4qIUlMXpcHebWpF8AkJhATowuA==",
"cpu": [
"riscv64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/linux-s390x": {
"version": "0.28.2",
"resolved": "https://registry.npmjs.org/@esbuild/linux-s390x/-/linux-s390x-0.28.2.tgz",
"integrity": "sha512-3H1weTYZPxt/WOhByszQZybS9w5lKzUn1FDMsgEChbHWQwHYQQRfBxgCcZvPhjHfKyJjIievvMmEUawJrdY9Dg==",
"cpu": [
"s390x"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/linux-x64": {
"version": "0.28.2",
"resolved": "https://registry.npmjs.org/@esbuild/linux-x64/-/linux-x64-0.28.2.tgz",
"integrity": "sha512-4xTZr1FUmSoQW4XIWmit3tzQrUTZM+N3P0XV8xROKYF50XfI7xeO90+1bZvNwxIufQ9hDQVRJH5YhgPVF8A/HQ==",
"cpu": [
"x64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/netbsd-arm64": {
"version": "0.28.2",
"resolved": "https://registry.npmjs.org/@esbuild/netbsd-arm64/-/netbsd-arm64-0.28.2.tgz",
"integrity": "sha512-sSATRjPeDBg3pdgHoQfoYBob11Kk1FGa9lui5RIHZCoCkJa9QKlvl3/vKz2usCmYYjs7ymJR/2Nnsqe+Hjt5nw==",
"cpu": [
"arm64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"netbsd"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/netbsd-x64": {
"version": "0.28.2",
"resolved": "https://registry.npmjs.org/@esbuild/netbsd-x64/-/netbsd-x64-0.28.2.tgz",
"integrity": "sha512-lqnzCV+mM0gIADaKihiCg6ifgfU2L3h5E33rNQBN1Y4MaVGnzryzmvvf7UHxprpQdE8hpqLolJ9Rl+SkIRDpyw==",
"cpu": [
"x64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"netbsd"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/openbsd-arm64": {
"version": "0.28.2",
"resolved": "https://registry.npmjs.org/@esbuild/openbsd-arm64/-/openbsd-arm64-0.28.2.tgz",
"integrity": "sha512-AL2qJILH7lNjrDmCQDvdxMfAUIv8KMNZOvrwAQ8i8//ntL9FflhOyMJ8OZSMBb8/AWXe3/5v5S20y3zCoZWKoQ==",
"cpu": [
"arm64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"openbsd"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/openbsd-x64": {
"version": "0.28.2",
"resolved": "https://registry.npmjs.org/@esbuild/openbsd-x64/-/openbsd-x64-0.28.2.tgz",
"integrity": "sha512-QtiuPytchRyC4rwUKhexJdQKvDuZ6hWloi3igqPQNUJCS1/v9EiO3UTOXR6A3FoMo4fnAKbWJdqaIwhOzh8qEw==",
"cpu": [
"x64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"openbsd"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/openharmony-arm64": {
"version": "0.28.2",
"resolved": "https://registry.npmjs.org/@esbuild/openharmony-arm64/-/openharmony-arm64-0.28.2.tgz",
"integrity": "sha512-WkhYDmpTjLvGlScA1rwjRUmhl4k8oXR3cIbtqWmELgU/dFeHHlEllxDvdWcNJV9rbzCexB5vz8gtNewWLgCT7Q==",
"cpu": [
"arm64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"openharmony"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/sunos-x64": {
"version": "0.28.2",
"resolved": "https://registry.npmjs.org/@esbuild/sunos-x64/-/sunos-x64-0.28.2.tgz",
"integrity": "sha512-GPMSkTOtMnv2U2F8gxe4Io6qmVs+YKyp832Etqqxr0hFngmXQ3rzwytelm3GIn7T4VviRUlf3sOgBOiTdvaf7g==",
"cpu": [
"x64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"sunos"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/win32-arm64": {
"version": "0.28.2",
"resolved": "https://registry.npmjs.org/@esbuild/win32-arm64/-/win32-arm64-0.28.2.tgz",
"integrity": "sha512-PIhhEkE9uPBleRBrQEJpUn7MBnibZzbGzYWPmY3x+YoVg/95zbjB4CxPPOQ8l5tYYM4mMaCthF8/1DIfBQQyWQ==",
"cpu": [
"arm64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"win32"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/win32-ia32": {
"version": "0.28.2",
"resolved": "https://registry.npmjs.org/@esbuild/win32-ia32/-/win32-ia32-0.28.2.tgz",
"integrity": "sha512-YmJbfTlvU7Sdn9BB+4PRES4oB6pxgS37MAONj+hBr/cpXS1aBPKXxNnDbu+QCWPj0o9dgyxeq79g6c5P8KeuYA==",
"cpu": [
"ia32"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"win32"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/win32-x64": {
"version": "0.28.2",
"resolved": "https://registry.npmjs.org/@esbuild/win32-x64/-/win32-x64-0.28.2.tgz",
"integrity": "sha512-5ebpxr3nWMzrL/rnUI755Jkuee0bHL/Gq0WTF9lvcpv73wAp5eu8MfBUgWK9bhWvZjj7yX8etf/8tI8Ney695g==",
"cpu": [
"x64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"win32"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@exodus/bytes": { "node_modules/@exodus/bytes": {
"version": "1.15.1", "version": "1.15.1",
"resolved": "https://registry.npmjs.org/@exodus/bytes/-/bytes-1.15.1.tgz", "resolved": "https://registry.npmjs.org/@exodus/bytes/-/bytes-1.15.1.tgz",
@@ -613,6 +1063,33 @@
} }
} }
}, },
"node_modules/@fontsource/orbitron": {
"version": "5.3.0",
"resolved": "https://registry.npmjs.org/@fontsource/orbitron/-/orbitron-5.3.0.tgz",
"integrity": "sha512-+BhPsbUAo/AF7sRcf+TEexm+EWWkgTiR9Zwb7AhYIPkc5oplfvw0C45Oc2xlKPhwVEmxG8hHuA6/Ansbl+Bc6w==",
"license": "OFL-1.1",
"funding": {
"url": "https://github.com/sponsors/ayuhito"
}
},
"node_modules/@fontsource/rajdhani": {
"version": "5.3.0",
"resolved": "https://registry.npmjs.org/@fontsource/rajdhani/-/rajdhani-5.3.0.tgz",
"integrity": "sha512-oBJgSIibWJvVBKseme1JczqPLoDHN97LHti4uh0LjfcJqQoVmnQQlRKIpEjTJV559rmVQ6qbVZZwKwk2wkBTtw==",
"license": "OFL-1.1",
"funding": {
"url": "https://github.com/sponsors/ayuhito"
}
},
"node_modules/@fontsource/share-tech-mono": {
"version": "5.3.0",
"resolved": "https://registry.npmjs.org/@fontsource/share-tech-mono/-/share-tech-mono-5.3.0.tgz",
"integrity": "sha512-hVEzVCzEid/fBfhggMyvkfDzn9ODrMihxC0EiOeQK3BuQNvdjmith2ERnoDNUkbW8zhufbKNuCgaSoT8LUNpuQ==",
"license": "OFL-1.1",
"funding": {
"url": "https://github.com/sponsors/ayuhito"
}
},
"node_modules/@hapi/address": { "node_modules/@hapi/address": {
"version": "5.1.1", "version": "5.1.1",
"resolved": "https://registry.npmjs.org/@hapi/address/-/address-5.1.1.tgz", "resolved": "https://registry.npmjs.org/@hapi/address/-/address-5.1.1.tgz",
@@ -1281,6 +1758,26 @@
"@types/node": "*" "@types/node": "*"
} }
}, },
"node_modules/@types/through": {
"version": "0.0.33",
"resolved": "https://registry.npmjs.org/@types/through/-/through-0.0.33.tgz",
"integrity": "sha512-HsJ+z3QuETzP3cswwtzt2vEIiHBk/dCcHGhbmG5X3ecnwFD/lPrMpliGXxSCg03L9AhrdwA4Oz/qfspkDW+xGQ==",
"dev": true,
"license": "MIT",
"dependencies": {
"@types/node": "*"
}
},
"node_modules/@types/unbzip2-stream": {
"version": "1.4.3",
"resolved": "https://registry.npmjs.org/@types/unbzip2-stream/-/unbzip2-stream-1.4.3.tgz",
"integrity": "sha512-D8X5uuJRISqc8YtwL8jNW2FpPdUOCYXbfD6zNROCTbVXK9nawucxh10tVXE3MPjnHdRA1LvB0zDxVya/lBsnYw==",
"dev": true,
"license": "MIT",
"dependencies": {
"@types/through": "*"
}
},
"node_modules/@types/unist": { "node_modules/@types/unist": {
"version": "3.0.3", "version": "3.0.3",
"resolved": "https://registry.npmjs.org/@types/unist/-/unist-3.0.3.tgz", "resolved": "https://registry.npmjs.org/@types/unist/-/unist-3.0.3.tgz",
@@ -1816,6 +2313,31 @@
"node": "20 || >=22" "node": "20 || >=22"
} }
}, },
"node_modules/buffer": {
"version": "5.7.1",
"resolved": "https://registry.npmjs.org/buffer/-/buffer-5.7.1.tgz",
"integrity": "sha512-EHcyIPBQ4BSGlvjB16k5KgAJ27CIsHY/2JBmCRReo48y9rQ3MaUzWX3KVlBa4U7MyX02HdVj0K7C3WaB3ju7FQ==",
"dev": true,
"funding": [
{
"type": "github",
"url": "https://github.com/sponsors/feross"
},
{
"type": "patreon",
"url": "https://www.patreon.com/feross"
},
{
"type": "consulting",
"url": "https://feross.org/support"
}
],
"license": "MIT",
"dependencies": {
"base64-js": "^1.3.1",
"ieee754": "^1.1.13"
}
},
"node_modules/buffer-from": { "node_modules/buffer-from": {
"version": "1.1.2", "version": "1.1.2",
"resolved": "https://registry.npmjs.org/buffer-from/-/buffer-from-1.1.2.tgz", "resolved": "https://registry.npmjs.org/buffer-from/-/buffer-from-1.1.2.tgz",
@@ -3027,6 +3549,48 @@
"license": "MIT", "license": "MIT",
"optional": true "optional": true
}, },
"node_modules/esbuild": {
"version": "0.28.2",
"resolved": "https://registry.npmjs.org/esbuild/-/esbuild-0.28.2.tgz",
"integrity": "sha512-HKVLS8dvII+xoKW9kmqxbRKrnWEXfJJr/FZhhJmiqIB0e053QNYFqOBouTMO/k5sID4MvCiUCvv8b9M4h32wIA==",
"dev": true,
"hasInstallScript": true,
"license": "MIT",
"bin": {
"esbuild": "bin/esbuild"
},
"engines": {
"node": ">=18"
},
"optionalDependencies": {
"@esbuild/aix-ppc64": "0.28.2",
"@esbuild/android-arm": "0.28.2",
"@esbuild/android-arm64": "0.28.2",
"@esbuild/android-x64": "0.28.2",
"@esbuild/darwin-arm64": "0.28.2",
"@esbuild/darwin-x64": "0.28.2",
"@esbuild/freebsd-arm64": "0.28.2",
"@esbuild/freebsd-x64": "0.28.2",
"@esbuild/linux-arm": "0.28.2",
"@esbuild/linux-arm64": "0.28.2",
"@esbuild/linux-ia32": "0.28.2",
"@esbuild/linux-loong64": "0.28.2",
"@esbuild/linux-mips64el": "0.28.2",
"@esbuild/linux-ppc64": "0.28.2",
"@esbuild/linux-riscv64": "0.28.2",
"@esbuild/linux-s390x": "0.28.2",
"@esbuild/linux-x64": "0.28.2",
"@esbuild/netbsd-arm64": "0.28.2",
"@esbuild/netbsd-x64": "0.28.2",
"@esbuild/openbsd-arm64": "0.28.2",
"@esbuild/openbsd-x64": "0.28.2",
"@esbuild/openharmony-arm64": "0.28.2",
"@esbuild/sunos-x64": "0.28.2",
"@esbuild/win32-arm64": "0.28.2",
"@esbuild/win32-ia32": "0.28.2",
"@esbuild/win32-x64": "0.28.2"
}
},
"node_modules/escalade": { "node_modules/escalade": {
"version": "3.2.0", "version": "3.2.0",
"resolved": "https://registry.npmjs.org/escalade/-/escalade-3.2.0.tgz", "resolved": "https://registry.npmjs.org/escalade/-/escalade-3.2.0.tgz",
@@ -3664,6 +4228,27 @@
"node": ">= 14" "node": ">= 14"
} }
}, },
"node_modules/ieee754": {
"version": "1.2.1",
"resolved": "https://registry.npmjs.org/ieee754/-/ieee754-1.2.1.tgz",
"integrity": "sha512-dcyqhDvX1C46lXZcVqCpK+FtMRQVdIMN6/Df5js2zouUsqG7I6sFxitIC+7KYK29KdXOLHdu9zL4sFnoVQnqaA==",
"dev": true,
"funding": [
{
"type": "github",
"url": "https://github.com/sponsors/feross"
},
{
"type": "patreon",
"url": "https://www.patreon.com/feross"
},
{
"type": "consulting",
"url": "https://feross.org/support"
}
],
"license": "BSD-3-Clause"
},
"node_modules/inflight": { "node_modules/inflight": {
"version": "1.0.6", "version": "1.0.6",
"resolved": "https://registry.npmjs.org/inflight/-/inflight-1.0.6.tgz", "resolved": "https://registry.npmjs.org/inflight/-/inflight-1.0.6.tgz",
@@ -6252,6 +6837,98 @@
"url": "https://github.com/sponsors/ljharb" "url": "https://github.com/sponsors/ljharb"
} }
}, },
"node_modules/sherpa-onnx-darwin-arm64": {
"version": "1.13.7",
"resolved": "https://registry.npmjs.org/sherpa-onnx-darwin-arm64/-/sherpa-onnx-darwin-arm64-1.13.7.tgz",
"integrity": "sha512-5NCE50hAvr3n2pdett0SgfPBJXaFZE0bqHwbHyiq+IKZ8Ids0l4M0VrG+ImGYIafCwie+oC3uAJ+pKj9xg/k+w==",
"cpu": [
"arm64"
],
"license": "Apache-2.0",
"optional": true,
"os": [
"darwin"
]
},
"node_modules/sherpa-onnx-darwin-x64": {
"version": "1.13.7",
"resolved": "https://registry.npmjs.org/sherpa-onnx-darwin-x64/-/sherpa-onnx-darwin-x64-1.13.7.tgz",
"integrity": "sha512-N3o+T+wn9WaQmsKV5DD8bTHdo+WN2+sXwmZcGJZiDjtOMR2zFz7uVCZnYCmEAMgvChC+oHcF5RvEEKcRCAu6Pw==",
"cpu": [
"x64"
],
"license": "Apache-2.0",
"optional": true,
"os": [
"darwin"
]
},
"node_modules/sherpa-onnx-linux-arm64": {
"version": "1.13.7",
"resolved": "https://registry.npmjs.org/sherpa-onnx-linux-arm64/-/sherpa-onnx-linux-arm64-1.13.7.tgz",
"integrity": "sha512-TFCVpXyTh69buhOtTS8KIfkRXOVKY4Y1qjAktSItrKS4A0chnnrlXO5bKWoNAPeI6fMxTF/uvMYbYgcvjEMfNg==",
"cpu": [
"arm64"
],
"license": "Apache-2.0",
"optional": true,
"os": [
"linux"
]
},
"node_modules/sherpa-onnx-linux-x64": {
"version": "1.13.7",
"resolved": "https://registry.npmjs.org/sherpa-onnx-linux-x64/-/sherpa-onnx-linux-x64-1.13.7.tgz",
"integrity": "sha512-npmxn5WwmAmlthgBhmbZ33t3i2j4mJwQt46dMEb3j7d41y1/uJrjrVAfa/DkvV+vn49ZWfcQ2UEWDipaZBVhuw==",
"cpu": [
"x64"
],
"license": "Apache-2.0",
"optional": true,
"os": [
"linux"
]
},
"node_modules/sherpa-onnx-node": {
"version": "1.13.7",
"resolved": "https://registry.npmjs.org/sherpa-onnx-node/-/sherpa-onnx-node-1.13.7.tgz",
"integrity": "sha512-0XGV7arGngBCnol0m8OLyqlnaUm19Q1KmetVj1DDBdymXa1upmAHZDwNdN47gjsEhqE5hXUEyc1vRQoXrNhNVg==",
"license": "Apache-2.0",
"optionalDependencies": {
"sherpa-onnx-darwin-arm64": "^1.13.7",
"sherpa-onnx-darwin-x64": "^1.13.7",
"sherpa-onnx-linux-arm64": "^1.13.7",
"sherpa-onnx-linux-x64": "^1.13.7",
"sherpa-onnx-win-ia32": "^1.13.7",
"sherpa-onnx-win-x64": "^1.13.7"
}
},
"node_modules/sherpa-onnx-win-ia32": {
"version": "1.13.7",
"resolved": "https://registry.npmjs.org/sherpa-onnx-win-ia32/-/sherpa-onnx-win-ia32-1.13.7.tgz",
"integrity": "sha512-sTwtpxPQ76XLn0giAbvknIDEDKD3XXi2mo2AVROEucf1pIK1DjQl+LjLkalTeFoQqbC4J3xGx/g+xgcHQD1dsw==",
"cpu": [
"ia32"
],
"license": "Apache-2.0",
"optional": true,
"os": [
"win32"
]
},
"node_modules/sherpa-onnx-win-x64": {
"version": "1.13.7",
"resolved": "https://registry.npmjs.org/sherpa-onnx-win-x64/-/sherpa-onnx-win-x64-1.13.7.tgz",
"integrity": "sha512-wBV1o+/zgsMrOjfCFIgGrH6S28xq6CqRCLSavCOjTZ6cqr80yGc07DUHxqsHFPZvfoJU+2JF5L2l3gyWFWoWdQ==",
"cpu": [
"x64"
],
"license": "Apache-2.0",
"optional": true,
"os": [
"win32"
]
},
"node_modules/siginfo": { "node_modules/siginfo": {
"version": "2.0.0", "version": "2.0.0",
"resolved": "https://registry.npmjs.org/siginfo/-/siginfo-2.0.0.tgz", "resolved": "https://registry.npmjs.org/siginfo/-/siginfo-2.0.0.tgz",
@@ -6514,6 +7191,13 @@
"fs-extra": "^10.0.0" "fs-extra": "^10.0.0"
} }
}, },
"node_modules/through": {
"version": "2.3.8",
"resolved": "https://registry.npmjs.org/through/-/through-2.3.8.tgz",
"integrity": "sha512-w89qg7PI8wAdvX60bMDP+bFoD5Dvhm9oLheFp5O4a2QF0cSBGsBX4qZmadPMvVqlLJBBci+WqGGOAPvcDeNSVg==",
"dev": true,
"license": "MIT"
},
"node_modules/tiny-async-pool": { "node_modules/tiny-async-pool": {
"version": "1.3.0", "version": "1.3.0",
"resolved": "https://registry.npmjs.org/tiny-async-pool/-/tiny-async-pool-1.3.0.tgz", "resolved": "https://registry.npmjs.org/tiny-async-pool/-/tiny-async-pool-1.3.0.tgz",
@@ -6712,6 +7396,17 @@
"node": ">=14.17" "node": ">=14.17"
} }
}, },
"node_modules/unbzip2-stream": {
"version": "1.4.3",
"resolved": "https://registry.npmjs.org/unbzip2-stream/-/unbzip2-stream-1.4.3.tgz",
"integrity": "sha512-mlExGW4w71ebDJviH16lQLtZS32VKqsSfk80GCfUlwT/4/hNRFsoscrF/c++9xinkMzECL1uL9DDwXqFWkruPg==",
"dev": true,
"license": "MIT",
"dependencies": {
"buffer": "^5.2.1",
"through": "^2.3.8"
}
},
"node_modules/undici": { "node_modules/undici": {
"version": "7.29.0", "version": "7.29.0",
"resolved": "https://registry.npmjs.org/undici/-/undici-7.29.0.tgz", "resolved": "https://registry.npmjs.org/undici/-/undici-7.29.0.tgz",
+24 -5
View File
@@ -1,22 +1,24 @@
{ {
"name": "eveflow", "name": "eveflow",
"version": "2.0.0", "version": "2.4.1",
"releaseVersion": "2.4.1",
"description": "JARVIS-style desktop HUD for Hermes Agent: voice, streaming runs, scheduled jobs, skills and telemetry", "description": "JARVIS-style desktop HUD for Hermes Agent: voice, streaming runs, scheduled jobs, skills and telemetry",
"main": "dist-electron/electron/main.js", "main": "dist-electron/main.js",
"private": true, "private": true,
"license": "MIT", "license": "MIT",
"scripts": { "scripts": {
"dev": "vite --host 127.0.0.1 --port 5173 --strictPort", "dev": "vite --host 127.0.0.1 --port 5173 --strictPort",
"build": "npm run typecheck && npm run build:renderer && npm run build:electron", "build": "npm run typecheck && npm run build:renderer && npm run build:electron",
"build:renderer": "vite build", "build:renderer": "vite build",
"build:electron": "tsc -p tsconfig.electron.json", "build:electron": "node scripts/build-electron.mjs",
"typecheck": "tsc -p tsconfig.json --noEmit && tsc -p tsconfig.electron.json --noEmit", "typecheck": "tsc -p tsconfig.json --noEmit && tsc -p tsconfig.electron.json --noEmit",
"test": "vitest run", "test": "vitest run",
"test:watch": "vitest", "test:watch": "vitest",
"electron": "electron .", "electron": "electron .",
"start": "npm run build:electron && concurrently -k -s first \"npm run dev\" \"wait-on http://127.0.0.1:5173 && cross-env NODE_ENV=development VITE_DEV_SERVER_URL=http://127.0.0.1:5173 electron .\"", "start": "npm run build:electron && concurrently -k -s first \"npm run dev\" \"wait-on http://127.0.0.1:5173 && cross-env NODE_ENV=development VITE_DEV_SERVER_URL=http://127.0.0.1:5173 electron .\"",
"preview": "vite preview", "preview": "vite preview",
"dist": "npm run build && electron-builder --win --publish never" "dist": "node scripts/dist.mjs",
"typecheck:electron": "tsc -p tsconfig.electron.json --noEmit"
}, },
"build": { "build": {
"appId": "com.romik3.eveflow", "appId": "com.romik3.eveflow",
@@ -51,27 +53,44 @@
"from": "build/icon.png", "from": "build/icon.png",
"to": "icon.png" "to": "icon.png"
} }
] ],
"asarUnpack": [
"dist-electron/voice-worker.js",
"node_modules/sherpa-onnx-node/**",
"node_modules/sherpa-onnx-win-x64/**",
"node_modules/sherpa-onnx-linux-x64/**",
"node_modules/sherpa-onnx-darwin-arm64/**",
"node_modules/sherpa-onnx-darwin-x64/**"
],
"npmRebuild": false
}, },
"dependencies": { "dependencies": {
"@fontsource/orbitron": "^5.3.0",
"@fontsource/rajdhani": "^5.3.0",
"@fontsource/share-tech-mono": "^5.3.0",
"lucide-react": "^1.40.0", "lucide-react": "^1.40.0",
"react": "^19.2.8", "react": "^19.2.8",
"react-dom": "^19.2.8", "react-dom": "^19.2.8",
"react-markdown": "^10.1.0", "react-markdown": "^10.1.0",
"remark-gfm": "^4.0.1", "remark-gfm": "^4.0.1",
"sherpa-onnx-node": "^1.13.7",
"zustand": "^5.0.15" "zustand": "^5.0.15"
}, },
"devDependencies": { "devDependencies": {
"@types/node": "^24.0.0", "@types/node": "^24.0.0",
"@types/react": "^19.2.18", "@types/react": "^19.2.18",
"@types/react-dom": "^19.2.0", "@types/react-dom": "^19.2.0",
"@types/unbzip2-stream": "^1.4.3",
"@vitejs/plugin-react": "^6.1.1", "@vitejs/plugin-react": "^6.1.1",
"concurrently": "^10.0.5", "concurrently": "^10.0.5",
"cross-env": "^10.0.0", "cross-env": "^10.0.0",
"electron": "^44.1.1", "electron": "^44.1.1",
"electron-builder": "^26.15.3", "electron-builder": "^26.15.3",
"esbuild": "^0.28.2",
"jsdom": "^30.0.1", "jsdom": "^30.0.1",
"tar": "^7.5.22",
"typescript": "^5.9.3", "typescript": "^5.9.3",
"unbzip2-stream": "^1.4.3",
"vite": "^8.2.2", "vite": "^8.2.2",
"vitest": "^5.0.0", "vitest": "^5.0.0",
"wait-on": "^9.1.0" "wait-on": "^9.1.0"
+27
View File
@@ -0,0 +1,27 @@
class EveFlowCaptureProcessor extends AudioWorkletProcessor {
constructor() {
super();
this.buffer = new Float32Array(2048);
this.offset = 0;
}
process(inputs) {
const input = inputs[0];
if (!input || input.length === 0) return true;
const channel = input[0];
if (!channel) return true;
let i = 0;
while (i < channel.length) {
const n = Math.min(channel.length - i, this.buffer.length - this.offset);
this.buffer.set(channel.subarray(i, i + n), this.offset);
this.offset += n;
i += n;
if (this.offset === this.buffer.length) {
this.port.postMessage(this.buffer, [this.buffer.buffer]);
this.buffer = new Float32Array(2048);
this.offset = 0;
}
}
return true;
}
}
registerProcessor('eveflow-capture', EveFlowCaptureProcessor);
+19
View File
@@ -0,0 +1,19 @@
class EveFlowWakeProcessor extends AudioWorkletProcessor {
constructor() { super(); this.buffer = new Float32Array(2048); this.offset = 0; }
process(inputs) {
const channel = inputs[0] && inputs[0][0];
if (!channel) return true;
let i = 0;
while (i < channel.length) {
const n = Math.min(channel.length - i, this.buffer.length - this.offset);
this.buffer.set(channel.subarray(i, i + n), this.offset);
this.offset += n; i += n;
if (this.offset === this.buffer.length) {
this.port.postMessage(this.buffer, [this.buffer.buffer]);
this.buffer = new Float32Array(2048); this.offset = 0;
}
}
return true;
}
}
registerProcessor('eveflow-wake', EveFlowWakeProcessor);
+24
View File
@@ -0,0 +1,24 @@
// Bundles the Electron main and preload scripts into single CommonJS files.
// The preload runs in a sandboxed renderer where `require()` of local modules is forbidden,
// so everything it imports must be inlined.
import { build } from 'esbuild';
import { rmSync } from 'node:fs';
rmSync('dist-electron', { recursive: true, force: true });
await build({
entryPoints: {
main: 'electron/main.ts',
preload: 'electron/preload.ts',
'voice-worker': 'electron/voice/worker.ts'
},
outdir: 'dist-electron',
bundle: true,
platform: 'node',
format: 'cjs',
target: 'node22',
// sherpa-onnx-node stays external: it is a native addon shipped unpacked from the asar.
external: ['electron', 'sherpa-onnx-node'],
sourcemap: false,
logLevel: 'info'
});
+21
View File
@@ -0,0 +1,21 @@
// Packages the app with electron-builder using the four-part release version
// (package.json "releaseVersion", e.g. 2.0.0.1). package.json "version" must stay semver.
import { spawnSync } from 'node:child_process';
import { existsSync, readFileSync } from 'node:fs';
const pkg = JSON.parse(readFileSync('package.json', 'utf8'));
const release = pkg.releaseVersion || pkg.version;
const build = release.split('.')[3] ?? '0';
const args = process.argv.slice(2);
if (!args.some((a) => a.startsWith('--win') || a.startsWith('--linux') || a.startsWith('--mac'))) args.unshift('--win');
// The Windows build must embed the Windows native addon (an optional dependency installed only on win32).
if (args.some((a) => a.startsWith('--win')) && !existsSync('node_modules/sherpa-onnx-win-x64/sherpa-onnx.node')) {
console.error('sherpa-onnx-win-x64 is not installed: build the Windows installer on Windows (npm ci) or install it with\n npm i --no-save --os=win32 --cpu=x64 sherpa-onnx-win-x64');
process.exit(1);
}
const result = spawnSync('npx', ['electron-builder', ...args, `--config.extraMetadata.version=${release}`], {
stdio: 'inherit',
shell: process.platform === 'win32',
env: { ...process.env, BUILD_NUMBER: process.env.BUILD_NUMBER ?? build }
});
process.exit(result.status ?? 1);
+62 -4
View File
@@ -11,6 +11,20 @@ import type {
WebhookStatus, WebhookStatus,
WindowMode WindowMode
} from './ipc'; } from './ipc';
import type { McpToolRequest, McpToolResponse } from './ipc';
import type {
KwsDetection,
KwsStartRequest,
VadEvent,
VadStartRequest,
SynthesizeRequest,
SynthesizeResult,
TranscribeRequest,
TranscribeResult,
VoiceDownloadProgress,
VoiceEngineStatus,
VoiceModelStatus
} from './voice';
export type Unsubscribe = () => void; export type Unsubscribe = () => void;
@@ -20,6 +34,8 @@ export interface EveFlowBridge {
control: (action: 'minimize' | 'maximize' | 'close' | 'hide') => void; control: (action: 'minimize' | 'maximize' | 'close' | 'hide') => void;
setMode: (mode: WindowMode) => void; setMode: (mode: WindowMode) => void;
onModeChanged: (cb: (mode: WindowMode) => void) => Unsubscribe; onModeChanged: (cb: (mode: WindowMode) => void) => Unsubscribe;
/** Fires when the window is shown/hidden/minimised (the Page Visibility API is disabled by backgroundThrottling:false). */
onVisibility: (cb: (visible: boolean) => void) => Unsubscribe;
}; };
log: (entry: LogEntry) => void; log: (entry: LogEntry) => void;
store: { store: {
@@ -33,10 +49,6 @@ export interface EveFlowBridge {
streamAbort: (id: string) => void; streamAbort: (id: string) => void;
onStreamEvent: (cb: (event: HttpStreamEvent) => void) => Unsubscribe; onStreamEvent: (cb: (event: HttpStreamEvent) => void) => Unsubscribe;
}; };
system: {
metrics: () => Promise<SystemMetrics>;
appInfo: () => Promise<AppInfo>;
};
files: { files: {
readLocal: (filePath: string) => Promise<string>; readLocal: (filePath: string) => Promise<string>;
writeShared: (filename: string, content: string, isBase64?: boolean) => Promise<{ path: string; url: string }>; writeShared: (filename: string, content: string, isBase64?: boolean) => Promise<{ path: string; url: string }>;
@@ -46,9 +58,55 @@ export interface EveFlowBridge {
hermes: { hermes: {
onPush: (cb: (event: HermesPushEvent) => void) => Unsubscribe; onPush: (cb: (event: HermesPushEvent) => void) => Unsubscribe;
webhookStatus: () => Promise<WebhookStatus>; webhookStatus: () => Promise<WebhookStatus>;
/** Tool calls from Hermes via the local MCP endpoint that need the renderer. */
onToolRequest: (cb: (req: McpToolRequest) => void) => Unsubscribe;
toolResponse: (res: McpToolResponse) => void;
webhookRestart: () => Promise<WebhookStatus>; webhookRestart: () => Promise<WebhookStatus>;
}; };
hotkeys: { hotkeys: {
on: (cb: (event: HotkeyEvent) => void) => Unsubscribe; on: (cb: (event: HotkeyEvent) => void) => Unsubscribe;
}; };
voice: {
status: () => Promise<VoiceEngineStatus>;
listModels: () => Promise<VoiceModelStatus[]>;
downloadModel: (id: string) => Promise<VoiceModelStatus>;
cancelDownload: (id: string) => Promise<boolean>;
removeModel: (id: string) => Promise<VoiceModelStatus[]>;
onProgress: (cb: (progress: VoiceDownloadProgress) => void) => Unsubscribe;
transcribe: (req: TranscribeRequest) => Promise<TranscribeResult>;
synthesize: (req: SynthesizeRequest) => Promise<SynthesizeResult>;
unload: (id?: string) => Promise<unknown>;
kwsStart: (req: KwsStartRequest) => Promise<{ accepted: string[]; rejected: string[] }>;
kwsStop: () => Promise<void>;
/** Fire-and-forget 16-bit PCM frames for the keyword spotter. */
kwsAudio: (pcm: Uint8Array, sampleRate: number) => void;
onKwsDetected: (cb: (detection: KwsDetection) => void) => Unsubscribe;
vadStart: (req: VadStartRequest) => Promise<void>;
vadStop: () => Promise<void>;
vadAudio: (pcm: Uint8Array, sampleRate: number) => void;
onVadEvent: (cb: (event: VadEvent) => void) => Unsubscribe;
};
system: {
metrics: () => Promise<SystemMetrics>;
appInfo: () => Promise<AppInfo>;
/** Screenshot of the primary display as a JPEG data URL. */
captureScreen: (maxWidth?: number) => Promise<string>;
/** Allow-listed local actions (lock session, open app/url, media keys, clipboard). */
action: (action: SystemAction) => Promise<SystemActionResult>;
};
}
export type SystemAction =
| { type: 'lock' }
| { type: 'open-app'; name: string }
| { type: 'open-url'; url: string }
| { type: 'media'; key: 'volume-up' | 'volume-down' | 'mute' | 'play-pause' | 'next' | 'previous' }
| { type: 'clipboard-read' }
| { type: 'clipboard-write'; text: string }
| { type: 'find-files'; query: string };
export interface SystemActionResult {
ok: boolean;
message?: string;
data?: unknown;
} }
+19
View File
@@ -100,6 +100,7 @@ export const IPC = {
windowControl: 'window:control', windowControl: 'window:control',
windowSetMode: 'window:set-mode', windowSetMode: 'window:set-mode',
windowModeChanged: 'window:mode-changed', windowModeChanged: 'window:mode-changed',
windowVisibility: 'window:visibility',
log: 'log:write', log: 'log:write',
storeGet: 'store:get', storeGet: 'store:get',
storeSet: 'store:set', storeSet: 'store:set',
@@ -110,6 +111,10 @@ export const IPC = {
httpStreamEvent: 'http:stream:event', httpStreamEvent: 'http:stream:event',
metrics: 'system:metrics', metrics: 'system:metrics',
appInfo: 'app:info', appInfo: 'app:info',
screenCapture: 'system:screen-capture',
systemAction: 'system:action',
mcpRequest: 'mcp:request',
mcpResponse: 'mcp:response',
readLocalFile: 'files:read-local', readLocalFile: 'files:read-local',
writeSharedFile: 'files:write-shared', writeSharedFile: 'files:write-shared',
openPath: 'files:open-path', openPath: 'files:open-path',
@@ -121,3 +126,17 @@ export const IPC = {
} as const; } as const;
export type HotkeyEvent = 'ptt-toggle' | 'toggle-window' | 'stop-speaking'; export type HotkeyEvent = 'ptt-toggle' | 'toggle-window' | 'stop-speaking';
/** A tool call arriving from Hermes through the local MCP endpoint that needs the renderer (UI, voice, chat). */
export interface McpToolRequest {
id: string;
name: string;
args: Record<string, unknown>;
}
export interface McpToolResponse {
id: string;
ok: boolean;
result?: unknown;
error?: string;
}
+108
View File
@@ -0,0 +1,108 @@
/**
* Encodes wake phrases into the BPE token sequences expected by the sherpa-onnx keyword
* spotter (gigaspeech BPE-500 model). Common phrases use sequences produced by the real
* SentencePiece model; anything else falls back to a greedy longest-match over the vocabulary,
* which is a close approximation for short words.
*/
const KNOWN: Record<string, string> = {
'JARVIS': '▁JA R VI S',
'HEY JARVIS': '▁HE Y ▁JA R VI S',
'OK JARVIS': '▁O K ▁JA R VI S',
'EVE': '▁E VE',
'HEY EVE': '▁HE Y ▁E VE',
'COMPUTER': '▁COMP U TER',
'HEY COMPUTER': '▁HE Y ▁COMP U TER',
'FRIDAY': '▁F RI DAY',
'HEY FRIDAY': '▁HE Y ▁F RI DAY',
'ALFRED': '▁A L F RE D',
'HERMES': '▁HER ME S',
'HEY HERMES': '▁HE Y ▁HER ME S',
'ASSISTANT': '▁AS S IST ANT',
'OK GOOGLE': '▁O K ▁GO O G LE',
'ALEXA': '▁A LE X A',
'SIRI': '▁S I RI',
'HEY SIRI': '▁HE Y ▁S I RI',
'NOVA': '▁NO V A',
'ATLAS': '▁AT LA S',
'HAL': '▁HA L'
};
export function normalizeKeyword(phrase: string): string {
return phrase
.normalize('NFD')
.replace(/[̀-ͯ]/g, '')
.toUpperCase()
.replace(/[^A-Z ]+/g, ' ')
.replace(/\s+/g, ' ')
.trim();
}
/** Parse a sherpa tokens.txt ("piece id" per line) into the set of pieces. */
export function parseTokens(tokensFile: string): Set<string> {
const pieces = new Set<string>();
for (const line of tokensFile.split(/\r?\n/)) {
const piece = line.trim().split(/\s+/)[0];
if (piece && !piece.startsWith('<')) pieces.add(piece);
}
return pieces;
}
function greedyWord(word: string, vocab: Set<string>): string[] | null {
const out: string[] = [];
let i = 0;
let first = true;
while (i < word.length) {
let matched = '';
for (let len = word.length - i; len >= 1; len--) {
const candidate = (first ? '▁' : '') + word.slice(i, i + len);
if (vocab.has(candidate)) {
matched = candidate;
break;
}
}
if (!matched && first) {
// No word-initial piece: use a bare "▁" if available, then continue without the prefix.
if (vocab.has('▁')) out.push('▁');
first = false;
continue;
}
if (!matched) return null;
out.push(matched);
i += matched.length - (first ? 1 : 0);
first = false;
}
return out;
}
/** Encode a phrase into space-separated BPE pieces, or null when it cannot be represented. */
export function encodeKeyword(phrase: string, vocab: Set<string>): string | null {
const normalized = normalizeKeyword(phrase);
if (!normalized) return null;
if (KNOWN[normalized]) return KNOWN[normalized];
const pieces: string[] = [];
for (const word of normalized.split(' ')) {
const encoded = greedyWord(word, vocab);
if (!encoded) return null;
pieces.push(...encoded);
}
return pieces.join(' ');
}
/** Build the keywords file content: one line per phrase, with the readable label. */
export function buildKeywordsFile(phrases: string[], vocab: Set<string>): { content: string; accepted: string[]; rejected: string[] } {
const lines: string[] = [];
const accepted: string[] = [];
const rejected: string[] = [];
for (const phrase of phrases) {
const encoded = encodeKeyword(phrase, vocab);
const label = normalizeKeyword(phrase).toLowerCase().replace(/ /g, '_');
if (!encoded || !label) {
rejected.push(phrase);
continue;
}
lines.push(`${encoded} @${label}`);
accepted.push(label);
}
return { content: lines.join('\n') + '\n', accepted, rejected };
}
+127
View File
@@ -0,0 +1,127 @@
/** Local voice engine contract (sherpa-onnx in a utility process). Shared by main and renderer. */
export type VoiceModelKind = 'stt' | 'tts' | 'kws' | 'vad';
export type VoiceEngineKind = 'whisper' | 'sense-voice' | 'nemo-transducer' | 'kokoro' | 'piper' | 'kws-transducer' | 'silero';
export interface VoiceSpeaker {
id: number;
name: string;
lang: string;
/** m = masculine, f = feminine (from the catalog label). */
gender?: 'm' | 'f';
}
export interface VoiceModelSpec {
id: string;
kind: VoiceModelKind;
engine: VoiceEngineKind;
name: string;
description: string;
languages: string[];
sizeMb: number;
url: string;
/** Folder created by the archive (files are referenced relative to it). */
dir: string;
/** Files that must exist once installed. */
files: string[];
speakers?: VoiceSpeaker[];
sampleRate?: number;
recommended?: boolean;
}
export interface VoiceModelStatus extends VoiceModelSpec {
installed: boolean;
downloading: boolean;
installedBytes: number;
}
export interface VoiceDownloadProgress {
id: string;
phase: 'download' | 'extract' | 'done' | 'error' | 'cancelled';
received: number;
total: number;
percent: number;
message?: string;
}
export interface TranscribeRequest {
modelId: string;
wav: Uint8Array;
language: string; // 'fr', 'en', 'auto'
}
export interface TranscribeResult {
text: string;
language?: string;
durationMs: number;
audioSec: number;
}
export interface SynthesizeRequest {
modelId: string;
text: string;
speaker: number;
speed: number;
}
export interface SynthesizeResult {
wav: Uint8Array;
sampleRate: number;
durationMs: number;
audioSec: number;
}
export interface KwsStartRequest {
modelId: string;
/** Wake phrases in plain text (e.g. "jarvis", "hey jarvis"). */
keywords: string[];
/** 1 (strict) .. 5 (eager) */
sensitivity: number;
}
export interface KwsDetection {
keyword: string;
at: number;
}
export interface VadStartRequest {
modelId: string;
/** Silence that ends an utterance, in ms. */
silenceMs: number;
/** Detection threshold 0..1 (0.5 default). */
threshold: number;
maxUtteranceSec: number;
}
export type VadEvent =
| { type: 'speech-start' }
| { type: 'segment'; wav: Uint8Array; durationSec: number }
| { type: 'error'; message: string };
export interface VoiceEngineStatus {
available: boolean;
error?: string;
version?: string;
loaded: string[];
modelsDir: string;
}
export const VOICE_IPC = {
status: 'voice:status',
modelsList: 'voice:models:list',
modelsDownload: 'voice:models:download',
modelsCancel: 'voice:models:cancel',
modelsRemove: 'voice:models:remove',
modelsProgress: 'voice:models:progress',
transcribe: 'voice:transcribe',
synthesize: 'voice:synthesize',
unload: 'voice:unload',
kwsStart: 'voice:kws:start',
kwsStop: 'voice:kws:stop',
kwsAudio: 'voice:kws:audio',
kwsDetected: 'voice:kws:detected',
vadStart: 'voice:vad:start',
vadStop: 'voice:vad:stop',
vadAudio: 'voice:vad:audio',
vadEvent: 'voice:vad:event'
} as const;
+71 -12
View File
@@ -1,15 +1,19 @@
import { useEffect, useState } from 'react'; import { useEffect, useRef, useState } from 'react';
import { AlertTriangle, X } from 'lucide-react'; import { AlertTriangle, X } from 'lucide-react';
import type { WindowMode } from '../shared/ipc'; import type { WindowMode } from '../shared/ipc';
import { bridge } from './lib/bridge'; import { bridge } from './lib/bridge';
import { Log } from './lib/log'; import { Log } from './lib/log';
import { handlePush, stopGeneration } from './services/conversation'; import { handlePush, localToolContext, stopGeneration } from './services/conversation';
import { initMcpBridge } from './services/mcpBridge';
import { ensurePreferredVoice } from './services/voice/voicePreference';
import { isQuietTime } from './lib/quietHours';
import { speech } from './services/voice/speech'; import { speech } from './services/voice/speech';
import { voiceController } from './services/voice/voiceController'; import { voiceController } from './services/voice/voiceController';
import { useChat } from './state/chat'; import { useChat } from './state/chat';
import { useHermes } from './state/hermes'; import { useHermes } from './state/hermes';
import { useSettings } from './state/settings'; import { useSettings } from './state/settings';
import { useVoice } from './state/voice'; import { useVoice } from './state/voice';
import { useVoiceModels } from './state/voiceModels';
import { ChatPanel } from './components/chat/ChatPanel'; import { ChatPanel } from './components/chat/ChatPanel';
import { PendingRequests } from './components/chat/PendingRequests'; import { PendingRequests } from './components/chat/PendingRequests';
import { CompactWidget } from './components/compact/CompactWidget'; import { CompactWidget } from './components/compact/CompactWidget';
@@ -36,8 +40,27 @@ function useBoot(): boolean {
void useHermes.getState().refreshWebhook(); void useHermes.getState().refreshWebhook();
const api = bridge(); const api = bridge();
if (!api && /Electron/i.test(navigator.userAgent)) {
useChat.getState().setError("Pont Electron indisponible : le script preload n'a pas été chargé. Réinstallez EveFlow ou consultez eveflow.log.");
Log.error('app', 'window.eveflow missing inside Electron');
}
if (api) { if (api) {
disposers.push(useVoiceModels.getState().subscribe());
void useVoiceModels.getState().refresh().then(() => ensurePreferredVoice());
if (settings.voice.wakeMode === 'kws') void voiceController.startWakeMode();
let lastWake = JSON.stringify([settings.voice.wakeMode, settings.voice.wakeWord, settings.voice.kwsSensitivity, settings.voice.micDeviceId]);
disposers.push(
useSettings.subscribe((s) => {
const v = s.settings.voice;
const key = JSON.stringify([v.wakeMode, v.wakeWord, v.kwsSensitivity, v.micDeviceId]);
if (key === lastWake) return;
lastWake = key;
if (v.wakeMode === 'kws') void voiceController.restartWakeMode();
else void voiceController.stopWakeMode();
})
);
disposers.push(api.hermes.onPush(handlePush)); disposers.push(api.hermes.onPush(handlePush));
initMcpBridge(localToolContext);
disposers.push( disposers.push(
api.hotkeys.on((event) => { api.hotkeys.on((event) => {
if (event === 'ptt-toggle') voiceController.toggle(); if (event === 'ptt-toggle') voiceController.toggle();
@@ -76,6 +99,24 @@ function useBoot(): boolean {
function useThemeSync(): void { function useThemeSync(): void {
const theme = useSettings((s) => s.settings.theme); const theme = useSettings((s) => s.settings.theme);
const reduce = useSettings((s) => s.settings.ui.reduceMotion); const reduce = useSettings((s) => s.settings.ui.reduceMotion);
// Quiet hours: computed every 30 s; dims the HUD (night theme) and silences pushes.
const notif = useSettings((s) => s.settings.notifications);
useEffect(() => {
const tick = () => {
const quiet = notif.quietEnabled && isQuietTime(notif.quietStart, notif.quietEnd);
useChat.getState().setQuiet(quiet);
document.documentElement.toggleAttribute('data-night', quiet && notif.nightTheme);
};
tick();
const timer = setInterval(tick, 30_000);
return () => clearInterval(timer);
}, [notif]);
useEffect(() => {
const onVisible = () => { if (!document.hidden) useChat.getState().markRead(); };
document.addEventListener('visibilitychange', onVisible);
return () => document.removeEventListener('visibilitychange', onVisible);
}, []);
useEffect(() => { useEffect(() => {
document.documentElement.dataset.theme = theme; document.documentElement.dataset.theme = theme;
document.documentElement.dataset.motion = reduce ? 'reduce' : 'full'; document.documentElement.dataset.motion = reduce ? 'reduce' : 'full';
@@ -92,15 +133,23 @@ function useWindowMode(): WindowMode {
return mode; return mode;
} }
function ErrorBanner() { /** Errors surface as a transient toast over the core instead of shifting the layout. */
function ErrorToast() {
const error = useChat((s) => s.error); const error = useChat((s) => s.error);
const setError = useChat((s) => s.setError); const setError = useChat((s) => s.setError);
useEffect(() => {
if (!error) return;
const timer = setTimeout(() => setError(null), 9000);
return () => clearTimeout(timer);
}, [error, setError]);
if (!error) return null; if (!error) return null;
return ( return (
<div className="error-banner"> <div className="toast-stack">
<AlertTriangle size={16} /> <div className="error-banner" role="alert">
<span>{error}</span> <AlertTriangle size={16} />
<button className="icon-btn" onClick={() => setError(null)}><X size={14} /></button> <span>{error}</span>
<button className="icon-btn" aria-label="Fermer" onClick={() => setError(null)}><X size={14} /></button>
</div>
</div> </div>
); );
} }
@@ -111,15 +160,25 @@ export default function App() {
const mode = useWindowMode(); const mode = useWindowMode();
const showTelemetry = useSettings((s) => s.settings.ui.showTelemetry); const showTelemetry = useSettings((s) => s.settings.ui.showTelemetry);
const [showSettings, setShowSettings] = useState(false); const [showSettings, setShowSettings] = useState(false);
const [showOps, setShowOps] = useState(false);
const opsBadge = useChat((s) => s.activity.filter((a) => a.status === 'running').length) + useHermes((s) => s.jobRuns.filter((r) => !r.read).length);
const showSettingsRef = useRef(false);
const showOpsRef = useRef(false);
showSettingsRef.current = showSettings;
showOpsRef.current = showOps;
useEffect(() => { useEffect(() => {
const onKey = (e: KeyboardEvent) => { const onKey = (e: KeyboardEvent) => {
if (e.key === 'Escape' && showSettings) setShowSettings(false); if (e.key === 'Escape') {
if (showSettingsRef.current) setShowSettings(false);
else if (showOpsRef.current) setShowOps(false);
}
if (e.key === ',' && (e.ctrlKey || e.metaKey)) setShowSettings((v) => !v); if (e.key === ',' && (e.ctrlKey || e.metaKey)) setShowSettings((v) => !v);
if (e.key.toLowerCase() === 'o' && (e.ctrlKey || e.metaKey) && e.shiftKey) setShowOps((v) => !v);
}; };
window.addEventListener('keydown', onKey); window.addEventListener('keydown', onKey);
return () => window.removeEventListener('keydown', onKey); return () => window.removeEventListener('keydown', onKey);
}, [showSettings]); }, []);
if (!loaded) { if (!loaded) {
return ( return (
@@ -140,22 +199,22 @@ export default function App() {
return ( return (
<div className="hud-root"> <div className="hud-root">
<TopBar onOpenSettings={() => setShowSettings(true)} /> <TopBar onOpenSettings={() => setShowSettings(true)} onToggleOps={() => setShowOps((v) => !v)} opsBadge={opsBadge} />
<div className="hud-body"> <div className="hud-body">
<div className="hud-column"> <div className="hud-column">
<ChatPanel /> <ChatPanel />
</div> </div>
<div className="hud-column"> <div className="hud-column">
<ErrorBanner />
<section className="panel bracket" style={{ flex: 1 }}> <section className="panel bracket" style={{ flex: 1 }}>
<CoreStage /> <CoreStage />
</section> </section>
</div> </div>
<div className="hud-column ops-column"> <div className={`hud-column ops-column${showOps ? ' open' : ''}`}>
<OpsPanel /> <OpsPanel />
{showTelemetry && <Telemetry />} {showTelemetry && <Telemetry />}
</div> </div>
</div> </div>
<ErrorToast />
{showSettings && <SettingsDrawer onClose={() => setShowSettings(false)} />} {showSettings && <SettingsDrawer onClose={() => setShowSettings(false)} />}
<PendingRequests /> <PendingRequests />
</div> </div>
+24 -6
View File
@@ -12,18 +12,36 @@ interface Props {
export function ChatPanel({ compact }: Props) { export function ChatPanel({ compact }: Props) {
const messages = useChat((s) => s.messages); const messages = useChat((s) => s.messages);
const clear = useChat((s) => s.clear); const clear = useChat((s) => s.clear);
const settings = useSettings((s) => s.settings); const assistantName = useSettings((s) => s.settings.assistantName);
const userName = useSettings((s) => s.settings.userName);
const showReasoning = useSettings((s) => s.settings.ui.showReasoning);
const listRef = useRef<HTMLDivElement>(null); const listRef = useRef<HTMLDivElement>(null);
const programmatic = useRef(false);
const [zoom, setZoom] = useState<string | null>(null); const [zoom, setZoom] = useState<string | null>(null);
const [stick, setStick] = useState(true); const [stick, setStick] = useState(true);
useEffect(() => { useEffect(() => {
if (!stick) return; if (!stick) return;
const el = listRef.current; const el = listRef.current;
if (el) el.scrollTop = el.scrollHeight; if (!el) return;
programmatic.current = true;
el.scrollTop = el.scrollHeight;
}, [messages, stick]); }, [messages, stick]);
useEffect(() => {
if (!zoom) return;
const onKey = (e: KeyboardEvent) => {
if (e.key === 'Escape') setZoom(null);
};
window.addEventListener('keydown', onKey);
return () => window.removeEventListener('keydown', onKey);
}, [zoom]);
const onScroll = () => { const onScroll = () => {
if (programmatic.current) {
programmatic.current = false;
return;
}
const el = listRef.current; const el = listRef.current;
if (!el) return; if (!el) return;
setStick(el.scrollHeight - el.scrollTop - el.clientHeight < 80); setStick(el.scrollHeight - el.scrollTop - el.clientHeight < 80);
@@ -47,7 +65,7 @@ export function ChatPanel({ compact }: Props) {
<div ref={listRef} className={`chat-list${compact ? ' compact' : ''}`} onScroll={onScroll}> <div ref={listRef} className={`chat-list${compact ? ' compact' : ''}`} onScroll={onScroll}>
{visible.length === 0 && ( {visible.length === 0 && (
<div className="empty"> <div className="empty">
{compact ? 'En attente…' : `Canal ouvert. Parlez ou écrivez à ${settings.assistantName}.`} {compact ? 'En attente…' : `Canal ouvert. Parlez ou écrivez à ${assistantName}.`}
{!compact && ( {!compact && (
<div className="muted" style={{ marginTop: 8, fontSize: 12 }}> <div className="muted" style={{ marginTop: 8, fontSize: 12 }}>
<span className="kbd">Ctrl</span> + <span className="kbd">Shift</span> + <span className="kbd">Espace</span> active le micro depuis n’importe où. <span className="kbd">Ctrl</span> + <span className="kbd">Shift</span> + <span className="kbd">Espace</span> active le micro depuis n’importe où.
@@ -59,9 +77,9 @@ export function ChatPanel({ compact }: Props) {
<MessageBubble <MessageBubble
key={m.id} key={m.id}
message={m} message={m}
assistantName={settings.assistantName} assistantName={assistantName}
userName={settings.userName} userName={userName}
showReasoning={settings.ui.showReasoning} showReasoning={showReasoning}
compact={compact} compact={compact}
onOpenImage={setZoom} onOpenImage={setZoom}
/> />
+43 -8
View File
@@ -1,6 +1,6 @@
import { useEffect, useRef, useState } from 'react'; import { useEffect, useRef, useState } from 'react';
import { Mic, Paperclip, Send, Square, X, Radio, Loader2, Volume2, VolumeX, Navigation } from 'lucide-react'; import { Mic, Paperclip, Send, Square, X, Radio, Loader2, Volume2, VolumeX, Navigation, ScanEye, Crosshair } from 'lucide-react';
import { sendMessage, steer, stopGeneration } from '../../services/conversation'; import { sendMessage, steer, stopGeneration , captureScreen } from '../../services/conversation';
import { speech } from '../../services/voice/speech'; import { speech } from '../../services/voice/speech';
import { voiceController } from '../../services/voice/voiceController'; import { voiceController } from '../../services/voice/voiceController';
import { useChat } from '../../state/chat'; import { useChat } from '../../state/chat';
@@ -50,6 +50,8 @@ export function CommandBar({ compact }: Props) {
const autoSpeak = useSettings((s) => s.settings.speech.autoSpeak); const autoSpeak = useSettings((s) => s.settings.speech.autoSpeak);
const update = useSettings((s) => s.update); const update = useSettings((s) => s.update);
const [images, setImages] = useState<string[]>([]); const [images, setImages] = useState<string[]>([]);
const missionMode = useChat((s) => s.missionMode);
const missionModel = useSettings((s) => s.settings.hermes.missionModel);
const [steerMode, setSteerMode] = useState(false); const [steerMode, setSteerMode] = useState(false);
const fileRef = useRef<HTMLInputElement>(null); const fileRef = useRef<HTMLInputElement>(null);
const inputRef = useRef<HTMLTextAreaElement>(null); const inputRef = useRef<HTMLTextAreaElement>(null);
@@ -67,17 +69,34 @@ export function CommandBar({ compact }: Props) {
const submit = async () => { const submit = async () => {
const text = draft.trim(); const text = draft.trim();
if (steerMode && currentRunId) { if (steerMode) {
if (!currentRunId) {
setSteerMode(false);
return;
}
if (await steer(text)) setDraft(''); if (await steer(text)) setDraft('');
inputRef.current?.focus();
return; return;
} }
if (!text && images.length === 0) return; if (!text && images.length === 0) return;
const attached = images; const attached = images;
setDraft(''); setDraft('');
setImages([]); setImages([]);
inputRef.current?.focus();
await sendMessage(text, attached); await sendMessage(text, attached);
}; };
useEffect(() => {
const onKey = (e: KeyboardEvent) => {
if ((e.ctrlKey || e.metaKey) && e.key.toLowerCase() === 'k') {
e.preventDefault();
inputRef.current?.focus();
}
};
window.addEventListener('keydown', onKey);
return () => window.removeEventListener('keydown', onKey);
}, []);
const onKeyDown = (e: React.KeyboardEvent<HTMLTextAreaElement>) => { const onKeyDown = (e: React.KeyboardEvent<HTMLTextAreaElement>) => {
if (e.key === 'Enter' && !e.shiftKey) { if (e.key === 'Enter' && !e.shiftKey) {
e.preventDefault(); e.preventDefault();
@@ -131,6 +150,8 @@ export function CommandBar({ compact }: Props) {
className={`mic-btn ${micClass}`} className={`mic-btn ${micClass}`}
style={{ ['--level' as string]: level }} style={{ ['--level' as string]: level }}
title={micTitle} title={micTitle}
aria-label={micTitle}
aria-pressed={listening}
onClick={() => voiceController.toggle()} onClick={() => voiceController.toggle()}
disabled={phase === 'transcribing'} disabled={phase === 'transcribing'}
> >
@@ -148,17 +169,28 @@ export function CommandBar({ compact }: Props) {
spellCheck={false} spellCheck={false}
/> />
{!compact && ( {!compact && (
<button className="icon-btn" title="Joindre une image" onClick={() => fileRef.current?.click()} style={{ height: 46, width: 40 }}> <>
<Paperclip size={18} /> <button
</button> className="icon-btn"
title="Joindre une capture de l’écran (Hermes la voit)"
aria-label="Capturer l’écran"
onClick={() => void captureScreen().then((shot) => { if (shot) setImages((prev) => [...prev, shot].slice(-4)); })}
style={{ height: 46, width: 40 }}
>
<ScanEye size={18} />
</button>
<button className="icon-btn" title="Joindre une image" onClick={() => fileRef.current?.click()} style={{ height: 46, width: 40 }}>
<Paperclip size={18} />
</button>
</>
)} )}
<input ref={fileRef} type="file" accept="image/*" multiple hidden onChange={(e) => void onFiles(e.target.files)} /> <input ref={fileRef} type="file" accept="image/*" multiple hidden onChange={(e) => void onFiles(e.target.files)} />
{isSending && !steerMode ? ( {isSending && !steerMode ? (
<button className="send-btn stop" title="Interrompre (Échap)" onClick={stopGeneration}> <button className="send-btn stop" title="Interrompre (Échap)" aria-label="Interrompre" onClick={stopGeneration}>
<Square size={18} /> <Square size={18} />
</button> </button>
) : ( ) : (
<button className="send-btn primary" title={steerMode ? 'Envoyer la consigne' : 'Envoyer (Entrée)'} onClick={() => void submit()} disabled={!draft.trim() && images.length === 0}> <button className="send-btn primary" title={steerMode ? 'Envoyer la consigne' : 'Envoyer (Entrée)'} aria-label={steerMode ? 'Envoyer la consigne' : 'Envoyer'} onClick={() => void submit()} disabled={!draft.trim() && images.length === 0}>
{steerMode ? <Navigation size={18} /> : <Send size={18} />} {steerMode ? <Navigation size={18} /> : <Send size={18} />}
</button> </button>
)} )}
@@ -168,6 +200,9 @@ export function CommandBar({ compact }: Props) {
<button className={`toggle-btn${handsFree ? ' on' : ''}`} onClick={() => voiceController.setHandsFree(!handsFree)} title="Écoute continue : le micro se réactive après chaque réponse"> <button className={`toggle-btn${handsFree ? ' on' : ''}`} onClick={() => voiceController.setHandsFree(!handsFree)} title="Écoute continue : le micro se réactive après chaque réponse">
<Radio size={12} /> mains libres <Radio size={12} /> mains libres
</button> </button>
<button className={`toggle-btn${missionMode ? ' on' : ''}`} onClick={() => useChat.getState().setMissionMode(!missionMode)} title={missionModel ? `Mode mission : modèle ${missionModel} pour les tâches longues` : 'Mode mission (définissez un modèle « mission » dans Paramètres → Hermes)'}>
<Crosshair size={14} /> Mission
</button>
<button className={`toggle-btn${autoSpeak && speechProvider !== 'off' ? ' on' : ''}`} onClick={() => update({ speech: { autoSpeak: !autoSpeak } })} title="Lire les réponses à voix haute"> <button className={`toggle-btn${autoSpeak && speechProvider !== 'off' ? ' on' : ''}`} onClick={() => update({ speech: { autoSpeak: !autoSpeak } })} title="Lire les réponses à voix haute">
{autoSpeak && speechProvider !== 'off' ? <Volume2 size={12} /> : <VolumeX size={12} />} voix {autoSpeak && speechProvider !== 'off' ? <Volume2 size={12} /> : <VolumeX size={12} />} voix
</button> </button>
+9 -4
View File
@@ -1,4 +1,4 @@
import { memo, useState } from 'react'; import { memo, useEffect, useRef, useState } from 'react';
import ReactMarkdown from 'react-markdown'; import ReactMarkdown from 'react-markdown';
import remarkGfm from 'remark-gfm'; import remarkGfm from 'remark-gfm';
import { Copy, Volume2, Check } from 'lucide-react'; import { Copy, Volume2, Check } from 'lucide-react';
@@ -28,6 +28,10 @@ function openLink(href: string): void {
export const MessageBubble = memo(function MessageBubble({ message, assistantName, userName, showReasoning, compact, onOpenImage }: Props) { export const MessageBubble = memo(function MessageBubble({ message, assistantName, userName, showReasoning, compact, onOpenImage }: Props) {
const [copied, setCopied] = useState(false); const [copied, setCopied] = useState(false);
const copyTimer = useRef<number | null>(null);
useEffect(() => () => {
if (copyTimer.current) window.clearTimeout(copyTimer.current);
}, []);
const isUser = message.role === 'user'; const isUser = message.role === 'user';
const who = isUser ? userName || 'VOUS' : message.role === 'system' ? 'SYSTÈME' : assistantName; const who = isUser ? userName || 'VOUS' : message.role === 'system' ? 'SYSTÈME' : assistantName;
const status = message.status ?? 'done'; const status = message.status ?? 'done';
@@ -35,7 +39,8 @@ export const MessageBubble = memo(function MessageBubble({ message, assistantNam
const copy = () => { const copy = () => {
navigator.clipboard.writeText(message.content).then(() => { navigator.clipboard.writeText(message.content).then(() => {
setCopied(true); setCopied(true);
setTimeout(() => setCopied(false), 1200); if (copyTimer.current) window.clearTimeout(copyTimer.current);
copyTimer.current = window.setTimeout(() => setCopied(false), 1200);
}).catch(() => undefined); }).catch(() => undefined);
}; };
@@ -50,8 +55,8 @@ export const MessageBubble = memo(function MessageBubble({ message, assistantNam
{message.usage?.total_tokens ? <span>{message.usage.total_tokens} tok</span> : null} {message.usage?.total_tokens ? <span>{message.usage.total_tokens} tok</span> : null}
{message.role === 'assistant' && status === 'done' && ( {message.role === 'assistant' && status === 'done' && (
<span className="msg-actions"> <span className="msg-actions">
<button className="icon-btn" title="Copier" onClick={copy}>{copied ? <Check size={13} /> : <Copy size={13} />}</button> <button className="icon-btn" title="Copier" aria-label="Copier le message" onClick={copy}>{copied ? <Check size={13} /> : <Copy size={13} />}</button>
<button className="icon-btn" title="Lire à voix haute" onClick={() => speech.say(message.content)}><Volume2 size={13} /></button> <button className="icon-btn" title="Lire à voix haute" aria-label="Lire à voix haute" onClick={() => speech.say(message.content)}><Volume2 size={13} /></button>
</span> </span>
)} )}
</div> </div>
+16 -4
View File
@@ -1,4 +1,4 @@
import { useState } from 'react'; import { useEffect, useState } from 'react';
import { ShieldAlert, HelpCircle, KeyRound } from 'lucide-react'; import { ShieldAlert, HelpCircle, KeyRound } from 'lucide-react';
import { resolvePending } from '../../services/conversation'; import { resolvePending } from '../../services/conversation';
import { useChat, type PendingRequest } from '../../state/chat'; import { useChat, type PendingRequest } from '../../state/chat';
@@ -7,10 +7,22 @@ function RequestCard({ request }: { request: PendingRequest }) {
const [answer, setAnswer] = useState(''); const [answer, setAnswer] = useState('');
const icon = request.kind === 'approval' ? <ShieldAlert size={16} /> : request.kind === 'clarify' ? <HelpCircle size={16} /> : <KeyRound size={16} />; const icon = request.kind === 'approval' ? <ShieldAlert size={16} /> : request.kind === 'clarify' ? <HelpCircle size={16} /> : <KeyRound size={16} />;
useEffect(() => {
const onKey = (e: KeyboardEvent) => {
if (e.key === 'Escape') {
e.stopPropagation();
if (request.kind === 'approval') void resolvePending(request, 'deny');
else useChat.getState().removePending(request.id);
}
};
window.addEventListener('keydown', onKey, true);
return () => window.removeEventListener('keydown', onKey, true);
}, [request]);
return ( return (
<div className="modal"> <div className="modal" role="dialog" aria-modal="true" aria-labelledby={`req-${request.id}`}>
<div className="request-card"> <div className="request-card">
<div className="title"> <div className="title" id={`req-${request.id}`}>
{icon} {icon}
{request.title} {request.title}
{request.tool && <span className="chip warn">{request.tool}</span>} {request.tool && <span className="chip warn">{request.tool}</span>}
@@ -20,7 +32,7 @@ function RequestCard({ request }: { request: PendingRequest }) {
{request.kind === 'approval' ? ( {request.kind === 'approval' ? (
<div className="actions"> <div className="actions">
<button className="btn danger" onClick={() => void resolvePending(request, 'deny')}>Refuser</button> <button className="btn danger" onClick={() => void resolvePending(request, 'deny')}>Refuser</button>
<button className="btn" onClick={() => void resolvePending(request, 'once')}>Autoriser une fois</button> <button className="btn" autoFocus onClick={() => void resolvePending(request, 'once')}>Autoriser une fois</button>
<button className="btn" onClick={() => void resolvePending(request, 'session')}>Pour la session</button> <button className="btn" onClick={() => void resolvePending(request, 'session')}>Pour la session</button>
<button className="btn primary" onClick={() => void resolvePending(request, 'always')}>Toujours</button> <button className="btn primary" onClick={() => void resolvePending(request, 'always')}>Toujours</button>
</div> </div>
+4
View File
@@ -40,6 +40,10 @@ export function ResilientImage({ src = '', alt, onOpen }: Props) {
setResolved(cache.get(src)!); setResolved(cache.get(src)!);
return; return;
} }
if (failures.has(src)) {
setError(failures.get(src)!);
return;
}
if (src.startsWith('data:') || src.startsWith('blob:') || /^https?:\/\//i.test(src)) { if (src.startsWith('data:') || src.startsWith('blob:') || /^https?:\/\//i.test(src)) {
ok(src); ok(src);
return; return;
+38 -4
View File
@@ -1,16 +1,41 @@
import { Maximize2, X, Minus } from 'lucide-react'; import { Maximize2, X, Minus, Moon, Crosshair } from 'lucide-react';
import { useShallow } from 'zustand/react/shallow';
import { bridge } from '../../lib/bridge'; import { bridge } from '../../lib/bridge';
import { useSettings } from '../../state/settings'; import { useSettings } from '../../state/settings';
import { useChat } from '../../state/chat';
import { useVoice } from '../../state/voice';
import { CoreStage } from '../hud/CoreStage'; import { CoreStage } from '../hud/CoreStage';
import { ChatPanel } from '../chat/ChatPanel'; import { ChatPanel } from '../chat/ChatPanel';
const STATE_LABEL: Record<string, string> = {
idle: 'veille',
listening: 'écoute',
thinking: 'réflexion',
speaking: 'parle',
alert: 'alerte',
error: 'erreur',
success: 'terminé'
};
/** Compact mode: a glanceable strip (state, unread, last sentence) above the mini chat. */
export function CompactWidget() { export function CompactWidget() {
const settings = useSettings((s) => s.settings); const assistantName = useSettings((s) => s.settings.assistantName);
const compactOpacity = useSettings((s) => s.settings.ui.compactOpacity);
const { hud, hudOverride, unread, missionMode, quiet, last } = useChat(
useShallow((s) => {
const lastAssistant = [...s.messages].reverse().find((m) => m.role === 'assistant' && m.content.trim());
return { hud: s.hud, hudOverride: s.hudOverride, unread: s.unread, missionMode: s.missionMode, quiet: s.quiet, last: lastAssistant?.content ?? '' };
})
);
const wake = useVoice((s) => s.wake);
const api = bridge(); const api = bridge();
const state = hudOverride ?? hud;
const label = state === 'idle' && wake === 'spotting' ? 'à l’écoute du mot-clé' : STATE_LABEL[state] ?? state;
const sentence = last.replace(/```[\s\S]*?```/g, ' ').replace(/[#*_>`|]/g, '').replace(/\s+/g, ' ').trim().slice(0, 140);
return ( return (
<div className="compact-root" style={{ ['--compact-opacity' as string]: settings.ui.compactOpacity }}> <div className="compact-root" style={{ ['--compact-opacity' as string]: compactOpacity }}>
<div className="compact-head"> <div className="compact-head">
<span className="title">{settings.assistantName}</span> <span className="title">{assistantName}</span>
{api && ( {api && (
<> <>
<button className="icon-btn" title="Mode HUD complet" onClick={() => api.window.setMode('hud')}><Maximize2 size={14} /></button> <button className="icon-btn" title="Mode HUD complet" onClick={() => api.window.setMode('hud')}><Maximize2 size={14} /></button>
@@ -20,6 +45,15 @@ export function CompactWidget() {
)} )}
</div> </div>
<CoreStage compact /> <CoreStage compact />
<div className="compact-glance" onClick={() => useChat.getState().markRead()} title={sentence}>
<span className={`state${state === 'alert' || state === 'error' ? ' alert' : ''}`}>{label}</span>
<span className="last">{sentence || 'Aucun message pour l’instant.'}</span>
<span className="badges">
{missionMode && <Crosshair size={12} className="mission" aria-label="mode mission" />}
{quiet && <Moon size={12} aria-label="heures calmes" />}
{unread > 0 && <span className="unread" aria-label={`${unread} non lus`}>{unread}</span>}
</span>
</div>
<div className="compact-body"> <div className="compact-body">
<ChatPanel compact /> <ChatPanel compact />
</div> </div>
+30 -7
View File
@@ -1,3 +1,4 @@
import { useEffect, useState } from 'react';
import { useChat } from '../../state/chat'; import { useChat } from '../../state/chat';
import { useHermes } from '../../state/hermes'; import { useHermes } from '../../state/hermes';
import { useSettings } from '../../state/settings'; import { useSettings } from '../../state/settings';
@@ -21,40 +22,62 @@ interface Props {
export function CoreStage({ compact }: Props) { export function CoreStage({ compact }: Props) {
const hud = useChat((s) => s.hudOverride ?? s.hud); const hud = useChat((s) => s.hudOverride ?? s.hud);
const activity = useChat((s) => s.activity); const running = useChat((s) => {
for (let i = s.activity.length - 1; i >= 0; i--) if (s.activity[i].status === 'running') return s.activity[i];
return undefined;
});
const isSending = useChat((s) => s.isSending); const isSending = useChat((s) => s.isSending);
const error = useChat((s) => s.error); const error = useChat((s) => s.error);
const pulse = useChat((s) => s.pingCount);
const latencyMs = useChat((s) => s.latencyMs);
const phase = useVoice((s) => s.phase); const phase = useVoice((s) => s.phase);
const level = useVoice((s) => s.inputLevel); const level = useVoice((s) => s.inputLevel);
const transcript = useVoice((s) => s.lastTranscript); const transcript = useVoice((s) => s.lastTranscript);
const interim = useVoice((s) => s.interim);
const voiceError = useVoice((s) => s.error); const voiceError = useVoice((s) => s.error);
const settings = useSettings((s) => s.settings); const wake = useVoice((s) => s.wake);
const wakeKeywords = useVoice((s) => s.wakeKeywords);
const assistantName = useSettings((s) => s.settings.assistantName);
const reduceMotion = useSettings((s) => s.settings.ui.reduceMotion);
const transport = useHermes((s) => s.transport); const transport = useHermes((s) => s.transport);
const link = useHermes((s) => s.link); const link = useHermes((s) => s.link);
const [elapsed, setElapsed] = useState(0);
useEffect(() => {
if (!isSending) {
setElapsed(0);
return;
}
const started = Date.now();
const timer = setInterval(() => setElapsed(Date.now() - started), 250);
return () => clearInterval(timer);
}, [isSending]);
const running = activity.filter((a) => a.status === 'running').slice(-1)[0];
let sub = ''; let sub = '';
if (hud === 'listening') sub = phase === 'speech' ? 'parole détectée…' : phase === 'transcribing' ? 'transcription…' : 'parlez maintenant'; if (hud === 'listening') sub = phase === 'speech' ? 'parole détectée…' : phase === 'transcribing' ? 'transcription…' : 'parlez maintenant';
else if (hud === 'thinking') sub = running ? `${running.name}${running.detail ? ` · ${previewText(running.detail, 60)}` : ''}` : isSending ? `Hermes · ${transport}` : ''; else if (hud === 'thinking') sub = running ? `${running.name}${running.detail ? ` · ${previewText(running.detail, 60)}` : ''}` : isSending ? `Hermes · ${transport} · ${(elapsed / 1000).toFixed(1)} s` : '';
else if (hud === 'error') sub = previewText(error ?? voiceError ?? 'erreur', 90); else if (hud === 'error') sub = previewText(error ?? voiceError ?? 'erreur', 90);
else if (hud === 'speaking') sub = 'synthèse vocale'; else if (hud === 'speaking') sub = 'synthèse vocale';
else if (transcript) sub = `« ${previewText(transcript, 80)} »`; else if (transcript) sub = `« ${previewText(transcript, 80)} »`;
else if (wake === 'spotting') sub = `dites « ${wakeKeywords[0] ?? 'jarvis'} »${link === 'online' ? ` · ${transport}` : link === 'offline' ? ' · Hermes hors ligne' : ''}`;
else sub = link === 'online' ? `liaison Hermes · ${transport}` : link === 'offline' ? 'Hermes hors ligne' : ''; else sub = link === 'online' ? `liaison Hermes · ${transport}` : link === 'offline' ? 'Hermes hors ligne' : '';
return ( return (
<div className={compact ? 'compact-core' : 'core-stage'}> <div className={compact ? 'compact-core' : 'core-stage'}>
{!compact && ( {!compact && (
<div className="core-name"> <div className="core-name">
{settings.assistantName} {assistantName}
<small>EVEFLOW · HERMES INTERFACE</small> <small>EVEFLOW · HERMES INTERFACE</small>
</div> </div>
)} )}
<div className={compact ? undefined : 'core-canvas-wrap'} style={compact ? { position: 'absolute', inset: '0 0 36px 0' } : undefined}> <div className={compact ? undefined : 'core-canvas-wrap'} style={compact ? { position: 'absolute', inset: '0 0 36px 0' } : undefined}>
<JarvisCore state={hud} inputLevel={level} reduceMotion={settings.ui.reduceMotion} /> <JarvisCore state={hud} inputLevel={level} reduceMotion={reduceMotion} pulse={pulse} />
</div> </div>
<div className="core-caption"> <div className="core-caption" aria-live="polite">
{!compact && (phase === 'speech' || phase === 'transcribing') && <div className="core-interim">{interim || (phase === 'transcribing' ? '…' : '')}</div>}
<div className={`core-state ${hud}`}>{STATE_LABEL[hud] ?? hud}</div> <div className={`core-state ${hud}`}>{STATE_LABEL[hud] ?? hud}</div>
{sub && <div className="core-sub">{sub}</div>} {sub && <div className="core-sub">{sub}</div>}
{!compact && latencyMs !== null && <div className="core-latency">premier mot · {latencyMs} ms</div>}
</div> </div>
</div> </div>
); );
+140 -62
View File
@@ -2,8 +2,13 @@
* JARVIS-style arc reactor core rendered on a 2D canvas. * JARVIS-style arc reactor core rendered on a 2D canvas.
* Rotating tick rings, segmented arcs and an audio-reactive radial spectrum that reads the * Rotating tick rings, segmented arcs and an audio-reactive radial spectrum that reads the
* real TTS output (or the microphone while listening). * real TTS output (or the microphone while listening).
*
* Performance notes: no canvas shadows (glow is a second wide low-alpha pass), colour strings
* are cached, the spectrum buffer is reused, the theme accent is read once per theme change,
* idle runs at 30 fps and the loop stops entirely while the window is hidden.
*/ */
import { useEffect, useRef } from 'react'; import { useEffect, useRef } from 'react';
import { bridge } from '../../lib/bridge';
import { audioBus } from '../../services/voice/audioBus'; import { audioBus } from '../../services/voice/audioBus';
import type { HudState } from '../../state/chat'; import type { HudState } from '../../state/chat';
@@ -12,12 +17,15 @@ interface Props {
/** 0..1 override for the microphone level (VAD), used while listening. */ /** 0..1 override for the microphone level (VAD), used while listening. */
inputLevel?: number; inputLevel?: number;
reduceMotion?: boolean; reduceMotion?: boolean;
/** Bumped by the parent to trigger a short "ping" flash (send, tool start, speech start). */
pulse?: number;
className?: string; className?: string;
} }
type Rgb = [number, number, number];
interface Palette { interface Palette {
main: [number, number, number]; main: Rgb;
glow: [number, number, number]; glow: Rgb;
} }
const PALETTES: Record<HudState, Palette> = { const PALETTES: Record<HudState, Palette> = {
@@ -31,23 +39,30 @@ const PALETTES: Record<HudState, Palette> = {
}; };
const BARS = 72; const BARS = 72;
const TICKS = 120;
const SEGMENTS = 6;
function readThemeAccent(): [number, number, number] | null { function readThemeAccent(): Rgb | null {
const raw = getComputedStyle(document.documentElement).getPropertyValue('--accent-rgb').trim(); const raw = getComputedStyle(document.documentElement).getPropertyValue('--accent-rgb').trim();
if (!raw) return null; if (!raw) return null;
const parts = raw.split(',').map((p) => Number(p.trim())); const parts = raw.split(',').map((p) => Number(p.trim()));
return parts.length === 3 && parts.every((n) => Number.isFinite(n)) ? (parts as [number, number, number]) : null; return parts.length === 3 && parts.every((n) => Number.isFinite(n)) ? (parts as Rgb) : null;
} }
export function JarvisCore({ state, inputLevel = 0, reduceMotion = false, className }: Props) { export function JarvisCore({ state, inputLevel = 0, reduceMotion = false, pulse = 0, className }: Props) {
const canvasRef = useRef<HTMLCanvasElement>(null); const canvasRef = useRef<HTMLCanvasElement>(null);
const stateRef = useRef(state); const stateRef = useRef(state);
const inputRef = useRef(inputLevel); const inputRef = useRef(inputLevel);
const motionRef = useRef(reduceMotion); const motionRef = useRef(reduceMotion);
const pulseRef = useRef(0);
stateRef.current = state; stateRef.current = state;
inputRef.current = inputLevel; inputRef.current = inputLevel;
motionRef.current = reduceMotion; motionRef.current = reduceMotion;
useEffect(() => {
if (pulse > 0) pulseRef.current = 1;
}, [pulse]);
useEffect(() => { useEffect(() => {
const canvas = canvasRef.current; const canvas = canvasRef.current;
if (!canvas) return; if (!canvas) return;
@@ -59,13 +74,33 @@ export function JarvisCore({ state, inputLevel = 0, reduceMotion = false, classN
let height = 0; let height = 0;
let dpr = 1; let dpr = 1;
const bars = new Float32Array(BARS); const bars = new Float32Array(BARS);
const color = { main: [52, 228, 255] as [number, number, number], glow: [52, 228, 255] as [number, number, number] }; const spectrumBuffer = new Float32Array(BARS);
const color = { main: [52, 228, 255] as Rgb, glow: [52, 228, 255] as Rgb };
let energy = 0; let energy = 0;
let spin = 0; let spin = 0;
let spin2 = 0; let spin2 = 0;
let spin3 = 0; let spin3 = 0;
let last = performance.now(); let last = performance.now();
let visible = true; let visible = document.visibilityState === 'visible';
let frame = 0;
let accent = readThemeAccent();
const colorCache = new Map<number, string>();
// Quantised rgba cache: colours are quantised to integers and alpha to 1/64 steps.
const rgba = (c: Rgb, a: number): string => {
const r = c[0] | 0;
const g = c[1] | 0;
const b = c[2] | 0;
const q = Math.max(0, Math.min(64, Math.round(a * 64)));
const key = (((r << 8) | g) << 8 | b) * 65 + q;
let s = colorCache.get(key);
if (!s) {
s = `rgba(${r},${g},${b},${(q / 64).toFixed(3)})`;
if (colorCache.size > 4000) colorCache.clear();
colorCache.set(key, s);
}
return s;
};
const resize = () => { const resize = () => {
const rect = canvas.getBoundingClientRect(); const rect = canvas.getBoundingClientRect();
@@ -79,41 +114,70 @@ export function JarvisCore({ state, inputLevel = 0, reduceMotion = false, classN
observer.observe(canvas); observer.observe(canvas);
resize(); resize();
const themeObserver = new MutationObserver(() => {
accent = readThemeAccent();
});
themeObserver.observe(document.documentElement, { attributes: true, attributeFilter: ['data-theme'] });
const start = () => {
if (!raf) {
last = performance.now();
raf = requestAnimationFrame(draw);
}
};
const onVisibility = () => { const onVisibility = () => {
visible = document.visibilityState === 'visible'; visible = document.visibilityState === 'visible';
if (visible) start();
}; };
document.addEventListener('visibilitychange', onVisibility); document.addEventListener('visibilitychange', onVisibility);
// backgroundThrottling is off, so the Page Visibility API never fires: main tells us instead.
const offWindow = bridge()?.window.onVisibility?.((shown) => {
visible = shown;
if (visible) start();
});
const lerp = (a: number, b: number, t: number) => a + (b - a) * t; const lerp = (a: number, b: number, t: number) => a + (b - a) * t;
const rgba = (c: [number, number, number], a: number) => `rgba(${c[0]},${c[1]},${c[2]},${a})`;
const draw = (now: number) => { const draw = (now: number) => {
if (!visible) {
raf = 0;
return;
}
raf = requestAnimationFrame(draw); raf = requestAnimationFrame(draw);
if (!visible) return; const st = stateRef.current;
frame++;
// Idle economy: 30 fps is plenty for the breathing animation.
if (st === 'idle' && pulseRef.current < 0.01 && frame % 2 === 1) return;
const dt = Math.min(0.05, (now - last) / 1000); const dt = Math.min(0.05, (now - last) / 1000);
last = now; last = now;
const st = stateRef.current;
const motion = motionRef.current ? 0.25 : 1; const motion = motionRef.current ? 0.25 : 1;
// ── target colours & energy
const target = PALETTES[st]; const target = PALETTES[st];
const themeAccent = st === 'idle' || st === 'listening' || st === 'speaking' ? readThemeAccent() : null; const themeAccent = st === 'idle' || st === 'listening' || st === 'speaking' ? accent : null;
const mainTarget = themeAccent ?? target.main; const mainTarget = themeAccent ?? target.main;
const glowTarget = themeAccent ?? target.glow;
for (let i = 0; i < 3; i++) { for (let i = 0; i < 3; i++) {
color.main[i] = lerp(color.main[i], mainTarget[i], 0.08); color.main[i] = lerp(color.main[i], mainTarget[i], 0.08);
color.glow[i] = lerp(color.glow[i], (themeAccent ?? target.glow)[i], 0.08); color.glow[i] = lerp(color.glow[i], glowTarget[i], 0.08);
} }
const speaking = st === 'speaking'; const speaking = st === 'speaking';
const listening = st === 'listening'; const listening = st === 'listening';
const spectrum = speaking ? audioBus.spectrum(BARS, 'output') : listening ? audioBus.spectrum(BARS, 'input') : null; const spectrum = speaking
? audioBus.spectrum(BARS, 'output', spectrumBuffer)
: listening
? audioBus.spectrum(BARS, 'input', spectrumBuffer)
: null;
const level = speaking ? audioBus.outputLevel() : listening ? Math.max(inputRef.current, audioBus.inputLevel()) : 0; const level = speaking ? audioBus.outputLevel() : listening ? Math.max(inputRef.current, audioBus.inputLevel()) : 0;
const targetEnergy = st === 'thinking' ? 0.45 : st === 'alert' || st === 'error' ? 0.6 : level; pulseRef.current = Math.max(0, pulseRef.current - dt * 2.2);
energy = lerp(energy, targetEnergy, speaking || listening ? 0.35 : 0.08); const ping = pulseRef.current;
const targetEnergy = Math.max(ping, st === 'thinking' ? 0.45 : st === 'alert' || st === 'error' ? 0.6 : level);
energy = lerp(energy, targetEnergy, speaking || listening || ping > 0 ? 0.35 : 0.08);
for (let i = 0; i < BARS; i++) { for (let i = 0; i < BARS; i++) {
let t = 0; let t = 0;
if (spectrum) { if (spectrum) {
const v = spectrum[i]; const v = spectrum[i];
t = Math.pow(v, 1.4); t = v * Math.sqrt(v);
if (listening) t = Math.max(t, inputRef.current * (0.5 + 0.5 * Math.sin(now / 90 + i))); if (listening) t = Math.max(t, inputRef.current * (0.5 + 0.5 * Math.sin(now / 90 + i)));
} else if (st === 'thinking') { } else if (st === 'thinking') {
t = 0.15 + 0.12 * Math.sin(now / 260 + i * 0.35); t = 0.15 + 0.12 * Math.sin(now / 260 + i * 0.35);
@@ -122,10 +186,10 @@ export function JarvisCore({ state, inputLevel = 0, reduceMotion = false, classN
} else { } else {
t = 0.12 + 0.08 * Math.sin(now / 140 + i * 0.7); t = 0.12 + 0.08 * Math.sin(now / 140 + i * 0.7);
} }
t = Math.max(t, ping * 0.6);
bars[i] = lerp(bars[i], t, t > bars[i] ? 0.5 : 0.12); bars[i] = lerp(bars[i], t, t > bars[i] ? 0.5 : 0.12);
} }
// ── rotation
const speed = (st === 'thinking' ? 1.6 : st === 'alert' || st === 'error' ? 1.2 : speaking ? 0.7 : 0.35) * motion; const speed = (st === 'thinking' ? 1.6 : st === 'alert' || st === 'error' ? 1.2 : speaking ? 0.7 : 0.35) * motion;
spin += dt * 0.35 * speed; spin += dt * 0.35 * speed;
spin2 -= dt * 0.55 * speed; spin2 -= dt * 0.55 * speed;
@@ -139,7 +203,7 @@ export function JarvisCore({ state, inputLevel = 0, reduceMotion = false, classN
const main = color.main; const main = color.main;
const glow = color.glow; const glow = color.glow;
// ── ambient glow // ambient glow
const halo = ctx.createRadialGradient(cx, cy, R * 0.1, cx, cy, R * 1.05); const halo = ctx.createRadialGradient(cx, cy, R * 0.1, cx, cy, R * 1.05);
halo.addColorStop(0, rgba(glow, 0.18 + energy * 0.25)); halo.addColorStop(0, rgba(glow, 0.18 + energy * 0.25));
halo.addColorStop(0.5, rgba(glow, 0.05)); halo.addColorStop(0.5, rgba(glow, 0.05));
@@ -147,53 +211,51 @@ export function JarvisCore({ state, inputLevel = 0, reduceMotion = false, classN
ctx.fillStyle = halo; ctx.fillStyle = halo;
ctx.fillRect(0, 0, width, height); ctx.fillRect(0, 0, width, height);
// ── outer tick ring // outer tick ring: three batched paths (long / mid / short ticks)
ctx.save(); ctx.save();
ctx.translate(cx, cy); ctx.translate(cx, cy);
ctx.rotate(spin); ctx.rotate(spin);
ctx.strokeStyle = rgba(main, 0.55);
ctx.lineWidth = 1; ctx.lineWidth = 1;
for (let i = 0; i < 120; i++) { for (let group = 0; group < 3; group++) {
const a = (i / 120) * Math.PI * 2;
const long = i % 10 === 0;
const mid = i % 5 === 0;
const r0 = R * (long ? 0.93 : mid ? 0.955 : 0.97);
ctx.globalAlpha = long ? 0.9 : mid ? 0.6 : 0.3;
ctx.beginPath(); ctx.beginPath();
ctx.moveTo(Math.cos(a) * r0, Math.sin(a) * r0); for (let i = 0; i < TICKS; i++) {
ctx.lineTo(Math.cos(a) * R, Math.sin(a) * R); const kind = i % 10 === 0 ? 0 : i % 5 === 0 ? 1 : 2;
if (kind !== group) continue;
const a = (i / TICKS) * Math.PI * 2;
const r0 = R * (kind === 0 ? 0.93 : kind === 1 ? 0.955 : 0.97);
ctx.moveTo(Math.cos(a) * r0, Math.sin(a) * r0);
ctx.lineTo(Math.cos(a) * R, Math.sin(a) * R);
}
ctx.strokeStyle = rgba(main, group === 0 ? 0.5 : group === 1 ? 0.33 : 0.17);
ctx.stroke(); ctx.stroke();
} }
ctx.globalAlpha = 1;
ctx.beginPath(); ctx.beginPath();
ctx.arc(0, 0, R, 0, Math.PI * 2); ctx.arc(0, 0, R, 0, Math.PI * 2);
ctx.strokeStyle = rgba(main, 0.35); ctx.strokeStyle = rgba(main, 0.35);
ctx.stroke(); ctx.stroke();
ctx.restore(); ctx.restore();
// ── segmented arc ring // segmented arc ring
ctx.save(); ctx.save();
ctx.translate(cx, cy); ctx.translate(cx, cy);
ctx.rotate(spin2); ctx.rotate(spin2);
ctx.lineCap = 'butt'; for (let i = 0; i < SEGMENTS; i++) {
const segments = 6; const startAngle = (i / SEGMENTS) * Math.PI * 2;
for (let i = 0; i < segments; i++) { const span = (Math.PI * 2) / SEGMENTS - 0.22;
const start = (i / segments) * Math.PI * 2;
const span = (Math.PI * 2) / segments - 0.22;
ctx.beginPath(); ctx.beginPath();
ctx.arc(0, 0, R * 0.86, start, start + span); ctx.arc(0, 0, R * 0.86, startAngle, startAngle + span);
ctx.lineWidth = 6; ctx.lineWidth = 6;
ctx.strokeStyle = rgba(main, 0.16 + (i % 2 === 0 ? 0.18 : 0.05) + energy * 0.2); ctx.strokeStyle = rgba(main, 0.16 + (i % 2 === 0 ? 0.18 : 0.05) + energy * 0.2);
ctx.stroke(); ctx.stroke();
ctx.beginPath(); ctx.beginPath();
ctx.arc(0, 0, R * 0.82, start + 0.08, start + span * 0.45); ctx.arc(0, 0, R * 0.82, startAngle + 0.08, startAngle + span * 0.45);
ctx.lineWidth = 2; ctx.lineWidth = 2;
ctx.strokeStyle = rgba(main, 0.75); ctx.strokeStyle = rgba(main, 0.75);
ctx.stroke(); ctx.stroke();
} }
ctx.restore(); ctx.restore();
// ── dashed ring // dashed ring
ctx.save(); ctx.save();
ctx.translate(cx, cy); ctx.translate(cx, cy);
ctx.rotate(spin3); ctx.rotate(spin3);
@@ -206,36 +268,46 @@ export function JarvisCore({ state, inputLevel = 0, reduceMotion = false, classN
ctx.setLineDash([]); ctx.setLineDash([]);
ctx.restore(); ctx.restore();
// ── audio spectrum bars // audio spectrum bars: one wide low-alpha glow pass, then one crisp pass per intensity bucket
ctx.save(); ctx.save();
ctx.translate(cx, cy); ctx.translate(cx, cy);
ctx.lineCap = 'round'; ctx.lineCap = 'round';
const inner = R * 0.5; const inner = R * 0.5;
const maxLen = R * 0.16; const maxLen = R * 0.16;
ctx.lineWidth = 7;
ctx.strokeStyle = rgba(glow, 0.1 + energy * 0.12);
ctx.beginPath();
for (let i = 0; i < BARS; i++) { for (let i = 0; i < BARS; i++) {
const a = (i / BARS) * Math.PI * 2 - Math.PI / 2; const a = (i / BARS) * Math.PI * 2 - Math.PI / 2;
const len = 2 + bars[i] * maxLen; const len = 2 + bars[i] * maxLen;
const x0 = Math.cos(a) * inner; ctx.moveTo(Math.cos(a) * inner, Math.sin(a) * inner);
const y0 = Math.sin(a) * inner; ctx.lineTo(Math.cos(a) * (inner + len), Math.sin(a) * (inner + len));
const x1 = Math.cos(a) * (inner + len); }
const y1 = Math.sin(a) * (inner + len); ctx.stroke();
ctx.lineWidth = 2.4; ctx.lineWidth = 2.4;
ctx.strokeStyle = rgba(main, 0.35 + bars[i] * 0.65); for (let bucket = 0; bucket < 4; bucket++) {
ctx.shadowBlur = 8 * bars[i];
ctx.shadowColor = rgba(glow, 0.8);
ctx.beginPath(); ctx.beginPath();
ctx.moveTo(x0, y0); let any = false;
ctx.lineTo(x1, y1); for (let i = 0; i < BARS; i++) {
const b = Math.min(3, Math.floor(bars[i] * 4));
if (b !== bucket) continue;
any = true;
const a = (i / BARS) * Math.PI * 2 - Math.PI / 2;
const len = 2 + bars[i] * maxLen;
ctx.moveTo(Math.cos(a) * inner, Math.sin(a) * inner);
ctx.lineTo(Math.cos(a) * (inner + len), Math.sin(a) * (inner + len));
}
if (!any) continue;
ctx.strokeStyle = rgba(main, 0.35 + (bucket + 0.5) * 0.16);
ctx.stroke(); ctx.stroke();
} }
ctx.shadowBlur = 0;
ctx.restore(); ctx.restore();
// ── reactor core // reactor core
const coreR = R * 0.36; const coreR = R * 0.36;
const breath = 1 + Math.sin(now / 700) * 0.02 + energy * 0.08; const breath = 1 + Math.sin(now / 700) * 0.02 + energy * 0.08;
const core = ctx.createRadialGradient(cx, cy, 0, cx, cy, coreR * breath); const core = ctx.createRadialGradient(cx, cy, 0, cx, cy, coreR * breath);
core.addColorStop(0, `rgba(255,255,255,${0.75 + energy * 0.25})`); core.addColorStop(0, `rgba(255,255,255,${(0.75 + energy * 0.25).toFixed(3)})`);
core.addColorStop(0.25, rgba(main, 0.55 + energy * 0.3)); core.addColorStop(0.25, rgba(main, 0.55 + energy * 0.3));
core.addColorStop(0.7, rgba(glow, 0.12)); core.addColorStop(0.7, rgba(glow, 0.12));
core.addColorStop(1, 'rgba(0,0,0,0)'); core.addColorStop(1, 'rgba(0,0,0,0)');
@@ -253,7 +325,6 @@ export function JarvisCore({ state, inputLevel = 0, reduceMotion = false, classN
ctx.strokeStyle = rgba(main, 0.7 - i * 0.2); ctx.strokeStyle = rgba(main, 0.7 - i * 0.2);
ctx.stroke(); ctx.stroke();
} }
// reactor triangle
ctx.rotate(-spin2 * 0.5); ctx.rotate(-spin2 * 0.5);
ctx.beginPath(); ctx.beginPath();
for (let i = 0; i < 3; i++) { for (let i = 0; i < 3; i++) {
@@ -268,24 +339,28 @@ export function JarvisCore({ state, inputLevel = 0, reduceMotion = false, classN
ctx.stroke(); ctx.stroke();
ctx.restore(); ctx.restore();
// ── thinking orbiters // thinking orbiters (radial gradient dots instead of shadow blur)
if (st === 'thinking' || st === 'alert') { if (st === 'thinking' || st === 'alert') {
ctx.save(); ctx.save();
ctx.translate(cx, cy); ctx.translate(cx, cy);
for (let i = 0; i < 3; i++) { for (let i = 0; i < 3; i++) {
const a = spin3 * 2 + (i / 3) * Math.PI * 2; const a = spin3 * 2 + (i / 3) * Math.PI * 2;
const r = R * 0.78; const r = R * 0.78;
const x = Math.cos(a) * r;
const y = Math.sin(a) * r;
const dot = ctx.createRadialGradient(x, y, 0, x, y, 11);
dot.addColorStop(0, rgba(main, 1));
dot.addColorStop(0.35, rgba(glow, 0.6));
dot.addColorStop(1, 'rgba(0,0,0,0)');
ctx.fillStyle = dot;
ctx.beginPath(); ctx.beginPath();
ctx.arc(Math.cos(a) * r, Math.sin(a) * r, 3.5, 0, Math.PI * 2); ctx.arc(x, y, 11, 0, Math.PI * 2);
ctx.fillStyle = rgba(main, 0.95);
ctx.shadowBlur = 14;
ctx.shadowColor = rgba(glow, 1);
ctx.fill(); ctx.fill();
} }
ctx.restore(); ctx.restore();
} }
// ── listening ring (input level) // listening ring (input level)
if (listening) { if (listening) {
ctx.save(); ctx.save();
ctx.translate(cx, cy); ctx.translate(cx, cy);
@@ -298,11 +373,14 @@ export function JarvisCore({ state, inputLevel = 0, reduceMotion = false, classN
} }
}; };
raf = requestAnimationFrame(draw); start();
return () => { return () => {
cancelAnimationFrame(raf); if (raf) cancelAnimationFrame(raf);
raf = 0;
observer.disconnect(); observer.disconnect();
themeObserver.disconnect();
document.removeEventListener('visibilitychange', onVisibility); document.removeEventListener('visibilitychange', onVisibility);
offWindow?.();
}; };
}, []); }, []);
+12 -4
View File
@@ -1,4 +1,4 @@
import { Minus, X, Settings, PictureInPicture2, RefreshCw, Webhook } from 'lucide-react'; import { Minus, X, Settings, PictureInPicture2, RefreshCw, Webhook, PanelRight } from 'lucide-react';
import { bridge } from '../../lib/bridge'; import { bridge } from '../../lib/bridge';
import { useClock } from '../../hooks/useMetrics'; import { useClock } from '../../hooks/useMetrics';
import { useHermes } from '../../state/hermes'; import { useHermes } from '../../state/hermes';
@@ -6,9 +6,11 @@ import { useSettings } from '../../state/settings';
interface Props { interface Props {
onOpenSettings: () => void; onOpenSettings: () => void;
onToggleOps?: () => void;
opsBadge?: number;
} }
export function TopBar({ onOpenSettings }: Props) { export function TopBar({ onOpenSettings, onToggleOps, opsBadge = 0 }: Props) {
const now = useClock(); const now = useClock();
const link = useHermes((s) => s.link); const link = useHermes((s) => s.link);
const linkDetail = useHermes((s) => s.linkDetail); const linkDetail = useHermes((s) => s.linkDetail);
@@ -55,9 +57,15 @@ export function TopBar({ onOpenSettings }: Props) {
<RefreshCw size={14} className={link === 'checking' ? 'spin' : undefined} /> <RefreshCw size={14} className={link === 'checking' ? 'spin' : undefined} />
</button> </button>
</div> </div>
<span className="clock">{now.toLocaleTimeString([], { hour: '2-digit', minute: '2-digit', second: '2-digit', hour12: false })}</span> <span className="clock">{now.toLocaleTimeString([], { hour: '2-digit', minute: '2-digit', hour12: false })}</span>
<div className="controls"> <div className="controls">
<button className="icon-btn" title="Paramètres" onClick={onOpenSettings}><Settings size={16} /></button> {onToggleOps && (
<button className="icon-btn ops-toggle" title="Panneau Hermes Ops" aria-label="Panneau Hermes Ops" onClick={onToggleOps} style={{ position: 'relative' }}>
<PanelRight size={16} />
{opsBadge > 0 && <span className="badge" style={{ position: 'absolute', top: 2, right: 2, minWidth: 14, height: 14, fontSize: 9, borderRadius: 7, background: 'var(--danger)', color: '#fff', display: 'inline-flex', alignItems: 'center', justifyContent: 'center', padding: '0 3px' }}>{opsBadge}</span>}
</button>
)}
<button className="icon-btn" title="Paramètres (Ctrl+,)" aria-label="Paramètres" onClick={onOpenSettings}><Settings size={16} /></button>
{api && ( {api && (
<> <>
<button className="icon-btn" title="Mode compact flottant" onClick={() => api.window.setMode('compact')}><PictureInPicture2 size={16} /></button> <button className="icon-btn" title="Mode compact flottant" onClick={() => api.window.setMode('compact')}><PictureInPicture2 size={16} /></button>
+15 -6
View File
@@ -1,5 +1,7 @@
import { useEffect, useState } from 'react'; import { Fragment, useEffect, useState } from 'react';
import { useShallow } from 'zustand/react/shallow';
import { import {
AlertTriangle,
Wrench, CalendarClock, Sparkles, Layers, History, Play, Pause, Trash2, Pencil, Plus, Save, X, RefreshCw, Wrench, CalendarClock, Sparkles, Layers, History, Play, Pause, Trash2, Pencil, Plus, Save, X, RefreshCw,
CheckCircle2, XCircle, Loader2, Bot, Cpu, GitBranch, Webhook, FolderOpen CheckCircle2, XCircle, Loader2, Bot, Cpu, GitBranch, Webhook, FolderOpen
} from 'lucide-react'; } from 'lucide-react';
@@ -40,6 +42,7 @@ function ActivityTab() {
function JobsTab() { function JobsTab() {
const jobs = useHermes((s) => s.jobs); const jobs = useHermes((s) => s.jobs);
const jobsError = useHermes((s) => s.jobsError);
const busy = useHermes((s) => s.busy); const busy = useHermes((s) => s.busy);
const lastSyncAt = useHermes((s) => s.lastSyncAt); const lastSyncAt = useHermes((s) => s.lastSyncAt);
const refreshJobs = useHermes((s) => s.refreshJobs); const refreshJobs = useHermes((s) => s.refreshJobs);
@@ -94,8 +97,9 @@ function JobsTab() {
</div> </div>
</div> </div>
)} )}
{jobsError && <div className="test-result fail" style={{ marginBottom: 8 }}><AlertTriangle size={13} /> Crons indisponibles : {jobsError}</div>}
{jobs.length === 0 ? ( {jobs.length === 0 ? (
<div className="empty">Aucun cron Hermes. Créez une mission planifiée en langage naturel.</div> <div className="empty">{jobsError ? 'La liaison Hermes reste utilisable ; seule la planification est indisponible.' : 'Aucun cron Hermes. Créez une mission planifiée en langage naturel.'}</div>
) : ( ) : (
<div className="list"> <div className="list">
{jobs.map((job) => { {jobs.map((job) => {
@@ -142,7 +146,10 @@ function HistoryList() {
<div <div
key={r.id} key={r.id}
className="row-item clickable" className="row-item clickable"
role="button"
tabIndex={0}
style={{ opacity: r.read ? 0.7 : 1 }} style={{ opacity: r.read ? 0.7 : 1 }}
onKeyDown={(e) => { if (e.key === 'Enter') (e.currentTarget as HTMLDivElement).click(); }}
onClick={() => { onClick={() => {
markRead(r.id); markRead(r.id);
addMessage({ role: 'assistant', content: r.output, source: 'cron', jobName: r.jobName, status: 'done', timestamp: new Date(r.at).getTime() || Date.now() }); addMessage({ role: 'assistant', content: r.output, source: 'cron', jobName: r.jobName, status: 'done', timestamp: new Date(r.at).getTime() || Date.now() });
@@ -222,7 +229,7 @@ function SessionsTab() {
const transport = useHermes((s) => s.transport); const transport = useHermes((s) => s.transport);
const sessionId = useSettings((s) => s.settings.hermesSessionId); const sessionId = useSettings((s) => s.settings.hermesSessionId);
const setSessionId = useSettings((s) => s.setHermesSessionId); const setSessionId = useSettings((s) => s.setHermesSessionId);
const chat = useChat(); const chat = useChat.getState();
const activeId = sessionId.replace(/^hs:/, ''); const activeId = sessionId.replace(/^hs:/, '');
const load = async (id: string) => { const load = async (id: string) => {
@@ -255,7 +262,7 @@ function SessionsTab() {
{sessions.length === 0 ? <div className="empty">Aucune session listée par Hermes.</div> : ( {sessions.length === 0 ? <div className="empty">Aucune session listée par Hermes.</div> : (
<div className="list"> <div className="list">
{sessions.map((s) => ( {sessions.map((s) => (
<div key={s.id} className={`row-item clickable`} style={s.id === activeId ? { borderColor: 'var(--accent)' } : undefined} onClick={() => void load(s.id)}> <div key={s.id} className="row-item clickable" role="button" tabIndex={0} style={s.id === activeId ? { borderColor: 'var(--accent)' } : undefined} onClick={() => void load(s.id)} onKeyDown={(e) => { if (e.key === 'Enter') void load(s.id); }}>
<div className="main"> <div className="main">
<span className="name">{s.title || s.id}</span> <span className="name">{s.title || s.id}</span>
<span className="meta">{[s.platform ?? s.source, s.message_count ? `${s.message_count} msg` : null, formatDateTime(s.updated_at ?? s.created_at)].filter(Boolean).join(' · ')}</span> <span className="meta">{[s.platform ?? s.source, s.message_count ? `${s.message_count} msg` : null, formatDateTime(s.updated_at ?? s.created_at)].filter(Boolean).join(' · ')}</span>
@@ -273,7 +280,9 @@ function SessionsTab() {
} }
function LinkTab() { function LinkTab() {
const { link, linkDetail, health, capabilities, transport, models, webhook } = useHermes(); const { link, linkDetail, health, capabilities, transport, models, webhook } = useHermes(
useShallow((s) => ({ link: s.link, linkDetail: s.linkDetail, health: s.health, capabilities: s.capabilities, transport: s.transport, models: s.models, webhook: s.webhook }))
);
const connect = useHermes((s) => s.connect); const connect = useHermes((s) => s.connect);
const hermes = useSettings((s) => s.settings.hermes); const hermes = useSettings((s) => s.settings.hermes);
const [info, setInfo] = useState<{ sharedDir: string; logPath: string; version: string } | null>(null); const [info, setInfo] = useState<{ sharedDir: string; logPath: string; version: string } | null>(null);
@@ -311,7 +320,7 @@ function LinkTab() {
<div className="section-title"><CheckCircle2 size={12} /> Santé</div> <div className="section-title"><CheckCircle2 size={12} /> Santé</div>
<dl className="kv"> <dl className="kv">
{Object.entries(checks).map(([k, v]) => ( {Object.entries(checks).map(([k, v]) => (
<><dt key={`${k}-t`}>{k}</dt><dd key={`${k}-d`}>{typeof v === 'object' ? JSON.stringify(v) : String(v)}</dd></> <Fragment key={k}><dt>{k}</dt><dd>{typeof v === 'object' ? JSON.stringify(v) : String(v)}</dd></Fragment>
))} ))}
</dl> </dl>
</> </>
+3 -3
View File
@@ -28,11 +28,11 @@ export function Telemetry() {
<Activity size={14} /> <Activity size={14} />
<span>Système</span> <span>Système</span>
<span className="spacer" /> <span className="spacer" />
<span className="chip">{m.hostname}</span> <span className="chip">{m.live ? m.hostname : 'HORS PONT'}</span>
</header> </header>
<div className="telemetry"> <div className="telemetry">
<Gauge value={m.cpuLoad} max={100} label="CPU" display={`${Math.round(m.cpuLoad)}%`} warn={90} /> <Gauge value={m.cpuLoad} max={100} label="CPU" display={m.live ? `${Math.round(m.cpuLoad)}%` : '—'} warn={90} />
<Gauge value={m.memUsedPct} max={100} label="RAM" display={`${Math.round(m.memUsedPct)}%`} warn={92} /> <Gauge value={m.memUsedPct} max={100} label="RAM" display={m.live ? `${Math.round(m.memUsedPct)}%` : '—'} warn={92} />
<Gauge value={m.fps} max={60} label="FPS" display={`${m.fps}`} /> <Gauge value={m.fps} max={60} label="FPS" display={`${m.fps}`} />
<Gauge value={m.cpuFreqMhz} max={5500} label="GHz" display={m.cpuFreqMhz ? (m.cpuFreqMhz / 1000).toFixed(1) : '—'} /> <Gauge value={m.cpuFreqMhz} max={5500} label="GHz" display={m.cpuFreqMhz ? (m.cpuFreqMhz / 1000).toFixed(1) : '—'} />
</div> </div>
+120
View File
@@ -0,0 +1,120 @@
import { useEffect } from 'react';
import { Download, Trash2, X, CheckCircle2, Cpu, Mic, Volume2, AlertTriangle, RefreshCw, Ear, Activity } from 'lucide-react';
import type { VoiceModelStatus } from '../../../shared/voice';
import { bridge } from '../../lib/bridge';
import { useShallow } from 'zustand/react/shallow';
import { useVoiceModels } from '../../state/voiceModels';
import { useSettings } from '../../state/settings';
function formatMb(bytes: number): string {
return bytes >= 1024 * 1024 * 1024 ? `${(bytes / 1024 ** 3).toFixed(2)} Go` : `${Math.round(bytes / 1024 / 1024)} Mo`;
}
function ModelRow({ model }: { model: VoiceModelStatus }) {
const progress = useVoiceModels((s) => s.progress[model.id]);
const { download, cancel, remove } = useVoiceModels.getState();
const activeStt = useSettings((s) => s.settings.voice.localModel);
const activeTts = useSettings((s) => s.settings.speech.localModel);
const update = useSettings((s) => s.update);
const isActive = model.kind === 'stt' ? activeStt === model.id : activeTts === model.id;
const busy = !!progress && (progress.phase === 'download' || progress.phase === 'extract');
const wakeMode = useSettings((s) => s.settings.voice.wakeMode);
const activate = () => {
if (model.kind === 'stt') update({ voice: { localModel: model.id, provider: 'local' } });
else if (model.kind === 'kws') update({ voice: { wakeMode: 'kws' } });
else if (model.kind === 'vad') update({ voice: { neuralVad: true } });
else update({ speech: { localModel: model.id, provider: 'local', localSpeaker: model.speakers?.[0]?.id ?? 0 } });
};
const neuralVad = useSettings((s) => s.settings.voice.neuralVad);
const activeNow = model.kind === 'kws' ? wakeMode === 'kws' : model.kind === 'vad' ? neuralVad : isActive;
return (
<div className="row-item" style={activeNow && model.installed ? { borderColor: 'var(--accent)' } : undefined}>
<div className="main">
<div className="row wrap" style={{ gap: 6 }}>
<span className="name">{model.name}</span>
{model.recommended && <span className="status-pill ok">recommandé</span>}
{model.installed && <span className="status-pill online">installé · {formatMb(model.installedBytes)}</span>}
{!model.installed && !busy && <span className="status-pill">{model.sizeMb} Mo</span>}
{activeNow && model.installed && <span className="status-pill active">actif</span>}
</div>
<span className="desc">{model.description}</span>
<span className="meta">langues : {model.languages.join(', ')}{model.speakers ? ` · ${model.speakers.length} voix` : ''}</span>
{busy && (
<div style={{ marginTop: 6 }}>
<div className="level-meter"><div style={{ width: `${progress.percent}%` }} /></div>
<span className="meta">{progress.phase === 'extract' ? 'extraction…' : `${progress.percent}% · ${formatMb(progress.received)} / ${formatMb(progress.total)}`}</span>
</div>
)}
{progress?.phase === 'error' && <span className="meta" style={{ color: 'var(--danger)' }}>échec : {progress.message}</span>}
</div>
<div className="actions">
{busy ? (
<button className="icon-btn danger" title="Annuler" onClick={() => void cancel(model.id)}><X size={13} /></button>
) : model.installed ? (
<>
{!activeNow && <button className="btn small" onClick={activate}>Utiliser</button>}
<button className="icon-btn danger" title="Supprimer du disque" onClick={() => { if (confirm(`Supprimer « ${model.name} » ?`)) void remove(model.id); }}><Trash2 size={13} /></button>
</>
) : (
<button className="btn small primary" onClick={() => void download(model.id)}><Download size={13} /> Télécharger</button>
)}
</div>
</div>
);
}
export function ModelsSection() {
const { models, engine, error } = useVoiceModels(useShallow((s) => ({ models: s.models, engine: s.engine, error: s.error })));
const { refresh, checkEngine } = useVoiceModels.getState();
useEffect(() => {
void refresh();
void checkEngine();
}, [refresh, checkEngine]);
if (!bridge()) return <div className="card"><div className="empty">Les modèles locaux ne sont disponibles que dans l’application Electron.</div></div>;
const stt = models.filter((m) => m.kind === 'stt');
const tts = models.filter((m) => m.kind === 'tts');
const kws = models.filter((m) => m.kind === 'kws');
const vad = models.filter((m) => m.kind === 'vad');
return (
<>
<div className="card">
<div className="section-title"><Cpu size={12} /> Moteur local (sherpa-onnx)</div>
{engine === null ? (
<span className="muted">vérification…</span>
) : engine.available ? (
<div className="test-result ok"><CheckCircle2 size={13} /> moteur prêt · version {engine.version}{engine.loaded.length ? ` · chargé : ${engine.loaded.join(', ')}` : ''}</div>
) : (
<div className="test-result fail"><AlertTriangle size={13} /> {engine.error}</div>
)}
<span className="hint" style={{ display: 'block', marginTop: 8 }}>
Les modèles tournent sur le processeur, sans connexion, dans un processus séparé. Dossier : <code>{engine?.modelsDir}</code>
</span>
{error && <div className="test-result fail" style={{ marginTop: 8 }}><AlertTriangle size={13} /> {error}</div>}
<div className="row" style={{ marginTop: 8 }}>
<button className="btn small ghost" onClick={() => { void refresh(); void checkEngine(); }}><RefreshCw size={13} /> Actualiser</button>
</div>
</div>
<div className="card">
<div className="section-title"><Ear size={12} /> Mot d’activation</div>
<div className="list">{kws.map((m) => <ModelRow key={m.id} model={m} />)}</div>
</div>
<div className="card">
<div className="section-title"><Activity size={12} /> Fin de phrase</div>
<div className="list">{vad.map((m) => <ModelRow key={m.id} model={m} />)}</div>
<span className="hint" style={{ display: 'block', marginTop: 6 }}>Utilisé automatiquement en écoute permanente dès qu’il est installé ; sinon le VAD énergétique prend le relais.</span>
</div>
<div className="card">
<div className="section-title"><Mic size={12} /> Reconnaissance vocale</div>
<div className="list">{stt.map((m) => <ModelRow key={m.id} model={m} />)}</div>
</div>
<div className="card">
<div className="section-title"><Volume2 size={12} /> Synthèse vocale</div>
<div className="list">{tts.map((m) => <ModelRow key={m.id} model={m} />)}</div>
</div>
</>
);
}
+237 -29
View File
@@ -1,17 +1,20 @@
import { useEffect, useState } from 'react'; import { useEffect, useState } from 'react';
import { X, Settings, CheckCircle2, XCircle, Loader2, Mic, Volume2, RotateCcw, Play } from 'lucide-react'; import { X, Settings, CheckCircle2, XCircle, Loader2, Mic, Volume2, RotateCcw, Play } from 'lucide-react';
import { bridge } from '../../lib/bridge'; import { bridge } from '../../lib/bridge';
import { HermesClient } from '../../services/hermes/client'; import { HermesClient, discoverHermesUrl, hermesUrlCandidates } from '../../services/hermes/client';
import { listSystemVoices } from '../../services/voice/tts'; import { listSystemVoices } from '../../services/voice/tts';
import { speech } from '../../services/voice/speech'; import { speech } from '../../services/voice/speech';
import { ensurePreferredVoice } from '../../services/voice/voicePreference';
import { listMicrophones } from '../../services/voice/capture'; import { listMicrophones } from '../../services/voice/capture';
import { transcribeWav } from '../../services/voice/stt'; import { transcribeWav } from '../../services/voice/stt';
import { encodeWav } from '../../services/voice/wav'; import { encodeWav } from '../../services/voice/wav';
import { useHermes } from '../../state/hermes'; import { useHermes } from '../../state/hermes';
import { DEFAULT_SETTINGS, useSettings, type HudTheme } from '../../state/settings'; import { DEFAULT_SETTINGS, useSettings, type HudTheme } from '../../state/settings';
import { useVoice } from '../../state/voice'; import { useVoice } from '../../state/voice';
import { installedModels, useVoiceModels } from '../../state/voiceModels';
import { ModelsSection } from './ModelsSection';
type Section = 'general' | 'hermes' | 'voice' | 'speech' | 'webhook' | 'ui'; type Section = 'general' | 'hermes' | 'voice' | 'speech' | 'models' | 'webhook' | 'notifications' | 'ui';
interface Props { interface Props {
onClose: () => void; onClose: () => void;
@@ -19,24 +22,65 @@ interface Props {
function Toggle({ on, onChange, label, hint }: { on: boolean; onChange: (v: boolean) => void; label: string; hint?: string }) { function Toggle({ on, onChange, label, hint }: { on: boolean; onChange: (v: boolean) => void; label: string; hint?: string }) {
return ( return (
<div className="switch" onClick={() => onChange(!on)}> <button type="button" role="switch" aria-checked={on} className="switch" onClick={() => onChange(!on)}>
<div className="text"> <div className="text">
<span>{label}</span> <span>{label}</span>
{hint && <small>{hint}</small>} {hint && <small>{hint}</small>}
</div> </div>
<span className={`toggle${on ? ' on' : ''}`} /> <span className={`toggle${on ? ' on' : ''}`} />
</div> </button>
); );
} }
type TestState = { status: 'idle' | 'running' | 'ok' | 'fail'; message: string }; type TestState = { status: 'idle' | 'running' | 'ok' | 'fail'; message: string };
function WakeStatus() {
const wake = useVoice((s) => s.wake);
const keywords = useVoice((s) => s.wakeKeywords);
const error = useVoice((s) => s.error);
const models = useVoiceModels((s) => s.models);
const download = useVoiceModels((s) => s.download);
const progress = useVoiceModels((s) => s.progress['kws-en']);
const installed = models.find((m) => m.id === 'kws-en')?.installed;
if (installed === false) {
return (
<div className="test-result fail" style={{ marginTop: 8 }}>
<XCircle size={13} />
<span>Détecteur non installé.</span>
<button className="btn small primary" style={{ marginLeft: 'auto' }} disabled={!!progress} onClick={() => void download('kws-en')}>
{progress ? `${progress.percent}%` : 'Télécharger (4 Mo)'}
</button>
</div>
);
}
return (
<div className={`test-result ${wake === 'spotting' ? 'ok' : wake === 'error' ? 'fail' : ''}`} style={{ marginTop: 8 }}>
{wake === 'spotting' ? <CheckCircle2 size={13} /> : wake === 'error' ? <XCircle size={13} /> : <Loader2 size={13} className="spin" />}
<span>{wake === 'spotting' ? `à l’écoute de « ${keywords.join(' », « ')} »` : wake === 'error' ? error ?? 'erreur' : 'démarrage…'}</span>
</div>
);
}
function TestResult({ t }: { t: TestState }) {
if (t.status === 'idle') return null;
return (
<div className={`test-result ${t.status === 'ok' ? 'ok' : t.status === 'fail' ? 'fail' : ''}`}>
{t.status === 'running' ? <Loader2 size={13} className="spin" /> : t.status === 'ok' ? <CheckCircle2 size={13} /> : <XCircle size={13} />}
<span>{t.message}</span>
</div>
);
}
export function SettingsDrawer({ onClose }: Props) { export function SettingsDrawer({ onClose }: Props) {
const settings = useSettings((s) => s.settings); const settings = useSettings((s) => s.settings);
const update = useSettings((s) => s.update); const update = useSettings((s) => s.update);
const reset = useSettings((s) => s.reset); const reset = useSettings((s) => s.reset);
const hermesStore = useHermes(); const hermesModels = useHermes((s) => s.models);
const hermesWebhook = useHermes((s) => s.webhook);
const hermesConnect = useHermes((s) => s.connect);
const micDevices = useVoice((s) => s.micDevices); const micDevices = useVoice((s) => s.micDevices);
const voiceModels = useVoiceModels((s) => s.models);
const refreshModels = useVoiceModels((s) => s.refresh);
const [section, setSection] = useState<Section>('hermes'); const [section, setSection] = useState<Section>('hermes');
const [hermesTest, setHermesTest] = useState<TestState>({ status: 'idle', message: '' }); const [hermesTest, setHermesTest] = useState<TestState>({ status: 'idle', message: '' });
const [sttTest, setSttTest] = useState<TestState>({ status: 'idle', message: '' }); const [sttTest, setSttTest] = useState<TestState>({ status: 'idle', message: '' });
@@ -49,8 +93,9 @@ export function SettingsDrawer({ onClose }: Props) {
refresh(); refresh();
speechSynthesis.addEventListener?.('voiceschanged', refresh); speechSynthesis.addEventListener?.('voiceschanged', refresh);
void listMicrophones().then((d) => useVoice.getState().setMicDevices(d)); void listMicrophones().then((d) => useVoice.getState().setMicDevices(d));
void refreshModels();
return () => speechSynthesis.removeEventListener?.('voiceschanged', refresh); return () => speechSynthesis.removeEventListener?.('voiceschanged', refresh);
}, []); }, [refreshModels]);
const testHermes = async () => { const testHermes = async () => {
setHermesTest({ status: 'running', message: 'connexion…' }); setHermesTest({ status: 'running', message: 'connexion…' });
@@ -60,14 +105,28 @@ export function SettingsDrawer({ onClose }: Props) {
let caps = ''; let caps = '';
try { try {
const c = await client.capabilities(); const c = await client.capabilities();
caps = ` · ${String(c.model ?? '')} · ${Object.entries(c.features ?? {}).filter(([, v]) => v === true).length} fonctions`; caps = ` · modèle ${String(c.model ?? '?')} · ${Object.entries(c.features ?? {}).filter(([, v]) => v === true).length} fonctions`;
} catch { } catch (err) {
caps = ' · capabilities indisponibles'; const message = (err as Error).message;
if (/page web/.test(message)) throw err;
caps = /404/.test(message)
? ' · /v1/capabilities absent (Hermes ancien : transport chat completions)'
: ` · capabilities : ${message}`;
} }
setHermesTest({ status: 'ok', message: `${health.status}${caps}` }); const via = bridge() ? 'via Electron' : 'via navigateur';
void hermesStore.connect(); setHermesTest({ status: 'ok', message: `${health.status} (${via})${caps}` });
void hermesConnect();
} catch (err) { } catch (err) {
setHermesTest({ status: 'fail', message: (err as Error).message }); const message = (err as Error).message;
setHermesTest({ status: 'running', message: `${message} — recherche de l’API sur le même hôte…` });
const found = await discoverHermesUrl(settings.hermes, (u) => setHermesTest({ status: 'running', message: `essai ${u}…` })).catch(() => null);
if (found) {
update({ hermes: { url: found } });
setHermesTest({ status: 'ok', message: `API Hermes trouvée : ${found} (URL corrigée automatiquement). Relancez le test.` });
void hermesConnect();
} else {
setHermesTest({ status: 'fail', message: `${message} Aucune API trouvée sur ${hermesUrlCandidates(settings.hermes.url).slice(0, 4).join(', ')}…` });
}
} }
}; };
@@ -97,19 +156,13 @@ export function SettingsDrawer({ onClose }: Props) {
useHermes.setState({ webhook: status }); useHermes.setState({ webhook: status });
}; };
const Result = ({ t }: { t: TestState }) =>
t.status === 'idle' ? null : (
<div className={`test-result ${t.status === 'ok' ? 'ok' : t.status === 'fail' ? 'fail' : ''}`}>
{t.status === 'running' ? <Loader2 size={13} className="spin" /> : t.status === 'ok' ? <CheckCircle2 size={13} /> : <XCircle size={13} />}
<span>{t.message}</span>
</div>
);
const nav: Array<[Section, string]> = [ const nav: Array<[Section, string]> = [
['hermes', 'Hermes'], ['hermes', 'Hermes'],
['voice', 'Micro / STT'], ['voice', 'Micro / STT'],
['speech', 'Voix / TTS'], ['speech', 'Voix / TTS'],
['models', 'Modèles locaux'],
['webhook', 'Webhook'], ['webhook', 'Webhook'],
['notifications', 'Notifications'],
['general', 'Général'], ['general', 'Général'],
['ui', 'Interface'] ['ui', 'Interface']
]; ];
@@ -145,7 +198,7 @@ export function SettingsDrawer({ onClose }: Props) {
<label>Modèle</label> <label>Modèle</label>
<input className="input" list="hermes-models" value={settings.hermes.model} placeholder="(défaut du serveur)" onChange={(e) => update({ hermes: { model: e.target.value } })} /> <input className="input" list="hermes-models" value={settings.hermes.model} placeholder="(défaut du serveur)" onChange={(e) => update({ hermes: { model: e.target.value } })} />
<datalist id="hermes-models"> <datalist id="hermes-models">
{hermesStore.models.map((m) => <option key={m.id} value={m.id} />)} {hermesModels.map((m) => <option key={m.id} value={m.id} />)}
</datalist> </datalist>
</div> </div>
<div className="field"> <div className="field">
@@ -174,6 +227,11 @@ export function SettingsDrawer({ onClose }: Props) {
</div> </div>
</div> </div>
<div className="field"> <div className="field">
<label>Modèle « mission » (tâches longues)</label>
<input className="input" list="hermes-models" value={settings.hermes.missionModel} placeholder="(même modèle)" onChange={(e) => update({ hermes: { missionModel: e.target.value } })} />
<span className="hint">Utilisé quand le mode Mission est activé dans la barre de commande. Laissez vide pour garder le modèle principal.</span>
</div>
<div className="field">
<label>Instructions EveFlow (superposées au prompt Hermes)</label> <label>Instructions EveFlow (superposées au prompt Hermes)</label>
<textarea className="textarea" value={settings.hermes.instructions} onChange={(e) => update({ hermes: { instructions: e.target.value } })} /> <textarea className="textarea" value={settings.hermes.instructions} onChange={(e) => update({ hermes: { instructions: e.target.value } })} />
</div> </div>
@@ -181,7 +239,7 @@ export function SettingsDrawer({ onClose }: Props) {
<div className="row" style={{ marginTop: 10 }}> <div className="row" style={{ marginTop: 10 }}>
<button className="btn primary small" onClick={() => void testHermes()}><Play size={13} /> Tester la liaison</button> <button className="btn primary small" onClick={() => void testHermes()}><Play size={13} /> Tester la liaison</button>
</div> </div>
<div style={{ marginTop: 8 }}><Result t={hermesTest} /></div> <div style={{ marginTop: 8 }}><TestResult t={hermesTest} /></div>
</div> </div>
</> </>
)} )}
@@ -191,10 +249,23 @@ export function SettingsDrawer({ onClose }: Props) {
<div className="field"> <div className="field">
<label>Moteur de reconnaissance</label> <label>Moteur de reconnaissance</label>
<select className="select" value={settings.voice.provider} onChange={(e) => update({ voice: { provider: e.target.value as typeof settings.voice.provider } })}> <select className="select" value={settings.voice.provider} onChange={(e) => update({ voice: { provider: e.target.value as typeof settings.voice.provider } })}>
<option value="local">Local dans l’application (Whisper via sherpa-onnx, hors ligne)</option>
<option value="openai-compatible">API compatible OpenAI (Qwen3-ASR, Whisper, Speaches, LocalAI…)</option> <option value="openai-compatible">API compatible OpenAI (Qwen3-ASR, Whisper, Speaches, LocalAI…)</option>
<option value="browser">Reconnaissance Chromium (en ligne, secours)</option> <option value="browser">Reconnaissance Chromium (en ligne, secours)</option>
</select> </select>
</div> </div>
{settings.voice.provider === 'local' && (
<div className="field">
<label>Modèle local</label>
{installedModels(voiceModels, 'stt').length === 0 ? (
<span className="hint">Aucun modèle installé. <a href="#" onClick={(e) => { e.preventDefault(); setSection('models'); }}>Téléchargez Whisper base dans « Modèles locaux »</a>.</span>
) : (
<select className="select" value={settings.voice.localModel} onChange={(e) => update({ voice: { localModel: e.target.value } })}>
{installedModels(voiceModels, 'stt').map((m) => <option key={m.id} value={m.id}>{m.name}</option>)}
</select>
)}
</div>
)}
{settings.voice.provider === 'openai-compatible' && ( {settings.voice.provider === 'openai-compatible' && (
<> <>
<div className="field"> <div className="field">
@@ -251,18 +322,89 @@ export function SettingsDrawer({ onClose }: Props) {
</div> </div>
<Toggle on={settings.voice.handsFree} onChange={(v) => { update({ voice: { handsFree: v } }); useVoice.getState().setHandsFree(v); }} label="Mains libres au démarrage" hint="Le micro se réactive automatiquement après chaque réponse." /> <Toggle on={settings.voice.handsFree} onChange={(v) => { update({ voice: { handsFree: v } }); useVoice.getState().setHandsFree(v); }} label="Mains libres au démarrage" hint="Le micro se réactive automatiquement après chaque réponse." />
<Toggle on={settings.voice.wakeChime} onChange={(v) => update({ voice: { wakeChime: v } })} label="Signal sonore d’écoute" /> <Toggle on={settings.voice.wakeChime} onChange={(v) => update({ voice: { wakeChime: v } })} label="Signal sonore d’écoute" />
<div className="row" style={{ marginTop: 10 }}> <div className="field" style={{ marginTop: 10 }}>
<button className="btn small" onClick={() => void testStt()} disabled={settings.voice.provider !== 'openai-compatible'}><Mic size={13} /> Tester l’API STT</button> <label>Mot d’activation</label>
<select className="select" value={settings.voice.wakeMode} onChange={(e) => update({ voice: { wakeMode: e.target.value as typeof settings.voice.wakeMode, wakeWordEnabled: e.target.value === 'transcript' } })}>
<option value="off">Désactivé (bouton, raccourci ou mains libres)</option>
<option value="kws">Écoute permanente : le micro reste ouvert et réagit au mot-clé (recommandé)</option>
<option value="transcript">Filtre après transcription (mains libres) : les phrases sans le mot sont ignorées</option>
</select>
<span className="hint">
{settings.voice.wakeMode === 'kws'
? 'Détection locale par un modèle de 3 Mo, quasi gratuite en CPU. Dire le mot seul ouvre l’écoute ; dire le mot puis la commande envoie directement. Le mot coupe aussi la voix en cours.'
: settings.voice.wakeMode === 'transcript'
? 'Chaque phrase est transcrite puis filtrée : plus coûteux, à réserver à la reconnaissance locale.'
: 'Le micro s’active avec le bouton, Ctrl+Shift+Espace ou la boucle mains libres.'}
</span>
</div> </div>
<div style={{ marginTop: 8 }}><Result t={sttTest} /></div> {settings.voice.wakeMode !== 'off' && (
<div className="grid-2">
<div className="field">
<label>Mot-clé</label>
<input className="input" value={settings.voice.wakeWord} placeholder="jarvis" onChange={(e) => update({ voice: { wakeWord: e.target.value.toLowerCase().slice(0, 40) } })} />
<span className="hint">Prononciation anglaise conseillée (« jarvis », « hey jarvis », « computer », « friday »…).</span>
</div>
{settings.voice.wakeMode === 'kws' && (
<div className="field">
<label>Sensibilité du mot-clé : {settings.voice.kwsSensitivity}/5</label>
<input className="range" type="range" min={1} max={5} step={1} value={settings.voice.kwsSensitivity} onChange={(e) => update({ voice: { kwsSensitivity: Number(e.target.value) } })} />
</div>
)}
</div>
)}
{settings.voice.wakeMode === 'kws' && (
<>
<WakeStatus />
<Toggle on={settings.voice.neuralVad} onChange={(v) => update({ voice: { neuralVad: v } })} label="Fin de phrase neuronale (Silero)" hint="Coupe l’écoute au bon moment, même avec du bruit de fond. Nécessite le modèle Silero VAD (0,6 Mo) dans Modèles locaux." />
</>
)}
<Toggle on={settings.voice.localCommands} onChange={(v) => update({ voice: { localCommands: v } })} label="Commandes locales instantanées" hint="« Verrouille la session », « monte le son », « ouvre Spotify », « regarde mon écran »… exécutées sur ce PC sans passer par Hermes." />
<div className="row" style={{ marginTop: 10 }}>
<button className="btn small" onClick={() => void testStt()} disabled={settings.voice.provider === 'browser'}><Mic size={13} /> Tester la reconnaissance</button>
</div>
<div style={{ marginTop: 8 }}><TestResult t={sttTest} /></div>
</div> </div>
)} )}
{section === 'speech' && ( {section === 'speech' && (
<div className="card"> <div className="card">
<div className="field">
<label>Voix</label>
<div className="segmented">
<button className={(settings.speech.voiceGender ?? 'male') === 'male' ? 'active' : ''} onClick={() => { update({ speech: { voiceGender: 'male' } }); void ensurePreferredVoice().then((m) => m && setTtsTest({ status: 'ok', message: m })); }}>Masculine</button>
<button className={settings.speech.voiceGender === 'female' ? 'active' : ''} onClick={() => { update({ speech: { voiceGender: 'female' } }); void ensurePreferredVoice().then((m) => m && setTtsTest({ status: 'ok', message: m })); }}>Féminine</button>
</div>
<span className="hint">S’applique à tous les moteurs : voix locale (Piper Tom ou Pierre pour le masculin, téléchargée automatiquement si besoin), voix OpenAI (onyx / nova), voix système Windows (Paul / Hortense). Kokoro n’a pas de voix française masculine.</span>
</div>
<div className="field">
<label>Timbre</label>
<div className="segmented">
<button className={settings.speech.timbre === 'jarvis' ? 'active' : ''} onClick={() => update({ speech: { timbre: 'jarvis' } })}>JARVIS</button>
<button className={settings.speech.timbre !== 'jarvis' ? 'active' : ''} onClick={() => update({ speech: { timbre: 'natural' } })}>Naturel</button>
</div>
<span className="hint">JARVIS : voix légèrement plus grave et posée, chaleur dans les basses, présence, courte réverbération d’intercom, comme dans le film.</span>
</div>
<div className="row" style={{ marginBottom: 10 }}>
<button
className="btn small primary"
onClick={() => {
update({ speech: { provider: 'local', voiceGender: 'male', timbre: 'jarvis', speed: 0.97, autoSpeak: true } });
setTtsTest({ status: 'running', message: 'préparation de la voix JARVIS (téléchargement de Piper Tom si nécessaire)…' });
void ensurePreferredVoice().then((m) => setTtsTest({ status: 'ok', message: m || 'Voix JARVIS prête.' })).catch((e: Error) => setTtsTest({ status: 'fail', message: e.message }));
}}
>
<Volume2 size={13} /> Préréglage voix JARVIS (français, masculine, locale)
</button>
<span className="status-pill">
{settings.speech.provider === 'local'
? `voix active : ${voiceModels.find((m) => m.id === settings.speech.localModel)?.speakers?.find((s) => s.id === settings.speech.localSpeaker)?.name ?? settings.speech.localModel}`
: `moteur : ${settings.speech.provider}`}
</span>
</div>
<div className="field"> <div className="field">
<label>Moteur de synthèse</label> <label>Moteur de synthèse</label>
<select className="select" value={settings.speech.provider} onChange={(e) => update({ speech: { provider: e.target.value as typeof settings.speech.provider } })}> <select className="select" value={settings.speech.provider} onChange={(e) => update({ speech: { provider: e.target.value as typeof settings.speech.provider } })}>
<option value="local">Local dans l’application (Kokoro / Piper via sherpa-onnx, hors ligne)</option>
<option value="openai-compatible">API compatible OpenAI /v1/audio/speech (Kokoro, Piper, OpenAI, LocalAI…)</option> <option value="openai-compatible">API compatible OpenAI /v1/audio/speech (Kokoro, Piper, OpenAI, LocalAI…)</option>
<option value="system">Voix système Windows</option> <option value="system">Voix système Windows</option>
<option value="google-free">Google Translate (gratuit, en ligne)</option> <option value="google-free">Google Translate (gratuit, en ligne)</option>
@@ -299,6 +441,33 @@ export function SettingsDrawer({ onClose }: Props) {
</div> </div>
</> </>
)} )}
{settings.speech.provider === 'local' && (
installedModels(voiceModels, 'tts').length === 0 ? (
<div className="field">
<span className="hint">Aucun modèle de voix installé. <a href="#" onClick={(e) => { e.preventDefault(); setSection('models'); }}>Téléchargez Kokoro dans « Modèles locaux »</a>.</span>
</div>
) : (
<div className="grid-2">
<div className="field">
<label>Modèle local</label>
<select className="select" value={settings.speech.localModel} onChange={(e) => {
const m = voiceModels.find((x) => x.id === e.target.value);
update({ speech: { localModel: e.target.value, localSpeaker: m?.speakers?.[0]?.id ?? 0 } });
}}>
{installedModels(voiceModels, 'tts').map((m) => <option key={m.id} value={m.id}>{m.name}</option>)}
</select>
</div>
<div className="field">
<label>Voix</label>
<select className="select" value={settings.speech.localSpeaker} onChange={(e) => update({ speech: { localSpeaker: Number(e.target.value) } })}>
{(voiceModels.find((m) => m.id === settings.speech.localModel)?.speakers ?? [{ id: 0, name: 'Voix 0', lang: '' }]).map((sp) => (
<option key={sp.id} value={sp.id}>{sp.name}</option>
))}
</select>
</div>
</div>
)
)}
{settings.speech.provider === 'system' && ( {settings.speech.provider === 'system' && (
<div className="field"> <div className="field">
<label>Voix système</label> <label>Voix système</label>
@@ -323,10 +492,12 @@ export function SettingsDrawer({ onClose }: Props) {
<div className="row" style={{ marginTop: 10 }}> <div className="row" style={{ marginTop: 10 }}>
<button className="btn small" onClick={testTts} disabled={settings.speech.provider === 'off'}><Volume2 size={13} /> Tester la voix</button> <button className="btn small" onClick={testTts} disabled={settings.speech.provider === 'off'}><Volume2 size={13} /> Tester la voix</button>
</div> </div>
<div style={{ marginTop: 8 }}><Result t={ttsTest} /></div> <div style={{ marginTop: 8 }}><TestResult t={ttsTest} /></div>
</div> </div>
)} )}
{section === 'models' && <ModelsSection />}
{section === 'webhook' && ( {section === 'webhook' && (
<div className="card"> <div className="card">
<Toggle on={settings.webhook.enabled} onChange={(v) => update({ webhook: { enabled: v } })} label="Serveur webhook local" hint="Hermes (crons, gateway, scripts) peut pousser des messages vers EveFlow." /> <Toggle on={settings.webhook.enabled} onChange={(v) => update({ webhook: { enabled: v } })} label="Serveur webhook local" hint="Hermes (crons, gateway, scripts) peut pousser des messages vers EveFlow." />
@@ -350,13 +521,50 @@ export function SettingsDrawer({ onClose }: Props) {
-d '{"role":"assistant","text":"Rapport terminé","source":"telegram"}'`}</pre> -d '{"role":"assistant","text":"Rapport terminé","source":"telegram"}'`}</pre>
<span className="hint">Formats acceptés : {'{role,text}'}, {'{event:"run.completed",input,output}'}, {'{event:"job.completed",job:{name},output,status}'}, {'{type:"message",payload:{text}}'}.</span> <span className="hint">Formats acceptés : {'{role,text}'}, {'{event:"run.completed",input,output}'}, {'{event:"job.completed",job:{name},output,status}'}, {'{type:"message",payload:{text}}'}.</span>
</div> </div>
<div className="field">
<label>Serveur MCP pour Hermes (outils du PC : écran, applications, volume, presse-papiers, voix)</label>
<pre className="input" style={{ whiteSpace: 'pre-wrap', fontSize: 11.5, userSelect: 'text' }}>{`# ~/.hermes/config.yaml côté Hermes
mcp_servers:
eveflow:
url: "http://<ip-de-ce-pc>:${settings.webhook.port}/mcp"${settings.webhook.secret ? '\n headers:\n Authorization: "Bearer <secret>"' : ''}`}</pre>
<span className="hint">Même port et même secret que le webhook. Sans secret, EveFlow n’écoute qu’en local (127.0.0.1) : définissez un secret pour un Hermes distant.</span>
</div>
<div className="row"> <div className="row">
<button className="btn primary small" onClick={() => void applyWebhook()} disabled={!bridge()}>Appliquer et redémarrer</button> <button className="btn primary small" onClick={() => void applyWebhook()} disabled={!bridge()}>Appliquer et redémarrer</button>
{hermesStore.webhook && <span className={`status-pill ${hermesStore.webhook.listening ? 'online' : 'offline'}`}>{hermesStore.webhook.listening ? `port ${hermesStore.webhook.port}` : hermesStore.webhook.error ?? 'inactif'}</span>} {hermesWebhook && <span className={`status-pill ${hermesWebhook.listening ? 'online' : 'offline'}`}>{hermesWebhook.listening ? `port ${hermesWebhook.port}` : hermesWebhook.error ?? 'inactif'}</span>}
</div> </div>
</div> </div>
)} )}
{section === 'notifications' && (
<div className="card">
<Toggle on={settings.notifications.quietEnabled} onChange={(v) => update({ notifications: { quietEnabled: v } })} label="Heures calmes" hint="Les messages poussés (crons, Telegram) s’affichent sans être lus à voix haute ni faire clignoter le noyau ; le HUD passe en mode nuit." />
<div className="grid-2">
<div className="field">
<label>Début</label>
<input className="input" type="time" value={settings.notifications.quietStart} onChange={(e) => update({ notifications: { quietStart: e.target.value } })} />
</div>
<div className="field">
<label>Fin</label>
<input className="input" type="time" value={settings.notifications.quietEnd} onChange={(e) => update({ notifications: { quietEnd: e.target.value } })} />
</div>
</div>
<Toggle on={settings.notifications.nightTheme} onChange={(v) => update({ notifications: { nightTheme: v } })} label="Thème nuit pendant les heures calmes" />
<div className="field" style={{ marginTop: 10 }}>
<label>Mots prioritaires (lus même en heures calmes)</label>
<input className="input" value={settings.notifications.priorityKeywords} onChange={(e) => update({ notifications: { priorityKeywords: e.target.value } })} placeholder="urgent, alerte, panne" />
<span className="hint">Séparés par des virgules ; comparés au texte et au nom du cron. Les échecs de crons sont toujours prioritaires.</span>
</div>
<Toggle on={settings.notifications.summarizeIncoming} onChange={(v) => update({ notifications: { summarizeIncoming: v } })} label="Résumé vocal des messages entrants" hint="Seules les premières phrases sont lues ; le message complet reste dans le fil." />
{settings.notifications.summarizeIncoming && (
<div className="field">
<label>Phrases lues : {settings.notifications.summarySentences}</label>
<input className="range" type="range" min={1} max={5} step={1} value={settings.notifications.summarySentences} onChange={(e) => update({ notifications: { summarySentences: Number(e.target.value) } })} />
</div>
)}
</div>
)}
{section === 'general' && ( {section === 'general' && (
<div className="card"> <div className="card">
<div className="grid-2"> <div className="grid-2">
@@ -374,13 +582,13 @@ export function SettingsDrawer({ onClose }: Props) {
<div className="theme-swatches"> <div className="theme-swatches">
{(['arc', 'gold', 'crimson', 'emerald'] as HudTheme[]).map((t) => { {(['arc', 'gold', 'crimson', 'emerald'] as HudTheme[]).map((t) => {
const colors: Record<HudTheme, string> = { arc: '#34e4ff', gold: '#ffc35a', crimson: '#ff5c6c', emerald: '#4ef0a8' }; const colors: Record<HudTheme, string> = { arc: '#34e4ff', gold: '#ffc35a', crimson: '#ff5c6c', emerald: '#4ef0a8' };
return <button key={t} className={`swatch${settings.theme === t ? ' active' : ''}`} style={{ background: colors[t] }} title={t} onClick={() => update({ theme: t })} />; return <button key={t} className={`swatch${settings.theme === t ? ' active' : ''}`} style={{ background: colors[t] }} title={t} aria-label={`Thème ${t}`} aria-pressed={settings.theme === t} onClick={() => update({ theme: t })} />;
})} })}
</div> </div>
</div> </div>
<div className="field"> <div className="field">
<label>Raccourcis globaux</label> <label>Raccourcis globaux</label>
<span className="hint"><span className="kbd">Ctrl+Shift+Espace</span> micro · <span className="kbd">Ctrl+Shift+J</span> afficher/masquer · <span className="kbd">Ctrl+Shift+Échap</span> couper la voix</span> <span className="hint"><span className="kbd">Ctrl+Shift+Espace</span> micro · <span className="kbd">Ctrl+Shift+J</span> afficher/masquer · <span className="kbd">Ctrl+Alt+Échap</span> couper la voix · <span className="kbd">Ctrl+K</span> saisie</span>
</div> </div>
<div className="divider" /> <div className="divider" />
<button className="btn danger small" onClick={() => { if (confirm('Réinitialiser tous les paramètres ?')) reset(); }}><RotateCcw size={13} /> Réinitialiser</button> <button className="btn danger small" onClick={() => { if (confirm('Réinitialiser tous les paramètres ?')) reset(); }}><RotateCcw size={13} /> Réinitialiser</button>
+6 -3
View File
@@ -4,6 +4,8 @@ import { bridge } from '../lib/bridge';
export interface LiveMetrics extends SystemMetrics { export interface LiveMetrics extends SystemMetrics {
fps: number; fps: number;
/** false until the main process delivered real measurements */
live: boolean;
} }
const EMPTY: LiveMetrics = { const EMPTY: LiveMetrics = {
@@ -16,7 +18,8 @@ const EMPTY: LiveMetrics = {
memUsedPct: 0, memUsedPct: 0,
memTotalGb: 0, memTotalGb: 0,
uptimeSec: 0, uptimeSec: 0,
fps: 60 fps: 60,
live: false
}; };
export function useMetrics(enabled = true, intervalMs = 2000): LiveMetrics { export function useMetrics(enabled = true, intervalMs = 2000): LiveMetrics {
@@ -43,7 +46,7 @@ export function useMetrics(enabled = true, intervalMs = 2000): LiveMetrics {
const api = bridge(); const api = bridge();
const poll = () => { const poll = () => {
if (!api) return; if (!api) return;
api.system.metrics().then((m) => setMetrics((prev) => ({ ...prev, ...m }))).catch(() => undefined); api.system.metrics().then((m) => setMetrics((prev) => ({ ...prev, ...m, live: true }))).catch(() => undefined);
}; };
poll(); poll();
const timer = setInterval(poll, intervalMs); const timer = setInterval(poll, intervalMs);
@@ -59,7 +62,7 @@ export function useMetrics(enabled = true, intervalMs = 2000): LiveMetrics {
export function useClock(): Date { export function useClock(): Date {
const [now, setNow] = useState(() => new Date()); const [now, setNow] = useState(() => new Date());
useEffect(() => { useEffect(() => {
const timer = setInterval(() => setNow(new Date()), 1000); const timer = setInterval(() => setNow(new Date()), 15_000);
return () => clearInterval(timer); return () => clearInterval(timer);
}, []); }, []);
return now; return now;
+43
View File
@@ -0,0 +1,43 @@
/** Quiet-hours and spoken-summary helpers (pure, unit tested). */
function minutesOf(hhmm: string): number | null {
const m = /^(\d{1,2}):(\d{2})$/.exec(hhmm.trim());
if (!m) return null;
const h = Number(m[1]);
const min = Number(m[2]);
if (h > 23 || min > 59) return null;
return h * 60 + min;
}
/** True when `now` falls inside [start, end), with ranges crossing midnight ("22:30" → "07:30"). */
export function isQuietTime(start: string, end: string, now: Date = new Date()): boolean {
const s = minutesOf(start);
const e = minutesOf(end);
if (s === null || e === null || s === e) return false;
const cur = now.getHours() * 60 + now.getMinutes();
return s < e ? cur >= s && cur < e : cur >= s || cur < e;
}
/** True when the text or job name contains one of the comma-separated priority words. */
export function isPriority(text: string, keywords: string, jobName = ''): boolean {
const words = keywords
.split(/[,;\n]/)
.map((w) => w.trim().toLowerCase())
.filter(Boolean);
if (!words.length) return false;
const hay = `${jobName} ${text}`.toLowerCase();
return words.some((w) => hay.includes(w));
}
/** First `count` sentences of a text, for a short spoken summary of a long push. */
export function summarize(text: string, count: number): string {
const clean = text
.replace(/```[\s\S]*?```/g, ' ')
.replace(/[#*_>`|]/g, '')
.replace(/\s+/g, ' ')
.trim();
if (!clean) return '';
const sentences = clean.match(/[^.!?…]+[.!?…]+["»)]?\s*|[^.!?…]+$/g) ?? [clean];
const picked = sentences.slice(0, Math.max(1, count)).join('').trim();
return picked.length < clean.length ? picked : clean;
}
+2
View File
@@ -20,6 +20,8 @@ export class SseParser {
this.buffer += chunk; this.buffer += chunk;
let index: number; let index: number;
while ((index = this.buffer.search(/\r\n|\n|\r/)) !== -1) { while ((index = this.buffer.search(/\r\n|\n|\r/)) !== -1) {
// A trailing "\r" may be the first half of a "\r\n" split across chunks: wait for more data.
if (this.buffer[index] === '\r' && index === this.buffer.length - 1) break;
const line = this.buffer.slice(0, index); const line = this.buffer.slice(0, index);
const sepLength = this.buffer.startsWith('\r\n', index) ? 2 : 1; const sepLength = this.buffer.startsWith('\r\n', index) ? 2 : 1;
this.buffer = this.buffer.slice(index + sepLength); this.buffer = this.buffer.slice(index + sepLength);
+30 -4
View File
@@ -65,12 +65,14 @@ export function cleanForSpeech(raw: string): string {
return applyPhonetics(text); return applyPhonetics(text);
} }
const PHONETIC_RULES: Array<[RegExp, string]> = Object.entries(PHONETIC).map(([key, value]) => {
const escaped = key.replace(/[-/\\^$*+?.()|[\]{}]/g, '\\$&');
return [new RegExp(`(?<=^|\\s|\\p{P})${escaped}(?=$|\\s|\\p{P})`, 'giu'), value];
});
function applyPhonetics(text: string): string { function applyPhonetics(text: string): string {
let out = text; let out = text;
for (const [key, value] of Object.entries(PHONETIC)) { for (const [regex, value] of PHONETIC_RULES) out = out.replace(regex, value);
const escaped = key.replace(/[-/\\^$*+?.()|[\]{}]/g, '\\$&');
out = out.replace(new RegExp(`(?<=^|\\s|\\p{P})${escaped}(?=$|\\s|\\p{P})`, 'giu'), value);
}
return out; return out;
} }
@@ -152,3 +154,27 @@ export function formatDuration(seconds: number): string {
if (h > 0) return `${h}h ${m.toString().padStart(2, '0')}m`; if (h > 0) return `${h}h ${m.toString().padStart(2, '0')}m`;
return `${m}m ${(s % 60).toString().padStart(2, '0')}s`; return `${m}m ${(s % 60).toString().padStart(2, '0')}s`;
} }
const NOISE_PHRASES = [
'sous-titres réalisés par la communauté d\'amara.org',
'sous-titrage société radio-canada',
'merci d\'avoir regardé',
'abonnez-vous',
'thank you for watching',
'thanks for watching',
'...'
];
/**
* Whisper-style hallucinations on silence or clicks: "(cliquant)", "*Claire*", "[Musique]",
* "Sous-titres réalisés par…". Such transcripts must not be sent to Hermes.
*/
export function isTranscriptNoise(text: string): boolean {
const t = text.trim();
if (!t) return true;
if (/^[\s\p{P}\p{S}]*$/u.test(t)) return true;
// whole transcript wrapped in brackets/asterisks: a sound description
if (/^[(\[*«"'\s]+[^()\[\]*]{0,60}[)\]*»"'\s]+$/u.test(t) && !/[a-zà-ÿ]{3,}\s+[a-zà-ÿ]{3,}\s+[a-zà-ÿ]{3,}/i.test(t)) return true;
const lower = t.toLowerCase().replace(/[.!?…\s]+$/u, '');
return NOISE_PHRASES.some((p) => lower === p.replace(/[.!?…\s]+$/u, ''));
}
-5
View File
@@ -130,8 +130,3 @@ export async function httpStream(req: HttpProxyRequest, handlers: StreamHandlers
return { start, done, abort: () => controller.abort() }; return { start, done, abort: () => controller.abort() };
} }
/** Collects the whole body of a streaming response (used to read error payloads). */
export async function collectStreamBody(handle: StreamHandle, first: string[]): Promise<string> {
await handle.done.catch(() => undefined);
return first.join('');
}
+66
View File
@@ -0,0 +1,66 @@
/** Voice gender preference applied to every TTS provider (pure helpers, unit tested). */
import type { VoiceModelStatus } from '../../shared/voice';
export type VoiceGender = 'male' | 'female';
const MALE_HINTS = /\b(homme|male|masculin|paul|thomas|claude|henri|guillaume|mathieu|antoine|nicolas|denis|pierre|tom|adam|michael|eric|liam|george|lewis|daniel|fenrir|puck|onyx|echo|david|mark|richard|james|ryan|guy)\b/i;
const FEMALE_HINTS = /\b(femme|female|f[ée]minin|hortense|julie|denise|eloise|am[ée]lie|audrey|siwis|jessica|heart|bella|sarah|nicole|sky|alloy|nova|shimmer|zira|aria|jenny|emma|isabella|sophie|charlotte|vivienne|coral|sage)\b/i;
/** Best-effort gender from a voice or speaker label ("Piper Tom (homme, français)", "Microsoft Paul", "am_adam"). */
export function inferGender(name: string): VoiceGender | undefined {
const n = name.replace(/_/g, ' ');
if (/^(am|bm|em|hm|im|jm|pm|zm)\b/i.test(n) || MALE_HINTS.test(n)) return 'male';
if (/^(af|bf|ef|ff|hf|if|jf|pf|zf)\b/i.test(n) || FEMALE_HINTS.test(n)) return 'female';
return undefined;
}
/** OpenAI-compatible default voice for the preferred gender (used when the user left the field empty). */
export function defaultOpenAiVoice(gender: VoiceGender): string {
return gender === 'male' ? 'onyx' : 'nova';
}
/** Rank system voices: language first, then gender, then quality hints. */
export function rankSystemVoice(v: { name: string; lang: string; localService?: boolean }, lang: string, gender: VoiceGender): number {
const name = v.name.toLowerCase();
let score = v.lang.toLowerCase().startsWith(lang) ? 100 : 0;
const g = inferGender(v.name);
if (g === gender) score += 50;
else if (g && g !== gender) score -= 30;
if (name.includes('natural') || name.includes('neural') || name.includes('online')) score += 20;
if (name.includes('google')) score += 10;
if (name.includes('microsoft')) score += 5;
if (v.localService) score += 2;
return score;
}
export interface LocalVoiceChoice {
modelId: string;
speaker: number;
}
/** Installed local voice matching language + gender, or null. */
export function findLocalVoice(models: VoiceModelStatus[], lang: string, gender: VoiceGender): LocalVoiceChoice | null {
const l = lang.toLowerCase().split('-')[0];
const installed = models.filter((m) => m.kind === 'tts' && m.installed);
for (const model of installed) {
for (const sp of model.speakers ?? []) {
const spLang = (sp.lang || '').toLowerCase();
if (spLang && spLang !== l && spLang !== 'multi') continue;
const g = (sp as { gender?: string }).gender === 'm' ? 'male' : (sp as { gender?: string }).gender === 'f' ? 'female' : inferGender(sp.name);
if (g === gender) return { modelId: model.id, speaker: sp.id };
}
}
return null;
}
/** Catalog model to download when nothing installed offers the preferred gender (French). */
export function suggestedDownload(models: VoiceModelStatus[], lang: string, gender: VoiceGender): string | null {
const l = lang.toLowerCase().split('-')[0];
if (l !== 'fr') return null;
const wanted = gender === 'male' ? ['piper-fr-tom', 'piper-fr-upmc'] : ['kokoro-v1', 'piper-fr-siwis'];
for (const id of wanted) {
const m = models.find((x) => x.id === id);
if (m && !m.installed) return id;
}
return null;
}
+8
View File
@@ -1,6 +1,14 @@
import React from 'react'; import React from 'react';
import ReactDOM from 'react-dom/client'; import ReactDOM from 'react-dom/client';
import App from './App'; import App from './App';
import '@fontsource/orbitron/500.css';
import '@fontsource/orbitron/700.css';
import '@fontsource/orbitron/900.css';
import '@fontsource/rajdhani/400.css';
import '@fontsource/rajdhani/500.css';
import '@fontsource/rajdhani/600.css';
import '@fontsource/rajdhani/700.css';
import '@fontsource/share-tech-mono/400.css';
import './styles/tokens.css'; import './styles/tokens.css';
import './styles/app.css'; import './styles/app.css';
import './styles/chat.css'; import './styles/chat.css';
+134 -28
View File
@@ -1,7 +1,7 @@
/** /**
* Conversation orchestrator: sends user input to Hermes through the resolved transport, * Conversation orchestrator: sends user input to Hermes through the resolved transport,
* streams the answer into the chat store, drives the HUD state, speaks the reply and * streams the answer into the chat store (coalesced per animation frame), drives the HUD
* surfaces approvals / clarifications as pending requests. * state, speaks the reply and surfaces approvals / clarifications as pending requests.
*/ */
import type { HermesPushEvent } from '../../shared/ipc'; import type { HermesPushEvent } from '../../shared/ipc';
import { Log } from '../lib/log'; import { Log } from '../lib/log';
@@ -9,16 +9,47 @@ import { previewText } from '../lib/text';
import { useChat, type PendingRequest } from '../state/chat'; import { useChat, type PendingRequest } from '../state/chat';
import { useHermes } from '../state/hermes'; import { useHermes } from '../state/hermes';
import { useSettings } from '../state/settings'; import { useSettings } from '../state/settings';
import { executeLocalTool, LOCAL_TOOL_DEFINITIONS } from './hermes/localTools'; import { executeLocalTool, LOCAL_TOOL_DEFINITIONS, type LocalToolContext } from './hermes/localTools';
import { isPriority, summarize } from '../lib/quietHours';
import type { HermesStreamEvent, SendHandle } from './hermes/types'; import type { HermesStreamEvent, SendHandle } from './hermes/types';
import { speech } from './voice/speech'; import { speech } from './voice/speech';
import { bridge } from '../lib/bridge';
import { parseLocalIntent, runLocalIntent } from './localCommands';
let active: SendHandle | null = null; let active: SendHandle | null = null;
let activeMessageId: string | null = null; let activeMessageId: string | null = null;
const toolLabel = (name: string) => name.replace(/_/g, ' '); const toolLabel = (name: string) => name.replace(/_/g, ' ');
function localToolContext() { // ── delta coalescing: one store update per animation frame ─────────────────
const pendingDelta = { id: '', content: '', reasoning: '', raf: 0 };
function flushDelta(): void {
if (pendingDelta.raf) cancelAnimationFrame(pendingDelta.raf);
const { id, content, reasoning } = pendingDelta;
pendingDelta.id = '';
pendingDelta.content = '';
pendingDelta.reasoning = '';
pendingDelta.raf = 0;
if (!id) return;
const chat = useChat.getState();
if (content) chat.appendToMessage(id, content);
if (reasoning) chat.appendToMessage(id, reasoning, 'reasoning');
}
function queueDelta(id: string, text: string, field: 'content' | 'reasoning'): void {
if (pendingDelta.id && pendingDelta.id !== id) flushDelta();
pendingDelta.id = id;
pendingDelta[field] += text;
if (!pendingDelta.raf) pendingDelta.raf = requestAnimationFrame(flushDelta);
}
/** Bumps a counter the core reads to flash briefly (send, tool start, speech start). */
export function pingCore(): void {
useChat.getState().ping();
}
export function localToolContext(): LocalToolContext {
const chat = useChat.getState(); const chat = useChat.getState();
const settings = useSettings.getState().settings; const settings = useSettings.getState().settings;
return { return {
@@ -30,13 +61,20 @@ function localToolContext() {
setTimeout(() => useChat.getState().setHudOverride(null), 6000); setTimeout(() => useChat.getState().setHudOverride(null), 6000);
}, },
speak: (text: string) => speech.say(text), speak: (text: string) => speech.say(text),
showMessage: (text: string, title?: string) => {
if (!text.trim()) return;
useChat.getState().addMessage({ role: 'assistant', content: text, source: 'hermes', jobName: title, status: 'done' });
},
getStatus: () => ({ getStatus: () => ({
transport: useHermes.getState().transport, transport: useHermes.getState().transport,
link: useHermes.getState().link, link: useHermes.getState().link,
assistant: settings.assistantName, assistant: settings.assistantName,
messages: chat.messages.length, messages: chat.messages.length,
speaking: speech.isSpeaking(), speaking: speech.isSpeaking(),
handsFree: settings.voice.handsFree handsFree: settings.voice.handsFree,
quietHours: chat.quiet,
missionMode: chat.missionMode,
wakeMode: settings.voice.wakeMode
}), }),
getHistory: (n: number) => chat.messages.slice(-n).map((m) => ({ role: m.role, content: m.content })), getHistory: (n: number) => chat.messages.slice(-n).map((m) => ({ role: m.role, content: m.content })),
notify: (title: string, body: string) => { notify: (title: string, body: string) => {
@@ -49,7 +87,7 @@ function localToolContext() {
}; };
} }
function handleEvent(event: HermesStreamEvent, messageId: string, runIdRef: { id: string | null }): void { function handleEvent(event: HermesStreamEvent, messageId: string, runIdRef: { id: string | null }, timing: { firstToken: number | null; startedAt: number }): void {
const chat = useChat.getState(); const chat = useChat.getState();
switch (event.kind) { switch (event.kind) {
case 'run.started': case 'run.started':
@@ -57,15 +95,20 @@ function handleEvent(event: HermesStreamEvent, messageId: string, runIdRef: { id
chat.setSending(true, event.runId); chat.setSending(true, event.runId);
break; break;
case 'delta': case 'delta':
chat.appendToMessage(messageId, event.text); if (timing.firstToken === null) {
timing.firstToken = Date.now() - timing.startedAt;
chat.setLatency(timing.firstToken);
}
queueDelta(messageId, event.text, 'content');
speech.pushStream(event.text); speech.pushStream(event.text);
break; break;
case 'reasoning': case 'reasoning':
chat.appendToMessage(messageId, event.text, 'reasoning'); queueDelta(messageId, event.text, 'reasoning');
break; break;
case 'tool.start': case 'tool.start':
chat.pushActivity({ id: event.id, kind: 'tool', name: event.name, status: 'running', detail: event.args ? previewText(event.args, 160) : undefined }); chat.pushActivity({ id: event.id, kind: 'tool', name: event.name, status: 'running', detail: event.args ? previewText(event.args, 160) : undefined });
chat.setHud('thinking'); chat.setHud('thinking');
pingCore();
break; break;
case 'tool.progress': case 'tool.progress':
if (!chat.activity.some((a) => (event.id && a.id === event.id) || (a.name === event.name && a.status === 'running'))) { if (!chat.activity.some((a) => (event.id && a.id === event.id) || (a.name === event.name && a.status === 'running'))) {
@@ -113,6 +156,7 @@ function handleEvent(event: HermesStreamEvent, messageId: string, runIdRef: { id
break; break;
case 'completed': case 'completed':
if (event.usage) chat.updateMessage(messageId, { usage: event.usage }); if (event.usage) chat.updateMessage(messageId, { usage: event.usage });
if (event.status === 'length') chat.updateMessage(messageId, { status: 'cancelled' });
break; break;
case 'error': case 'error':
chat.setError(event.message); chat.setError(event.message);
@@ -127,18 +171,60 @@ function addPending(request: Omit<PendingRequest, 'id' | 'createdAt'>): void {
const chat = useChat.getState(); const chat = useChat.getState();
chat.addPending(request); chat.addPending(request);
chat.setHud('alert'); chat.setHud('alert');
speech.say(request.kind === 'approval' ? 'Autorisation requise.' : request.kind === 'clarify' ? request.description : 'Saisie requise.'); // Spoken without interrupting the streamed answer.
speech.say(request.kind === 'approval' ? 'Autorisation requise.' : request.kind === 'clarify' ? request.description : 'Saisie requise.', { interrupt: false });
}
/** Screenshot of the primary display as a data URL (Electron only). */
export async function captureScreen(): Promise<string | null> {
const api = bridge();
if (!api) return null;
try {
return await api.system.captureScreen(1600);
} catch (err) {
useChat.getState().setError(`Capture d’écran : ${(err as Error).message}`);
return null;
}
}
/** Short system intents handled on the machine without a Hermes round trip. Returns true when consumed. */
async function tryLocalIntent(text: string, source: string): Promise<{ handled: boolean; images?: string[]; text?: string }> {
const settings = useSettings.getState().settings;
if (!settings.voice.localCommands || !bridge()) return { handled: false };
const intent = parseLocalIntent(text);
if (!intent) return { handled: false };
if (intent.kind === 'screenshot') {
const shot = await captureScreen();
if (!shot) return { handled: false };
return { handled: false, images: [shot], text: intent.question || text };
}
const chat = useChat.getState();
chat.addMessage({ role: 'user', content: text, source, status: 'done' });
const result = await runLocalIntent(intent);
chat.addMessage({ role: 'assistant', content: result.message, source: 'local', status: result.ok ? 'done' : 'error' });
chat.setHud(result.ok ? 'idle' : 'error');
if (settings.speech.autoSpeak) speech.say(result.message, { interrupt: true });
return { handled: true };
} }
export async function sendMessage(text: string, images: string[] = [], source = 'eveflow'): Promise<void> { export async function sendMessage(text: string, images: string[] = [], source = 'eveflow'): Promise<void> {
const trimmed = text.trim(); let trimmed = text.trim();
if ((!trimmed && images.length === 0) || active) return; if ((!trimmed && images.length === 0) || active) return;
if (trimmed && images.length === 0) {
const local = await tryLocalIntent(trimmed, source);
if (local.handled) return;
if (local.images) {
images = local.images;
trimmed = (local.text ?? trimmed).trim();
}
}
const chat = useChat.getState(); const chat = useChat.getState();
const hermes = useHermes.getState(); const hermes = useHermes.getState();
const settings = useSettings.getState().settings; const settings = useSettings.getState().settings;
speech.stop(); speech.stop();
chat.setError(null); chat.setError(null);
chat.setLatency(null);
chat.addMessage({ role: 'user', content: trimmed || '(image)', images: images.length ? images : undefined, source, status: 'done' }); chat.addMessage({ role: 'user', content: trimmed || '(image)', images: images.length ? images : undefined, source, status: 'done' });
const history = chat.messages const history = chat.messages
.filter((m) => m.role !== 'system' && m.status !== 'error') .filter((m) => m.role !== 'system' && m.status !== 'error')
@@ -148,17 +234,21 @@ export async function sendMessage(text: string, images: string[] = [], source =
activeMessageId = messageId; activeMessageId = messageId;
chat.setSending(true); chat.setSending(true);
chat.setHud('thinking'); chat.setHud('thinking');
pingCore();
const startedAt = Date.now(); const startedAt = Date.now();
const timing = { firstToken: null as number | null, startedAt };
const runIdRef = { id: null as string | null }; const runIdRef = { id: null as string | null };
const client = hermes.client(); const mission = chat.missionMode && settings.hermes.missionModel.trim();
const client = hermes.client(mission ? settings.hermes.missionModel : undefined);
if (mission) Log.info('hermes', `mission mode → model ${settings.hermes.missionModel}`);
const handle = client.send( const handle = client.send(
{ {
text: trimmed || 'Analyse cette image.', text: trimmed || 'Analyse cette image.',
images, images,
history, history,
sessionId: settings.hermesSessionId, sessionId: settings.hermesSessionId,
onEvent: (event) => handleEvent(event, messageId, runIdRef), onEvent: (event) => handleEvent(event, messageId, runIdRef, timing),
localToolDefinitions: LOCAL_TOOL_DEFINITIONS, localToolDefinitions: LOCAL_TOOL_DEFINITIONS,
localToolExecutor: (name, args) => executeLocalTool(name, args, localToolContext()) localToolExecutor: (name, args) => executeLocalTool(name, args, localToolContext())
}, },
@@ -169,20 +259,26 @@ export async function sendMessage(text: string, images: string[] = [], source =
try { try {
const reply = await handle.result; const reply = await handle.result;
const current = useChat.getState().messages.find((m) => m.id === messageId); flushDelta();
const finalText = reply || current?.content || ''; const state = useChat.getState();
useChat.getState().updateMessage(messageId, { content: finalText, status: 'done' }); const current = state.messages.find((m) => m.id === messageId);
// Anything not streamed (non-delta transports) is spoken now. const streamedText = current?.content ?? '';
if (settings.speech.autoSpeak) { const finalText = handle.aborted ? streamedText : reply || streamedText;
if (!current?.content && finalText) speech.say(finalText); const truncated = current?.status === 'cancelled';
state.updateMessage(messageId, { content: finalText || (handle.aborted ? '(interrompu)' : ''), status: handle.aborted || truncated ? 'cancelled' : 'done' });
if (handle.aborted) {
speech.discardStream();
} else if (settings.speech.autoSpeak) {
if (!streamedText && finalText) speech.say(finalText);
else speech.endStream(); else speech.endStream();
} else { } else {
speech.discardStream(); speech.discardStream();
} }
useChat.getState().setHud(speech.isSpeaking() ? 'speaking' : 'idle'); state.setHud(speech.isSpeaking() ? 'speaking' : 'idle');
if (useHermes.getState().link === 'offline') useHermes.getState().connect().catch(() => undefined); if (!handle.aborted && useHermes.getState().link === 'offline') useHermes.getState().connect().catch(() => undefined);
Log.info('hermes', `reply in ${Date.now() - startedAt} ms via ${handle.transport}`, { chars: finalText.length }); Log.info('hermes', `${handle.aborted ? 'aborted' : 'reply'} in ${Date.now() - startedAt} ms via ${handle.transport}`, { chars: finalText.length, firstToken: timing.firstToken });
} catch (err) { } catch (err) {
flushDelta();
const message = (err as Error).message || String(err); const message = (err as Error).message || String(err);
const partial = useChat.getState().messages.find((m) => m.id === messageId)?.content ?? ''; const partial = useChat.getState().messages.find((m) => m.id === messageId)?.content ?? '';
useChat.getState().updateMessage(messageId, { useChat.getState().updateMessage(messageId, {
@@ -200,10 +296,14 @@ export async function sendMessage(text: string, images: string[] = [], source =
} finally { } finally {
active = null; active = null;
activeMessageId = null; activeMessageId = null;
useChat.getState().setSending(false); const state = useChat.getState();
for (const a of useChat.getState().activity) { state.setSending(false);
if (a.status === 'running') useChat.getState().finishActivity({ id: a.id }, { status: 'done' }); state.setLatency(null);
for (const a of state.activity) {
if (a.status === 'running') state.finishActivity({ id: a.id }, { status: 'done' });
} }
// Requests bound to this run cannot be answered once it is over.
for (const p of state.pending) if (!runIdRef.id || p.runId === runIdRef.id) state.removePending(p.id);
} }
} }
@@ -235,14 +335,13 @@ export async function resolvePending(request: PendingRequest, decision: string):
chat.removePending(request.id); chat.removePending(request.id);
const client = useHermes.getState().client(); const client = useHermes.getState().client();
try { try {
if (!request.runId) throw new Error('run inconnu');
if (request.kind === 'approval') { if (request.kind === 'approval') {
const choice = (['once', 'session', 'always', 'deny'].includes(decision) ? decision : decision === 'approve' ? 'once' : 'deny') as 'once' | 'session' | 'always' | 'deny'; const choice = (['once', 'session', 'always', 'deny'].includes(decision) ? decision : decision === 'approve' ? 'once' : 'deny') as 'once' | 'session' | 'always' | 'deny';
if (!request.runId) throw new Error('run inconnu');
await client.approveRun(request.runId, choice, request.requestId || undefined); await client.approveRun(request.runId, choice, request.requestId || undefined);
chat.addMessage({ role: 'system', content: `Autorisation ${choice === 'deny' ? 'refusée' : 'accordée'}${request.tool ? ` (${toolLabel(request.tool)})` : ''}.`, status: 'done' }); chat.addMessage({ role: 'system', content: `Autorisation ${choice === 'deny' ? 'refusée' : 'accordée'}${request.tool ? ` (${toolLabel(request.tool)})` : ''}.`, status: 'done' });
} else { } else {
// Clarifications and secrets are answered by steering the run with the reply. // Clarifications and secrets are answered by steering the run with the reply.
if (!request.runId) throw new Error('run inconnu');
await client.steerRun(request.runId, decision); await client.steerRun(request.runId, decision);
chat.addMessage({ role: 'system', content: request.secret ? 'Saisie transmise à Hermes.' : `Réponse transmise : ${decision}`, status: 'done' }); chat.addMessage({ role: 'system', content: request.secret ? 'Saisie transmise à Hermes.' : `Réponse transmise : ${decision}`, status: 'done' });
} }
@@ -274,10 +373,17 @@ export function handlePush(event: HermesPushEvent): void {
chat.pushActivity({ kind: 'job', name: event.jobName || 'cron', status: event.status?.includes('fail') ? 'error' : 'done', detail: previewText(event.text, 160) }); chat.pushActivity({ kind: 'job', name: event.jobName || 'cron', status: event.status?.includes('fail') ? 'error' : 'done', detail: previewText(event.text, 160) });
void useHermes.getState().refreshJobs(); void useHermes.getState().refreshJobs();
} }
if (event.role !== 'user' && settings.speech.speakIncoming) { const n = settings.notifications;
speech.say(event.jobName ? `Résultat de ${event.jobName}. ${event.text}` : event.text); const priority = isPriority(event.text, n.priorityKeywords, event.jobName) || !!event.status?.includes('fail');
const silenced = chat.quiet && !priority;
if (document.hidden || silenced) chat.incUnread();
if (event.role !== 'user' && settings.speech.speakIncoming && !silenced) {
const body = n.summarizeIncoming ? summarize(event.text, n.summarySentences) : event.text;
speech.say(event.jobName ? `Résultat de ${event.jobName}. ${body}` : body);
} }
if (silenced) Log.info('webhook', 'quiet hours: push shown silently');
chat.setHud(event.status?.includes('fail') ? 'alert' : 'success'); chat.setHud(event.status?.includes('fail') ? 'alert' : 'success');
if (!silenced) pingCore();
setTimeout(() => { setTimeout(() => {
const s = useChat.getState(); const s = useChat.getState();
if (s.hud === 'success' || s.hud === 'alert') s.setHud(speech.isSpeaking() ? 'speaking' : 'idle'); if (s.hud === 'success' || s.hud === 'alert') s.setHud(speech.isSpeaking() ? 'speaking' : 'idle');
+177 -26
View File
@@ -31,6 +31,75 @@ const isRec = (v: unknown): v is Rec => !!v && typeof v === 'object' && !Array.i
export type ResolvedTransport = Exclude<HermesTransport, 'auto'>; export type ResolvedTransport = Exclude<HermesTransport, 'auto'>;
/**
* A web page (login portal, dashboard, reverse-proxy error) instead of JSON means the URL does not
* point at the Hermes API. Returns a human explanation, or null when the body is not HTML.
*/
export function describeHtml(body: string): string | null {
const head = body.slice(0, 600).trimStart().toLowerCase();
if (!head.startsWith('<!doctype html') && !head.startsWith('<html') && !/^<\?xml[^>]*>\s*<html/.test(head)) return null;
const title = /<title[^>]*>([^<]{1,120})<\/title>/i.exec(body)?.[1]?.trim();
const login = /connecter|login|sign in|authentif|mot de passe|password/i.test(body.slice(0, 20_000));
return `Le serveur renvoie une page web${title ? ` « ${title} »` : ''} au lieu de l'API Hermes${login ? ' (page de connexion : l’URL passe par un portail web)' : ''}. Utilisez l'URL directe du serveur API Hermes (port 8642 par défaut, ou le chemin /v1 exposé par votre proxy).`;
}
/** Candidate API URLs derived from what the user typed (same host, other port or path). */
export function hermesUrlCandidates(url: string): string[] {
const base = hermesBaseUrl(url);
if (!base) return [];
const out = new Set<string>();
const add = (u: string) => out.add(u.replace(/\/+$/, ''));
try {
const u = new URL(base.includes('://') ? base : `http://${base}`);
const host = u.hostname;
const scheme = u.protocol.replace(':', '');
const path = u.pathname.replace(/\/+$/, '');
if (path) add(`${scheme}://${u.host}`);
for (const suffix of ['/api', '/hermes', '/hermes/api', '/v1', '/api/v1']) add(`${scheme}://${u.host}${path}${suffix}`);
if (!u.port) {
add(`${scheme}://${host}:8642`);
add(`http://${host}:8642`);
add(`https://${host}:8642`);
}
for (const sub of ['api', 'hermes-api']) {
if (!host.startsWith(`${sub}.`) && host.includes('.')) add(`${scheme}://${sub}.${host}`);
}
if (host.startsWith('jarvis.')) add(`${scheme}://hermes.${host.slice('jarvis.'.length)}`);
} catch {
return [];
}
out.delete(base);
return [...out];
}
/** True when this base URL answers like the Hermes API (JSON on /v1/*, or 401/403 = key required). */
export async function probeHermesUrl(config: HermesConfig, url: string): Promise<boolean> {
const client = new HermesClient({ ...config, url });
for (const path of ['/v1/capabilities', '/v1/models', '/health']) {
try {
const payload = await client.request<unknown>(path, { timeoutMs: 4000 });
if (payload && typeof payload === 'object') return true;
} catch (err) {
if (err instanceof HttpError) {
if (err.status === 401 || err.status === 403) return true; // the API is there, only the key is missing
if (/page web/.test(err.message)) return false; // a portal answers on this host: not the API
if (err.status === 404) continue; // older Hermes without this endpoint
}
return false;
}
}
return false;
}
/** Probe candidate URLs until one answers like the Hermes API. */
export async function discoverHermesUrl(config: HermesConfig, onProgress?: (url: string) => void): Promise<string | null> {
for (const candidate of hermesUrlCandidates(config.url)) {
onProgress?.(candidate);
if (await probeHermesUrl(config, candidate)) return candidate;
}
return null;
}
export function hermesBaseUrl(url: string): string { export function hermesBaseUrl(url: string): string {
let base = url.trim().replace(/\/+$/, ''); let base = url.trim().replace(/\/+$/, '');
base = base.replace(/\/chat\/completions$/, ''); base = base.replace(/\/chat\/completions$/, '');
@@ -93,7 +162,7 @@ export class HermesClient {
return h; return h;
} }
private async request<T>(path: string, init: { method?: 'GET' | 'POST' | 'PATCH' | 'DELETE'; body?: unknown; timeoutMs?: number } = {}): Promise<T> { async request<T>(path: string, init: { method?: 'GET' | 'POST' | 'PATCH' | 'DELETE'; body?: unknown; timeoutMs?: number } = {}): Promise<T> {
if (!this.config.url.trim()) throw new Error("URL Hermes non configurée."); if (!this.config.url.trim()) throw new Error("URL Hermes non configurée.");
const url = `${this.base}${path}`; const url = `${this.base}${path}`;
const res = await httpFetch({ const res = await httpFetch({
@@ -103,11 +172,13 @@ export class HermesClient {
body: init.body !== undefined ? JSON.stringify(init.body) : undefined, body: init.body !== undefined ? JSON.stringify(init.body) : undefined,
timeoutMs: init.timeoutMs ?? 20_000 timeoutMs: init.timeoutMs ?? 20_000
}); });
if (!res.ok) throw new HttpError(res.status, errorMessage(res.status, res.text ?? ''), res.text);
const text = res.text ?? ''; const text = res.text ?? '';
const html = describeHtml(text);
if (html) throw new HttpError(res.status, html, text);
if (!res.ok) throw new HttpError(res.status, errorMessage(res.status, res.text ?? ''), res.text);
if (!text.trim()) return null as T; if (!text.trim()) return null as T;
const parsed = tryParseJson<T>(text); const parsed = tryParseJson<T>(text);
if (parsed === null) throw new Error(`Réponse Hermes illisible depuis ${path}`); if (parsed === null) throw new Error(`Réponse Hermes illisible depuis ${path} : ${text.slice(0, 120).replace(/\s+/g, ' ')}`);
return parsed; return parsed;
} }
@@ -261,6 +332,7 @@ export class HermesClient {
for (const event of normalizeLifecycleEvent(eventName, data)) onEvent(event); for (const event of normalizeLifecycleEvent(eventName, data)) onEvent(event);
}); });
const errorChunks: string[] = []; const errorChunks: string[] = [];
let handleReady = false;
const handle = await httpStream( const handle = await httpStream(
{ {
url: `${this.base}${path}`, url: `${this.base}${path}`,
@@ -271,7 +343,6 @@ export class HermesClient {
}, },
{ onChunk: (text) => (handleReady ? parser.feed(text) : errorChunks.push(text)) } { onChunk: (text) => (handleReady ? parser.feed(text) : errorChunks.push(text)) }
); );
let handleReady = handle.start.ok;
if (!handle.start.ok) { if (!handle.start.ok) {
await handle.done.catch(() => undefined); await handle.done.catch(() => undefined);
throw new HttpError(handle.start.status, errorMessage(handle.start.status, errorChunks.join(''))); throw new HttpError(handle.start.status, errorMessage(handle.start.status, errorChunks.join('')));
@@ -285,25 +356,45 @@ export class HermesClient {
send(options: SendOptions, transport: ResolvedTransport): SendHandle { send(options: SendOptions, transport: ResolvedTransport): SendHandle {
const effective: ResolvedTransport = options.images?.length && transport === 'runs' ? 'completions' : transport; const effective: ResolvedTransport = options.images?.length && transport === 'runs' ? 'completions' : transport;
let abort: () => void = () => undefined; let aborted = false;
let abortImpl: () => void = () => undefined;
const setAbort = (fn: () => void) => {
abortImpl = fn;
if (aborted) fn();
};
const isAborted = () => aborted;
const result = const result =
effective === 'runs' effective === 'runs'
? this.sendViaRuns(options, (fn) => (abort = fn)) ? this.sendViaRuns(options, setAbort, isAborted)
: effective === 'sessions' : effective === 'sessions'
? this.sendViaSessions(options, (fn) => (abort = fn)) ? this.sendViaSessions(options, setAbort, isAborted)
: this.sendViaCompletions(options, (fn) => (abort = fn)); : this.sendViaCompletions(options, setAbort, isAborted);
return { result, abort: () => abort(), transport: effective }; return {
result,
transport: effective,
get aborted() {
return aborted;
},
abort: () => {
aborted = true;
abortImpl();
}
};
} }
private async sendViaRuns(options: SendOptions, setAbort: (fn: () => void) => void): Promise<string> { private async sendViaRuns(options: SendOptions, setAbort: (fn: () => void) => void, isAborted: () => boolean): Promise<string> {
const { onEvent } = options; const { onEvent } = options;
const run = await this.startRun({ const run = await this.startRun({
input: options.text, input: options.text,
session_id: options.sessionId || undefined, session_id: plainSession(options.sessionId) || undefined,
instructions: this.config.instructions || undefined, instructions: this.config.instructions || undefined,
model: this.config.model || undefined model: this.config.model || undefined
}); });
const runId = run.run_id; const runId = run.run_id;
if (isAborted()) {
this.stopRun(runId).catch(() => undefined);
return '';
}
onEvent({ kind: 'run.started', runId }); onEvent({ kind: 'run.started', runId });
Log.info('hermes', `run ${runId} started`); Log.info('hermes', `run ${runId} started`);
@@ -329,7 +420,7 @@ export class HermesClient {
}); });
await handle.done; await handle.done;
if (stopped) return streamedText; if (stopped || isAborted()) return streamedText;
if (failure) throw new Error(failure); if (failure) throw new Error(failure);
if (!completed) { if (!completed) {
@@ -349,7 +440,7 @@ export class HermesClient {
return finalText || streamedText; return finalText || streamedText;
} }
private async sendViaSessions(options: SendOptions, setAbort: (fn: () => void) => void): Promise<string> { private async sendViaSessions(options: SendOptions, setAbort: (fn: () => void) => void, isAborted: () => boolean): Promise<string> {
const { onEvent } = options; const { onEvent } = options;
let sessionId = options.sessionId; let sessionId = options.sessionId;
if (!sessionId || !sessionId.startsWith('hs:')) { if (!sessionId || !sessionId.startsWith('hs:')) {
@@ -357,6 +448,7 @@ export class HermesClient {
sessionId = `hs:${session.id}`; sessionId = `hs:${session.id}`;
onEvent({ kind: 'session', sessionId }); onEvent({ kind: 'session', sessionId });
} }
if (isAborted()) return '';
const realId = sessionId.slice(3); const realId = sessionId.slice(3);
const body: Rec = { input: options.text }; const body: Rec = { input: options.text };
if (this.config.instructions) body.instructions = this.config.instructions; if (this.config.instructions) body.instructions = this.config.instructions;
@@ -374,11 +466,12 @@ export class HermesClient {
}); });
setAbort(() => handle.abort()); setAbort(() => handle.abort());
await handle.done; await handle.done;
if (isAborted()) return streamed;
if (failure) throw new Error(failure); if (failure) throw new Error(failure);
return finalText || streamed; return finalText || streamed;
} }
private async sendViaCompletions(options: SendOptions, setAbort: (fn: () => void) => void): Promise<string> { private async sendViaCompletions(options: SendOptions, setAbort: (fn: () => void) => void, isAborted: () => boolean): Promise<string> {
const { onEvent } = options; const { onEvent } = options;
const useTools = this.config.localTools && !!options.localToolExecutor && !!options.localToolDefinitions?.length; const useTools = this.config.localTools && !!options.localToolExecutor && !!options.localToolDefinitions?.length;
const messages: HermesChatMessage[] = []; const messages: HermesChatMessage[] = [];
@@ -389,17 +482,15 @@ export class HermesClient {
: options.text; : options.text;
messages.push({ role: 'user', content: userContent }); messages.push({ role: 'user', content: userContent });
let aborted = false;
let currentHandle: StreamHandle | null = null; let currentHandle: StreamHandle | null = null;
setAbort(() => { setAbort(() => currentHandle?.abort());
aborted = true; const aborted = () => isAborted();
currentHandle?.abort();
});
let fullText = ''; let fullText = '';
for (let iteration = 0; iteration < 6 && !aborted; iteration++) { let toolsAllowed = useTools;
for (let iteration = 0; iteration < 6 && !aborted(); iteration++) {
const payload: Rec = { model: this.config.model || 'hermes-agent', messages, stream: true }; const payload: Rec = { model: this.config.model || 'hermes-agent', messages, stream: true };
if (useTools) { if (toolsAllowed) {
payload.tools = options.localToolDefinitions; payload.tools = options.localToolDefinitions;
payload.tool_choice = 'auto'; payload.tool_choice = 'auto';
} }
@@ -410,6 +501,7 @@ export class HermesClient {
let iterationText = ''; let iterationText = '';
const errorChunks: string[] = []; const errorChunks: string[] = [];
let ready = false; let ready = false;
let rawBody = '';
const parser = new SseParser((message) => { const parser = new SseParser((message) => {
if (message.data === '[DONE]') return; if (message.data === '[DONE]') return;
const data = tryParseJson<unknown>(message.data); const data = tryParseJson<unknown>(message.data);
@@ -427,25 +519,53 @@ export class HermesClient {
}); });
const headers = this.headers({ Accept: 'text/event-stream' }); const headers = this.headers({ Accept: 'text/event-stream' });
if (options.sessionId) headers['X-Hermes-Session-Id'] = options.sessionId; const sessionId = plainSession(options.sessionId);
if (sessionId) headers['X-Hermes-Session-Id'] = sessionId;
const handle = await httpStream( const handle = await httpStream(
{ url: `${this.base}/v1/chat/completions`, method: 'POST', headers, body: JSON.stringify(payload), timeoutMs: 10 * 60_000 }, { url: `${this.base}/v1/chat/completions`, method: 'POST', headers, body: JSON.stringify(payload), timeoutMs: 10 * 60_000 },
{ onChunk: (text) => (ready ? parser.feed(text) : errorChunks.push(text)) } {
onChunk: (text) => {
if (rawBody.length < 512_000) rawBody += text;
if (ready) parser.feed(text);
else errorChunks.push(text);
}
}
); );
currentHandle = handle; currentHandle = handle;
if (!handle.start.ok) { if (!handle.start.ok) {
await handle.done.catch(() => undefined); await handle.done.catch(() => undefined);
throw new HttpError(handle.start.status, errorMessage(handle.start.status, errorChunks.join(''))); const detail = errorMessage(handle.start.status, errorChunks.join(''));
// Some Hermes builds reject client-side tool definitions: retry once without them.
if (toolsAllowed && handle.start.status === 400 && iteration === 0) {
Log.warn('hermes', `chat completions rejected the tools payload (${detail}); retrying without local tools`);
toolsAllowed = false;
iteration--;
continue;
}
throw new HttpError(handle.start.status, detail);
} }
ready = true; ready = true;
// Chunks that raced ahead of the start event were buffered as potential error bodies: replay them.
for (const chunk of errorChunks.splice(0)) parser.feed(chunk);
const rotated = handle.start.headers['x-hermes-session-id']; const rotated = handle.start.headers['x-hermes-session-id'];
if (rotated && rotated !== options.sessionId) onEvent({ kind: 'session', sessionId: rotated }); if (rotated && rotated !== options.sessionId) onEvent({ kind: 'session', sessionId: rotated });
await handle.done; await handle.done;
parser.end(); parser.end();
if (!iterationText && toolCalls.size === 0 && !aborted()) {
// Nothing streamed: the server may have answered with a plain JSON completion (stream ignored)
// or with a 200 carrying an error object. Surface it instead of an empty bubble.
const recovered = recoverCompletion(rawBody);
if (recovered.text) {
iterationText = recovered.text;
onEvent({ kind: 'delta', text: recovered.text });
} else {
throw new Error(recovered.error ?? `Réponse vide de Hermes (${rawBody.length} octets reçus${rawBody ? ` : ${rawBody.slice(0, 160).replace(/\s+/g, ' ')}` : ''})`);
}
}
fullText += iterationText; fullText += iterationText;
if (finishReason === 'tool_calls' && useTools && toolCalls.size > 0 && !aborted) { if (finishReason === 'tool_calls' && toolsAllowed && toolCalls.size > 0 && !aborted()) {
const calls = [...toolCalls.values()]; const calls = [...toolCalls.values()];
messages.push({ messages.push({
role: 'assistant', role: 'assistant',
@@ -462,11 +582,42 @@ export class HermesClient {
} }
break; break;
} }
onEvent({ kind: 'completed', status: aborted ? 'cancelled' : 'completed', text: fullText }); onEvent({ kind: 'completed', status: aborted() ? 'cancelled' : 'completed', text: fullText });
return fullText; return fullText;
} }
} }
/** Extract text or an error message from a non-streamed chat completion body. */
export function recoverCompletion(raw: string): { text?: string; error?: string } {
const body = raw.trim();
if (!body) return {};
const html = describeHtml(body);
if (html) return { error: html };
const candidates = body.startsWith('data:') || body.startsWith('event:')
? body.split(/\n+/).filter((l) => l.startsWith('data:')).map((l) => l.replace(/^data:\s*/, '')).filter((l) => l && l !== '[DONE]')
: [body];
let text = '';
for (const c of candidates) {
const data = tryParseJson<Rec>(c);
if (!data || !isRec(data)) continue;
if (data.error) {
const e = data.error as Rec | string;
return { error: `Hermes : ${typeof e === 'string' ? e : String((e as Rec).message ?? JSON.stringify(e))}` };
}
const choice = Array.isArray(data.choices) && isRec(data.choices[0]) ? (data.choices[0] as Rec) : null;
const msg = choice && isRec(choice.message) ? (choice.message as Rec) : choice && isRec(choice.delta) ? (choice.delta as Rec) : null;
if (msg && typeof msg.content === 'string') text += msg.content;
else if (typeof data.output === 'string') text += data.output;
else if (typeof data.text === 'string') text += data.text;
}
return text ? { text } : {};
}
/** Session ids created by the sessions transport carry an `hs:` prefix; other endpoints get the bare id. */
export function plainSession(id: string): string {
return id.startsWith('hs:') ? id.slice(3) : id;
}
export function normalizeJob(job: HermesJob): HermesJob { export function normalizeJob(job: HermesJob): HermesJob {
const raw = job as Rec; const raw = job as Rec;
const id = String(raw.id ?? raw.job_id ?? raw.name ?? ''); const id = String(raw.id ?? raw.job_id ?? raw.name ?? '');
+1
View File
@@ -212,6 +212,7 @@ export function normalizeCompletionChunk(
): { events: HermesStreamEvent[]; finishReason: string | null } { ): { events: HermesStreamEvent[]; finishReason: string | null } {
const events: HermesStreamEvent[] = []; const events: HermesStreamEvent[] = [];
if (!isRec(chunk)) return { events, finishReason: null }; if (!isRec(chunk)) return { events, finishReason: null };
if (isRec(chunk.usage)) events.push({ kind: 'completed', status: 'usage', usage: chunk.usage as HermesUsage });
const choice = Array.isArray(chunk.choices) && isRec(chunk.choices[0]) ? (chunk.choices[0] as Rec) : null; const choice = Array.isArray(chunk.choices) && isRec(chunk.choices[0]) ? (chunk.choices[0] as Rec) : null;
if (!choice) return { events, finishReason: null }; if (!choice) return { events, finishReason: null };
const delta = isRec(choice.delta) ? (choice.delta as Rec) : isRec(choice.message) ? (choice.message as Rec) : null; const delta = isRec(choice.delta) ? (choice.delta as Rec) : isRec(choice.message) ? (choice.message as Rec) : null;
+42
View File
@@ -3,16 +3,35 @@
* With the runs/sessions transports Hermes uses its own server-side toolsets instead. * With the runs/sessions transports Hermes uses its own server-side toolsets instead.
*/ */
import { bridge } from '../../lib/bridge'; import { bridge } from '../../lib/bridge';
import type { SystemAction } from '../../../shared/bridge';
export interface LocalToolContext { export interface LocalToolContext {
setEmotion: (emotion: string) => void; setEmotion: (emotion: string) => void;
showMessage?: (text: string, title?: string) => void;
speak: (text: string) => void; speak: (text: string) => void;
getStatus: () => Record<string, unknown>; getStatus: () => Record<string, unknown>;
getHistory: (n: number) => Array<{ role: string; content: string }>; getHistory: (n: number) => Array<{ role: string; content: string }>;
notify: (title: string, body: string) => void; notify: (title: string, body: string) => void;
} }
const fn = (name: string, description: string, properties: Record<string, unknown> = {}, required: string[] = []) => ({
type: 'function',
function: { name, description, parameters: { type: 'object', properties, required } }
});
/** Tools executed on the user's PC through the Electron main process (allow-listed). */
export const SYSTEM_TOOL_DEFINITIONS = [
fn('lock_session', "Verrouille la session de l'utilisateur."),
fn('open_app', "Lance une application du PC de l'utilisateur (bloc-notes, calculatrice, chrome, spotify, vscode, terminal, explorateur…).", { name: { type: 'string' } }, ['name']),
fn('open_url', "Ouvre une URL http(s) dans le navigateur de l'utilisateur.", { url: { type: 'string' } }, ['url']),
fn('media_key', 'Touche média : volume-up, volume-down, mute, play-pause, next, previous.', { key: { type: 'string', enum: ['volume-up', 'volume-down', 'mute', 'play-pause', 'next', 'previous'] } }, ['key']),
fn('clipboard_get', 'Lit le texte du presse-papiers.'),
fn('clipboard_set', 'Place un texte dans le presse-papiers.', { text: { type: 'string' } }, ['text']),
fn('find_files', "Cherche des fichiers par nom dans Documents, Bureau, Téléchargements et Images (25 max).", { query: { type: 'string' } }, ['query'])
];
export const LOCAL_TOOL_DEFINITIONS = [ export const LOCAL_TOOL_DEFINITIONS = [
...SYSTEM_TOOL_DEFINITIONS,
{ {
type: 'function', type: 'function',
function: { function: {
@@ -95,6 +114,29 @@ export async function executeLocalTool(name: string, rawArgs: string, ctx: Local
case 'notify_user': case 'notify_user':
ctx.notify(String(args.title ?? 'EveFlow'), String(args.body ?? '')); ctx.notify(String(args.title ?? 'EveFlow'), String(args.body ?? ''));
return JSON.stringify({ ok: true }); return JSON.stringify({ ok: true });
case 'show_message':
ctx.showMessage?.(String(args.text ?? ''), typeof args.title === 'string' ? args.title : undefined);
return JSON.stringify({ ok: true });
case 'lock_session':
case 'open_app':
case 'open_url':
case 'media_key':
case 'clipboard_get':
case 'clipboard_set':
case 'find_files': {
const api = bridge();
if (!api) return JSON.stringify({ error: 'system actions unavailable outside Electron' });
const action: SystemAction =
name === 'lock_session' ? { type: 'lock' }
: name === 'open_app' ? { type: 'open-app', name: String(args.name ?? '') }
: name === 'open_url' ? { type: 'open-url', url: String(args.url ?? '') }
: name === 'media_key' ? { type: 'media', key: String(args.key ?? '') as 'mute' }
: name === 'clipboard_get' ? { type: 'clipboard-read' }
: name === 'clipboard_set' ? { type: 'clipboard-write', text: String(args.text ?? '') }
: { type: 'find-files', query: String(args.query ?? '') };
const result = await api.system.action(action);
return JSON.stringify(result.ok ? { ok: true, message: result.message, data: result.data } : { error: result.message ?? 'failed' });
}
case 'write_shared_file': { case 'write_shared_file': {
const api = bridge(); const api = bridge();
if (!api) return JSON.stringify({ error: 'file system unavailable outside Electron' }); if (!api) return JSON.stringify({ error: 'file system unavailable outside Electron' });
+5 -1
View File
@@ -10,6 +10,8 @@ export interface HermesConfig {
reasoningEffort: '' | 'low' | 'medium' | 'high'; reasoningEffort: '' | 'low' | 'medium' | 'high';
/** Extra instructions layered on top of the Hermes system prompt. */ /** Extra instructions layered on top of the Hermes system prompt. */
instructions: string; instructions: string;
/** Model used in "mission" mode (long tasks); empty = same as `model`. */
missionModel: string;
/** Expose EveFlow client tools in chat-completions mode. */ /** Expose EveFlow client tools in chat-completions mode. */
localTools: boolean; localTools: boolean;
} }
@@ -167,8 +169,10 @@ export interface SendOptions {
} }
export interface SendHandle { export interface SendHandle {
/** Resolves with the final assistant text. */ /** Resolves with the final assistant text (partial text when aborted). */
result: Promise<string>; result: Promise<string>;
abort: () => void; abort: () => void;
/** True once abort() was called; the caller must not treat the result as a completed reply. */
readonly aborted: boolean;
transport: Exclude<HermesTransport, 'auto'>; transport: Exclude<HermesTransport, 'auto'>;
} }
+75
View File
@@ -0,0 +1,75 @@
/**
* Instant local commands: short French/English intents handled on the machine without a
* round trip to Hermes (lock, open app/url, volume, media, screenshot to Hermes).
* Anything unmatched goes to Hermes as usual.
*/
import type { SystemAction } from '../../shared/bridge';
import { bridge } from '../lib/bridge';
import { Log } from '../lib/log';
export interface LocalIntent {
kind: 'action' | 'screenshot';
action?: SystemAction;
/** Spoken confirmation. */
reply: string;
/** For screenshot: the question to send to Hermes with the image. */
question?: string;
}
const norm = (t: string) =>
t
.normalize('NFD')
.replace(/[̀-ͯ]/g, '')
.toLowerCase()
.replace(/[’']/g, ' ')
.replace(/[^a-z0-9 :/._-]+/g, ' ')
.replace(/\s+/g, ' ')
.trim();
const LOCK = /^(verrouille|verrouiller|bloque|lock)( (la |ma )?(session|le pc|l ordinateur|the (pc|computer|screen)))?$/;
const VOL_UP = /^(monte|augmente|hausse|plus fort|volume plus|turn up|raise)( (le|the)? ?(son|volume))?( de \d+)?$/;
const VOL_DOWN = /^(baisse|diminue|moins fort|volume moins|turn down|lower)( (le|the)? ?(son|volume))?( de \d+)?$/;
const MUTE = /^(coupe|couper|mute|silence|desactive)( (le|the)? ?(son|volume|audio))?$/;
const PLAY = /^(pause|play|lecture|reprends|reprendre|mets en pause|met en pause|stop la musique|arrete la musique)$/;
const NEXT = /^(suivant|suivante|piste suivante|musique suivante|next|skip)$/;
const PREV = /^(precedent|precedente|piste precedente|previous)$/;
const OPEN_APP = /^(ouvre|ouvrir|lance|lancer|demarre|open|launch|start) ((l application|l appli|le logiciel|le programme|the app|le|la|les|l|un|une|the|moi) )*(.+)$/;
const OPEN_URL = /^(ouvre|ouvrir|va sur|open|go to) (https?:\/\/\S+|(www\.)?[a-z0-9-]+\.[a-z]{2,}(\/\S*)?)$/;
const SCREEN = /(regarde|regardes|analyse|decris|decrit|lis|explique|qu est ce qu il y a sur|que vois tu sur|what is on|look at|read) (mon |l |the )?(ecran|screen)|capture (d )?ecran|screenshot/;
export function parseLocalIntent(text: string): LocalIntent | null {
const t = norm(text).replace(/^(jarvis|hey jarvis|ok jarvis)[ ,]*/, '');
if (!t) return null;
if (LOCK.test(t)) return { kind: 'action', action: { type: 'lock' }, reply: 'Session verrouillée.' };
if (MUTE.test(t)) return { kind: 'action', action: { type: 'media', key: 'mute' }, reply: 'Son coupé.' };
if (VOL_UP.test(t)) return { kind: 'action', action: { type: 'media', key: 'volume-up' }, reply: 'Volume augmenté.' };
if (VOL_DOWN.test(t)) return { kind: 'action', action: { type: 'media', key: 'volume-down' }, reply: 'Volume baissé.' };
if (PLAY.test(t)) return { kind: 'action', action: { type: 'media', key: 'play-pause' }, reply: 'Lecture.' };
if (NEXT.test(t)) return { kind: 'action', action: { type: 'media', key: 'next' }, reply: 'Piste suivante.' };
if (PREV.test(t)) return { kind: 'action', action: { type: 'media', key: 'previous' }, reply: 'Piste précédente.' };
if (SCREEN.test(t)) return { kind: 'screenshot', reply: 'Je regarde votre écran.', question: text.trim() };
const url = OPEN_URL.exec(t);
if (url) {
const target = url[2].startsWith('http') ? url[2] : `https://${url[2]}`;
return { kind: 'action', action: { type: 'open-url', url: target }, reply: `J’ouvre ${url[2]}.` };
}
const app = OPEN_APP.exec(t);
if (app) {
const name = app[app.length - 1].trim();
if (name.length <= 40 && !/\s(et|puis|and)\s/.test(name)) return { kind: 'action', action: { type: 'open-app', name }, reply: `J’ouvre ${name}.` };
}
return null;
}
/** Execute an action intent through the bridge. Returns the spoken outcome. */
export async function runLocalIntent(intent: LocalIntent): Promise<{ ok: boolean; message: string }> {
const api = bridge();
if (!api || !intent.action) return { ok: false, message: 'Actions locales indisponibles hors Electron.' };
try {
const result = await api.system.action(intent.action);
Log.info('local', `${intent.action.type}: ${result.ok ? 'ok' : result.message}`);
return { ok: result.ok, message: result.ok ? intent.reply : result.message || 'Échec de la commande locale.' };
} catch (err) {
return { ok: false, message: (err as Error).message };
}
}
-70
View File
@@ -1,70 +0,0 @@
/**
* LogService — Centralized structured logger for EveFlow.
*
* In Electron (production/dev): sends log entries via window.electronAPI.log()
* which the main process writes to %APPDATA%/EveFlow/eveflow.log
*
* Fallback (browser only): console only.
*/
export type LogLevel = 'DEBUG' | 'INFO' | 'WARN' | 'ERROR';
export interface LogEntry {
ts: string; // ISO timestamp
level: LogLevel;
tag: string; // Component / service tag (e.g. "HermesAgent", "Minimax")
message: string;
data?: unknown; // Any extra serializable payload
}
export class LogService {
private static readonly MAX_CONSOLE_DATA_LEN = 2000;
// Core write method
public static log(level: LogLevel, tag: string, message: string, data?: unknown): void {
const entry: LogEntry = {
ts: new Date().toISOString(),
level,
tag,
message,
data
};
// Always write to console for DevTools visibility
const consoleMsg = `[${entry.ts}] [${level}] [${tag}] ${message}`;
const dataStr = data !== undefined ? JSON.stringify(data, null, 2) : '';
switch (level) {
case 'ERROR': console.error(consoleMsg, dataStr ? dataStr.slice(0, this.MAX_CONSOLE_DATA_LEN) : ''); break;
case 'WARN': console.warn(consoleMsg, dataStr ? dataStr.slice(0, this.MAX_CONSOLE_DATA_LEN) : ''); break;
case 'DEBUG': console.debug(consoleMsg, dataStr ? dataStr.slice(0, this.MAX_CONSOLE_DATA_LEN) : ''); break;
default: console.info(consoleMsg, dataStr ? dataStr.slice(0, this.MAX_CONSOLE_DATA_LEN) : ''); break;
}
// Send to Electron main process for file persistence
try {
const electronAPI = (window as any).electronAPI;
if (electronAPI?.writeLog) {
electronAPI.writeLog(entry);
}
} catch {
// Silently ignore: browser context has no electronAPI
}
}
public static debug(tag: string, message: string, data?: unknown): void {
this.log('DEBUG', tag, message, data);
}
public static info(tag: string, message: string, data?: unknown): void {
this.log('INFO', tag, message, data);
}
public static warn(tag: string, message: string, data?: unknown): void {
this.log('WARN', tag, message, data);
}
public static error(tag: string, message: string, data?: unknown): void {
this.log('ERROR', tag, message, data);
}
}
+42
View File
@@ -0,0 +1,42 @@
/**
* Answers tool calls that Hermes sends through the local MCP endpoint and that need the renderer
* (voice, notifications, HUD, chat). System tools are executed in the main process directly.
*/
import type { McpToolRequest } from '../../shared/ipc';
import { bridge } from '../lib/bridge';
import { Log } from '../lib/log';
import { useChat } from '../state/chat';
import { executeLocalTool, type LocalToolContext } from './hermes/localTools';
let unsubscribe: (() => void) | null = null;
export function initMcpBridge(context: () => LocalToolContext): void {
const api = bridge();
if (!api || unsubscribe) return;
unsubscribe = api.hermes.onToolRequest((req: McpToolRequest) => {
void handle(req, context)
.then((result) => api.hermes.toolResponse({ id: req.id, ok: true, result }))
.catch((err: Error) => api.hermes.toolResponse({ id: req.id, ok: false, error: err.message }));
});
}
async function handle(req: McpToolRequest, context: () => LocalToolContext): Promise<unknown> {
Log.info('mcp', `tool ${req.name}`);
const chat = useChat.getState();
if (req.name === 'show_message') {
const text = String(req.args.text ?? '');
if (!text.trim()) throw new Error('texte vide');
chat.addMessage({ role: 'assistant', content: text, source: 'mcp', jobName: typeof req.args.title === 'string' ? req.args.title : undefined, status: 'done' });
chat.pushActivity({ kind: 'system', name: 'hermes → eveflow', status: 'done', detail: text.slice(0, 160) });
return { ok: true };
}
const raw = await executeLocalTool(req.name, JSON.stringify(req.args ?? {}), context());
try {
const parsed = JSON.parse(raw) as { error?: string };
if (parsed && typeof parsed === 'object' && parsed.error) throw new Error(parsed.error);
return parsed;
} catch (err) {
if ((err as Error).message && !(err instanceof SyntaxError)) throw err;
return raw;
}
}
+2 -2
View File
@@ -71,9 +71,9 @@ class AudioBus {
} }
/** Normalised output spectrum (0..1) with `bins` values, low to high frequency. */ /** Normalised output spectrum (0..1) with `bins` values, low to high frequency. */
spectrum(bins: number, source: 'output' | 'input' = 'output'): Float32Array { spectrum(bins: number, source: 'output' | 'input' = 'output', out: Float32Array = new Float32Array(bins)): Float32Array {
const analyser = source === 'output' ? this.outputAnalyser : this.inputAnalyser; const analyser = source === 'output' ? this.outputAnalyser : this.inputAnalyser;
const out = new Float32Array(bins); out.fill(0);
if (!analyser) { if (!analyser) {
if (this.synthetic > 0) { if (this.synthetic > 0) {
const t = performance.now() / 1000; const t = performance.now() / 1000;
+28 -2
View File
@@ -66,6 +66,21 @@ function getWorkletUrl(): string {
return workletUrl; return workletUrl;
} }
/**
* Load an AudioWorklet module: first the static file shipped with the app (allowed by the strict
* CSP, script-src 'self'), then a blob URL for dev servers that do not serve /worklets.
*/
export async function loadWorklet(ctx: AudioContext, name: string, blobUrl: () => string): Promise<void> {
const staticUrl = new URL(`worklets/${name}.js`, document.baseURI).href;
try {
await ctx.audioWorklet.addModule(staticUrl);
return;
} catch (err) {
Log.debug('audio', `static worklet ${name} unavailable (${(err as Error).message}), trying blob`);
}
await ctx.audioWorklet.addModule(blobUrl());
}
export async function listMicrophones(): Promise<MicDevice[]> { export async function listMicrophones(): Promise<MicDevice[]> {
try { try {
const devices = await navigator.mediaDevices.enumerateDevices(); const devices = await navigator.mediaDevices.enumerateDevices();
@@ -88,6 +103,7 @@ export class MicCapture {
private active = false; private active = false;
private options: CaptureOptions | null = null; private options: CaptureOptions | null = null;
private sampleRate = 16_000; private sampleRate = 16_000;
private hadSpeech = false;
get isActive(): boolean { get isActive(): boolean {
return this.active; return this.active;
@@ -98,6 +114,7 @@ export class MicCapture {
this.options = options; this.options = options;
this.chunks = []; this.chunks = [];
this.totalSamples = 0; this.totalSamples = 0;
this.hadSpeech = false;
this.vad = options.mode === 'auto' ? new EnergyVad(options.vad ?? DEFAULT_VAD) : null; this.vad = options.mode === 'auto' ? new EnergyVad(options.vad ?? DEFAULT_VAD) : null;
if (!navigator.mediaDevices?.getUserMedia) { if (!navigator.mediaDevices?.getUserMedia) {
@@ -145,7 +162,7 @@ export class MicCapture {
audioBus.setInputAnalyser(analyser); audioBus.setInputAnalyser(analyser);
try { try {
await this.ctx.audioWorklet.addModule(getWorkletUrl()); await loadWorklet(this.ctx, 'eveflow-capture', getWorkletUrl);
this.worklet = new AudioWorkletNode(this.ctx, 'eveflow-capture', { numberOfInputs: 1, numberOfOutputs: 0, channelCount: 1 }); this.worklet = new AudioWorkletNode(this.ctx, 'eveflow-capture', { numberOfInputs: 1, numberOfOutputs: 0, channelCount: 1 });
this.worklet.port.onmessage = (event: MessageEvent<Float32Array>) => this.onSamples(event.data); this.worklet.port.onmessage = (event: MessageEvent<Float32Array>) => this.onSamples(event.data);
source.connect(this.worklet); source.connect(this.worklet);
@@ -169,6 +186,12 @@ export class MicCapture {
if (!this.active || !this.options) return; if (!this.active || !this.options) return;
this.chunks.push(samples); this.chunks.push(samples);
this.totalSamples += samples.length; this.totalSamples += samples.length;
// Before speech starts only a short pre-roll (~400 ms) is kept, so silence is never shipped to STT.
if (this.vad && !this.hadSpeech) {
while (this.chunks.length > 1 && this.totalSamples - this.chunks[0].length > this.sampleRate * 0.4) {
this.totalSamples -= this.chunks.shift()!.length;
}
}
const level = rms(samples); const level = rms(samples);
this.options.callbacks.onLevel?.(Math.min(1, level * 6)); this.options.callbacks.onLevel?.(Math.min(1, level * 6));
@@ -184,6 +207,7 @@ export class MicCapture {
private handleVad(signal: VadSignal): void { private handleVad(signal: VadSignal): void {
switch (signal) { switch (signal) {
case 'speech-start': case 'speech-start':
this.hadSpeech = true;
this.options?.callbacks.onSpeechStart?.(); this.options?.callbacks.onSpeechStart?.();
break; break;
case 'speech-end': case 'speech-end':
@@ -222,10 +246,12 @@ export class MicCapture {
/** Stop capture. In manual mode this finalises the utterance; otherwise reports the reason. */ /** Stop capture. In manual mode this finalises the utterance; otherwise reports the reason. */
stop(reason: 'stopped' | 'no-speech' | 'too-short' = 'stopped'): void { stop(reason: 'stopped' | 'no-speech' | 'too-short' = 'stopped'): void {
if (!this.active) return; if (!this.active) return;
if (reason === 'stopped') { // Manual stop: transcribe only if something was actually said (or no VAD is running).
if (reason === 'stopped' && (!this.vad || this.hadSpeech)) {
this.finishUtterance('stopped'); this.finishUtterance('stopped');
return; return;
} }
if (reason === 'stopped') reason = 'no-speech';
const opts = this.options; const opts = this.options;
this.teardown(); this.teardown();
opts?.callbacks.onEnd(reason); opts?.callbacks.onEnd(reason);
+3 -2
View File
@@ -35,9 +35,10 @@ class SpeechFacade {
return this.engine?.isActive ?? false; return this.engine?.isActive ?? false;
} }
say(text: string): void { say(text: string, options: { interrupt?: boolean } = {}): void {
if (useSettings.getState().settings.speech.provider === 'off') return; if (useSettings.getState().settings.speech.provider === 'off') return;
this.tts.speak(text); // While an answer streams, spoken notices are inserted without discarding the rest.
this.tts.speak(text, { interrupt: options.interrupt ?? !this.streaming });
} }
pushStream(delta: string): void { pushStream(delta: string): void {
+17 -2
View File
@@ -1,7 +1,8 @@
import { Log } from '../../lib/log'; import { Log } from '../../lib/log';
import { bridge } from '../../lib/bridge';
import { httpFetch } from '../../lib/transport'; import { httpFetch } from '../../lib/transport';
export type SttProvider = 'openai-compatible' | 'browser'; export type SttProvider = 'openai-compatible' | 'browser' | 'local';
export interface SttConfig { export interface SttConfig {
provider: SttProvider; provider: SttProvider;
@@ -10,6 +11,8 @@ export interface SttConfig {
model: string; model: string;
language: string; // BCP-47, e.g. fr-FR language: string; // BCP-47, e.g. fr-FR
prompt?: string; prompt?: string;
/** Catalog id of the local sherpa-onnx model (provider 'local'). */
localModel: string;
} }
export function sttEndpoint(apiUrl: string): string { export function sttEndpoint(apiUrl: string): string {
@@ -20,8 +23,20 @@ export function sttEndpoint(apiUrl: string): string {
return `${base}/v1/audio/transcriptions`; return `${base}/v1/audio/transcriptions`;
} }
/** Transcribe a WAV buffer through any OpenAI-compatible `/v1/audio/transcriptions` endpoint. */ /** Transcribe with the in-app sherpa-onnx engine (runs in the Electron main process). */
export async function transcribeLocal(wav: Uint8Array, config: SttConfig): Promise<string> {
const api = bridge();
if (!api) throw new Error('La reconnaissance locale nécessite l’application Electron.');
if (!config.localModel) throw new Error('Aucun modèle de reconnaissance local sélectionné.');
const started = Date.now();
const result = await api.voice.transcribe({ modelId: config.localModel, wav, language: config.language.split('-')[0] || 'auto' });
Log.info('stt', `local transcription in ${Date.now() - started} ms (engine ${result.durationMs} ms, audio ${result.audioSec.toFixed(1)}s)`);
return result.text.trim();
}
/** Transcribe a WAV buffer with the configured provider. */
export async function transcribeWav(wav: Uint8Array, config: SttConfig): Promise<string> { export async function transcribeWav(wav: Uint8Array, config: SttConfig): Promise<string> {
if (config.provider === 'local') return transcribeLocal(wav, config);
if (!config.apiUrl.trim()) throw new Error("URL de l'API de transcription non configurée."); if (!config.apiUrl.trim()) throw new Error("URL de l'API de transcription non configurée.");
const endpoint = sttEndpoint(config.apiUrl); const endpoint = sttEndpoint(config.apiUrl);
const fields: Record<string, string> = { const fields: Record<string, string> = {
+156 -42
View File
@@ -1,14 +1,16 @@
/** /**
* Text-to-speech engine with a sentence queue, prefetching and Web Audio playback so the HUD * Text-to-speech engine with a sentence queue, bounded prefetching and Web Audio playback so
* reacts to the actual waveform. Providers: OpenAI-compatible /v1/audio/speech, system voices, * the HUD reacts to the actual waveform. Providers: in-app sherpa-onnx (Kokoro / Piper),
* and the legacy Google Translate endpoint (no key, online only). * OpenAI-compatible /v1/audio/speech, system voices, and the legacy Google Translate endpoint.
*/ */
import { Log } from '../../lib/log'; import { Log } from '../../lib/log';
import { defaultOpenAiVoice, rankSystemVoice } from '../../lib/voicePreference';
import { bridge } from '../../lib/bridge';
import { chunkForSpeech, cleanForSpeech, extractSentences } from '../../lib/text'; import { chunkForSpeech, cleanForSpeech, extractSentences } from '../../lib/text';
import { httpFetch } from '../../lib/transport'; import { httpFetch } from '../../lib/transport';
import { audioBus } from './audioBus'; import { audioBus } from './audioBus';
export type TtsProvider = 'openai-compatible' | 'system' | 'google-free' | 'off'; export type TtsProvider = 'openai-compatible' | 'system' | 'google-free' | 'local' | 'off';
export interface TtsConfig { export interface TtsConfig {
provider: TtsProvider; provider: TtsProvider;
@@ -21,6 +23,14 @@ export interface TtsConfig {
systemVoice: string; systemVoice: string;
language: string; language: string;
volume: number; volume: number;
/** Catalog id of the local sherpa-onnx voice model (provider 'local'). */
localModel: string;
/** Speaker id inside the local model. */
localSpeaker: number;
/** Preferred voice gender, applied to every provider's default voice. */
voiceGender?: 'male' | 'female';
/** 'jarvis' adds a subtle AI timbre: slightly lower pitch, warm/crisp EQ, short room reverb. */
timbre?: 'natural' | 'jarvis';
} }
export type TtsState = 'idle' | 'loading' | 'speaking'; export type TtsState = 'idle' | 'loading' | 'speaking';
@@ -30,6 +40,8 @@ interface QueueItem {
audio?: Promise<AudioBuffer | null>; audio?: Promise<AudioBuffer | null>;
} }
const LOOKAHEAD = 2;
export function ttsEndpoint(apiUrl: string): string { export function ttsEndpoint(apiUrl: string): string {
const base = apiUrl.trim().replace(/\/+$/, ''); const base = apiUrl.trim().replace(/\/+$/, '');
if (base.endsWith('/speech')) return base; if (base.endsWith('/speech')) return base;
@@ -41,8 +53,7 @@ export function ttsEndpoint(apiUrl: string): string {
export class TtsEngine { export class TtsEngine {
private queue: QueueItem[] = []; private queue: QueueItem[] = [];
private playing = false; private playing = false;
private currentSource: AudioBufferSourceNode | null = null; private cancelCurrent: (() => void) | null = null;
private currentUtterance: SpeechSynthesisUtterance | null = null;
private streamBuffer = ''; private streamBuffer = '';
private generation = 0; private generation = 0;
private config: TtsConfig; private config: TtsConfig;
@@ -55,7 +66,6 @@ export class TtsEngine {
updateConfig(config: TtsConfig): void { updateConfig(config: TtsConfig): void {
this.config = config; this.config = config;
audioBus.setVolume(config.volume);
} }
onState(listener: (state: TtsState) => void): () => void { onState(listener: (state: TtsState) => void): () => void {
@@ -73,10 +83,18 @@ export class TtsEngine {
for (const l of this.listeners) l(state); for (const l of this.listeners) l(state);
} }
/** Speak a finished text (clears any pending speech first). */ /**
speak(text: string): void { * Speak a finished text. With `interrupt` (default) pending speech is discarded first;
this.stop(); * without it the text is inserted ahead of the queue and streamed speech resumes afterwards.
for (const chunk of chunkForSpeech(text)) this.enqueue(chunk); */
speak(text: string, options: { interrupt?: boolean } = {}): void {
const interrupt = options.interrupt ?? true;
if (interrupt) this.stop();
const items = chunkForSpeech(text).map((chunk) => this.makeItem(chunk)).filter((i): i is QueueItem => !!i);
if (interrupt) this.queue.push(...items);
else this.queue.unshift(...items);
this.fillPrefetch();
void this.drain();
} }
/** Feed streamed tokens; complete sentences are spoken as soon as they are available. */ /** Feed streamed tokens; complete sentences are spoken as soon as they are available. */
@@ -95,32 +113,32 @@ export class TtsEngine {
} }
enqueue(text: string): void { enqueue(text: string): void {
if (this.config.provider === 'off') return; const item = this.makeItem(text);
const clean = cleanForSpeech(text); if (!item) return;
if (!clean || !/[\p{L}\p{N}]/u.test(clean)) return;
const item: QueueItem = { text: clean };
if (this.config.provider !== 'system') item.audio = this.prefetch(clean, this.generation);
this.queue.push(item); this.queue.push(item);
this.fillPrefetch();
void this.drain(); void this.drain();
} }
private makeItem(text: string): QueueItem | null {
if (this.config.provider === 'off') return null;
const clean = cleanForSpeech(text);
if (!clean || !/[\p{L}\p{N}]/u.test(clean)) return null;
return { text: clean };
}
/** Keep at most LOOKAHEAD synthesis requests in flight ahead of playback. */
private fillPrefetch(): void {
if (this.config.provider === 'system') return;
for (const item of this.queue.slice(0, LOOKAHEAD)) item.audio ??= this.prefetch(item.text, this.generation);
}
stop(): void { stop(): void {
this.generation++; this.generation++;
this.queue = []; this.queue = [];
this.streamBuffer = ''; this.streamBuffer = '';
if (this.currentSource) { this.cancelCurrent?.();
try { this.cancelCurrent = null;
this.currentSource.onended = null;
this.currentSource.stop();
} catch {
/* already stopped */
}
this.currentSource = null;
}
if (this.currentUtterance) {
this.currentUtterance.onend = null;
this.currentUtterance = null;
}
if (typeof speechSynthesis !== 'undefined') speechSynthesis.cancel(); if (typeof speechSynthesis !== 'undefined') speechSynthesis.cancel();
audioBus.setSyntheticLevel(0); audioBus.setSyntheticLevel(0);
this.playing = false; this.playing = false;
@@ -133,15 +151,17 @@ export class TtsEngine {
const gen = this.generation; const gen = this.generation;
while (this.queue.length > 0 && gen === this.generation) { while (this.queue.length > 0 && gen === this.generation) {
const item = this.queue.shift()!; const item = this.queue.shift()!;
this.fillPrefetch();
try { try {
if (item.audio) { if (this.config.provider !== 'system') {
this.setState('loading'); this.setState('loading');
const buffer = await item.audio; const buffer = await (item.audio ?? this.prefetch(item.text, gen));
if (gen !== this.generation) break; if (gen !== this.generation) break;
if (buffer) { if (buffer) {
this.setState('speaking'); this.setState('speaking');
await this.playBuffer(buffer, gen); await this.playBuffer(buffer, gen);
} else { } else {
this.setState('speaking');
await this.speakSystem(item.text, gen); await this.speakSystem(item.text, gen);
} }
} else { } else {
@@ -159,7 +179,8 @@ export class TtsEngine {
} }
private prefetch(text: string, gen: number): Promise<AudioBuffer | null> { private prefetch(text: string, gen: number): Promise<AudioBuffer | null> {
const task = this.config.provider === 'google-free' ? this.fetchGoogle(text) : this.fetchOpenAi(text); const task =
this.config.provider === 'google-free' ? this.fetchGoogle(text) : this.config.provider === 'local' ? this.fetchLocal(text) : this.fetchOpenAi(text);
return task return task
.then(async (bytes) => { .then(async (bytes) => {
if (gen !== this.generation || !bytes) return null; if (gen !== this.generation || !bytes) return null;
@@ -168,7 +189,7 @@ export class TtsEngine {
return audioBus.context.decodeAudioData(copy); return audioBus.context.decodeAudioData(copy);
}) })
.catch((err) => { .catch((err) => {
Log.warn('tts', `synthesis failed, falling back to system voice: ${(err as Error).message}`); if (gen === this.generation) Log.warn('tts', `synthesis failed, falling back to system voice: ${(err as Error).message}`);
return null; return null;
}); });
} }
@@ -185,7 +206,7 @@ export class TtsEngine {
body: JSON.stringify({ body: JSON.stringify({
model: model || 'tts-1', model: model || 'tts-1',
input: text, input: text,
voice: voice || 'alloy', voice: voice || defaultOpenAiVoice(this.config.voiceGender ?? 'male'),
speed: Math.max(0.5, Math.min(2, speed || 1)), speed: Math.max(0.5, Math.min(2, speed || 1)),
response_format: format || 'mp3' response_format: format || 'mp3'
}), }),
@@ -199,6 +220,20 @@ export class TtsEngine {
return res.binary; return res.binary;
} }
private async fetchLocal(text: string): Promise<Uint8Array> {
const api = bridge();
if (!api) throw new Error('La synthèse locale nécessite l’application Electron.');
if (!this.config.localModel) throw new Error('Aucun modèle de voix local sélectionné.');
const result = await api.voice.synthesize({
modelId: this.config.localModel,
text,
speaker: this.config.localSpeaker,
speed: Math.max(0.5, Math.min(2, this.config.speed || 1))
});
Log.debug('tts', `local synthesis ${result.durationMs} ms for ${result.audioSec.toFixed(1)}s of audio`);
return result.wav;
}
private async fetchGoogle(text: string): Promise<Uint8Array> { private async fetchGoogle(text: string): Promise<Uint8Array> {
const url = `https://translate.google.com/translate_tts?ie=UTF-8&tl=${encodeURIComponent(this.config.language.split('-')[0] || 'fr')}&client=tw-ob&q=${encodeURIComponent(text.slice(0, 200))}`; const url = `https://translate.google.com/translate_tts?ie=UTF-8&tl=${encodeURIComponent(this.config.language.split('-')[0] || 'fr')}&client=tw-ob&q=${encodeURIComponent(text.slice(0, 200))}`;
const res = await httpFetch({ const res = await httpFetch({
@@ -218,15 +253,32 @@ export class TtsEngine {
return new Promise((resolve) => { return new Promise((resolve) => {
if (gen !== this.generation) return resolve(); if (gen !== this.generation) return resolve();
const ctx = audioBus.context; const ctx = audioBus.context;
audioBus.setVolume(this.config.volume);
const source = ctx.createBufferSource(); const source = ctx.createBufferSource();
source.buffer = buffer; source.buffer = buffer;
if (this.config.provider === 'google-free') source.playbackRate.value = Math.max(0.5, Math.min(2, this.config.speed || 1)); if (this.config.provider === 'google-free') source.playbackRate.value = Math.max(0.5, Math.min(2, this.config.speed || 1));
source.connect(audioBus.output); if (this.config.timbre === 'jarvis') {
source.onended = () => { // Slightly lower and calmer, warm low end, crisp presence, a touch of room: the film's intercom feel.
if (this.currentSource === source) this.currentSource = null; source.playbackRate.value *= 0.94;
source.connect(jarvisChain(ctx));
} else {
source.connect(audioBus.output);
}
const finish = () => {
if (this.cancelCurrent === cancel) this.cancelCurrent = null;
resolve(); resolve();
}; };
this.currentSource = source; const cancel = () => {
source.onended = null;
try {
source.stop();
} catch {
/* already stopped */
}
finish();
};
source.onended = finish;
this.cancelCurrent = cancel;
source.start(); source.start();
}); });
} }
@@ -248,12 +300,17 @@ export class TtsEngine {
if (pulse) clearInterval(pulse); if (pulse) clearInterval(pulse);
pulse = null; pulse = null;
audioBus.setSyntheticLevel(0); audioBus.setSyntheticLevel(0);
if (this.currentUtterance === utterance) this.currentUtterance = null; if (this.cancelCurrent === cancel) this.cancelCurrent = null;
resolve(); resolve();
}; };
const cancel = () => {
utterance.onend = null;
utterance.onerror = null;
finish();
};
utterance.onend = finish; utterance.onend = finish;
utterance.onerror = finish; utterance.onerror = finish;
this.currentUtterance = utterance; this.cancelCurrent = cancel;
speechSynthesis.speak(utterance); speechSynthesis.speak(utterance);
}); });
} }
@@ -266,8 +323,8 @@ export class TtsEngine {
if (exact) return exact; if (exact) return exact;
} }
const lang = (this.config.language || 'fr').toLowerCase().split('-')[0]; const lang = (this.config.language || 'fr').toLowerCase().split('-')[0];
const candidates = voices.filter((v) => v.lang.toLowerCase().startsWith(lang)); const gender = this.config.voiceGender ?? 'male';
return candidates.sort((a, b) => scoreVoice(b) - scoreVoice(a))[0] ?? voices[0]; return [...voices].sort((a, b) => rankSystemVoice(b, lang, gender) - rankSystemVoice(a, lang, gender))[0];
} }
} }
@@ -285,3 +342,60 @@ export function listSystemVoices(): SpeechSynthesisVoice[] {
if (typeof speechSynthesis === 'undefined') return []; if (typeof speechSynthesis === 'undefined') return [];
return [...speechSynthesis.getVoices()].sort((a, b) => a.lang.localeCompare(b.lang) || scoreVoice(b) - scoreVoice(a)); return [...speechSynthesis.getVoices()].sort((a, b) => a.lang.localeCompare(b.lang) || scoreVoice(b) - scoreVoice(a));
} }
let jarvisInput: AudioNode | null = null;
let jarvisCtx: AudioContext | null = null;
/** Shared effect chain for the JARVIS timbre (built once per AudioContext). */
function jarvisChain(ctx: AudioContext): AudioNode {
if (jarvisInput && jarvisCtx === ctx) return jarvisInput;
const input = ctx.createGain();
const warmth = ctx.createBiquadFilter();
warmth.type = 'lowshelf';
warmth.frequency.value = 180;
warmth.gain.value = 3.5;
const presence = ctx.createBiquadFilter();
presence.type = 'peaking';
presence.frequency.value = 3200;
presence.Q.value = 0.9;
presence.gain.value = 2.5;
const air = ctx.createBiquadFilter();
air.type = 'highshelf';
air.frequency.value = 7000;
air.gain.value = -2;
const comp = ctx.createDynamicsCompressor();
comp.threshold.value = -20;
comp.ratio.value = 3;
comp.attack.value = 0.005;
comp.release.value = 0.12;
const dry = ctx.createGain();
dry.gain.value = 0.86;
const wet = ctx.createGain();
wet.gain.value = 0.14;
const reverb = ctx.createConvolver();
reverb.buffer = impulse(ctx, 0.22, 3.2);
input.connect(warmth);
warmth.connect(presence);
presence.connect(air);
air.connect(comp);
comp.connect(dry);
comp.connect(reverb);
reverb.connect(wet);
dry.connect(audioBus.output);
wet.connect(audioBus.output);
jarvisInput = input;
jarvisCtx = ctx;
return input;
}
/** Short synthetic room impulse (exponentially decaying noise). */
function impulse(ctx: AudioContext, seconds: number, decay: number): AudioBuffer {
const rate = ctx.sampleRate;
const length = Math.max(1, Math.floor(rate * seconds));
const buffer = ctx.createBuffer(2, length, rate);
for (let ch = 0; ch < 2; ch++) {
const data = buffer.getChannelData(ch);
for (let i = 0; i < length; i++) data[i] = (Math.random() * 2 - 1) * Math.pow(1 - i / length, decay);
}
return buffer;
}
+212 -12
View File
@@ -2,6 +2,7 @@
* Voice controller: microphone → VAD → STT → conversation, plus hands-free loop and barge-in. * Voice controller: microphone → VAD → STT → conversation, plus hands-free loop and barge-in.
*/ */
import { Log } from '../../lib/log'; import { Log } from '../../lib/log';
import { isTranscriptNoise } from '../../lib/text';
import { useChat } from '../../state/chat'; import { useChat } from '../../state/chat';
import { useSettings } from '../../state/settings'; import { useSettings } from '../../state/settings';
import { useVoice } from '../../state/voice'; import { useVoice } from '../../state/voice';
@@ -10,16 +11,52 @@ import { audioBus } from './audioBus';
import { listMicrophones, MicCapture } from './capture'; import { listMicrophones, MicCapture } from './capture';
import { speech } from './speech'; import { speech } from './speech';
import { BrowserRecognizer, transcribeWav } from './stt'; import { BrowserRecognizer, transcribeWav } from './stt';
import { WakeListener } from './wakeListener';
import type { WavResult } from './wav'; import type { WavResult } from './wav';
import { bridge } from '../../lib/bridge';
const SENSITIVITY_RATIO: Record<number, number> = { 1: 4.5, 2: 3.4, 3: 2.6, 4: 2.0, 5: 1.6 }; const SENSITIVITY_RATIO: Record<number, number> = { 1: 4.5, 2: 3.4, 3: 2.6, 4: 2.0, 5: 1.6 };
const SENSITIVITY_MIN_RMS: Record<number, number> = { 1: 0.03, 2: 0.02, 3: 0.012, 4: 0.008, 5: 0.005 }; const SENSITIVITY_MIN_RMS: Record<number, number> = { 1: 0.03, 2: 0.02, 3: 0.012, 4: 0.008, 5: 0.005 };
/** Levenshtein distance, used to tolerate transcription slips in the wake word ("javis" for "jarvis"). */
function editDistance(a: string, b: string): number {
const dp = Array.from({ length: a.length + 1 }, (_, i) => [i, ...new Array<number>(b.length).fill(0)]);
for (let j = 1; j <= b.length; j++) dp[0][j] = j;
for (let i = 1; i <= a.length; i++) {
for (let j = 1; j <= b.length; j++) {
dp[i][j] = Math.min(dp[i - 1][j] + 1, dp[i][j - 1] + 1, dp[i - 1][j - 1] + (a[i - 1] === b[j - 1] ? 0 : 1));
}
}
return dp[a.length][b.length];
}
/** Compare the start of an utterance with the wake word, tolerant to accents, case, punctuation and small slips. */
export function matchWakeWord(text: string, wakeWord: string): { matched: boolean; rest: string } {
const norm = (v: string) => v.normalize('NFD').replace(/[\u0300-\u036f]/g, '').toLowerCase();
const target = norm(wakeWord).split(/[\s\p{P}]+/u).filter(Boolean);
if (target.length === 0) return { matched: true, rest: text.trim() };
const original = text.trim().replace(/^[\s\p{P}]+/u, '');
const words = original.split(/\s+/);
const head = words.slice(0, target.length).map((w) => norm(w).replace(/[\p{P}]+/gu, ''));
if (head.length < target.length) return { matched: false, rest: '' };
const joinedHead = head.join(' ');
const joinedTarget = target.join(' ');
const tolerance = joinedTarget.length >= 6 ? 2 : joinedTarget.length >= 4 ? 1 : 0;
if (editDistance(joinedHead, joinedTarget) > tolerance) return { matched: false, rest: '' };
const rest = words.slice(target.length).join(' ').replace(/^[\s\p{P}]+/u, '');
return { matched: true, rest };
}
class VoiceController { class VoiceController {
private capture = new MicCapture(); private capture = new MicCapture();
private wake = new WakeListener();
private unsubscribeKws: (() => void) | null = null;
/** After a bare wake word, the next utterance is accepted without the wake word. */
private attentionUntil = 0;
private browser = new BrowserRecognizer(); private browser = new BrowserRecognizer();
private handsFreeTimer: ReturnType<typeof setTimeout> | null = null; private handsFreeTimer: ReturnType<typeof setTimeout> | null = null;
private stopping = false; private stopping = false;
private startSeq = 0;
private unsubscribeVoice: (() => void) | null = null; private unsubscribeVoice: (() => void) | null = null;
init(): void { init(): void {
@@ -30,28 +67,141 @@ class VoiceController {
this.scheduleHandsFree(350); this.scheduleHandsFree(350);
} }
}); });
// Hands-free also re-arms when a reply ends without speech (voice off, empty answer, error).
useChat.subscribe((state, prev) => {
if (prev.isSending && !state.isSending) {
const v = useVoice.getState();
if (v.handsFree && v.phase === 'off' && !speech.isSpeaking()) this.scheduleHandsFree(400);
}
});
} }
dispose(): void { dispose(): void {
this.unsubscribeVoice?.(); this.unsubscribeVoice?.();
this.stop(); this.stop();
void this.stopWakeMode();
} }
get isListening(): boolean { get isListening(): boolean {
return useVoice.getState().phase !== 'off'; return useVoice.getState().phase !== 'off';
} }
get wakeActive(): boolean {
return this.wake.isActive;
}
toggle(): void { toggle(): void {
if (this.wake.isActive) {
// Always-on mode: the button starts/cancels a command capture on the shared microphone.
if (this.wake.currentPhase === 'command' || this.wake.currentPhase === 'speech') this.wake.cancelCommand();
else this.onWakeDetected('manuel');
return;
}
if (this.isListening) this.stop(); if (this.isListening) this.stop();
else void this.start(); else void this.start();
} }
/** Always-on keyword spotting: the microphone stays open and the main process spots the wake word. */
async startWakeMode(): Promise<void> {
const api = bridge();
const voice = useVoice.getState();
if (!api || this.wake.isActive) return;
const settings = useSettings.getState().settings.voice;
voice.setWake('starting');
try {
const phrases = [settings.wakeWord || 'jarvis', `hey ${settings.wakeWord || 'jarvis'}`];
const result = await api.voice.kwsStart({ modelId: 'kws-en', keywords: phrases, sensitivity: settings.kwsSensitivity });
this.unsubscribeKws?.();
this.unsubscribeKws = api.voice.onKwsDetected((d) => this.onWakeDetected(d.keyword));
const sensitivity = Math.min(5, Math.max(1, Math.round(settings.sensitivity))) as 1 | 2 | 3 | 4 | 5;
let neuralVad = false;
if (settings.neuralVad) {
try {
const installed = (await api.voice.listModels()).some((m) => m.id === 'silero-vad' && m.installed);
if (installed) {
await api.voice.vadStart({ modelId: 'silero-vad', silenceMs: Math.max(300, Math.min(1500, settings.silenceMs - 200)), threshold: 0.5, maxUtteranceSec: 25 });
neuralVad = true;
}
} catch (err) {
Log.warn('voice', `silero unavailable, energy VAD fallback: ${(err as Error).message}`);
}
}
await this.wake.start({
deviceId: settings.micDeviceId || undefined,
neuralVad,
vad: { silenceMs: settings.silenceMs, speechRatio: SENSITIVITY_RATIO[sensitivity], minRms: SENSITIVITY_MIN_RMS[sensitivity] },
callbacks: {
onPhase: (phase) => {
const v = useVoice.getState();
if (phase === 'spotting') {
v.setPhase('off');
const chat = useChat.getState();
if (chat.hud === 'listening') chat.setHud(chat.isSending ? 'thinking' : 'idle');
} else if (phase === 'command') {
v.setPhase('listening');
useChat.getState().setHud('listening');
} else if (phase === 'speech') {
v.setPhase('speech');
useChat.getState().ping();
}
},
onLevel: (level) => {
const q = Math.round(level * 20) / 20;
if (q !== useVoice.getState().inputLevel) useVoice.getState().setInputLevel(q);
},
onUtterance: (wav) => void this.transcribe(wav, true),
onNoSpeech: () => useVoice.getState().setInputLevel(0),
onError: (message) => useVoice.getState().setError(message)
}
});
voice.setWake('spotting', result.accepted.map((k) => k.replace(/_/g, ' ')));
voice.setNeuralVad(neuralVad);
Log.info('voice', `wake mode on: ${result.accepted.join(', ')}${result.rejected.length ? ` (rejetés : ${result.rejected.join(', ')})` : ''}${neuralVad ? ' · Silero' : ''}`);
} catch (err) {
const message = (err as Error).message;
voice.setWake('error');
voice.setError(message);
useChat.getState().setError(`Mot d’activation : ${message}`);
Log.error('voice', `wake mode failed: ${message}`);
await api.voice.kwsStop().catch(() => undefined);
await api.voice.vadStop().catch(() => undefined);
}
}
async stopWakeMode(): Promise<void> {
this.unsubscribeKws?.();
this.unsubscribeKws = null;
this.wake.stop();
useVoice.getState().setWake('off');
useVoice.getState().setNeuralVad(false);
const api = bridge();
await api?.voice.kwsStop().catch(() => undefined);
await api?.voice.vadStop().catch(() => undefined);
}
/** Restart spotting with the current settings (wake word, sensitivity, microphone). */
async restartWakeMode(): Promise<void> {
await this.stopWakeMode();
if (useSettings.getState().settings.voice.wakeMode === 'kws') await this.startWakeMode();
}
private onWakeDetected(keyword: string): void {
if (!this.wake.isActive || this.wake.currentPhase !== 'spotting') return;
if (useChat.getState().isSending) return;
Log.info('voice', `wake: ${keyword}`);
speech.stop();
useChat.getState().ping();
this.playChime(true);
this.wake.beginCommand();
}
setHandsFree(on: boolean): void { setHandsFree(on: boolean): void {
useVoice.getState().setHandsFree(on); useVoice.getState().setHandsFree(on);
useSettings.getState().update({ voice: { handsFree: on } }); useSettings.getState().update({ voice: { handsFree: on } });
if (on && !this.isListening && !speech.isSpeaking()) void this.start(); if (on && !this.isListening && !speech.isSpeaking()) void this.start();
if (!on) { if (!on) {
this.clearHandsFreeTimer(); this.clearHandsFreeTimer();
if (this.isListening) this.cancel();
} }
} }
@@ -59,7 +209,8 @@ class VoiceController {
this.clearHandsFreeTimer(); this.clearHandsFreeTimer();
this.handsFreeTimer = setTimeout(() => { this.handsFreeTimer = setTimeout(() => {
const v = useVoice.getState(); const v = useVoice.getState();
if (v.handsFree && v.phase === 'off' && !useChat.getState().isSending && !speech.isSpeaking()) void this.start(); const bargeIn = useSettings.getState().settings.voice.bargeIn;
if (v.handsFree && v.phase === 'off' && !useChat.getState().isSending && (bargeIn || !speech.isSpeaking())) void this.start(true);
}, delay); }, delay);
} }
@@ -68,14 +219,21 @@ class VoiceController {
this.handsFreeTimer = null; this.handsFreeTimer = null;
} }
async start(): Promise<void> { /** @param auto true when re-armed by the hands-free loop (no chime, patient timeout). */
async start(auto = false): Promise<void> {
const voice = useVoice.getState(); const voice = useVoice.getState();
if (voice.phase !== 'off') return; if (voice.phase !== 'off') return;
if (this.wake.isActive) {
// The always-on listener owns the microphone: open a command capture on it instead.
if (!auto) this.onWakeDetected('manuel');
return;
}
const settings = useSettings.getState().settings.voice; const settings = useSettings.getState().settings.voice;
const seq = ++this.startSeq;
voice.setError(null); voice.setError(null);
voice.setInterim(''); voice.setInterim('');
if (!settings.bargeIn) speech.stop(); // Barge-in keeps the assistant talking while the mic opens (echo cancellation handles the overlap).
else speech.stop(); if (!(auto && settings.bargeIn)) speech.stop();
useChat.getState().setHud('listening'); useChat.getState().setHud('listening');
voice.setPhase('arming'); voice.setPhase('arming');
@@ -85,6 +243,8 @@ class VoiceController {
} }
const sensitivity = Math.min(5, Math.max(1, Math.round(settings.sensitivity))) as 1 | 2 | 3 | 4 | 5; const sensitivity = Math.min(5, Math.max(1, Math.round(settings.sensitivity))) as 1 | 2 | 3 | 4 | 5;
// Barge-in: while the assistant talks, demand a clearly louder signal so the speaker echo cannot trigger.
const barging = auto && settings.bargeIn && speech.isSpeaking();
try { try {
await audioBus.resume(); await audioBus.resume();
await this.capture.start({ await this.capture.start({
@@ -92,20 +252,33 @@ class VoiceController {
mode: settings.captureMode, mode: settings.captureMode,
vad: { vad: {
silenceMs: settings.silenceMs, silenceMs: settings.silenceMs,
speechRatio: SENSITIVITY_RATIO[sensitivity], speechRatio: barging ? Math.min(0.9, SENSITIVITY_RATIO[sensitivity] + 0.15) : SENSITIVITY_RATIO[sensitivity],
minRms: SENSITIVITY_MIN_RMS[sensitivity], minRms: barging ? SENSITIVITY_MIN_RMS[sensitivity] * 2.5 : SENSITIVITY_MIN_RMS[sensitivity],
noSpeechTimeoutMs: useVoice.getState().handsFree ? 20_000 : 9_000 noSpeechTimeoutMs: useVoice.getState().handsFree ? Number.POSITIVE_INFINITY : 9_000
}, },
callbacks: { callbacks: {
onLevel: (level) => useVoice.getState().setInputLevel(level), onLevel: (level) => {
onSpeechStart: () => useVoice.getState().setPhase('speech'), const q = Math.round(level * 20) / 20;
if (q !== useVoice.getState().inputLevel) useVoice.getState().setInputLevel(q);
},
onSpeechStart: () => {
// The user started talking over the assistant: cut the voice (barge-in).
if (speech.isSpeaking()) speech.stop();
useVoice.getState().setPhase('speech');
useChat.getState().ping();
},
onUtterance: (wav) => void this.transcribe(wav), onUtterance: (wav) => void this.transcribe(wav),
onEnd: (reason) => this.onCaptureEnd(reason), onEnd: (reason) => this.onCaptureEnd(reason),
onError: (message) => useVoice.getState().setError(message) onError: (message) => useVoice.getState().setError(message)
} }
}); });
if (seq !== this.startSeq || useVoice.getState().phase !== 'arming') {
// stop() was called while the microphone was opening.
this.capture.cancel();
return;
}
useVoice.getState().setPhase('listening'); useVoice.getState().setPhase('listening');
this.playChime(true); if (!auto) this.playChime(true);
} catch (err) { } catch (err) {
const message = (err as Error).message; const message = (err as Error).message;
useVoice.getState().setError(message); useVoice.getState().setError(message);
@@ -135,6 +308,7 @@ class VoiceController {
stop(): void { stop(): void {
this.stopping = true; this.stopping = true;
this.startSeq++;
this.clearHandsFreeTimer(); this.clearHandsFreeTimer();
if (useSettings.getState().settings.voice.provider === 'browser') this.browser.stop(); if (useSettings.getState().settings.voice.provider === 'browser') this.browser.stop();
else if (this.capture.isActive) this.capture.stop('stopped'); else if (this.capture.isActive) this.capture.stop('stopped');
@@ -143,6 +317,7 @@ class VoiceController {
} }
cancel(): void { cancel(): void {
this.startSeq++;
this.clearHandsFreeTimer(); this.clearHandsFreeTimer();
this.browser.stop(); this.browser.stop();
this.capture.cancel(); this.capture.cancel();
@@ -162,19 +337,43 @@ class VoiceController {
} }
} }
private async transcribe(wav: WavResult): Promise<void> { private async transcribe(wav: WavResult, fromWake = false): Promise<void> {
const voice = useVoice.getState(); const voice = useVoice.getState();
voice.setPhase('transcribing'); voice.setPhase('transcribing');
useChat.getState().setHud('thinking'); useChat.getState().setHud('thinking');
this.playChime(false); this.playChime(false);
const settings = useSettings.getState().settings.voice; const settings = useSettings.getState().settings.voice;
try { try {
const text = await transcribeWav(wav.bytes, settings); let text = await transcribeWav(wav.bytes, settings);
if (isTranscriptNoise(text)) {
Log.debug('voice', `transcript ignored as noise: ${text}`);
text = '';
}
voice.setTranscript(text); voice.setTranscript(text);
voice.setPhase('off'); voice.setPhase('off');
Log.info('voice', `transcript (${wav.durationSec.toFixed(1)}s): ${text}`); Log.info('voice', `transcript (${wav.durationSec.toFixed(1)}s): ${text}`);
const filterByTranscript = settings.wakeMode === 'transcript' || (settings.wakeMode === 'off' && settings.wakeWordEnabled);
if (!fromWake && voice.handsFree && filterByTranscript && Date.now() > this.attentionUntil) {
const { matched, rest } = matchWakeWord(text, settings.wakeWord || 'jarvis');
if (!matched) {
Log.debug('voice', 'utterance ignored: no wake word');
voice.setTranscript('');
useChat.getState().setHud('idle');
this.scheduleHandsFree(200);
return;
}
if (!rest.trim()) {
this.attentionUntil = Date.now() + 10_000;
useChat.getState().setHud('listening');
speech.say('Oui ?');
return;
}
text = rest;
}
this.attentionUntil = 0;
if (text.trim()) { if (text.trim()) {
await sendMessage(text, [], 'voice'); await sendMessage(text, [], 'voice');
if (useVoice.getState().handsFree && !speech.isSpeaking()) this.scheduleHandsFree(350);
} else { } else {
voice.setError('Transcription vide.'); voice.setError('Transcription vide.');
useChat.getState().setHud('idle'); useChat.getState().setHud('idle');
@@ -188,6 +387,7 @@ class VoiceController {
useChat.getState().setHud('error'); useChat.getState().setHud('error');
Log.error('voice', `transcription failed: ${message}`); Log.error('voice', `transcription failed: ${message}`);
setTimeout(() => useChat.getState().setHud('idle'), 3000); setTimeout(() => useChat.getState().setHud('idle'), 3000);
if (useVoice.getState().handsFree) this.scheduleHandsFree(3000);
} }
} }
+48
View File
@@ -0,0 +1,48 @@
/**
* Applies the preferred voice gender to the active TTS provider: switches the local model/speaker,
* downloads a matching French voice when none is installed, and logs what it did.
*/
import { Log } from '../../lib/log';
import { findLocalVoice, inferGender, suggestedDownload } from '../../lib/voicePreference';
import { useSettings } from '../../state/settings';
import { useVoiceModels } from '../../state/voiceModels';
let downloading: string | null = null;
/** Make the local voice match `speech.voiceGender`. Returns a short status message. */
export async function ensurePreferredVoice(): Promise<string> {
const { settings, update } = useSettings.getState();
const { speech } = settings;
const gender = speech.voiceGender ?? 'male';
if (speech.provider !== 'local') return '';
const models = useVoiceModels.getState().models;
if (!models.length) return '';
const lang = speech.language || 'fr-FR';
const current = models.find((m) => m.id === speech.localModel);
const currentSpeaker = current?.speakers?.find((s) => s.id === speech.localSpeaker);
const currentGender = currentSpeaker ? (currentSpeaker as { gender?: string }).gender === 'm' ? 'male' : (currentSpeaker as { gender?: string }).gender === 'f' ? 'female' : inferGender(currentSpeaker.name) : undefined;
if (current?.installed && currentGender === gender) return '';
const choice = findLocalVoice(models, lang, gender);
if (choice) {
update({ speech: { localModel: choice.modelId, localSpeaker: choice.speaker } });
const name = models.find((m) => m.id === choice.modelId)?.speakers?.find((s) => s.id === choice.speaker)?.name ?? choice.modelId;
Log.info('tts', `voice preference ${gender}: ${name}`);
return `Voix ${gender === 'male' ? 'masculine' : 'féminine'} : ${name}`;
}
const download = suggestedDownload(models, lang, gender);
if (!download || downloading === download) return download ? 'Téléchargement de la voix en cours…' : '';
downloading = download;
Log.info('tts', `no ${gender} voice installed, downloading ${download}`);
try {
await useVoiceModels.getState().download(download);
const after = findLocalVoice(useVoiceModels.getState().models, lang, gender);
if (after) update({ speech: { localModel: after.modelId, localSpeaker: after.speaker } });
return after ? `Voix ${gender === 'male' ? 'masculine' : 'féminine'} installée.` : 'Voix téléchargée.';
} catch (err) {
Log.warn('tts', `voice download failed: ${(err as Error).message}`);
return `Téléchargement impossible : ${(err as Error).message}`;
} finally {
downloading = null;
}
}
+276
View File
@@ -0,0 +1,276 @@
/**
* Always-on microphone for the keyword spotter. Streams 16 kHz PCM to the main process
* (which feeds the sherpa-onnx spotter); once the wake word is detected it captures the
* following utterance with the energy VAD and hands back a WAV, then resumes spotting.
* One microphone stream, no re-opening between phrases.
*/
import { bridge } from '../../lib/bridge';
import { Log } from '../../lib/log';
import { audioBus } from './audioBus';
import { loadWorklet } from './capture';
import { DEFAULT_VAD, EnergyVad, type VadOptions } from './vad';
import { buildWav16k, rms, type WavResult } from './wav';
import type { VadEvent } from '../../../shared/voice';
const WORKLET_SOURCE = `
class EveFlowWakeProcessor extends AudioWorkletProcessor {
constructor() { super(); this.buffer = new Float32Array(2048); this.offset = 0; }
process(inputs) {
const channel = inputs[0] && inputs[0][0];
if (!channel) return true;
let i = 0;
while (i < channel.length) {
const n = Math.min(channel.length - i, this.buffer.length - this.offset);
this.buffer.set(channel.subarray(i, i + n), this.offset);
this.offset += n; i += n;
if (this.offset === this.buffer.length) {
this.port.postMessage(this.buffer, [this.buffer.buffer]);
this.buffer = new Float32Array(2048); this.offset = 0;
}
}
return true;
}
}
registerProcessor('eveflow-wake', EveFlowWakeProcessor);
`;
export type WakePhase = 'off' | 'spotting' | 'command' | 'speech';
export interface WakeCallbacks {
onPhase: (phase: WakePhase) => void;
onLevel?: (level: number) => void;
onUtterance: (wav: WavResult) => void;
onNoSpeech: () => void;
onError: (message: string) => void;
}
let workletUrl: string | null = null;
export class WakeListener {
private ctx: AudioContext | null = null;
private stream: MediaStream | null = null;
private node: AudioWorkletNode | null = null;
private phase: WakePhase = 'off';
private vad: EnergyVad | null = null;
private chunks: Float32Array[] = [];
private totalSamples = 0;
private sampleRate = 16_000;
private pending: Float32Array[] = [];
private pendingSamples = 0;
private callbacks: WakeCallbacks | null = null;
private vadOptions: Partial<VadOptions> = {};
private noSpeechTimer: ReturnType<typeof setTimeout> | null = null;
/** Neural end-of-speech (Silero in the main process) instead of the energy VAD. */
private neural = false;
private unsubscribeVad: (() => void) | null = null;
private vadPending: Float32Array[] = [];
private vadPendingSamples = 0;
get isActive(): boolean {
return this.phase !== 'off';
}
get currentPhase(): WakePhase {
return this.phase;
}
async start(options: { deviceId?: string; vad?: Partial<VadOptions>; neuralVad?: boolean; callbacks: WakeCallbacks }): Promise<void> {
if (this.phase !== 'off') return;
this.callbacks = options.callbacks;
this.vadOptions = options.vad ?? {};
this.neural = !!options.neuralVad;
if (this.neural) {
this.unsubscribeVad?.();
this.unsubscribeVad = bridge()?.voice.onVadEvent((event) => this.onVadEvent(event)) ?? null;
}
this.stream = await navigator.mediaDevices.getUserMedia({
audio: {
deviceId: options.deviceId ? { exact: options.deviceId } : undefined,
echoCancellation: true,
noiseSuppression: true,
autoGainControl: true,
channelCount: 1
}
});
try {
this.ctx = new AudioContext({ sampleRate: 16_000, latencyHint: 'playback' });
} catch {
this.ctx = new AudioContext({ latencyHint: 'playback' });
}
if (this.ctx.state === 'suspended') await this.ctx.resume().catch(() => undefined);
this.sampleRate = this.ctx.sampleRate;
const source = this.ctx.createMediaStreamSource(this.stream);
const analyser = this.ctx.createAnalyser();
analyser.fftSize = 512;
source.connect(analyser);
audioBus.setInputAnalyser(analyser);
await loadWorklet(this.ctx, 'eveflow-wake', () => {
if (!workletUrl) workletUrl = URL.createObjectURL(new Blob([WORKLET_SOURCE], { type: 'application/javascript' }));
return workletUrl;
});
this.node = new AudioWorkletNode(this.ctx, 'eveflow-wake', { numberOfInputs: 1, numberOfOutputs: 0, channelCount: 1 });
this.node.port.onmessage = (event: MessageEvent<Float32Array>) => this.onSamples(event.data);
source.connect(this.node);
this.setPhase('spotting');
Log.info('wake', `listener started (${this.sampleRate} Hz, fin de phrase ${this.neural ? 'Silero' : 'énergie'})`);
}
private onVadEvent(event: VadEvent): void {
if (!this.neural || (this.phase !== 'command' && this.phase !== 'speech')) return;
if (event.type === 'speech-start') {
if (this.phase === 'command') {
this.setPhase('speech');
if (this.noSpeechTimer) clearTimeout(this.noSpeechTimer);
this.noSpeechTimer = null;
}
} else if (event.type === 'segment') {
const wav: WavResult = { bytes: event.wav, sampleRate: 16_000, durationSec: event.durationSec };
this.resumeSpotting();
if (wav.durationSec >= 0.25) this.callbacks?.onUtterance(wav);
else this.callbacks?.onNoSpeech();
} else if (event.type === 'error') {
this.callbacks?.onError(event.message);
this.resumeSpotting();
}
}
/** Called by the controller when the main process reports the wake word (or on manual trigger). */
beginCommand(): void {
if (this.phase === 'off' || this.phase === 'command' || this.phase === 'speech') return;
this.vad = this.neural ? null : new EnergyVad({ ...DEFAULT_VAD, ...this.vadOptions, noSpeechTimeoutMs: Number.POSITIVE_INFINITY });
this.vadPending = [];
this.vadPendingSamples = 0;
this.chunks = [];
this.totalSamples = 0;
this.setPhase('command');
if (this.noSpeechTimer) clearTimeout(this.noSpeechTimer);
this.noSpeechTimer = setTimeout(() => {
if (this.phase === 'command') {
this.resumeSpotting();
this.callbacks?.onNoSpeech();
}
}, 8000);
}
/** Abort a command capture and go back to spotting. */
cancelCommand(): void {
if (this.phase === 'command' || this.phase === 'speech') this.resumeSpotting();
}
stop(): void {
if (this.phase === 'off') return;
if (this.noSpeechTimer) clearTimeout(this.noSpeechTimer);
this.noSpeechTimer = null;
this.unsubscribeVad?.();
this.unsubscribeVad = null;
audioBus.setInputAnalyser(null);
if (this.node) {
this.node.port.onmessage = null;
this.node.disconnect();
this.node = null;
}
this.stream?.getTracks().forEach((t) => t.stop());
this.stream = null;
void this.ctx?.close().catch(() => undefined);
this.ctx = null;
this.chunks = [];
this.pending = [];
this.pendingSamples = 0;
this.vadPending = [];
this.vadPendingSamples = 0;
this.setPhase('off');
Log.info('wake', 'listener stopped');
}
private setPhase(phase: WakePhase): void {
this.phase = phase;
this.callbacks?.onPhase(phase);
}
private resumeSpotting(): void {
if (this.noSpeechTimer) clearTimeout(this.noSpeechTimer);
this.noSpeechTimer = null;
this.vad = null;
this.chunks = [];
this.totalSamples = 0;
this.setPhase('spotting');
}
private onSamples(samples: Float32Array): void {
if (this.phase === 'off') return;
const level = rms(samples);
this.callbacks?.onLevel?.(Math.min(1, level * 6));
if (this.phase === 'spotting') {
// Batch ~256 ms of audio per IPC message for the keyword spotter.
this.pending.push(samples);
this.pendingSamples += samples.length;
if (this.pendingSamples >= this.sampleRate * 0.25) this.flushToSpotter();
return;
}
if (this.neural) {
// command / speech with Silero: stream ~128 ms frames to the main process, which returns the segment.
this.vadPending.push(samples);
this.vadPendingSamples += samples.length;
if (this.vadPendingSamples >= this.sampleRate * 0.128) this.flushToVad();
return;
}
// command / speech: collect the utterance
this.chunks.push(samples);
this.totalSamples += samples.length;
if (this.vad && this.phase === 'command') {
// keep only a short pre-roll before speech starts
while (this.chunks.length > 1 && this.totalSamples - this.chunks[0].length > this.sampleRate * 0.4) {
this.totalSamples -= this.chunks.shift()!.length;
}
}
if (!this.vad) return;
const signal = this.vad.feed(level, (samples.length / this.sampleRate) * 1000);
if (signal === 'speech-start') {
this.setPhase('speech');
if (this.noSpeechTimer) clearTimeout(this.noSpeechTimer);
this.noSpeechTimer = null;
} else if (signal === 'speech-end' || signal === 'max-length') {
const wav = buildWav16k(this.chunks, this.sampleRate);
this.resumeSpotting();
if (wav.durationSec >= 0.25) this.callbacks?.onUtterance(wav);
else this.callbacks?.onNoSpeech();
} else if (signal === 'too-short') {
this.resumeSpotting();
this.callbacks?.onNoSpeech();
}
}
private static toInt16(chunks: Float32Array[], total: number): Uint8Array {
const merged = new Int16Array(total);
let offset = 0;
for (const chunk of chunks) {
for (let i = 0; i < chunk.length; i++) {
const s = Math.max(-1, Math.min(1, chunk[i]));
merged[offset + i] = s < 0 ? s * 0x8000 : s * 0x7fff;
}
offset += chunk.length;
}
return new Uint8Array(merged.buffer);
}
private flushToSpotter(): void {
const api = bridge();
if (!api) return;
const bytes = WakeListener.toInt16(this.pending, this.pendingSamples);
this.pending = [];
this.pendingSamples = 0;
api.voice.kwsAudio(bytes, this.sampleRate);
}
private flushToVad(): void {
const api = bridge();
if (!api) return;
const bytes = WakeListener.toInt16(this.vadPending, this.vadPendingSamples);
this.vadPending = [];
this.vadPendingSamples = 0;
api.voice.vadAudio(bytes, this.sampleRate);
}
}
+28 -1
View File
@@ -53,6 +53,20 @@ interface ChatStore {
hudOverride: HudState | null; hudOverride: HudState | null;
error: string | null; error: string | null;
draft: string; draft: string;
/** Incremented to make the core flash briefly. */
pingCount: number;
/** Time to first token of the current reply, in ms. */
latencyMs: number | null;
/** Pushes received while the window was hidden / compact, cleared when the user looks. */
unread: number;
/** Mission mode routes the next messages to the "mission" model (long tasks). */
missionMode: boolean;
/** True while quiet hours apply (computed by App). */
quiet: boolean;
incUnread: () => void;
markRead: () => void;
setMissionMode: (on: boolean) => void;
setQuiet: (quiet: boolean) => void;
addMessage: (message: Omit<ChatMessage, 'id' | 'timestamp'> & Partial<Pick<ChatMessage, 'id' | 'timestamp'>>) => string; addMessage: (message: Omit<ChatMessage, 'id' | 'timestamp'> & Partial<Pick<ChatMessage, 'id' | 'timestamp'>>) => string;
updateMessage: (id: string, patch: Partial<ChatMessage> | ((m: ChatMessage) => Partial<ChatMessage>)) => void; updateMessage: (id: string, patch: Partial<ChatMessage> | ((m: ChatMessage) => Partial<ChatMessage>)) => void;
@@ -68,6 +82,8 @@ interface ChatStore {
setHudOverride: (state: HudState | null) => void; setHudOverride: (state: HudState | null) => void;
setError: (error: string | null) => void; setError: (error: string | null) => void;
setDraft: (draft: string) => void; setDraft: (draft: string) => void;
ping: () => void;
setLatency: (ms: number | null) => void;
} }
const MAX_MESSAGES = 400; const MAX_MESSAGES = 400;
@@ -83,6 +99,15 @@ export const useChat = create<ChatStore>((set, get) => ({
hudOverride: null, hudOverride: null,
error: null, error: null,
draft: '', draft: '',
pingCount: 0,
latencyMs: null,
unread: 0,
missionMode: false,
quiet: false,
incUnread: () => set((s) => ({ unread: Math.min(99, s.unread + 1) })),
markRead: () => set((s) => (s.unread ? { unread: 0 } : {})),
setMissionMode: (missionMode) => set({ missionMode }),
setQuiet: (quiet) => set((s) => (s.quiet === quiet ? {} : { quiet })),
addMessage: (message) => { addMessage: (message) => {
const id = message.id ?? uid('msg'); const id = message.id ?? uid('msg');
@@ -137,5 +162,7 @@ export const useChat = create<ChatStore>((set, get) => ({
}, },
setHudOverride: (hudOverride) => set({ hudOverride }), setHudOverride: (hudOverride) => set({ hudOverride }),
setError: (error) => set({ error }), setError: (error) => set({ error }),
setDraft: (draft) => set({ draft }) setDraft: (draft) => set({ draft }),
ping: () => set((s) => ({ pingCount: s.pingCount + 1 })),
setLatency: (latencyMs) => set({ latencyMs })
})); }));
+79 -10
View File
@@ -2,7 +2,7 @@ import { create } from 'zustand';
import type { WebhookStatus } from '../../shared/ipc'; import type { WebhookStatus } from '../../shared/ipc';
import { Log } from '../lib/log'; import { Log } from '../lib/log';
import { persistGet, persistSet } from '../lib/persist'; import { persistGet, persistSet } from '../lib/persist';
import { HermesClient, jobOutput, jobStatus, resolveTransport, type ResolvedTransport } from '../services/hermes/client'; import { HermesClient, jobOutput, jobStatus, resolveTransport, type ResolvedTransport, discoverHermesUrl } from '../services/hermes/client';
import type { import type {
HermesCapabilities, HermesCapabilities,
HermesHealth, HermesHealth,
@@ -38,12 +38,16 @@ interface HermesStore {
sessions: HermesSession[]; sessions: HermesSession[];
jobs: HermesJob[]; jobs: HermesJob[];
jobRuns: JobRun[]; jobRuns: JobRun[];
/** Last cron sync failure (the chat link stays independent of it). */
jobsError: string | null;
/** True while probing alternative API URLs after a failed connection. */
discovering: boolean;
transport: ResolvedTransport; transport: ResolvedTransport;
lastSyncAt: number | null; lastSyncAt: number | null;
webhook: WebhookStatus | null; webhook: WebhookStatus | null;
busy: boolean; busy: boolean;
client: () => HermesClient; client: (modelOverride?: string) => HermesClient;
connect: () => Promise<void>; connect: () => Promise<void>;
refreshJobs: () => Promise<void>; refreshJobs: () => Promise<void>;
refreshSessions: () => Promise<void>; refreshSessions: () => Promise<void>;
@@ -57,6 +61,8 @@ interface HermesStore {
} }
const CACHE_KEY = 'eveflow.hermes.cache.v2'; const CACHE_KEY = 'eveflow.hermes.cache.v2';
let connectInflight: Promise<void> | null = null;
let lastCacheSnapshot = '';
const isTerminal = (status: string) => ['ok', 'failed', 'delivery_failed', 'completed', 'error'].includes(status.toLowerCase()); const isTerminal = (status: string) => ['ok', 'failed', 'delivery_failed', 'completed', 'error'].includes(status.toLowerCase());
function runFromJob(job: HermesJob): JobRun | null { function runFromJob(job: HermesJob): JobRun | null {
@@ -86,14 +92,23 @@ export const useHermes = create<HermesStore>((set, get) => ({
sessions: [], sessions: [],
jobs: [], jobs: [],
jobRuns: [], jobRuns: [],
jobsError: null,
discovering: false,
transport: 'completions', transport: 'completions',
lastSyncAt: null, lastSyncAt: null,
webhook: null, webhook: null,
busy: false, busy: false,
client: () => new HermesClient(useSettings.getState().settings.hermes), client: (modelOverride) => {
const config = useSettings.getState().settings.hermes;
// Without an explicit model, use the alias advertised by /v1/models (Hermes rejects unknown names).
const model = (modelOverride ?? '').trim() || config.model.trim() || get().models[0]?.id || '';
return new HermesClient({ ...config, model });
},
connect: async () => { connect: () => {
if (connectInflight) return connectInflight;
connectInflight = (async () => {
const config = useSettings.getState().settings.hermes; const config = useSettings.getState().settings.hermes;
if (!config.url.trim()) { if (!config.url.trim()) {
set({ link: 'offline', linkDetail: 'URL Hermes non configurée' }); set({ link: 'offline', linkDetail: 'URL Hermes non configurée' });
@@ -107,20 +122,42 @@ export const useHermes = create<HermesStore>((set, get) => ({
try { try {
capabilities = await client.capabilities(); capabilities = await client.capabilities();
} catch (err) { } catch (err) {
// A web page on /v1/* means the whole API is behind a portal even if /health passed through.
if (/page web/.test((err as Error).message)) throw err;
Log.warn('hermes', `capabilities unavailable: ${(err as Error).message}`); Log.warn('hermes', `capabilities unavailable: ${(err as Error).message}`);
} }
const transport = resolveTransport(config, capabilities); const transport = resolveTransport(config, capabilities);
const degraded = String(health.status ?? 'ok').toLowerCase() !== 'ok'; const degraded = !isHealthyStatus(health.status);
set({ health, capabilities, transport, link: degraded ? 'degraded' : 'online', linkDetail: degraded ? `état ${health.status}` : '' }); set({ health, capabilities, transport, link: degraded ? 'degraded' : 'online', linkDetail: degraded ? describeHealth(health) : '' });
Log.info('hermes', `connected (${transport})`, { status: health.status, model: capabilities?.model }); Log.info('hermes', `connected (${transport})`, { status: health.status, model: capabilities?.model });
void get().refreshCatalog(); void get().refreshCatalog();
void get().refreshJobs(); void get().refreshJobs();
void get().refreshSessions(); void get().refreshSessions();
} catch (err) { } catch (err) {
const message = (err as Error).message; const message = (err as Error).message;
// The URL answers with a web page (portal, dashboard) or nothing: look for the API on the same host.
if (!get().discovering && /page web|illisible|fetch failed|ECONNREFUSED|404/i.test(message)) {
set({ discovering: true, linkDetail: 'recherche de l’API Hermes…' });
try {
const found = await discoverHermesUrl(config);
if (found) {
Log.info('hermes', `API found at ${found} (was ${config.url})`);
useSettings.getState().update({ hermes: { url: found } });
set({ discovering: false, linkDetail: `URL corrigée automatiquement : ${found}` });
await get().connect();
return;
}
} finally {
set({ discovering: false });
}
}
set({ link: 'offline', linkDetail: message, transport: resolveTransport(config, null) }); set({ link: 'offline', linkDetail: message, transport: resolveTransport(config, null) });
Log.warn('hermes', `connection failed: ${message}`); Log.warn('hermes', `connection failed: ${message}`);
} }
})().finally(() => {
connectInflight = null;
});
return connectInflight;
}, },
refreshCatalog: async () => { refreshCatalog: async () => {
@@ -152,13 +189,17 @@ export const useHermes = create<HermesStore>((set, get) => ({
const merged = [...incoming, ...previous.filter((r) => !incoming.some((i) => i.id === r.id))] const merged = [...incoming, ...previous.filter((r) => !incoming.some((i) => i.id === r.id))]
.sort((a, b) => new Date(b.at).getTime() - new Date(a.at).getTime()) .sort((a, b) => new Date(b.at).getTime() - new Date(a.at).getTime())
.slice(0, 100); .slice(0, 100);
set({ jobs, jobRuns: merged, lastSyncAt: Date.now(), link: get().link === 'offline' ? 'online' : get().link }); set({ jobs, jobRuns: merged, jobsError: null, lastSyncAt: Date.now(), link: get().link === 'offline' ? 'online' : get().link });
persistSet(CACHE_KEY, { jobs, jobRuns: merged, syncedAt: Date.now() }); const snapshot = JSON.stringify({ jobs, jobRuns: merged });
if (snapshot !== lastCacheSnapshot) {
lastCacheSnapshot = snapshot;
persistSet(CACHE_KEY, { jobs, jobRuns: merged, syncedAt: Date.now() });
}
} catch (err) { } catch (err) {
const message = (err as Error).message; const message = (err as Error).message;
Log.warn('hermes', `jobs sync failed: ${message}`); Log.warn('hermes', `jobs sync failed: ${message}`);
if (/HTTP 404/.test(message)) return; // jobs API disabled on this server // The cron API can be absent or restricted on a given Hermes: the chat link is unaffected.
set({ link: 'offline', linkDetail: message }); set({ jobsError: /HTTP 404/.test(message) ? 'API des crons absente sur ce serveur Hermes' : message });
} }
}, },
@@ -215,3 +256,31 @@ export const useHermes = create<HermesStore>((set, get) => ({
})); }));
export { jobStatus }; export { jobStatus };
const HEALTHY = new Set(['ok', 'healthy', 'up', 'alive', 'pass', 'ready', 'running', 'online', 'true']);
/** Hermes /health reports "ok"; /health/detailed may report "healthy", "degraded" or "unhealthy". */
export function isHealthyStatus(status: unknown): boolean {
if (status === undefined || status === null || status === '') return true;
return HEALTHY.has(String(status).toLowerCase());
}
/** Human summary of a degraded health payload: failing checks by name. */
export function describeHealth(health: Record<string, unknown>): string {
const failing: string[] = [];
const visit = (node: unknown, prefix: string, depth: number) => {
if (!node || typeof node !== 'object' || depth > 3) return;
for (const [key, value] of Object.entries(node as Record<string, unknown>)) {
if (key === 'status') continue;
if (value && typeof value === 'object') {
const rec = value as Record<string, unknown>;
const st = rec.status ?? rec.ok ?? rec.healthy;
if (st !== undefined && !isHealthyStatus(st)) failing.push(prefix + key);
else visit(value, `${prefix}${key}.`, depth + 1);
} else if (typeof value === 'boolean' && !value && /ok|healthy|ready|connected|available/i.test(key)) failing.push(prefix + key);
}
};
visit(health, '', 0);
const base = `état ${String(health.status)}`;
return failing.length ? `${base} · ${failing.slice(0, 4).join(', ')}` : base;
}
+41 -2
View File
@@ -16,6 +16,16 @@ export interface VoiceSettings extends SttConfig {
silenceMs: number; silenceMs: number;
sensitivity: number; // 1 (low) .. 5 (high) sensitivity: number; // 1 (low) .. 5 (high)
wakeChime: boolean; wakeChime: boolean;
/** Hands-free: only react to utterances starting with this word (local STT recommended). */
wakeWordEnabled: boolean;
wakeWord: string;
/** off = push-to-talk / hands-free; transcript = filter after transcription; kws = always-on keyword spotting. */
wakeMode: 'off' | 'transcript' | 'kws';
kwsSensitivity: number; // 1..5
/** Use Silero VAD for end-of-speech in always-on mode when the model is installed. */
neuralVad: boolean;
/** Execute short system intents locally (lock, volume, open app…) instead of asking Hermes. */
localCommands: boolean;
} }
export interface SpeechSettings extends TtsConfig { export interface SpeechSettings extends TtsConfig {
@@ -29,6 +39,20 @@ export interface WebhookSettings {
secret: string; secret: string;
} }
export interface NotificationSettings {
/** Quiet hours: no spoken pushes, no chime, dimmed HUD (24 h "HH:MM"). */
quietEnabled: boolean;
quietStart: string;
quietEnd: string;
/** Pushes whose text or job name contains one of these words are spoken even during quiet hours. */
priorityKeywords: string;
/** Speak only the first sentences of incoming pushes (cron reports can be long). */
summarizeIncoming: boolean;
summarySentences: number;
/** Dim the HUD (night theme) during quiet hours. */
nightTheme: boolean;
}
export interface Settings { export interface Settings {
version: 2; version: 2;
assistantName: string; assistantName: string;
@@ -39,6 +63,7 @@ export interface Settings {
voice: VoiceSettings; voice: VoiceSettings;
speech: SpeechSettings; speech: SpeechSettings;
webhook: WebhookSettings; webhook: WebhookSettings;
notifications: NotificationSettings;
ui: { ui: {
showTelemetry: boolean; showTelemetry: boolean;
showReasoning: boolean; showReasoning: boolean;
@@ -63,7 +88,8 @@ export const DEFAULT_SETTINGS: Settings = {
reasoningEffort: '', reasoningEffort: '',
instructions: instructions:
"Tu es l'interface vocale EveFlow (style JARVIS). Réponds en français, de façon concise et orale quand la question est simple; utilise le Markdown uniquement pour le contenu structuré (code, listes, tableaux). Les images doivent être des URL http(s) ou des fichiers du dossier partagé.", "Tu es l'interface vocale EveFlow (style JARVIS). Réponds en français, de façon concise et orale quand la question est simple; utilise le Markdown uniquement pour le contenu structuré (code, listes, tableaux). Les images doivent être des URL http(s) ou des fichiers du dossier partagé.",
localTools: true localTools: true,
missionModel: ''
}, },
voice: { voice: {
provider: 'openai-compatible', provider: 'openai-compatible',
@@ -77,7 +103,14 @@ export const DEFAULT_SETTINGS: Settings = {
micDeviceId: '', micDeviceId: '',
silenceMs: 900, silenceMs: 900,
sensitivity: 3, sensitivity: 3,
wakeChime: true wakeChime: true,
wakeWordEnabled: false,
wakeWord: 'jarvis',
wakeMode: 'off',
kwsSensitivity: 3,
neuralVad: true,
localCommands: true,
localModel: 'whisper-base'
}, },
speech: { speech: {
provider: 'openai-compatible', provider: 'openai-compatible',
@@ -90,10 +123,15 @@ export const DEFAULT_SETTINGS: Settings = {
systemVoice: '', systemVoice: '',
language: 'fr-FR', language: 'fr-FR',
volume: 1, volume: 1,
localModel: 'kokoro-v1',
localSpeaker: 30,
voiceGender: 'male',
timbre: 'jarvis',
autoSpeak: true, autoSpeak: true,
speakIncoming: true speakIncoming: true
}, },
webhook: { enabled: true, port: 7842, secret: '' }, webhook: { enabled: true, port: 7842, secret: '' },
notifications: { quietEnabled: false, quietStart: '22:30', quietEnd: '07:30', priorityKeywords: 'urgent, alerte, alarme, panne', summarizeIncoming: true, summarySentences: 2, nightTheme: true },
ui: { showTelemetry: true, showReasoning: false, reduceMotion: false, compactOpacity: 0.92 }, ui: { showTelemetry: true, showReasoning: false, reduceMotion: false, compactOpacity: 0.92 },
hermesSessionId: '' hermesSessionId: ''
}; };
@@ -173,6 +211,7 @@ export const useSettings = create<SettingsStore>((set, get) => ({
} }
const merged = merge(DEFAULT_SETTINGS, saved); const merged = merge(DEFAULT_SETTINGS, saved);
if (!merged.hermesSessionId) merged.hermesSessionId = uid('eveflow'); if (!merged.hermesSessionId) merged.hermesSessionId = uid('eveflow');
if (saved && saved.voice && saved.voice.wakeMode === undefined && saved.voice.wakeWordEnabled) merged.voice.wakeMode = 'transcript';
set({ settings: merged, loaded: true }); set({ settings: merged, loaded: true });
persistSet(STORAGE_KEY, merged); persistSet(STORAGE_KEY, merged);
}, },
+11
View File
@@ -2,6 +2,7 @@ import { create } from 'zustand';
import type { TtsState } from '../services/voice/tts'; import type { TtsState } from '../services/voice/tts';
export type ListenPhase = 'off' | 'arming' | 'listening' | 'speech' | 'transcribing'; export type ListenPhase = 'off' | 'arming' | 'listening' | 'speech' | 'transcribing';
export type WakeState = 'off' | 'starting' | 'spotting' | 'error';
interface VoiceStore { interface VoiceStore {
phase: ListenPhase; phase: ListenPhase;
@@ -12,6 +13,11 @@ interface VoiceStore {
interim: string; interim: string;
error: string | null; error: string | null;
micDevices: Array<{ deviceId: string; label: string }>; micDevices: Array<{ deviceId: string; label: string }>;
wake: WakeState;
wakeKeywords: string[];
neuralVad: boolean;
setNeuralVad: (on: boolean) => void;
setWake: (state: WakeState, keywords?: string[]) => void;
setPhase: (phase: ListenPhase) => void; setPhase: (phase: ListenPhase) => void;
setInputLevel: (level: number) => void; setInputLevel: (level: number) => void;
setTts: (state: TtsState) => void; setTts: (state: TtsState) => void;
@@ -31,6 +37,11 @@ export const useVoice = create<VoiceStore>((set) => ({
interim: '', interim: '',
error: null, error: null,
micDevices: [], micDevices: [],
wake: 'off',
wakeKeywords: [],
neuralVad: false,
setNeuralVad: (neuralVad) => set({ neuralVad }),
setWake: (wake, wakeKeywords) => set(wakeKeywords ? { wake, wakeKeywords } : { wake }),
setPhase: (phase) => set({ phase }), setPhase: (phase) => set({ phase }),
setInputLevel: (inputLevel) => set({ inputLevel }), setInputLevel: (inputLevel) => set({ inputLevel }),
setTts: (tts) => set({ tts }), setTts: (tts) => set({ tts }),
+91
View File
@@ -0,0 +1,91 @@
import { create } from 'zustand';
import type { VoiceDownloadProgress, VoiceEngineStatus, VoiceModelStatus } from '../../shared/voice';
import { bridge } from '../lib/bridge';
import { Log } from '../lib/log';
interface VoiceModelsStore {
models: VoiceModelStatus[];
progress: Record<string, VoiceDownloadProgress>;
engine: VoiceEngineStatus | null;
error: string | null;
refresh: () => Promise<void>;
checkEngine: () => Promise<void>;
download: (id: string) => Promise<void>;
cancel: (id: string) => Promise<void>;
remove: (id: string) => Promise<void>;
subscribe: () => () => void;
}
export const useVoiceModels = create<VoiceModelsStore>((set, get) => ({
models: [],
progress: {},
engine: null,
error: null,
refresh: async () => {
const api = bridge();
if (!api) return;
try {
set({ models: await api.voice.listModels(), error: null });
} catch (err) {
set({ error: (err as Error).message });
}
},
checkEngine: async () => {
const api = bridge();
if (!api) return;
try {
set({ engine: await api.voice.status() });
} catch (err) {
set({ engine: { available: false, error: (err as Error).message, loaded: [], modelsDir: '' } });
}
},
download: async (id) => {
const api = bridge();
if (!api) return;
set((s) => ({ progress: { ...s.progress, [id]: { id, phase: 'download', received: 0, total: 0, percent: 0 } }, error: null }));
try {
await api.voice.downloadModel(id);
} catch (err) {
const message = (err as Error).message;
Log.error('voice', `download ${id} failed: ${message}`);
set({ error: message });
} finally {
await get().refresh();
}
},
cancel: async (id) => {
await bridge()?.voice.cancelDownload(id);
},
remove: async (id) => {
const api = bridge();
if (!api) return;
try {
set({ models: await api.voice.removeModel(id) });
} catch (err) {
set({ error: (err as Error).message });
}
},
subscribe: () => {
const api = bridge();
if (!api) return () => undefined;
return api.voice.onProgress((p) => {
set((s) => {
const progress = { ...s.progress, [p.id]: p };
if (p.phase === 'done' || p.phase === 'error' || p.phase === 'cancelled') {
setTimeout(() => {
set((s2) => {
const next = { ...s2.progress };
delete next[p.id];
return { progress: next };
});
void get().refresh();
}, 1500);
}
return { progress };
});
});
}
}));
export const installedModels = (models: VoiceModelStatus[], kind: 'stt' | 'tts') => models.filter((m) => m.kind === kind && m.installed);
+54 -8
View File
@@ -7,6 +7,7 @@
grid-template-rows: 46px 1fr; grid-template-rows: 46px 1fr;
border-radius: 14px; border-radius: 14px;
overflow: hidden; overflow: hidden;
background-clip: padding-box;
background: background:
radial-gradient(1200px 700px at 50% 40%, rgba(var(--accent-rgb), 0.08), transparent 60%), radial-gradient(1200px 700px at 50% 40%, rgba(var(--accent-rgb), 0.08), transparent 60%),
linear-gradient(180deg, var(--bg-1) 0%, var(--bg-0) 100%); linear-gradient(180deg, var(--bg-1) 0%, var(--bg-0) 100%);
@@ -51,6 +52,17 @@
grid-template-columns: minmax(340px, 1fr) minmax(380px, 1fr); grid-template-columns: minmax(340px, 1fr) minmax(380px, 1fr);
} }
.hud-body > .ops-column { .hud-body > .ops-column {
position: absolute;
right: 14px;
top: 12px;
bottom: 14px;
width: min(360px, 60%);
z-index: 6;
background: var(--bg-1);
border-radius: var(--radius);
box-shadow: -20px 0 60px rgba(0, 0, 0, 0.5);
}
.hud-body > .ops-column:not(.open) {
display: none; display: none;
} }
} }
@@ -68,7 +80,6 @@
background: var(--panel); background: var(--panel);
border: 1px solid var(--line); border: 1px solid var(--line);
border-radius: var(--radius); border-radius: var(--radius);
backdrop-filter: blur(14px);
display: flex; display: flex;
flex-direction: column; flex-direction: column;
min-height: 0; min-height: 0;
@@ -310,7 +321,8 @@
background: linear-gradient(180deg, rgba(var(--accent-rgb), 0.06), transparent); background: linear-gradient(180deg, rgba(var(--accent-rgb), 0.06), transparent);
-webkit-app-region: drag; -webkit-app-region: drag;
} }
.topbar > * { .topbar button,
.topbar .chip {
-webkit-app-region: no-drag; -webkit-app-region: no-drag;
} }
.topbar .brand { .topbar .brand {
@@ -346,9 +358,6 @@
overflow: hidden; overflow: hidden;
-webkit-app-region: drag; -webkit-app-region: drag;
} }
.topbar .status-strip > * {
-webkit-app-region: no-drag;
}
.topbar .clock { .topbar .clock {
font-family: var(--font-mono); font-family: var(--font-mono);
font-size: 13px; font-size: 13px;
@@ -421,6 +430,21 @@
.core-state.success { .core-state.success {
color: var(--success); color: var(--success);
} }
.core-interim {
font-family: var(--font-ui);
font-size: 15px;
color: var(--ink-1);
max-width: 82%;
text-align: center;
opacity: 0.85;
min-height: 1.4em;
}
.core-latency {
font-family: var(--font-mono);
font-size: 10px;
letter-spacing: 0.2em;
color: var(--ink-3);
}
.core-sub { .core-sub {
font-family: var(--font-mono); font-family: var(--font-mono);
font-size: 12px; font-size: 12px;
@@ -438,10 +462,13 @@
right: 0; right: 0;
text-align: center; text-align: center;
font-family: var(--font-display); font-family: var(--font-display);
font-size: 22px; font-size: clamp(13px, 2.2vw, 22px);
font-weight: 900; font-weight: 900;
letter-spacing: 0.5em; letter-spacing: clamp(0.15em, 0.8vw, 0.5em);
padding-left: 0.5em; padding: 0 12px 0 calc(12px + 0.4em);
white-space: nowrap;
overflow: hidden;
text-overflow: ellipsis;
color: var(--ink-0); color: var(--ink-0);
text-shadow: 0 0 22px var(--accent-glow); text-shadow: 0 0 22px var(--accent-glow);
pointer-events: none; pointer-events: none;
@@ -474,6 +501,25 @@
margin-left: auto; margin-left: auto;
} }
/* ── Toasts ────────────────────────────────────────────────────────────── */
.toast-stack {
position: absolute;
left: 50%;
bottom: 22px;
transform: translateX(-50%);
z-index: 18;
display: flex;
flex-direction: column;
gap: 8px;
width: min(560px, 80%);
pointer-events: none;
}
.toast-stack .error-banner {
pointer-events: auto;
box-shadow: 0 20px 50px rgba(0, 0, 0, 0.5);
animation: rise-in 200ms ease-out;
}
/* ── Overlay / modal ───────────────────────────────────────────────────── */ /* ── Overlay / modal ───────────────────────────────────────────────────── */
.overlay { .overlay {
position: absolute; position: absolute;
-1
View File
@@ -11,7 +11,6 @@
display: flex; display: flex;
flex-direction: column; flex-direction: column;
gap: 12px; gap: 12px;
scroll-behavior: smooth;
} }
.chat-list.compact { .chat-list.compact {
padding: 8px 10px; padding: 8px 10px;
+61 -2
View File
@@ -7,7 +7,15 @@
background: linear-gradient(180deg, rgba(5, 12, 22, var(--compact-opacity, 0.92)), rgba(2, 6, 12, var(--compact-opacity, 0.92))); background: linear-gradient(180deg, rgba(5, 12, 22, var(--compact-opacity, 0.92)), rgba(2, 6, 12, var(--compact-opacity, 0.92)));
border: 1px solid rgba(var(--accent-rgb), 0.3); border: 1px solid rgba(var(--accent-rgb), 0.3);
box-shadow: 0 20px 60px rgba(0, 0, 0, 0.6), 0 0 30px rgba(var(--accent-rgb), 0.12); box-shadow: 0 20px 60px rgba(0, 0, 0, 0.6), 0 0 30px rgba(var(--accent-rgb), 0.12);
backdrop-filter: blur(16px); }
.compact-body .panel {
background: transparent;
border: none;
border-radius: 0;
}
.compact-body .panel.bracket::before,
.compact-body .panel.bracket::after {
display: none;
} }
.compact-head { .compact-head {
display: flex; display: flex;
@@ -29,7 +37,7 @@
} }
.compact-core { .compact-core {
position: relative; position: relative;
height: 220px; height: clamp(140px, 38vh, 220px);
flex-shrink: 0; flex-shrink: 0;
} }
.compact-core canvas { .compact-core canvas {
@@ -59,3 +67,54 @@
font-size: 13px; font-size: 13px;
padding: 8px 11px; padding: 8px 11px;
} }
/* Glanceable strip: state, unread badge, last sentence. */
.compact-glance {
display: grid;
grid-template-columns: auto 1fr auto;
align-items: center;
gap: 8px;
padding: 4px 12px 8px;
font-family: var(--font-mono);
font-size: 10.5px;
letter-spacing: 0.12em;
text-transform: uppercase;
color: var(--ink-2);
}
.compact-glance .state {
color: var(--accent);
}
.compact-glance .state.alert {
color: var(--danger);
}
.compact-glance .last {
font-family: var(--font-body);
font-size: 12.5px;
letter-spacing: 0;
text-transform: none;
color: var(--ink-1);
white-space: nowrap;
overflow: hidden;
text-overflow: ellipsis;
}
.compact-glance .badges {
display: flex;
gap: 6px;
align-items: center;
}
.compact-glance .unread {
min-width: 18px;
height: 18px;
padding: 0 5px;
border-radius: 9px;
background: var(--accent);
color: #03101c;
font-weight: 700;
font-size: 11px;
display: inline-flex;
align-items: center;
justify-content: center;
}
.compact-glance .mission {
color: var(--warn, #f5c451);
}
+43
View File
@@ -104,3 +104,46 @@
background: linear-gradient(90deg, var(--accent), var(--success)); background: linear-gradient(90deg, var(--accent), var(--success));
transition: width 60ms linear; transition: width 60ms linear;
} }
.switch {
width: 100%;
background: transparent;
border-left: none;
border-right: none;
border-top: none;
color: inherit;
text-align: left;
font: inherit;
}
/* Segmented choice (voice gender, capture mode…) */
.segmented {
display: inline-flex;
border: 1px solid var(--line);
border-radius: 10px;
overflow: hidden;
background: rgba(255, 255, 255, 0.03);
}
.segmented button {
padding: 8px 16px;
background: transparent;
border: none;
color: var(--ink-1);
font: inherit;
font-size: 13px;
letter-spacing: 0.02em;
cursor: pointer;
transition: background 0.15s, color 0.15s;
}
.segmented button + button {
border-left: 1px solid var(--line);
}
.segmented button:hover {
color: var(--ink-0);
background: rgba(var(--accent-rgb), 0.08);
}
.segmented button.active {
color: #03101c;
background: var(--accent);
font-weight: 600;
}
+33 -3
View File
@@ -1,5 +1,3 @@
@import url('https://fonts.googleapis.com/css2?family=Orbitron:wght@500;700;900&family=Rajdhani:wght@400;500;600;700&family=Share+Tech+Mono&display=swap');
:root { :root {
--font-display: 'Orbitron', 'Rajdhani', 'Segoe UI', sans-serif; --font-display: 'Orbitron', 'Rajdhani', 'Segoe UI', sans-serif;
--font-ui: 'Rajdhani', 'Segoe UI', 'Helvetica Neue', sans-serif; --font-ui: 'Rajdhani', 'Segoe UI', 'Helvetica Neue', sans-serif;
@@ -137,7 +135,39 @@ textarea {
background: rgba(var(--accent-rgb), 0.45); background: rgba(var(--accent-rgb), 0.45);
} }
:root[data-motion='reduce'] * { :root[data-motion='reduce'] *,
:root[data-motion='reduce'] *::before,
:root[data-motion='reduce'] *::after {
animation-iteration-count: 1 !important;
animation-duration: 0.001s !important; animation-duration: 0.001s !important;
transition-duration: 0.001s !important; transition-duration: 0.001s !important;
} }
@media (prefers-reduced-motion: reduce) {
:root:not([data-motion='full']) *,
:root:not([data-motion='full']) *::before,
:root:not([data-motion='full']) *::after {
animation-iteration-count: 1 !important;
animation-duration: 0.001s !important;
transition-duration: 0.001s !important;
}
}
.error-banner span,
.activity-item .detail,
.row-item .desc,
.row-item .meta,
.core-sub,
.kv dd {
user-select: text;
}
/* Quiet hours: dimmer HUD, softer glow. */
:root[data-night] {
--accent-glow: rgba(var(--accent-rgb), 0.18);
}
:root[data-night] .hud-root,
:root[data-night] .compact-root {
filter: brightness(0.72) saturate(0.85);
}
:root[data-night] .core-stage canvas {
opacity: 0.8;
}
+33
View File
@@ -0,0 +1,33 @@
import { describe, expect, it } from 'vitest';
import { describeHtml, hermesUrlCandidates, recoverCompletion } from '../src/services/hermes/client';
describe('describeHtml', () => {
it('explains a login page instead of the API', () => {
const msg = describeHtml('<!doctype html><html lang="fr-FR"><head><title>Jarvis – Se connecter</title></head><body>Mot de passe</body></html>');
expect(msg).toContain('Jarvis – Se connecter');
expect(msg).toContain('page de connexion');
expect(msg).toContain('8642');
});
it('ignores JSON and SSE', () => {
expect(describeHtml('{"status":"ok"}')).toBeNull();
expect(describeHtml('data: {"choices":[]}')).toBeNull();
});
it('is used by recoverCompletion', () => {
expect(recoverCompletion('<html><head><title>Portal</title></head></html>').error).toContain('Portal');
});
});
describe('hermesUrlCandidates', () => {
it('tries the API port, common paths and sibling hosts', () => {
const c = hermesUrlCandidates('http://jarvis.vonrodbox.eu');
expect(c).toContain('http://jarvis.vonrodbox.eu:8642');
expect(c).toContain('http://jarvis.vonrodbox.eu/api');
expect(c).toContain('http://api.jarvis.vonrodbox.eu');
expect(c).toContain('http://hermes.vonrodbox.eu');
expect(c).not.toContain('http://jarvis.vonrodbox.eu');
});
it('keeps an explicit port and handles garbage', () => {
expect(hermesUrlCandidates('http://10.0.0.5:8642').some((u) => u.includes(':8642:'))).toBe(false);
expect(hermesUrlCandidates('')).toEqual([]);
});
});
+27
View File
@@ -0,0 +1,27 @@
import { describe, expect, it } from 'vitest';
import { recoverCompletion } from '../src/services/hermes/client';
import { describeHealth, isHealthyStatus } from '../src/state/hermes';
describe('health status', () => {
it('accepts the usual healthy words', () => {
for (const s of ['ok', 'OK', 'healthy', 'ready', undefined]) expect(isHealthyStatus(s)).toBe(true);
for (const s of ['degraded', 'unhealthy', 'error']) expect(isHealthyStatus(s)).toBe(false);
});
it('names the failing checks', () => {
expect(describeHealth({ status: 'degraded', checks: { memory: { status: 'ok' }, sessions_db: { status: 'error' } } })).toBe('état degraded · checks.sessions_db');
expect(describeHealth({ status: 'degraded' })).toBe('état degraded');
});
});
describe('recoverCompletion', () => {
it('reads a plain JSON completion when the server ignored streaming', () => {
expect(recoverCompletion(JSON.stringify({ choices: [{ message: { role: 'assistant', content: 'Bonjour.' } }] }))).toEqual({ text: 'Bonjour.' });
});
it('surfaces an error object returned with HTTP 200', () => {
expect(recoverCompletion(JSON.stringify({ error: { message: 'model not found' } })).error).toContain('model not found');
});
it('handles SSE bodies and empty input', () => {
expect(recoverCompletion('data: {"choices":[{"delta":{"content":"A"}}]}\n\ndata: {"choices":[{"delta":{"content":"B"}}]}\n\ndata: [DONE]\n\n')).toEqual({ text: 'AB' });
expect(recoverCompletion('')).toEqual({});
});
});
+25
View File
@@ -0,0 +1,25 @@
import { describe, expect, it } from 'vitest';
import { buildKeywordsFile, encodeKeyword, normalizeKeyword, parseTokens } from '../shared/keywords';
const vocab = parseTokens(['<blk> 0', '▁ 3', '▁JA 4', 'R 5', 'VI 6', 'S 7', '▁HE 8', 'Y 9', '▁NO 10', 'V 11', 'A 12', '▁MA 13', 'X 14', '▁T 15', 'ON 16', 'Y 17'].join('\n'));
describe('keywords', () => {
it('normalises accents, case and punctuation', () => {
expect(normalizeKeyword(' Hé, Jarvis ! ')).toBe('HE JARVIS');
});
it('uses the known SentencePiece encodings', () => {
expect(encodeKeyword('jarvis', vocab)).toBe('▁JA R VI S');
expect(encodeKeyword('Hey Jarvis', vocab)).toBe('▁HE Y ▁JA R VI S');
});
it('falls back to greedy longest match', () => {
expect(encodeKeyword('max', vocab)).toBe('▁MA X');
expect(encodeKeyword('tony', vocab)).toBe('▁T ON Y');
expect(encodeKeyword('zzz', vocab)).toBeNull();
});
it('builds a keywords file with labels', () => {
const file = buildKeywordsFile(['jarvis', 'hey jarvis', 'zzz'], vocab);
expect(file.content).toBe('▁JA R VI S @jarvis\n▁HE Y ▁JA R VI S @hey_jarvis\n');
expect(file.accepted).toEqual(['jarvis', 'hey_jarvis']);
expect(file.rejected).toEqual(['zzz']);
});
});
+24
View File
@@ -0,0 +1,24 @@
import { describe, expect, it } from 'vitest';
import { parseLocalIntent } from '../src/services/localCommands';
describe('parseLocalIntent', () => {
it('recognises system actions in French and English', () => {
expect(parseLocalIntent('Jarvis, verrouille la session')?.action).toEqual({ type: 'lock' });
expect(parseLocalIntent('monte le son')?.action).toEqual({ type: 'media', key: 'volume-up' });
expect(parseLocalIntent('Baisse le volume')?.action).toEqual({ type: 'media', key: 'volume-down' });
expect(parseLocalIntent('coupe le son')?.action).toEqual({ type: 'media', key: 'mute' });
expect(parseLocalIntent('piste suivante')?.action).toEqual({ type: 'media', key: 'next' });
expect(parseLocalIntent('ouvre le bloc-notes')?.action).toEqual({ type: 'open-app', name: 'bloc-notes' });
expect(parseLocalIntent('open spotify')?.action).toEqual({ type: 'open-app', name: 'spotify' });
expect(parseLocalIntent('ouvre github.com')?.action).toEqual({ type: 'open-url', url: 'https://github.com' });
});
it('detects screen questions', () => {
expect(parseLocalIntent('Jarvis, regarde mon écran et dis-moi ce que tu vois')?.kind).toBe('screenshot');
expect(parseLocalIntent('fais une capture d’écran')?.kind).toBe('screenshot');
});
it('leaves everything else to Hermes', () => {
expect(parseLocalIntent('Quelle est la météo à Paris demain ?')).toBeNull();
expect(parseLocalIntent('ouvre le fichier puis envoie-le à Marc')).toBeNull();
expect(parseLocalIntent('')).toBeNull();
});
});
+37
View File
@@ -0,0 +1,37 @@
import { describe, expect, it } from 'vitest';
import { isPriority, isQuietTime, summarize } from '../src/lib/quietHours';
const at = (h: number, m = 0) => new Date(2026, 0, 1, h, m);
describe('isQuietTime', () => {
it('handles ranges crossing midnight', () => {
expect(isQuietTime('22:30', '07:30', at(23))).toBe(true);
expect(isQuietTime('22:30', '07:30', at(3, 15))).toBe(true);
expect(isQuietTime('22:30', '07:30', at(7, 30))).toBe(false);
expect(isQuietTime('22:30', '07:30', at(12))).toBe(false);
});
it('handles same-day ranges and invalid input', () => {
expect(isQuietTime('13:00', '14:00', at(13, 30))).toBe(true);
expect(isQuietTime('13:00', '14:00', at(14))).toBe(false);
expect(isQuietTime('bad', '14:00', at(13))).toBe(false);
expect(isQuietTime('10:00', '10:00', at(10))).toBe(false);
});
});
describe('isPriority', () => {
it('matches keywords in text or job name', () => {
expect(isPriority('Serveur en panne depuis 5 min', 'urgent, panne')).toBe(true);
expect(isPriority('Rapport quotidien', 'urgent', 'Alerte disque')).toBe(false);
expect(isPriority('Rapport quotidien', 'alerte', 'Alerte disque')).toBe(true);
expect(isPriority('x', '')).toBe(false);
});
});
describe('summarize', () => {
it('keeps the first sentences and strips markdown', () => {
const text = '## Rapport\n\nTrois annonces **majeures** aujourd’hui. Le marché monte de 2 %. Détails ci-dessous :\n\n```\ncode\n```\n- point 1';
expect(summarize(text, 2)).toBe('Rapport Trois annonces majeures aujourd’hui. Le marché monte de 2 %.');
expect(summarize('Une seule phrase', 2)).toBe('Une seule phrase');
expect(summarize('', 2)).toBe('');
});
});
+16 -1
View File
@@ -1,5 +1,5 @@
import { describe, expect, it } from 'vitest'; import { describe, expect, it } from 'vitest';
import { chunkForSpeech, cleanForSpeech, extractSentences, preprocessMedia } from '../src/lib/text'; import { chunkForSpeech, cleanForSpeech, extractSentences, isTranscriptNoise, preprocessMedia } from '../src/lib/text';
describe('text', () => { describe('text', () => {
it('cleans markdown for speech', () => { it('cleans markdown for speech', () => {
@@ -25,3 +25,18 @@ describe('text', () => {
expect(preprocessMedia('MEDIA:/tmp/a.png')).toContain('![image](/tmp/a.png)'); expect(preprocessMedia('MEDIA:/tmp/a.png')).toContain('![image](/tmp/a.png)');
}); });
}); });
describe('isTranscriptNoise', () => {
it('drops Whisper hallucinations on silence', () => {
expect(isTranscriptNoise('(cliquant)')).toBe(true);
expect(isTranscriptNoise('*Claire*')).toBe(true);
expect(isTranscriptNoise('[Musique]')).toBe(true);
expect(isTranscriptNoise('...')).toBe(true);
expect(isTranscriptNoise("Sous-titres réalisés par la communauté d'Amara.org")).toBe(true);
});
it('keeps real sentences', () => {
expect(isTranscriptNoise('Jarvis, allume la lumière du salon.')).toBe(false);
expect(isTranscriptNoise('(Jarvis) quelle heure est-il maintenant ?')).toBe(false);
expect(isTranscriptNoise('Oui')).toBe(false);
});
});
+49
View File
@@ -0,0 +1,49 @@
import { describe, expect, it } from 'vitest';
import { defaultOpenAiVoice, findLocalVoice, inferGender, rankSystemVoice, suggestedDownload } from '../src/lib/voicePreference';
import type { VoiceModelStatus } from '../shared/voice';
const model = (id: string, installed: boolean, speakers: Array<{ id: number; name: string; lang: string }>): VoiceModelStatus =>
({ id, kind: 'tts', engine: 'piper', name: id, description: '', languages: ['fr'], sizeMb: 1, url: '', dir: id, files: [], speakers, installed, installedBytes: 0 }) as unknown as VoiceModelStatus;
describe('inferGender', () => {
it('reads catalog labels, system voices and kokoro ids', () => {
expect(inferGender('Piper Tom (homme, français)')).toBe('male');
expect(inferGender('Siwis (femme, français)')).toBe('female');
expect(inferGender('Microsoft Paul - French (France)')).toBe('male');
expect(inferGender('Microsoft Hortense - French (France)')).toBe('female');
expect(inferGender('am_adam')).toBe('male');
expect(inferGender('Voix 3')).toBeUndefined();
});
});
describe('local voice selection', () => {
const models = [
model('kokoro-v1', true, [{ id: 30, name: 'Siwis (femme, français)', lang: 'fr' }, { id: 4, name: 'Adam (homme, anglais US)', lang: 'en' }]),
model('piper-fr-tom', false, [{ id: 0, name: 'Tom', lang: 'fr' }]),
model('piper-fr-upmc', true, [{ id: 0, name: 'Jessica (femme)', lang: 'fr' }, { id: 1, name: 'Pierre (homme)', lang: 'fr' }])
];
it('prefers an installed speaker of the wanted gender in the right language', () => {
expect(findLocalVoice(models, 'fr-FR', 'male')).toEqual({ modelId: 'piper-fr-upmc', speaker: 1 });
expect(findLocalVoice(models, 'fr-FR', 'female')).toEqual({ modelId: 'kokoro-v1', speaker: 30 });
expect(findLocalVoice(models.slice(0, 2), 'fr', 'male')).toBeNull();
});
it('suggests a French download when nothing matches', () => {
expect(suggestedDownload(models, 'fr', 'male')).toBe('piper-fr-tom');
expect(suggestedDownload(models, 'en', 'male')).toBeNull();
});
});
describe('provider defaults', () => {
it('picks onyx for male and nova for female on OpenAI-compatible APIs', () => {
expect(defaultOpenAiVoice('male')).toBe('onyx');
expect(defaultOpenAiVoice('female')).toBe('nova');
});
it('ranks system voices by language then gender', () => {
const paul = { name: 'Microsoft Paul - French (France)', lang: 'fr-FR', localService: true };
const hortense = { name: 'Microsoft Hortense - French (France)', lang: 'fr-FR', localService: true };
const david = { name: 'Microsoft David - English (US)', lang: 'en-US', localService: true };
expect(rankSystemVoice(paul, 'fr', 'male')).toBeGreaterThan(rankSystemVoice(hortense, 'fr', 'male'));
expect(rankSystemVoice(hortense, 'fr', 'female')).toBeGreaterThan(rankSystemVoice(paul, 'fr', 'female'));
expect(rankSystemVoice(paul, 'fr', 'male')).toBeGreaterThan(rankSystemVoice(david, 'fr', 'male'));
});
});
+16
View File
@@ -0,0 +1,16 @@
import { describe, expect, it } from 'vitest';
import { matchWakeWord } from '../src/services/voice/voiceController';
describe('matchWakeWord', () => {
it('accepts the wake word with punctuation, accents and small slips', () => {
expect(matchWakeWord('Jarvis, quelle heure est-il ?', 'jarvis')).toEqual({ matched: true, rest: 'quelle heure est-il ?' });
expect(matchWakeWord('Javis allume la lumière', 'jarvis')).toEqual({ matched: true, rest: 'allume la lumière' });
expect(matchWakeWord('JARVIS.', 'jarvis')).toEqual({ matched: true, rest: '' });
expect(matchWakeWord('Hé Jarvis, bonjour', 'hé jarvis')).toEqual({ matched: true, rest: 'bonjour' });
});
it('rejects unrelated speech', () => {
expect(matchWakeWord('Il fait beau aujourd’hui', 'jarvis').matched).toBe(false);
expect(matchWakeWord('Service client', 'jarvis').matched).toBe(false);
expect(matchWakeWord('', 'jarvis').matched).toBe(false);
});
});