perf: fix model reload thrash and cut time-to-first-token

Ollama reloaded the chat model mid-message because plan/summary/router
calls sent divergent runner options (no num_ctx/num_batch) and omitted
keep_alive — the main cause of perceived slowness.

- share one keep-alive httpx.AsyncClient for all Ollama calls
- unify runner options (runner_options) + keep_alive on every model
  call, embeddings included
- drop the blocking LLM routing fallback (pure lexical heuristic)
- run RAG recall + plan + code-model pick in parallel inside the SSE
  stream, after the start event
- RAG cosine scoring off the event loop; cache /api/tags 30s and
  nvidia-smi 5s; frontend polls 2s->5s, warm poll backoff, dedup
  config fetch

feat: working session menu in TopBar (switch/create/rename/delete)
feat: workspace file deletion (DELETE /api/files + UI trash buttons)
docs: recommended Ollama env vars (KEEP_ALIVE, MAX_LOADED_MODELS...)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
MichaelandClaude Opus 4.8 committed 2026-07-18 13:58:18 +02:00
1 parent 552548e4b4
commit cb2872c78a
23 files changed
+672 -224

No files matched your search

+8 -6
View File
@@ -14,27 +14,29 @@ export default function App() {
refreshStatus,
refreshSystemStats,
refreshModels,
refreshConfig,
refreshLoadedModels,
refreshSessions,
} = useStore();
// Au démarrage : statut Ollama + modèles + config. Poll du statut, des stats
// système et des modèles chargés en VRAM (indicateur de préchargement).
// Au démarrage : statut Ollama + modèles + sessions. La config est chargée
// par setSelectedModel (via refreshModels) — pas de double fetch. Poll du
// statut, des stats système (5 s, apparié au cache GPU serveur) et des
// modèles chargés en VRAM (indicateur de préchargement).
useEffect(() => {
refreshStatus();
refreshSystemStats();
refreshModels();
refreshConfig();
refreshLoadedModels();
refreshSessions();
const statusId = setInterval(refreshStatus, 10000);
const statsId = setInterval(refreshSystemStats, 2000);
const statsId = setInterval(refreshSystemStats, 5000);
const loadedId = setInterval(refreshLoadedModels, 8000);
return () => {
clearInterval(statusId);
clearInterval(statsId);
clearInterval(loadedId);
};
}, [refreshStatus, refreshSystemStats, refreshModels, refreshConfig, refreshLoadedModels]);
}, [refreshStatus, refreshSystemStats, refreshModels, refreshLoadedModels, refreshSessions]);
return (
<div className="flex h-full flex-col bg-base text-ink">