mirror of
https://github.com/R0m1k3/Loki.git
synced 2026-10-21 05:58:13 +02:00
perf: fix model reload thrash and cut time-to-first-token
Ollama reloaded the chat model mid-message because plan/summary/router calls sent divergent runner options (no num_ctx/num_batch) and omitted keep_alive — the main cause of perceived slowness. - share one keep-alive httpx.AsyncClient for all Ollama calls - unify runner options (runner_options) + keep_alive on every model call, embeddings included - drop the blocking LLM routing fallback (pure lexical heuristic) - run RAG recall + plan + code-model pick in parallel inside the SSE stream, after the start event - RAG cosine scoring off the event loop; cache /api/tags 30s and nvidia-smi 5s; frontend polls 2s->5s, warm poll backoff, dedup config fetch feat: working session menu in TopBar (switch/create/rename/delete) feat: workspace file deletion (DELETE /api/files + UI trash buttons) docs: recommended Ollama env vars (KEEP_ALIVE, MAX_LOADED_MODELS...) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
1 parent
552548e4b4
commit
cb2872c78a
23 files changed
+672
-224
No files matched your search
@@ -14,27 +14,29 @@ export default function App() {
|
||||
refreshStatus,
|
||||
refreshSystemStats,
|
||||
refreshModels,
|
||||
refreshConfig,
|
||||
refreshLoadedModels,
|
||||
refreshSessions,
|
||||
} = useStore();
|
||||
|
||||
// Au démarrage : statut Ollama + modèles + config. Poll du statut, des stats
|
||||
// système et des modèles chargés en VRAM (indicateur de préchargement).
|
||||
// Au démarrage : statut Ollama + modèles + sessions. La config est chargée
|
||||
// par setSelectedModel (via refreshModels) — pas de double fetch. Poll du
|
||||
// statut, des stats système (5 s, apparié au cache GPU serveur) et des
|
||||
// modèles chargés en VRAM (indicateur de préchargement).
|
||||
useEffect(() => {
|
||||
refreshStatus();
|
||||
refreshSystemStats();
|
||||
refreshModels();
|
||||
refreshConfig();
|
||||
refreshLoadedModels();
|
||||
refreshSessions();
|
||||
const statusId = setInterval(refreshStatus, 10000);
|
||||
const statsId = setInterval(refreshSystemStats, 2000);
|
||||
const statsId = setInterval(refreshSystemStats, 5000);
|
||||
const loadedId = setInterval(refreshLoadedModels, 8000);
|
||||
return () => {
|
||||
clearInterval(statusId);
|
||||
clearInterval(statsId);
|
||||
clearInterval(loadedId);
|
||||
};
|
||||
}, [refreshStatus, refreshSystemStats, refreshModels, refreshConfig, refreshLoadedModels]);
|
||||
}, [refreshStatus, refreshSystemStats, refreshModels, refreshLoadedModels, refreshSessions]);
|
||||
|
||||
return (
|
||||
<div className="flex h-full flex-col bg-base text-ink">
|
||||
|
||||
Reference in new issue
Block a user