perf: fix model reload thrash and cut time-to-first-token

Ollama reloaded the chat model mid-message because plan/summary/router
calls sent divergent runner options (no num_ctx/num_batch) and omitted
keep_alive — the main cause of perceived slowness.

- share one keep-alive httpx.AsyncClient for all Ollama calls
- unify runner options (runner_options) + keep_alive on every model
  call, embeddings included
- drop the blocking LLM routing fallback (pure lexical heuristic)
- run RAG recall + plan + code-model pick in parallel inside the SSE
  stream, after the start event
- RAG cosine scoring off the event loop; cache /api/tags 30s and
  nvidia-smi 5s; frontend polls 2s->5s, warm poll backoff, dedup
  config fetch

feat: working session menu in TopBar (switch/create/rename/delete)
feat: workspace file deletion (DELETE /api/files + UI trash buttons)
docs: recommended Ollama env vars (KEEP_ALIVE, MAX_LOADED_MODELS...)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
MichaelandClaude Opus 4.8 committed 2026-07-18 13:58:18 +02:00
1 parent 552548e4b4
commit cb2872c78a
23 files changed
+672 -224

No files matched your search

+30 -4
View File
@@ -1,6 +1,7 @@
import { create } from "zustand";
import {
createSession,
deleteFile,
deleteSession,
getConfig,
getModels,
@@ -12,6 +13,7 @@ import {
fileContent,
listFiles,
listSessions,
renameSession as apiRenameSession,
runShell,
saveConfig,
streamChat,
@@ -73,6 +75,8 @@ interface LokiState {
newSession: () => Promise<void>;
openSession: (id: string) => Promise<void>;
removeSession: (id: string) => Promise<void>;
renameSession: (id: string, title: string) => Promise<void>;
removeFile: (path: string) => Promise<void>;
sendMessage: (content: string) => Promise<void>;
stopStreaming: () => void;
@@ -278,6 +282,23 @@ export const useStore = create<LokiState>((set, get) => ({
await get().refreshSessions();
},
renameSession: async (id, title) => {
const clean = title.trim();
if (!clean) return;
await apiRenameSession(id, clean);
await get().refreshSessions();
},
removeFile: async (path) => {
await deleteFile(path);
// Ferme l'aperçu si le fichier supprimé (ou son dossier) y est affiché.
const preview = get().previewPath;
if (preview && (preview === path || preview.startsWith(path + "/"))) {
set({ previewPath: null, previewContent: "" });
}
await get().refreshFiles();
},
stopStreaming: () => {
activeStreamController?.abort();
activeStreamController = null;
@@ -403,10 +424,15 @@ export const useStore = create<LokiState>((set, get) => ({
streamNotice: null,
streamTools: [],
});
// Recharge depuis la base + l'arborescence (fichiers créés par l'agent).
if (get().currentSessionId === sid) await get().openSession(sid!);
await get().refreshSessions();
await get().refreshFiles();
// Recharge depuis la base + l'arborescence (fichiers créés par
// l'agent) — trois requêtes indépendantes, en parallèle.
await Promise.all([
get().currentSessionId === sid
? get().openSession(sid!)
: Promise.resolve(),
get().refreshSessions(),
get().refreshFiles(),
]);
if (writtenHtml) await get().openPreview(writtenHtml.args.path as string);
},
onError: (msg) => {