perf: fix model reload thrash and cut time-to-first-token

Ollama reloaded the chat model mid-message because plan/summary/router
calls sent divergent runner options (no num_ctx/num_batch) and omitted
keep_alive — the main cause of perceived slowness.

- share one keep-alive httpx.AsyncClient for all Ollama calls
- unify runner options (runner_options) + keep_alive on every model
  call, embeddings included
- drop the blocking LLM routing fallback (pure lexical heuristic)
- run RAG recall + plan + code-model pick in parallel inside the SSE
  stream, after the start event
- RAG cosine scoring off the event loop; cache /api/tags 30s and
  nvidia-smi 5s; frontend polls 2s->5s, warm poll backoff, dedup
  config fetch

feat: working session menu in TopBar (switch/create/rename/delete)
feat: workspace file deletion (DELETE /api/files + UI trash buttons)
docs: recommended Ollama env vars (KEEP_ALIVE, MAX_LOADED_MODELS...)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
MichaelandClaude Opus 4.8 committed 2026-07-18 13:58:18 +02:00
1 parent 552548e4b4
commit cb2872c78a
23 files changed
+672 -224

No files matched your search

+27 -2
View File
@@ -158,7 +158,11 @@ export async function warmModel(
throw await apiError(res, `préchargement refusé (${res.status})`);
}
for (let attempt = 0; attempt < 300; attempt++) {
// Backoff progressif : réactif au début (modèle déjà chargé), espacé ensuite
// pour ne pas marteler l'API pendant un long chargement.
const deadline = Date.now() + 10 * 60 * 1000;
let delay = 1000;
while (Date.now() < deadline) {
const statusRes = await fetch(
`/api/models/warm/status?name=${encodeURIComponent(name)}`,
{ cache: "no-store" }
@@ -171,7 +175,8 @@ export async function warmModel(
if (status.state === "error") {
throw new Error(status.error ?? "préchargement impossible");
}
await wait(2000);
await wait(delay);
delay = Math.min(delay * 1.5, 5000);
}
throw new Error("préchargement toujours en cours après 10 minutes");
}
@@ -405,6 +410,15 @@ export async function fileContent(path: string): Promise<string> {
return (await res.json()).content;
}
export async function deleteFile(path: string): Promise<void> {
const res = await fetch(`/api/files?path=${encodeURIComponent(path)}`, {
method: "DELETE",
});
if (!res.ok) {
throw await apiError(res, `suppression impossible (${res.status})`);
}
}
export function downloadFile(path: string): void {
const link = document.createElement("a");
link.href = `/api/files/download?path=${encodeURIComponent(path)}`;
@@ -439,6 +453,17 @@ export async function deleteSession(id: string): Promise<void> {
await fetch(`/api/sessions/${id}`, { method: "DELETE" });
}
export async function renameSession(id: string, title: string): Promise<void> {
const res = await fetch(`/api/sessions/${id}`, {
method: "PATCH",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ title }),
});
if (!res.ok) {
throw await apiError(res, `renommage impossible (${res.status})`);
}
}
/** Envoie un message et streame la réponse de l'agent via SSE. */
export async function streamChat(
body: { session_id: string; content: string; model?: string; mode?: string },