diff --git a/README.md b/README.md index 03c5ae6..adc92da 100644 --- a/README.md +++ b/README.md @@ -235,6 +235,7 @@ redémarrages (volume `/data`). Variables d'environnement du conteneur : | `LOKI_PUSH` | `rust` : les notifications Web Push (clé VAPID, abonnements, envoi « réponse prête ») sont tenues par la façade Rust (expérimental, migration en cours) | Go | | `LOKI_MEM` | `rust` : la mémoire (éditeur des pages, index, déplacement entre projets, trackers, mode mémoire, interrupteurs du mode agent) est tenue par la façade Rust ; le chiffrement reste au Go (expérimental, migration en cours) | Go | | `LOKI_SERVE` | `rust` : la ligne de commande de llama-server est composée par la façade Rust (`loki-rs engine serve`) au lieu de `loki serve` ; Strata et l'optimiseur restent au Go (expérimental, migration en cours) | Go | +| `LOKI_PRESETS` | `rust` : les presets (liste, éditeur et aperçus, suppression, placement auto, presets externes, bascule) sont tenus par la façade Rust ; l'application au moteur reste au Go (expérimental, migration en cours) | Go | **Accès par nom de domaine.** Sans clé de pilotage, l'API n'accepte que les hôtes locaux (IP, `localhost`, nom sans point, `.local`/`.lan`…) et refuse diff --git a/docs/superpowers/specs/2026-10-15-rust-migration-lot15-presets-design.md b/docs/superpowers/specs/2026-10-15-rust-migration-lot15-presets-design.md new file mode 100644 index 0000000..3364fbb --- /dev/null +++ b/docs/superpowers/specs/2026-10-15-rust-migration-lot15-presets-design.md @@ -0,0 +1,86 @@ +# Lot 15 — presets tenus par la façade + +## Contexte + +Les presets vivaient dans le Go : +- `backend_presets.go` pour l'écriture ; +- les handlers de `web_api.go` et `web_external.go` ; +- les aperçus de l'éditeur (`cacheRAMPreview`, `sideSlotPreview`, + `moePreview`, `autoPlacementPreset`). + +Lot 15 : la façade tient ces routes quand `LOKI_PRESETS=rust` +(`presets_api.rs`, expérimental). Sans la variable, rien ne change. + +| Route | Rôle | +|---|---| +| `GET /api/presets` | liste, avec pastilles (contexte, vision, quantification, raisonnement, Strata, externe) et mesure du bench si elle correspond à la configuration actuelle | +| `POST /api/presets/order` | ordre choisi par glisser-déposer | +| `GET /api/preset?id=` | contenu et prompt système ; sans id, la graine d'un preset neuf (BIN personnalisé, HOST, PORT) | +| `POST /api/preset/save` | crée (nom de fichier unique) ou met à jour ; le preset EN SERVICE est appliqué tout de suite | +| `POST /api/preset/delete` | refusé pour le preset actif ; emporte prompt système et mesure, et le modèle sur demande | +| `POST /api/preset/cacheram`, `/api/preset/moe` | aperçus de l'éditeur, calculés comme au lancement (lot 14) | +| `POST /api/preset/autoplace` | copie en placement automatique d'un preset MoE placé à la main | +| `GET /api/preset/external?id=`, `POST /api/preset/external/save`, `/api/preset/external/test` | presets externes ; la clé n'est jamais rendue, le test fait un vrai appel de complétion | +| `POST /api/switch` | bascule de preset | + +Les presets restent des fichiers `presets/*.env`, et leur prompt système une +clé de la base. Les aperçus réutilisent le port de `loki serve` (lot 14), +sondes comprises. + +## Restent au Go + +Tout ce qui touche au moteur reste au Go, jusqu'au lot qui le portera. Le Go +expose deux routes internes pour cela (`facade_presets.go`). + +`POST /internal/v1/preset/apply {path, mode}` applique un preset à la +configuration vivante : +- `switch` : l'état du slot est relevé avant (SLOT_PERSIST), puis le moteur + est relancé, ou arrêté pour un preset externe ; +- `save` : application, puis relance ; +- `apply` : la configuration seule (preset externe en service modifié). + +`GET /internal/v1/tune/guard` dit si une optimisation tient le moteur. La +réponse est alors un 409 `{ok: false, tuning: true}`, comme le Go. + +## Écarts assumés + +- La suppression d'un preset retire sa mesure sans réécrire les autres avec + leurs champs à zéro, comme le faisait la struct du Go. Le Go les relit + pareil. +- Le message d'un échec de réseau du test externe est celui de reqwest. + +## Parité + +`internal/loki/parity_presets_test.go` tire du Go deux séries. + +Les fonctions pures : +- nom d'affichage, guillemets, format `.env`, noms refusés ; +- quantification, raisonnement, modèle ; +- URL de complétions, contenu d'un preset externe, message d'erreur d'API ; +- arguments rejoints, clé réécrite dans un preset ; +- huit copies en placement auto ; +- empreinte de mesure, moteur imposé par un preset. + +Puis 70 appels de route, avec après chacun les dossiers `presets/` et +`models/`, et les clés touchées en base. Ils couvrent : +- créations, doublons, noms refusés, erreurs JSON ; +- ordre, pastilles et mesures (empreinte, repli sur le modèle, autre + configuration) ; +- preset actif et suppressions (modèle compris, partagé, absent) ; +- presets externes et leur test contre un faux service (succès, 401 au + format OpenAI, 404 brut, 500 vide) ; +- placement auto et aperçus. + +`tests/presets_api.rs` rejoue le tout par la façade, et une mutation fait +échouer le rejeu. + +La bascule et l'enregistrement du preset en service sont vérifiés de bout en +bout contre le faux Go : +- appels d'application, avec leur mode ; +- refus quand une optimisation tourne ; +- erreur d'application rendue ; +- prompt système. + +Côté Go, `TestFacadePresetApply` vérifie `presetApply` : configuration +appliquée, appareils préservés, moteur relancé, chemin hors du dossier +refusé. diff --git a/internal/loki/facade_chat.go b/internal/loki/facade_chat.go index 28de305..c300872 100644 --- a/internal/loki/facade_chat.go +++ b/internal/loki/facade_chat.go @@ -162,6 +162,8 @@ func withFacadeChat(path string, h http.HandlerFunc) http.HandlerFunc { held, unavailable = facadePush, "notifications indisponibles : " case facadeMemRoutes[path]: held, unavailable = facadeMem, "mémoire indisponible : " + case facadePresetsRoutes[path]: + held, unavailable = facadePresets, "presets indisponibles : " default: return h } diff --git a/internal/loki/facade_chat_test.go b/internal/loki/facade_chat_test.go index 18289fb..daddc16 100644 --- a/internal/loki/facade_chat_test.go +++ b/internal/loki/facade_chat_test.go @@ -545,3 +545,28 @@ func TestFacadeMemoire(t *testing.T) { } } } + +func TestFacadePresets(t *testing.T) { + testHome(t) + f := demarrerFacade(t) + t.Setenv(facadeChatEnv, "") + local := func(w http.ResponseWriter, r *http.Request) { io.WriteString(w, "go") } + for p := range facadePresetsRoutes { + w := httptest.NewRecorder() + withFacadeChat(p, local)(w, httptest.NewRequest("GET", p, nil)) + if w.Body.String() != "go" { + t.Fatalf("%s servi ailleurs sans LOKI_PRESETS : %q", p, w.Body.String()) + } + } + t.Setenv(facadePresetsEnv, "rust") + for p := range facadePresetsRoutes { + w := httptest.NewRecorder() + withFacadeChat(p, local)(w, httptest.NewRequest("POST", p, strings.NewReader("{}"))) + if w.Body.String() != f.reply { + t.Fatalf("%s : réponse %q", p, w.Body.String()) + } + } + if n := len(f.recu()); n != len(facadePresetsRoutes) { + t.Fatalf("reçu par la façade : %d appels", n) + } +} diff --git a/internal/loki/facade_presets.go b/internal/loki/facade_presets.go new file mode 100644 index 0000000..19cd213 --- /dev/null +++ b/internal/loki/facade_presets.go @@ -0,0 +1,136 @@ +package loki + +import ( + "fmt" + "net/http" + "os" + "path/filepath" + "strings" +) + +// facade_presets.go — presets tenus par la façade Rust (LOKI_PRESETS=rust, +// lot 15 de la migration ; expérimental, le Go reste le défaut) : liste, +// ordre, lecture, éditeur, suppression, aperçus de l'éditeur, copie en +// placement auto, presets externes, bascule. +// +// Les presets restent des fichiers presets/*.env et leur prompt système une +// clé de la base. Ce qui touche au moteur reste ici jusqu'au lot qui le +// portera : verrou de l'optimiseur (/internal/v1/tune/guard) et application +// d'un preset à la configuration vivante, avec l'état du slot gardé et le +// moteur relancé ou arrêté (/internal/v1/preset/apply). + +const facadePresetsEnv = "LOKI_PRESETS" + +// facadePresets : la façade tient-elle les presets ? +func facadePresets() bool { + return strings.EqualFold(strings.TrimSpace(os.Getenv(facadePresetsEnv)), "rust") && + facadeAddr() != "" && internalToken() != "" +} + +// facadePresetsRoutes : les routes que la façade sert alors (presets_api.rs). +var facadePresetsRoutes = map[string]bool{ + "/api/presets": true, + "/api/presets/order": true, + "/api/preset": true, + "/api/preset/save": true, + "/api/preset/cacheram": true, + "/api/preset/moe": true, + "/api/preset/autoplace": true, + "/api/preset/delete": true, + "/api/preset/external": true, + "/api/preset/external/save": true, + "/api/preset/external/test": true, + "/api/switch": true, +} + +// presetServiceAction : le pilotage du moteur après une application de +// preset. Variable pour les tests (aucun moteur n'y tourne). +var presetServiceAction = serviceAction + +// presetApply applique un preset (chemin dans presets/) à la configuration +// vivante, comme les handlers d'origine : +// +// switch : bascule (handleSwitch) — état du slot relevé avant, moteur arrêté +// pour un preset externe, relancé sinon, en arrière-plan ; +// save : preset en service modifié (handlePresetSave) — relancé ; +// apply : preset externe en service modifié (handlePresetExternalSave) — +// configuration seule. +func presetApply(path, mode string) error { + root, err := filepath.Abs(presetsDir()) + if err != nil { + return err + } + abs, err := filepath.Abs(path) + if err != nil || !strings.HasPrefix(abs, root+string(filepath.Separator)) || !strings.HasSuffix(abs, ".env") { + return fmt.Errorf("preset hors du dossier des presets") + } + switch mode { + case "switch": + persist := slotPersistPrepare() + if err := applyPresetFile(abs); err != nil { + return err + } + fmt.Printf("%s config.env <- %s\n", green("[ok]"), filepath.Base(abs)) + if isExternalConfig(ReadConfig()) { + fmt.Println(dim("[info] preset externe — arrêt du moteur local")) + go func() { + if serviceIsActive() { + persist.save() + if err := presetServiceAction("stop"); err != nil { + fmt.Printf("%s arrêt du moteur après bascule externe : %v\n", red("[ERREUR]"), err) + } + } + }() + return nil + } + fmt.Println(dim("[info] redémarrage du service...")) + go func() { + persist.save() + if err := presetServiceAction("restart"); err != nil { + fmt.Printf("%s redémarrage après bascule: %v\n", red("[ERREUR]"), err) + } + }() + return nil + case "save": + if err := applyPresetFile(abs); err != nil { + return err + } + fmt.Printf("%s config.env <- %s (preset actif modifié)\n", green("[ok]"), filepath.Base(abs)) + fmt.Println(dim("[info] redémarrage du service...")) + go func() { + if err := presetServiceAction("restart"); err != nil { + fmt.Printf("%s redémarrage après modification du preset: %v\n", red("[ERREUR]"), err) + } + }() + return nil + case "apply": + return applyPresetFile(abs) + } + return fmt.Errorf("mode inconnu : %q", mode) +} + +// handleInternalPresetApply : POST {path, mode} → 204, ou 400/500 + {error}. +func handleInternalPresetApply(w http.ResponseWriter, r *http.Request) { + var in struct { + Path string `json:"path"` + Mode string `json:"mode"` + } + if !readInternalJSON(w, r, &in) { + return + } + if err := presetApply(in.Path, in.Mode); err != nil { + sendJSON(w, http.StatusInternalServerError, map[string]any{"error": err.Error()}) + return + } + w.WriteHeader(http.StatusNoContent) +} + +// handleInternalTuneGuard : GET → {busy, error} — une optimisation tient-elle +// le moteur (tuneGuard) ? +func handleInternalTuneGuard(w http.ResponseWriter, r *http.Request) { + if err := tuneGuard(); err != nil { + sendJSON(w, http.StatusOK, map[string]any{"busy": true, "error": err.Error()}) + return + } + sendJSON(w, http.StatusOK, map[string]any{"busy": false}) +} diff --git a/internal/loki/parity_presets_test.go b/internal/loki/parity_presets_test.go new file mode 100644 index 0000000..5d9ffb0 --- /dev/null +++ b/internal/loki/parity_presets_test.go @@ -0,0 +1,425 @@ +package loki + +import ( + "encoding/json" + "io" + "net/http" + "net/http/httptest" + "os" + "path/filepath" + "sort" + "strings" + "testing" +) + +// parity_presets_test.go — journal de référence des presets (lot 15 de la +// migration Rust) : fonctions pures (nom d'affichage, valeurs, environnement, +// quantification, raisonnement, URL des presets externes, copie en placement +// auto…) puis une suite d'appels aux routes (liste et pastilles, ordre, +// éditeur, suppression avec le modèle, presets externes et leur test, aperçus +// de l'éditeur), avec après chacun le dossier presets/ et les clés touchées en +// base. Rangé dans rust/loki-rs/tests/parity_presets/presets.json ; +// tests/presets_api.rs le rejoue par la façade. +// +// La bascule et l'enregistrement du preset EN SERVICE pilotent le moteur : +// ils sont vérifiés à part (TestFacadePresetApply côté Go, essais de bout en +// bout côté Rust). +// +// LOKI_PARITY_UPDATE=1 go test ./internal/loki -run TestParitePresets + +const parityPresetsFile = "../../rust/loki-rs/tests/parity_presets/presets.json" + +type parityPresetStep struct { + Files map[string]string `json:"files,omitempty"` // relatif à LOKI_HOME ; "" = suppression + Store map[string]string `json:"store,omitempty"` // « bucket/clé » ; "" = suppression + Method string `json:"method"` + Path string `json:"path"` + Body string `json:"body,omitempty"` + Status int `json:"status"` + Out any `json:"out"` + Disk map[string]string `json:"disk"` + State map[string]any `json:"state"` +} + +func TestParitePresets(t *testing.T) { + home := testHome(t) + t.Setenv("LOKI_ENGINE_BIN", "") + t.Setenv("LOKI_MODEL_DIRS", "") + for _, k := range append(append(append(append([]string{}, serveArgEnv...), fidelityArgEnv...), cacheArgEnv...), sideSlotArgEnv...) { + t.Setenv(k, "") + os.Unsetenv(k) + } + + var fx struct { + DisplayName []map[string]string `json:"display_name"` + Quote map[string]string `json:"quote"` + FormatEnv string `json:"format_env"` + Names map[string]string `json:"names"` + Quant map[string]string `json:"quant"` + Reasoning map[string]string `json:"reasoning"` + Model map[string]string `json:"model"` + Completions map[string]string `json:"completions"` + External []map[string]any `json:"external"` + APIError map[string]string `json:"api_error"` + Join []map[string]any `json:"join"` + SetKey []map[string]string `json:"set_key"` + Autoplace []map[string]any `json:"autoplace"` + BenchFP []map[string]any `json:"bench_fp"` + Imposes map[string]bool `json:"imposes"` + Seed map[string]string `json:"seed"` + Steps []parityPresetStep `json:"steps"` + } + for _, c := range [][2]string{ + {"MODEL=a.gguf\n", "Alpha"}, {"# NAME=Vieux\nMODEL=a\n", "Neuf"}, {" name = x\nCTX=1\n", "Y"}, + {"#NAME=a\n# name=b\n", "Deux"}, {"MODEL=a\n# NAME=x", "Prix $1 et ${x}"}, {"", "Vide"}, + } { + fx.DisplayName = append(fx.DisplayName, map[string]string{"content": c[0], "name": c[1], "out": withDisplayName(c[0], c[1])}) + } + fx.Quote = map[string]string{} + for _, v := range []string{"", "simple", "a b", "a\tb", `a"b`, "'x'", `"y"`, "'z"} { + fx.Quote[v] = quoteValue(v) + } + fx.FormatEnv = formatEnv(map[string]string{"Z": "1", "A": "deux mots", "M": "", "Q": `q"q`}) + fx.Names = map[string]string{} + for _, n := range []string{"", " ", ".", "..", "a/b", `a\b`, "a\x00b", "a\x01b", "Qwen 3 (27B) é", " pad "} { + if _, err := safePresetPath(n); err != nil { + fx.Names[n] = err.Error() + } else { + fx.Names[n] = "ok" + } + } + fx.Quant, fx.Reasoning, fx.Model = map[string]string{}, map[string]string{}, map[string]string{} + for _, c := range []string{"MODEL=Qwen3-27B-Q4_K_M.gguf\n", "# QUANT=iq3_xs\nMODEL=x-Q8_0.gguf\n", "QUANT='q5'\n", + "MODEL=\"/m/Llama-3-UD-Q6_K_XL-00001-of-00002.gguf\"\nREASONING=On\n", "reasoning = 'off'\n#REASONING=on\n", + "export MODEL=z.gguf\n", "MODEL = C:\\models\\w-BF16.gguf\n", ""} { + fx.Quant[c], fx.Reasoning[c], fx.Model[c] = detectQuant(c), presetReasoning(c), modelFromPresetContent(c) + } + fx.Completions = map[string]string{} + for _, u := range []string{"", " https://api.groq.com/openai/v1/ ", "http://x/chat/completions", "http://loki:8090", "https://o.ai/api/v1"} { + fx.Completions[u] = completionsURL(u) + } + for _, c := range []struct { + url, model, key, ctx string + vision bool + }{{"https://a/v1", "gpt", "", "", false}, {" u ", " m ", " k ", " 128000 ", true}} { + fx.External = append(fx.External, map[string]any{"url": c.url, "model": c.model, "key": c.key, "ctx": c.ctx, + "vision": c.vision, "out": externalPresetContent(c.url, c.model, c.key, c.ctx, c.vision)}) + } + fx.APIError = map[string]string{} + for _, b := range []string{`{"error":{"message":" clé invalide "}}`, `{"error":"x"}`, `pas du json`, `{}`} { + fx.APIError[b] = extractAPIError([]byte(b)) + } + for _, a := range [][]string{{"-ngl", "30"}, {"--chat-template-file", "/a b/t.jinja"}, {"x", "l'apostrophe ici"}, {""}} { + fx.Join = append(fx.Join, map[string]any{"args": a, "out": joinArgs(a)}) + } + for _, c := range [][3]string{ + {"A=1\nB=2\n", "B", "3"}, {"A=1\nexport B=2\nB=4\n", "B", ""}, {"A=1", "C", "deux mots"}, {"", "X", "1"}, + {"NGL=999\n# NGL=1\n", "NGL", ""}, {"K.E=1\nKXE=2\n", "K.E", "v"}, + } { + fx.SetKey = append(fx.SetKey, map[string]string{"content": c[0], "key": c[1], "val": c[2], "out": presetSetKey(c[0], c[1], c[2])}) + } + for _, c := range []string{ + "# NAME=MoE\nMODEL=m.gguf\nCTX=65536\nNGL=999\nEXTRA_ARGS=-ot 'exps=CPU,per_layer_token_embd=CPU,blk\\.0=CUDA0' --n-cpu-moe 0 -ub 512 --mlock --fit off -ctk q8_0\n", + "MODEL=m.gguf\nEXTRA_ARGS=--n-cpu-moe 30 -ts 1,1\n", "MODEL=m.gguf\nEXTRA_ARGS=--cpu-moe -sm row\n", + "MODEL=m.gguf\nEXTRA_ARGS=--cpu-moe\nCTX=0\n", "MODEL=m.gguf\nEXTRA_ARGS=--cpu-moe\nSPLIT_MODE=tensor\n", + "MODEL=m.gguf\nEXTRA_ARGS=-ngl 40\n", "EXTERNAL=1\n", "MODEL=m\nEXTRA_ARGS=--cpu-moe --fit=on -b=1024\nKV_TYPE=q4_0\nUBATCH=1024\n", + } { + out, changes, err := autoPlacementPreset(c) + e := "" + if err != nil { + e = err.Error() + } + fx.Autoplace = append(fx.Autoplace, map[string]any{"in": c, "out": out, "changes": changes, "err": e}) + } + for _, c := range []struct{ cfg, cur map[string]string }{ + {map[string]string{"MODEL": "a.gguf"}, map[string]string{}}, + {map[string]string{"MODEL": "a.gguf", "CUDA_VISIBLE_DEVICES": "1"}, map[string]string{"CUDA_VISIBLE_DEVICES": "0"}}, + {map[string]string{"MODEL": "a.gguf"}, map[string]string{"CUDA_VISIBLE_DEVICES": " 0,1 "}}, + } { + fx.BenchFP = append(fx.BenchFP, map[string]any{"cfg": c.cfg, "cur": c.cur, "out": benchFingerprint(c.cfg, c.cur)}) + } + fx.Imposes = map[string]bool{} + for _, p := range []string{"", "/app/llama-server", filepath.Join(home, "engine", "b1", "llama-server"), + filepath.Join(home, "backends", "llama.cpp-prebuilt", "llama-b2", "llama-server"), "/opt/custom/llama-server"} { + fx.Imposes[strings.ReplaceAll(p, home, "{{HOME}}")] = presetImposesEngine(p) + } + + fx.Seed = map[string]string{ + "config/BIN": "/opt/custom/llama-server", + "config/HOST": "127.0.0.1", + "config/PORT": "8081", + "config/MODEL": "live.gguf", + } + apply := func(store map[string]string) { + keys := make([]string, 0, len(store)) + for k := range store { + keys = append(keys, k) + } + sort.Strings(keys) + for _, k := range keys { + b, key, _ := strings.Cut(k, "/") + if err := putStr(b, key, store[k]); err != nil { + t.Fatal(err) + } + } + } + apply(fx.Seed) + + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + b, _ := io.ReadAll(r.Body) + switch { + case strings.HasPrefix(r.URL.Path, "/ok/"): + if !strings.Contains(string(b), `"max_tokens":1`) || r.Header.Get("Authorization") != "Bearer k1" { + w.WriteHeader(400) + return + } + io.WriteString(w, "{}") + case strings.HasPrefix(r.URL.Path, "/refus/"): + w.WriteHeader(401) + io.WriteString(w, `{"error":{"message":"clé invalide"}}`) + case strings.HasPrefix(r.URL.Path, "/brut/"): + w.WriteHeader(404) + io.WriteString(w, " page absente ") + default: + w.WriteHeader(500) + } + })) + defer srv.Close() + + disk := func() map[string]string { + out := map[string]string{} + for _, d := range []string{"presets", "models"} { + entries, _ := os.ReadDir(filepath.Join(home, d)) + for _, e := range entries { + b, _ := os.ReadFile(filepath.Join(home, d, e.Name())) + out[d+"/"+e.Name()] = string(b) + } + } + return out + } + state := func() map[string]any { + st := map[string]any{} + for k, v := range allKV(bkState) { + switch { + case strings.HasPrefix(k, sysPromptKeyPrefix): + st[k] = v + case k == presetOrderKey || k == "bench_presets": + var x any + _ = json.Unmarshal([]byte(v), &x) + st[k] = x + } + } + return st + } + run := func(st parityPresetStep) parityPresetStep { + for rel, content := range st.Files { + p := filepath.Join(home, filepath.FromSlash(rel)) + if content == "" { + _ = os.Remove(p) + continue + } + _ = os.MkdirAll(filepath.Dir(p), 0o755) + if err := os.WriteFile(p, []byte(content), 0o644); err != nil { + t.Fatal(err) + } + } + apply(st.Store) + body := strings.ReplaceAll(st.Body, "{{URL}}", srv.URL) + w := httptest.NewRecorder() + r := httptest.NewRequest(st.Method, st.Path, strings.NewReader(body)) + switch r.URL.Path { + case "/api/presets": + handlePresets(w, r) + case "/api/presets/order": + handlePresetsOrder(w, r) + case "/api/preset": + handlePreset(w, r) + case "/api/preset/save": + handlePresetSave(w, r) + case "/api/preset/cacheram": + handlePresetCacheRAM(w, r) + case "/api/preset/moe": + handlePresetMoE(w, r) + case "/api/preset/autoplace": + handlePresetAutoPlace(w, r) + case "/api/preset/delete": + handlePresetDelete(w, r) + case "/api/preset/external": + handlePresetExternal(w, r) + case "/api/preset/external/save": + handlePresetExternalSave(w, r) + case "/api/preset/external/test": + handlePresetExternalTest(w, r) + default: + t.Fatalf("route inattendue %s", st.Path) + } + st.Status = w.Code + if err := json.Unmarshal(w.Body.Bytes(), &st.Out); err != nil { + t.Fatalf("%s %s : %v (%s)", st.Method, st.Path, err, w.Body.String()) + } + st.Disk = disk() + st.State = state() + return st + } + + moe := "# NAME=MoE\nMODEL=absent.gguf\nCTX=32768\nEXTRA_ARGS=--n-cpu-moe 20 --load-mode none\n" + for _, st := range []parityPresetStep{ + {Method: "GET", Path: "/api/presets"}, + {Method: "GET", Path: "/api/preset"}, + {Method: "POST", Path: "/api/preset/save", Body: `{"name":"Qwen 27B","content":"MODEL=Qwen3-27B-Q4_K_M.gguf\nCTX=65536\nREASONING=on\n"}`}, + {Method: "POST", Path: "/api/preset/save", Body: `{"name":"Qwen 27B","content":"MODEL=Qwen3-27B-Q8_0.gguf\nMMPROJ=mm.gguf\n","sysprompt":" Réponds en français. "}`}, + {Method: "POST", Path: "/api/preset/save", Body: `{"name":"a/b\\c","content":"MODEL=x.gguf\nCTX=40000\n"}`}, + {Method: "POST", Path: "/api/preset/save", Body: `{"name":" ","content":"X=1"}`}, + {Method: "POST", Path: "/api/preset/save", Body: `{"id":"../evil","name":"E","content":"X=1"}`}, + {Method: "POST", Path: "/api/preset/save", Body: `{"id":"Qwen 27B (2)","name":"Qwen Q8","content":"# NAME=ancien\nMODEL=Qwen3-27B-Q8_0.gguf\nQUANT=q8\n","sysprompt":""}`}, + {Method: "POST", Path: "/api/preset/save", Body: `pas du json`}, + {Method: "POST", Path: "/api/preset/save", Body: `{"deleteModel":"oui"}`}, + {Method: "POST", Path: "/api/preset/save", Body: `{"name":"Strata","content":"ENGINE=strata\nSTRATA_VISION=0\nCTX=1048576\n"}`}, + {Method: "GET", Path: "/api/presets"}, + {Method: "GET", Path: "/api/preset?id=Qwen%2027B"}, + {Method: "GET", Path: "/api/preset?id=Qwen%2027B%20(2)"}, + {Method: "GET", Path: "/api/preset?id=absent"}, + {Method: "GET", Path: "/api/preset?id=../x"}, + {Method: "POST", Path: "/api/presets/order", Body: `{"ids":["Strata","Qwen 27B (2)"]}`}, + {Method: "GET", Path: "/api/presets"}, + {Method: "POST", Path: "/api/presets/order", Body: `{}`}, + {Method: "GET", Path: "/api/presets/order"}, + {Method: "POST", Path: "/api/presets/order", Body: `{"ids":"x"}`}, + {Method: "POST", Path: "/api/presets/order", Body: `{"ids":["a-b-c"]}`}, + // Mesures : empreinte, repli sur le nom du modèle, autre configuration. + {Store: map[string]string{"state/bench_presets": `{"Qwen 27B":{"result":{"prompt_per_second":1234.5,"predicted_per_second":40,"kv":"q8_0","kv_quantized":true,"depth":{"cold":{"new":30000},"decode_per_second":31.25}},"model":"Qwen3-27B-Q4_K_M.gguf","at":1700000000},` + + `"Qwen 27B (2)":{"result":{"prompt_per_second":900,"predicted_per_second":20.5},"model":"autre.gguf","at":1700000001},` + + `"a-b-c":{"result":{"prompt_per_second":1,"predicted_per_second":2},"model":"x.gguf","at":1,"fingerprint":"mauvaise"}}`}, + Method: "GET", Path: "/api/presets"}, + // Preset actif : la configuration est celle de « a-b-c ». + {Store: map[string]string{"config/BIN": "", "config/HOST": "", "config/PORT": "", "config/MODEL": "x.gguf", "config/CTX": "40000", "config/MEM_MODE": "always"}, + Method: "GET", Path: "/api/presets"}, + {Method: "POST", Path: "/api/preset/delete", Body: `{"id":"a-b-c"}`}, + {Store: map[string]string{"config/MODEL": "autre.gguf"}, Method: "POST", Path: "/api/preset/delete", Body: `{"id":"a-b-c"}`}, + {Method: "POST", Path: "/api/preset/delete", Body: `{"id":"absent"}`}, + {Method: "POST", Path: "/api/preset/delete", Body: `{"id":"a/b"}`}, + {Files: map[string]string{"models/Qwen3-27B-Q4_K_M.gguf": "poids"}, Method: "POST", Path: "/api/preset/delete", Body: `{"id":"Qwen 27B","deleteModel":true}`}, + {Files: map[string]string{"models/Qwen3-27B-Q8_0.gguf": "poids8", "presets/Copie.env": "# NAME=Copie\nMODEL=Qwen3-27B-Q8_0.gguf\n"}, + Method: "POST", Path: "/api/preset/delete", Body: `{"id":"Qwen 27B (2)","deleteModel":true}`}, + {Method: "POST", Path: "/api/preset/delete", Body: `{"id":"Copie","deleteModel":true}`}, + {Method: "POST", Path: "/api/preset/delete", Body: `{"id":3}`}, + {Method: "GET", Path: "/api/presets"}, + // Presets externes. + {Method: "GET", Path: "/api/preset/external"}, + {Method: "POST", Path: "/api/preset/external/save", Body: `{"name":" Groq ","url":" https://api.groq.com/openai/v1 ","model":" llama ","key":"k1","ctx":"131072","vision":true}`}, + {Method: "POST", Path: "/api/preset/external/save", Body: `{"name":"X","url":"","model":"m"}`}, + {Method: "POST", Path: "/api/preset/external/save", Body: `{"name":"X","url":"u","model":" "}`}, + {Method: "POST", Path: "/api/preset/external/save", Body: `{"name":"","url":"u","model":"m"}`}, + {Method: "GET", Path: "/api/preset/external?id=Groq"}, + {Method: "POST", Path: "/api/preset/external/save", Body: `{"id":"Groq","name":"Groq 2","url":"https://api.groq.com","model":"llama-3"}`}, + {Method: "GET", Path: "/api/preset/external?id=Groq"}, + {Method: "POST", Path: "/api/preset/external/save", Body: `{"id":"Groq","name":"Groq","url":"https://api.groq.com","model":"llama-3","keyTouched":true,"key":""}`}, + {Method: "GET", Path: "/api/preset/external?id=Groq"}, + {Method: "GET", Path: "/api/preset/external?id=Strata"}, + {Method: "GET", Path: "/api/preset/external?id=absent"}, + {Method: "GET", Path: "/api/presets"}, + {Method: "POST", Path: "/api/preset/external/test", Body: `{"url":"{{URL}}/ok","model":"m","key":"k1"}`}, + {Method: "POST", Path: "/api/preset/external/test", Body: `{"url":"{{URL}}/refus/v1","model":"m"}`}, + {Method: "POST", Path: "/api/preset/external/test", Body: `{"url":"{{URL}}/brut/v1/chat/completions","model":"m"}`}, + {Method: "POST", Path: "/api/preset/external/test", Body: `{"url":"{{URL}}/rien","model":"m"}`}, + {Method: "POST", Path: "/api/preset/external/test", Body: `{"url":"","model":"m"}`}, + {Method: "POST", Path: "/api/preset/external/save", Body: `{"name":"Clé","url":"{{URL}}/ok","model":"m","key":"k1"}`}, + {Method: "POST", Path: "/api/preset/external/test", Body: `{"id":"Clé","url":"{{URL}}/ok","model":"m"}`}, + {Method: "POST", Path: "/api/preset/external/test", Body: `[]`}, + // Placement auto et aperçus de l'éditeur. + {Method: "GET", Path: "/api/preset/autoplace"}, + {Method: "POST", Path: "/api/preset/autoplace", Body: `{"id":"MoE","content":` + jsonString(moe) + `}`}, + {Method: "POST", Path: "/api/preset/autoplace", Body: `{"name":" Mon MoE ","content":"MODEL=m\nEXTRA_ARGS=-ngl 30\n"}`}, + {Method: "POST", Path: "/api/preset/autoplace", Body: `{"content":"MODEL=m\nEXTRA_ARGS=--cpu-moe\n"}`}, + {Method: "POST", Path: "/api/preset/moe", Body: `{"content":` + jsonString(moe) + `}`}, + {Method: "POST", Path: "/api/preset/moe", Body: `{"content":"EXTERNAL=1\n"}`}, + {Method: "POST", Path: "/api/preset/moe", Body: `{"content":"MODEL=absent.gguf\nEXTRA_ARGS=--cpu-moe -sm row\n"}`}, + {Method: "POST", Path: "/api/preset/moe", Body: `{"content":5}`}, + {Method: "POST", Path: "/api/preset/cacheram", Body: `{"content":"EXTERNAL=1\nSIDE_SLOT=on\n"}`}, + {Method: "POST", Path: "/api/preset/cacheram", Body: `{"content":"EXTRA_ARGS=-cram 2048\n"}`}, + {Method: "POST", Path: "/api/preset/cacheram", Body: `{"content":"CACHE_RAM=4096\nSIDE_SLOT=on\nMODEL=absent.gguf\n"}`}, + {Method: "POST", Path: "/api/preset/cacheram", Body: `{"content":"CACHE_RAM=beaucoup\n"}`}, + {Method: "POST", Path: "/api/preset/cacheram", Body: `{"content":"CACHE_RAM=auto\n"}`}, + {Method: "POST", Path: "/api/preset/cacheram", Body: `{"content":"MODEL=absent.gguf\nSIDE_SLOT=on\nPARALLEL=3\n"}`}, + {Method: "POST", Path: "/api/preset/cacheram", Body: `{}`}, + {Method: "GET", Path: "/api/presets"}, + } { + fx.Steps = append(fx.Steps, run(st)) + } + + got, err := parityJSON(fx) + if err != nil { + t.Fatal(err) + } + got = []byte(strings.ReplaceAll(strings.ReplaceAll(string(got), srv.URL, "{{URL}}"), home, "{{HOME}}")) + if os.Getenv("LOKI_PARITY_UPDATE") == "1" { + if err := os.MkdirAll(filepath.Dir(parityPresetsFile), 0o755); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(parityPresetsFile, got, 0o644); err != nil { + t.Fatal(err) + } + return + } + want, err := os.ReadFile(parityPresetsFile) + if err != nil { + t.Fatalf("journal de référence absent (%v) : LOKI_PARITY_UPDATE=1 pour le créer", err) + } + if string(got) != string(want) { + t.Errorf("les presets ne produisent plus le journal rangé dans %s.\n"+ + "Si c'est voulu : LOKI_PARITY_UPDATE=1 go test ./internal/loki -run TestParitePresets,\n"+ + "puis cargo test --test presets_api dit si le port Rust suit.\n--- obtenu ---\n%s", parityPresetsFile, got) + } +} + +func jsonString(s string) string { + b, _ := json.Marshal(s) + return string(b) +} + +// La bascule et l'enregistrement du preset en service passent par +// presetApply (façade) : configuration appliquée, moteur relancé ou arrêté. +func TestFacadePresetApply(t *testing.T) { + testHome(t) + var actions []string + old := presetServiceAction + done := make(chan struct{}, 4) + presetServiceAction = func(a string) error { actions = append(actions, a); done <- struct{}{}; return nil } + t.Cleanup(func() { presetServiceAction = old }) + if err := os.MkdirAll(presetsDir(), 0o755); err != nil { + t.Fatal(err) + } + local := filepath.Join(presetsDir(), "local.env") + ext := filepath.Join(presetsDir(), "ext.env") + _ = os.WriteFile(local, []byte("MODEL=a.gguf\nCTX=8192\n"), 0o644) + _ = os.WriteFile(ext, []byte("EXTERNAL=1\nEXTERNAL_URL=u\n"), 0o644) + _ = SetConfigKey("HOST", "127.0.0.1") + if err := presetApply(local, "switch"); err != nil { + t.Fatal(err) + } + <-done + if cfg := ReadConfig(); cfg["CTX"] != "8192" || cfg["HOST"] != "127.0.0.1" { + t.Fatalf("configuration après bascule : %v", cfg) + } + if err := presetApply(local, "save"); err != nil { + t.Fatal(err) + } + <-done + if err := presetApply(ext, "apply"); err != nil { + t.Fatal(err) + } + if !isExternalConfig(ReadConfig()) || len(actions) != 2 || actions[0] != "restart" || actions[1] != "restart" { + t.Fatalf("actions %v, config %v", actions, ReadConfig()) + } + if err := presetApply("/etc/passwd", "switch"); err == nil { + t.Fatal("preset hors du dossier accepté") + } + if err := presetApply(local, "bizarre"); err == nil { + t.Fatal("mode inconnu accepté") + } + w := httptest.NewRecorder() + handleInternalTuneGuard(w, httptest.NewRequest("GET", "/internal/v1/tune/guard", nil)) + if w.Body.String() != "{\"busy\":false}\n" { + t.Fatalf("garde : %s", w.Body.String()) + } +} diff --git a/internal/loki/web_internal.go b/internal/loki/web_internal.go index 719f34d..e2d0e41 100644 --- a/internal/loki/web_internal.go +++ b/internal/loki/web_internal.go @@ -109,6 +109,10 @@ func withInternalAPI(next http.Handler) http.Handler { handleInternalConvDropped(w, r) case "/internal/v1/conv/workspace": handleInternalConvWorkspace(w, r) + case "/internal/v1/preset/apply": + handleInternalPresetApply(w, r) + case "/internal/v1/tune/guard": + handleInternalTuneGuard(w, r) default: http.NotFound(w, r) } diff --git a/rust/loki-rs/src/api.rs b/rust/loki-rs/src/api.rs index 9f2a1a5..e44c354 100644 --- a/rust/loki-rs/src/api.rs +++ b/rust/loki-rs/src/api.rs @@ -40,6 +40,8 @@ pub struct Ctx { pub push: Option>, /// Mémoire tenue par la façade (`LOKI_MEM=rust`). pub mem: Option>, + /// Presets tenus par la façade (`LOKI_PRESETS=rust`). + pub presets: bool, /// Jeton de l'API interne : une requête qui le porte vient du Go. pub internal_token: String, } @@ -55,6 +57,7 @@ impl Ctx { || self.files.is_some() || self.push.is_some() || self.mem.is_some() + || self.presets } /// La requête vient du Go (jeton interne valide) : déjà authentifiée par diff --git a/rust/loki-rs/src/lib.rs b/rust/loki-rs/src/lib.rs index b92ecef..91fa5e4 100644 --- a/rust/loki-rs/src/lib.rs +++ b/rust/loki-rs/src/lib.rs @@ -28,6 +28,7 @@ pub mod models; pub mod oai; pub mod paths; pub mod presets; +pub mod presets_api; pub mod proxy; pub mod push; pub mod serve; diff --git a/rust/loki-rs/src/main.rs b/rust/loki-rs/src/main.rs index 343a2d4..da75b67 100644 --- a/rust/loki-rs/src/main.rs +++ b/rust/loki-rs/src/main.rs @@ -145,6 +145,9 @@ fn serve() -> ExitCode { // LOKI_MEM=rust : la mémoire (pages, index, trackers, mode mémoire, // interrupteurs du mode agent). let mem_rust = held_by_facade("LOKI_MEM"); + // LOKI_PRESETS=rust : les presets (liste, éditeur, aperçus, externes, + // bascule ; l'application au moteur reste au Go). + let presets_rust = held_by_facade("LOKI_PRESETS"); let mut env = vec![ (auth::INTERNAL_ENV.to_string(), token.clone()), // Adresse de l'API interne du Go, héritée par ses enfants : @@ -159,6 +162,7 @@ fn serve() -> ExitCode { ("LOKI_FILES", files_rust), ("LOKI_PUSH", push_rust), ("LOKI_MEM", mem_rust), + ("LOKI_PRESETS", presets_rust), ]; for (var, on) in held { if on { @@ -238,6 +242,7 @@ fn serve() -> ExitCode { files, push: push_rust.then(|| std::sync::Arc::new(loki_rs::push::Push::new())), mem: mem_rust.then(|| std::sync::Arc::new(loki_rs::memory::Memory::new(mem_home))), + presets: presets_rust, }; let res = proxy::serve(listener, cfg, shutdown_signal()).await; eprintln!("[loki-rs] arrêt demandé — arrêt du serveur Go"); diff --git a/rust/loki-rs/src/models.rs b/rust/loki-rs/src/models.rs index 338402f..d23bd71 100644 --- a/rust/loki-rs/src/models.rs +++ b/rust/loki-rs/src/models.rs @@ -244,7 +244,7 @@ impl ModelRefs { } /// `modelUsers` : pour UN modèle (nom de fichier ou chemin). - fn users(&self, dirs: &ModelDirs, name: &str) -> (Vec, bool) { + pub(crate) fn users(&self, dirs: &ModelDirs, name: &str) -> (Vec, bool) { let k = dirs.ref_key(name); if k.is_empty() { return (Vec::new(), false); @@ -400,7 +400,11 @@ pub fn delete_model( } /// `deleteModelFile` : rend les octets libérés. -fn delete_model_file(dirs: &ModelDirs, refs: &ModelRefs, name: &str) -> Result { +pub(crate) fn delete_model_file( + dirs: &ModelDirs, + refs: &ModelRefs, + name: &str, +) -> Result { let p = gguf::resolve_model_path(name, &dirs.home, &dirs.stored, dirs.env.as_deref())? .to_string_lossy() .into_owned(); diff --git a/rust/loki-rs/src/presets_api.rs b/rust/loki-rs/src/presets_api.rs new file mode 100644 index 0000000..35fc1ff --- /dev/null +++ b/rust/loki-rs/src/presets_api.rs @@ -0,0 +1,1458 @@ +//! Presets tenus par la façade — port de `backend_presets.go` (écriture), +//! des handlers de presets de `web_api.go`, de `web_external.go`, et des +//! aperçus de l'éditeur (`cacheRAMPreview`, `sideSlotPreview`, `moePreview`, +//! `autoPlacementPreset`) — lot 15 de la migration, servi quand +//! `LOKI_PRESETS=rust` (expérimental ; le Go par défaut) : +//! +//! - `GET /api/presets` (pastilles et mesures), `POST /api/presets/order` ; +//! - `GET /api/preset?id=`, `POST /api/preset/save`, `/api/preset/delete` ; +//! - `POST /api/preset/cacheram`, `/api/preset/moe`, `/api/preset/autoplace` ; +//! - `GET /api/preset/external?id=`, `POST /api/preset/external/save`, +//! `/api/preset/external/test` ; +//! - `POST /api/switch`. +//! +//! Les presets restent des fichiers `presets/*.env`. Ce qui touche au moteur +//! reste au Go jusqu'à son lot : le verrou de l'optimiseur +//! (`/internal/v1/tune/guard`) et l'application d'un preset à la +//! configuration vivante, état du slot gardé et moteur relancé +//! (`/internal/v1/preset/apply`). + +use std::collections::{BTreeMap, HashMap}; +use std::path::{Path, PathBuf}; + +use http_body_util::{BodyExt, Limited}; +use hyper::body::Incoming; +use hyper::{Method, Request, Response, StatusCode}; +use serde_json::{Value, json}; + +use crate::api::Ctx; +use crate::auth::json as send_json; +use crate::gojson::{self, Kind}; +use crate::presets::{ + PRESET_ORDER_KEY, config_fingerprint, list_presets, parse_env, parse_preset_order, preset_ctx, + preset_display_name, preset_fingerprint, presets_dir, unquote_value, +}; +use crate::proxy::Body; +use crate::serve::{self, Cfg, SysInfo}; +use crate::store::{BK_STATE, Store}; + +const SYSPROMPT_PREFIX: &str = "sysprompt:"; +const BENCH_KEY: &str = "bench_presets"; +const BENCH_PROTOCOL: &str = "2"; +/// `previewServeHelp` : l'aide supposée d'un moteur récent. +pub const PREVIEW_SERVE_HELP: &str = "--load-mode --fit [on|off] --fit-target"; + +/// Routes servies ici. +pub fn handles(path: &str) -> bool { + matches!( + path, + "/api/presets" + | "/api/presets/order" + | "/api/preset" + | "/api/preset/save" + | "/api/preset/cacheram" + | "/api/preset/moe" + | "/api/preset/autoplace" + | "/api/preset/delete" + | "/api/preset/external" + | "/api/preset/external/save" + | "/api/preset/external/test" + | "/api/switch" + ) +} + +fn trim(s: &str) -> &str { + s.trim_matches(char::is_whitespace) +} + +fn get<'a>(cfg: &'a Cfg, k: &str) -> &'a str { + cfg.get(k).map_or("", String::as_str) +} + +// --------------------------------------------------------------------------- +// Fonctions pures +// --------------------------------------------------------------------------- + +/// `withDisplayName` : la ligne `# NAME=` remplacée, ou ajoutée en tête. +pub fn with_display_name(content: &str, name: &str) -> String { + let re = regex::Regex::new(r"(?mi)^[ \t]*#?[ \t]*NAME[ \t]*=.*$").expect("motif"); + let line = format!("# NAME={name}"); + if re.is_match(content) { + // ReplaceAllString du Go : « $x » est développé, comme ici. + return re.replace_all(content, line.as_str()).into_owned(); + } + format!("{line}\n{content}") +} + +/// `quoteValue`. +pub fn quote_value(v: &str) -> String { + if v.is_empty() || v.contains('"') { + return v.to_string(); + } + if v.contains([' ', '\t']) || unquote_value(v) != v { + return format!("\"{v}\""); + } + v.to_string() +} + +/// `formatEnv` : clés triées. +pub fn format_env(m: &BTreeMap) -> String { + m.iter() + .map(|(k, v)| format!("{k}={}\n", quote_value(v))) + .collect() +} + +/// `validPresetName`. +pub fn valid_preset_name(name: &str) -> Result<(), String> { + if name.is_empty() { + return Err("nom vide".into()); + } + if name == "." || name == ".." { + return Err("nom réservé".into()); + } + if name.contains(['/', '\\', '\0']) { + return Err("le nom ne peut pas contenir / ni \\".into()); + } + if name.chars().any(|c| (c as u32) < 0x20) { + return Err("le nom contient un caractère de contrôle invalide".into()); + } + Ok(()) +} + +/// `safePresetPath`. +pub fn safe_preset_path(home: &Path, name: &str) -> Result { + let name = trim(name); + valid_preset_name(name)?; + Ok(presets_dir(home).join(format!("{name}.env"))) +} + +/// `uniquePresetID`. +pub fn unique_preset_id(home: &Path, name: &str) -> Result { + let replaced = name.replace(['/', '\\'], "-"); + let mut base = trim(&replaced).to_string(); + if base.is_empty() { + base = "preset".into(); + } + let mut cand = base.clone(); + for n in 2..10000 { + let p = safe_preset_path(home, &cand)?; + if matches!(std::fs::metadata(&p), Err(e) if e.kind() == std::io::ErrorKind::NotFound) { + return Ok(cand); + } + cand = format!("{base} ({n})"); + } + Err("impossible de générer un nom de fichier unique".into()) +} + +/// `ReadPreset`. +pub fn read_preset(home: &Path, id: &str) -> Result { + let p = safe_preset_path(home, id)?; + std::fs::read(&p) + .map(|b| String::from_utf8_lossy(&b).into_owned()) + .map_err(|e| crate::paths::os_err("open", &p.to_string_lossy(), &e)) +} + +fn write_preset(p: &Path, content: &str) -> Result<(), String> { + if let Some(d) = p.parent() { + let _ = std::fs::create_dir_all(d); + } + let mut o = std::fs::OpenOptions::new(); + o.write(true).create(true).truncate(true); + #[cfg(unix)] + { + use std::os::unix::fs::OpenOptionsExt; + o.mode(0o644); + } + use std::io::Write as _; + o.open(p) + .and_then(|mut f| f.write_all(content.as_bytes())) + .map_err(|e| crate::paths::os_err("open", &p.to_string_lossy(), &e)) +} + +/// `SavePreset` : création (id vide, nom de fichier unique) ou mise à jour. +pub fn save_preset(home: &Path, id: &str, name: &str, content: &str) -> Result { + let name = trim(name); + if name.is_empty() { + return Err("nom requis".into()); + } + let content = with_display_name(content, name); + let id = if id.is_empty() { + unique_preset_id(home, name)? + } else { + id.to_string() + }; + let p = safe_preset_path(home, &id)?; + write_preset(&p, &content)?; + Ok(id) +} + +/// `presetReasoning`. +pub fn preset_reasoning(content: &str) -> String { + for line in content.split('\n') { + let s = trim(line); + if s.is_empty() || s.starts_with('#') { + continue; + } + if let Some((k, v)) = s.split_once('=') + && trim(k).eq_ignore_ascii_case("REASONING") + { + return unquote_value(trim(v)).to_string(); + } + } + String::new() +} + +/// `modelFromPresetContent`. +pub fn model_from_preset_content(content: &str) -> String { + for line in content.split('\n') { + let s = trim(line); + if s.is_empty() || s.starts_with('#') { + continue; + } + if let Some((k, v)) = s.split_once('=') + && trim(k) == "MODEL" + { + return unquote_value(trim(v)).to_string(); + } + } + String::new() +} + +/// `detectQuant` : une ligne QUANT= (avec ou sans `#`), sinon le nom du +/// modèle. +pub fn detect_quant(content: &str) -> String { + for line in content.split('\n') { + let t = trim(line); + let s = trim(t.strip_prefix('#').unwrap_or(t)); + if let Some((k, v)) = s.split_once('=') + && trim(k).eq_ignore_ascii_case("QUANT") + { + let v = unquote_value(trim(v)); + if !v.is_empty() { + return v.to_uppercase(); + } + } + } + crate::models::quant_from_name(&crate::paths::base_name(&model_from_preset_content( + content, + ))) +} + +/// `completionsURL`. +pub fn completions_url(raw: &str) -> String { + let u = trim(raw).trim_end_matches('/'); + if u.is_empty() { + return String::new(); + } + if u.ends_with("/chat/completions") { + return u.to_string(); + } + if u.ends_with("/v1") { + return format!("{u}/chat/completions"); + } + format!("{u}/v1/chat/completions") +} + +/// `externalPresetContent`. +pub fn external_preset_content( + url: &str, + model: &str, + key: &str, + ctx: &str, + vision: bool, +) -> String { + let mut m = BTreeMap::new(); + m.insert("EXTERNAL".to_string(), "1".to_string()); + m.insert("EXTERNAL_URL".into(), trim(url).into()); + m.insert("EXTERNAL_MODEL".into(), trim(model).into()); + if !trim(key).is_empty() { + m.insert("EXTERNAL_KEY".into(), trim(key).into()); + } + if vision { + m.insert("EXTERNAL_VISION".into(), "1".into()); + } + if !trim(ctx).is_empty() { + m.insert("CTX".into(), trim(ctx).into()); + } + format_env(&m) +} + +/// `extractAPIError` : `error.message` d'une réponse au format OpenAI. +pub fn extract_api_error(b: &[u8]) -> String { + match serde_json::from_slice::(b) { + Ok(v) => v + .get("error") + .and_then(|e| e.get("message")) + .and_then(Value::as_str) + .map(|s| trim(s).to_string()) + .unwrap_or_default(), + Err(_) => String::new(), + } +} + +/// `presetImposesEngine` : seul un backend personnalisé est imposé par un +/// preset ; le moteur de l'image, un téléchargé ou un précompilé suivent la +/// machine. +pub fn preset_imposes_engine(home: &Path, p: &str) -> bool { + let p = trim(p); + let ep = serve::run::EnginePaths { + home: home.to_path_buf(), + }; + if p.is_empty() || ep.engine_owns(p) || ep.prebuilt_owns(p) { + return false; + } + let img = std::env::var("LOKI_ENGINE_BIN").unwrap_or_default(); + let img = trim(&img); + if !img.is_empty() && serve::run::same_path(p, img) { + return false; + } + true +} + +/// `newPresetSeed` : BIN (backend personnalisé seulement), HOST, PORT. +pub fn new_preset_seed(home: &Path, cur: &Cfg) -> BTreeMap { + let mut seed = BTreeMap::new(); + for k in ["BIN", "HOST", "PORT"] { + let v = get(cur, k); + if v.is_empty() || (k == "BIN" && !preset_imposes_engine(home, v)) { + continue; + } + seed.insert(k.to_string(), v.to_string()); + } + seed +} + +/// `benchFingerprint`. +pub fn bench_fingerprint(cfg: &Cfg, cur: &Cfg) -> String { + let mut gpus = trim(get(cfg, "CUDA_VISIBLE_DEVICES")).to_string(); + if gpus.is_empty() { + gpus = trim(get(cur, "CUDA_VISIBLE_DEVICES")).to_string(); + } + let mut m = HashMap::new(); + m.insert("config".to_string(), config_fingerprint(cfg)); + m.insert("gpus".into(), gpus); + m.insert("protocol".into(), BENCH_PROTOCOL.into()); + config_fingerprint(&m) +} + +/// `benchMatchesPreset`. +pub fn bench_matches_preset(sb: &Value, cfg: &Cfg, cur: &Cfg) -> bool { + let fp = sb.get("fingerprint").and_then(Value::as_str).unwrap_or(""); + if !fp.is_empty() { + return fp == bench_fingerprint(cfg, cur); + } + let m = trim(get(cfg, "MODEL")); + !m.is_empty() && sb.get("model").and_then(Value::as_str) == Some(&crate::paths::base_name(m)) +} + +/// Nombre JSON au rendu d'`encoding/json` (un flottant entier s'écrit sans +/// « .0 »). +fn go_num(f: f64) -> Value { + if f.fract() == 0.0 && f.abs() < 1e15 { + json!(f as i64) + } else { + json!(f) + } +} + +fn num(v: &Value, k: &str) -> f64 { + v.get(k).and_then(Value::as_f64).unwrap_or(0.0) +} + +/// Pastille de mesure d'un preset (`bench` de `handlePresets`). +fn bench_badge(sb: &Value) -> Value { + let r = sb.get("result").cloned().unwrap_or(Value::Null); + let mut b = serde_json::Map::new(); + b.insert("prefill".into(), go_num(num(&r, "prompt_per_second"))); + b.insert("decode".into(), go_num(num(&r, "predicted_per_second"))); + b.insert( + "at".into(), + json!(sb.get("at").and_then(Value::as_i64).unwrap_or(0)), + ); + let kv = r.get("kv").and_then(Value::as_str).unwrap_or(""); + if !kv.is_empty() { + b.insert("kv".into(), json!(kv)); + b.insert( + "kv_quantized".into(), + json!( + r.get("kv_quantized") + .and_then(Value::as_bool) + .unwrap_or(false) + ), + ); + } + if let Some(d) = r.get("depth").filter(|d| d.is_object()) { + let dd = num(d, "decode_per_second"); + if let Some(c) = d.get("cold").filter(|c| c.is_object()) + && dd > 0.0 + { + b.insert( + "depth".into(), + json!(c.get("new").and_then(Value::as_i64).unwrap_or(0)), + ); + b.insert("depth_decode".into(), go_num(dd)); + } + } + Value::Object(b) +} + +/// `joinArgs` : l'inverse de `splitArgs`. +pub fn join_args(args: &[String]) -> String { + args.iter() + .map(|a| { + if a.is_empty() { + "\"\"".to_string() + } else if a.contains([' ', '\t']) { + if a.contains('\'') { + format!("\"{a}\"") + } else { + format!("'{a}'") + } + } else { + a.clone() + } + }) + .collect::>() + .join(" ") +} + +/// `presetSetKey` : lignes KEY= remplacées (toutes), ajoutée à la fin si +/// absente, retirées si `val` est vide. +pub fn preset_set_key(content: &str, key: &str, val: &str) -> String { + let k = regex::escape(key); + if val.is_empty() { + let re = regex::Regex::new(&format!( + r"(?m)^[ \t]*(?:export[ \t]+)?{k}[ \t]*=.*(?:\n|$)" + )) + .expect("motif"); + return re.replace_all(content, regex::NoExpand("")).into_owned(); + } + let re = + regex::Regex::new(&format!(r"(?m)^[ \t]*(?:export[ \t]+)?{k}[ \t]*=.*$")).expect("motif"); + let line = format!("{key}={}", quote_value(val)); + if re.is_match(content) { + return re.replace_all(content, regex::NoExpand(&line)).into_owned(); + } + let mut c = content.to_string(); + if !c.is_empty() && !c.ends_with('\n') { + c.push('\n'); + } + c + &line + "\n" +} + +/// `autoPlacementPreset` : la COPIE d'un preset MoE placé à la main, en +/// placement automatique, et ce qui change. +pub fn auto_placement_preset(content: &str) -> Result<(String, Vec), String> { + let cfg = parse_env(content); + if serve::is_external(&cfg) { + return Err("preset externe : le modèle tourne ailleurs".into()); + } + let extra = crate::models::split_args(get(&cfg, "EXTRA_ARGS")); + let mut kept: Vec = Vec::new(); + let mut changes: Vec = Vec::new(); + let mut migrated = false; + let mut i = 0; + while i < extra.len() { + let a = &extra[i]; + let (name, val, has_eq) = match a.split_once('=') { + Some((n, v)) => (n.to_string(), v.to_string(), true), + None => (a.clone(), String::new(), false), + }; + let mut value = || -> String { + if has_eq { + return val.clone(); + } + if i + 1 < extra.len() { + i += 1; + return extra[i].clone(); + } + String::new() + }; + match name.as_str() { + "-ot" | "--override-tensor" => { + let v = value(); + let mut rest: Vec = Vec::new(); + for el in v.split(',') { + let el = trim(el); + let pat = el.split_once('=').map_or(el, |(p, _)| p); + if el.is_empty() { + } else if serve::ot_targets_experts(pat) { + changes.push(format!("retiré : {name} {el} (experts : --fit les place)")); + migrated = true; + } else if pat.to_lowercase().contains("per_layer_token_embd") + && serve::ot_on_cpu(el) + { + changes.push(format!( + "retiré : {name} {el} (couche d'entrée, toujours au CPU)" + )); + } else { + rest.push(el.to_string()); + } + } + if !rest.is_empty() { + kept.push(name.clone()); + kept.push(rest.join(",")); + } + } + "-ncmoe" | "--n-cpu-moe" => { + let v = value(); + changes.push(format!("retiré : {name} {v}")); + if trim(&v) != "0" { + migrated = true; + } + } + "-cmoe" | "--cpu-moe" => { + changes.push(format!("retiré : {name}")); + migrated = true; + } + "-ngl" | "--n-gpu-layers" | "--gpu-layers" | "-ts" | "--tensor-split" | "-b" + | "--batch-size" | "-ub" | "--ubatch-size" | "--load-mode" | "-lm" => { + let v = value(); + changes.push(format!("retiré : {name} {v}")); + } + "--mlock" | "--no-mmap" | "--mmap" => changes.push(format!("retiré : {name}")), + "-fit" | "--fit" => { + let v = value(); + if matches!( + trim(&v).to_lowercase().as_str(), + "off" | "disabled" | "false" | "0" + ) { + changes.push(format!("retiré : {name} {v}")); + } else { + kept.push(name.clone()); + kept.push(v); + } + } + _ => kept.push(a.clone()), + } + i += 1; + } + if !migrated { + return Err("rien à migrer : ce preset ne place pas d'experts à la main (-ot …exps…, --n-cpu-moe, --cpu-moe)".into()); + } + kept.push("--load-mode".into()); + kept.push("mmap".into()); + let mut nc = cfg.clone(); + nc.insert("EXTRA_ARGS".into(), join_args(&kept)); + nc.insert("UBATCH".into(), "2048".into()); + nc.insert("BATCH".into(), "4096".into()); + nc.remove("NGL"); + let (ctx, ok) = serve::ctx_fixed(&nc, &kept); + if !ok { + return Err(format!( + "contexte « {ctx} » non chiffré : --fit pourrait le réduire pour tenir — donne un CTX chiffré d'abord" + )); + } + if serve::split_mode_key(&nc) == "tensor" { + return Err("SPLIT_MODE=tensor coupe --fit : retire-le d'abord".into()); + } + let si = SysInfo { + help: PREVIEW_SERVE_HELP.into(), + ..SysInfo::default() + }; + let why = serve::fit_blocker(&nc, &kept, &si); + if !why.is_empty() { + return Err(format!( + "--fit resterait inactif ({why}) : retire-le à la main d'abord" + )); + } + let ngl = trim(get(&cfg, "NGL")); + if !ngl.is_empty() { + changes.push(format!( + "NGL={ngl} retiré : le moteur place les couches (-ngl auto)" + )); + } + changes.push(format!( + "UBATCH=2048, BATCH=4096 (avant : {} / {})", + serve_first(&[get(&cfg, "UBATCH"), "512"]), + serve_first(&[get(&cfg, "BATCH"), "2048"]) + )); + changes.push("--load-mode mmap".into()); + changes.push(format!("CTX conservé : {ctx}")); + let (k, v) = serve::serve_kv_types(&nc); + if !k.is_empty() + || !v.is_empty() + || !serve::flag_value(&kept, &["-ctk", "--cache-type-k"]).is_empty() + || !serve::flag_value(&kept, &["-ctv", "--cache-type-v"]).is_empty() + { + changes.push( + "cache KV conservé tel quel — garde-le identique pour comparer les deux presets".into(), + ); + } + let mut out = content.to_string(); + out = preset_set_key(&out, "EXTRA_ARGS", get(&nc, "EXTRA_ARGS")); + out = preset_set_key(&out, "UBATCH", "2048"); + out = preset_set_key(&out, "BATCH", "4096"); + out = preset_set_key(&out, "NGL", ""); + Ok((out, changes)) +} + +fn serve_first<'a>(vals: &[&'a str]) -> &'a str { + vals.iter() + .copied() + .find(|v| !trim(v).is_empty()) + .unwrap_or("") +} + +// --------------------------------------------------------------------------- +// Aperçus de l'éditeur (lectures du monde : GGUF, RAM, nvidia-smi) +// --------------------------------------------------------------------------- + +/// Résolution d'un MODEL= pour les aperçus. +pub struct Resolver { + pub home: PathBuf, + pub stored: Vec, + pub env: Option, +} + +impl Resolver { + fn resolve(&self, m: &str) -> String { + crate::gguf::resolve_serve_model_path(m, &self.home, &self.stored, self.env.as_deref()) + .map(|p| p.to_string_lossy().into_owned()) + .unwrap_or_default() + } +} + +fn lookup_env(si: &mut SysInfo, keys: &[&[&str]], trimmed: bool) { + for group in keys { + for k in *group { + if let Ok(v) = std::env::var(k) { + let keep = if trimmed { + !trim(&v).is_empty() + } else { + !v.is_empty() + }; + if keep { + si.arg_env.insert((*k).into(), v); + } + } + } + } +} + +/// `cacheRAMPreview` : (Mio, pourquoi) ; 0 = défaut du moteur. +pub fn cache_ram_preview(r: &Resolver, content: &str) -> (i64, String) { + let cfg = parse_env(content); + if serve::is_external(&cfg) { + return (0, "preset externe".into()); + } + let extra = crate::models::split_args(get(&cfg, "EXTRA_ARGS")); + let v = serve::flag_value(&extra, &["-cram", "--cache-ram"]); + if !v.is_empty() { + return (v.parse().unwrap_or(0), "fixé par EXTRA_ARGS".into()); + } + let e = std::env::var("LLAMA_ARG_CACHE_RAM").unwrap_or_default(); + if !trim(&e).is_empty() { + return ( + trim(&e).parse().unwrap_or(0), + "fixé par LLAMA_ARG_CACHE_RAM".into(), + ); + } + let (manual, auto, ok) = serve::cache_ram_setting(&cfg); + if !ok { + return (0, "CACHE_RAM illisible".into()); + } + if !auto { + return (manual.parse().unwrap_or(0), "fixé par CACHE_RAM".into()); + } + let mut si = SysInfo::default(); + lookup_env( + &mut si, + &[ + serve::SERVE_ARG_ENV, + serve::FIDELITY_ARG_ENV, + serve::CACHE_ARG_ENV, + ], + false, + ); + let m = trim(get(&cfg, "MODEL")); + if !m.is_empty() { + si.model = r.resolve(m); + } + if si.model.is_empty() { + return (0, "modèle introuvable".into()); + } + serve::run::probe_cache_ram(&serve::run::Env::default(), &cfg, &extra, &mut si); + serve::cache_ram_auto(&cfg, &extra, &si) +} + +/// `sideSlotPreview` : (accepté, Mio, pourquoi). +pub fn side_slot_preview(r: &Resolver, content: &str) -> (bool, i64, String) { + let cfg = parse_env(content); + if !serve::side_slot_on(&cfg) { + return (false, 0, String::new()); + } + if serve::is_external(&cfg) { + return (false, 0, "preset externe".into()); + } + let extra = crate::models::split_args(get(&cfg, "EXTRA_ARGS")); + let mut si = SysInfo { + help: "--no-kv-unified".into(), + ..SysInfo::default() + }; + lookup_env( + &mut si, + &[ + serve::SERVE_ARG_ENV, + serve::FIDELITY_ARG_ENV, + serve::CACHE_ARG_ENV, + serve::SIDE_SLOT_ARG_ENV, + ], + true, + ); + let m = trim(get(&cfg, "MODEL")); + if !m.is_empty() { + si.model = r.resolve(m); + } + if si.model.is_empty() { + return (false, 0, "modèle introuvable".into()); + } + serve::run::probe_cache_ram(&serve::run::Env::default(), &cfg, &extra, &mut si); + let (ctx, mib, why) = serve::side_slot_plan(&cfg, &extra, &si); + (ctx > 0, mib, why) +} + +/// `moePreview` : (notes, avis de chargement). +pub fn moe_preview(r: &Resolver, live_bin: &str, content: &str) -> (Vec, String) { + let cfg = parse_env(content); + if serve::is_external(&cfg) { + return (Vec::new(), String::new()); + } + let mut si = SysInfo { + help: PREVIEW_SERVE_HELP.into(), + ..SysInfo::default() + }; + lookup_env( + &mut si, + &[ + serve::SERVE_ARG_ENV, + serve::FIT_ARG_ENV, + serve::LOAD_ARG_ENV, + ], + true, + ); + let m = trim(get(&cfg, "MODEL")); + if !m.is_empty() { + si.model = r.resolve(m); + } + if si.model.is_empty() { + return (Vec::new(), String::new()); + } + let extra = crate::models::split_args(get(&cfg, "EXTRA_ARGS")); + serve::run::probe_cache_ram(&serve::run::Env::default(), &cfg, &extra, &mut si); + let (norm, _) = serve::normalize_load_flags(&extra, true); + if serve::load_guard_needs_devices(&cfg, &norm, &si) { + let ep = serve::run::EnginePaths { + home: r.home.clone(), + }; + let bin = ep.prebuilt_resolve_bin(serve_first(&[get(&cfg, "BIN"), live_bin])); + let key = format!("{bin}\0{}", get(&cfg, "CUDA_VISIBLE_DEVICES")); + if let Some(devs) = dev_persist_get(&r.home, &key) { + si.engine_vram_mib = serve::engine_devs_vram(&devs); + } + } + let (mut warn, refuse) = serve::load_mode_risk(&cfg, &norm, &si); + if !refuse.is_empty() { + warn = format!("refusé au lancement — {refuse}"); + } + (serve::moe_notes(&cfg, &extra, &si), warn) +} + +/// `devPersistGet` : la dernière liste de cartes donnée à l'éditeur +/// (`devices.json`). +fn dev_persist_get(home: &Path, key: &str) -> Option> { + let b = std::fs::read(home.join("devices.json")).ok()?; + let m: HashMap> = serde_json::from_slice(&b).ok()?; + let list = m.get(key).filter(|l| !l.is_empty())?; + Some( + list.iter() + .map(|d| serve::SplitDev { + id: d.get("id").and_then(Value::as_str).unwrap_or("").into(), + total_mib: d.get("total_mib").and_then(Value::as_f64).unwrap_or(0.0) as i64, + }) + .collect(), + ) +} + +// --------------------------------------------------------------------------- +// Routes +// --------------------------------------------------------------------------- + +fn fail(code: StatusCode, e: impl Into) -> Response { + send_json(code, &json!({ "ok": false, "error": e.into() })) +} + +async fn read_body(body: Incoming, limit: usize) -> Result, String> { + Limited::new(body, limit) + .collect() + .await + .map(|b| b.to_bytes().to_vec()) + .map_err(|_| "http: request body too large".to_string()) +} + +const PRESET_SAVE_REQ: &[(&str, Kind)] = &[ + ("id", Kind::Str), + ("name", Kind::Str), + ("content", Kind::Str), + ("deleteModel", Kind::Bool), + ("sysprompt", Kind::OptStr), +]; + +const EXTERNAL_SAVE_REQ: &[(&str, Kind)] = &[ + ("id", Kind::Str), + ("name", Kind::Str), + ("url", Kind::Str), + ("model", Kind::Str), + ("key", Kind::Str), + ("ctx", Kind::Str), + ("keyTouched", Kind::Bool), + ("vision", Kind::Bool), +]; + +const CONTENT_REQ_TYPE: &str = r#"struct { Content string "json:\"content\"" }"#; +const ORDER_REQ_TYPE: &str = r#"struct { IDs []string "json:\"ids\"" }"#; +const AUTOPLACE_REQ_TYPE: &str = r#"struct { ID string "json:\"id\""; Name string "json:\"name\""; Content string "json:\"content\"" }"#; +const SWITCH_REQ_TYPE: &str = r#"struct { N int "json:\"n\""; ID string "json:\"id\"" }"#; + +/// Le verrou de l'optimiseur (au Go) : `Some(réponse 409)` s'il tient le +/// moteur. +async fn tune_deny(store: &Store) -> Option> { + let v = store + .internal_json(Method::GET, "/internal/v1/tune/guard", None) + .await + .ok()?; + if v.get("busy").and_then(Value::as_bool) == Some(true) { + return Some(send_json( + StatusCode::CONFLICT, + &json!({ "ok": false, "tuning": true, "error": v.get("error").and_then(Value::as_str).unwrap_or("") }), + )); + } + None +} + +/// Applique un preset à la configuration vivante (au Go, voir +/// `facade_presets.go`). +async fn preset_apply(store: &Store, path: &Path, mode: &str) -> Result<(), String> { + let (code, v) = store + .internal_post( + "/internal/v1/preset/apply", + &json!({ "path": path.to_string_lossy(), "mode": mode }), + ) + .await?; + if code.is_success() { + return Ok(()); + } + Err(v + .get("error") + .and_then(Value::as_str) + .map(str::to_string) + .unwrap_or_else(|| format!("application du preset : {code}"))) +} + +struct Live { + cfg: Cfg, + order: Vec, +} + +async fn live(store: &Store) -> Live { + Live { + cfg: store.config().await, + order: parse_preset_order( + store + .get(BK_STATE, PRESET_ORDER_KEY) + .await + .ok() + .flatten() + .as_deref(), + ), + } +} + +/// `SavePresetApplying` : enregistre ; si c'était le preset EN SERVICE (avant +/// l'écriture), l'applique (`mode` : save ou apply). +async fn save_applying( + ctx: &Ctx, + id: &str, + name: &str, + content: &str, + mode: &str, +) -> Result<(String, bool), String> { + let home = &ctx.engine.home; + let mut was_active = false; + if !id.is_empty() + && let Ok(old) = read_preset(home, id) + { + was_active = preset_fingerprint(&old) == config_fingerprint(&ctx.store.config().await); + } + let new_id = save_preset(home, id, name, content)?; + if !was_active { + return Ok((new_id, false)); + } + let p = safe_preset_path(home, &new_id)?; + preset_apply(&ctx.store, &p, mode).await?; + Ok((new_id, true)) +} + +async fn resolver(ctx: &Ctx) -> Resolver { + let d = crate::models::model_dirs(ctx).await; + Resolver { + home: d.home, + stored: d.stored, + env: d.env, + } +} + +/// `from_go` : remise par le Go (tunnel), déjà authentifiée par lui. +pub async fn serve(ctx: &Ctx, req: Request, from_go: bool) -> Response { + let (parts, body) = req.into_parts(); + if !from_go && let Err(refus) = ctx.guard(&parts.headers).await { + return *refus; + } + let store = &ctx.store; + let home = ctx.engine.home.clone(); + let mut query: HashMap = HashMap::new(); + for (k, v) in url::form_urlencoded::parse(parts.uri.query().unwrap_or("").as_bytes()) { + query + .entry(k.into_owned()) + .or_insert_with(|| v.into_owned()); + } + let q = |k: &str| trim(query.get(k).map_or("", String::as_str)).to_string(); + match parts.uri.path() { + "/api/presets" => { + let lv = live(store).await; + let list = match list_presets(&home, &lv.cfg, &lv.order) { + Ok(l) => l, + Err(e) => { + return send_json(StatusCode::INTERNAL_SERVER_ERROR, &json!({ "error": e })); + } + }; + let benches: HashMap = store + .get(BK_STATE, BENCH_KEY) + .await + .ok() + .flatten() + .and_then(|b| serde_json::from_slice(&b).ok()) + .unwrap_or_default(); + let mut out = Vec::new(); + for p in list { + let mut item = serde_json::Map::new(); + item.insert("id".into(), json!(p.id)); + item.insert("name".into(), json!(p.name)); + item.insert("active".into(), json!(p.active)); + let content = read_preset(&home, &p.id).unwrap_or_default(); + let cfg = parse_env(&content); + if !content.is_empty() { + if serve::is_external(&cfg) { + item.insert("external".into(), json!(true)); + let m = trim(get(&cfg, "EXTERNAL_MODEL")); + if !m.is_empty() { + item.insert("model".into(), json!(m)); + } + if trim(get(&cfg, "EXTERNAL_VISION")) == "1" { + item.insert("vision".into(), json!(true)); + } + out.push(Value::Object(item)); + continue; + } + let c = preset_ctx(&content); + if !c.is_empty() { + item.insert("ctx".into(), json!(c)); + } + if !trim(get(&cfg, "MMPROJ")).is_empty() { + item.insert("vision".into(), json!(true)); + } + if serve::is_strata(&cfg) { + item.insert("strata".into(), json!(true)); + if get(&cfg, "STRATA_VISION") != "0" { + item.insert("vision".into(), json!(true)); + } + } + let qn = detect_quant(&content); + if !qn.is_empty() { + item.insert("quant".into(), json!(qn)); + } + let r = preset_reasoning(&content); + if crate::chat::wire::reasoning_active(&r) { + item.insert("reasoning".into(), json!(r.to_lowercase())); + } + } + if let Some(sb) = benches.get(&p.id) + && bench_matches_preset(sb, &cfg, &lv.cfg) + { + item.insert("bench".into(), bench_badge(sb)); + } + out.push(Value::Object(item)); + } + send_json(StatusCode::OK, &Value::Array(out)) + } + "/api/presets/order" => { + if parts.method != Method::POST { + return fail(StatusCode::METHOD_NOT_ALLOWED, "POST attendu"); + } + let b = read_body(body, 8 << 20).await.unwrap_or_default(); + let d = gojson::decode_struct(&b, ORDER_REQ_TYPE, &[("ids", Kind::StrList)]); + if let Some(e) = d.err { + return fail(StatusCode::BAD_REQUEST, e); + } + let ids = d.str_list("ids"); + if let Err(e) = store + .put( + BK_STATE, + PRESET_ORDER_KEY, + gojson::marshal(&ids).into_bytes(), + ) + .await + { + return fail(StatusCode::INTERNAL_SERVER_ERROR, e); + } + send_json(StatusCode::OK, &json!({ "ok": true })) + } + "/api/preset" => { + let id = q("id"); + if id.is_empty() { + let cur = store.config().await; + return send_json( + StatusCode::OK, + &json!({ "id": "", "name": "", "content": format_env(&new_preset_seed(&home, &cur)) }), + ); + } + match read_preset(&home, &id) { + Ok(content) => { + let sp = store + .get_str(BK_STATE, &format!("{SYSPROMPT_PREFIX}{id}")) + .await; + send_json( + StatusCode::OK, + &json!({ "id": id, "name": preset_display_name(&content, &id), "content": content, "sysprompt": sp }), + ) + } + Err(_) => send_json(StatusCode::NOT_FOUND, &json!({ "error": "not found" })), + } + } + "/api/preset/save" => { + if let Some(r) = tune_deny(store).await { + return r; + } + let b = read_body(body, 8 << 20).await.unwrap_or_default(); + let d = gojson::decode_struct(&b, "presetSaveReq", PRESET_SAVE_REQ); + if let Some(e) = d.err.clone() { + return fail(StatusCode::BAD_REQUEST, e); + } + let (new_id, applied) = + match save_applying(ctx, &d.str("id"), &d.str("name"), &d.str("content"), "save") + .await + { + Ok(x) => x, + Err(e) => return fail(StatusCode::BAD_REQUEST, e), + }; + if let Some(sp) = d.opt_str("sysprompt") { + let key = format!("{SYSPROMPT_PREFIX}{new_id}"); + let res = if trim(&sp).is_empty() { + store.del(BK_STATE, &key).await + } else { + store + .put(BK_STATE, &key, trim(&sp).as_bytes().to_vec()) + .await + }; + if let Err(e) = res { + return fail(StatusCode::INTERNAL_SERVER_ERROR, e); + } + } + send_json( + StatusCode::OK, + &json!({ "ok": true, "id": new_id, "name": d.str("name"), "applied": applied }), + ) + } + "/api/preset/cacheram" => { + let b = match read_body(body, 1 << 20).await { + Ok(b) => b, + Err(e) => return fail(StatusCode::BAD_REQUEST, e), + }; + let d = gojson::decode_struct(&b, CONTENT_REQ_TYPE, &[("content", Kind::Str)]); + if let Some(e) = d.err.clone() { + return fail(StatusCode::BAD_REQUEST, e); + } + let r = resolver(ctx).await; + let content = d.str("content"); + let res = tokio::task::spawn_blocking(move || { + ( + cache_ram_preview(&r, &content), + side_slot_preview(&r, &content), + ) + }) + .await; + let Ok(((mib, why), (on, extra, swhy))) = res else { + return fail(StatusCode::INTERNAL_SERVER_ERROR, "aperçu interrompu"); + }; + let mut out = serde_json::Map::new(); + out.insert("ok".into(), json!(true)); + out.insert("mib".into(), json!(mib)); + out.insert("default".into(), json!(serve::ENGINE_CACHE_RAM_DEFAULT)); + out.insert("why".into(), json!(why)); + if on || !swhy.is_empty() { + out.insert( + "side".into(), + json!({ "on": on, "vramMiB": extra, "why": swhy }), + ); + } + send_json(StatusCode::OK, &Value::Object(out)) + } + "/api/preset/moe" => { + let b = match read_body(body, 1 << 20).await { + Ok(b) => b, + Err(e) => return fail(StatusCode::BAD_REQUEST, e), + }; + let d = gojson::decode_struct(&b, CONTENT_REQ_TYPE, &[("content", Kind::Str)]); + if let Some(e) = d.err.clone() { + return fail(StatusCode::BAD_REQUEST, e); + } + let r = resolver(ctx).await; + let live_bin = get(&store.config().await, "BIN").to_string(); + let content = d.str("content"); + let c2 = content.clone(); + let Ok((notes, load)) = + tokio::task::spawn_blocking(move || moe_preview(&r, &live_bin, &c2)).await + else { + return fail(StatusCode::INTERNAL_SERVER_ERROR, "aperçu interrompu"); + }; + let mut out = serde_json::Map::new(); + out.insert("ok".into(), json!(true)); + out.insert( + "notes".into(), + if notes.is_empty() { + Value::Null + } else { + json!(notes) + }, + ); + out.insert("load".into(), json!(load)); + let cfg = parse_env(&content); + if !serve::is_external(&cfg) + && !serve::cpu_experts( + &crate::models::split_args(get(&cfg, "EXTRA_ARGS")), + &HashMap::new(), + ) + .is_empty() + { + out.insert( + "autoplace".into(), + match auto_placement_preset(&content) { + Ok((_, changes)) => json!({ "ok": true, "changes": changes }), + Err(e) => json!({ "ok": false, "why": e }), + }, + ); + } + send_json(StatusCode::OK, &Value::Object(out)) + } + "/api/preset/autoplace" => { + if parts.method != Method::POST { + let mut r = fail( + StatusCode::METHOD_NOT_ALLOWED, + format!("méthode {} refusée : POST attendu", parts.method), + ); + r.headers_mut().insert( + hyper::header::ALLOW, + hyper::header::HeaderValue::from_static("POST"), + ); + return r; + } + let b = match read_body(body, 1 << 20).await { + Ok(b) => b, + Err(e) => return fail(StatusCode::BAD_REQUEST, e), + }; + let d = gojson::decode_struct( + &b, + AUTOPLACE_REQ_TYPE, + &[ + ("id", Kind::Str), + ("name", Kind::Str), + ("content", Kind::Str), + ], + ); + if let Some(e) = d.err.clone() { + return fail(StatusCode::BAD_REQUEST, e); + } + let content = d.str("content"); + let (out, changes) = match auto_placement_preset(&content) { + Ok(x) => x, + Err(e) => return send_json(StatusCode::OK, &json!({ "ok": false, "error": e })), + }; + let shown = preset_display_name(&content, &d.str("id")); + let name = format!( + "{} (placement auto)", + serve_first(&[trim(&d.str("name")), &shown, "preset"]) + ); + match save_preset(&home, "", &name, &out) { + Ok(id) => send_json( + StatusCode::OK, + &json!({ "ok": true, "id": id, "name": name, "changes": changes }), + ), + Err(e) => send_json(StatusCode::OK, &json!({ "ok": false, "error": e })), + } + } + "/api/preset/delete" => { + let b = read_body(body, 8 << 20).await.unwrap_or_default(); + let d = gojson::decode_struct(&b, "presetSaveReq", PRESET_SAVE_REQ); + if let Some(e) = d.err.clone() { + return fail(StatusCode::BAD_REQUEST, e); + } + let id = d.str("id"); + let mut model = String::new(); + if d.bool("deleteModel") + && let Ok(c) = read_preset(&home, &id) + { + model = model_from_preset_content(&c); + } + if let Err(e) = delete_preset(ctx, &id).await { + return fail(StatusCode::BAD_REQUEST, e); + } + let (mut model_deleted, mut model_err, mut freed) = + (String::new(), String::new(), 0i64); + if d.bool("deleteModel") && !model.is_empty() { + let dirs = crate::models::model_dirs(ctx).await; + let lv = live(store).await; + let m2 = model.clone(); + let res = tokio::task::spawn_blocking(move || { + let refs = crate::models::build_refs(&dirs, &lv.cfg, &lv.order); + let (others, _) = refs.users(&dirs, &m2); + (others, crate::models::delete_model_file(&dirs, &refs, &m2)) + }) + .await; + if let Ok((others, r)) = res { + match r { + Err(e) => model_err = e, + Ok(n) => { + model_deleted = model; + freed = n; + if !others.is_empty() { + model_err = format!( + "supprimé, mais il était aussi référencé par : {}", + others.join(", ") + ); + } + } + } + } + } + send_json( + StatusCode::OK, + &json!({ "ok": true, "modelDeleted": model_deleted, "modelError": model_err, "freed": freed }), + ) + } + "/api/preset/external" => { + let id = q("id"); + if id.is_empty() { + return send_json( + StatusCode::OK, + &json!({ "id": "", "name": "", "url": "", "model": "", "hasKey": false }), + ); + } + let Ok(content) = read_preset(&home, &id) else { + return send_json(StatusCode::NOT_FOUND, &json!({ "error": "not found" })); + }; + let cfg = parse_env(&content); + if !serve::is_external(&cfg) { + return send_json( + StatusCode::BAD_REQUEST, + &json!({ "error": "pas un preset externe" }), + ); + } + send_json( + StatusCode::OK, + &json!({ + "id": id, + "name": preset_display_name(&content, &id), + "url": trim(get(&cfg, "EXTERNAL_URL")), + "model": trim(get(&cfg, "EXTERNAL_MODEL")), + "ctx": trim(get(&cfg, "CTX")), + "hasKey": !trim(get(&cfg, "EXTERNAL_KEY")).is_empty(), + "vision": trim(get(&cfg, "EXTERNAL_VISION")) == "1", + }), + ) + } + "/api/preset/external/save" => { + let b = read_body(body, 8 << 20).await.unwrap_or_default(); + let d = gojson::decode_struct(&b, "externalSaveReq", EXTERNAL_SAVE_REQ); + if let Some(e) = d.err.clone() { + return fail(StatusCode::BAD_REQUEST, e); + } + let name = trim(&d.str("name")).to_string(); + let url = trim(&d.str("url")).to_string(); + let model = trim(&d.str("model")).to_string(); + if name.is_empty() { + return fail(StatusCode::BAD_REQUEST, "nom requis"); + } + if url.is_empty() { + return fail(StatusCode::BAD_REQUEST, "URL requise"); + } + if model.is_empty() { + return fail(StatusCode::BAD_REQUEST, "modèle requis"); + } + let id = d.str("id"); + let mut key = d.str("key"); + if !id.is_empty() + && !d.bool("keyTouched") + && let Ok(c) = read_preset(&home, &id) + { + key = trim(get(&parse_env(&c), "EXTERNAL_KEY")).to_string(); + } + let content = + external_preset_content(&url, &model, &key, &d.str("ctx"), d.bool("vision")); + match save_applying(ctx, &id, &name, &content, "apply").await { + Ok((new_id, applied)) => send_json( + StatusCode::OK, + &json!({ "ok": true, "id": new_id, "name": name, "applied": applied }), + ), + Err(e) => fail(StatusCode::BAD_REQUEST, e), + } + } + "/api/preset/external/test" => { + let b = read_body(body, 8 << 20).await.unwrap_or_default(); + let d = gojson::decode_struct(&b, "externalSaveReq", EXTERNAL_SAVE_REQ); + if let Some(e) = d.err.clone() { + return fail(StatusCode::BAD_REQUEST, e); + } + let url = completions_url(&d.str("url")); + let model = trim(&d.str("model")).to_string(); + if url.is_empty() || model.is_empty() { + return send_json( + StatusCode::OK, + &json!({ "ok": false, "error": "URL et modèle requis" }), + ); + } + let mut key = trim(&d.str("key")).to_string(); + let id = d.str("id"); + if key.is_empty() + && !id.is_empty() + && !d.bool("keyTouched") + && let Ok(c) = read_preset(&home, &id) + { + key = trim(get(&parse_env(&c), "EXTERNAL_KEY")).to_string(); + } + send_json(StatusCode::OK, &external_test(&url, &model, &key).await) + } + "/api/switch" => { + if let Some(r) = tune_deny(store).await { + return r; + } + let b = read_body(body, 8 << 20).await.unwrap_or_default(); + let d = + gojson::decode_struct(&b, SWITCH_REQ_TYPE, &[("n", Kind::Int), ("id", Kind::Str)]); + if let Some(e) = d.err.clone() { + return fail(StatusCode::BAD_REQUEST, e); + } + let lv = live(store).await; + let list = match list_presets(&home, &lv.cfg, &lv.order) { + Ok(l) => l, + Err(e) => return fail(StatusCode::INTERNAL_SERVER_ERROR, e), + }; + let id = trim(&d.str("id")).to_string(); + let target = if !id.is_empty() { + match list.into_iter().find(|p| p.id == id) { + Some(p) => p, + None => { + return fail( + StatusCode::NOT_FOUND, + "preset introuvable (supprimé ou renommé ?)", + ); + } + } + } else { + let n = d.int("n"); + if n < 1 || n as usize > list.len() { + return fail(StatusCode::BAD_REQUEST, "index hors limites"); + } + list.into_iter().nth(n as usize - 1).expect("borné") + }; + if let Err(e) = preset_apply(store, &target.path, "switch").await { + return fail(StatusCode::INTERNAL_SERVER_ERROR, e); + } + send_json( + StatusCode::OK, + &json!({ "ok": true, "preset": target.name }), + ) + } + _ => fail(StatusCode::NOT_FOUND, "route inconnue"), + } +} + +/// `DeletePreset` : refusé pour le preset actif ; emporte son prompt système +/// et sa mesure. +async fn delete_preset(ctx: &Ctx, id: &str) -> Result<(), String> { + let home = &ctx.engine.home; + let p = safe_preset_path(home, id)?; + let target = std::fs::read(&p).map_err(|_| "introuvable".to_string())?; + if preset_fingerprint(&String::from_utf8_lossy(&target)) + == config_fingerprint(&ctx.store.config().await) + { + return Err("preset actif, switche d'abord".into()); + } + std::fs::remove_file(&p) + .map_err(|e| crate::paths::os_err("remove", &p.to_string_lossy(), &e))?; + let _ = ctx + .store + .del(BK_STATE, &format!("{SYSPROMPT_PREFIX}{id}")) + .await; + if let Ok(Some(b)) = ctx.store.get(BK_STATE, BENCH_KEY).await + && let Ok(mut m) = serde_json::from_slice::>(&b) + && m.remove(id).is_some() + { + let _ = ctx + .store + .put(BK_STATE, BENCH_KEY, gojson::marshal(&m).into_bytes()) + .await; + } + Ok(()) +} + +/// `handlePresetExternalTest` : un appel de complétion minimal. +pub async fn external_test(url: &str, model: &str, key: &str) -> Value { + crate::chat::engine::install_crypto(); + let client = reqwest::Client::builder() + .timeout(std::time::Duration::from_secs(20)) + .build() + .unwrap_or_default(); + let payload = json!({ + "max_tokens": 1, + "messages": [{ "content": "ping", "role": "user" }], + "model": model, + "stream": false, + }); + let mut rq = client + .post(url) + .header("Content-Type", "application/json") + .body(serde_json::to_vec(&payload).unwrap_or_default()); + if !key.is_empty() { + rq = rq.header("Authorization", format!("Bearer {key}")); + } + let resp = match rq.send().await { + Ok(r) => r, + Err(e) => return json!({ "ok": false, "error": format!("injoignable : {e}") }), + }; + let status = resp.status(); + let mut body = Vec::new(); + let mut stream = resp; + while body.len() < 2000 { + match stream.chunk().await { + Ok(Some(c)) => body.extend_from_slice(&c), + _ => break, + } + } + body.truncate(2000); + if status != reqwest::StatusCode::OK { + let mut msg = trim(&String::from_utf8_lossy(&body)).to_string(); + let m = extract_api_error(&body); + if !m.is_empty() { + msg = m; + } + if msg.is_empty() { + msg = format!( + "{} {}", + status.as_u16(), + status.canonical_reason().unwrap_or("") + ); + } + return json!({ "ok": false, "status": status.as_u16(), "error": msg }); + } + json!({ "ok": true }) +} diff --git a/rust/loki-rs/src/proxy.rs b/rust/loki-rs/src/proxy.rs index 71e77f4..dde5fee 100644 --- a/rust/loki-rs/src/proxy.rs +++ b/rust/loki-rs/src/proxy.rs @@ -77,6 +77,8 @@ pub struct Config { pub push: Option>, /// Mémoire (pages, trackers, mode) tenue par la façade (`LOKI_MEM=rust`). pub mem: Option>, + /// Presets tenus par la façade (`LOKI_PRESETS=rust`). + pub presets: bool, } struct Proxy { @@ -110,6 +112,7 @@ pub async fn serve( files: cfg.files, push: cfg.push, mem: cfg.mem, + presets: cfg.presets, internal_token: cfg.internal_token.clone(), }, }); @@ -197,6 +200,9 @@ impl Proxy { { return chatapi::serve(&self.ctx, chat, req, from_go).await; } + if self.ctx.presets && crate::presets_api::handles(path) { + return crate::presets_api::serve(&self.ctx, req, from_go).await; + } if let Some(m) = &self.ctx.mem && crate::memory::handles(path) { diff --git a/rust/loki-rs/src/serve/run.rs b/rust/loki-rs/src/serve/run.rs index 800f95c..ed73ac3 100644 --- a/rust/loki-rs/src/serve/run.rs +++ b/rust/loki-rs/src/serve/run.rs @@ -384,7 +384,7 @@ fn probe_serve_gpus(env: &Env, cfg: &Cfg, extra: &[String], si: &mut SysInfo) { } /// `probeCacheRAM`. -fn probe_cache_ram(env: &Env, cfg: &Cfg, extra: &[String], si: &mut SysInfo) { +pub(crate) fn probe_cache_ram(env: &Env, cfg: &Cfg, extra: &[String], si: &mut SysInfo) { probe_env(env, si, CACHE_ARG_ENV, true); if !si.model.is_empty() { let mp = Path::new(&si.model); diff --git a/rust/loki-rs/tests/common/fakego.rs b/rust/loki-rs/tests/common/fakego.rs index 6f133d4..cc13f81 100644 --- a/rust/loki-rs/tests/common/fakego.rs +++ b/rust/loki-rs/tests/common/fakego.rs @@ -74,6 +74,10 @@ pub struct FakeGo { pub turn_view: Mutex>, pub turn_verifier: Mutex>, pub tool_results: Mutex>, + /// /internal/v1/tune/guard : message d'une optimisation en cours. + pub tune_busy: Mutex>, + /// /internal/v1/preset/apply reçus : (chemin, mode). + pub preset_applies: Mutex>, /// Comportement de /internal/v1/crypt/* (préfixe « SCELLE: » par défaut). pub crypt_mode: Mutex, /// Réponse de /internal/v1/conv/workspace (None : 404). @@ -325,6 +329,24 @@ async fn handle(st: Arc, req: Request) -> Response *st.go_busy.lock().unwrap() = None; resp(204, "") } + "/internal/v1/tune/guard" => match st.tune_busy.lock().unwrap().clone() { + Some(e) => resp( + 200, + serde_json::json!({ "busy": true, "error": e }).to_string(), + ), + None => resp(200, r#"{"busy":false}"#), + }, + "/internal/v1/preset/apply" => { + let body = req.into_body().collect().await.unwrap().to_bytes(); + let v: serde_json::Value = serde_json::from_slice(&body).unwrap_or_default(); + let path = v["path"].as_str().unwrap_or("").to_string(); + let mode = v["mode"].as_str().unwrap_or("").to_string(); + if mode == "refus" || path.ends_with("refus.env") { + return resp(500, r#"{"error":"application refusée"}"#); + } + st.preset_applies.lock().unwrap().push((path, mode)); + resp(204, "") + } "/internal/v1/conv/workspace" => match st.conv_workspace.lock().unwrap().clone() { Some(v) => resp(200, v.to_string()), None => resp(404, "404 page not found"), @@ -507,6 +529,7 @@ pub struct Opts { pub files: Option>, pub push: Option>, pub mem: Option>, + pub presets: bool, } /// Comme [`start`], avec la conversation tenue par la façade @@ -602,6 +625,7 @@ pub async fn start_with( files: opts.files, push: opts.push, mem: opts.mem, + presets: opts.presets, }; tokio::spawn(loki_rs::proxy::serve(fl, cfg, async move { let _ = rx.await; diff --git a/rust/loki-rs/tests/lot2.rs b/rust/loki-rs/tests/lot2.rs index 166e75e..5c8b14b 100644 --- a/rust/loki-rs/tests/lot2.rs +++ b/rust/loki-rs/tests/lot2.rs @@ -100,6 +100,7 @@ async fn banc(cle: Option<&str>) -> Banc { files: None, push: None, mem: None, + presets: false, }; tokio::spawn(loki_rs::proxy::serve(l, cfg, async move { let _ = rx.await; diff --git a/rust/loki-rs/tests/parity_presets/presets.json b/rust/loki-rs/tests/parity_presets/presets.json new file mode 100644 index 0000000..b62b1cb --- /dev/null +++ b/rust/loki-rs/tests/parity_presets/presets.json @@ -0,0 +1,2206 @@ +{ + "display_name": [ + { + "content": "MODEL=a.gguf\n", + "name": "Alpha", + "out": "# NAME=Alpha\nMODEL=a.gguf\n" + }, + { + "content": "# NAME=Vieux\nMODEL=a\n", + "name": "Neuf", + "out": "# NAME=Neuf\nMODEL=a\n" + }, + { + "content": " name = x\nCTX=1\n", + "name": "Y", + "out": "# NAME=Y\nCTX=1\n" + }, + { + "content": "#NAME=a\n# name=b\n", + "name": "Deux", + "out": "# NAME=Deux\n# NAME=Deux\n" + }, + { + "content": "MODEL=a\n# NAME=x", + "name": "Prix $1 et ${x}", + "out": "MODEL=a\n# NAME=Prix et " + }, + { + "content": "", + "name": "Vide", + "out": "# NAME=Vide\n" + } + ], + "quote": { + "": "", + "\"y\"": "\"y\"", + "'x'": "\"'x'\"", + "'z": "'z", + "a\tb": "\"a\tb\"", + "a b": "\"a b\"", + "a\"b": "a\"b", + "simple": "simple" + }, + "format_env": "A=\"deux mots\"\nM=\nQ=q\"q\nZ=1\n", + "names": { + "": "nom vide", + " ": "nom vide", + " pad ": "ok", + ".": "nom réservé", + "..": "nom réservé", + "Qwen 3 (27B) é": "ok", + "a\u0000b": "le nom ne peut pas contenir / ni \\", + "a\u0001b": "le nom contient un caractère de contrôle invalide", + "a/b": "le nom ne peut pas contenir / ni \\", + "a\\b": "le nom ne peut pas contenir / ni \\" + }, + "quant": { + "": "", + "# QUANT=iq3_xs\nMODEL=x-Q8_0.gguf\n": "IQ3_XS", + "MODEL = C:\\models\\w-BF16.gguf\n": "BF16", + "MODEL=\"/m/Llama-3-UD-Q6_K_XL-00001-of-00002.gguf\"\nREASONING=On\n": "Q6_K_XL", + "MODEL=Qwen3-27B-Q4_K_M.gguf\n": "Q4_K_M", + "QUANT='q5'\n": "Q5", + "export MODEL=z.gguf\n": "", + "reasoning = 'off'\n#REASONING=on\n": "" + }, + "reasoning": { + "": "", + "# QUANT=iq3_xs\nMODEL=x-Q8_0.gguf\n": "", + "MODEL = C:\\models\\w-BF16.gguf\n": "", + "MODEL=\"/m/Llama-3-UD-Q6_K_XL-00001-of-00002.gguf\"\nREASONING=On\n": "On", + "MODEL=Qwen3-27B-Q4_K_M.gguf\n": "", + "QUANT='q5'\n": "", + "export MODEL=z.gguf\n": "", + "reasoning = 'off'\n#REASONING=on\n": "off" + }, + "model": { + "": "", + "# QUANT=iq3_xs\nMODEL=x-Q8_0.gguf\n": "x-Q8_0.gguf", + "MODEL = C:\\models\\w-BF16.gguf\n": "C:\\models\\w-BF16.gguf", + "MODEL=\"/m/Llama-3-UD-Q6_K_XL-00001-of-00002.gguf\"\nREASONING=On\n": "/m/Llama-3-UD-Q6_K_XL-00001-of-00002.gguf", + "MODEL=Qwen3-27B-Q4_K_M.gguf\n": "Qwen3-27B-Q4_K_M.gguf", + "QUANT='q5'\n": "", + "export MODEL=z.gguf\n": "", + "reasoning = 'off'\n#REASONING=on\n": "" + }, + "completions": { + "": "", + " https://api.groq.com/openai/v1/ ": "https://api.groq.com/openai/v1/chat/completions", + "http://loki:8090": "http://loki:8090/v1/chat/completions", + "http://x/chat/completions": "http://x/chat/completions", + "https://o.ai/api/v1": "https://o.ai/api/v1/chat/completions" + }, + "external": [ + { + "ctx": "", + "key": "", + "model": "gpt", + "out": "EXTERNAL=1\nEXTERNAL_MODEL=gpt\nEXTERNAL_URL=https://a/v1\n", + "url": "https://a/v1", + "vision": false + }, + { + "ctx": " 128000 ", + "key": " k ", + "model": " m ", + "out": "CTX=128000\nEXTERNAL=1\nEXTERNAL_KEY=k\nEXTERNAL_MODEL=m\nEXTERNAL_URL=u\nEXTERNAL_VISION=1\n", + "url": " u ", + "vision": true + } + ], + "api_error": { + "pas du json": "", + "{\"error\":\"x\"}": "", + "{\"error\":{\"message\":\" clé invalide \"}}": "clé invalide", + "{}": "" + }, + "join": [ + { + "args": [ + "-ngl", + "30" + ], + "out": "-ngl 30" + }, + { + "args": [ + "--chat-template-file", + "/a b/t.jinja" + ], + "out": "--chat-template-file '/a b/t.jinja'" + }, + { + "args": [ + "x", + "l'apostrophe ici" + ], + "out": "x \"l'apostrophe ici\"" + }, + { + "args": [ + "" + ], + "out": "\"\"" + } + ], + "set_key": [ + { + "content": "A=1\nB=2\n", + "key": "B", + "out": "A=1\nB=3\n", + "val": "3" + }, + { + "content": "A=1\nexport B=2\nB=4\n", + "key": "B", + "out": "A=1\n", + "val": "" + }, + { + "content": "A=1", + "key": "C", + "out": "A=1\nC=\"deux mots\"\n", + "val": "deux mots" + }, + { + "content": "", + "key": "X", + "out": "X=1\n", + "val": "1" + }, + { + "content": "NGL=999\n# NGL=1\n", + "key": "NGL", + "out": "# NGL=1\n", + "val": "" + }, + { + "content": "K.E=1\nKXE=2\n", + "key": "K.E", + "out": "K.E=v\nKXE=2\n", + "val": "v" + } + ], + "autoplace": [ + { + "changes": null, + "err": "--fit resterait inactif (surcharge de tenseurs) : retire-le à la main d'abord", + "in": "# NAME=MoE\nMODEL=m.gguf\nCTX=65536\nNGL=999\nEXTRA_ARGS=-ot 'exps=CPU,per_layer_token_embd=CPU,blk\\.0=CUDA0' --n-cpu-moe 0 -ub 512 --mlock --fit off -ctk q8_0\n", + "out": "" + }, + { + "changes": [ + "retiré : --n-cpu-moe 30", + "retiré : -ts 1,1", + "UBATCH=2048, BATCH=4096 (avant : 512 / 2048)", + "--load-mode mmap", + "CTX conservé : 32768" + ], + "err": "", + "in": "MODEL=m.gguf\nEXTRA_ARGS=--n-cpu-moe 30 -ts 1,1\n", + "out": "MODEL=m.gguf\nEXTRA_ARGS=\"--load-mode mmap\"\nUBATCH=2048\nBATCH=4096\n" + }, + { + "changes": null, + "err": "--fit resterait inactif (-sm row) : retire-le à la main d'abord", + "in": "MODEL=m.gguf\nEXTRA_ARGS=--cpu-moe -sm row\n", + "out": "" + }, + { + "changes": null, + "err": "contexte « 0 » non chiffré : --fit pourrait le réduire pour tenir — donne un CTX chiffré d'abord", + "in": "MODEL=m.gguf\nEXTRA_ARGS=--cpu-moe\nCTX=0\n", + "out": "" + }, + { + "changes": null, + "err": "SPLIT_MODE=tensor coupe --fit : retire-le d'abord", + "in": "MODEL=m.gguf\nEXTRA_ARGS=--cpu-moe\nSPLIT_MODE=tensor\n", + "out": "" + }, + { + "changes": null, + "err": "rien à migrer : ce preset ne place pas d'experts à la main (-ot …exps…, --n-cpu-moe, --cpu-moe)", + "in": "MODEL=m.gguf\nEXTRA_ARGS=-ngl 40\n", + "out": "" + }, + { + "changes": null, + "err": "preset externe : le modèle tourne ailleurs", + "in": "EXTERNAL=1\n", + "out": "" + }, + { + "changes": [ + "retiré : --cpu-moe", + "retiré : -b 1024", + "UBATCH=2048, BATCH=4096 (avant : 1024 / 2048)", + "--load-mode mmap", + "CTX conservé : 32768", + "cache KV conservé tel quel — garde-le identique pour comparer les deux presets" + ], + "err": "", + "in": "MODEL=m\nEXTRA_ARGS=--cpu-moe --fit=on -b=1024\nKV_TYPE=q4_0\nUBATCH=1024\n", + "out": "MODEL=m\nEXTRA_ARGS=\"--fit on --load-mode mmap\"\nKV_TYPE=q4_0\nUBATCH=2048\nBATCH=4096\n" + } + ], + "bench_fp": [ + { + "cfg": { + "MODEL": "a.gguf" + }, + "cur": {}, + "out": "e3fbbb2b984dad580f7fc2c6085660e1013a2d4d" + }, + { + "cfg": { + "CUDA_VISIBLE_DEVICES": "1", + "MODEL": "a.gguf" + }, + "cur": { + "CUDA_VISIBLE_DEVICES": "0" + }, + "out": "0c52a09078a6809ccde29379f7bf6e01c51a9a11" + }, + { + "cfg": { + "MODEL": "a.gguf" + }, + "cur": { + "CUDA_VISIBLE_DEVICES": " 0,1 " + }, + "out": "63d3caa22a8aadb3db007d11f044263fcfacfcc5" + } + ], + "imposes": { + "": false, + "/app/llama-server": true, + "/opt/custom/llama-server": true, + "{{HOME}}/backends/llama.cpp-prebuilt/llama-b2/llama-server": false, + "{{HOME}}/engine/b1/llama-server": false + }, + "seed": { + "config/BIN": "/opt/custom/llama-server", + "config/HOST": "127.0.0.1", + "config/MODEL": "live.gguf", + "config/PORT": "8081" + }, + "steps": [ + { + "method": "GET", + "path": "/api/presets", + "status": 200, + "out": [], + "disk": {}, + "state": {} + }, + { + "method": "GET", + "path": "/api/preset", + "status": 200, + "out": { + "content": "BIN=/opt/custom/llama-server\nHOST=127.0.0.1\nPORT=8081\n", + "id": "", + "name": "" + }, + "disk": {}, + "state": {} + }, + { + "method": "POST", + "path": "/api/preset/save", + "body": "{\"name\":\"Qwen 27B\",\"content\":\"MODEL=Qwen3-27B-Q4_K_M.gguf\\nCTX=65536\\nREASONING=on\\n\"}", + "status": 200, + "out": { + "applied": false, + "id": "Qwen 27B", + "name": "Qwen 27B", + "ok": true + }, + "disk": { + "presets/Qwen 27B.env": "# NAME=Qwen 27B\nMODEL=Qwen3-27B-Q4_K_M.gguf\nCTX=65536\nREASONING=on\n" + }, + "state": {} + }, + { + "method": "POST", + "path": "/api/preset/save", + "body": "{\"name\":\"Qwen 27B\",\"content\":\"MODEL=Qwen3-27B-Q8_0.gguf\\nMMPROJ=mm.gguf\\n\",\"sysprompt\":\" Réponds en français. \"}", + "status": 200, + "out": { + "applied": false, + "id": "Qwen 27B (2)", + "name": "Qwen 27B", + "ok": true + }, + "disk": { + "presets/Qwen 27B (2).env": "# NAME=Qwen 27B\nMODEL=Qwen3-27B-Q8_0.gguf\nMMPROJ=mm.gguf\n", + "presets/Qwen 27B.env": "# NAME=Qwen 27B\nMODEL=Qwen3-27B-Q4_K_M.gguf\nCTX=65536\nREASONING=on\n" + }, + "state": { + "sysprompt:Qwen 27B (2)": "Réponds en français." + } + }, + { + "method": "POST", + "path": "/api/preset/save", + "body": "{\"name\":\"a/b\\\\c\",\"content\":\"MODEL=x.gguf\\nCTX=40000\\n\"}", + "status": 200, + "out": { + "applied": false, + "id": "a-b-c", + "name": "a/b\\c", + "ok": true + }, + "disk": { + "presets/Qwen 27B (2).env": "# NAME=Qwen 27B\nMODEL=Qwen3-27B-Q8_0.gguf\nMMPROJ=mm.gguf\n", + "presets/Qwen 27B.env": "# NAME=Qwen 27B\nMODEL=Qwen3-27B-Q4_K_M.gguf\nCTX=65536\nREASONING=on\n", + "presets/a-b-c.env": "# NAME=a/b\\c\nMODEL=x.gguf\nCTX=40000\n" + }, + "state": { + "sysprompt:Qwen 27B (2)": "Réponds en français." + } + }, + { + "method": "POST", + "path": "/api/preset/save", + "body": "{\"name\":\" \",\"content\":\"X=1\"}", + "status": 400, + "out": { + "error": "nom requis", + "ok": false + }, + "disk": { + "presets/Qwen 27B (2).env": "# NAME=Qwen 27B\nMODEL=Qwen3-27B-Q8_0.gguf\nMMPROJ=mm.gguf\n", + "presets/Qwen 27B.env": "# NAME=Qwen 27B\nMODEL=Qwen3-27B-Q4_K_M.gguf\nCTX=65536\nREASONING=on\n", + "presets/a-b-c.env": "# NAME=a/b\\c\nMODEL=x.gguf\nCTX=40000\n" + }, + "state": { + "sysprompt:Qwen 27B (2)": "Réponds en français." + } + }, + { + "method": "POST", + "path": "/api/preset/save", + "body": "{\"id\":\"../evil\",\"name\":\"E\",\"content\":\"X=1\"}", + "status": 400, + "out": { + "error": "le nom ne peut pas contenir / ni \\", + "ok": false + }, + "disk": { + "presets/Qwen 27B (2).env": "# NAME=Qwen 27B\nMODEL=Qwen3-27B-Q8_0.gguf\nMMPROJ=mm.gguf\n", + "presets/Qwen 27B.env": "# NAME=Qwen 27B\nMODEL=Qwen3-27B-Q4_K_M.gguf\nCTX=65536\nREASONING=on\n", + "presets/a-b-c.env": "# NAME=a/b\\c\nMODEL=x.gguf\nCTX=40000\n" + }, + "state": { + "sysprompt:Qwen 27B (2)": "Réponds en français." + } + }, + { + "method": "POST", + "path": "/api/preset/save", + "body": "{\"id\":\"Qwen 27B (2)\",\"name\":\"Qwen Q8\",\"content\":\"# NAME=ancien\\nMODEL=Qwen3-27B-Q8_0.gguf\\nQUANT=q8\\n\",\"sysprompt\":\"\"}", + "status": 200, + "out": { + "applied": false, + "id": "Qwen 27B (2)", + "name": "Qwen Q8", + "ok": true + }, + "disk": { + "presets/Qwen 27B (2).env": "# NAME=Qwen Q8\nMODEL=Qwen3-27B-Q8_0.gguf\nQUANT=q8\n", + "presets/Qwen 27B.env": "# NAME=Qwen 27B\nMODEL=Qwen3-27B-Q4_K_M.gguf\nCTX=65536\nREASONING=on\n", + "presets/a-b-c.env": "# NAME=a/b\\c\nMODEL=x.gguf\nCTX=40000\n" + }, + "state": {} + }, + { + "method": "POST", + "path": "/api/preset/save", + "body": "pas du json", + "status": 400, + "out": { + "error": "invalid character 'p' looking for beginning of value", + "ok": false + }, + "disk": { + "presets/Qwen 27B (2).env": "# NAME=Qwen Q8\nMODEL=Qwen3-27B-Q8_0.gguf\nQUANT=q8\n", + "presets/Qwen 27B.env": "# NAME=Qwen 27B\nMODEL=Qwen3-27B-Q4_K_M.gguf\nCTX=65536\nREASONING=on\n", + "presets/a-b-c.env": "# NAME=a/b\\c\nMODEL=x.gguf\nCTX=40000\n" + }, + "state": {} + }, + { + "method": "POST", + "path": "/api/preset/save", + "body": "{\"deleteModel\":\"oui\"}", + "status": 400, + "out": { + "error": "json: cannot unmarshal string into Go struct field presetSaveReq.deleteModel of type bool", + "ok": false + }, + "disk": { + "presets/Qwen 27B (2).env": "# NAME=Qwen Q8\nMODEL=Qwen3-27B-Q8_0.gguf\nQUANT=q8\n", + "presets/Qwen 27B.env": "# NAME=Qwen 27B\nMODEL=Qwen3-27B-Q4_K_M.gguf\nCTX=65536\nREASONING=on\n", + "presets/a-b-c.env": "# NAME=a/b\\c\nMODEL=x.gguf\nCTX=40000\n" + }, + "state": {} + }, + { + "method": "POST", + "path": "/api/preset/save", + "body": "{\"name\":\"Strata\",\"content\":\"ENGINE=strata\\nSTRATA_VISION=0\\nCTX=1048576\\n\"}", + "status": 200, + "out": { + "applied": false, + "id": "Strata", + "name": "Strata", + "ok": true + }, + "disk": { + "presets/Qwen 27B (2).env": "# NAME=Qwen Q8\nMODEL=Qwen3-27B-Q8_0.gguf\nQUANT=q8\n", + "presets/Qwen 27B.env": "# NAME=Qwen 27B\nMODEL=Qwen3-27B-Q4_K_M.gguf\nCTX=65536\nREASONING=on\n", + "presets/Strata.env": "# NAME=Strata\nENGINE=strata\nSTRATA_VISION=0\nCTX=1048576\n", + "presets/a-b-c.env": "# NAME=a/b\\c\nMODEL=x.gguf\nCTX=40000\n" + }, + "state": {} + }, + { + "method": "GET", + "path": "/api/presets", + "status": 200, + "out": [ + { + "active": false, + "ctx": "64K", + "id": "Qwen 27B", + "name": "Qwen 27B", + "quant": "Q4_K_M", + "reasoning": "on" + }, + { + "active": false, + "ctx": "32K", + "id": "Qwen 27B (2)", + "name": "Qwen Q8", + "quant": "Q8" + }, + { + "active": false, + "ctx": "1M", + "id": "Strata", + "name": "Strata", + "strata": true + }, + { + "active": false, + "ctx": "40K", + "id": "a-b-c", + "name": "a/b\\c" + } + ], + "disk": { + "presets/Qwen 27B (2).env": "# NAME=Qwen Q8\nMODEL=Qwen3-27B-Q8_0.gguf\nQUANT=q8\n", + "presets/Qwen 27B.env": "# NAME=Qwen 27B\nMODEL=Qwen3-27B-Q4_K_M.gguf\nCTX=65536\nREASONING=on\n", + "presets/Strata.env": "# NAME=Strata\nENGINE=strata\nSTRATA_VISION=0\nCTX=1048576\n", + "presets/a-b-c.env": "# NAME=a/b\\c\nMODEL=x.gguf\nCTX=40000\n" + }, + "state": {} + }, + { + "method": "GET", + "path": "/api/preset?id=Qwen%2027B", + "status": 200, + "out": { + "content": "# NAME=Qwen 27B\nMODEL=Qwen3-27B-Q4_K_M.gguf\nCTX=65536\nREASONING=on\n", + "id": "Qwen 27B", + "name": "Qwen 27B", + "sysprompt": "" + }, + "disk": { + "presets/Qwen 27B (2).env": "# NAME=Qwen Q8\nMODEL=Qwen3-27B-Q8_0.gguf\nQUANT=q8\n", + "presets/Qwen 27B.env": "# NAME=Qwen 27B\nMODEL=Qwen3-27B-Q4_K_M.gguf\nCTX=65536\nREASONING=on\n", + "presets/Strata.env": "# NAME=Strata\nENGINE=strata\nSTRATA_VISION=0\nCTX=1048576\n", + "presets/a-b-c.env": "# NAME=a/b\\c\nMODEL=x.gguf\nCTX=40000\n" + }, + "state": {} + }, + { + "method": "GET", + "path": "/api/preset?id=Qwen%2027B%20(2)", + "status": 200, + "out": { + "content": "# NAME=Qwen Q8\nMODEL=Qwen3-27B-Q8_0.gguf\nQUANT=q8\n", + "id": "Qwen 27B (2)", + "name": "Qwen Q8", + "sysprompt": "" + }, + "disk": { + "presets/Qwen 27B (2).env": "# NAME=Qwen Q8\nMODEL=Qwen3-27B-Q8_0.gguf\nQUANT=q8\n", + "presets/Qwen 27B.env": "# NAME=Qwen 27B\nMODEL=Qwen3-27B-Q4_K_M.gguf\nCTX=65536\nREASONING=on\n", + "presets/Strata.env": "# NAME=Strata\nENGINE=strata\nSTRATA_VISION=0\nCTX=1048576\n", + "presets/a-b-c.env": "# NAME=a/b\\c\nMODEL=x.gguf\nCTX=40000\n" + }, + "state": {} + }, + { + "method": "GET", + "path": "/api/preset?id=absent", + "status": 404, + "out": { + "error": "not found" + }, + "disk": { + "presets/Qwen 27B (2).env": "# NAME=Qwen Q8\nMODEL=Qwen3-27B-Q8_0.gguf\nQUANT=q8\n", + "presets/Qwen 27B.env": "# NAME=Qwen 27B\nMODEL=Qwen3-27B-Q4_K_M.gguf\nCTX=65536\nREASONING=on\n", + "presets/Strata.env": "# NAME=Strata\nENGINE=strata\nSTRATA_VISION=0\nCTX=1048576\n", + "presets/a-b-c.env": "# NAME=a/b\\c\nMODEL=x.gguf\nCTX=40000\n" + }, + "state": {} + }, + { + "method": "GET", + "path": "/api/preset?id=../x", + "status": 404, + "out": { + "error": "not found" + }, + "disk": { + "presets/Qwen 27B (2).env": "# NAME=Qwen Q8\nMODEL=Qwen3-27B-Q8_0.gguf\nQUANT=q8\n", + "presets/Qwen 27B.env": "# NAME=Qwen 27B\nMODEL=Qwen3-27B-Q4_K_M.gguf\nCTX=65536\nREASONING=on\n", + "presets/Strata.env": "# NAME=Strata\nENGINE=strata\nSTRATA_VISION=0\nCTX=1048576\n", + "presets/a-b-c.env": "# NAME=a/b\\c\nMODEL=x.gguf\nCTX=40000\n" + }, + "state": {} + }, + { + "method": "POST", + "path": "/api/presets/order", + "body": "{\"ids\":[\"Strata\",\"Qwen 27B (2)\"]}", + "status": 200, + "out": { + "ok": true + }, + "disk": { + "presets/Qwen 27B (2).env": "# NAME=Qwen Q8\nMODEL=Qwen3-27B-Q8_0.gguf\nQUANT=q8\n", + "presets/Qwen 27B.env": "# NAME=Qwen 27B\nMODEL=Qwen3-27B-Q4_K_M.gguf\nCTX=65536\nREASONING=on\n", + "presets/Strata.env": "# NAME=Strata\nENGINE=strata\nSTRATA_VISION=0\nCTX=1048576\n", + "presets/a-b-c.env": "# NAME=a/b\\c\nMODEL=x.gguf\nCTX=40000\n" + }, + "state": { + "preset_order": [ + "Strata", + "Qwen 27B (2)" + ] + } + }, + { + "method": "GET", + "path": "/api/presets", + "status": 200, + "out": [ + { + "active": false, + "ctx": "1M", + "id": "Strata", + "name": "Strata", + "strata": true + }, + { + "active": false, + "ctx": "32K", + "id": "Qwen 27B (2)", + "name": "Qwen Q8", + "quant": "Q8" + }, + { + "active": false, + "ctx": "64K", + "id": "Qwen 27B", + "name": "Qwen 27B", + "quant": "Q4_K_M", + "reasoning": "on" + }, + { + "active": false, + "ctx": "40K", + "id": "a-b-c", + "name": "a/b\\c" + } + ], + "disk": { + "presets/Qwen 27B (2).env": "# NAME=Qwen Q8\nMODEL=Qwen3-27B-Q8_0.gguf\nQUANT=q8\n", + "presets/Qwen 27B.env": "# NAME=Qwen 27B\nMODEL=Qwen3-27B-Q4_K_M.gguf\nCTX=65536\nREASONING=on\n", + "presets/Strata.env": "# NAME=Strata\nENGINE=strata\nSTRATA_VISION=0\nCTX=1048576\n", + "presets/a-b-c.env": "# NAME=a/b\\c\nMODEL=x.gguf\nCTX=40000\n" + }, + "state": { + "preset_order": [ + "Strata", + "Qwen 27B (2)" + ] + } + }, + { + "method": "POST", + "path": "/api/presets/order", + "body": "{}", + "status": 200, + "out": { + "ok": true + }, + "disk": { + "presets/Qwen 27B (2).env": "# NAME=Qwen Q8\nMODEL=Qwen3-27B-Q8_0.gguf\nQUANT=q8\n", + "presets/Qwen 27B.env": "# NAME=Qwen 27B\nMODEL=Qwen3-27B-Q4_K_M.gguf\nCTX=65536\nREASONING=on\n", + "presets/Strata.env": "# NAME=Strata\nENGINE=strata\nSTRATA_VISION=0\nCTX=1048576\n", + "presets/a-b-c.env": "# NAME=a/b\\c\nMODEL=x.gguf\nCTX=40000\n" + }, + "state": { + "preset_order": null + } + }, + { + "method": "GET", + "path": "/api/presets/order", + "status": 405, + "out": { + "error": "POST attendu", + "ok": false + }, + "disk": { + "presets/Qwen 27B (2).env": "# NAME=Qwen Q8\nMODEL=Qwen3-27B-Q8_0.gguf\nQUANT=q8\n", + "presets/Qwen 27B.env": "# NAME=Qwen 27B\nMODEL=Qwen3-27B-Q4_K_M.gguf\nCTX=65536\nREASONING=on\n", + "presets/Strata.env": "# NAME=Strata\nENGINE=strata\nSTRATA_VISION=0\nCTX=1048576\n", + "presets/a-b-c.env": "# NAME=a/b\\c\nMODEL=x.gguf\nCTX=40000\n" + }, + "state": { + "preset_order": null + } + }, + { + "method": "POST", + "path": "/api/presets/order", + "body": "{\"ids\":\"x\"}", + "status": 400, + "out": { + "error": "json: cannot unmarshal string into Go struct field .ids of type []string", + "ok": false + }, + "disk": { + "presets/Qwen 27B (2).env": "# NAME=Qwen Q8\nMODEL=Qwen3-27B-Q8_0.gguf\nQUANT=q8\n", + "presets/Qwen 27B.env": "# NAME=Qwen 27B\nMODEL=Qwen3-27B-Q4_K_M.gguf\nCTX=65536\nREASONING=on\n", + "presets/Strata.env": "# NAME=Strata\nENGINE=strata\nSTRATA_VISION=0\nCTX=1048576\n", + "presets/a-b-c.env": "# NAME=a/b\\c\nMODEL=x.gguf\nCTX=40000\n" + }, + "state": { + "preset_order": null + } + }, + { + "method": "POST", + "path": "/api/presets/order", + "body": "{\"ids\":[\"a-b-c\"]}", + "status": 200, + "out": { + "ok": true + }, + "disk": { + "presets/Qwen 27B (2).env": "# NAME=Qwen Q8\nMODEL=Qwen3-27B-Q8_0.gguf\nQUANT=q8\n", + "presets/Qwen 27B.env": "# NAME=Qwen 27B\nMODEL=Qwen3-27B-Q4_K_M.gguf\nCTX=65536\nREASONING=on\n", + "presets/Strata.env": "# NAME=Strata\nENGINE=strata\nSTRATA_VISION=0\nCTX=1048576\n", + "presets/a-b-c.env": "# NAME=a/b\\c\nMODEL=x.gguf\nCTX=40000\n" + }, + "state": { + "preset_order": [ + "a-b-c" + ] + } + }, + { + "store": { + "state/bench_presets": "{\"Qwen 27B\":{\"result\":{\"prompt_per_second\":1234.5,\"predicted_per_second\":40,\"kv\":\"q8_0\",\"kv_quantized\":true,\"depth\":{\"cold\":{\"new\":30000},\"decode_per_second\":31.25}},\"model\":\"Qwen3-27B-Q4_K_M.gguf\",\"at\":1700000000},\"Qwen 27B (2)\":{\"result\":{\"prompt_per_second\":900,\"predicted_per_second\":20.5},\"model\":\"autre.gguf\",\"at\":1700000001},\"a-b-c\":{\"result\":{\"prompt_per_second\":1,\"predicted_per_second\":2},\"model\":\"x.gguf\",\"at\":1,\"fingerprint\":\"mauvaise\"}}" + }, + "method": "GET", + "path": "/api/presets", + "status": 200, + "out": [ + { + "active": false, + "ctx": "40K", + "id": "a-b-c", + "name": "a/b\\c" + }, + { + "active": false, + "bench": { + "at": 1700000000, + "decode": 40, + "depth": 30000, + "depth_decode": 31.25, + "kv": "q8_0", + "kv_quantized": true, + "prefill": 1234.5 + }, + "ctx": "64K", + "id": "Qwen 27B", + "name": "Qwen 27B", + "quant": "Q4_K_M", + "reasoning": "on" + }, + { + "active": false, + "ctx": "32K", + "id": "Qwen 27B (2)", + "name": "Qwen Q8", + "quant": "Q8" + }, + { + "active": false, + "ctx": "1M", + "id": "Strata", + "name": "Strata", + "strata": true + } + ], + "disk": { + "presets/Qwen 27B (2).env": "# NAME=Qwen Q8\nMODEL=Qwen3-27B-Q8_0.gguf\nQUANT=q8\n", + "presets/Qwen 27B.env": "# NAME=Qwen 27B\nMODEL=Qwen3-27B-Q4_K_M.gguf\nCTX=65536\nREASONING=on\n", + "presets/Strata.env": "# NAME=Strata\nENGINE=strata\nSTRATA_VISION=0\nCTX=1048576\n", + "presets/a-b-c.env": "# NAME=a/b\\c\nMODEL=x.gguf\nCTX=40000\n" + }, + "state": { + "bench_presets": { + "Qwen 27B": { + "at": 1700000000, + "model": "Qwen3-27B-Q4_K_M.gguf", + "result": { + "depth": { + "cold": { + "new": 30000 + }, + "decode_per_second": 31.25 + }, + "kv": "q8_0", + "kv_quantized": true, + "predicted_per_second": 40, + "prompt_per_second": 1234.5 + } + }, + "Qwen 27B (2)": { + "at": 1700000001, + "model": "autre.gguf", + "result": { + "predicted_per_second": 20.5, + "prompt_per_second": 900 + } + }, + "a-b-c": { + "at": 1, + "fingerprint": "mauvaise", + "model": "x.gguf", + "result": { + "predicted_per_second": 2, + "prompt_per_second": 1 + } + } + }, + "preset_order": [ + "a-b-c" + ] + } + }, + { + "store": { + "config/BIN": "", + "config/CTX": "40000", + "config/HOST": "", + "config/MEM_MODE": "always", + "config/MODEL": "x.gguf", + "config/PORT": "" + }, + "method": "GET", + "path": "/api/presets", + "status": 200, + "out": [ + { + "active": true, + "ctx": "40K", + "id": "a-b-c", + "name": "a/b\\c" + }, + { + "active": false, + "bench": { + "at": 1700000000, + "decode": 40, + "depth": 30000, + "depth_decode": 31.25, + "kv": "q8_0", + "kv_quantized": true, + "prefill": 1234.5 + }, + "ctx": "64K", + "id": "Qwen 27B", + "name": "Qwen 27B", + "quant": "Q4_K_M", + "reasoning": "on" + }, + { + "active": false, + "ctx": "32K", + "id": "Qwen 27B (2)", + "name": "Qwen Q8", + "quant": "Q8" + }, + { + "active": false, + "ctx": "1M", + "id": "Strata", + "name": "Strata", + "strata": true + } + ], + "disk": { + "presets/Qwen 27B (2).env": "# NAME=Qwen Q8\nMODEL=Qwen3-27B-Q8_0.gguf\nQUANT=q8\n", + "presets/Qwen 27B.env": "# NAME=Qwen 27B\nMODEL=Qwen3-27B-Q4_K_M.gguf\nCTX=65536\nREASONING=on\n", + "presets/Strata.env": "# NAME=Strata\nENGINE=strata\nSTRATA_VISION=0\nCTX=1048576\n", + "presets/a-b-c.env": "# NAME=a/b\\c\nMODEL=x.gguf\nCTX=40000\n" + }, + "state": { + "bench_presets": { + "Qwen 27B": { + "at": 1700000000, + "model": "Qwen3-27B-Q4_K_M.gguf", + "result": { + "depth": { + "cold": { + "new": 30000 + }, + "decode_per_second": 31.25 + }, + "kv": "q8_0", + "kv_quantized": true, + "predicted_per_second": 40, + "prompt_per_second": 1234.5 + } + }, + "Qwen 27B (2)": { + "at": 1700000001, + "model": "autre.gguf", + "result": { + "predicted_per_second": 20.5, + "prompt_per_second": 900 + } + }, + "a-b-c": { + "at": 1, + "fingerprint": "mauvaise", + "model": "x.gguf", + "result": { + "predicted_per_second": 2, + "prompt_per_second": 1 + } + } + }, + "preset_order": [ + "a-b-c" + ] + } + }, + { + "method": "POST", + "path": "/api/preset/delete", + "body": "{\"id\":\"a-b-c\"}", + "status": 400, + "out": { + "error": "preset actif, switche d'abord", + "ok": false + }, + "disk": { + "presets/Qwen 27B (2).env": "# NAME=Qwen Q8\nMODEL=Qwen3-27B-Q8_0.gguf\nQUANT=q8\n", + "presets/Qwen 27B.env": "# NAME=Qwen 27B\nMODEL=Qwen3-27B-Q4_K_M.gguf\nCTX=65536\nREASONING=on\n", + "presets/Strata.env": "# NAME=Strata\nENGINE=strata\nSTRATA_VISION=0\nCTX=1048576\n", + "presets/a-b-c.env": "# NAME=a/b\\c\nMODEL=x.gguf\nCTX=40000\n" + }, + "state": { + "bench_presets": { + "Qwen 27B": { + "at": 1700000000, + "model": "Qwen3-27B-Q4_K_M.gguf", + "result": { + "depth": { + "cold": { + "new": 30000 + }, + "decode_per_second": 31.25 + }, + "kv": "q8_0", + "kv_quantized": true, + "predicted_per_second": 40, + "prompt_per_second": 1234.5 + } + }, + "Qwen 27B (2)": { + "at": 1700000001, + "model": "autre.gguf", + "result": { + "predicted_per_second": 20.5, + "prompt_per_second": 900 + } + }, + "a-b-c": { + "at": 1, + "fingerprint": "mauvaise", + "model": "x.gguf", + "result": { + "predicted_per_second": 2, + "prompt_per_second": 1 + } + } + }, + "preset_order": [ + "a-b-c" + ] + } + }, + { + "store": { + "config/MODEL": "autre.gguf" + }, + "method": "POST", + "path": "/api/preset/delete", + "body": "{\"id\":\"a-b-c\"}", + "status": 200, + "out": { + "freed": 0, + "modelDeleted": "", + "modelError": "", + "ok": true + }, + "disk": { + "presets/Qwen 27B (2).env": "# NAME=Qwen Q8\nMODEL=Qwen3-27B-Q8_0.gguf\nQUANT=q8\n", + "presets/Qwen 27B.env": "# NAME=Qwen 27B\nMODEL=Qwen3-27B-Q4_K_M.gguf\nCTX=65536\nREASONING=on\n", + "presets/Strata.env": "# NAME=Strata\nENGINE=strata\nSTRATA_VISION=0\nCTX=1048576\n" + }, + "state": { + "bench_presets": { + "Qwen 27B": { + "at": 1700000000, + "model": "Qwen3-27B-Q4_K_M.gguf", + "result": { + "depth": { + "cold": { + "cached": 0, + "expected": 0, + "new": 30000, + "predicted_ms": 0, + "predicted_n": 0, + "predicted_per_second": 0, + "prompt_ms": 0, + "prompt_per_second": 0 + }, + "ctx": 0, + "decode_per_second": 31.25, + "reuse": 0, + "target": 0 + }, + "elapsed_sec": 0, + "kv": "q8_0", + "kv_quantized": true, + "predicted_ms": 0, + "predicted_n": 0, + "predicted_per_second": 40, + "prompt_ms": 0, + "prompt_n": 0, + "prompt_per_second": 1234.5 + } + }, + "Qwen 27B (2)": { + "at": 1700000001, + "model": "autre.gguf", + "result": { + "elapsed_sec": 0, + "predicted_ms": 0, + "predicted_n": 0, + "predicted_per_second": 20.5, + "prompt_ms": 0, + "prompt_n": 0, + "prompt_per_second": 900 + } + } + }, + "preset_order": [ + "a-b-c" + ] + } + }, + { + "method": "POST", + "path": "/api/preset/delete", + "body": "{\"id\":\"absent\"}", + "status": 400, + "out": { + "error": "introuvable", + "ok": false + }, + "disk": { + "presets/Qwen 27B (2).env": "# NAME=Qwen Q8\nMODEL=Qwen3-27B-Q8_0.gguf\nQUANT=q8\n", + "presets/Qwen 27B.env": "# NAME=Qwen 27B\nMODEL=Qwen3-27B-Q4_K_M.gguf\nCTX=65536\nREASONING=on\n", + "presets/Strata.env": "# NAME=Strata\nENGINE=strata\nSTRATA_VISION=0\nCTX=1048576\n" + }, + "state": { + "bench_presets": { + "Qwen 27B": { + "at": 1700000000, + "model": "Qwen3-27B-Q4_K_M.gguf", + "result": { + "depth": { + "cold": { + "cached": 0, + "expected": 0, + "new": 30000, + "predicted_ms": 0, + "predicted_n": 0, + "predicted_per_second": 0, + "prompt_ms": 0, + "prompt_per_second": 0 + }, + "ctx": 0, + "decode_per_second": 31.25, + "reuse": 0, + "target": 0 + }, + "elapsed_sec": 0, + "kv": "q8_0", + "kv_quantized": true, + "predicted_ms": 0, + "predicted_n": 0, + "predicted_per_second": 40, + "prompt_ms": 0, + "prompt_n": 0, + "prompt_per_second": 1234.5 + } + }, + "Qwen 27B (2)": { + "at": 1700000001, + "model": "autre.gguf", + "result": { + "elapsed_sec": 0, + "predicted_ms": 0, + "predicted_n": 0, + "predicted_per_second": 20.5, + "prompt_ms": 0, + "prompt_n": 0, + "prompt_per_second": 900 + } + } + }, + "preset_order": [ + "a-b-c" + ] + } + }, + { + "method": "POST", + "path": "/api/preset/delete", + "body": "{\"id\":\"a/b\"}", + "status": 400, + "out": { + "error": "le nom ne peut pas contenir / ni \\", + "ok": false + }, + "disk": { + "presets/Qwen 27B (2).env": "# NAME=Qwen Q8\nMODEL=Qwen3-27B-Q8_0.gguf\nQUANT=q8\n", + "presets/Qwen 27B.env": "# NAME=Qwen 27B\nMODEL=Qwen3-27B-Q4_K_M.gguf\nCTX=65536\nREASONING=on\n", + "presets/Strata.env": "# NAME=Strata\nENGINE=strata\nSTRATA_VISION=0\nCTX=1048576\n" + }, + "state": { + "bench_presets": { + "Qwen 27B": { + "at": 1700000000, + "model": "Qwen3-27B-Q4_K_M.gguf", + "result": { + "depth": { + "cold": { + "cached": 0, + "expected": 0, + "new": 30000, + "predicted_ms": 0, + "predicted_n": 0, + "predicted_per_second": 0, + "prompt_ms": 0, + "prompt_per_second": 0 + }, + "ctx": 0, + "decode_per_second": 31.25, + "reuse": 0, + "target": 0 + }, + "elapsed_sec": 0, + "kv": "q8_0", + "kv_quantized": true, + "predicted_ms": 0, + "predicted_n": 0, + "predicted_per_second": 40, + "prompt_ms": 0, + "prompt_n": 0, + "prompt_per_second": 1234.5 + } + }, + "Qwen 27B (2)": { + "at": 1700000001, + "model": "autre.gguf", + "result": { + "elapsed_sec": 0, + "predicted_ms": 0, + "predicted_n": 0, + "predicted_per_second": 20.5, + "prompt_ms": 0, + "prompt_n": 0, + "prompt_per_second": 900 + } + } + }, + "preset_order": [ + "a-b-c" + ] + } + }, + { + "files": { + "models/Qwen3-27B-Q4_K_M.gguf": "poids" + }, + "method": "POST", + "path": "/api/preset/delete", + "body": "{\"id\":\"Qwen 27B\",\"deleteModel\":true}", + "status": 200, + "out": { + "freed": 5, + "modelDeleted": "Qwen3-27B-Q4_K_M.gguf", + "modelError": "", + "ok": true + }, + "disk": { + "presets/Qwen 27B (2).env": "# NAME=Qwen Q8\nMODEL=Qwen3-27B-Q8_0.gguf\nQUANT=q8\n", + "presets/Strata.env": "# NAME=Strata\nENGINE=strata\nSTRATA_VISION=0\nCTX=1048576\n" + }, + "state": { + "bench_presets": { + "Qwen 27B (2)": { + "at": 1700000001, + "model": "autre.gguf", + "result": { + "elapsed_sec": 0, + "predicted_ms": 0, + "predicted_n": 0, + "predicted_per_second": 20.5, + "prompt_ms": 0, + "prompt_n": 0, + "prompt_per_second": 900 + } + } + }, + "preset_order": [ + "a-b-c" + ] + } + }, + { + "files": { + "models/Qwen3-27B-Q8_0.gguf": "poids8", + "presets/Copie.env": "# NAME=Copie\nMODEL=Qwen3-27B-Q8_0.gguf\n" + }, + "method": "POST", + "path": "/api/preset/delete", + "body": "{\"id\":\"Qwen 27B (2)\",\"deleteModel\":true}", + "status": 200, + "out": { + "freed": 6, + "modelDeleted": "Qwen3-27B-Q8_0.gguf", + "modelError": "supprimé, mais il était aussi référencé par : Copie", + "ok": true + }, + "disk": { + "presets/Copie.env": "# NAME=Copie\nMODEL=Qwen3-27B-Q8_0.gguf\n", + "presets/Strata.env": "# NAME=Strata\nENGINE=strata\nSTRATA_VISION=0\nCTX=1048576\n" + }, + "state": { + "bench_presets": {}, + "preset_order": [ + "a-b-c" + ] + } + }, + { + "method": "POST", + "path": "/api/preset/delete", + "body": "{\"id\":\"Copie\",\"deleteModel\":true}", + "status": 200, + "out": { + "freed": 0, + "modelDeleted": "", + "modelError": "modèle introuvable: Qwen3-27B-Q8_0.gguf", + "ok": true + }, + "disk": { + "presets/Strata.env": "# NAME=Strata\nENGINE=strata\nSTRATA_VISION=0\nCTX=1048576\n" + }, + "state": { + "bench_presets": {}, + "preset_order": [ + "a-b-c" + ] + } + }, + { + "method": "POST", + "path": "/api/preset/delete", + "body": "{\"id\":3}", + "status": 400, + "out": { + "error": "json: cannot unmarshal number into Go struct field presetSaveReq.id of type string", + "ok": false + }, + "disk": { + "presets/Strata.env": "# NAME=Strata\nENGINE=strata\nSTRATA_VISION=0\nCTX=1048576\n" + }, + "state": { + "bench_presets": {}, + "preset_order": [ + "a-b-c" + ] + } + }, + { + "method": "GET", + "path": "/api/presets", + "status": 200, + "out": [ + { + "active": false, + "ctx": "1M", + "id": "Strata", + "name": "Strata", + "strata": true + } + ], + "disk": { + "presets/Strata.env": "# NAME=Strata\nENGINE=strata\nSTRATA_VISION=0\nCTX=1048576\n" + }, + "state": { + "bench_presets": {}, + "preset_order": [ + "a-b-c" + ] + } + }, + { + "method": "GET", + "path": "/api/preset/external", + "status": 200, + "out": { + "hasKey": false, + "id": "", + "model": "", + "name": "", + "url": "" + }, + "disk": { + "presets/Strata.env": "# NAME=Strata\nENGINE=strata\nSTRATA_VISION=0\nCTX=1048576\n" + }, + "state": { + "bench_presets": {}, + "preset_order": [ + "a-b-c" + ] + } + }, + { + "method": "POST", + "path": "/api/preset/external/save", + "body": "{\"name\":\" Groq \",\"url\":\" https://api.groq.com/openai/v1 \",\"model\":\" llama \",\"key\":\"k1\",\"ctx\":\"131072\",\"vision\":true}", + "status": 200, + "out": { + "applied": false, + "id": "Groq", + "name": "Groq", + "ok": true + }, + "disk": { + "presets/Groq.env": "# NAME=Groq\nCTX=131072\nEXTERNAL=1\nEXTERNAL_KEY=k1\nEXTERNAL_MODEL=llama\nEXTERNAL_URL=https://api.groq.com/openai/v1\nEXTERNAL_VISION=1\n", + "presets/Strata.env": "# NAME=Strata\nENGINE=strata\nSTRATA_VISION=0\nCTX=1048576\n" + }, + "state": { + "bench_presets": {}, + "preset_order": [ + "a-b-c" + ] + } + }, + { + "method": "POST", + "path": "/api/preset/external/save", + "body": "{\"name\":\"X\",\"url\":\"\",\"model\":\"m\"}", + "status": 400, + "out": { + "error": "URL requise", + "ok": false + }, + "disk": { + "presets/Groq.env": "# NAME=Groq\nCTX=131072\nEXTERNAL=1\nEXTERNAL_KEY=k1\nEXTERNAL_MODEL=llama\nEXTERNAL_URL=https://api.groq.com/openai/v1\nEXTERNAL_VISION=1\n", + "presets/Strata.env": "# NAME=Strata\nENGINE=strata\nSTRATA_VISION=0\nCTX=1048576\n" + }, + "state": { + "bench_presets": {}, + "preset_order": [ + "a-b-c" + ] + } + }, + { + "method": "POST", + "path": "/api/preset/external/save", + "body": "{\"name\":\"X\",\"url\":\"u\",\"model\":\" \"}", + "status": 400, + "out": { + "error": "modèle requis", + "ok": false + }, + "disk": { + "presets/Groq.env": "# NAME=Groq\nCTX=131072\nEXTERNAL=1\nEXTERNAL_KEY=k1\nEXTERNAL_MODEL=llama\nEXTERNAL_URL=https://api.groq.com/openai/v1\nEXTERNAL_VISION=1\n", + "presets/Strata.env": "# NAME=Strata\nENGINE=strata\nSTRATA_VISION=0\nCTX=1048576\n" + }, + "state": { + "bench_presets": {}, + "preset_order": [ + "a-b-c" + ] + } + }, + { + "method": "POST", + "path": "/api/preset/external/save", + "body": "{\"name\":\"\",\"url\":\"u\",\"model\":\"m\"}", + "status": 400, + "out": { + "error": "nom requis", + "ok": false + }, + "disk": { + "presets/Groq.env": "# NAME=Groq\nCTX=131072\nEXTERNAL=1\nEXTERNAL_KEY=k1\nEXTERNAL_MODEL=llama\nEXTERNAL_URL=https://api.groq.com/openai/v1\nEXTERNAL_VISION=1\n", + "presets/Strata.env": "# NAME=Strata\nENGINE=strata\nSTRATA_VISION=0\nCTX=1048576\n" + }, + "state": { + "bench_presets": {}, + "preset_order": [ + "a-b-c" + ] + } + }, + { + "method": "GET", + "path": "/api/preset/external?id=Groq", + "status": 200, + "out": { + "ctx": "131072", + "hasKey": true, + "id": "Groq", + "model": "llama", + "name": "Groq", + "url": "https://api.groq.com/openai/v1", + "vision": true + }, + "disk": { + "presets/Groq.env": "# NAME=Groq\nCTX=131072\nEXTERNAL=1\nEXTERNAL_KEY=k1\nEXTERNAL_MODEL=llama\nEXTERNAL_URL=https://api.groq.com/openai/v1\nEXTERNAL_VISION=1\n", + "presets/Strata.env": "# NAME=Strata\nENGINE=strata\nSTRATA_VISION=0\nCTX=1048576\n" + }, + "state": { + "bench_presets": {}, + "preset_order": [ + "a-b-c" + ] + } + }, + { + "method": "POST", + "path": "/api/preset/external/save", + "body": "{\"id\":\"Groq\",\"name\":\"Groq 2\",\"url\":\"https://api.groq.com\",\"model\":\"llama-3\"}", + "status": 200, + "out": { + "applied": false, + "id": "Groq", + "name": "Groq 2", + "ok": true + }, + "disk": { + "presets/Groq.env": "# NAME=Groq 2\nEXTERNAL=1\nEXTERNAL_KEY=k1\nEXTERNAL_MODEL=llama-3\nEXTERNAL_URL=https://api.groq.com\n", + "presets/Strata.env": "# NAME=Strata\nENGINE=strata\nSTRATA_VISION=0\nCTX=1048576\n" + }, + "state": { + "bench_presets": {}, + "preset_order": [ + "a-b-c" + ] + } + }, + { + "method": "GET", + "path": "/api/preset/external?id=Groq", + "status": 200, + "out": { + "ctx": "", + "hasKey": true, + "id": "Groq", + "model": "llama-3", + "name": "Groq 2", + "url": "https://api.groq.com", + "vision": false + }, + "disk": { + "presets/Groq.env": "# NAME=Groq 2\nEXTERNAL=1\nEXTERNAL_KEY=k1\nEXTERNAL_MODEL=llama-3\nEXTERNAL_URL=https://api.groq.com\n", + "presets/Strata.env": "# NAME=Strata\nENGINE=strata\nSTRATA_VISION=0\nCTX=1048576\n" + }, + "state": { + "bench_presets": {}, + "preset_order": [ + "a-b-c" + ] + } + }, + { + "method": "POST", + "path": "/api/preset/external/save", + "body": "{\"id\":\"Groq\",\"name\":\"Groq\",\"url\":\"https://api.groq.com\",\"model\":\"llama-3\",\"keyTouched\":true,\"key\":\"\"}", + "status": 200, + "out": { + "applied": false, + "id": "Groq", + "name": "Groq", + "ok": true + }, + "disk": { + "presets/Groq.env": "# NAME=Groq\nEXTERNAL=1\nEXTERNAL_MODEL=llama-3\nEXTERNAL_URL=https://api.groq.com\n", + "presets/Strata.env": "# NAME=Strata\nENGINE=strata\nSTRATA_VISION=0\nCTX=1048576\n" + }, + "state": { + "bench_presets": {}, + "preset_order": [ + "a-b-c" + ] + } + }, + { + "method": "GET", + "path": "/api/preset/external?id=Groq", + "status": 200, + "out": { + "ctx": "", + "hasKey": false, + "id": "Groq", + "model": "llama-3", + "name": "Groq", + "url": "https://api.groq.com", + "vision": false + }, + "disk": { + "presets/Groq.env": "# NAME=Groq\nEXTERNAL=1\nEXTERNAL_MODEL=llama-3\nEXTERNAL_URL=https://api.groq.com\n", + "presets/Strata.env": "# NAME=Strata\nENGINE=strata\nSTRATA_VISION=0\nCTX=1048576\n" + }, + "state": { + "bench_presets": {}, + "preset_order": [ + "a-b-c" + ] + } + }, + { + "method": "GET", + "path": "/api/preset/external?id=Strata", + "status": 400, + "out": { + "error": "pas un preset externe" + }, + "disk": { + "presets/Groq.env": "# NAME=Groq\nEXTERNAL=1\nEXTERNAL_MODEL=llama-3\nEXTERNAL_URL=https://api.groq.com\n", + "presets/Strata.env": "# NAME=Strata\nENGINE=strata\nSTRATA_VISION=0\nCTX=1048576\n" + }, + "state": { + "bench_presets": {}, + "preset_order": [ + "a-b-c" + ] + } + }, + { + "method": "GET", + "path": "/api/preset/external?id=absent", + "status": 404, + "out": { + "error": "not found" + }, + "disk": { + "presets/Groq.env": "# NAME=Groq\nEXTERNAL=1\nEXTERNAL_MODEL=llama-3\nEXTERNAL_URL=https://api.groq.com\n", + "presets/Strata.env": "# NAME=Strata\nENGINE=strata\nSTRATA_VISION=0\nCTX=1048576\n" + }, + "state": { + "bench_presets": {}, + "preset_order": [ + "a-b-c" + ] + } + }, + { + "method": "GET", + "path": "/api/presets", + "status": 200, + "out": [ + { + "active": false, + "external": true, + "id": "Groq", + "model": "llama-3", + "name": "Groq" + }, + { + "active": false, + "ctx": "1M", + "id": "Strata", + "name": "Strata", + "strata": true + } + ], + "disk": { + "presets/Groq.env": "# NAME=Groq\nEXTERNAL=1\nEXTERNAL_MODEL=llama-3\nEXTERNAL_URL=https://api.groq.com\n", + "presets/Strata.env": "# NAME=Strata\nENGINE=strata\nSTRATA_VISION=0\nCTX=1048576\n" + }, + "state": { + "bench_presets": {}, + "preset_order": [ + "a-b-c" + ] + } + }, + { + "method": "POST", + "path": "/api/preset/external/test", + "body": "{\"url\":\"{{URL}}/ok\",\"model\":\"m\",\"key\":\"k1\"}", + "status": 200, + "out": { + "ok": true + }, + "disk": { + "presets/Groq.env": "# NAME=Groq\nEXTERNAL=1\nEXTERNAL_MODEL=llama-3\nEXTERNAL_URL=https://api.groq.com\n", + "presets/Strata.env": "# NAME=Strata\nENGINE=strata\nSTRATA_VISION=0\nCTX=1048576\n" + }, + "state": { + "bench_presets": {}, + "preset_order": [ + "a-b-c" + ] + } + }, + { + "method": "POST", + "path": "/api/preset/external/test", + "body": "{\"url\":\"{{URL}}/refus/v1\",\"model\":\"m\"}", + "status": 200, + "out": { + "error": "clé invalide", + "ok": false, + "status": 401 + }, + "disk": { + "presets/Groq.env": "# NAME=Groq\nEXTERNAL=1\nEXTERNAL_MODEL=llama-3\nEXTERNAL_URL=https://api.groq.com\n", + "presets/Strata.env": "# NAME=Strata\nENGINE=strata\nSTRATA_VISION=0\nCTX=1048576\n" + }, + "state": { + "bench_presets": {}, + "preset_order": [ + "a-b-c" + ] + } + }, + { + "method": "POST", + "path": "/api/preset/external/test", + "body": "{\"url\":\"{{URL}}/brut/v1/chat/completions\",\"model\":\"m\"}", + "status": 200, + "out": { + "error": "page absente", + "ok": false, + "status": 404 + }, + "disk": { + "presets/Groq.env": "# NAME=Groq\nEXTERNAL=1\nEXTERNAL_MODEL=llama-3\nEXTERNAL_URL=https://api.groq.com\n", + "presets/Strata.env": "# NAME=Strata\nENGINE=strata\nSTRATA_VISION=0\nCTX=1048576\n" + }, + "state": { + "bench_presets": {}, + "preset_order": [ + "a-b-c" + ] + } + }, + { + "method": "POST", + "path": "/api/preset/external/test", + "body": "{\"url\":\"{{URL}}/rien\",\"model\":\"m\"}", + "status": 200, + "out": { + "error": "500 Internal Server Error", + "ok": false, + "status": 500 + }, + "disk": { + "presets/Groq.env": "# NAME=Groq\nEXTERNAL=1\nEXTERNAL_MODEL=llama-3\nEXTERNAL_URL=https://api.groq.com\n", + "presets/Strata.env": "# NAME=Strata\nENGINE=strata\nSTRATA_VISION=0\nCTX=1048576\n" + }, + "state": { + "bench_presets": {}, + "preset_order": [ + "a-b-c" + ] + } + }, + { + "method": "POST", + "path": "/api/preset/external/test", + "body": "{\"url\":\"\",\"model\":\"m\"}", + "status": 200, + "out": { + "error": "URL et modèle requis", + "ok": false + }, + "disk": { + "presets/Groq.env": "# NAME=Groq\nEXTERNAL=1\nEXTERNAL_MODEL=llama-3\nEXTERNAL_URL=https://api.groq.com\n", + "presets/Strata.env": "# NAME=Strata\nENGINE=strata\nSTRATA_VISION=0\nCTX=1048576\n" + }, + "state": { + "bench_presets": {}, + "preset_order": [ + "a-b-c" + ] + } + }, + { + "method": "POST", + "path": "/api/preset/external/save", + "body": "{\"name\":\"Clé\",\"url\":\"{{URL}}/ok\",\"model\":\"m\",\"key\":\"k1\"}", + "status": 200, + "out": { + "applied": false, + "id": "Clé", + "name": "Clé", + "ok": true + }, + "disk": { + "presets/Clé.env": "# NAME=Clé\nEXTERNAL=1\nEXTERNAL_KEY=k1\nEXTERNAL_MODEL=m\nEXTERNAL_URL={{URL}}/ok\n", + "presets/Groq.env": "# NAME=Groq\nEXTERNAL=1\nEXTERNAL_MODEL=llama-3\nEXTERNAL_URL=https://api.groq.com\n", + "presets/Strata.env": "# NAME=Strata\nENGINE=strata\nSTRATA_VISION=0\nCTX=1048576\n" + }, + "state": { + "bench_presets": {}, + "preset_order": [ + "a-b-c" + ] + } + }, + { + "method": "POST", + "path": "/api/preset/external/test", + "body": "{\"id\":\"Clé\",\"url\":\"{{URL}}/ok\",\"model\":\"m\"}", + "status": 200, + "out": { + "ok": true + }, + "disk": { + "presets/Clé.env": "# NAME=Clé\nEXTERNAL=1\nEXTERNAL_KEY=k1\nEXTERNAL_MODEL=m\nEXTERNAL_URL={{URL}}/ok\n", + "presets/Groq.env": "# NAME=Groq\nEXTERNAL=1\nEXTERNAL_MODEL=llama-3\nEXTERNAL_URL=https://api.groq.com\n", + "presets/Strata.env": "# NAME=Strata\nENGINE=strata\nSTRATA_VISION=0\nCTX=1048576\n" + }, + "state": { + "bench_presets": {}, + "preset_order": [ + "a-b-c" + ] + } + }, + { + "method": "POST", + "path": "/api/preset/external/test", + "body": "[]", + "status": 400, + "out": { + "error": "json: cannot unmarshal array into Go value of type loki.externalSaveReq", + "ok": false + }, + "disk": { + "presets/Clé.env": "# NAME=Clé\nEXTERNAL=1\nEXTERNAL_KEY=k1\nEXTERNAL_MODEL=m\nEXTERNAL_URL={{URL}}/ok\n", + "presets/Groq.env": "# NAME=Groq\nEXTERNAL=1\nEXTERNAL_MODEL=llama-3\nEXTERNAL_URL=https://api.groq.com\n", + "presets/Strata.env": "# NAME=Strata\nENGINE=strata\nSTRATA_VISION=0\nCTX=1048576\n" + }, + "state": { + "bench_presets": {}, + "preset_order": [ + "a-b-c" + ] + } + }, + { + "method": "GET", + "path": "/api/preset/autoplace", + "status": 405, + "out": { + "error": "méthode GET refusée : POST attendu", + "ok": false + }, + "disk": { + "presets/Clé.env": "# NAME=Clé\nEXTERNAL=1\nEXTERNAL_KEY=k1\nEXTERNAL_MODEL=m\nEXTERNAL_URL={{URL}}/ok\n", + "presets/Groq.env": "# NAME=Groq\nEXTERNAL=1\nEXTERNAL_MODEL=llama-3\nEXTERNAL_URL=https://api.groq.com\n", + "presets/Strata.env": "# NAME=Strata\nENGINE=strata\nSTRATA_VISION=0\nCTX=1048576\n" + }, + "state": { + "bench_presets": {}, + "preset_order": [ + "a-b-c" + ] + } + }, + { + "method": "POST", + "path": "/api/preset/autoplace", + "body": "{\"id\":\"MoE\",\"content\":\"# NAME=MoE\\nMODEL=absent.gguf\\nCTX=32768\\nEXTRA_ARGS=--n-cpu-moe 20 --load-mode none\\n\"}", + "status": 200, + "out": { + "changes": [ + "retiré : --n-cpu-moe 20", + "retiré : --load-mode none", + "UBATCH=2048, BATCH=4096 (avant : 512 / 2048)", + "--load-mode mmap", + "CTX conservé : 32768" + ], + "id": "MoE (placement auto)", + "name": "MoE (placement auto)", + "ok": true + }, + "disk": { + "presets/Clé.env": "# NAME=Clé\nEXTERNAL=1\nEXTERNAL_KEY=k1\nEXTERNAL_MODEL=m\nEXTERNAL_URL={{URL}}/ok\n", + "presets/Groq.env": "# NAME=Groq\nEXTERNAL=1\nEXTERNAL_MODEL=llama-3\nEXTERNAL_URL=https://api.groq.com\n", + "presets/MoE (placement auto).env": "# NAME=MoE (placement auto)\nMODEL=absent.gguf\nCTX=32768\nEXTRA_ARGS=\"--load-mode mmap\"\nUBATCH=2048\nBATCH=4096\n", + "presets/Strata.env": "# NAME=Strata\nENGINE=strata\nSTRATA_VISION=0\nCTX=1048576\n" + }, + "state": { + "bench_presets": {}, + "preset_order": [ + "a-b-c" + ] + } + }, + { + "method": "POST", + "path": "/api/preset/autoplace", + "body": "{\"name\":\" Mon MoE \",\"content\":\"MODEL=m\\nEXTRA_ARGS=-ngl 30\\n\"}", + "status": 200, + "out": { + "error": "rien à migrer : ce preset ne place pas d'experts à la main (-ot …exps…, --n-cpu-moe, --cpu-moe)", + "ok": false + }, + "disk": { + "presets/Clé.env": "# NAME=Clé\nEXTERNAL=1\nEXTERNAL_KEY=k1\nEXTERNAL_MODEL=m\nEXTERNAL_URL={{URL}}/ok\n", + "presets/Groq.env": "# NAME=Groq\nEXTERNAL=1\nEXTERNAL_MODEL=llama-3\nEXTERNAL_URL=https://api.groq.com\n", + "presets/MoE (placement auto).env": "# NAME=MoE (placement auto)\nMODEL=absent.gguf\nCTX=32768\nEXTRA_ARGS=\"--load-mode mmap\"\nUBATCH=2048\nBATCH=4096\n", + "presets/Strata.env": "# NAME=Strata\nENGINE=strata\nSTRATA_VISION=0\nCTX=1048576\n" + }, + "state": { + "bench_presets": {}, + "preset_order": [ + "a-b-c" + ] + } + }, + { + "method": "POST", + "path": "/api/preset/autoplace", + "body": "{\"content\":\"MODEL=m\\nEXTRA_ARGS=--cpu-moe\\n\"}", + "status": 200, + "out": { + "changes": [ + "retiré : --cpu-moe", + "UBATCH=2048, BATCH=4096 (avant : 512 / 2048)", + "--load-mode mmap", + "CTX conservé : 32768" + ], + "id": "preset (placement auto)", + "name": "preset (placement auto)", + "ok": true + }, + "disk": { + "presets/Clé.env": "# NAME=Clé\nEXTERNAL=1\nEXTERNAL_KEY=k1\nEXTERNAL_MODEL=m\nEXTERNAL_URL={{URL}}/ok\n", + "presets/Groq.env": "# NAME=Groq\nEXTERNAL=1\nEXTERNAL_MODEL=llama-3\nEXTERNAL_URL=https://api.groq.com\n", + "presets/MoE (placement auto).env": "# NAME=MoE (placement auto)\nMODEL=absent.gguf\nCTX=32768\nEXTRA_ARGS=\"--load-mode mmap\"\nUBATCH=2048\nBATCH=4096\n", + "presets/Strata.env": "# NAME=Strata\nENGINE=strata\nSTRATA_VISION=0\nCTX=1048576\n", + "presets/preset (placement auto).env": "# NAME=preset (placement auto)\nMODEL=m\nEXTRA_ARGS=\"--load-mode mmap\"\nUBATCH=2048\nBATCH=4096\n" + }, + "state": { + "bench_presets": {}, + "preset_order": [ + "a-b-c" + ] + } + }, + { + "method": "POST", + "path": "/api/preset/moe", + "body": "{\"content\":\"# NAME=MoE\\nMODEL=absent.gguf\\nCTX=32768\\nEXTRA_ARGS=--n-cpu-moe 20 --load-mode none\\n\"}", + "status": 200, + "out": { + "autoplace": { + "changes": [ + "retiré : --n-cpu-moe 20", + "retiré : --load-mode none", + "UBATCH=2048, BATCH=4096 (avant : 512 / 2048)", + "--load-mode mmap", + "CTX conservé : 32768" + ], + "ok": true + }, + "load": "", + "notes": null, + "ok": true + }, + "disk": { + "presets/Clé.env": "# NAME=Clé\nEXTERNAL=1\nEXTERNAL_KEY=k1\nEXTERNAL_MODEL=m\nEXTERNAL_URL={{URL}}/ok\n", + "presets/Groq.env": "# NAME=Groq\nEXTERNAL=1\nEXTERNAL_MODEL=llama-3\nEXTERNAL_URL=https://api.groq.com\n", + "presets/MoE (placement auto).env": "# NAME=MoE (placement auto)\nMODEL=absent.gguf\nCTX=32768\nEXTRA_ARGS=\"--load-mode mmap\"\nUBATCH=2048\nBATCH=4096\n", + "presets/Strata.env": "# NAME=Strata\nENGINE=strata\nSTRATA_VISION=0\nCTX=1048576\n", + "presets/preset (placement auto).env": "# NAME=preset (placement auto)\nMODEL=m\nEXTRA_ARGS=\"--load-mode mmap\"\nUBATCH=2048\nBATCH=4096\n" + }, + "state": { + "bench_presets": {}, + "preset_order": [ + "a-b-c" + ] + } + }, + { + "method": "POST", + "path": "/api/preset/moe", + "body": "{\"content\":\"EXTERNAL=1\\n\"}", + "status": 200, + "out": { + "load": "", + "notes": null, + "ok": true + }, + "disk": { + "presets/Clé.env": "# NAME=Clé\nEXTERNAL=1\nEXTERNAL_KEY=k1\nEXTERNAL_MODEL=m\nEXTERNAL_URL={{URL}}/ok\n", + "presets/Groq.env": "# NAME=Groq\nEXTERNAL=1\nEXTERNAL_MODEL=llama-3\nEXTERNAL_URL=https://api.groq.com\n", + "presets/MoE (placement auto).env": "# NAME=MoE (placement auto)\nMODEL=absent.gguf\nCTX=32768\nEXTRA_ARGS=\"--load-mode mmap\"\nUBATCH=2048\nBATCH=4096\n", + "presets/Strata.env": "# NAME=Strata\nENGINE=strata\nSTRATA_VISION=0\nCTX=1048576\n", + "presets/preset (placement auto).env": "# NAME=preset (placement auto)\nMODEL=m\nEXTRA_ARGS=\"--load-mode mmap\"\nUBATCH=2048\nBATCH=4096\n" + }, + "state": { + "bench_presets": {}, + "preset_order": [ + "a-b-c" + ] + } + }, + { + "method": "POST", + "path": "/api/preset/moe", + "body": "{\"content\":\"MODEL=absent.gguf\\nEXTRA_ARGS=--cpu-moe -sm row\\n\"}", + "status": 200, + "out": { + "autoplace": { + "ok": false, + "why": "--fit resterait inactif (-sm row) : retire-le à la main d'abord" + }, + "load": "", + "notes": null, + "ok": true + }, + "disk": { + "presets/Clé.env": "# NAME=Clé\nEXTERNAL=1\nEXTERNAL_KEY=k1\nEXTERNAL_MODEL=m\nEXTERNAL_URL={{URL}}/ok\n", + "presets/Groq.env": "# NAME=Groq\nEXTERNAL=1\nEXTERNAL_MODEL=llama-3\nEXTERNAL_URL=https://api.groq.com\n", + "presets/MoE (placement auto).env": "# NAME=MoE (placement auto)\nMODEL=absent.gguf\nCTX=32768\nEXTRA_ARGS=\"--load-mode mmap\"\nUBATCH=2048\nBATCH=4096\n", + "presets/Strata.env": "# NAME=Strata\nENGINE=strata\nSTRATA_VISION=0\nCTX=1048576\n", + "presets/preset (placement auto).env": "# NAME=preset (placement auto)\nMODEL=m\nEXTRA_ARGS=\"--load-mode mmap\"\nUBATCH=2048\nBATCH=4096\n" + }, + "state": { + "bench_presets": {}, + "preset_order": [ + "a-b-c" + ] + } + }, + { + "method": "POST", + "path": "/api/preset/moe", + "body": "{\"content\":5}", + "status": 400, + "out": { + "error": "json: cannot unmarshal number into Go struct field .content of type string", + "ok": false + }, + "disk": { + "presets/Clé.env": "# NAME=Clé\nEXTERNAL=1\nEXTERNAL_KEY=k1\nEXTERNAL_MODEL=m\nEXTERNAL_URL={{URL}}/ok\n", + "presets/Groq.env": "# NAME=Groq\nEXTERNAL=1\nEXTERNAL_MODEL=llama-3\nEXTERNAL_URL=https://api.groq.com\n", + "presets/MoE (placement auto).env": "# NAME=MoE (placement auto)\nMODEL=absent.gguf\nCTX=32768\nEXTRA_ARGS=\"--load-mode mmap\"\nUBATCH=2048\nBATCH=4096\n", + "presets/Strata.env": "# NAME=Strata\nENGINE=strata\nSTRATA_VISION=0\nCTX=1048576\n", + "presets/preset (placement auto).env": "# NAME=preset (placement auto)\nMODEL=m\nEXTRA_ARGS=\"--load-mode mmap\"\nUBATCH=2048\nBATCH=4096\n" + }, + "state": { + "bench_presets": {}, + "preset_order": [ + "a-b-c" + ] + } + }, + { + "method": "POST", + "path": "/api/preset/cacheram", + "body": "{\"content\":\"EXTERNAL=1\\nSIDE_SLOT=on\\n\"}", + "status": 200, + "out": { + "default": 8192, + "mib": 0, + "ok": true, + "side": { + "on": false, + "vramMiB": 0, + "why": "preset externe" + }, + "why": "preset externe" + }, + "disk": { + "presets/Clé.env": "# NAME=Clé\nEXTERNAL=1\nEXTERNAL_KEY=k1\nEXTERNAL_MODEL=m\nEXTERNAL_URL={{URL}}/ok\n", + "presets/Groq.env": "# NAME=Groq\nEXTERNAL=1\nEXTERNAL_MODEL=llama-3\nEXTERNAL_URL=https://api.groq.com\n", + "presets/MoE (placement auto).env": "# NAME=MoE (placement auto)\nMODEL=absent.gguf\nCTX=32768\nEXTRA_ARGS=\"--load-mode mmap\"\nUBATCH=2048\nBATCH=4096\n", + "presets/Strata.env": "# NAME=Strata\nENGINE=strata\nSTRATA_VISION=0\nCTX=1048576\n", + "presets/preset (placement auto).env": "# NAME=preset (placement auto)\nMODEL=m\nEXTRA_ARGS=\"--load-mode mmap\"\nUBATCH=2048\nBATCH=4096\n" + }, + "state": { + "bench_presets": {}, + "preset_order": [ + "a-b-c" + ] + } + }, + { + "method": "POST", + "path": "/api/preset/cacheram", + "body": "{\"content\":\"EXTRA_ARGS=-cram 2048\\n\"}", + "status": 200, + "out": { + "default": 8192, + "mib": 2048, + "ok": true, + "why": "fixé par EXTRA_ARGS" + }, + "disk": { + "presets/Clé.env": "# NAME=Clé\nEXTERNAL=1\nEXTERNAL_KEY=k1\nEXTERNAL_MODEL=m\nEXTERNAL_URL={{URL}}/ok\n", + "presets/Groq.env": "# NAME=Groq\nEXTERNAL=1\nEXTERNAL_MODEL=llama-3\nEXTERNAL_URL=https://api.groq.com\n", + "presets/MoE (placement auto).env": "# NAME=MoE (placement auto)\nMODEL=absent.gguf\nCTX=32768\nEXTRA_ARGS=\"--load-mode mmap\"\nUBATCH=2048\nBATCH=4096\n", + "presets/Strata.env": "# NAME=Strata\nENGINE=strata\nSTRATA_VISION=0\nCTX=1048576\n", + "presets/preset (placement auto).env": "# NAME=preset (placement auto)\nMODEL=m\nEXTRA_ARGS=\"--load-mode mmap\"\nUBATCH=2048\nBATCH=4096\n" + }, + "state": { + "bench_presets": {}, + "preset_order": [ + "a-b-c" + ] + } + }, + { + "method": "POST", + "path": "/api/preset/cacheram", + "body": "{\"content\":\"CACHE_RAM=4096\\nSIDE_SLOT=on\\nMODEL=absent.gguf\\n\"}", + "status": 200, + "out": { + "default": 8192, + "mib": 4096, + "ok": true, + "side": { + "on": false, + "vramMiB": 0, + "why": "VRAM NVIDIA inconnue (macOS, AMD, --device…)" + }, + "why": "fixé par CACHE_RAM" + }, + "disk": { + "presets/Clé.env": "# NAME=Clé\nEXTERNAL=1\nEXTERNAL_KEY=k1\nEXTERNAL_MODEL=m\nEXTERNAL_URL={{URL}}/ok\n", + "presets/Groq.env": "# NAME=Groq\nEXTERNAL=1\nEXTERNAL_MODEL=llama-3\nEXTERNAL_URL=https://api.groq.com\n", + "presets/MoE (placement auto).env": "# NAME=MoE (placement auto)\nMODEL=absent.gguf\nCTX=32768\nEXTRA_ARGS=\"--load-mode mmap\"\nUBATCH=2048\nBATCH=4096\n", + "presets/Strata.env": "# NAME=Strata\nENGINE=strata\nSTRATA_VISION=0\nCTX=1048576\n", + "presets/preset (placement auto).env": "# NAME=preset (placement auto)\nMODEL=m\nEXTRA_ARGS=\"--load-mode mmap\"\nUBATCH=2048\nBATCH=4096\n" + }, + "state": { + "bench_presets": {}, + "preset_order": [ + "a-b-c" + ] + } + }, + { + "method": "POST", + "path": "/api/preset/cacheram", + "body": "{\"content\":\"CACHE_RAM=beaucoup\\n\"}", + "status": 200, + "out": { + "default": 8192, + "mib": 0, + "ok": true, + "why": "CACHE_RAM illisible" + }, + "disk": { + "presets/Clé.env": "# NAME=Clé\nEXTERNAL=1\nEXTERNAL_KEY=k1\nEXTERNAL_MODEL=m\nEXTERNAL_URL={{URL}}/ok\n", + "presets/Groq.env": "# NAME=Groq\nEXTERNAL=1\nEXTERNAL_MODEL=llama-3\nEXTERNAL_URL=https://api.groq.com\n", + "presets/MoE (placement auto).env": "# NAME=MoE (placement auto)\nMODEL=absent.gguf\nCTX=32768\nEXTRA_ARGS=\"--load-mode mmap\"\nUBATCH=2048\nBATCH=4096\n", + "presets/Strata.env": "# NAME=Strata\nENGINE=strata\nSTRATA_VISION=0\nCTX=1048576\n", + "presets/preset (placement auto).env": "# NAME=preset (placement auto)\nMODEL=m\nEXTRA_ARGS=\"--load-mode mmap\"\nUBATCH=2048\nBATCH=4096\n" + }, + "state": { + "bench_presets": {}, + "preset_order": [ + "a-b-c" + ] + } + }, + { + "method": "POST", + "path": "/api/preset/cacheram", + "body": "{\"content\":\"CACHE_RAM=auto\\n\"}", + "status": 200, + "out": { + "default": 8192, + "mib": 0, + "ok": true, + "why": "modèle introuvable" + }, + "disk": { + "presets/Clé.env": "# NAME=Clé\nEXTERNAL=1\nEXTERNAL_KEY=k1\nEXTERNAL_MODEL=m\nEXTERNAL_URL={{URL}}/ok\n", + "presets/Groq.env": "# NAME=Groq\nEXTERNAL=1\nEXTERNAL_MODEL=llama-3\nEXTERNAL_URL=https://api.groq.com\n", + "presets/MoE (placement auto).env": "# NAME=MoE (placement auto)\nMODEL=absent.gguf\nCTX=32768\nEXTRA_ARGS=\"--load-mode mmap\"\nUBATCH=2048\nBATCH=4096\n", + "presets/Strata.env": "# NAME=Strata\nENGINE=strata\nSTRATA_VISION=0\nCTX=1048576\n", + "presets/preset (placement auto).env": "# NAME=preset (placement auto)\nMODEL=m\nEXTRA_ARGS=\"--load-mode mmap\"\nUBATCH=2048\nBATCH=4096\n" + }, + "state": { + "bench_presets": {}, + "preset_order": [ + "a-b-c" + ] + } + }, + { + "method": "POST", + "path": "/api/preset/cacheram", + "body": "{\"content\":\"MODEL=absent.gguf\\nSIDE_SLOT=on\\nPARALLEL=3\\n\"}", + "status": 200, + "out": { + "default": 8192, + "mib": 0, + "ok": true, + "side": { + "on": false, + "vramMiB": 0, + "why": "PARALLEL=3 (SIDE_SLOT ouvre exactement 2 slots)" + }, + "why": "VRAM NVIDIA inconnue" + }, + "disk": { + "presets/Clé.env": "# NAME=Clé\nEXTERNAL=1\nEXTERNAL_KEY=k1\nEXTERNAL_MODEL=m\nEXTERNAL_URL={{URL}}/ok\n", + "presets/Groq.env": "# NAME=Groq\nEXTERNAL=1\nEXTERNAL_MODEL=llama-3\nEXTERNAL_URL=https://api.groq.com\n", + "presets/MoE (placement auto).env": "# NAME=MoE (placement auto)\nMODEL=absent.gguf\nCTX=32768\nEXTRA_ARGS=\"--load-mode mmap\"\nUBATCH=2048\nBATCH=4096\n", + "presets/Strata.env": "# NAME=Strata\nENGINE=strata\nSTRATA_VISION=0\nCTX=1048576\n", + "presets/preset (placement auto).env": "# NAME=preset (placement auto)\nMODEL=m\nEXTRA_ARGS=\"--load-mode mmap\"\nUBATCH=2048\nBATCH=4096\n" + }, + "state": { + "bench_presets": {}, + "preset_order": [ + "a-b-c" + ] + } + }, + { + "method": "POST", + "path": "/api/preset/cacheram", + "body": "{}", + "status": 200, + "out": { + "default": 8192, + "mib": 0, + "ok": true, + "why": "modèle introuvable" + }, + "disk": { + "presets/Clé.env": "# NAME=Clé\nEXTERNAL=1\nEXTERNAL_KEY=k1\nEXTERNAL_MODEL=m\nEXTERNAL_URL={{URL}}/ok\n", + "presets/Groq.env": "# NAME=Groq\nEXTERNAL=1\nEXTERNAL_MODEL=llama-3\nEXTERNAL_URL=https://api.groq.com\n", + "presets/MoE (placement auto).env": "# NAME=MoE (placement auto)\nMODEL=absent.gguf\nCTX=32768\nEXTRA_ARGS=\"--load-mode mmap\"\nUBATCH=2048\nBATCH=4096\n", + "presets/Strata.env": "# NAME=Strata\nENGINE=strata\nSTRATA_VISION=0\nCTX=1048576\n", + "presets/preset (placement auto).env": "# NAME=preset (placement auto)\nMODEL=m\nEXTRA_ARGS=\"--load-mode mmap\"\nUBATCH=2048\nBATCH=4096\n" + }, + "state": { + "bench_presets": {}, + "preset_order": [ + "a-b-c" + ] + } + }, + { + "method": "GET", + "path": "/api/presets", + "status": 200, + "out": [ + { + "active": false, + "external": true, + "id": "Clé", + "model": "m", + "name": "Clé" + }, + { + "active": false, + "external": true, + "id": "Groq", + "model": "llama-3", + "name": "Groq" + }, + { + "active": false, + "ctx": "32K", + "id": "MoE (placement auto)", + "name": "MoE (placement auto)" + }, + { + "active": false, + "ctx": "1M", + "id": "Strata", + "name": "Strata", + "strata": true + }, + { + "active": false, + "ctx": "32K", + "id": "preset (placement auto)", + "name": "preset (placement auto)" + } + ], + "disk": { + "presets/Clé.env": "# NAME=Clé\nEXTERNAL=1\nEXTERNAL_KEY=k1\nEXTERNAL_MODEL=m\nEXTERNAL_URL={{URL}}/ok\n", + "presets/Groq.env": "# NAME=Groq\nEXTERNAL=1\nEXTERNAL_MODEL=llama-3\nEXTERNAL_URL=https://api.groq.com\n", + "presets/MoE (placement auto).env": "# NAME=MoE (placement auto)\nMODEL=absent.gguf\nCTX=32768\nEXTRA_ARGS=\"--load-mode mmap\"\nUBATCH=2048\nBATCH=4096\n", + "presets/Strata.env": "# NAME=Strata\nENGINE=strata\nSTRATA_VISION=0\nCTX=1048576\n", + "presets/preset (placement auto).env": "# NAME=preset (placement auto)\nMODEL=m\nEXTRA_ARGS=\"--load-mode mmap\"\nUBATCH=2048\nBATCH=4096\n" + }, + "state": { + "bench_presets": {}, + "preset_order": [ + "a-b-c" + ] + } + } + ] +} diff --git a/rust/loki-rs/tests/presets_api.rs b/rust/loki-rs/tests/presets_api.rs new file mode 100644 index 0000000..86e99ef --- /dev/null +++ b/rust/loki-rs/tests/presets_api.rs @@ -0,0 +1,414 @@ +//! Lot 15 : presets tenus par la façade (`LOKI_PRESETS=rust`). +//! +//! Le journal tiré du Go (`internal/loki/parity_presets_test.go` → +//! `tests/parity_presets/presets.json`) est rejoué : fonctions pures, puis la +//! suite d'appels aux routes par la façade, avec après chacun le dossier +//! presets/ (et models/) et les clés touchées en base. La bascule et +//! l'enregistrement du preset en service, qui pilotent le moteur au Go, sont +//! vérifiés de bout en bout contre le faux Go. + +mod common; + +use std::collections::{BTreeMap, HashMap}; +use std::path::Path; + +use common::fakego; +use loki_rs::presets_api as pa; +use serde_json::{Value, json}; + +fn fixture() -> Value { + serde_json::from_str(include_str!("parity_presets/presets.json")).expect("journal") +} + +fn s(v: &Value) -> &str { + v.as_str().unwrap() +} + +fn strs(v: &Value) -> Vec { + v.as_array() + .map(|a| a.iter().map(|x| s(x).to_string()).collect()) + .unwrap_or_default() +} + +fn cfg(v: &Value) -> HashMap { + v.as_object() + .unwrap() + .iter() + .map(|(k, x)| (k.clone(), s(x).to_string())) + .collect() +} + +#[test] +fn parite_des_fonctions_pures() { + let fx = fixture(); + for c in fx["display_name"].as_array().unwrap() { + assert_eq!( + pa::with_display_name(s(&c["content"]), s(&c["name"])), + s(&c["out"]), + "{c}" + ); + } + for (v, want) in fx["quote"].as_object().unwrap() { + assert_eq!(pa::quote_value(v), s(want), "{v:?}"); + } + let m: BTreeMap = [("Z", "1"), ("A", "deux mots"), ("M", ""), ("Q", "q\"q")] + .iter() + .map(|(k, v)| (k.to_string(), v.to_string())) + .collect(); + assert_eq!(pa::format_env(&m), s(&fx["format_env"])); + let home = Path::new("/h"); + for (n, want) in fx["names"].as_object().unwrap() { + let got = match pa::safe_preset_path(home, n) { + Ok(_) => "ok".to_string(), + Err(e) => e, + }; + assert_eq!(got, s(want), "{n:?}"); + } + for (c, want) in fx["quant"].as_object().unwrap() { + assert_eq!(pa::detect_quant(c), s(want), "{c:?}"); + } + for (c, want) in fx["reasoning"].as_object().unwrap() { + assert_eq!(pa::preset_reasoning(c), s(want), "{c:?}"); + } + for (c, want) in fx["model"].as_object().unwrap() { + assert_eq!(pa::model_from_preset_content(c), s(want), "{c:?}"); + } + for (u, want) in fx["completions"].as_object().unwrap() { + assert_eq!(pa::completions_url(u), s(want), "{u:?}"); + } + for c in fx["external"].as_array().unwrap() { + assert_eq!( + pa::external_preset_content( + s(&c["url"]), + s(&c["model"]), + s(&c["key"]), + s(&c["ctx"]), + c["vision"].as_bool().unwrap() + ), + s(&c["out"]) + ); + } + for (b, want) in fx["api_error"].as_object().unwrap() { + assert_eq!(pa::extract_api_error(b.as_bytes()), s(want), "{b:?}"); + } + for c in fx["join"].as_array().unwrap() { + assert_eq!(pa::join_args(&strs(&c["args"])), s(&c["out"]), "{c}"); + } + for c in fx["set_key"].as_array().unwrap() { + assert_eq!( + pa::preset_set_key(s(&c["content"]), s(&c["key"]), s(&c["val"])), + s(&c["out"]), + "{c}" + ); + } + for c in fx["autoplace"].as_array().unwrap() { + match pa::auto_placement_preset(s(&c["in"])) { + Ok((out, changes)) => { + assert_eq!(s(&c["err"]), "", "{c}"); + assert_eq!(out, s(&c["out"]), "{c}"); + assert_eq!(changes, strs(&c["changes"]), "{c}"); + } + Err(e) => assert_eq!(e, s(&c["err"]), "{c}"), + } + } + for c in fx["bench_fp"].as_array().unwrap() { + assert_eq!( + pa::bench_fingerprint(&cfg(&c["cfg"]), &cfg(&c["cur"])), + s(&c["out"]), + "{c}" + ); + } + let tmp = tempfile::tempdir().unwrap(); + for (p, want) in fx["imposes"].as_object().unwrap() { + let p = p.replace("{{HOME}}", tmp.path().to_str().unwrap()); + assert_eq!( + pa::preset_imposes_engine(tmp.path(), &p), + want.as_bool().unwrap(), + "{p:?}" + ); + } +} + +/// Faux service de complétions : mêmes réponses que celui du test Go. +async fn fake_completions() -> String { + use http_body_util::{BodyExt, Full}; + use hyper::service::service_fn; + use hyper_util::rt::TokioIo; + let l = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap(); + let addr = l.local_addr().unwrap(); + tokio::spawn(async move { + while let Ok((sock, _)) = l.accept().await { + tokio::spawn(async move { + let svc = service_fn(|req: hyper::Request| async move { + let path = req.uri().path().to_string(); + let auth = req + .headers() + .get("authorization") + .map(|v| v.to_str().unwrap().to_string()) + .unwrap_or_default(); + let body = req.into_body().collect().await.unwrap().to_bytes(); + let (code, out): (u16, &str) = if path.starts_with("/ok/") { + if String::from_utf8_lossy(&body).contains("\"max_tokens\":1") + && auth == "Bearer k1" + { + (200, "{}") + } else { + (400, "") + } + } else if path.starts_with("/refus/") { + (401, r#"{"error":{"message":"clé invalide"}}"#) + } else if path.starts_with("/brut/") { + (404, " page absente ") + } else { + (500, "") + }; + let mut r = hyper::Response::new(Full::new(bytes::Bytes::from(out))); + *r.status_mut() = hyper::StatusCode::from_u16(code).unwrap(); + Ok::<_, std::convert::Infallible>(r) + }); + let _ = hyper::server::conn::http1::Builder::new() + .serve_connection(TokioIo::new(sock), svc) + .await; + }); + } + }); + format!("http://{addr}") +} + +fn apply_store(go: &fakego::FakeGo, m: &serde_json::Map) { + for (k, v) in m { + let (b, key) = k.split_once('/').unwrap(); + let mut st = go.store.lock().unwrap(); + let bucket = st.entry(b.to_string()).or_default(); + if s(v).is_empty() { + bucket.remove(key); + } else { + bucket.insert(key.to_string(), s(v).as_bytes().to_vec()); + } + } +} + +fn disk(home: &Path) -> BTreeMap { + let mut out = BTreeMap::new(); + for d in ["presets", "models"] { + if let Ok(rd) = std::fs::read_dir(home.join(d)) { + for e in rd.flatten() { + out.insert( + format!("{d}/{}", e.file_name().to_string_lossy()), + String::from_utf8_lossy(&std::fs::read(e.path()).unwrap()).into_owned(), + ); + } + } + } + out +} + +/// Retire récursivement les champs à zéro : le Go réécrit les mesures restantes +/// par sa struct (champs nuls ajoutés), la façade les garde telles quelles — +/// même lecture pour le Go. +fn sans_zeros(v: &Value) -> Value { + match v { + Value::Object(m) => Value::Object( + m.iter() + .filter(|(_, x)| { + !(x.is_null() + || x.as_f64() == Some(0.0) + || x.as_str() == Some("") + || x.as_bool() == Some(false)) + }) + .map(|(k, x)| (k.clone(), sans_zeros(x))) + .collect(), + ), + Value::Array(a) => Value::Array(a.iter().map(sans_zeros).collect()), + x => x.clone(), + } +} + +fn state(go: &fakego::FakeGo) -> Value { + let mut st = serde_json::Map::new(); + if let Some(b) = go.store.lock().unwrap().get("state") { + for (k, v) in b { + let txt = String::from_utf8_lossy(v).into_owned(); + if k.starts_with("sysprompt:") { + st.insert(k.clone(), json!(txt)); + } else if k == "preset_order" || k == "bench_presets" { + st.insert(k.clone(), serde_json::from_str(&txt).unwrap_or(Value::Null)); + } + } + } + Value::Object(st) +} + +#[tokio::test] +async fn parite_des_routes() { + let fx = fixture(); + let tmp = tempfile::tempdir().unwrap(); + let home = tmp.path().to_path_buf(); + let h = home.to_string_lossy().into_owned(); + let url = fake_completions().await; + let opts = fakego::Opts { + presets: true, + ..Default::default() + }; + let (facade, go, _stop, _) = fakego::start_with(home.clone(), opts).await; + apply_store(&go, fx["seed"].as_object().unwrap()); + for (i, st) in fx["steps"].as_array().unwrap().iter().enumerate() { + if let Some(files) = st.get("files").and_then(Value::as_object) { + for (rel, content) in files { + let p = home.join(rel); + if s(content).is_empty() { + let _ = std::fs::remove_file(&p); + } else { + std::fs::create_dir_all(p.parent().unwrap()).unwrap(); + std::fs::write(&p, s(content)).unwrap(); + } + } + } + if let Some(m) = st.get("store").and_then(Value::as_object) { + apply_store(&go, m); + } + let (m, p) = (s(&st["method"]), s(&st["path"])); + let body = st + .get("body") + .and_then(Value::as_str) + .unwrap_or("") + .replace("{{URL}}", &url); + let (code, b) = fakego::send(facade, m, p, &[], &body).await; + let ctx = format!("étape {i} : {m} {p} {body}"); + let txt = String::from_utf8_lossy(&b) + .replace(&url, "{{URL}}") + .replace(&h, "{{HOME}}"); + assert_eq!( + u64::from(code.as_u16()), + st["status"].as_u64().unwrap(), + "{ctx} : {txt}" + ); + let got: Value = serde_json::from_str(&txt).unwrap(); + assert_eq!(got, st["out"], "{ctx}"); + let d: BTreeMap = disk(&home) + .into_iter() + .map(|(k, v)| (k, v.replace(&url, "{{URL}}"))) + .collect(); + let want_disk: BTreeMap = + serde_json::from_value(st["disk"].clone()).unwrap(); + assert_eq!(d, want_disk, "{ctx} (disque)"); + assert_eq!( + sans_zeros(&state(&go)), + sans_zeros(&st["state"]), + "{ctx} (base)" + ); + } + assert!(go.preset_applies.lock().unwrap().is_empty()); +} + +/// Bascule, enregistrement du preset en service, verrou de l'optimiseur : +/// la façade fait appliquer par le Go. +#[tokio::test] +async fn bascule_et_application() { + let tmp = tempfile::tempdir().unwrap(); + let home = tmp.path().to_path_buf(); + let opts = fakego::Opts { + presets: true, + ..Default::default() + }; + let (facade, go, _stop, _) = fakego::start_with(home.clone(), opts).await; + let dir = home.join("presets"); + std::fs::create_dir_all(&dir).unwrap(); + std::fs::write(dir.join("a.env"), "# NAME=Alpha\nMODEL=a.gguf\n").unwrap(); + std::fs::write(dir.join("b.env"), "# NAME=Beta\nMODEL=b.gguf\n").unwrap(); + std::fs::write(dir.join("refus.env"), "# NAME=Zut\nMODEL=z.gguf\n").unwrap(); + go.set("config", "MODEL", "a.gguf"); + + let (c, b) = fakego::send(facade, "POST", "/api/switch", &[], r#"{"id":"b"}"#).await; + assert_eq!(c, 200); + assert_eq!( + serde_json::from_slice::(&b).unwrap(), + json!({ "ok": true, "preset": "Beta" }) + ); + let (c, b) = fakego::send(facade, "POST", "/api/switch", &[], r#"{"n":1}"#).await; + assert_eq!(c, 200, "{}", String::from_utf8_lossy(&b)); + let (c, b) = fakego::send(facade, "POST", "/api/switch", &[], r#"{"id":"zz"}"#).await; + assert_eq!(c, 404); + assert_eq!( + serde_json::from_slice::(&b).unwrap()["error"], + "preset introuvable (supprimé ou renommé ?)" + ); + let (c, _) = fakego::send(facade, "POST", "/api/switch", &[], r#"{"n":9}"#).await; + assert_eq!(c, 400); + let (c, b) = fakego::send(facade, "POST", "/api/switch", &[], r#"{"id":"refus"}"#).await; + assert_eq!(c, 500); + assert_eq!( + serde_json::from_slice::(&b).unwrap()["error"], + "application refusée" + ); + // Le preset en service (a) modifié : appliqué, moteur relancé par le Go. + let (c, b) = fakego::send( + facade, + "POST", + "/api/preset/save", + &[], + r#"{"id":"a","name":"Alpha","content":"MODEL=a.gguf\nCTX=100000\n","sysprompt":"Bref."}"#, + ) + .await; + assert_eq!(c, 200); + assert_eq!( + serde_json::from_slice::(&b).unwrap()["applied"], + true + ); + assert_eq!(go.get("state", "sysprompt:a").as_deref(), Some("Bref.")); + let applies = go.preset_applies.lock().unwrap().clone(); + let names: Vec<(String, String)> = applies + .iter() + .map(|(p, m)| { + ( + Path::new(p) + .file_name() + .unwrap() + .to_string_lossy() + .into_owned(), + m.clone(), + ) + }) + .collect(); + assert_eq!( + names, + vec![ + ("b.env".to_string(), "switch".to_string()), + ("a.env".to_string(), "switch".to_string()), + ("a.env".to_string(), "save".to_string()), + ] + ); + // Optimisation en cours : refus, rien d'appliqué. + *go.tune_busy.lock().unwrap() = Some("une optimisation tient le moteur".into()); + let (c, b) = fakego::send(facade, "POST", "/api/switch", &[], r#"{"id":"b"}"#).await; + assert_eq!(c, 409); + assert_eq!( + serde_json::from_slice::(&b).unwrap(), + json!({ "ok": false, "tuning": true, "error": "une optimisation tient le moteur" }) + ); + let (c, _) = fakego::send( + facade, + "POST", + "/api/preset/save", + &[], + r#"{"name":"N","content":"X=1"}"#, + ) + .await; + assert_eq!(c, 409); + assert_eq!(go.preset_applies.lock().unwrap().len(), 3); +} + +#[tokio::test] +async fn sans_la_variable_les_presets_restent_au_go() { + let tmp = tempfile::tempdir().unwrap(); + let (facade, _go, _stop) = fakego::start(tmp.path().to_path_buf()).await; + for p in [ + "/api/presets", + "/api/preset/save", + "/api/switch", + "/api/preset/external/test", + ] { + let (_, _, b) = fakego::get(facade, p, &[]).await; + assert_eq!(&b[..], format!("go:{p}").as_bytes()); + } +} diff --git a/rust/loki-rs/tests/proxy.rs b/rust/loki-rs/tests/proxy.rs index 8bdaadb..da89148 100644 --- a/rust/loki-rs/tests/proxy.rs +++ b/rust/loki-rs/tests/proxy.rs @@ -106,6 +106,7 @@ async fn facade(backend: SocketAddr) -> (SocketAddr, oneshot::Sender<()>) { files: None, push: None, mem: None, + presets: false, }; tokio::spawn(loki_rs::proxy::serve(l, cfg, async move { let _ = rx.await;