diff --git a/README.md b/README.md index 1cbf23a..6f3c613 100644 --- a/README.md +++ b/README.md @@ -236,6 +236,7 @@ redémarrages (volume `/data`). Variables d'environnement du conteneur : | `LOKI_MEM` | `rust` : la mémoire (éditeur des pages, index, déplacement entre projets, trackers, mode mémoire, interrupteurs du mode agent) est tenue par la façade Rust ; le chiffrement reste au Go (expérimental, migration en cours) | Go | | `LOKI_SERVE` | `rust` : la ligne de commande de llama-server est composée par la façade Rust (`loki-rs engine serve`) au lieu de `loki serve` ; Strata et l'optimiseur restent au Go (expérimental, migration en cours) | Go | | `LOKI_PRESETS` | `rust` : les presets (liste, éditeur et aperçus, suppression, placement auto, presets externes, bascule) sont tenus par la façade Rust ; l'application au moteur reste au Go (expérimental, migration en cours) | Go | +| `LOKI_SYSTEM` | `rust` : les interrupteurs système — accès internet et Crawl4AI, pilotage du navigateur, clé de complétion, écoute réseau du moteur — sont tenus par la façade Rust ; le navigateur piloté reste au Go (expérimental, migration en cours) | Go | | `LOKI_ENGINE` | `rust` : le pilotage du moteur (démarrage, arrêt, redémarrage, déchargement et rechargement de la VRAM) est tenu par la façade Rust, en conteneur avec son superviseur ; le verrou de l'optimiseur reste au Go (expérimental, migration en cours) | Go | **Accès par nom de domaine.** Sans clé de pilotage, l'API n'accepte que les diff --git a/docs/superpowers/specs/2026-10-15-rust-migration-lot17-systeme-design.md b/docs/superpowers/specs/2026-10-15-rust-migration-lot17-systeme-design.md new file mode 100644 index 0000000..c95b1f3 --- /dev/null +++ b/docs/superpowers/specs/2026-10-15-rust-migration-lot17-systeme-design.md @@ -0,0 +1,92 @@ +# Lot 17 — interrupteurs système tenus par la façade + +## Contexte + +Quatre réglages de l'interface vivaient dans le Go : +- l'accès internet de l'IA : `handleInternet`, `chat_internet.go`, + `web_fetch_go.go` (moteur web, serveur Crawl4AI, clé, joignabilité) ; +- le pilotage du navigateur : `handleComputer`, `chromePath`, + `visionEnabled` ; +- la clé de complétion du moteur : `handleAPIKey`, `backend_config.go`, + `oaiBaseURL` ; +- l'écoute réseau du moteur : `handleNetwork`, `sys_network.go`. + +Lot 17 : la façade tient ces routes quand `LOKI_SYSTEM=rust` (`system.rs`, +expérimental). Sans la variable, rien ne change. + +| Route | Rôle | +|---|---| +| `GET\|POST /api/internet` | interrupteur, moteur web (`go` ou `crawl4ai`), URL et clé du serveur Crawl4AI (la clé n'est jamais rendue, seulement ses quatre derniers caractères), joignabilité | +| `GET\|POST /api/computer` | interrupteur du navigateur, binaire trouvé, vision du modèle | +| `GET\|POST /api/apikey` | clé de complétion : génération, saisie, effacement ; adresse `/v1` à coller dans un client tiers | +| `GET\|POST /api/network` | moteur ouvert au réseau ou à cette machine seulement (`HOST`), adresse de l'endpoint | + +Les réglages restent des clés de la base, écrites comme le Go : une chaîne +vide efface la clé, et l'URL comme les clés ne laissent rien traîner dans la +configuration. + +La joignabilité de Crawl4AI suit `crawlReachable` : +- `/health`, puis `/`, en moins de deux secondes en tout ; +- clé envoyée en `Authorization: Bearer` ; +- moins de 500 = joignable ; +- cache de 30 s par URL, vidé quand l'URL, la clé ou le moteur changent. + +## Restent au Go + +Le Go expose trois routes internes (`facade_system.go`) : +- `POST /internal/v1/web/reach/reset` : le cache de joignabilité des outils + web, encore exécutés par le Go, est vidé à chaque changement ; +- `POST /internal/v1/cdp/shutdown` : le Chromium des outils `browser_*` se + ferme quand l'interrupteur se coupe ; +- `POST /internal/v1/service/action {action, if_active}` : une clé neuve ne + s'applique qu'au redémarrage du moteur. La façade le relance elle-même si + elle le pilote (lot 16), sinon le Go, seulement s'il tourne. + +`GET /internal/v1/tune/guard` (lot 15) refuse un changement de clé pendant une +optimisation. + +Le relais du tunnel vers la façade garde désormais l'hôte et +`X-Forwarded-Proto` du client : l'adresse de la clé de complétion est celle +que le client a tapée, pas celle de la façade. + +## Correction en passant + +`maskAPIKey` paniquait sur une clé d'un seul octet (`k[len(k)-2:]`). +`/api/apikey` tombait alors à chaque lecture tant que la clé restait en place. +Une clé d'un ou deux octets s'affiche maintenant « … », côté Go comme côté +Rust. + +## Écarts assumés + +- Hors Windows, aucun pare-feu n'est piloté : la façade ne tourne qu'en + conteneur Linux, où le Go répondait déjà « inconnu ». +- La clé masquée est coupée sur les octets, comme le Go. Un caractère coupé + devient U+FFFD, octet par octet, comme son encodeur JSON. + +## Parité + +`internal/loki/parity_system_test.go` tire du Go 61 appels de route, avec +après chacun les clés touchées en base et les en-têtes reçus par trois faux +serveurs Crawl4AI (200, 500, 404). Ils couvrent : +- moteurs inconnus, URL avec et sans slashs, injoignable, sans schéma ; +- clés longues, courtes, d'un caractère, résiduelle en configuration ; +- erreurs de type qui laissent passer les autres champs, `null` ; +- vision par projecteur, preset externe ou Strata ; +- clé générée (masquée dans le journal), saisie, de treize octets, repli sur + `config.env`, adresse publique, `X-Forwarded-Proto`, refus pendant une + optimisation ; +- écoute ouverte, fermée, `localhost`, `::1`, port illisible. + +Il tire aussi huit dispositions du dossier de Playwright pour le Chromium +retenu. + +`tests/system.rs` rejoue le tout par une seule façade, cache de +joignabilité compris. Il vérifie de bout en bout ce qui reste au Go : cache +vidé, navigateur fermé, moteur relancé par le Go, ou par le superviseur +quand la façade pilote le moteur. Une mutation de chaque règle fait échouer +les tests. + +Côté Go : +- `TestFacadeSysteme` vérifie le relais et les trois routes internes ; +- `TestFacadeRelaisGardeHote` vérifie l'hôte gardé ; +- `TestMaskAPIKeyCourte` vérifie le masque des clés courtes. diff --git a/internal/loki/backend_config.go b/internal/loki/backend_config.go index 653f195..be0cc4e 100644 --- a/internal/loki/backend_config.go +++ b/internal/loki/backend_config.go @@ -73,11 +73,16 @@ func writeAPIKey(key string) error { } // maskAPIKey renders a key for display: keep the "sk-loki-" prefix and last 4 -// chars, elide the middle. Empty in → empty out. +// chars, elide the middle. Empty in → empty out. Une clé d'un ou deux octets +// n'est pas montrée du tout : k[len(k)-2:] paniquait sur un seul octet, et +// /api/apikey tombait à chaque lecture tant que la clé restait en place. func maskAPIKey(k string) string { if k == "" { return "" } + if len(k) <= 2 { + return "…" + } if len(k) <= 13 { return "…" + k[len(k)-2:] } diff --git a/internal/loki/facade_chat.go b/internal/loki/facade_chat.go index b651fbb..61f9793 100644 --- a/internal/loki/facade_chat.go +++ b/internal/loki/facade_chat.go @@ -145,7 +145,7 @@ var facadeChatRoutes = map[string]bool{ // (facade_dictate.go), les réglages partagés (facade_settings.go), les fichiers // de la discussion (facade_files.go), les notifications (facade_push.go), la // mémoire (facade_mem.go), les presets (facade_presets.go), le pilotage du -// moteur (facade_engine.go). +// moteur (facade_engine.go), les interrupteurs système (facade_system.go). // Posé APRÈS requireWebAuth : seule une requête authentifiée ici part, avec le // jeton interne qui en tient lieu là-bas. func withFacadeChat(path string, h http.HandlerFunc) http.HandlerFunc { @@ -168,6 +168,8 @@ func withFacadeChat(path string, h http.HandlerFunc) http.HandlerFunc { held, unavailable = facadePresets, "presets indisponibles : " case facadeEngineRoutes[path]: held, unavailable = facadeEngine, "pilotage du moteur indisponible : " + case facadeSystemRoutes[path]: + held, unavailable = facadeSystem, "réglages système indisponibles : " default: return h } @@ -186,6 +188,12 @@ func facadeRelayAs(w http.ResponseWriter, r *http.Request, unavailable string) { rp := &httputil.ReverseProxy{ Rewrite: func(pr *httputil.ProxyRequest) { pr.SetURL(&url.URL{Scheme: "http", Host: facadeAddr()}) + // L'hôte et le schéma du client, pas ceux du relais : la clé de + // complétion affiche l'adresse à coller dans un client tiers. + pr.Out.Host = pr.In.Host + if p := pr.In.Header.Get("X-Forwarded-Proto"); p != "" { + pr.Out.Header.Set("X-Forwarded-Proto", p) + } pr.Out.Header.Set(internalTokenHeader, internalToken()) }, FlushInterval: -1, // SSE : chaque trame part aussitôt diff --git a/internal/loki/facade_chat_test.go b/internal/loki/facade_chat_test.go index 2a7485a..3f7e05e 100644 --- a/internal/loki/facade_chat_test.go +++ b/internal/loki/facade_chat_test.go @@ -645,3 +645,116 @@ func TestFacadeMoteur(t *testing.T) { t.Fatalf("arrêt de la dictée : %d", w.Code) } } + +// Une clé de complétion d'un ou deux octets ne fait plus tomber /api/apikey. +func TestMaskAPIKeyCourte(t *testing.T) { + for k, want := range map[string]string{ + "": "", + "k": "…", + "ab": "…", + "abc": "…bc", + "abcdefghijklm": "…lm", + "sk-loki-0123456789abcdef": "sk-loki-0…cdef", + } { + if got := maskAPIKey(k); got != want { + t.Errorf("maskAPIKey(%q) = %q, attendu %q", k, got, want) + } + } +} + +// Les interrupteurs système (LOKI_SYSTEM=rust, lot 17) partent à la façade ; +// l'API interne lui rend ce qui vit ici : cache de joignabilité des outils +// web, navigateur piloté, redémarrage du moteur. +func TestFacadeSysteme(t *testing.T) { + testHome(t) + t.Setenv("LOKI_CONTAINER", "1") + f := demarrerFacade(t) + t.Setenv(facadeChatEnv, "") + local := func(w http.ResponseWriter, r *http.Request) { io.WriteString(w, "go") } + for p := range facadeSystemRoutes { + w := httptest.NewRecorder() + withFacadeChat(p, local)(w, httptest.NewRequest("GET", p, nil)) + if w.Body.String() != "go" { + t.Fatalf("%s servi ailleurs sans LOKI_SYSTEM : %q", p, w.Body.String()) + } + } + t.Setenv(facadeSystemEnv, "rust") + for p := range facadeSystemRoutes { + w := httptest.NewRecorder() + withFacadeChat(p, local)(w, httptest.NewRequest("POST", p, strings.NewReader("{}"))) + if w.Body.String() != f.reply { + t.Fatalf("%s : réponse %q", p, w.Body.String()) + } + } + if n := len(f.recu()); n != len(facadeSystemRoutes) { + t.Fatalf("reçu par la façade : %d appels", n) + } + + // Cache de joignabilité vidé. + reachMu.Lock() + reachURL = "http://ancien" + reachMu.Unlock() + w := httptest.NewRecorder() + handleInternalReachReset(w, httptest.NewRequest("POST", "/internal/v1/web/reach/reset", nil)) + reachMu.Lock() + vide := reachURL == "" + reachMu.Unlock() + if w.Code != 204 || !vide { + t.Fatalf("reach/reset : %d, cache vidé : %v", w.Code, vide) + } + w = httptest.NewRecorder() + handleInternalCDPShutdown(w, httptest.NewRequest("POST", "/internal/v1/cdp/shutdown", nil)) + if w.Code != 204 { + t.Fatalf("cdp/shutdown : %d", w.Code) + } + + // Redémarrage : seulement si le moteur tourne quand if_active est posé. + var actions []string + old := internalServiceAction + t.Cleanup(func() { internalServiceAction = old }) + internalServiceAction = func(a string) error { + actions = append(actions, a) + if a == "start" { + return errTuneBusy + } + return nil + } + for _, c := range []struct { + body, want string + code int + }{ + {`{"action":"restart","if_active":true}`, `{"error":"","ran":false}`, 200}, + {`{"action":"restart"}`, `{"error":"","ran":true}`, 200}, + {`{"action":"start"}`, `{"error":"` + errTuneBusy.Error() + `","ran":true}`, 200}, + {`{"action":"reload"}`, `{"error":"action inconnue : reload"}`, 400}, + {`{"action":`, `{"error":"unexpected EOF"}`, 400}, + } { + w := httptest.NewRecorder() + handleInternalServiceAction(w, httptest.NewRequest("POST", "/internal/v1/service/action", strings.NewReader(c.body))) + if got := strings.TrimSpace(w.Body.String()); w.Code != c.code || got != c.want { + t.Fatalf("%s : %d %s", c.body, w.Code, got) + } + } + if strings.Join(actions, ",") != "restart,start" { + t.Fatalf("actions : %v", actions) + } +} + +// Le relais vers la façade garde l'hôte et le schéma du client : la clé de +// complétion y calcule l'adresse à coller dans un client tiers. +func TestFacadeRelaisGardeHote(t *testing.T) { + var host, proto string + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + host, proto = r.Host, r.Header.Get("X-Forwarded-Proto") + })) + defer srv.Close() + t.Setenv(internalTokenEnv, jetonTest) + t.Setenv(facadeAddrEnv, strings.TrimPrefix(srv.URL, "http://")) + r := httptest.NewRequest("GET", "/api/apikey", nil) + r.Host = "loki.exemple.fr" + r.Header.Set("X-Forwarded-Proto", "https") + facadeRelayAs(httptest.NewRecorder(), r, "") + if host != "loki.exemple.fr" || proto != "https" { + t.Fatalf("hôte %q, schéma %q", host, proto) + } +} diff --git a/internal/loki/facade_system.go b/internal/loki/facade_system.go new file mode 100644 index 0000000..4682466 --- /dev/null +++ b/internal/loki/facade_system.go @@ -0,0 +1,98 @@ +package loki + +import ( + "encoding/json" + "net/http" + "os" + "strings" +) + +// facade_system.go — interrupteurs système tenus par la façade Rust +// (LOKI_SYSTEM=rust, lot 17 de la migration ; expérimental, le Go reste le +// défaut) : accès internet (/api/internet), pilotage du navigateur +// (/api/computer), clé de complétion (/api/apikey) et écoute réseau du moteur +// (/api/network). +// +// Les réglages restent des clés de la base. Ce qui vit dans ce processus reste +// ici, derrière l'API interne : +// +// - le cache de joignabilité de Crawl4AI des outils web, vidé quand l'URL, la +// clé ou le moteur changent (/internal/v1/web/reach/reset) ; +// - le Chromium piloté par les outils browser_*, fermé quand l'interrupteur +// se coupe (/internal/v1/cdp/shutdown) ; +// - le redémarrage du moteur qui applique une nouvelle clé, quand la façade +// ne pilote pas le moteur (/internal/v1/service/action). + +const facadeSystemEnv = "LOKI_SYSTEM" + +// facadeSystem : la façade tient-elle les interrupteurs système ? +func facadeSystem() bool { + return strings.EqualFold(strings.TrimSpace(os.Getenv(facadeSystemEnv)), "rust") && + facadeAddr() != "" && internalToken() != "" +} + +// facadeSystemRoutes : les routes que la façade sert alors (system.rs). +var facadeSystemRoutes = map[string]bool{ + "/api/internet": true, + "/api/computer": true, + "/api/apikey": true, + "/api/network": true, +} + +// handleInternalReachReset : POST → 204 — le prochain tour reteste Crawl4AI. +func handleInternalReachReset(w http.ResponseWriter, r *http.Request) { + if r.Method != http.MethodPost { + sendJSON(w, http.StatusMethodNotAllowed, map[string]any{"error": "POST attendu"}) + return + } + reachMu.Lock() + reachURL = "" + reachMu.Unlock() + w.WriteHeader(http.StatusNoContent) +} + +// handleInternalCDPShutdown : POST → 204 — ferme le navigateur piloté. +func handleInternalCDPShutdown(w http.ResponseWriter, r *http.Request) { + if r.Method != http.MethodPost { + sendJSON(w, http.StatusMethodNotAllowed, map[string]any{"error": "POST attendu"}) + return + } + cdpShutdown() + w.WriteHeader(http.StatusNoContent) +} + +// internalServiceAction : serviceAction, variable pour les tests. +var internalServiceAction = serviceAction + +// handleInternalServiceAction : POST {action, if_active} → 200 {ran, error}. +// if_active : seulement si le moteur tourne (une clé neuve ne s'applique qu'au +// redémarrage, inutile de démarrer un moteur arrêté pour elle). +func handleInternalServiceAction(w http.ResponseWriter, r *http.Request) { + if r.Method != http.MethodPost { + sendJSON(w, http.StatusMethodNotAllowed, map[string]any{"error": "POST attendu"}) + return + } + var in struct { + Action string `json:"action"` + IfActive bool `json:"if_active"` + } + if err := json.NewDecoder(r.Body).Decode(&in); err != nil { + sendJSON(w, http.StatusBadRequest, map[string]any{"error": err.Error()}) + return + } + switch in.Action { + case "start", "stop", "restart": + default: + sendJSON(w, http.StatusBadRequest, map[string]any{"error": "action inconnue : " + in.Action}) + return + } + if in.IfActive && !serviceIsActive() { + sendJSON(w, http.StatusOK, map[string]any{"ran": false, "error": ""}) + return + } + msg := "" + if err := internalServiceAction(in.Action); err != nil { + msg = err.Error() + } + sendJSON(w, http.StatusOK, map[string]any{"ran": true, "error": msg}) +} diff --git a/internal/loki/parity_system_test.go b/internal/loki/parity_system_test.go new file mode 100644 index 0000000..3501d23 --- /dev/null +++ b/internal/loki/parity_system_test.go @@ -0,0 +1,294 @@ +package loki + +import ( + "encoding/json" + "net/http" + "net/http/httptest" + "os" + "path/filepath" + "regexp" + "strings" + "sync" + "testing" +) + +// parity_system_test.go — journal de référence des interrupteurs système +// (lot 17 de la migration Rust) : accès internet et Crawl4AI (moteur, URL, +// clé jamais rendue, joignabilité et son cache), pilotage du navigateur et +// vision, clé de complétion (génération, saisie, effacement, repli sur +// config.env, adresse à coller dans un client tiers, refus pendant une +// optimisation) et écoute réseau du moteur. Après chaque appel : les clés +// touchées en base et les en-têtes reçus par les faux serveurs Crawl4AI. +// Rangé dans rust/loki-rs/tests/parity_system/system.json ; tests/system.rs +// le rejoue par la façade. +// +// LOKI_PARITY_UPDATE=1 go test ./internal/loki -run TestPariteSysteme + +const paritySystemFile = "../../rust/loki-rs/tests/parity_system/system.json" + +type paritySystemStep struct { + Name string `json:"name"` + Config map[string]string `json:"config,omitempty"` // posé avant l'appel ; "" = supprimé + State map[string]string `json:"state,omitempty"` + Tune bool `json:"tune,omitempty"` + Method string `json:"method"` + Path string `json:"path"` + Body string `json:"body,omitempty"` + Headers map[string]string `json:"headers,omitempty"` + Status int `json:"status"` + Out any `json:"out"` + After map[string]string `json:"after"` + Probes []string `json:"probes"` // « chemin Authorization » reçus depuis l'appel précédent +} + +// paritySystemKeys : ce que les routes touchent, relu après chaque appel. +var paritySystemKeys = []string{ + "config/WEB_ENGINE", "config/CRAWL4AI_URL", "config/CRAWL4AI_KEY", "config/API_KEY", "config/HOST", + "state/internet", "state/computer", "state/crawl_key", "state/api_key", "state/public_url", +} + +var ( + parityKeyRe = regexp.MustCompile(`sk-loki-[0-9a-f]{48}`) + parityMaskedRe = regexp.MustCompile(`sk-loki-[0-9a-f]…[0-9a-f]{4}`) +) + +func TestPariteSysteme(t *testing.T) { + home := testHome(t) + t.Setenv("LOKI_CONTAINER", "1") + t.Setenv("LOKI_CHROME", "/opt/faux/chrome") + + var mu sync.Mutex + var probes []string + fake := func(code int) *httptest.Server { + return httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + mu.Lock() + probes = append(probes, strings.TrimSpace(r.URL.Path+" "+r.Header.Get("Authorization"))) + mu.Unlock() + w.WriteHeader(code) + })) + } + ok, down, notFound := fake(200), fake(500), fake(404) + defer ok.Close() + defer down.Close() + defer notFound.Close() + + mask := func(s string) string { + s = strings.ReplaceAll(s, ok.URL, "{{OK}}") + s = strings.ReplaceAll(s, down.URL, "{{DOWN}}") + s = strings.ReplaceAll(s, notFound.URL, "{{NOTFOUND}}") + s = strings.ReplaceAll(s, home, "{{HOME}}") + if ip := localIP(); ip != "" { + s = strings.ReplaceAll(s, "http://"+ip+":", "http://{{IP}}:") + } + s = parityKeyRe.ReplaceAllString(s, "{{KEY}}") + return parityMaskedRe.ReplaceAllString(s, "{{MASKED}}") + } + unmask := func(s string) string { + s = strings.ReplaceAll(s, "{{OK}}", ok.URL) + s = strings.ReplaceAll(s, "{{DOWN}}", down.URL) + return strings.ReplaceAll(s, "{{NOTFOUND}}", notFound.URL) + } + handlers := map[string]http.HandlerFunc{ + "/api/internet": handleInternet, + "/api/computer": handleComputer, + "/api/apikey": handleAPIKey, + "/api/network": handleNetwork, + } + run := func(st paritySystemStep) paritySystemStep { + for k, v := range st.Config { + if err := SetConfigKey(k, unmask(v)); err != nil { + t.Fatal(err) + } + } + for k, v := range st.State { + if err := putStr(bkState, k, unmask(v)); err != nil { + t.Fatal(err) + } + } + var lock *tuneLock + if st.Tune { + l, err := tuneLockAcquire("web", false) + if err != nil { + t.Fatal(err) + } + lock = l + } + mu.Lock() + probes = nil + mu.Unlock() + req := httptest.NewRequest(st.Method, st.Path, strings.NewReader(unmask(st.Body))) + req.Host = "loki.local:8090" + for k, v := range st.Headers { + req.Header.Set(k, v) + } + rec := httptest.NewRecorder() + handlers[st.Path](rec, req) + if lock != nil { + lock.release() + } + st.Status = rec.Code + var out any + if err := json.Unmarshal([]byte(mask(rec.Body.String())), &out); err != nil { + t.Fatalf("%s : corps illisible %q", st.Name, rec.Body.String()) + } + st.Out = out + st.After = map[string]string{} + for _, k := range paritySystemKeys { + b, key, _ := strings.Cut(k, "/") + if v := getStr(b, key); v != "" { + st.After[k] = mask(v) + } + } + mu.Lock() + st.Probes = append([]string{}, probes...) + mu.Unlock() + for i := range st.Probes { + st.Probes[i] = mask(st.Probes[i]) + } + return st + } + + var steps []paritySystemStep + for _, st := range []paritySystemStep{ + // --- accès internet --- + {Name: "internet par défaut", Method: "GET", Path: "/api/internet"}, + {Name: "allumé, moteur intégré", Method: "POST", Path: "/api/internet", Body: `{"enabled":true}`}, + {Name: "moteur crawl4ai sans URL", Method: "POST", Path: "/api/internet", Body: `{"engine":" CRAWL4AI "}`}, + {Name: "moteur inconnu", Method: "POST", Path: "/api/internet", Body: `{"engine":"bing"}`}, + {Name: "moteur inconnu vide", Method: "POST", Path: "/api/internet", Body: `{"engine":" "}`}, + {Name: "URL joignable", Method: "POST", Path: "/api/internet", Body: `{"url":" {{OK}}/// "}`}, + {Name: "relu dans le cache", Method: "GET", Path: "/api/internet"}, + {Name: "clé posée", Method: "POST", Path: "/api/internet", Body: `{"key":" secret-clé-crawl "}`}, + {Name: "clé courte", Method: "POST", Path: "/api/internet", Body: `{"key":"abcd"}`}, + {Name: "clé d'un caractère", Method: "POST", Path: "/api/internet", Body: `{"key":"é"}`}, + {Name: "clé résiduelle en config", Config: map[string]string{"CRAWL4AI_KEY": "vieille"}, + Method: "POST", Path: "/api/internet", Body: `{"key":"neuve12345"}`}, + {Name: "URL en panne", Method: "POST", Path: "/api/internet", Body: `{"url":"{{DOWN}}"}`}, + {Name: "URL 404", Method: "POST", Path: "/api/internet", Body: `{"url":"{{NOTFOUND}}"}`}, + {Name: "URL injoignable", Method: "POST", Path: "/api/internet", Body: `{"url":"http://127.0.0.1:1"}`}, + {Name: "URL sans schéma", Method: "POST", Path: "/api/internet", Body: `{"url":"exemple"}`}, + {Name: "type faux, le reste passe", Method: "POST", Path: "/api/internet", Body: `{"enabled":"oui","url":"{{OK}}"}`}, + {Name: "corps illisible", Method: "POST", Path: "/api/internet", Body: `{"enabled":false`}, + {Name: "éteint, clé effacée", Method: "POST", Path: "/api/internet", Body: `{"enabled":false,"key":""}`}, + {Name: "null ne touche à rien", Method: "POST", Path: "/api/internet", Body: `{"enabled":null,"url":null,"engine":null}`}, + {Name: "URL vidée, moteur choisi gardé", Method: "POST", Path: "/api/internet", Body: `{"url":"","enabled":true}`}, + {Name: "moteur go", Method: "POST", Path: "/api/internet", Body: `{"engine":"Go"}`}, + {Name: "moteur non choisi, URL héritée", Config: map[string]string{"WEB_ENGINE": "", "CRAWL4AI_URL": "{{OK}}"}, + Method: "GET", Path: "/api/internet"}, + {Name: "moteur illisible en config", Config: map[string]string{"WEB_ENGINE": "autre", "CRAWL4AI_URL": ""}, + Method: "GET", Path: "/api/internet"}, + {Name: "URL en config avec slashs", Config: map[string]string{"WEB_ENGINE": "crawl4ai", "CRAWL4AI_URL": " {{DOWN}}// "}, + Method: "GET", Path: "/api/internet"}, + {Name: "internet en PUT", Method: "PUT", Path: "/api/internet", Body: `{"enabled":true}`}, + + // --- navigateur --- + {Name: "navigateur par défaut", Method: "GET", Path: "/api/computer"}, + {Name: "navigateur allumé", Method: "POST", Path: "/api/computer", Body: `{"on":true}`}, + {Name: "navigateur, corps illisible", Method: "POST", Path: "/api/computer", Body: `oui`}, + {Name: "vision par projecteur", Config: map[string]string{"MMPROJ": " mmproj.gguf "}, + Method: "GET", Path: "/api/computer"}, + {Name: "vision externe", Config: map[string]string{"MMPROJ": "", "EXTERNAL": "1", "EXTERNAL_VISION": "1"}, + Method: "GET", Path: "/api/computer"}, + {Name: "externe sans vision", Config: map[string]string{"EXTERNAL_VISION": "0", "MMPROJ": "m.gguf"}, + Method: "GET", Path: "/api/computer"}, + {Name: "vision strata", Config: map[string]string{"EXTERNAL": "", "ENGINE": "strata"}, + Method: "GET", Path: "/api/computer"}, + {Name: "strata sans vision", Config: map[string]string{"STRATA_VISION": "0", "MMPROJ": ""}, + Method: "GET", Path: "/api/computer"}, + {Name: "navigateur éteint", Config: map[string]string{"ENGINE": "", "STRATA_VISION": ""}, + Method: "POST", Path: "/api/computer", Body: `{"on":false}`}, + + // --- clé de complétion --- + {Name: "clé par défaut", Method: "GET", Path: "/api/apikey"}, + {Name: "clé générée", Method: "POST", Path: "/api/apikey", Body: `{"action":"generate"}`}, + {Name: "clé relue", Method: "GET", Path: "/api/apikey", Headers: map[string]string{"X-Forwarded-Proto": "HTTPS, http"}}, + {Name: "clé saisie", Method: "POST", Path: "/api/apikey", Body: `{"action":"set","key":" ma-clé-perso-é "}`}, + {Name: "clé saisie courte", Method: "POST", Path: "/api/apikey", Body: `{"action":"set","key":"abc"}`}, + {Name: "clé de 13 octets", Method: "POST", Path: "/api/apikey", Body: `{"action":"set","key":"abcdefghijklm"}`}, + {Name: "clé effacée", Method: "POST", Path: "/api/apikey", Body: `{"action":"clear"}`}, + {Name: "repli sur config.env", Config: map[string]string{"API_KEY": " résiduelle-config "}, + Method: "GET", Path: "/api/apikey", Headers: map[string]string{"X-Forwarded-Proto": "ftp"}}, + {Name: "adresse publique", State: map[string]string{"public_url": "https://loki.exemple.fr"}, + Method: "POST", Path: "/api/apikey", Body: `{"action":"set","key":"k"}`}, + {Name: "action inconnue", Method: "POST", Path: "/api/apikey", Body: `{"action":"rotate"}`}, + {Name: "action vide", Method: "POST", Path: "/api/apikey", Body: `{}`}, + {Name: "corps vide", Method: "POST", Path: "/api/apikey", Body: ``}, + {Name: "corps illisible", Method: "POST", Path: "/api/apikey", Body: `{"action":`}, + {Name: "type faux", Method: "POST", Path: "/api/apikey", Body: `{"action":1}`}, + {Name: "optimisation en cours", Tune: true, Method: "POST", Path: "/api/apikey", Body: `{"action":"clear"}`}, + {Name: "lecture pendant une optimisation", Tune: true, Method: "GET", Path: "/api/apikey"}, + + // --- réseau --- + {Name: "réseau par défaut", Method: "GET", Path: "/api/network"}, + {Name: "réseau fermé", Method: "POST", Path: "/api/network", Body: `{"exposed":false}`}, + {Name: "réseau ouvert", Method: "POST", Path: "/api/network", Body: `{"exposed":true}`}, + {Name: "réseau sans champ", Method: "POST", Path: "/api/network", Body: `{}`}, + {Name: "réseau illisible", Method: "POST", Path: "/api/network", Body: `{"exposed":1}`}, + {Name: "réseau corps vide", Method: "POST", Path: "/api/network", Body: ``}, + {Name: "HOST localhost", Config: map[string]string{"HOST": " LocalHost ", "PORT": "9090"}, Method: "GET", Path: "/api/network"}, + {Name: "HOST ::1", Config: map[string]string{"HOST": "::1", "PORT": "abc"}, Method: "GET", Path: "/api/network"}, + {Name: "HOST d'une carte", Config: map[string]string{"HOST": "192.168.1.5", "PORT": ""}, Method: "GET", Path: "/api/network"}, + } { + steps = append(steps, run(st)) + } + + // --- le Chromium de Playwright, le plus récent --- + type chromiumCase struct { + Files []string `json:"files"` // relatifs à la racine ; « / » final = dossier + Out string `json:"out"` + } + var chromium []chromiumCase + for _, files := range [][]string{ + {"chromium-1000/chrome-linux/chrome", "chromium-1200/chrome-linux/chrome"}, + {"chromium-1000/chrome-linux/chrome", "chromium-1200/chrome-linux/headless_shell"}, + {"chromium-1200/chrome-linux/headless_shell", "chromium-1200/chrome-linux/chrome"}, + {"chromium-1000/chrome-linux/chrome", "chromium_headless_shell-1100/chrome-linux/headless_shell"}, + {"chromium-1000/chrome-linux/chrome", "chromium-1300/chrome-linux/chrome/"}, + {"chromiumX/chrome-linux/chrome", "chromium-1/chrome-linux/chrome"}, + {"firefox-1/chrome-linux/chrome", "chromium-/chrome-linux/chrome"}, + {"chromium-9/chrome-linux/chrome", "chromium-10/chrome-linux/chrome"}, + } { + root := t.TempDir() + for _, f := range files { + p := filepath.Join(root, f) + if strings.HasSuffix(f, "/") { + if err := os.MkdirAll(p, 0o755); err != nil { + t.Fatal(err) + } + continue + } + if err := os.MkdirAll(filepath.Dir(p), 0o755); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(p, nil, 0o755); err != nil { + t.Fatal(err) + } + } + t.Setenv("PLAYWRIGHT_BROWSERS_PATH", root) + chromium = append(chromium, chromiumCase{Files: files, + Out: strings.TrimPrefix(playwrightChromium(), root+"/")}) + } + + got, err := parityJSON(map[string]any{"steps": steps, "chromium": chromium}) + if err != nil { + t.Fatal(err) + } + if os.Getenv("LOKI_PARITY_UPDATE") == "1" { + if err := os.MkdirAll(filepath.Dir(paritySystemFile), 0o755); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(paritySystemFile, got, 0o644); err != nil { + t.Fatal(err) + } + return + } + want, err := os.ReadFile(paritySystemFile) + if err != nil { + t.Fatalf("journal de référence absent (%v) : LOKI_PARITY_UPDATE=1 pour le créer", err) + } + if string(got) != string(want) { + t.Errorf("les interrupteurs système ne produisent plus le journal rangé dans %s.\n"+ + "Si c'est voulu : LOKI_PARITY_UPDATE=1 go test ./internal/loki -run TestPariteSysteme,\n"+ + "puis cargo test --test system dit si le port Rust suit.\n--- obtenu ---\n%s", paritySystemFile, got) + } +} diff --git a/internal/loki/web_internal.go b/internal/loki/web_internal.go index 2274d93..66d09cf 100644 --- a/internal/loki/web_internal.go +++ b/internal/loki/web_internal.go @@ -119,6 +119,12 @@ func withInternalAPI(next http.Handler) http.Handler { handleInternalChatGenerating(w, r) case "/internal/v1/dictate/stop": handleInternalDictateStop(w, r) + case "/internal/v1/web/reach/reset": + handleInternalReachReset(w, r) + case "/internal/v1/cdp/shutdown": + handleInternalCDPShutdown(w, r) + case "/internal/v1/service/action": + handleInternalServiceAction(w, r) default: http.NotFound(w, r) } diff --git a/rust/loki-rs/src/api.rs b/rust/loki-rs/src/api.rs index 0487d3d..9b7866b 100644 --- a/rust/loki-rs/src/api.rs +++ b/rust/loki-rs/src/api.rs @@ -44,6 +44,8 @@ pub struct Ctx { pub presets: bool, /// Pilotage du moteur tenu par la façade (`LOKI_ENGINE=rust`). pub engine_ctl: Option>, + /// Interrupteurs système tenus par la façade (`LOKI_SYSTEM=rust`). + pub system: Option>, /// Jeton de l'API interne : une requête qui le porte vient du Go. pub internal_token: String, } @@ -61,6 +63,7 @@ impl Ctx { || self.mem.is_some() || self.presets || self.engine_ctl.is_some() + || self.system.is_some() } /// La requête vient du Go (jeton interne valide) : déjà authentifiée par diff --git a/rust/loki-rs/src/enginectl.rs b/rust/loki-rs/src/enginectl.rs index b3eb06a..d112773 100644 --- a/rust/loki-rs/src/enginectl.rs +++ b/rust/loki-rs/src/enginectl.rs @@ -343,7 +343,7 @@ pub async fn service_action(ctx: &Ctx, ctl: &EngineCtl, action: &str) -> Result< } /// `serviceIsActive` en conteneur : le PID du fichier vit. -fn service_is_active(ctx: &Ctx) -> bool { +pub(crate) fn service_is_active(ctx: &Ctx) -> bool { pid_alive(crate::engine::read_pid(&ctx.engine.pid())) } diff --git a/rust/loki-rs/src/lib.rs b/rust/loki-rs/src/lib.rs index f1c81c1..25bd3c4 100644 --- a/rust/loki-rs/src/lib.rs +++ b/rust/loki-rs/src/lib.rs @@ -36,6 +36,7 @@ pub mod serve; pub mod settings; pub mod store; pub mod sysinfo; +pub mod system; pub mod tools; pub mod ui; pub mod update; diff --git a/rust/loki-rs/src/main.rs b/rust/loki-rs/src/main.rs index 506136e..16074f8 100644 --- a/rust/loki-rs/src/main.rs +++ b/rust/loki-rs/src/main.rs @@ -151,6 +151,9 @@ fn serve() -> ExitCode { // LOKI_ENGINE=rust : le pilotage du moteur (démarrage, arrêt, // déchargement de la VRAM), en conteneur avec le superviseur Rust // seulement — sous systemd, le Go pilote son unité. + // LOKI_SYSTEM=rust : les interrupteurs système (accès internet, + // navigateur, clé de complétion, écoute réseau du moteur). + let system_rust = held_by_facade("LOKI_SYSTEM"); let engine_rust = held_by_facade("LOKI_ENGINE") && match loki_rs::enginectl::usable( std::env::var("LOKI_CONTAINER").ok().as_deref(), @@ -178,6 +181,7 @@ fn serve() -> ExitCode { ("LOKI_MEM", mem_rust), ("LOKI_PRESETS", presets_rust), ("LOKI_ENGINE", engine_rust), + ("LOKI_SYSTEM", system_rust), ]; for (var, on) in held { if on { @@ -280,6 +284,7 @@ fn serve() -> ExitCode { mem: mem_rust.then(|| std::sync::Arc::new(loki_rs::memory::Memory::new(mem_home))), presets: presets_rust, engine_ctl, + system: system_rust.then(|| std::sync::Arc::new(loki_rs::system::System::new())), }; let res = proxy::serve(listener, cfg, shutdown_signal()).await; eprintln!("[loki-rs] arrêt demandé — arrêt du serveur Go"); diff --git a/rust/loki-rs/src/proxy.rs b/rust/loki-rs/src/proxy.rs index ee26fb6..6e58743 100644 --- a/rust/loki-rs/src/proxy.rs +++ b/rust/loki-rs/src/proxy.rs @@ -82,6 +82,9 @@ pub struct Config { /// Pilotage du moteur tenu par la façade (`LOKI_ENGINE=rust`) : /// démarrage, arrêt, déchargement de la VRAM ; `None` : au Go. pub engine_ctl: Option>, + /// Interrupteurs système (internet, navigateur, clé de complétion, + /// réseau) tenus par la façade (`LOKI_SYSTEM=rust`) ; `None` : au Go. + pub system: Option>, } struct Proxy { @@ -117,6 +120,7 @@ pub async fn serve( mem: cfg.mem, presets: cfg.presets, engine_ctl: cfg.engine_ctl, + system: cfg.system, internal_token: cfg.internal_token.clone(), }, }); @@ -204,6 +208,11 @@ impl Proxy { { return chatapi::serve(&self.ctx, chat, req, from_go).await; } + if let Some(s) = &self.ctx.system + && crate::system::handles(path) + { + return crate::system::serve(&self.ctx, s, req, from_go).await; + } if let Some(e) = &self.ctx.engine_ctl && crate::enginectl::handles(path) { diff --git a/rust/loki-rs/src/system.rs b/rust/loki-rs/src/system.rs new file mode 100644 index 0000000..018b459 --- /dev/null +++ b/rust/loki-rs/src/system.rs @@ -0,0 +1,664 @@ +//! Interrupteurs système tenus par la façade (lot 17 de la migration, +//! `LOKI_SYSTEM=rust` ; expérimental, le Go reste le défaut) : +//! +//! - `/api/internet` — port de `handleInternet` (`web_api.go`) et de +//! `chat_internet.go` / `web_fetch_go.go` : moteur web, serveur Crawl4AI et +//! sa clé (jamais rendue), joignabilité avec son cache de 30 s ; +//! - `/api/computer` — `handleComputer`, `chromePath`, `visionEnabled` ; +//! - `/api/apikey` — `handleAPIKey` et `backend_config.go` : clé de +//! complétion, repli sur `config.env`, adresse à coller dans un client tiers +//! (`oaiBaseURL`) ; +//! - `/api/network` — `handleNetwork`, `sys_network.go` (en conteneur Linux, +//! aucun pare-feu à piloter : « inconnu »). +//! +//! Les réglages restent des clés de la base. Ce qui vit dans le processus Go +//! reste au Go : son cache de joignabilité (vidé par +//! `/internal/v1/web/reach/reset`), le Chromium des outils `browser_*` +//! (`/internal/v1/cdp/shutdown`) et le redémarrage du moteur quand la façade ne +//! le pilote pas (`/internal/v1/service/action`). + +use std::time::{Duration, Instant}; + +use http_body_util::{BodyExt, Limited}; +use hyper::body::Incoming; +use hyper::header::HOST; +use hyper::{Method, Request, Response, StatusCode}; +use serde_json::{Value, json}; + +use crate::api::Ctx; +use crate::auth::json as send_json; +use crate::gojson::{self, Kind}; +use crate::proxy::Body; +use crate::serve::{self, Cfg}; +use crate::store::{BK_CONFIG, BK_STATE, Store}; + +/// Moteur web intégré (`engineGo`). +pub const ENGINE_GO: &str = "go"; +/// Serveur Crawl4AI (`engineCrawl`). +pub const ENGINE_CRAWL: &str = "crawl4ai"; +/// Durée de vie du cache de joignabilité (`reachTTL`). +pub const REACH_TTL: Duration = Duration::from_secs(30); + +/// État tenu par la façade pour ce lot. +pub struct System { + /// Binaire du navigateur imposé (tests) ; `None` : [`chrome_path`]. + pub chrome: Option, + reach: tokio::sync::Mutex, + client: reqwest::Client, +} + +#[derive(Default)] +struct Reach { + ok: bool, + url: String, + when: Option, +} + +impl Default for System { + fn default() -> Self { + Self::new() + } +} + +impl System { + pub fn new() -> Self { + crate::chat::engine::install_crypto(); + System { + chrome: None, + reach: tokio::sync::Mutex::new(Reach::default()), + client: reqwest::Client::builder().build().unwrap_or_default(), + } + } + + /// Le cache de joignabilité est périmé : ici, et dans le Go (outils web). + async fn reach_reset(&self, store: &Store) { + self.reach.lock().await.url.clear(); + if let Err(e) = store + .internal_post("/internal/v1/web/reach/reset", &json!({})) + .await + { + eprintln!("[internet] cache de joignabilité du Go : {e}"); + } + } + + /// `crawlReachable` : le serveur Crawl4AI répond-il (moins de 500 sur + /// `/health` ou `/`, deux secondes en tout) ? Le moteur intégré n'a rien à + /// joindre. + async fn reachable(&self, cfg: &Cfg, key: &str) -> bool { + if web_engine(cfg) == ENGINE_GO { + return true; + } + let base = crawl4ai_url(cfg); + if base.is_empty() { + return false; + } + let mut r = self.reach.lock().await; + if base == r.url && r.when.is_some_and(|w| w.elapsed() < REACH_TTL) { + return r.ok; + } + let probe = async { + for path in ["/health", "/"] { + let mut rq = self.client.get(format!("{base}{path}")); + if !key.is_empty() { + rq = rq.header("Authorization", format!("Bearer {key}")); + } + if let Ok(resp) = rq.send().await + && resp.status().as_u16() < 500 + { + return true; + } + } + false + }; + let ok = tokio::time::timeout(Duration::from_secs(2), probe) + .await + .unwrap_or(false); + *r = Reach { + ok, + url: base, + when: Some(Instant::now()), + }; + ok + } +} + +// --- Fonctions pures --------------------------------------------------------- + +/// `crawl4aiURL` : sans slash final, vide si non configurée. +pub fn crawl4ai_url(cfg: &Cfg) -> String { + serve::trim(serve::get(cfg, "CRAWL4AI_URL")) + .trim_end_matches('/') + .to_string() +} + +/// `webEngine` : le moteur choisi ; sinon Crawl4AI si une URL est déjà là +/// (installation d'avant le choix), le moteur intégré par défaut. +pub fn web_engine(cfg: &Cfg) -> &'static str { + match serve::trim(serve::get(cfg, "WEB_ENGINE")) + .to_lowercase() + .as_str() + { + ENGINE_GO => return ENGINE_GO, + ENGINE_CRAWL => return ENGINE_CRAWL, + _ => {} + } + if crawl4ai_url(cfg).is_empty() { + ENGINE_GO + } else { + ENGINE_CRAWL + } +} + +/// `internetEnabled` : interrupteur armé ET moteur utilisable. +pub fn internet_enabled(cfg: &Cfg, switch_on: bool) -> bool { + if web_engine(cfg) == ENGINE_CRAWL && crawl4ai_url(cfg).is_empty() { + return false; + } + switch_on +} + +/// Indice de la clé Crawl4AI : ses quatre derniers caractères. +pub fn key_hint(key: &str) -> String { + let runes: Vec = key.chars().collect(); + if runes.len() > 4 { + format!("…{}", runes[runes.len() - 4..].iter().collect::()) + } else if !key.is_empty() { + "…".into() + } else { + String::new() + } +} + +/// Octets → texte comme l'encodeur JSON du Go : chaque octet invalide devient +/// U+FFFD. +fn go_lossy(mut b: &[u8]) -> String { + let mut out = String::new(); + loop { + match std::str::from_utf8(b) { + Ok(s) => { + out.push_str(s); + return out; + } + Err(e) => { + let (ok, rest) = b.split_at(e.valid_up_to()); + out.push_str(std::str::from_utf8(ok).unwrap_or_default()); + let bad = e.error_len().unwrap_or(rest.len()); + out.extend(std::iter::repeat_n('\u{FFFD}', bad)); + b = &rest[bad..]; + } + } + } +} + +/// `maskAPIKey` : préfixe « sk-loki-? » et quatre derniers octets ; une clé de +/// treize octets au plus ne montre que les deux derniers, une clé d'un ou deux +/// octets rien. +pub fn mask_api_key(k: &str) -> String { + let b = k.as_bytes(); + match b.len() { + 0 => String::new(), + 1 | 2 => "…".into(), + n if n <= 13 => format!("…{}", go_lossy(&b[n - 2..])), + n => format!("{}…{}", go_lossy(&b[..9]), go_lossy(&b[n - 4..])), + } +} + +/// `genAPIKey`. +pub fn gen_api_key() -> String { + let mut b = [0u8; 24]; + let _ = getrandom::fill(&mut b); + format!("sk-loki-{}", hex::encode(b)) +} + +/// `visionEnabled` : un projecteur multimodal est-il configuré ? +pub fn vision_enabled(cfg: &Cfg) -> bool { + if serve::is_external(cfg) { + return serve::trim(serve::get(cfg, "EXTERNAL_VISION")) == "1"; + } + if serve::is_strata(cfg) && serve::get(cfg, "STRATA_VISION") != "0" { + return true; + } + !serve::trim(serve::get(cfg, "MMPROJ")).is_empty() +} + +/// `playwrightChromium` : le Chromium déposé par Playwright, le plus récent. +pub fn playwright_chromium(browsers_env: Option<&str>, home: Option<&str>) -> String { + let mut roots = Vec::new(); + if let Some(p) = browsers_env + .map(str::trim) + .filter(|p| !p.is_empty() && *p != "0") + { + roots.push(p.to_string()); + } + roots.push("/opt/pw-browsers".into()); + if let Some(h) = home.filter(|h| !h.is_empty()) { + roots.push(crate::paths::go_join(&[h, ".cache", "ms-playwright"])); + } + for root in roots { + let Ok(rd) = std::fs::read_dir(&root) else { + continue; + }; + let mut dirs: Vec = rd + .filter_map(|e| e.ok()) + .map(|e| e.file_name().to_string_lossy().into_owned()) + .filter(|n| { + n.strip_prefix("chromium") + .is_some_and(|rest| rest.contains('-')) + }) + .map(|n| crate::paths::go_join(&[&root, &n])) + .collect(); + dirs.sort(); + for d in dirs.iter().rev() { + for rel in [ + ["chrome-linux", "chrome"], + ["chrome-linux", "headless_shell"], + ] { + let p = crate::paths::go_join(&[d, rel[0], rel[1]]); + if std::fs::metadata(&p).is_ok_and(|m| !m.is_dir()) { + return p; + } + } + } + } + String::new() +} + +/// `chromePath` : `LOKI_CHROME`, le Chromium de Playwright, puis le PATH. +pub fn chrome_path() -> String { + let env = |k: &str| std::env::var(k).ok(); + if let Some(p) = env("LOKI_CHROME") + .map(|p| p.trim().to_string()) + .filter(|p| !p.is_empty()) + { + return p; + } + let p = playwright_chromium( + env("PLAYWRIGHT_BROWSERS_PATH").as_deref(), + env("HOME").as_deref(), + ); + if !p.is_empty() { + return p; + } + for n in [ + "google-chrome", + "google-chrome-stable", + "chromium", + "chromium-browser", + "microsoft-edge", + "chrome", + ] { + if let Some(p) = crate::paths::look_path(n) { + return p; + } + } + String::new() +} + +/// `engineHost`. +pub fn engine_host(cfg: &Cfg) -> String { + let h = serve::trim(serve::get(cfg, "HOST")); + if h.is_empty() { + "0.0.0.0".into() + } else { + h.into() + } +} + +/// `lanExposed`. +pub fn lan_exposed(cfg: &Cfg) -> bool { + let h = engine_host(cfg); + h != "127.0.0.1" && h != "::1" && !h.eq_ignore_ascii_case("localhost") +} + +/// `localIP` : l'adresse par laquelle cette machine sort sur le réseau. +pub fn local_ip() -> String { + let s = match std::net::UdpSocket::bind("0.0.0.0:0") { + Ok(s) => s, + Err(_) => return "localhost".into(), + }; + if s.connect("8.8.8.8:80").is_err() { + return "localhost".into(); + } + s.local_addr() + .map(|a| a.ip().to_string()) + .unwrap_or_else(|_| "localhost".into()) +} + +/// `networkStatus` (aucun pare-feu piloté hors Windows). +pub fn network_status(cfg: &Cfg) -> Value { + let port = crate::engstatus::llm_port(cfg); + json!({ + "exposed": lan_exposed(cfg), + "host": engine_host(cfg), + "port": port, + "url": format!("http://{}:{port}/v1", local_ip()), + "firewall": "inconnu", + "hint": "", + }) +} + +/// `requestBase` : « schéma://hôte » de la requête (la façade ne termine pas +/// le TLS : le schéma ne vient que de X-Forwarded-Proto). +pub fn request_base(headers: &hyper::HeaderMap, uri: &hyper::Uri) -> String { + let mut scheme = "http".to_string(); + if let Some(fwd) = headers + .get("x-forwarded-proto") + .and_then(|v| v.to_str().ok()) + .filter(|v| !v.is_empty()) + { + let first = fwd.split(',').next().unwrap_or("").trim().to_lowercase(); + if first == "http" || first == "https" { + scheme = first; + } + } + let host = headers + .get(HOST) + .and_then(|v| v.to_str().ok()) + .map(str::to_string) + .or_else(|| uri.authority().map(|a| a.to_string())) + .filter(|h| !h.is_empty()) + .unwrap_or_else(|| "localhost".into()); + format!("{scheme}://{host}") +} + +// --- Routes ------------------------------------------------------------------ + +/// Les routes servies par ce module. +pub fn handles(path: &str) -> bool { + matches!( + path, + "/api/internet" | "/api/computer" | "/api/apikey" | "/api/network" + ) +} + +/// `putStr` : une chaîne vide efface la clé. +async fn put_str(store: &Store, bucket: &str, key: &str, val: &str) -> Result<(), String> { + if val.is_empty() { + store.del(bucket, key).await + } else { + store.put(bucket, key, val.as_bytes().to_vec()).await + } +} + +/// `putBool`. +async fn put_bool(store: &Store, key: &str, on: bool) -> Result<(), String> { + put_str(store, BK_STATE, key, if on { "1" } else { "" }).await +} + +async fn get_bool(store: &Store, key: &str) -> bool { + store.get_str(BK_STATE, key).await == "1" +} + +fn fail(code: StatusCode, e: impl Into) -> Response { + send_json(code, &json!({ "ok": false, "error": e.into() })) +} + +/// Le verrou de l'optimiseur (au Go) : `Some(réponse 409)` s'il tient le +/// moteur. +async fn tune_deny(store: &Store) -> Option> { + let v = store + .internal_json(Method::GET, "/internal/v1/tune/guard", None) + .await + .ok()?; + if v.get("busy").and_then(Value::as_bool) == Some(true) { + return Some(send_json( + StatusCode::CONFLICT, + &json!({ "ok": false, "tuning": true, "error": v.get("error").and_then(Value::as_str).unwrap_or("") }), + )); + } + None +} + +/// La clé ne s'applique qu'au redémarrage du moteur : relancé s'il tourne, +/// par la façade si elle le pilote, sinon par le Go. +async fn restart_if_active(ctx: &Ctx) { + if let Some(ctl) = &ctx.engine_ctl { + if crate::enginectl::service_is_active(ctx) { + let _ = crate::enginectl::service_action(ctx, ctl, "restart").await; + } + return; + } + if let Err(e) = ctx + .store + .internal_post( + "/internal/v1/service/action", + &json!({ "action": "restart", "if_active": true }), + ) + .await + { + eprintln!("[clé] redémarrage du moteur : {e}"); + } +} + +const INTERNET_REQ_TYPE: &str = r#"struct { Enabled *bool "json:\"enabled\""; URL *string "json:\"url\""; Key *string "json:\"key\""; Engine *string "json:\"engine\"" }"#; +const COMPUTER_REQ_TYPE: &str = r#"struct { On bool "json:\"on\"" }"#; +const APIKEY_REQ_TYPE: &str = + r#"struct { Action string "json:\"action\""; Key string "json:\"key\"" }"#; +const NETWORK_REQ_TYPE: &str = r#"struct { Exposed *bool "json:\"exposed\"" }"#; + +/// `from_go` : remise par le Go (tunnel), déjà authentifiée par lui. +pub async fn serve( + ctx: &Ctx, + sys: &System, + req: Request, + from_go: bool, +) -> Response { + let (parts, body) = req.into_parts(); + if !from_go && let Err(refus) = ctx.guard(&parts.headers).await { + return *refus; + } + let post = parts.method == Method::POST; + let raw = if post { + Limited::new(body, 1 << 20) + .collect() + .await + .map(|b| b.to_bytes().to_vec()) + .unwrap_or_default() + } else { + Vec::new() + }; + let store = &ctx.store; + match parts.uri.path() { + "/api/internet" => { + if post { + let d = gojson::decode_struct( + &raw, + INTERNET_REQ_TYPE, + &[ + ("enabled", Kind::OptBool), + ("url", Kind::OptStr), + ("key", Kind::OptStr), + ("engine", Kind::OptStr), + ], + ); + if let Some(engine) = d.opt_str("engine") { + let e = engine.trim().to_lowercase(); + if e != ENGINE_GO && e != ENGINE_CRAWL { + return fail( + StatusCode::BAD_REQUEST, + format!("moteur web inconnu : {}", gojson::quote(&e)), + ); + } + if let Err(err) = store.put(BK_CONFIG, "WEB_ENGINE", e.into_bytes()).await { + return fail(StatusCode::BAD_REQUEST, err); + } + sys.reach_reset(store).await; + } + if let Some(key) = d.opt_str("key") { + // Rien ne traîne dans la configuration. + let _ = store.del(BK_CONFIG, "CRAWL4AI_KEY").await; + if let Err(err) = put_str(store, BK_STATE, "crawl_key", key.trim()).await { + return fail(StatusCode::INTERNAL_SERVER_ERROR, err); + } + sys.reach_reset(store).await; + } + if let Some(u) = d.opt_str("url") { + let u = u.trim().trim_end_matches('/'); + if let Err(err) = put_str(store, BK_CONFIG, "CRAWL4AI_URL", u).await { + return fail(StatusCode::INTERNAL_SERVER_ERROR, err); + } + sys.reach_reset(store).await; + } + if let Some(on) = d.opt_bool("enabled") + && let Err(err) = put_bool(store, "internet", on).await + { + return fail(StatusCode::INTERNAL_SERVER_ERROR, err); + } + } + let cfg = store.config().await; + let key = store.get_str(BK_STATE, "crawl_key").await; + let switch_on = get_bool(store, "internet").await; + send_json( + StatusCode::OK, + &json!({ + "ok": true, + "enabled": internet_enabled(&cfg, switch_on), + "engine": web_engine(&cfg), + "url": crawl4ai_url(&cfg), + "reachable": sys.reachable(&cfg, &key).await, + "key_set": !key.is_empty(), + "key_hint": key_hint(&key), + }), + ) + } + "/api/computer" => { + if post { + let on = gojson::decode_struct(&raw, COMPUTER_REQ_TYPE, &[("on", Kind::Bool)]) + .bool("on"); + if let Err(e) = put_bool(store, "computer", on).await { + return fail(StatusCode::INTERNAL_SERVER_ERROR, e); + } + // Couper l'interrupteur ferme le navigateur (au Go, qui le + // pilote pour les outils browser_*). + if !on + && let Err(e) = store + .internal_post("/internal/v1/cdp/shutdown", &json!({})) + .await + { + eprintln!("[navigateur] fermeture demandée au Go : {e}"); + } + } + let bin = match &sys.chrome { + Some(c) => c.clone(), + None => tokio::task::spawn_blocking(chrome_path) + .await + .unwrap_or_default(), + }; + let cfg = store.config().await; + send_json( + StatusCode::OK, + &json!({ + "ok": true, + "enabled": get_bool(store, "computer").await, + "browser": bin, + "browser_ok": !bin.is_empty(), + "vision": vision_enabled(&cfg), + }), + ) + } + "/api/apikey" => { + if post { + if let Some(r) = tune_deny(store).await { + return r; + } + let d = gojson::decode_struct( + &raw, + APIKEY_REQ_TYPE, + &[("action", Kind::Str), ("key", Kind::Str)], + ); + if let Some(e) = d.err.clone() { + return fail(StatusCode::BAD_REQUEST, e); + } + let key = match d.str("action").as_str() { + "generate" => gen_api_key(), + "set" => d.str("key").trim().to_string(), + "clear" => String::new(), + _ => return fail(StatusCode::BAD_REQUEST, "action inconnue"), + }; + // Aucune ambiguïté avec une valeur résiduelle de config.env. + let _ = store.del(BK_CONFIG, "API_KEY").await; + if let Err(e) = put_str(store, BK_STATE, "api_key", &key).await { + return fail(StatusCode::INTERNAL_SERVER_ERROR, e); + } + restart_if_active(ctx).await; + } + // `effectiveAPIKeyErr` : la clé en base, sinon celle de config.env. + let mut k = store + .get(BK_STATE, "api_key") + .await + .ok() + .flatten() + .map(|b| go_lossy(&b)) + .unwrap_or_default(); + if k.is_empty() { + k = serve::trim(serve::get(&store.config().await, "API_KEY")).to_string(); + } + let public = store.get_str(BK_STATE, "public_url").await; + let base = if public.is_empty() { + request_base(&parts.headers, &parts.uri) + } else { + public.clone() + }; + send_json( + StatusCode::OK, + &json!({ + "ok": true, + "set": !k.is_empty(), + "masked": mask_api_key(&k), + "key": k, + "url": format!("{base}/v1"), + "public_url": public, + }), + ) + } + "/api/network" => { + if post { + let d = + gojson::decode_struct(&raw, NETWORK_REQ_TYPE, &[("exposed", Kind::OptBool)]); + if let Some(e) = d.err.clone() { + return fail(StatusCode::BAD_REQUEST, e); + } + if let Some(on) = d.opt_bool("exposed") { + let host = if on { "0.0.0.0" } else { "127.0.0.1" }; + if let Err(e) = store.put(BK_CONFIG, "HOST", host.as_bytes().to_vec()).await { + return fail(StatusCode::INTERNAL_SERVER_ERROR, e); + } + let cfg = store.config().await; + let st = tokio::task::spawn_blocking(move || network_status(&cfg)) + .await + .unwrap_or_default(); + return send_json(StatusCode::OK, &json!({ "ok": true, "status": st })); + } + } + let cfg = store.config().await; + let st = tokio::task::spawn_blocking(move || network_status(&cfg)) + .await + .unwrap_or_default(); + send_json(StatusCode::OK, &json!({ "ok": true, "status": st })) + } + _ => send_json(StatusCode::NOT_FOUND, &json!({ "error": "route inconnue" })), + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn octets_invalides_comme_go() { + assert_eq!(go_lossy(b"ab"), "ab"); + assert_eq!(go_lossy(&[0x61, 0xC3]), "a\u{FFFD}"); + assert_eq!(go_lossy(&[0xA9, 0x62]), "\u{FFFD}b"); + assert_eq!(go_lossy(&[0xE2, 0x82, 0x41]), "\u{FFFD}\u{FFFD}A"); + } + + #[test] + fn masque_sur_les_octets() { + assert_eq!(mask_api_key("k"), "…"); + assert_eq!(mask_api_key("abc"), "…bc"); + assert_eq!(mask_api_key("sk-loki-0123456789abcdef"), "sk-loki-0…cdef"); + // Coupé au milieu d'un caractère : U+FFFD, comme l'encodeur du Go. + assert_eq!(mask_api_key("abcdefghijé0"), "…\u{FFFD}0"); + assert_eq!(mask_api_key("abcdefgh€xyz0123"), "abcdefgh\u{FFFD}…0123"); + } +} diff --git a/rust/loki-rs/tests/common/fakego.rs b/rust/loki-rs/tests/common/fakego.rs index 0632d7c..2590362 100644 --- a/rust/loki-rs/tests/common/fakego.rs +++ b/rust/loki-rs/tests/common/fakego.rs @@ -84,6 +84,12 @@ pub struct FakeGo { pub prestarts: Mutex, /// /internal/v1/dictate/stop reçus. pub dictate_stops: Mutex, + /// /internal/v1/web/reach/reset reçus. + pub reach_resets: Mutex, + /// /internal/v1/cdp/shutdown reçus. + pub cdp_shutdowns: Mutex, + /// /internal/v1/service/action reçus : (action, if_active). + pub service_actions: Mutex>, /// Comportement de /internal/v1/crypt/* (préfixe « SCELLE: » par défaut). pub crypt_mode: Mutex, /// Réponse de /internal/v1/conv/workspace (None : 404). @@ -99,6 +105,11 @@ impl FakeGo { .or_default() .insert(key.into(), val.as_bytes().to_vec()); } + pub fn del(&self, bucket: &str, key: &str) { + if let Some(b) = self.store.lock().unwrap().get_mut(bucket) { + b.remove(key); + } + } pub fn get(&self, bucket: &str, key: &str) -> Option { let s = self.store.lock().unwrap(); s.get(bucket)? @@ -351,6 +362,23 @@ async fn handle(st: Arc, req: Request) -> Response 200, serde_json::json!({ "generating": *st.generating.lock().unwrap() }).to_string(), ), + "/internal/v1/web/reach/reset" => { + *st.reach_resets.lock().unwrap() += 1; + resp(204, "") + } + "/internal/v1/cdp/shutdown" => { + *st.cdp_shutdowns.lock().unwrap() += 1; + resp(204, "") + } + "/internal/v1/service/action" => { + let body = req.into_body().collect().await.unwrap().to_bytes(); + let v: serde_json::Value = serde_json::from_slice(&body).unwrap_or_default(); + st.service_actions.lock().unwrap().push(( + v["action"].as_str().unwrap_or("").to_string(), + v["if_active"].as_bool().unwrap_or(false), + )); + resp(200, r#"{"ran":true,"error":""}"#) + } "/internal/v1/dictate/stop" => { *st.dictate_stops.lock().unwrap() += 1; resp(204, "") @@ -550,6 +578,7 @@ pub struct Opts { pub mem: Option>, pub presets: bool, pub engine_ctl: Option>, + pub system: Option>, } /// Comme [`start`], avec la conversation tenue par la façade @@ -647,6 +676,7 @@ pub async fn start_with( mem: opts.mem, presets: opts.presets, engine_ctl: opts.engine_ctl, + system: opts.system, }; tokio::spawn(loki_rs::proxy::serve(fl, cfg, async move { let _ = rx.await; diff --git a/rust/loki-rs/tests/lot2.rs b/rust/loki-rs/tests/lot2.rs index fcdffd7..a9f3c29 100644 --- a/rust/loki-rs/tests/lot2.rs +++ b/rust/loki-rs/tests/lot2.rs @@ -102,6 +102,7 @@ async fn banc(cle: Option<&str>) -> Banc { mem: None, presets: false, engine_ctl: None, + system: None, }; tokio::spawn(loki_rs::proxy::serve(l, cfg, async move { let _ = rx.await; diff --git a/rust/loki-rs/tests/parity_system/system.json b/rust/loki-rs/tests/parity_system/system.json new file mode 100644 index 0000000..ec6f436 --- /dev/null +++ b/rust/loki-rs/tests/parity_system/system.json @@ -0,0 +1,1407 @@ +{ + "chromium": [ + { + "files": [ + "chromium-1000/chrome-linux/chrome", + "chromium-1200/chrome-linux/chrome" + ], + "out": "chromium-1200/chrome-linux/chrome" + }, + { + "files": [ + "chromium-1000/chrome-linux/chrome", + "chromium-1200/chrome-linux/headless_shell" + ], + "out": "chromium-1200/chrome-linux/headless_shell" + }, + { + "files": [ + "chromium-1200/chrome-linux/headless_shell", + "chromium-1200/chrome-linux/chrome" + ], + "out": "chromium-1200/chrome-linux/chrome" + }, + { + "files": [ + "chromium-1000/chrome-linux/chrome", + "chromium_headless_shell-1100/chrome-linux/headless_shell" + ], + "out": "chromium_headless_shell-1100/chrome-linux/headless_shell" + }, + { + "files": [ + "chromium-1000/chrome-linux/chrome", + "chromium-1300/chrome-linux/chrome/" + ], + "out": "chromium-1000/chrome-linux/chrome" + }, + { + "files": [ + "chromiumX/chrome-linux/chrome", + "chromium-1/chrome-linux/chrome" + ], + "out": "chromium-1/chrome-linux/chrome" + }, + { + "files": [ + "firefox-1/chrome-linux/chrome", + "chromium-/chrome-linux/chrome" + ], + "out": "chromium-/chrome-linux/chrome" + }, + { + "files": [ + "chromium-9/chrome-linux/chrome", + "chromium-10/chrome-linux/chrome" + ], + "out": "chromium-9/chrome-linux/chrome" + } + ], + "steps": [ + { + "name": "internet par défaut", + "method": "GET", + "path": "/api/internet", + "status": 200, + "out": { + "enabled": false, + "engine": "go", + "key_hint": "", + "key_set": false, + "ok": true, + "reachable": true, + "url": "" + }, + "after": {}, + "probes": [] + }, + { + "name": "allumé, moteur intégré", + "method": "POST", + "path": "/api/internet", + "body": "{\"enabled\":true}", + "status": 200, + "out": { + "enabled": true, + "engine": "go", + "key_hint": "", + "key_set": false, + "ok": true, + "reachable": true, + "url": "" + }, + "after": { + "state/internet": "1" + }, + "probes": [] + }, + { + "name": "moteur crawl4ai sans URL", + "method": "POST", + "path": "/api/internet", + "body": "{\"engine\":\" CRAWL4AI \"}", + "status": 200, + "out": { + "enabled": false, + "engine": "crawl4ai", + "key_hint": "", + "key_set": false, + "ok": true, + "reachable": false, + "url": "" + }, + "after": { + "config/WEB_ENGINE": "crawl4ai", + "state/internet": "1" + }, + "probes": [] + }, + { + "name": "moteur inconnu", + "method": "POST", + "path": "/api/internet", + "body": "{\"engine\":\"bing\"}", + "status": 400, + "out": { + "error": "moteur web inconnu : \"bing\"", + "ok": false + }, + "after": { + "config/WEB_ENGINE": "crawl4ai", + "state/internet": "1" + }, + "probes": [] + }, + { + "name": "moteur inconnu vide", + "method": "POST", + "path": "/api/internet", + "body": "{\"engine\":\" \"}", + "status": 400, + "out": { + "error": "moteur web inconnu : \"\"", + "ok": false + }, + "after": { + "config/WEB_ENGINE": "crawl4ai", + "state/internet": "1" + }, + "probes": [] + }, + { + "name": "URL joignable", + "method": "POST", + "path": "/api/internet", + "body": "{\"url\":\" {{OK}}/// \"}", + "status": 200, + "out": { + "enabled": true, + "engine": "crawl4ai", + "key_hint": "", + "key_set": false, + "ok": true, + "reachable": true, + "url": "{{OK}}" + }, + "after": { + "config/CRAWL4AI_URL": "{{OK}}", + "config/WEB_ENGINE": "crawl4ai", + "state/internet": "1" + }, + "probes": [ + "/health" + ] + }, + { + "name": "relu dans le cache", + "method": "GET", + "path": "/api/internet", + "status": 200, + "out": { + "enabled": true, + "engine": "crawl4ai", + "key_hint": "", + "key_set": false, + "ok": true, + "reachable": true, + "url": "{{OK}}" + }, + "after": { + "config/CRAWL4AI_URL": "{{OK}}", + "config/WEB_ENGINE": "crawl4ai", + "state/internet": "1" + }, + "probes": [] + }, + { + "name": "clé posée", + "method": "POST", + "path": "/api/internet", + "body": "{\"key\":\" secret-clé-crawl \"}", + "status": 200, + "out": { + "enabled": true, + "engine": "crawl4ai", + "key_hint": "…rawl", + "key_set": true, + "ok": true, + "reachable": true, + "url": "{{OK}}" + }, + "after": { + "config/CRAWL4AI_URL": "{{OK}}", + "config/WEB_ENGINE": "crawl4ai", + "state/crawl_key": "secret-clé-crawl", + "state/internet": "1" + }, + "probes": [ + "/health Bearer secret-clé-crawl" + ] + }, + { + "name": "clé courte", + "method": "POST", + "path": "/api/internet", + "body": "{\"key\":\"abcd\"}", + "status": 200, + "out": { + "enabled": true, + "engine": "crawl4ai", + "key_hint": "…", + "key_set": true, + "ok": true, + "reachable": true, + "url": "{{OK}}" + }, + "after": { + "config/CRAWL4AI_URL": "{{OK}}", + "config/WEB_ENGINE": "crawl4ai", + "state/crawl_key": "abcd", + "state/internet": "1" + }, + "probes": [ + "/health Bearer abcd" + ] + }, + { + "name": "clé d'un caractère", + "method": "POST", + "path": "/api/internet", + "body": "{\"key\":\"é\"}", + "status": 200, + "out": { + "enabled": true, + "engine": "crawl4ai", + "key_hint": "…", + "key_set": true, + "ok": true, + "reachable": true, + "url": "{{OK}}" + }, + "after": { + "config/CRAWL4AI_URL": "{{OK}}", + "config/WEB_ENGINE": "crawl4ai", + "state/crawl_key": "é", + "state/internet": "1" + }, + "probes": [ + "/health Bearer é" + ] + }, + { + "name": "clé résiduelle en config", + "config": { + "CRAWL4AI_KEY": "vieille" + }, + "method": "POST", + "path": "/api/internet", + "body": "{\"key\":\"neuve12345\"}", + "status": 200, + "out": { + "enabled": true, + "engine": "crawl4ai", + "key_hint": "…2345", + "key_set": true, + "ok": true, + "reachable": true, + "url": "{{OK}}" + }, + "after": { + "config/CRAWL4AI_URL": "{{OK}}", + "config/WEB_ENGINE": "crawl4ai", + "state/crawl_key": "neuve12345", + "state/internet": "1" + }, + "probes": [ + "/health Bearer neuve12345" + ] + }, + { + "name": "URL en panne", + "method": "POST", + "path": "/api/internet", + "body": "{\"url\":\"{{DOWN}}\"}", + "status": 200, + "out": { + "enabled": true, + "engine": "crawl4ai", + "key_hint": "…2345", + "key_set": true, + "ok": true, + "reachable": false, + "url": "{{DOWN}}" + }, + "after": { + "config/CRAWL4AI_URL": "{{DOWN}}", + "config/WEB_ENGINE": "crawl4ai", + "state/crawl_key": "neuve12345", + "state/internet": "1" + }, + "probes": [ + "/health Bearer neuve12345", + "/ Bearer neuve12345" + ] + }, + { + "name": "URL 404", + "method": "POST", + "path": "/api/internet", + "body": "{\"url\":\"{{NOTFOUND}}\"}", + "status": 200, + "out": { + "enabled": true, + "engine": "crawl4ai", + "key_hint": "…2345", + "key_set": true, + "ok": true, + "reachable": true, + "url": "{{NOTFOUND}}" + }, + "after": { + "config/CRAWL4AI_URL": "{{NOTFOUND}}", + "config/WEB_ENGINE": "crawl4ai", + "state/crawl_key": "neuve12345", + "state/internet": "1" + }, + "probes": [ + "/health Bearer neuve12345" + ] + }, + { + "name": "URL injoignable", + "method": "POST", + "path": "/api/internet", + "body": "{\"url\":\"http://127.0.0.1:1\"}", + "status": 200, + "out": { + "enabled": true, + "engine": "crawl4ai", + "key_hint": "…2345", + "key_set": true, + "ok": true, + "reachable": false, + "url": "http://127.0.0.1:1" + }, + "after": { + "config/CRAWL4AI_URL": "http://127.0.0.1:1", + "config/WEB_ENGINE": "crawl4ai", + "state/crawl_key": "neuve12345", + "state/internet": "1" + }, + "probes": [] + }, + { + "name": "URL sans schéma", + "method": "POST", + "path": "/api/internet", + "body": "{\"url\":\"exemple\"}", + "status": 200, + "out": { + "enabled": true, + "engine": "crawl4ai", + "key_hint": "…2345", + "key_set": true, + "ok": true, + "reachable": false, + "url": "exemple" + }, + "after": { + "config/CRAWL4AI_URL": "exemple", + "config/WEB_ENGINE": "crawl4ai", + "state/crawl_key": "neuve12345", + "state/internet": "1" + }, + "probes": [] + }, + { + "name": "type faux, le reste passe", + "method": "POST", + "path": "/api/internet", + "body": "{\"enabled\":\"oui\",\"url\":\"{{OK}}\"}", + "status": 200, + "out": { + "enabled": false, + "engine": "crawl4ai", + "key_hint": "…2345", + "key_set": true, + "ok": true, + "reachable": true, + "url": "{{OK}}" + }, + "after": { + "config/CRAWL4AI_URL": "{{OK}}", + "config/WEB_ENGINE": "crawl4ai", + "state/crawl_key": "neuve12345" + }, + "probes": [ + "/health Bearer neuve12345" + ] + }, + { + "name": "corps illisible", + "method": "POST", + "path": "/api/internet", + "body": "{\"enabled\":false", + "status": 200, + "out": { + "enabled": false, + "engine": "crawl4ai", + "key_hint": "…2345", + "key_set": true, + "ok": true, + "reachable": true, + "url": "{{OK}}" + }, + "after": { + "config/CRAWL4AI_URL": "{{OK}}", + "config/WEB_ENGINE": "crawl4ai", + "state/crawl_key": "neuve12345" + }, + "probes": [] + }, + { + "name": "éteint, clé effacée", + "method": "POST", + "path": "/api/internet", + "body": "{\"enabled\":false,\"key\":\"\"}", + "status": 200, + "out": { + "enabled": false, + "engine": "crawl4ai", + "key_hint": "", + "key_set": false, + "ok": true, + "reachable": true, + "url": "{{OK}}" + }, + "after": { + "config/CRAWL4AI_URL": "{{OK}}", + "config/WEB_ENGINE": "crawl4ai" + }, + "probes": [ + "/health" + ] + }, + { + "name": "null ne touche à rien", + "method": "POST", + "path": "/api/internet", + "body": "{\"enabled\":null,\"url\":null,\"engine\":null}", + "status": 200, + "out": { + "enabled": false, + "engine": "crawl4ai", + "key_hint": "", + "key_set": false, + "ok": true, + "reachable": true, + "url": "{{OK}}" + }, + "after": { + "config/CRAWL4AI_URL": "{{OK}}", + "config/WEB_ENGINE": "crawl4ai" + }, + "probes": [] + }, + { + "name": "URL vidée, moteur choisi gardé", + "method": "POST", + "path": "/api/internet", + "body": "{\"url\":\"\",\"enabled\":true}", + "status": 200, + "out": { + "enabled": false, + "engine": "crawl4ai", + "key_hint": "", + "key_set": false, + "ok": true, + "reachable": false, + "url": "" + }, + "after": { + "config/WEB_ENGINE": "crawl4ai", + "state/internet": "1" + }, + "probes": [] + }, + { + "name": "moteur go", + "method": "POST", + "path": "/api/internet", + "body": "{\"engine\":\"Go\"}", + "status": 200, + "out": { + "enabled": true, + "engine": "go", + "key_hint": "", + "key_set": false, + "ok": true, + "reachable": true, + "url": "" + }, + "after": { + "config/WEB_ENGINE": "go", + "state/internet": "1" + }, + "probes": [] + }, + { + "name": "moteur non choisi, URL héritée", + "config": { + "CRAWL4AI_URL": "{{OK}}", + "WEB_ENGINE": "" + }, + "method": "GET", + "path": "/api/internet", + "status": 200, + "out": { + "enabled": true, + "engine": "crawl4ai", + "key_hint": "", + "key_set": false, + "ok": true, + "reachable": true, + "url": "{{OK}}" + }, + "after": { + "config/CRAWL4AI_URL": "{{OK}}", + "state/internet": "1" + }, + "probes": [ + "/health" + ] + }, + { + "name": "moteur illisible en config", + "config": { + "CRAWL4AI_URL": "", + "WEB_ENGINE": "autre" + }, + "method": "GET", + "path": "/api/internet", + "status": 200, + "out": { + "enabled": true, + "engine": "go", + "key_hint": "", + "key_set": false, + "ok": true, + "reachable": true, + "url": "" + }, + "after": { + "config/WEB_ENGINE": "autre", + "state/internet": "1" + }, + "probes": [] + }, + { + "name": "URL en config avec slashs", + "config": { + "CRAWL4AI_URL": " {{DOWN}}// ", + "WEB_ENGINE": "crawl4ai" + }, + "method": "GET", + "path": "/api/internet", + "status": 200, + "out": { + "enabled": true, + "engine": "crawl4ai", + "key_hint": "", + "key_set": false, + "ok": true, + "reachable": false, + "url": "{{DOWN}}" + }, + "after": { + "config/CRAWL4AI_URL": " {{DOWN}}// ", + "config/WEB_ENGINE": "crawl4ai", + "state/internet": "1" + }, + "probes": [ + "/health", + "/" + ] + }, + { + "name": "internet en PUT", + "method": "PUT", + "path": "/api/internet", + "body": "{\"enabled\":true}", + "status": 200, + "out": { + "enabled": true, + "engine": "crawl4ai", + "key_hint": "", + "key_set": false, + "ok": true, + "reachable": false, + "url": "{{DOWN}}" + }, + "after": { + "config/CRAWL4AI_URL": " {{DOWN}}// ", + "config/WEB_ENGINE": "crawl4ai", + "state/internet": "1" + }, + "probes": [] + }, + { + "name": "navigateur par défaut", + "method": "GET", + "path": "/api/computer", + "status": 200, + "out": { + "browser": "/opt/faux/chrome", + "browser_ok": true, + "enabled": false, + "ok": true, + "vision": false + }, + "after": { + "config/CRAWL4AI_URL": " {{DOWN}}// ", + "config/WEB_ENGINE": "crawl4ai", + "state/internet": "1" + }, + "probes": [] + }, + { + "name": "navigateur allumé", + "method": "POST", + "path": "/api/computer", + "body": "{\"on\":true}", + "status": 200, + "out": { + "browser": "/opt/faux/chrome", + "browser_ok": true, + "enabled": true, + "ok": true, + "vision": false + }, + "after": { + "config/CRAWL4AI_URL": " {{DOWN}}// ", + "config/WEB_ENGINE": "crawl4ai", + "state/computer": "1", + "state/internet": "1" + }, + "probes": [] + }, + { + "name": "navigateur, corps illisible", + "method": "POST", + "path": "/api/computer", + "body": "oui", + "status": 200, + "out": { + "browser": "/opt/faux/chrome", + "browser_ok": true, + "enabled": false, + "ok": true, + "vision": false + }, + "after": { + "config/CRAWL4AI_URL": " {{DOWN}}// ", + "config/WEB_ENGINE": "crawl4ai", + "state/internet": "1" + }, + "probes": [] + }, + { + "name": "vision par projecteur", + "config": { + "MMPROJ": " mmproj.gguf " + }, + "method": "GET", + "path": "/api/computer", + "status": 200, + "out": { + "browser": "/opt/faux/chrome", + "browser_ok": true, + "enabled": false, + "ok": true, + "vision": true + }, + "after": { + "config/CRAWL4AI_URL": " {{DOWN}}// ", + "config/WEB_ENGINE": "crawl4ai", + "state/internet": "1" + }, + "probes": [] + }, + { + "name": "vision externe", + "config": { + "EXTERNAL": "1", + "EXTERNAL_VISION": "1", + "MMPROJ": "" + }, + "method": "GET", + "path": "/api/computer", + "status": 200, + "out": { + "browser": "/opt/faux/chrome", + "browser_ok": true, + "enabled": false, + "ok": true, + "vision": true + }, + "after": { + "config/CRAWL4AI_URL": " {{DOWN}}// ", + "config/WEB_ENGINE": "crawl4ai", + "state/internet": "1" + }, + "probes": [] + }, + { + "name": "externe sans vision", + "config": { + "EXTERNAL_VISION": "0", + "MMPROJ": "m.gguf" + }, + "method": "GET", + "path": "/api/computer", + "status": 200, + "out": { + "browser": "/opt/faux/chrome", + "browser_ok": true, + "enabled": false, + "ok": true, + "vision": false + }, + "after": { + "config/CRAWL4AI_URL": " {{DOWN}}// ", + "config/WEB_ENGINE": "crawl4ai", + "state/internet": "1" + }, + "probes": [] + }, + { + "name": "vision strata", + "config": { + "ENGINE": "strata", + "EXTERNAL": "" + }, + "method": "GET", + "path": "/api/computer", + "status": 200, + "out": { + "browser": "/opt/faux/chrome", + "browser_ok": true, + "enabled": false, + "ok": true, + "vision": true + }, + "after": { + "config/CRAWL4AI_URL": " {{DOWN}}// ", + "config/WEB_ENGINE": "crawl4ai", + "state/internet": "1" + }, + "probes": [] + }, + { + "name": "strata sans vision", + "config": { + "MMPROJ": "", + "STRATA_VISION": "0" + }, + "method": "GET", + "path": "/api/computer", + "status": 200, + "out": { + "browser": "/opt/faux/chrome", + "browser_ok": true, + "enabled": false, + "ok": true, + "vision": false + }, + "after": { + "config/CRAWL4AI_URL": " {{DOWN}}// ", + "config/WEB_ENGINE": "crawl4ai", + "state/internet": "1" + }, + "probes": [] + }, + { + "name": "navigateur éteint", + "config": { + "ENGINE": "", + "STRATA_VISION": "" + }, + "method": "POST", + "path": "/api/computer", + "body": "{\"on\":false}", + "status": 200, + "out": { + "browser": "/opt/faux/chrome", + "browser_ok": true, + "enabled": false, + "ok": true, + "vision": false + }, + "after": { + "config/CRAWL4AI_URL": " {{DOWN}}// ", + "config/WEB_ENGINE": "crawl4ai", + "state/internet": "1" + }, + "probes": [] + }, + { + "name": "clé par défaut", + "method": "GET", + "path": "/api/apikey", + "status": 200, + "out": { + "key": "", + "masked": "", + "ok": true, + "public_url": "", + "set": false, + "url": "http://loki.local:8090/v1" + }, + "after": { + "config/CRAWL4AI_URL": " {{DOWN}}// ", + "config/WEB_ENGINE": "crawl4ai", + "state/internet": "1" + }, + "probes": [] + }, + { + "name": "clé générée", + "method": "POST", + "path": "/api/apikey", + "body": "{\"action\":\"generate\"}", + "status": 200, + "out": { + "key": "{{KEY}}", + "masked": "{{MASKED}}", + "ok": true, + "public_url": "", + "set": true, + "url": "http://loki.local:8090/v1" + }, + "after": { + "config/CRAWL4AI_URL": " {{DOWN}}// ", + "config/WEB_ENGINE": "crawl4ai", + "state/api_key": "{{KEY}}", + "state/internet": "1" + }, + "probes": [] + }, + { + "name": "clé relue", + "method": "GET", + "path": "/api/apikey", + "headers": { + "X-Forwarded-Proto": "HTTPS, http" + }, + "status": 200, + "out": { + "key": "{{KEY}}", + "masked": "{{MASKED}}", + "ok": true, + "public_url": "", + "set": true, + "url": "https://loki.local:8090/v1" + }, + "after": { + "config/CRAWL4AI_URL": " {{DOWN}}// ", + "config/WEB_ENGINE": "crawl4ai", + "state/api_key": "{{KEY}}", + "state/internet": "1" + }, + "probes": [] + }, + { + "name": "clé saisie", + "method": "POST", + "path": "/api/apikey", + "body": "{\"action\":\"set\",\"key\":\" ma-clé-perso-é \"}", + "status": 200, + "out": { + "key": "ma-clé-perso-é", + "masked": "ma-clé-p…o-é", + "ok": true, + "public_url": "", + "set": true, + "url": "http://loki.local:8090/v1" + }, + "after": { + "config/CRAWL4AI_URL": " {{DOWN}}// ", + "config/WEB_ENGINE": "crawl4ai", + "state/api_key": "ma-clé-perso-é", + "state/internet": "1" + }, + "probes": [] + }, + { + "name": "clé saisie courte", + "method": "POST", + "path": "/api/apikey", + "body": "{\"action\":\"set\",\"key\":\"abc\"}", + "status": 200, + "out": { + "key": "abc", + "masked": "…bc", + "ok": true, + "public_url": "", + "set": true, + "url": "http://loki.local:8090/v1" + }, + "after": { + "config/CRAWL4AI_URL": " {{DOWN}}// ", + "config/WEB_ENGINE": "crawl4ai", + "state/api_key": "abc", + "state/internet": "1" + }, + "probes": [] + }, + { + "name": "clé de 13 octets", + "method": "POST", + "path": "/api/apikey", + "body": "{\"action\":\"set\",\"key\":\"abcdefghijklm\"}", + "status": 200, + "out": { + "key": "abcdefghijklm", + "masked": "…lm", + "ok": true, + "public_url": "", + "set": true, + "url": "http://loki.local:8090/v1" + }, + "after": { + "config/CRAWL4AI_URL": " {{DOWN}}// ", + "config/WEB_ENGINE": "crawl4ai", + "state/api_key": "abcdefghijklm", + "state/internet": "1" + }, + "probes": [] + }, + { + "name": "clé effacée", + "method": "POST", + "path": "/api/apikey", + "body": "{\"action\":\"clear\"}", + "status": 200, + "out": { + "key": "", + "masked": "", + "ok": true, + "public_url": "", + "set": false, + "url": "http://loki.local:8090/v1" + }, + "after": { + "config/CRAWL4AI_URL": " {{DOWN}}// ", + "config/WEB_ENGINE": "crawl4ai", + "state/internet": "1" + }, + "probes": [] + }, + { + "name": "repli sur config.env", + "config": { + "API_KEY": " résiduelle-config " + }, + "method": "GET", + "path": "/api/apikey", + "headers": { + "X-Forwarded-Proto": "ftp" + }, + "status": 200, + "out": { + "key": "résiduelle-config", + "masked": "résiduel…nfig", + "ok": true, + "public_url": "", + "set": true, + "url": "http://loki.local:8090/v1" + }, + "after": { + "config/API_KEY": " résiduelle-config ", + "config/CRAWL4AI_URL": " {{DOWN}}// ", + "config/WEB_ENGINE": "crawl4ai", + "state/internet": "1" + }, + "probes": [] + }, + { + "name": "adresse publique", + "state": { + "public_url": "https://loki.exemple.fr" + }, + "method": "POST", + "path": "/api/apikey", + "body": "{\"action\":\"set\",\"key\":\"k\"}", + "status": 200, + "out": { + "key": "k", + "masked": "…", + "ok": true, + "public_url": "https://loki.exemple.fr", + "set": true, + "url": "https://loki.exemple.fr/v1" + }, + "after": { + "config/CRAWL4AI_URL": " {{DOWN}}// ", + "config/WEB_ENGINE": "crawl4ai", + "state/api_key": "k", + "state/internet": "1", + "state/public_url": "https://loki.exemple.fr" + }, + "probes": [] + }, + { + "name": "action inconnue", + "method": "POST", + "path": "/api/apikey", + "body": "{\"action\":\"rotate\"}", + "status": 400, + "out": { + "error": "action inconnue", + "ok": false + }, + "after": { + "config/CRAWL4AI_URL": " {{DOWN}}// ", + "config/WEB_ENGINE": "crawl4ai", + "state/api_key": "k", + "state/internet": "1", + "state/public_url": "https://loki.exemple.fr" + }, + "probes": [] + }, + { + "name": "action vide", + "method": "POST", + "path": "/api/apikey", + "body": "{}", + "status": 400, + "out": { + "error": "action inconnue", + "ok": false + }, + "after": { + "config/CRAWL4AI_URL": " {{DOWN}}// ", + "config/WEB_ENGINE": "crawl4ai", + "state/api_key": "k", + "state/internet": "1", + "state/public_url": "https://loki.exemple.fr" + }, + "probes": [] + }, + { + "name": "corps vide", + "method": "POST", + "path": "/api/apikey", + "status": 400, + "out": { + "error": "EOF", + "ok": false + }, + "after": { + "config/CRAWL4AI_URL": " {{DOWN}}// ", + "config/WEB_ENGINE": "crawl4ai", + "state/api_key": "k", + "state/internet": "1", + "state/public_url": "https://loki.exemple.fr" + }, + "probes": [] + }, + { + "name": "corps illisible", + "method": "POST", + "path": "/api/apikey", + "body": "{\"action\":", + "status": 400, + "out": { + "error": "unexpected EOF", + "ok": false + }, + "after": { + "config/CRAWL4AI_URL": " {{DOWN}}// ", + "config/WEB_ENGINE": "crawl4ai", + "state/api_key": "k", + "state/internet": "1", + "state/public_url": "https://loki.exemple.fr" + }, + "probes": [] + }, + { + "name": "type faux", + "method": "POST", + "path": "/api/apikey", + "body": "{\"action\":1}", + "status": 400, + "out": { + "error": "json: cannot unmarshal number into Go struct field .action of type string", + "ok": false + }, + "after": { + "config/CRAWL4AI_URL": " {{DOWN}}// ", + "config/WEB_ENGINE": "crawl4ai", + "state/api_key": "k", + "state/internet": "1", + "state/public_url": "https://loki.exemple.fr" + }, + "probes": [] + }, + { + "name": "optimisation en cours", + "tune": true, + "method": "POST", + "path": "/api/apikey", + "body": "{\"action\":\"clear\"}", + "status": 409, + "out": { + "error": "optimisation en cours (loki tune) : le moteur est réservé aux essais — attends la fin ou annule-la", + "ok": false, + "tuning": true + }, + "after": { + "config/CRAWL4AI_URL": " {{DOWN}}// ", + "config/WEB_ENGINE": "crawl4ai", + "state/api_key": "k", + "state/internet": "1", + "state/public_url": "https://loki.exemple.fr" + }, + "probes": [] + }, + { + "name": "lecture pendant une optimisation", + "tune": true, + "method": "GET", + "path": "/api/apikey", + "status": 200, + "out": { + "key": "k", + "masked": "…", + "ok": true, + "public_url": "https://loki.exemple.fr", + "set": true, + "url": "https://loki.exemple.fr/v1" + }, + "after": { + "config/CRAWL4AI_URL": " {{DOWN}}// ", + "config/WEB_ENGINE": "crawl4ai", + "state/api_key": "k", + "state/internet": "1", + "state/public_url": "https://loki.exemple.fr" + }, + "probes": [] + }, + { + "name": "réseau par défaut", + "method": "GET", + "path": "/api/network", + "status": 200, + "out": { + "ok": true, + "status": { + "exposed": true, + "firewall": "inconnu", + "hint": "", + "host": "0.0.0.0", + "port": 8080, + "url": "http://{{IP}}:8080/v1" + } + }, + "after": { + "config/CRAWL4AI_URL": " {{DOWN}}// ", + "config/WEB_ENGINE": "crawl4ai", + "state/api_key": "k", + "state/internet": "1", + "state/public_url": "https://loki.exemple.fr" + }, + "probes": [] + }, + { + "name": "réseau fermé", + "method": "POST", + "path": "/api/network", + "body": "{\"exposed\":false}", + "status": 200, + "out": { + "ok": true, + "status": { + "exposed": false, + "firewall": "inconnu", + "hint": "", + "host": "127.0.0.1", + "port": 8080, + "url": "http://{{IP}}:8080/v1" + } + }, + "after": { + "config/CRAWL4AI_URL": " {{DOWN}}// ", + "config/HOST": "127.0.0.1", + "config/WEB_ENGINE": "crawl4ai", + "state/api_key": "k", + "state/internet": "1", + "state/public_url": "https://loki.exemple.fr" + }, + "probes": [] + }, + { + "name": "réseau ouvert", + "method": "POST", + "path": "/api/network", + "body": "{\"exposed\":true}", + "status": 200, + "out": { + "ok": true, + "status": { + "exposed": true, + "firewall": "inconnu", + "hint": "", + "host": "0.0.0.0", + "port": 8080, + "url": "http://{{IP}}:8080/v1" + } + }, + "after": { + "config/CRAWL4AI_URL": " {{DOWN}}// ", + "config/HOST": "0.0.0.0", + "config/WEB_ENGINE": "crawl4ai", + "state/api_key": "k", + "state/internet": "1", + "state/public_url": "https://loki.exemple.fr" + }, + "probes": [] + }, + { + "name": "réseau sans champ", + "method": "POST", + "path": "/api/network", + "body": "{}", + "status": 200, + "out": { + "ok": true, + "status": { + "exposed": true, + "firewall": "inconnu", + "hint": "", + "host": "0.0.0.0", + "port": 8080, + "url": "http://{{IP}}:8080/v1" + } + }, + "after": { + "config/CRAWL4AI_URL": " {{DOWN}}// ", + "config/HOST": "0.0.0.0", + "config/WEB_ENGINE": "crawl4ai", + "state/api_key": "k", + "state/internet": "1", + "state/public_url": "https://loki.exemple.fr" + }, + "probes": [] + }, + { + "name": "réseau illisible", + "method": "POST", + "path": "/api/network", + "body": "{\"exposed\":1}", + "status": 400, + "out": { + "error": "json: cannot unmarshal number into Go struct field .exposed of type bool", + "ok": false + }, + "after": { + "config/CRAWL4AI_URL": " {{DOWN}}// ", + "config/HOST": "0.0.0.0", + "config/WEB_ENGINE": "crawl4ai", + "state/api_key": "k", + "state/internet": "1", + "state/public_url": "https://loki.exemple.fr" + }, + "probes": [] + }, + { + "name": "réseau corps vide", + "method": "POST", + "path": "/api/network", + "status": 400, + "out": { + "error": "EOF", + "ok": false + }, + "after": { + "config/CRAWL4AI_URL": " {{DOWN}}// ", + "config/HOST": "0.0.0.0", + "config/WEB_ENGINE": "crawl4ai", + "state/api_key": "k", + "state/internet": "1", + "state/public_url": "https://loki.exemple.fr" + }, + "probes": [] + }, + { + "name": "HOST localhost", + "config": { + "HOST": " LocalHost ", + "PORT": "9090" + }, + "method": "GET", + "path": "/api/network", + "status": 200, + "out": { + "ok": true, + "status": { + "exposed": false, + "firewall": "inconnu", + "hint": "", + "host": "LocalHost", + "port": 9090, + "url": "http://{{IP}}:9090/v1" + } + }, + "after": { + "config/CRAWL4AI_URL": " {{DOWN}}// ", + "config/HOST": " LocalHost ", + "config/WEB_ENGINE": "crawl4ai", + "state/api_key": "k", + "state/internet": "1", + "state/public_url": "https://loki.exemple.fr" + }, + "probes": [] + }, + { + "name": "HOST ::1", + "config": { + "HOST": "::1", + "PORT": "abc" + }, + "method": "GET", + "path": "/api/network", + "status": 200, + "out": { + "ok": true, + "status": { + "exposed": false, + "firewall": "inconnu", + "hint": "", + "host": "::1", + "port": 8080, + "url": "http://{{IP}}:8080/v1" + } + }, + "after": { + "config/CRAWL4AI_URL": " {{DOWN}}// ", + "config/HOST": "::1", + "config/WEB_ENGINE": "crawl4ai", + "state/api_key": "k", + "state/internet": "1", + "state/public_url": "https://loki.exemple.fr" + }, + "probes": [] + }, + { + "name": "HOST d'une carte", + "config": { + "HOST": "192.168.1.5", + "PORT": "" + }, + "method": "GET", + "path": "/api/network", + "status": 200, + "out": { + "ok": true, + "status": { + "exposed": true, + "firewall": "inconnu", + "hint": "", + "host": "192.168.1.5", + "port": 8080, + "url": "http://{{IP}}:8080/v1" + } + }, + "after": { + "config/CRAWL4AI_URL": " {{DOWN}}// ", + "config/HOST": "192.168.1.5", + "config/WEB_ENGINE": "crawl4ai", + "state/api_key": "k", + "state/internet": "1", + "state/public_url": "https://loki.exemple.fr" + }, + "probes": [] + } + ] +} diff --git a/rust/loki-rs/tests/proxy.rs b/rust/loki-rs/tests/proxy.rs index 6339327..2253c6b 100644 --- a/rust/loki-rs/tests/proxy.rs +++ b/rust/loki-rs/tests/proxy.rs @@ -108,6 +108,7 @@ async fn facade(backend: SocketAddr) -> (SocketAddr, oneshot::Sender<()>) { mem: None, presets: false, engine_ctl: None, + system: None, }; tokio::spawn(loki_rs::proxy::serve(l, cfg, async move { let _ = rx.await; diff --git a/rust/loki-rs/tests/system.rs b/rust/loki-rs/tests/system.rs new file mode 100644 index 0000000..35f1307 --- /dev/null +++ b/rust/loki-rs/tests/system.rs @@ -0,0 +1,369 @@ +//! Lot 17 : interrupteurs système tenus par la façade (`LOKI_SYSTEM=rust`). +//! +//! Le journal tiré du Go (`internal/loki/parity_system_test.go` → +//! `tests/parity_system/system.json`) est rejoué par une seule façade : accès +//! internet et Crawl4AI (en-têtes reçus par les faux serveurs et cache de +//! joignabilité compris), navigateur et vision, clé de complétion, écoute +//! réseau, avec après chaque appel les clés touchées en base ; puis le +//! Chromium de Playwright. Ce qui reste au Go (cache de ses outils web, +//! Chromium piloté, redémarrage du moteur) est vérifié de bout en bout contre +//! le faux Go. + +mod common; + +use std::collections::BTreeMap; +use std::convert::Infallible; +use std::net::SocketAddr; +use std::sync::{Arc, Mutex}; + +use bytes::Bytes; +use common::fakego; +use http_body_util::{BodyExt, Full}; +use hyper::service::service_fn; +use hyper::{Request, Response}; +use hyper_util::rt::TokioIo; +use loki_rs::system::{self as sy, System}; +use serde_json::{Value, json}; +use tokio::net::TcpListener; + +fn fixture() -> Value { + serde_json::from_str(include_str!("parity_system/system.json")).expect("journal") +} + +fn s(v: &Value) -> &str { + v.as_str().unwrap() +} + +const KEYS: &[&str] = &[ + "config/WEB_ENGINE", + "config/CRAWL4AI_URL", + "config/CRAWL4AI_KEY", + "config/API_KEY", + "config/HOST", + "state/internet", + "state/computer", + "state/crawl_key", + "state/api_key", + "state/public_url", +]; + +/// Un faux Crawl4AI : répond `code` partout, note « chemin Authorization ». +async fn fake_crawl(code: u16, log: Arc>>) -> String { + let l = TcpListener::bind("127.0.0.1:0").await.unwrap(); + let addr = l.local_addr().unwrap(); + tokio::spawn(async move { + while let Ok((st, _)) = l.accept().await { + let log = log.clone(); + tokio::spawn(async move { + let svc = service_fn(move |req: Request| { + let log = log.clone(); + async move { + let auth = req + .headers() + .get("authorization") + .map(|v| String::from_utf8_lossy(v.as_bytes()).into_owned()) + .unwrap_or_default(); + log.lock() + .unwrap() + .push(format!("{} {auth}", req.uri().path()).trim().to_string()); + let mut r = Response::new(Full::new(Bytes::new())); + *r.status_mut() = hyper::StatusCode::from_u16(code).unwrap(); + Ok::<_, Infallible>(r) + } + }); + let _ = hyper::server::conn::http1::Builder::new() + .serve_connection(TokioIo::new(st), svc) + .await; + }); + } + }); + format!("http://{addr}") +} + +async fn call( + facade: SocketAddr, + method: &str, + path: &str, + body: &str, + headers: &[(String, String)], +) -> (u16, String) { + use hyper_util::client::legacy::Client; + use hyper_util::rt::TokioExecutor; + let mut req = Request::builder() + .method(method) + .uri(format!("http://{facade}{path}")) + .header("host", "loki.local:8090"); + for (k, v) in headers { + req = req.header(k.as_str(), v.as_str()); + } + let client: Client<_, Full> = Client::builder(TokioExecutor::new()).build_http(); + let r = client + .request(req.body(Full::new(Bytes::from(body.to_string()))).unwrap()) + .await + .unwrap(); + let code = r.status().as_u16(); + let b = r.into_body().collect().await.unwrap().to_bytes(); + (code, String::from_utf8_lossy(&b).into_owned()) +} + +async fn banc( + opts: fakego::Opts, +) -> ( + SocketAddr, + Arc, + tokio::sync::oneshot::Sender<()>, + tempfile::TempDir, +) { + let tmp = tempfile::tempdir().unwrap(); + let (facade, go, stop, _) = fakego::start_with(tmp.path().to_path_buf(), opts).await; + (facade, go, stop, tmp) +} + +fn system() -> Arc { + let mut s = System::new(); + s.chrome = Some("/opt/faux/chrome".into()); + Arc::new(s) +} + +#[tokio::test] +async fn parite_des_routes() { + let log = Arc::new(Mutex::new(Vec::new())); + let ok = fake_crawl(200, log.clone()).await; + let down = fake_crawl(500, log.clone()).await; + let nf = fake_crawl(404, log.clone()).await; + let (facade, go, _stop, _tmp) = banc(fakego::Opts { + system: Some(system()), + ..Default::default() + }) + .await; + let ip = sy::local_ip(); + let key_re = regex::Regex::new(r"sk-loki-[0-9a-f]{48}").unwrap(); + let masked_re = regex::Regex::new(r"sk-loki-[0-9a-f]…[0-9a-f]{4}").unwrap(); + let unmask = |v: &str| { + v.replace("{{OK}}", &ok) + .replace("{{DOWN}}", &down) + .replace("{{NOTFOUND}}", &nf) + }; + let mask = |v: &str| { + let v = v + .replace(&ok, "{{OK}}") + .replace(&down, "{{DOWN}}") + .replace(&nf, "{{NOTFOUND}}") + .replace(&format!("http://{ip}:"), "http://{{IP}}:"); + let v = key_re.replace_all(&v, "{{KEY}}").into_owned(); + masked_re.replace_all(&v, "{{MASKED}}").into_owned() + }; + for st in fixture()["steps"].as_array().unwrap() { + let name = s(&st["name"]); + for (bucket, field) in [("config", "config"), ("state", "state")] { + for (k, v) in st[field].as_object().into_iter().flatten() { + let v = unmask(s(v)); + if v.is_empty() { + go.del(bucket, k); + } else { + go.set(bucket, k, &v); + } + } + } + *go.tune_busy.lock().unwrap() = (st["tune"].as_bool() == Some(true)).then(|| { + "optimisation en cours (loki tune) : le moteur est réservé aux essais — attends la fin ou annule-la".into() + }); + log.lock().unwrap().clear(); + let headers: Vec<(String, String)> = st["headers"] + .as_object() + .into_iter() + .flatten() + .map(|(k, v)| (k.clone(), s(v).to_string())) + .collect(); + let (code, body) = call( + facade, + s(&st["method"]), + s(&st["path"]), + &unmask(st["body"].as_str().unwrap_or("")), + &headers, + ) + .await; + assert_eq!(code as i64, st["status"].as_i64().unwrap(), "{name}"); + let out: Value = serde_json::from_str(&mask(&body)).unwrap(); + assert_eq!(out, st["out"], "{name}"); + let mut after = BTreeMap::new(); + for k in KEYS { + let (b, key) = k.split_once('/').unwrap(); + if let Some(v) = go.get(b, key).filter(|v| !v.is_empty()) { + after.insert(k.to_string(), Value::String(mask(&v))); + } + } + assert_eq!( + Value::Object(after.into_iter().collect()), + st["after"], + "{name} : base" + ); + let probes: Vec = log + .lock() + .unwrap() + .iter() + .map(|p| Value::String(mask(p))) + .collect(); + assert_eq!(Value::Array(probes), st["probes"], "{name} : sondes"); + } +} + +#[test] +fn parite_du_chromium_de_playwright() { + for c in fixture()["chromium"].as_array().unwrap() { + let tmp = tempfile::tempdir().unwrap(); + let root = tmp.path().to_string_lossy().into_owned(); + for f in c["files"].as_array().unwrap() { + let f = s(f); + let p = tmp.path().join(f.trim_end_matches('/')); + if f.ends_with('/') { + std::fs::create_dir_all(&p).unwrap(); + } else { + std::fs::create_dir_all(p.parent().unwrap()).unwrap(); + std::fs::write(&p, "").unwrap(); + } + } + let got = sy::playwright_chromium(Some(&root), None); + assert_eq!( + got.strip_prefix(&format!("{root}/")).unwrap_or(&got), + s(&c["out"]), + "{c}" + ); + } +} + +#[tokio::test] +async fn ce_qui_reste_au_go() { + let (facade, go, _stop, _tmp) = banc(fakego::Opts { + system: Some(system()), + ..Default::default() + }) + .await; + // Le cache des outils web du Go est vidé à chaque changement d'URL, de clé + // ou de moteur — pas pour l'interrupteur seul, ni en lecture. + for (body, resets) in [ + (r#"{"enabled":true}"#, 0), + (r#"{"url":"http://127.0.0.1:1"}"#, 1), + (r#"{"key":"k1"}"#, 2), + (r#"{"engine":"go"}"#, 3), + (r#"{"engine":"go","key":"","url":""}"#, 6), + (r#"{"engine":"autre","url":"x"}"#, 6), + ] { + call(facade, "POST", "/api/internet", body, &[]).await; + assert_eq!(*go.reach_resets.lock().unwrap(), resets, "{body}"); + } + call(facade, "GET", "/api/internet", "", &[]).await; + assert_eq!(*go.reach_resets.lock().unwrap(), 6); + + // Le navigateur piloté (au Go) se ferme quand l'interrupteur se coupe. + call(facade, "POST", "/api/computer", r#"{"on":true}"#, &[]).await; + assert_eq!(*go.cdp_shutdowns.lock().unwrap(), 0); + call(facade, "POST", "/api/computer", r#"{"on":false}"#, &[]).await; + call(facade, "GET", "/api/computer", "", &[]).await; + assert_eq!(*go.cdp_shutdowns.lock().unwrap(), 1); + + // Une clé neuve : le moteur est relancé s'il tourne, par le Go quand la + // façade ne le pilote pas ; jamais sur une lecture ni sur un refus. + call(facade, "GET", "/api/apikey", "", &[]).await; + call(facade, "POST", "/api/apikey", r#"{"action":"bof"}"#, &[]).await; + assert!(go.service_actions.lock().unwrap().is_empty()); + call( + facade, + "POST", + "/api/apikey", + r#"{"action":"generate"}"#, + &[], + ) + .await; + assert_eq!( + *go.service_actions.lock().unwrap(), + [("restart".to_string(), true)] + ); +} + +/// Faux superviseur du moteur (voir tests/enginectl.rs). +fn supervisor(dir: &std::path::Path) -> std::path::PathBuf { + let p = dir.join("loki-rs"); + let body = "#!/bin/sh\necho \"$2\" >> \"$LOKI_HOME/trace\"\nexit 0\n"; + let ok = std::process::Command::new("sh") + .args(["-c", "printf %s \"$1\" > \"$2\" && chmod 755 \"$2\""]) + .arg("sh") + .arg(body) + .arg(&p) + .status() + .unwrap() + .success(); + assert!(ok); + p +} + +#[tokio::test] +async fn cle_neuve_moteur_pilote_par_la_facade() { + let tmp = tempfile::tempdir().unwrap(); + let home = tmp.path().to_path_buf(); + let ctl = loki_rs::enginectl::EngineCtl::new( + supervisor(&home), + vec![("LOKI_HOME".into(), home.to_string_lossy().into_owned())], + ); + let (facade, go, _stop, _) = fakego::start_with( + home.clone(), + fakego::Opts { + system: Some(system()), + engine_ctl: Some(Arc::new(ctl)), + ..Default::default() + }, + ) + .await; + let trace = || std::fs::read_to_string(home.join("trace")).unwrap_or_default(); + // Moteur arrêté : rien à relancer. + let (_, b) = call( + facade, + "POST", + "/api/apikey", + r#"{"action":"set","key":"abc"}"#, + &[], + ) + .await; + assert_eq!( + serde_json::from_str::(&b).unwrap()["masked"], + json!("…bc") + ); + assert_eq!(trace(), ""); + // Moteur actif : relancé par le superviseur, sans passer par le Go. + let mut sleeper = std::process::Command::new("sleep") + .arg("30") + .spawn() + .unwrap(); + std::fs::write(home.join("loki-engine.pid"), sleeper.id().to_string()).unwrap(); + call(facade, "POST", "/api/apikey", r#"{"action":"clear"}"#, &[]).await; + assert_eq!(trace(), "restart\n"); + assert_eq!(*go.prestarts.lock().unwrap(), 1); + assert!(go.service_actions.lock().unwrap().is_empty()); + let _ = sleeper.kill(); + let _ = sleeper.wait(); +} + +#[tokio::test] +async fn sans_la_variable_les_interrupteurs_restent_au_go() { + let tmp = tempfile::tempdir().unwrap(); + let (facade, _go, _stop) = fakego::start(tmp.path().to_path_buf()).await; + for p in [ + "/api/internet", + "/api/computer", + "/api/apikey", + "/api/network", + ] { + let (_, _, b) = fakego::get(facade, p, &[]).await; + assert_eq!(&b[..], format!("go:{p}").as_bytes()); + } +} + +#[test] +fn fonctions_pures() { + assert_eq!(sy::key_hint("abcdé"), "…bcdé"); + assert_eq!(sy::key_hint("abcd"), "…"); + assert_eq!(sy::key_hint(""), ""); + let k = sy::gen_api_key(); + assert!(k.starts_with("sk-loki-") && k.len() == 56, "{k}"); + assert_ne!(k, sy::gen_api_key()); +}