Michael/Loki
1
0 Fork 0
mirror of https://github.com/R0m1k3/Loki.git synced 2026-10-17 12:16:21 +02:00
Code Issues Packages Projects Releases Wiki Activity

Strata : seconde carte en partage des couches au choix, l'aide aux experts reste le défaut

Browse Source 
Repris d'AJEAN 38919238 (#121 amont) et de la partie moteur de 6db3f3d0 :
STRATA_SPLIT=layers dans le preset fait lister les deux cartes dans la clé
« gpu » de la config (principale d'abord, numéros nvidia-smi) ; le serveur de
Strata en déduit --layer-split. Rien du mode d'aide ne subsiste alors : ni
--expert-cache-device1 / --remote-expert-opt, ni source d'aide à la lecture
des prompts, ni encodeur d'images déplacé ; la réserve de VRAM de
l'installeur reste. Sans la clé, ou la seconde carte coupée, rien ne change.

Propre à Loki : le placement des experts selon la RAM disponible ne choisit
jamais le mode hors RAM en partage des couches (la seconde carte n'y garde
pas d'experts, la marge calculée pour elle serait fausse).

Réglages : « split » (helper ou layers ; absent = inchangé pour un ancien
client), détail split_layers. Fenêtre Strata : la case « Carte d'appoint »
devient un choix « Seconde carte » (non utilisée, aide aux experts, partage
des couches) ; la carte de configuration du moteur actif l'affiche.

Façade Rust alignée (strata/mod.rs, strata/api.rs, tests portés) ; journal
de parité régénéré sous Linux avec les réglages et l'affichage du partage.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
MichaelandClaude Opus 5.5 committed 2026-10-16 15:16:37 +02:00
1 parent 9792953d62
commit 615e58dd2f
13 files changed
+823 -120
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

README.md
+9 -1
View File
@@ -120,7 +120,15 @@ puis installe et active le modèle en un clic (téléchargement de 66 à 84 Go d
`/data/strata`). Le modèle installé devient un preset comme les autres :
basculer dessus lance Strata, basculer sur un autre preset revient à llama.cpp.
Son engrenage ouvre les réglages propres à Strata (contexte jusqu'à 262 144,
cache KV fp16/8 bits, prédiction MTP, lecture des images, carte d'appoint).
cache KV fp16/8 bits, prédiction MTP, lecture des images, seconde carte).
La seconde carte a trois réglages : **non utilisée**, **aide aux experts** (le
défaut : elle garde et calcule une part des experts) ou **partage des couches**
(les deux cartes se répartissent les couches du modèle). Le partage est souvent
plus rapide avec deux cartes identiques (+50 % en génération mesuré par un
utilisateur d'AJEAN sur deux RTX 3060) ; sur deux cartes très différentes,
l'aide aux experts reste le meilleur choix. En partage des couches, le mode
« hors RAM » n'est jamais choisi : la seconde carte n'y garde pas d'experts.
Depuis le paquet 1.1, la carte d'appoint calcule elle-même ses experts à la
lecture des prompts (au lieu de les recopier vers la principale) : les ajouts
internal/loki/backend_strata.go
+41 -1
View File
@@ -220,7 +220,9 @@ func strataPackHasExpertsBin(args []string) bool {
// Pure : la décision se teste en table.
func strataResolveExperts(cfg map[string]string, args []string, availGB, helperGB float64) string {
binReady := strataPackHasExpertsBin(args) && strataEngineInstalled() == strataEngineAsset
dropReady := binReady && strataHelperOn(cfg) && cfg["STRATA_DROP"] != "0"
// (experts de la carte d'aide hors RAM : propre au mode d'aide, jamais en
// partage des couches, où la seconde carte ne garde pas d'experts)
dropReady := binReady && strataHelperOn(cfg) && !strataLayerSplit(cfg) && cfg["STRATA_DROP"] != "0"
arena := strataArenaGB(cfg)
switch strings.ToLower(strings.TrimSpace(cfg["STRATA_EXPERTS"])) {
case strataExpertsDisk:
@@ -504,6 +506,7 @@ type strataDetails struct {
// pour les réglages : ce que la machine permet et l'état actuel
HasHelper bool `json:"has_helper"` // une carte d'aide est disponible
HelperOn bool `json:"helper_on"`
Split bool `json:"split_layers"` // les deux cartes se partagent les couches
Vision bool `json:"vision"`
}
@@ -565,6 +568,7 @@ func strataDetailsFor(presetID string, env strataEnv) *strataDetails {
if strataHelperOn(pc) {
d.HelperOn = true
d.HelperGPU = gpuName(h)
d.Split = strataLayerSplit(pc)
}
}
d.Vision = pc["STRATA_VISION"] != "0"
@@ -968,6 +972,16 @@ func strataBuildConfig(base map[string]any, cfg map[string]string, apiKey, mode
}
helper := strings.TrimSpace(cfg["STRATA_HELPER_GPU"])
hasHelper := strataHelperOn(cfg)
if strataLayerSplit(cfg) {
// Partage des couches : le serveur de Strata le fait de lui-même quand sa
// clé « gpu » liste plusieurs cartes (--layer-split auto, réparti selon la
// VRAM libre), numérotées comme nvidia-smi. Pas de chemin « aide » : la
// réserve de VRAM de l'installeur et l'encodeur d'images restent en place.
main, _ := strconv.Atoi(strings.TrimSpace(cfg["STRATA_MAIN_GPU"]))
second, _ := strconv.Atoi(helper)
out["gpu"] = []any{main, second}
hasHelper = false
}
// Deuxième carte en aide : elle garde et CALCULE sa part des experts (le
// chemin « helper » du moteur), la carte principale reste CUDA0. Les cartes
// sont posées par CUDA_VISIBLE_DEVICES (serveStrata), PAS par la clé « gpu » :
@@ -1191,6 +1205,16 @@ func strataHelperOn(cfg map[string]string) bool {
return h != "" && h != "-1" && cfg["STRATA_HELPER"] != "0"
}
// strataLayerSplit : les deux cartes se partagent les COUCHES du modèle (clé de
// preset STRATA_SPLIT=layers) au lieu que la seconde n'aide que pour une part
// des experts. Mesuré par un utilisateur d'AJEAN sur deux RTX 3060 identiques
// (#121 amont) : la carte d'aide ne travaillait qu'à ~15 %, le partage des
// couches donnait +50 % en génération. Sur deux cartes très différentes, le
// mode d'aide reste le défaut mesuré : le partage est un choix, jamais imposé.
func strataLayerSplit(cfg map[string]string) bool {
return strataHelperOn(cfg) && cfg["STRATA_SPLIT"] == "layers"
}
// strataCudaDevices : ordre des cartes vu par le moteur, la principale en CUDA0.
func strataCudaDevices(cfg map[string]string) string {
main := strings.TrimSpace(cfg["STRATA_MAIN_GPU"])
@@ -1437,6 +1461,9 @@ type strataSettingsReq struct {
Vision bool `json:"vision"`
Helper bool `json:"helper"`
Experts string `json:"experts"` // auto, ram ou disk ; vide = inchangé
// Split : rôle de la seconde carte, « helper » (aide aux experts) ou
// « layers » (partage des couches). Vide = inchangé (anciens clients).
Split string `json:"split"`
}
// strataApplySettings réécrit les clés du preset (pure, testable).
@@ -1451,6 +1478,8 @@ func strataApplySettings(content string, r strataSettingsReq) (string, error) {
return "", fmt.Errorf("MTP hors limites : %d", r.Spec)
case r.Experts != "" && r.Experts != strataExpertsAuto && r.Experts != strataExpertsRAM && r.Experts != strataExpertsDisk:
return "", fmt.Errorf("placement des experts inconnu : %s", r.Experts)
case r.Split != "" && r.Split != "helper" && r.Split != "layers":
return "", fmt.Errorf("répartition inconnue : %s", r.Split)
}
set := map[string]string{
"CTX": strconv.Itoa(r.Ctx), "STRATA_KV": r.KV, "STRATA_SPEC": strconv.Itoa(r.Spec),
@@ -1461,6 +1490,14 @@ func strataApplySettings(content string, r strataSettingsReq) (string, error) {
set["STRATA_EXPERTS"] = r.Experts
keys = append(keys, "STRATA_EXPERTS")
}
if r.Split != "" {
// « helper » est le défaut : la clé disparaît plutôt que d'être écrite
set["STRATA_SPLIT"] = ""
if r.Split == "layers" {
set["STRATA_SPLIT"] = "layers"
}
keys = append(keys, "STRATA_SPLIT")
}
var lines []string
for _, l := range strings.Split(strings.TrimRight(content, "\n"), "\n") {
k, _, _ := strings.Cut(strings.TrimSpace(l), "=")
@@ -1470,6 +1507,9 @@ func strataApplySettings(content string, r strataSettingsReq) (string, error) {
lines = append(lines, l)
}
for _, k := range keys {
if set[k] == "" && k == "STRATA_SPLIT" {
continue
}
lines = append(lines, k+"="+set[k])
}
return strings.Join(lines, "\n") + "\n", nil
internal/loki/backend_strata_split_test.go
+124
View File
@@ -0,0 +1,124 @@
package loki
import (
"os"
"path/filepath"
"strings"
"testing"
)
// Partage des couches (#121 amont) : la clé « gpu » liste les deux cartes (le
// serveur de Strata en déduit --layer-split), et rien du mode d'aide ne subsiste.
func TestStrataBuildConfigLayerSplit(t *testing.T) {
base := map[string]any{
"args": []any{"--pack", "/p", "--vram-reserve-mib", "900"},
"vision": map[string]any{"exe": "/v"},
}
cfg := map[string]string{"STRATA_MAIN_GPU": "1", "STRATA_HELPER_GPU": "0", "STRATA_SPLIT": "layers"}
out, err := strataBuildConfig(base, cfg, "", "drop")
if err != nil {
t.Fatal(err)
}
g, ok := out["gpu"].([]any)
if !ok || len(g) != 2 || g[0] != 1 || g[1] != 0 {
t.Fatalf("gpu = %v, attendu [1 0] (principale d'abord, numéros nvidia-smi)", out["gpu"])
}
a := strataArgs(t, out)
if strataHasArg(a, "--remote-expert-opt") || strataHasArg(a, "--expert-cache-device1") {
t.Errorf("options du mode d'aide en partage des couches : %v", a)
}
if argAfter(a, "--vram-reserve-mib") != "900" {
t.Errorf("la réserve de VRAM de l'installeur doit rester : %v", a)
}
if argAfter(a, "--short-read") != "150" {
t.Errorf("lecture des prompts par la carte d'aide en partage des couches : %v", a)
}
env := out["env"].(map[string]any)
if env["STRATA_REMOTE_DROP"] != nil || env["STRATA_PREFILL_HELPER_SOURCE"] != nil {
t.Errorf("experts hors RAM ou source d'aide (mode d'aide) en partage des couches : %v", env)
}
if _, has := out["vision"].(map[string]any)["cuda_device"]; has {
t.Errorf("encodeur d'images déplacé sur la seconde carte en partage des couches")
}
// Sans la clé, rien ne change : le mode d'aide reste le défaut.
delete(cfg, "STRATA_SPLIT")
out, _ = strataBuildConfig(base, cfg, "", "drop")
if _, has := out["gpu"]; has {
t.Errorf("clé gpu posée en mode d'aide : %v", out["gpu"])
}
if !strataHasArg(strataArgs(t, out), "--remote-expert-opt") {
t.Errorf("mode d'aide perdu sans STRATA_SPLIT")
}
// Seconde carte coupée : pas de partage même si la clé traîne.
cfg["STRATA_SPLIT"], cfg["STRATA_HELPER"] = "layers", "0"
if strataLayerSplit(cfg) {
t.Errorf("partage des couches avec la seconde carte coupée")
}
}
// En partage des couches, la seconde carte ne garde pas d'experts : le mode
// hors RAM (qui compte sur elle pour en loger une part) n'est jamais choisi.
func TestStrataResolveExpertsLayerSplit(t *testing.T) {
home := t.TempDir()
t.Setenv("LOKI_HOME", home)
pack := filepath.Join(home, "pack")
_ = os.MkdirAll(pack, 0o755)
_ = os.WriteFile(filepath.Join(pack, "experts.bin"), []byte("x"), 0o644)
_ = os.MkdirAll(filepath.Join(strataSrcDir(), "engine-cuda12"), 0o755)
_ = os.WriteFile(strataEngineMarker(), []byte(strataEngineAsset+"\n"), 0o644)
args := []string{"--pack", pack}
cfg := map[string]string{"STRATA_CONFIG": "/x/strata-swift-iq3_xxs.json", "STRATA_HELPER_GPU": "1", "STRATA_DROP": "1"}
if got := strataResolveExperts(cfg, args, 44, 12); got != "drop" {
t.Fatalf("mode d'aide : %s, attendu drop", got)
}
cfg["STRATA_SPLIT"] = "layers"
if got := strataResolveExperts(cfg, args, 44, 12); got != "arena" {
t.Errorf("partage des couches : %s, attendu arena", got)
}
cfg["STRATA_EXPERTS"] = "ram"
if got := strataResolveExperts(cfg, args, 44, 12); got != "ram" {
t.Errorf("partage des couches, experts en RAM : %s, attendu ram", got)
}
}
func TestStrataApplySettingsSplit(t *testing.T) {
in := "ENGINE=strata\nSTRATA_HELPER_GPU=0\n"
req := strataSettingsReq{Ctx: 131072, KV: "fp16", Spec: 3, Vision: true, Helper: true, Split: "layers"}
out, err := strataApplySettings(in, req)
if err != nil || !strings.Contains(out, "STRATA_SPLIT=layers\n") {
t.Fatalf("STRATA_SPLIT=layers attendu : %q (%v)", out, err)
}
req.Split = "helper"
if out, _ = strataApplySettings(out, req); strings.Contains(out, "STRATA_SPLIT") {
t.Errorf("retour au mode d'aide : la clé doit disparaître : %q", out)
}
// un ancien client n'envoie pas « split » : la clé est conservée
req.Split = ""
out, _ = strataApplySettings("STRATA_SPLIT=layers\n", req)
if !strings.Contains(out, "STRATA_SPLIT=layers") {
t.Errorf("clé effacée par un client qui ne la connaît pas : %q", out)
}
req.Split = "nimporte"
if _, err := strataApplySettings(in, req); err == nil {
t.Errorf("répartition inconnue acceptée")
}
}
func TestStrataDetailsForLayerSplit(t *testing.T) {
home := t.TempDir()
t.Setenv("LOKI_HOME", home)
base := filepath.Join(home, "base.json")
_ = os.WriteFile(base, []byte(`{"args":["--pack","/p"],"vision":{"exe":"v"}}`), 0o644)
_ = os.MkdirAll(presetsDir(), 0o755)
preset := "ENGINE=strata\nSTRATA_CONFIG=" + base + "\nSTRATA_MAIN_GPU=1\nSTRATA_HELPER_GPU=0\nSTRATA_SPLIT=layers\n"
_ = os.WriteFile(filepath.Join(presetsDir(), "X.env"), []byte(preset), 0o644)
env := strataEnv{GPUs: []strataGPU{{Index: 0, Name: "NVIDIA GeForce RTX 3060", VRAMGB: 12}, {Index: 1, Name: "NVIDIA GeForce RTX 3060", VRAMGB: 12}}}
d := strataDetailsFor("X", env)
if d == nil {
t.Fatal("pas de détails")
}
if !d.Split || !d.HelperOn || d.HelperGPU == "" || d.VisionGPU != d.MainGPU || d.ShortRead != "150" {
t.Errorf("%+v", *d)
}
}
internal/loki/parity_strata_test.go
+10 -1
View File
@@ -333,7 +333,7 @@ func TestPariteStrata(t *testing.T) {
mu.Unlock()
st.After = map[string]string{}
cfg := ReadConfig()
for _, k := range []string{"ENGINE", "STRATA_CONFIG", "CTX", "STRATA_KV", "STRATA_HELPER"} {
for _, k := range []string{"ENGINE", "STRATA_CONFIG", "CTX", "STRATA_KV", "STRATA_HELPER", "STRATA_SPLIT"} {
if v := cfg[k]; v != "" {
st.After["config/"+k] = mask(v)
}
@@ -382,6 +382,10 @@ func TestPariteStrata(t *testing.T) {
Method: "POST", Path: "/api/strata/settings", Body: settings("S", 98304, "")},
{Name: "réglages du preset actif pendant une optimisation", Tune: true,
Method: "POST", Path: "/api/strata/settings", Body: settings("S", 98304, "")},
{Name: "réglages, répartition inconnue", Method: "POST", Path: "/api/strata/settings", Body: settings("S", 98304, `,"split":"nimporte"`)},
{Name: "réglages, partage des couches", Method: "POST", Path: "/api/strata/settings", Body: settings("S", 98304, `,"split":"layers"`)},
{Name: "réglages d'un ancien client, partage gardé", Method: "POST", Path: "/api/strata/settings", Body: settings("S", 98304, "")},
{Name: "réglages, retour à l'aide aux experts", Method: "POST", Path: "/api/strata/settings", Body: settings("S", 98304, `,"split":"helper"`)},
// Installation.
{Name: "installation en GET", Method: "GET", Path: "/api/strata/install"},
@@ -417,6 +421,11 @@ func TestPariteStrata(t *testing.T) {
"STRATA_CONFIG={{HOME}}/strata/" + srcDir + "/strata-swift-iq3_xxs.json\nSTRATA_MAIN_GPU=1\nSTRATA_HELPER_GPU=0\n" +
"STRATA_VISION=1\nSTRATA_EXPERTS=auto\nSTRATA_DROP=1\nSTRATA_KV=fp16\nSTRATA_KV_RESIDENT=32768\nSTRATA_SPEC=3\nCTX=196608\nREASONING=off\n",
}, Method: "GET", Path: "/api/strata"},
{Name: "seconde carte en partage des couches", Files: map[string]string{
"presets/STRATA FLASH NEXT SWIFT 1.5 IQ3_XXS.env": "# NAME=Qwen3.8 Flash Next Swift 1.5 IQ3_XXS (Strata)\nENGINE=strata\n" +
"STRATA_CONFIG={{HOME}}/strata/" + srcDir + "/strata-swift-iq3_xxs.json\nSTRATA_MAIN_GPU=1\nSTRATA_HELPER_GPU=0\n" +
"STRATA_VISION=1\nSTRATA_EXPERTS=auto\nSTRATA_DROP=1\nSTRATA_KV=fp16\nSTRATA_KV_RESIDENT=32768\nSTRATA_SPEC=3\nCTX=196608\nREASONING=off\nSTRATA_SPLIT=layers\n",
}, Method: "GET", Path: "/api/strata"},
{Name: "mode hors RAM coupé dans le preset", Files: map[string]string{
"presets/STRATA FLASH NEXT SWIFT 1.5 IQ3_XXS.env": "# NAME=Qwen3.8 Flash Next Swift 1.5 IQ3_XXS (Strata)\nENGINE=strata\n" +
"STRATA_CONFIG={{HOME}}/strata/" + srcDir + "/strata-swift-iq3_xxs.json\nSTRATA_MAIN_GPU=1\nSTRATA_HELPER_GPU=0\n" +
internal/loki/ui/index.html
+11 -3
View File
@@ -4521,6 +4521,7 @@ async function loadCfg(){
rows.push(row('CTX', d.ctx));
rows.push(row('KV', d.kv+(d.kv_resident ? ' · '+k(d.kv_resident)+' VRAM' : '')));
rows.push(row('MTP', d.spec));
if(d.helper_gpu) rows.push(row('GPU 2', d.split_layers ? 'couches' : 'aide', d.helper_gpu));
rows.push(row('EXPERTS', ({drop:'mlock (hors appoint)', ram:'RAM', arena:'experts.bin', disk:'mmap (disque)'})[d.mode] || (d.mmap ? 'mmap' : 'RAM'), d.mode_label||''));
if(d.vision_gpu) rows.push(row('VISION', 'on', d.vision_gpu));
}else if(c.CTX) rows.push(row('CTX', c.CTX));
@@ -9281,7 +9282,11 @@ function strataRenderSettings(fit){
['Experts', sel('strata-s-experts', [['auto','auto (selon la RAM libre)'],['ram','en RAM (le plus rapide)'],['disk','depuis le disque (mmap, économe)']], d.experts||'auto')],
['Lire les images', sw('strata-s-vision', d.vision)],
];
if(d.has_helper) rows.push(["Carte d'appoint", sw('strata-s-helper', d.helper_on)]);
// Seconde carte : aide aux experts (défaut mesuré) ou partage des couches,
// souvent plus rapide avec deux cartes identiques (#121 d'AJEAN).
if(d.has_helper) rows.push(['Seconde carte', sel('strata-s-helper',
[['off', 'non utilisée'], ['helper', 'aide aux experts'], ['layers', 'partage des couches']],
!d.helper_on ? 'off' : d.split_layers ? 'layers' : 'helper')]);
document.getElementById('strata-settings').innerHTML = rows.map(([a,b])=>
'<div class="moe-kv"><span>'+escHtml(a)+'</span><span>'+b+'</span></div>').join('');
strataCtxLbl();
@@ -9301,8 +9306,11 @@ async function strataSaveSettings(){
if(!fit || !fit.preset) return;
const v = id => document.getElementById(id);
const body = {preset: fit.preset, ctx: +v('strata-s-ctx').value, kv: v('strata-s-kv').value, spec: +v('strata-s-spec').value,
vision: v('strata-s-vision').checked, helper: v('strata-s-helper') ? v('strata-s-helper').checked : false,
vision: v('strata-s-vision').checked,
experts: v('strata-s-experts') ? v('strata-s-experts').value : ''};
const second = v('strata-s-helper') ? v('strata-s-helper').value : 'off';
body.helper = second !== 'off';
if(body.helper) body.split = second;
if(fit.active && !await askConfirm('Le modèle va redémarrer avec ces réglages (quelques minutes).', {title:'Strata — Qwen3.8 Flash Next', okText:'Enregistrer'})) return;
const r = await jpost('/api/strata/settings', body);
if(!r.ok){ toast('Erreur : '+(r.error||'')); return; }
@@ -9321,7 +9329,7 @@ function strataRenderDetails(fit){
const k = n => (+n >= 1024 ? Math.round(+n/1024)+'K' : n);
const rows = [
['Carte principale', d.main_gpu],
["Carte d'appoint", d.helper_gpu || 'aucune'],
['Seconde carte', d.helper_gpu ? d.helper_gpu + (d.split_layers ? ' · partage des couches' : ' · aide aux experts') : 'aucune'],
['Lecture des images', d.vision_gpu || 'aucune'],
['Contexte', (+d.ctx).toLocaleString('fr-FR')+' jetons'],
['Cache KV', d.kv + (d.kv_resident ? ' · '+k(d.kv_resident)+' en VRAM, le reste en RAM' : '')],
internal/loki/ui/src/js/05-status.js
+1
View File
@@ -330,6 +330,7 @@ async function loadCfg(){
rows.push(row('CTX', d.ctx));
rows.push(row('KV', d.kv+(d.kv_resident ? ' · '+k(d.kv_resident)+' VRAM' : '')));
rows.push(row('MTP', d.spec));
if(d.helper_gpu) rows.push(row('GPU 2', d.split_layers ? 'couches' : 'aide', d.helper_gpu));
rows.push(row('EXPERTS', ({drop:'mlock (hors appoint)', ram:'RAM', arena:'experts.bin', disk:'mmap (disque)'})[d.mode] || (d.mmap ? 'mmap' : 'RAM'), d.mode_label||''));
if(d.vision_gpu) rows.push(row('VISION', 'on', d.vision_gpu));
}else if(c.CTX) rows.push(row('CTX', c.CTX));
internal/loki/ui/src/js/10b-strata.js
+10 -3
View File
@@ -128,7 +128,11 @@ function strataRenderSettings(fit){
['Experts', sel('strata-s-experts', [['auto','auto (selon la RAM libre)'],['ram','en RAM (le plus rapide)'],['disk','depuis le disque (mmap, économe)']], d.experts||'auto')],
['Lire les images', sw('strata-s-vision', d.vision)],
];
if(d.has_helper) rows.push(["Carte d'appoint", sw('strata-s-helper', d.helper_on)]);
// Seconde carte : aide aux experts (défaut mesuré) ou partage des couches,
// souvent plus rapide avec deux cartes identiques (#121 d'AJEAN).
if(d.has_helper) rows.push(['Seconde carte', sel('strata-s-helper',
[['off', 'non utilisée'], ['helper', 'aide aux experts'], ['layers', 'partage des couches']],
!d.helper_on ? 'off' : d.split_layers ? 'layers' : 'helper')]);
document.getElementById('strata-settings').innerHTML = rows.map(([a,b])=>
'<div class="moe-kv"><span>'+escHtml(a)+'</span><span>'+b+'</span></div>').join('');
strataCtxLbl();
@@ -148,8 +152,11 @@ async function strataSaveSettings(){
if(!fit || !fit.preset) return;
const v = id => document.getElementById(id);
const body = {preset: fit.preset, ctx: +v('strata-s-ctx').value, kv: v('strata-s-kv').value, spec: +v('strata-s-spec').value,
vision: v('strata-s-vision').checked, helper: v('strata-s-helper') ? v('strata-s-helper').checked : false,
vision: v('strata-s-vision').checked,
experts: v('strata-s-experts') ? v('strata-s-experts').value : ''};
const second = v('strata-s-helper') ? v('strata-s-helper').value : 'off';
body.helper = second !== 'off';
if(body.helper) body.split = second;
if(fit.active && !await askConfirm('Le modèle va redémarrer avec ces réglages (quelques minutes).', {title:'Strata — Qwen3.8 Flash Next', okText:'Enregistrer'})) return;
const r = await jpost('/api/strata/settings', body);
if(!r.ok){ toast('Erreur : '+(r.error||'')); return; }
@@ -168,7 +175,7 @@ function strataRenderDetails(fit){
const k = n => (+n >= 1024 ? Math.round(+n/1024)+'K' : n);
const rows = [
['Carte principale', d.main_gpu],
["Carte d'appoint", d.helper_gpu || 'aucune'],
['Seconde carte', d.helper_gpu ? d.helper_gpu + (d.split_layers ? ' · partage des couches' : ' · aide aux experts') : 'aucune'],
['Lecture des images', d.vision_gpu || 'aucune'],
['Contexte', (+d.ctx).toLocaleString('fr-FR')+' jetons'],
['Cache KV', d.kv + (d.kv_resident ? ' · '+k(d.kv_resident)+' en VRAM, le reste en RAM' : '')],
rust/loki-rs/src/strata/api.rs
+4 -1
View File
@@ -345,7 +345,7 @@ impl Strata {
"mmap": mode == "disk" || mode == "arena", "drop": mode == "drop",
"experts": experts, "mode_label": experts_label(&mode), "mode": mode,
"main_gpu": main_gpu,
"has_helper": false, "helper_on": false,
"has_helper": false, "helper_on": false, "split_layers": false,
});
let kvr = val("--kv-resident");
if !kvr.is_empty() {
@@ -356,6 +356,7 @@ impl Strata {
d["has_helper"] = json!(true);
if helper_on(&pc) {
d["helper_on"] = json!(true);
d["split_layers"] = json!(layer_split(&pc));
let hg = gpu_name(h);
if !hg.is_empty() {
d["helper_gpu"] = json!(hg);
@@ -825,6 +826,7 @@ pub async fn serve(
("vision", Kind::Bool),
("helper", Kind::Bool),
("experts", Kind::Str),
("split", Kind::Str),
],
);
if let Some(e) = d.err.clone() {
@@ -838,6 +840,7 @@ pub async fn serve(
vision: d.bool("vision"),
helper: d.bool("helper"),
experts: d.str("experts"),
split: d.str("split"),
};
if r.preset.contains(['/', '\\']) || r.preset.is_empty() {
return fail(StatusCode::BAD_REQUEST, "preset invalide");
rust/loki-rs/src/strata/mod.rs
+35 -2
View File
@@ -287,6 +287,13 @@ pub fn helper_on(cfg: &Cfg) -> bool {
!h.is_empty() && h != "-1" && cfg_get(cfg, "STRATA_HELPER") != "0"
}
/// `strataLayerSplit` : les deux cartes se partagent les couches
/// (`STRATA_SPLIT=layers`) au lieu que la seconde n'aide que pour une part
/// des experts. Un choix, jamais imposé : le mode d'aide reste le défaut.
pub fn layer_split(cfg: &Cfg) -> bool {
helper_on(cfg) && cfg_get(cfg, "STRATA_SPLIT") == "layers"
}
/// `strataResolveExperts` : le mode lancé (drop, ram, arena, disk), d'après
/// le choix du preset et la RAM disponible.
pub fn resolve_experts(
@@ -297,7 +304,9 @@ pub fn resolve_experts(
helper_gb: f64,
) -> &'static str {
let bin_ready = pack_has_experts_bin(args) && paths.engine_installed() == ENGINE_ASSET;
let drop_ready = bin_ready && helper_on(cfg) && cfg_get(cfg, "STRATA_DROP") != "0";
// (jamais en partage des couches : la seconde carte n'y garde pas d'experts)
let drop_ready =
bin_ready && helper_on(cfg) && !layer_split(cfg) && cfg_get(cfg, "STRATA_DROP") != "0";
let arena = arena_gb(cfg);
match cfg_get(cfg, "STRATA_EXPERTS")
.trim()
@@ -411,7 +420,18 @@ pub fn build_config(base: &Value, cfg: &Cfg, api_key: &str, mode: &str) -> Resul
args = set_arg(args, flag, v);
}
let helper = cfg_get(cfg, "STRATA_HELPER_GPU").trim();
let has_helper = helper_on(cfg);
let mut has_helper = helper_on(cfg);
if layer_split(cfg) {
// Partage des couches : la clé « gpu » liste les deux cartes (numéros
// nvidia-smi), le serveur de Strata en déduit --layer-split.
let main = cfg_get(cfg, "STRATA_MAIN_GPU")
.trim()
.parse::<i64>()
.unwrap_or(0);
let second = helper.parse::<i64>().unwrap_or(0);
out.insert("gpu".into(), json!([main, second]));
has_helper = false;
}
if has_helper {
args = set_arg(args, "--expert-cache-device1", "auto");
if !has_arg(&args, "--remote-expert-opt") {
@@ -788,6 +808,8 @@ pub struct SettingsReq {
pub vision: bool,
pub helper: bool,
pub experts: String,
/// « helper » ou « layers » ; vide = inchangé (anciens clients).
pub split: String,
}
/// `strataApplySettings` : réécrit les clés du preset. Pure.
@@ -808,6 +830,9 @@ pub fn apply_settings(content: &str, r: &SettingsReq) -> Result<String, String>
{
return Err(format!("placement des experts inconnu : {}", r.experts));
}
if !r.split.is_empty() && r.split != "helper" && r.split != "layers" {
return Err(format!("répartition inconnue : {}", r.split));
}
let mut set: Vec<(&str, String)> = vec![
("CTX", r.ctx.to_string()),
("STRATA_KV", r.kv.clone()),
@@ -818,6 +843,11 @@ pub fn apply_settings(content: &str, r: &SettingsReq) -> Result<String, String>
if !r.experts.is_empty() {
set.push(("STRATA_EXPERTS", r.experts.clone()));
}
if !r.split.is_empty() {
// « helper » est le défaut : la clé disparaît plutôt que d'être écrite
let v = if r.split == "layers" { "layers" } else { "" };
set.push(("STRATA_SPLIT", v.into()));
}
let mut lines: Vec<String> = Vec::new();
for l in content.trim_end_matches('\n').split('\n') {
let k = l.trim().split_once('=').map_or(l.trim(), |(k, _)| k);
@@ -827,6 +857,9 @@ pub fn apply_settings(content: &str, r: &SettingsReq) -> Result<String, String>
lines.push(l.to_string());
}
for (k, v) in set {
if k == "STRATA_SPLIT" && v.is_empty() {
continue;
}
lines.push(format!("{k}={v}"));
}
Ok(lines.join("\n") + "\n")
rust/loki-rs/src/strata/tests.rs
+83
View File
@@ -405,6 +405,89 @@ fn hors_ram_reserve_et_contexte() {
}
}
// `TestStrataBuildConfigLayerSplit`.
#[test]
fn partage_des_couches() {
let base = json!({
"args": ["--pack", "/p", "--vram-reserve-mib", "900"],
"vision": { "exe": "/v" },
});
let mut c = cfg(&[
("STRATA_MAIN_GPU", "1"),
("STRATA_HELPER_GPU", "0"),
("STRATA_SPLIT", "layers"),
]);
let out = build_config(&base, &c, "", "drop").unwrap();
assert_eq!(out["gpu"], json!([1, 0]));
let a = args_of(&out);
assert!(!has_arg(&a, "--remote-expert-opt") && !has_arg(&a, "--expert-cache-device1"));
assert_eq!(arg_after(&a, "--vram-reserve-mib"), "900");
assert_eq!(arg_after(&a, "--short-read"), "150");
assert!(out["env"].get("STRATA_REMOTE_DROP").is_none());
assert!(out["env"].get("STRATA_PREFILL_HELPER_SOURCE").is_none());
assert!(out["vision"].get("cuda_device").is_none());
c.remove("STRATA_SPLIT");
let out = build_config(&base, &c, "", "drop").unwrap();
assert!(out.get("gpu").is_none());
assert!(has_arg(&args_of(&out), "--remote-expert-opt"));
c.insert("STRATA_SPLIT".into(), "layers".into());
c.insert("STRATA_HELPER".into(), "0".into());
assert!(!layer_split(&c));
}
// `TestStrataResolveExpertsLayerSplit`.
#[test]
fn partage_des_couches_sans_hors_ram() {
let d = tempfile::tempdir().unwrap();
let p = Paths {
home: d.path().to_path_buf(),
};
let pack = d.path().join("pack");
std::fs::create_dir_all(&pack).unwrap();
std::fs::write(pack.join("experts.bin"), "x").unwrap();
std::fs::create_dir_all(p.src_dir().join("engine-cuda12")).unwrap();
std::fs::write(p.engine_marker(), format!("{ENGINE_ASSET}\n")).unwrap();
let args = vec!["--pack".to_string(), pack.to_string_lossy().into_owned()];
let mut c = cfg(&[
("STRATA_CONFIG", "/x/strata-swift-iq3_xxs.json"),
("STRATA_HELPER_GPU", "1"),
("STRATA_DROP", "1"),
]);
assert_eq!(resolve_experts(&p, &c, &args, 44.0, 12.0), "drop");
c.insert("STRATA_SPLIT".into(), "layers".into());
assert_eq!(resolve_experts(&p, &c, &args, 44.0, 12.0), "arena");
c.insert("STRATA_EXPERTS".into(), "ram".into());
assert_eq!(resolve_experts(&p, &c, &args, 44.0, 12.0), "ram");
}
// `TestStrataApplySettingsSplit`.
#[test]
fn reglages_de_la_seconde_carte() {
let input = "ENGINE=strata\nSTRATA_HELPER_GPU=0\n";
let mut r = SettingsReq {
ctx: 131072,
kv: "fp16".into(),
spec: 3,
vision: true,
helper: true,
split: "layers".into(),
..SettingsReq::default()
};
let out = apply_settings(input, &r).unwrap();
assert!(out.contains("STRATA_SPLIT=layers\n"), "{out:?}");
r.split = "helper".into();
let out = apply_settings(&out, &r).unwrap();
assert!(!out.contains("STRATA_SPLIT"), "{out:?}");
r.split = String::new();
let out = apply_settings("STRATA_SPLIT=layers\n", &r).unwrap();
assert!(out.contains("STRATA_SPLIT=layers"), "{out:?}");
r.split = "nimporte".into();
assert_eq!(
apply_settings(input, &r).unwrap_err(),
"répartition inconnue : nimporte"
);
}
#[test]
fn reglages_des_experts() {
let input = "ENGINE=strata\nSTRATA_EXPERTS=auto\n";
rust/loki-rs/tests/parity_bench/bench.json
+108 -108
View File
@@ -231,8 +231,8 @@
"predicted_ms": 6000,
"predicted_n": 300,
"predicted_per_second": 50,
"prompt_ms": 1018,
"prompt_n": 2036,
"prompt_ms": 1014.5,
"prompt_n": 2029,
"prompt_per_second": 2000,
"protocol": 2
},
@@ -304,7 +304,7 @@
"max_tokens": 300,
"messages": [
{
"content": "<8129 octets a3e9aa90e552>",
"content": "<8102 octets f67301f37ff4>",
"role": "user"
}
],
@@ -335,8 +335,8 @@
"predicted_ms": 6000,
"predicted_n": 300,
"predicted_per_second": 50,
"prompt_ms": 1018,
"prompt_n": 2036,
"prompt_ms": 1014.5,
"prompt_n": 2029,
"prompt_per_second": 2000,
"protocol": 2
}
@@ -358,8 +358,8 @@
"predicted_ms": 6000,
"predicted_n": 300,
"predicted_per_second": 50,
"prompt_ms": 1018,
"prompt_n": 2036,
"prompt_ms": 1014.5,
"prompt_n": 2029,
"prompt_per_second": 2000,
"protocol": 2
}
@@ -387,8 +387,8 @@
"predicted_ms": 6000,
"predicted_n": 300,
"predicted_per_second": 50,
"prompt_ms": 1018,
"prompt_n": 2036,
"prompt_ms": 1014.5,
"prompt_n": 2029,
"prompt_per_second": 2000,
"protocol": 2
}
@@ -410,8 +410,8 @@
"predicted_ms": 6000,
"predicted_n": 300,
"predicted_per_second": 50,
"prompt_ms": 1018,
"prompt_n": 2036,
"prompt_ms": 1014.5,
"prompt_n": 2029,
"prompt_per_second": 2000,
"protocol": 2
}
@@ -433,8 +433,8 @@
"predicted_ms": 6000,
"predicted_n": 300,
"predicted_per_second": 50,
"prompt_ms": 1018,
"prompt_n": 2036,
"prompt_ms": 1014.5,
"prompt_n": 2029,
"prompt_per_second": 2000,
"protocol": 2
}
@@ -542,8 +542,8 @@
"predicted_ms": 6000,
"predicted_n": 300,
"predicted_per_second": 50,
"prompt_ms": 1018,
"prompt_n": 2036,
"prompt_ms": 1014.5,
"prompt_n": 2029,
"prompt_per_second": 2000,
"protocol": 2
},
@@ -615,7 +615,7 @@
"max_tokens": 300,
"messages": [
{
"content": "<8129 octets a3e9aa90e552>",
"content": "<8102 octets f67301f37ff4>",
"role": "user"
}
],
@@ -855,8 +855,8 @@
"predicted_ms": 6000,
"predicted_n": 300,
"predicted_per_second": 50,
"prompt_ms": 1018,
"prompt_n": 2036,
"prompt_ms": 1014.5,
"prompt_n": 2029,
"prompt_per_second": 2000,
"protocol": 2
}
@@ -947,8 +947,8 @@
"predicted_ms": 6000,
"predicted_n": 300,
"predicted_per_second": 50,
"prompt_ms": 1018,
"prompt_n": 2036,
"prompt_ms": 1014.5,
"prompt_n": 2029,
"prompt_per_second": 2000,
"protocol": 2
}
@@ -993,8 +993,8 @@
"predicted_ms": 6000,
"predicted_n": 300,
"predicted_per_second": 50,
"prompt_ms": 1018,
"prompt_n": 2036,
"prompt_ms": 1014.5,
"prompt_n": 2029,
"prompt_per_second": 2000,
"protocol": 2
},
@@ -1066,7 +1066,7 @@
"max_tokens": 300,
"messages": [
{
"content": "<8129 octets a3e9aa90e552>",
"content": "<8102 octets f67301f37ff4>",
"role": "user"
}
],
@@ -1103,8 +1103,8 @@
"predicted_ms": 6000,
"predicted_n": 300,
"predicted_per_second": 50,
"prompt_ms": 1018,
"prompt_n": 2036,
"prompt_ms": 1014.5,
"prompt_n": 2029,
"prompt_per_second": 2000,
"protocol": 2
}
@@ -1132,8 +1132,8 @@
"predicted_ms": 6000,
"predicted_n": 300,
"predicted_per_second": 50,
"prompt_ms": 1018,
"prompt_n": 2036,
"prompt_ms": 1014.5,
"prompt_n": 2029,
"prompt_per_second": 2000,
"protocol": 2
}
@@ -1241,8 +1241,8 @@
"predicted_ms": 6000,
"predicted_n": 300,
"predicted_per_second": 50,
"prompt_ms": 1018,
"prompt_n": 2036,
"prompt_ms": 1014.5,
"prompt_n": 2029,
"prompt_per_second": 2000,
"protocol": 2
},
@@ -1314,7 +1314,7 @@
"max_tokens": 300,
"messages": [
{
"content": "<8129 octets a3e9aa90e552>",
"content": "<8102 octets f67301f37ff4>",
"role": "user"
}
],
@@ -1554,8 +1554,8 @@
"predicted_ms": 6000,
"predicted_n": 300,
"predicted_per_second": 50,
"prompt_ms": 1018,
"prompt_n": 2036,
"prompt_ms": 1014.5,
"prompt_n": 2029,
"prompt_per_second": 2000,
"protocol": 2
}
@@ -1646,8 +1646,8 @@
"predicted_ms": 6000,
"predicted_n": 300,
"predicted_per_second": 50,
"prompt_ms": 1018,
"prompt_n": 2036,
"prompt_ms": 1014.5,
"prompt_n": 2029,
"prompt_per_second": 2000,
"protocol": 2
}
@@ -1686,8 +1686,8 @@
"predicted_ms": 81920,
"predicted_n": 4096,
"predicted_per_second": 50,
"prompt_ms": 16401.5,
"prompt_n": 32803,
"prompt_ms": 16403,
"prompt_n": 32806,
"prompt_per_second": 2000,
"protocol": 2
},
@@ -1759,7 +1759,7 @@
"max_tokens": 4096,
"messages": [
{
"content": "<131199 octets aa7e053a18e5>",
"content": "<131211 octets 96987502f054>",
"role": "user"
}
],
@@ -1790,8 +1790,8 @@
"predicted_ms": 81920,
"predicted_n": 4096,
"predicted_per_second": 50,
"prompt_ms": 16401.5,
"prompt_n": 32803,
"prompt_ms": 16403,
"prompt_n": 32806,
"prompt_per_second": 2000,
"protocol": 2
}
@@ -1813,8 +1813,8 @@
"predicted_ms": 81920,
"predicted_n": 4096,
"predicted_per_second": 50,
"prompt_ms": 16401.5,
"prompt_n": 32803,
"prompt_ms": 16403,
"prompt_n": 32806,
"prompt_per_second": 2000,
"protocol": 2
}
@@ -2093,7 +2093,7 @@
"max_tokens": 16,
"messages": [
{
"content": "<3740 octets 48e62bdc3898>",
"content": "<3743 octets 302346ae5a44>",
"role": "user"
}
],
@@ -2187,8 +2187,8 @@
"predicted_ms": 6000,
"predicted_n": 300,
"predicted_per_second": 50,
"prompt_ms": 1018,
"prompt_n": 2036,
"prompt_ms": 1014.5,
"prompt_n": 2029,
"prompt_per_second": 2000,
"protocol": 2
},
@@ -2260,7 +2260,7 @@
"max_tokens": 300,
"messages": [
{
"content": "<8129 octets a3e9aa90e552>",
"content": "<8102 octets f67301f37ff4>",
"role": "user"
}
],
@@ -2291,8 +2291,8 @@
"predicted_ms": 6000,
"predicted_n": 300,
"predicted_per_second": 50,
"prompt_ms": 1018,
"prompt_n": 2036,
"prompt_ms": 1014.5,
"prompt_n": 2029,
"prompt_per_second": 2000,
"protocol": 2
}
@@ -2314,8 +2314,8 @@
"predicted_ms": 6000,
"predicted_n": 300,
"predicted_per_second": 50,
"prompt_ms": 1018,
"prompt_n": 2036,
"prompt_ms": 1014.5,
"prompt_n": 2029,
"prompt_per_second": 2000,
"protocol": 2
}
@@ -2420,8 +2420,8 @@
"predicted_ms": 6000,
"predicted_n": 300,
"predicted_per_second": 50,
"prompt_ms": 1018,
"prompt_n": 2036,
"prompt_ms": 1014.5,
"prompt_n": 2029,
"prompt_per_second": 2000,
"protocol": 2
}
@@ -2443,8 +2443,8 @@
"predicted_ms": 6000,
"predicted_n": 300,
"predicted_per_second": 50,
"prompt_ms": 1018,
"prompt_n": 2036,
"prompt_ms": 1014.5,
"prompt_n": 2029,
"prompt_per_second": 2000,
"protocol": 2
}
@@ -2541,7 +2541,7 @@
"max_tokens": 300,
"messages": [
{
"content": "<8129 octets a3e9aa90e552>",
"content": "<8102 octets f67301f37ff4>",
"role": "user"
}
],
@@ -2572,8 +2572,8 @@
"predicted_ms": 6000,
"predicted_n": 300,
"predicted_per_second": 50,
"prompt_ms": 1018,
"prompt_n": 2036,
"prompt_ms": 1014.5,
"prompt_n": 2029,
"prompt_per_second": 2000,
"protocol": 2
}
@@ -2595,8 +2595,8 @@
"predicted_ms": 6000,
"predicted_n": 300,
"predicted_per_second": 50,
"prompt_ms": 1018,
"prompt_n": 2036,
"prompt_ms": 1014.5,
"prompt_n": 2029,
"prompt_per_second": 2000,
"protocol": 2
}
@@ -2693,7 +2693,7 @@
"max_tokens": 300,
"messages": [
{
"content": "<8129 octets a3e9aa90e552>",
"content": "<8102 octets f67301f37ff4>",
"role": "user"
}
],
@@ -2747,8 +2747,8 @@
"predicted_ms": 6000,
"predicted_n": 300,
"predicted_per_second": 50,
"prompt_ms": 1018,
"prompt_n": 2036,
"prompt_ms": 1014.5,
"prompt_n": 2029,
"prompt_per_second": 2000,
"protocol": 2
}
@@ -2770,8 +2770,8 @@
"predicted_ms": 6000,
"predicted_n": 300,
"predicted_per_second": 50,
"prompt_ms": 1018,
"prompt_n": 2036,
"prompt_ms": 1014.5,
"prompt_n": 2029,
"prompt_per_second": 2000,
"protocol": 2
}
@@ -2868,7 +2868,7 @@
"max_tokens": 300,
"messages": [
{
"content": "<8129 octets a3e9aa90e552>",
"content": "<8102 octets f67301f37ff4>",
"role": "user"
}
],
@@ -2992,8 +2992,8 @@
"predicted_ms": 6000,
"predicted_n": 300,
"predicted_per_second": 50,
"prompt_ms": 1018,
"prompt_n": 2036,
"prompt_ms": 1014.5,
"prompt_n": 2029,
"prompt_per_second": 2000,
"protocol": 2
}
@@ -3015,8 +3015,8 @@
"predicted_ms": 6000,
"predicted_n": 300,
"predicted_per_second": 50,
"prompt_ms": 1018,
"prompt_n": 2036,
"prompt_ms": 1014.5,
"prompt_n": 2029,
"prompt_per_second": 2000,
"protocol": 2
}
@@ -3121,8 +3121,8 @@
"predicted_ms": 6000,
"predicted_n": 300,
"predicted_per_second": 50,
"prompt_ms": 1018,
"prompt_n": 2036,
"prompt_ms": 1014.5,
"prompt_n": 2029,
"prompt_per_second": 2000,
"protocol": 2
}
@@ -3144,8 +3144,8 @@
"predicted_ms": 6000,
"predicted_n": 300,
"predicted_per_second": 50,
"prompt_ms": 1018,
"prompt_n": 2036,
"prompt_ms": 1014.5,
"prompt_n": 2029,
"prompt_per_second": 2000,
"protocol": 2
}
@@ -3189,8 +3189,8 @@
"predicted_ms": 6000,
"predicted_n": 300,
"predicted_per_second": 50,
"prompt_ms": 1018,
"prompt_n": 2036,
"prompt_ms": 1014.5,
"prompt_n": 2029,
"prompt_per_second": 2000,
"protocol": 2
},
@@ -3287,7 +3287,7 @@
"max_tokens": 300,
"messages": [
{
"content": "<8129 octets a3e9aa90e552>",
"content": "<8102 octets f67301f37ff4>",
"role": "user"
}
],
@@ -3319,8 +3319,8 @@
"predicted_ms": 6000,
"predicted_n": 300,
"predicted_per_second": 50,
"prompt_ms": 1018,
"prompt_n": 2036,
"prompt_ms": 1014.5,
"prompt_n": 2029,
"prompt_per_second": 2000,
"protocol": 2
}
@@ -3342,8 +3342,8 @@
"predicted_ms": 6000,
"predicted_n": 300,
"predicted_per_second": 50,
"prompt_ms": 1018,
"prompt_n": 2036,
"prompt_ms": 1014.5,
"prompt_n": 2029,
"prompt_per_second": 2000,
"protocol": 2
}
@@ -3383,8 +3383,8 @@
"predicted_ms": 6000,
"predicted_n": 300,
"predicted_per_second": 50,
"prompt_ms": 1018,
"prompt_n": 2036,
"prompt_ms": 1014.5,
"prompt_n": 2029,
"prompt_per_second": 2000,
"protocol": 2
},
@@ -3457,7 +3457,7 @@
"max_tokens": 300,
"messages": [
{
"content": "<8129 octets a3e9aa90e552>",
"content": "<8102 octets f67301f37ff4>",
"role": "user"
}
],
@@ -3489,8 +3489,8 @@
"predicted_ms": 6000,
"predicted_n": 300,
"predicted_per_second": 50,
"prompt_ms": 1018,
"prompt_n": 2036,
"prompt_ms": 1014.5,
"prompt_n": 2029,
"prompt_per_second": 2000,
"protocol": 2
}
@@ -3512,8 +3512,8 @@
"predicted_ms": 6000,
"predicted_n": 300,
"predicted_per_second": 50,
"prompt_ms": 1018,
"prompt_n": 2036,
"prompt_ms": 1014.5,
"prompt_n": 2029,
"prompt_per_second": 2000,
"protocol": 2
}
@@ -3579,8 +3579,8 @@
"predicted_ms": 6000,
"predicted_n": 300,
"predicted_per_second": 50,
"prompt_ms": 1018,
"prompt_n": 2036,
"prompt_ms": 1014.5,
"prompt_n": 2029,
"prompt_per_second": 2000,
"protocol": 2
},
@@ -3652,7 +3652,7 @@
"max_tokens": 300,
"messages": [
{
"content": "<8129 octets a3e9aa90e552>",
"content": "<8102 octets f67301f37ff4>",
"role": "user"
}
],
@@ -3737,8 +3737,8 @@
"predicted_ms": 6000,
"predicted_n": 300,
"predicted_per_second": 50,
"prompt_ms": 1018,
"prompt_n": 2036,
"prompt_ms": 1014.5,
"prompt_n": 2029,
"prompt_per_second": 2000,
"protocol": 2
}
@@ -3760,8 +3760,8 @@
"predicted_ms": 6000,
"predicted_n": 300,
"predicted_per_second": 50,
"prompt_ms": 1018,
"prompt_n": 2036,
"prompt_ms": 1014.5,
"prompt_n": 2029,
"prompt_per_second": 2000,
"protocol": 2
}
@@ -3867,7 +3867,7 @@
"max_tokens": 300,
"messages": [
{
"content": "<8129 octets a3e9aa90e552>",
"content": "<8102 octets f67301f37ff4>",
"role": "user"
}
],
@@ -3952,8 +3952,8 @@
"predicted_ms": 6000,
"predicted_n": 300,
"predicted_per_second": 50,
"prompt_ms": 1018,
"prompt_n": 2036,
"prompt_ms": 1014.5,
"prompt_n": 2029,
"prompt_per_second": 2000,
"protocol": 2
}
@@ -3975,8 +3975,8 @@
"predicted_ms": 6000,
"predicted_n": 300,
"predicted_per_second": 50,
"prompt_ms": 1018,
"prompt_n": 2036,
"prompt_ms": 1014.5,
"prompt_n": 2029,
"prompt_per_second": 2000,
"protocol": 2
}
@@ -4034,8 +4034,8 @@
"predicted_ms": 6000,
"predicted_n": 300,
"predicted_per_second": 50,
"prompt_ms": 1018,
"prompt_n": 2036,
"prompt_ms": 1014.5,
"prompt_n": 2029,
"prompt_per_second": 2000,
"protocol": 2
}
@@ -4057,8 +4057,8 @@
"predicted_ms": 6000,
"predicted_n": 300,
"predicted_per_second": 50,
"prompt_ms": 1018,
"prompt_n": 2036,
"prompt_ms": 1014.5,
"prompt_n": 2029,
"prompt_per_second": 2000,
"protocol": 2
}
@@ -4089,8 +4089,8 @@
"predicted_ms": 6000,
"predicted_n": 300,
"predicted_per_second": 50,
"prompt_ms": 1018,
"prompt_n": 2036,
"prompt_ms": 1014.5,
"prompt_n": 2029,
"prompt_per_second": 2000,
"protocol": 2
}
@@ -4112,8 +4112,8 @@
"predicted_ms": 6000,
"predicted_n": 300,
"predicted_per_second": 50,
"prompt_ms": 1018,
"prompt_n": 2036,
"prompt_ms": 1014.5,
"prompt_n": 2029,
"prompt_per_second": 2000,
"protocol": 2
}
@@ -4135,8 +4135,8 @@
"predicted_ms": 6000,
"predicted_n": 300,
"predicted_per_second": 50,
"prompt_ms": 1018,
"prompt_n": 2036,
"prompt_ms": 1014.5,
"prompt_n": 2029,
"prompt_per_second": 2000,
"protocol": 2
}
rust/loki-rs/tests/parity_strata/strata.json
+386
View File
@@ -3006,6 +3006,126 @@
"f strata/data/models/swift-IQ3_XXS/a.gguf -rw-r--r-- xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
]
},
{
"name": "réglages, répartition inconnue",
"method": "POST",
"path": "/api/strata/settings",
"body": "{\"preset\":\"S\",\"ctx\":98304,\"kv\":\"int8\",\"spec\":4,\"vision\":false,\"helper\":true,\"split\":\"nimporte\"}",
"status": 400,
"out": {
"error": "répartition inconnue : nimporte",
"ok": false
},
"actions": [],
"after": {
"config/CTX": "98304",
"config/ENGINE": "strata",
"config/STRATA_HELPER": "1",
"config/STRATA_KV": "int8"
},
"tree": [
"d presets",
"d strata",
"d strata/data",
"d strata/data/models",
"d strata/data/models/swift-IQ3_XXS",
"f presets/S.env -rw-r--r-- # NAME=Strata S\nENGINE=strata\nSTRATA_HELPER_GPU=0\nSTRATA_EXPERTS=disk\nCTX=98304\nSTRATA_KV=int8\nSTRATA_SPEC=4\nSTRATA_VISION=0\nSTRATA_HELPER=1\n",
"f presets/llama.env -rw-r--r-- MODEL=/m.gguf\n",
"f strata/data/models/swift-IQ3_XXS/a.gguf -rw-r--r-- xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
]
},
{
"name": "réglages, partage des couches",
"method": "POST",
"path": "/api/strata/settings",
"body": "{\"preset\":\"S\",\"ctx\":98304,\"kv\":\"int8\",\"spec\":4,\"vision\":false,\"helper\":true,\"split\":\"layers\"}",
"status": 200,
"out": {
"ok": true,
"restarted": true
},
"actions": [
"restart"
],
"after": {
"config/CTX": "98304",
"config/ENGINE": "strata",
"config/STRATA_HELPER": "1",
"config/STRATA_KV": "int8",
"config/STRATA_SPLIT": "layers"
},
"tree": [
"d presets",
"d strata",
"d strata/data",
"d strata/data/models",
"d strata/data/models/swift-IQ3_XXS",
"f presets/S.env -rw-r--r-- # NAME=Strata S\nENGINE=strata\nSTRATA_HELPER_GPU=0\nSTRATA_EXPERTS=disk\nCTX=98304\nSTRATA_KV=int8\nSTRATA_SPEC=4\nSTRATA_VISION=0\nSTRATA_HELPER=1\nSTRATA_SPLIT=layers\n",
"f presets/llama.env -rw-r--r-- MODEL=/m.gguf\n",
"f strata/data/models/swift-IQ3_XXS/a.gguf -rw-r--r-- xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
]
},
{
"name": "réglages d'un ancien client, partage gardé",
"method": "POST",
"path": "/api/strata/settings",
"body": "{\"preset\":\"S\",\"ctx\":98304,\"kv\":\"int8\",\"spec\":4,\"vision\":false,\"helper\":true}",
"status": 200,
"out": {
"ok": true,
"restarted": true
},
"actions": [
"restart"
],
"after": {
"config/CTX": "98304",
"config/ENGINE": "strata",
"config/STRATA_HELPER": "1",
"config/STRATA_KV": "int8",
"config/STRATA_SPLIT": "layers"
},
"tree": [
"d presets",
"d strata",
"d strata/data",
"d strata/data/models",
"d strata/data/models/swift-IQ3_XXS",
"f presets/S.env -rw-r--r-- # NAME=Strata S\nENGINE=strata\nSTRATA_HELPER_GPU=0\nSTRATA_EXPERTS=disk\nSTRATA_SPLIT=layers\nCTX=98304\nSTRATA_KV=int8\nSTRATA_SPEC=4\nSTRATA_VISION=0\nSTRATA_HELPER=1\n",
"f presets/llama.env -rw-r--r-- MODEL=/m.gguf\n",
"f strata/data/models/swift-IQ3_XXS/a.gguf -rw-r--r-- xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
]
},
{
"name": "réglages, retour à l'aide aux experts",
"method": "POST",
"path": "/api/strata/settings",
"body": "{\"preset\":\"S\",\"ctx\":98304,\"kv\":\"int8\",\"spec\":4,\"vision\":false,\"helper\":true,\"split\":\"helper\"}",
"status": 200,
"out": {
"ok": true,
"restarted": true
},
"actions": [
"restart"
],
"after": {
"config/CTX": "98304",
"config/ENGINE": "strata",
"config/STRATA_HELPER": "1",
"config/STRATA_KV": "int8"
},
"tree": [
"d presets",
"d strata",
"d strata/data",
"d strata/data/models",
"d strata/data/models/swift-IQ3_XXS",
"f presets/S.env -rw-r--r-- # NAME=Strata S\nENGINE=strata\nSTRATA_HELPER_GPU=0\nSTRATA_EXPERTS=disk\nCTX=98304\nSTRATA_KV=int8\nSTRATA_SPEC=4\nSTRATA_VISION=0\nSTRATA_HELPER=1\n",
"f presets/llama.env -rw-r--r-- MODEL=/m.gguf\n",
"f strata/data/models/swift-IQ3_XXS/a.gguf -rw-r--r-- xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
]
},
{
"name": "installation en GET",
"method": "GET",
@@ -3529,6 +3649,7 @@
"prefill": "16384",
"short_read": "48",
"spec": "3",
"split_layers": false,
"version": "1.1",
"vision": true,
"vision_gpu": "RTX 3070 (8 Go)"
@@ -3731,6 +3852,7 @@
"prefill": "16384",
"short_read": "48",
"spec": "3",
"split_layers": false,
"version": "1.1",
"vision": true,
"vision_gpu": "RTX 3070 (8 Go)"
@@ -3775,6 +3897,7 @@
"prefill": "16384",
"short_read": "150",
"spec": "4",
"split_layers": false,
"version": "1.1",
"vision": false
},
@@ -3806,6 +3929,7 @@
"prefill": "16384",
"short_read": "150",
"spec": "4",
"split_layers": false,
"version": "1.1",
"vision": true,
"vision_gpu": "RTX 5060 Ti (16 Go)"
@@ -3975,6 +4099,7 @@
"prefill": "16384",
"short_read": "48",
"spec": "3",
"split_layers": false,
"version": "1.1",
"vision": true,
"vision_gpu": "RTX 3070 (8 Go)"
@@ -4019,6 +4144,7 @@
"prefill": "16384",
"short_read": "150",
"spec": "4",
"split_layers": false,
"version": "1.1",
"vision": false
},
@@ -4050,6 +4176,7 @@
"prefill": "16384",
"short_read": "150",
"spec": "4",
"split_layers": false,
"version": "1.1",
"vision": true,
"vision_gpu": "RTX 5060 Ti (16 Go)"
@@ -4215,6 +4342,7 @@
"prefill": "16384",
"short_read": "48",
"spec": "3",
"split_layers": false,
"version": "1.1",
"vision": true,
"vision_gpu": "RTX 3070 (8 Go)"
@@ -4259,6 +4387,7 @@
"prefill": "16384",
"short_read": "150",
"spec": "4",
"split_layers": false,
"version": "1.1",
"vision": false
},
@@ -4290,6 +4419,7 @@
"prefill": "16384",
"short_read": "150",
"spec": "4",
"split_layers": false,
"version": "1.1",
"vision": true,
"vision_gpu": "RTX 5060 Ti (16 Go)"
@@ -4455,6 +4585,7 @@
"prefill": "16384",
"short_read": "48",
"spec": "3",
"split_layers": false,
"version": "1.1",
"vision": true,
"vision_gpu": "RTX 3070 (8 Go)"
@@ -4499,6 +4630,7 @@
"prefill": "16384",
"short_read": "150",
"spec": "4",
"split_layers": false,
"version": "1.1",
"vision": false
},
@@ -4530,6 +4662,7 @@
"prefill": "16384",
"short_read": "150",
"spec": "4",
"split_layers": false,
"version": "1.1",
"vision": true,
"vision_gpu": "RTX 5060 Ti (16 Go)"
@@ -4696,6 +4829,7 @@
"prefill": "16384",
"short_read": "48",
"spec": "3",
"split_layers": false,
"version": "1.1",
"vision": true,
"vision_gpu": "RTX 3070 (8 Go)"
@@ -4740,6 +4874,7 @@
"prefill": "16384",
"short_read": "150",
"spec": "4",
"split_layers": false,
"version": "1.1",
"vision": false
},
@@ -4771,6 +4906,7 @@
"prefill": "16384",
"short_read": "150",
"spec": "4",
"split_layers": false,
"version": "1.1",
"vision": true,
"vision_gpu": "RTX 5060 Ti (16 Go)"
@@ -4870,6 +5006,250 @@
"f strata/prebuilt/strata-linux-x64-cuda12.zip"
]
},
{
"name": "seconde carte en partage des couches",
"files": {
"presets/STRATA FLASH NEXT SWIFT 1.5 IQ3_XXS.env": "# NAME=Qwen3.8 Flash Next Swift 1.5 IQ3_XXS (Strata)\nENGINE=strata\nSTRATA_CONFIG={{HOME}}/strata/ajean-moe-1.1/strata-swift-iq3_xxs.json\nSTRATA_MAIN_GPU=1\nSTRATA_HELPER_GPU=0\nSTRATA_VISION=1\nSTRATA_EXPERTS=auto\nSTRATA_DROP=1\nSTRATA_KV=fp16\nSTRATA_KV_RESIDENT=32768\nSTRATA_SPEC=3\nCTX=196608\nREASONING=off\nSTRATA_SPLIT=layers\n"
},
"method": "GET",
"path": "/api/strata",
"status": 200,
"out": {
"active": true,
"env": {
"avail_gb": 45.500000256,
"disk_free_gb": 400,
"gpus": [
{
"arch": 86,
"index": 0,
"name": "NVIDIA GeForce RTX 3070",
"vram_gb": 8
},
{
"arch": 120,
"index": 1,
"name": "NVIDIA GeForce RTX 5060 Ti",
"vram_gb": 15.9287109375
}
],
"helper": 0,
"main": 1,
"python": "3.12",
"ram_gb": 47,
"supported": true
},
"families": [
{
"about": "réfléchit moins, répond plus vite",
"fits": [
{
"disk_gb": 68,
"drop": false,
"mmap": false,
"ok": true,
"quant": {
"about": "un peu plus fidèle, presque aussi rapide",
"arena_gb": 35.5,
"download_gb": 68,
"id": "IQ2_XS"
}
},
{
"details": {
"cache_root": "256",
"ctx": "196608",
"drop": false,
"experts": "auto",
"has_helper": true,
"helper_gpu": "RTX 3070 (8 Go)",
"helper_on": true,
"kv": "fp16",
"kv_resident": "32768",
"main_gpu": "RTX 5060 Ti (16 Go)",
"mmap": true,
"mode": "arena",
"mode_label": "experts.bin mappé (cache système)",
"prefill": "16384",
"short_read": "150",
"spec": "3",
"split_layers": true,
"version": "1.1",
"vision": true,
"vision_gpu": "RTX 5060 Ti (16 Go)"
},
"disk_gb": 75.799997999,
"drop": false,
"mmap": true,
"ok": true,
"preset": "STRATA FLASH NEXT SWIFT 1.5 IQ3_XXS",
"quant": {
"about": "99 % de la qualité du modèle complet",
"arena_gb": 42.9,
"download_gb": 75.8,
"id": "IQ3_XXS"
}
}
],
"id": "swift",
"label": "Swift 1.5",
"quants": [
"IQ2_XS",
"IQ3_XXS"
],
"recommend": "IQ3_XXS"
},
{
"about": "le Qwen3.8-Flash-Next original",
"fits": [
{
"details": {
"cache_root": "256",
"ctx": "131072",
"drop": false,
"experts": "ram",
"has_helper": true,
"helper_on": false,
"kv": "int8",
"main_gpu": "RTX 3070 (8 Go)",
"mmap": false,
"mode": "ram",
"mode_label": "chargés et verrouillés en RAM",
"prefill": "16384",
"short_read": "150",
"spec": "4",
"split_layers": false,
"version": "1.1",
"vision": false
},
"disk_gb": 66.4,
"drop": false,
"mmap": false,
"ok": true,
"preset": "STRATA FLASH NEXT CLASSIQUE Q2_0",
"quant": {
"about": "le plus rapide",
"arena_gb": 34,
"download_gb": 66.4,
"id": "Q2_0"
}
},
{
"details": {
"cache_root": "256",
"ctx": "131072",
"drop": false,
"experts": "disk",
"has_helper": false,
"helper_on": false,
"kv": "int8",
"main_gpu": "RTX 5060 Ti (16 Go)",
"mmap": true,
"mode": "arena",
"mode_label": "experts.bin mappé (cache système)",
"prefill": "16384",
"short_read": "150",
"spec": "4",
"split_layers": false,
"version": "1.1",
"vision": true,
"vision_gpu": "RTX 5060 Ti (16 Go)"
},
"disk_gb": 68,
"drop": false,
"mmap": false,
"ok": true,
"preset": "STRATA FLASH NEXT CLASSIQUE IQ2_XS",
"quant": {
"about": "un peu plus fidèle, presque aussi rapide",
"arena_gb": 35.5,
"download_gb": 68,
"id": "IQ2_XS"
}
},
{
"disk_gb": 75.8,
"drop": false,
"mmap": true,
"ok": true,
"quant": {
"about": "99 % de la qualité du modèle complet",
"arena_gb": 42.9,
"download_gb": 75.8,
"id": "IQ3_XXS"
}
},
{
"disk_gb": 83.6,
"drop": false,
"mmap": true,
"ok": true,
"quant": {
"about": "la meilleure qualité, le plus lent",
"arena_gb": 50.3,
"download_gb": 83.6,
"id": "IQ3_S"
}
}
],
"id": "qwen",
"label": "Classique",
"quants": [
"Q2_0",
"IQ2_XS",
"IQ3_XXS",
"IQ3_S"
],
"recommend": "IQ3_XXS"
}
],
"installed": [
"STRATA FLASH NEXT CLASSIQUE IQ2_XS",
"STRATA FLASH NEXT SWIFT 1.5 IQ3_XXS",
"STRATA FLASH NEXT CLASSIQUE Q2_0",
"S"
],
"version": "1.1"
},
"actions": [],
"after": {
"config/CTX": "131072",
"config/ENGINE": "strata",
"config/STRATA_CONFIG": "{{HOME}}/strata/ajean-moe-1.1/strata-swift-iq3_xxs.json",
"config/STRATA_KV": "fp16"
},
"tree": [
"d presets",
"d strata",
"d strata/ajean-moe-1.1",
"d strata/ajean-moe-1.1/.venv",
"d strata/ajean-moe-1.1/.venv/bin",
"d strata/ajean-moe-1.1/engine-cuda12",
"d strata/ajean-moe-1.1/serve",
"d strata/data",
"d strata/data/models",
"d strata/data/models/swift-IQ3_XXS",
"d strata/data/packs",
"d strata/data/packs/swift-iq3_xxs",
"d strata/dl",
"d strata/prebuilt",
"f presets/S.env -rw-r--r-- # NAME=Strata S\nENGINE=strata\nSTRATA_HELPER_GPU=0\nSTRATA_EXPERTS=disk\nCTX=98304\nSTRATA_KV=int8\nSTRATA_SPEC=4\nSTRATA_VISION=0\nSTRATA_HELPER=1\n",
"f presets/STRATA FLASH NEXT CLASSIQUE IQ2_XS.env -rw-r--r-- # NAME=Classique IQ2_XS\nENGINE=strata\nSTRATA_CONFIG={{HOME}}/strata/ajean-moe-1.1/strata-swift-iq3_xxs.json\nSTRATA_MAIN_GPU=1\nSTRATA_HELPER_GPU=-1\nSTRATA_EXPERTS=disk\n",
"f presets/STRATA FLASH NEXT CLASSIQUE Q2_0.env -rw-r--r-- ENGINE=moe\nSTRATA_CONFIG={{HOME}}/strata/ajean-moe-1.1/strata-swift-iq3_xxs.json\nSTRATA_MAIN_GPU=0\nSTRATA_HELPER_GPU=1\nSTRATA_HELPER=0\nSTRATA_VISION=0\nSTRATA_EXPERTS=ram\n",
"f presets/STRATA FLASH NEXT SWIFT 1.5 IQ3_XXS.env -rw-r--r-- # NAME=Qwen3.8 Flash Next Swift 1.5 IQ3_XXS (Strata)\nENGINE=strata\nSTRATA_CONFIG={{HOME}}/strata/ajean-moe-1.1/strata-swift-iq3_xxs.json\nSTRATA_MAIN_GPU=1\nSTRATA_HELPER_GPU=0\nSTRATA_VISION=1\nSTRATA_EXPERTS=auto\nSTRATA_DROP=1\nSTRATA_KV=fp16\nSTRATA_KV_RESIDENT=32768\nSTRATA_SPEC=3\nCTX=196608\nREASONING=off\nSTRATA_SPLIT=layers\n",
"f presets/llama.env -rw-r--r-- MODEL=/m.gguf\n",
"f strata/ajean-moe-1.1/engine-cuda12/.loki-engine -rw-r--r-- ajean-moe-engine-1.1-linux-x64-cuda12.zip\n",
"f strata/ajean-moe-1.1/serve/server.py -rw-r--r-- # serveur\n",
"f strata/ajean-moe-1.1/setup.py -rw-r--r-- # installeur\n",
"f strata/ajean-moe-1.1/setup.sh -rwxr-xr-x #!/bin/bash\necho \"installeur : $*\"\necho \"cache : $XDG_CACHE_HOME\"\nprintf 'modèle 10%%\\r'\nprintf 'modèle 20%%\\r'\necho \"modèle téléchargé\"\nmkdir -p engine-cuda12 .venv/bin\nfam=\"\"; model=\"\"; data=\"\"\nwhile [ $# -gt 0 ]; do\n case \"$1\" in\n --family) fam=\"$2\"; shift;;\n --model) model=\"$2\"; shift;;\n --data-dir) data=\"$2\"; shift;;\n esac\n shift\ndone\ncase \"$model\" in\nQ2_0) echo \"erreur : téléchargement interrompu\" >&2; exit 3;;\nIQ2_XS) echo \"rien d'écrit\"; exit 0;;\nesac\nlow=$(echo \"$model\" | tr 'A-Z' 'a-z')\nif [ \"$fam\" = \"swift\" ]; then tag=\"swift-$low\"; else tag=\"$low\"; fi\nmkdir -p \"$data/packs/$tag\"\nprintf '{\"exe\":\"engine-cuda12/strata\",\"args\":[\"--pack\",\"%s\",\"--kv\",\"int8\",\"--spec\",\"4\",\"--max-context\",\"131072\",\"--vision\",\"--vram-reserve-mib\",\"700\"],\"vision\":{\"exe\":\"v\",\"gpu\":true},\"gpu\":0,\"port\":8080}\\n' \"$data/packs/$tag\" > \"strata-$tag.json\"\necho \"prêt : strata-$tag.json\"\n",
"f strata/ajean-moe-1.1/strata-swift-iq3_xxs.json -rw-r--r-- {\"exe\":\"engine-cuda12/strata\",\"args\":[\"--pack\",\"{{HOME}}/strata/data/packs/swift-iq3_xxs\",\"--kv\",\"int8\",\"--spec\",\"4\",\"--max-context\",\"131072\",\"--vision\",\"--vram-reserve-mib\",\"700\"],\"vision\":{\"exe\":\"v\",\"gpu\":true},\"gpu\":0,\"port\":8080}\n",
"f strata/data/models/swift-IQ3_XXS/a.gguf -rw-r--r-- xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
"f strata/data/packs/swift-iq3_xxs/experts.bin -rw-r--r-- x",
"f strata/dl/ajean-moe-engine-1.1-linux-x64-cuda12.zip",
"f strata/dl/ajean-moe-src-1.1.tar.gz",
"f strata/prebuilt/strata-linux-x64-cuda12.zip"
]
},
{
"name": "mode hors RAM coupé dans le preset",
"files": {
@@ -4937,6 +5317,7 @@
"prefill": "16384",
"short_read": "48",
"spec": "3",
"split_layers": false,
"version": "1.1",
"vision": true,
"vision_gpu": "RTX 3070 (8 Go)"
@@ -4981,6 +5362,7 @@
"prefill": "16384",
"short_read": "150",
"spec": "4",
"split_layers": false,
"version": "1.1",
"vision": false
},
@@ -5012,6 +5394,7 @@
"prefill": "16384",
"short_read": "150",
"spec": "4",
"split_layers": false,
"version": "1.1",
"vision": true,
"vision_gpu": "RTX 5060 Ti (16 Go)"
@@ -5179,6 +5562,7 @@
"prefill": "16384",
"short_read": "48",
"spec": "3",
"split_layers": false,
"version": "1.1",
"vision": true,
"vision_gpu": "RTX 3070 (8 Go)"
@@ -5223,6 +5607,7 @@
"prefill": "16384",
"short_read": "150",
"spec": "4",
"split_layers": false,
"version": "1.1",
"vision": false
},
@@ -5254,6 +5639,7 @@
"prefill": "16384",
"short_read": "150",
"spec": "4",
"split_layers": false,
"version": "1.1",
"vision": true,
"vision_gpu": "RTX 5060 Ti (16 Go)"
rust/loki-rs/tests/strata.rs
+1
View File
@@ -318,6 +318,7 @@ async fn parite_de_strata() {
"CTX",
"STRATA_KV",
"STRATA_HELPER",
"STRATA_SPLIT",
] {
if let Some(v) = bn.go.get("config", k).filter(|v| !v.is_empty()) {
after.insert(format!("config/{k}"), Value::String(bn.mask(&v)));