mirror of
https://github.com/R0m1k3/Loki.git
synced 2026-10-17 04:10:04 +02:00
Trois défauts sans rapport entre eux, tous visibles à l'écran. Jauge de contexte bloquée à « 0 / N (0 %) » : le comptage exact vient de usage.prompt_tokens (include_usage), qu'un llama-server récent a cessé de renvoyer. On note désormais si l'usage est arrivé ; sinon la fin de tour publie l'estimation qui pilote déjà la compaction — approximative, mais jamais absente. Et la souscription envoie la valeur courante juste après le rattrapage : une page rechargée affichait zéro sur une discussion pourtant pleine, faute d'événement ctx_used dans la fenêtre rejouée. Espace disque : « il ne reste que 11,9 Go » sur un partage Unraid qui en a des centaines. /mnt/user est servi par shfs, un FUSE qui agrège plusieurs disques — statfs y renvoie l'espace d'un seul, et ce chiffre REFUSAIT l'installation du modèle. Les systèmes de fichiers qui approximent (FUSE, overlay, NFS, CIFS, Ceph, Gluster) sont reconnus par leur magie : leur chiffre reste affiché, précédé d'un « ~ », mais ne bloque plus rien. Panneau Fichiers : le panneau entier défilait, emportant vers le haut l'en-tête, le fil d'Ariane et le pied (occupation disque, bouton .zip) dès qu'il y avait quelques fichiers. Seule la liste défile maintenant ; le pied reste en bas et l'état vide se centre. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
182 lines
4.5 KiB
Markdown
182 lines
4.5 KiB
Markdown
# micromark-util-subtokenize
|
||
|
||
[![Build][build-badge]][build]
|
||
[![Coverage][coverage-badge]][coverage]
|
||
[![Downloads][downloads-badge]][downloads]
|
||
[![Size][bundle-size-badge]][bundle-size]
|
||
[![Sponsors][sponsors-badge]][opencollective]
|
||
[![Backers][backers-badge]][opencollective]
|
||
[![Chat][chat-badge]][chat]
|
||
|
||
[micromark][] utility to tokenize subtokens.
|
||
|
||
## Contents
|
||
|
||
* [What is this?](#what-is-this)
|
||
* [When should I use this?](#when-should-i-use-this)
|
||
* [Install](#install)
|
||
* [Use](#use)
|
||
* [API](#api)
|
||
* [`subtokenize(events)`](#subtokenizeevents)
|
||
* [Types](#types)
|
||
* [Compatibility](#compatibility)
|
||
* [Security](#security)
|
||
* [Contribute](#contribute)
|
||
* [License](#license)
|
||
|
||
## What is this?
|
||
|
||
This package exposes a micromark internal that you probably don’t need.
|
||
|
||
## When should I use this?
|
||
|
||
This package might be useful when you are making your own micromark extensions.
|
||
|
||
## Install
|
||
|
||
This package is [ESM only][esm].
|
||
In Node.js (version 16+), install with [npm][]:
|
||
|
||
```sh
|
||
npm install micromark-util-subtokenize
|
||
```
|
||
|
||
In Deno with [`esm.sh`][esmsh]:
|
||
|
||
```js
|
||
import {subtokenize} from 'https://esm.sh/micromark-util-subtokenize@1'
|
||
```
|
||
|
||
In browsers with [`esm.sh`][esmsh]:
|
||
|
||
```html
|
||
<script type="module">
|
||
import {subtokenize} from 'https://esm.sh/micromark-util-subtokenize@1?bundle'
|
||
</script>
|
||
```
|
||
|
||
## Use
|
||
|
||
```js
|
||
import {subtokenize} from 'micromark-util-subtokenize'
|
||
|
||
/**
|
||
* Content is transparent: it’s parsed right now. That way, definitions are also
|
||
* parsed right now: before text in paragraphs (specifically, media) are parsed.
|
||
*
|
||
* @type {Resolver}
|
||
*/
|
||
function resolveContent(events) {
|
||
subtokenize(events)
|
||
return events
|
||
}
|
||
```
|
||
|
||
## API
|
||
|
||
This module exports the identifiers [`subtokenize`][api-subtokenize].
|
||
There is no default export.
|
||
|
||
### `subtokenize(events)`
|
||
|
||
Tokenize subcontent.
|
||
|
||
###### Parameters
|
||
|
||
* `events` (`Array<Event>`)
|
||
— list of events
|
||
|
||
###### Returns
|
||
|
||
Whether subtokens were found (`boolean`).
|
||
|
||
## Types
|
||
|
||
This package is fully typed with [TypeScript][].
|
||
It exports no additional types.
|
||
|
||
## Compatibility
|
||
|
||
Projects maintained by the unified collective are compatible with maintained
|
||
versions of Node.js.
|
||
|
||
When we cut a new major release, we drop support for unmaintained versions of
|
||
Node.
|
||
This means we try to keep the current release line,
|
||
`micromark-util-subtokenize@2`, compatible with Node.js 16.
|
||
This package works with `micromark@3`.
|
||
|
||
## Security
|
||
|
||
This package is safe.
|
||
See [`security.md`][securitymd] in [`micromark/.github`][health] for how to
|
||
submit a security report.
|
||
|
||
## Contribute
|
||
|
||
See [`contributing.md`][contributing] in [`micromark/.github`][health] for ways
|
||
to get started.
|
||
See [`support.md`][support] for ways to get help.
|
||
|
||
This project has a [code of conduct][coc].
|
||
By interacting with this repository, organisation, or community you agree to
|
||
abide by its terms.
|
||
|
||
## License
|
||
|
||
[MIT][license] © [Titus Wormer][author]
|
||
|
||
<!-- Definitions -->
|
||
|
||
[api-subtokenize]: #subtokenizeevents
|
||
|
||
[author]: https://wooorm.com
|
||
|
||
[backers-badge]: https://opencollective.com/unified/backers/badge.svg
|
||
|
||
[build]: https://github.com/micromark/micromark/actions
|
||
|
||
[build-badge]: https://github.com/micromark/micromark/workflows/main/badge.svg
|
||
|
||
[bundle-size]: https://bundlejs.com/?q=micromark-util-subtokenize
|
||
|
||
[bundle-size-badge]: https://img.shields.io/badge/dynamic/json?label=minzipped%20size&query=$.size.compressedSize&url=https://deno.bundlejs.com/?q=micromark-util-subtokenize
|
||
|
||
[chat]: https://github.com/micromark/micromark/discussions
|
||
|
||
[chat-badge]: https://img.shields.io/badge/chat-discussions-success.svg
|
||
|
||
[coc]: https://github.com/micromark/.github/blob/main/code-of-conduct.md
|
||
|
||
[contributing]: https://github.com/micromark/.github/blob/main/contributing.md
|
||
|
||
[coverage]: https://codecov.io/github/micromark/micromark
|
||
|
||
[coverage-badge]: https://img.shields.io/codecov/c/github/micromark/micromark.svg
|
||
|
||
[downloads]: https://www.npmjs.com/package/micromark-util-subtokenize
|
||
|
||
[downloads-badge]: https://img.shields.io/npm/dm/micromark-util-subtokenize.svg
|
||
|
||
[esm]: https://gist.github.com/sindresorhus/a39789f98801d908bbc7ff3ecc99d99c
|
||
|
||
[esmsh]: https://esm.sh
|
||
|
||
[health]: https://github.com/micromark/.github
|
||
|
||
[license]: https://github.com/micromark/micromark/blob/main/license
|
||
|
||
[micromark]: https://github.com/micromark/micromark
|
||
|
||
[npm]: https://docs.npmjs.com/cli/install
|
||
|
||
[opencollective]: https://opencollective.com/unified
|
||
|
||
[securitymd]: https://github.com/micromark/.github/blob/main/security.md
|
||
|
||
[sponsors-badge]: https://opencollective.com/unified/sponsors/badge.svg
|
||
|
||
[support]: https://github.com/micromark/.github/blob/main/support.md
|
||
|
||
[typescript]: https://www.typescriptlang.org
|