toksuite/facebook-xglm-564M
Text Generation • 2B • Updated • 282
Edit Models filters
Model Tree
Apps
Inference Providers
One-click Deployment
Models
645
Active filters: tokenization
toksuite/mistralai-tekken
Text Generation • 2B • Updated • 293
toksuite/google-byt5-small
Text Generation • 1B • Updated • 367
toksuite/tokenmonster-englishcode-32000-consistent-v1
Text Generation • 1B • Updated • 380
toksuite/google-bert-bert-base-multilingual-cased
Text Generation • 2B • Updated • 367
toksuite/Qwen-Qwen3-8B
Text Generation • 2B • Updated • 329
toksuite/tiktoken-gpt-4o
Text Generation • 2B • Updated • 283
ggunio/intelligent-tokenizer-v6
Updated • 34
toksuite/CohereLabs-aya-expanse-8b
Text Generation • 2B • Updated • 295
DiscreteSpeech/DSTK
Updated • 12 • 8
manoskary/miditok-REMI
Updated
toksuite/meta-llama-Llama-3.2-7B
Text Generation • 8B • Updated • 31
kaanilker/Mini-Turkish-Tokenizer
Token Classification • Updated • 1
toksuite/meta-llama-Llama-3.2-300M
Text Generation • 0.6B • Updated • 28
wikilangs/ary
Text Generation • Updated
wikilangs/ab
Text Generation • Updated
wikilangs/ace
Text Generation • Updated
wikilangs/ady
Text Generation • Updated
wikilangs/af
Text Generation • Updated
wikilangs/alt
Text Generation • Updated
wikilangs/am
Text Generation • Updated
wikilangs/ami
Text Generation • Updated
wikilangs/an
Text Generation • Updated
wikilangs/ang
Text Generation • Updated
wikilangs/ann
Text Generation • Updated
wikilangs/anp
Text Generation • Updated
wikilangs/ar
Text Generation • Updated
wikilangs/arc
Text Generation • Updated
wikilangs/arz
Text Generation • Updated
wikilangs/as
Text Generation • Updated