Conductor Tokenizer Library
A common library of tokenizers for Conductor
Setup
Run generate script to download tokenizers and generate tokenizers/index.
bun run generate@foundation0/tokenizers
0.3.1 • Public • PublishedReadme
Keywords
nonePackage Sidebar
Install
npm i @foundation0/tokenizers
Weekly Downloads
0
Version
0.3.1
License
SSPL-1.0
Unpacked Size
24.7 MB
Total Files
37