Search results
65 packages found
Sort by: Default
- Default
- Most downloaded this week
- Most downloaded this month
- Most dependents
- Recently published
English word and sentence tokenizer, for natural language processing.
Textmate token-based language service for Visual Studio Code.
- vscode
- vscode-extension
- textmate
- grammar
- language-features
- language-service
- language
- lsp
- parse
- syntax
- tokenization
- tokenizer
OpenVINO™ Tokenizers adds text processing operations to openvino-node package
- OpenVINO
- openvino
- tokenizers
- OpenVINO tokenizers
- openvino tokenizers
- openvino-tokenizers
- openvino-tokenizers-node
- tokenization
- text processing
- string tensors
- text tensors
## How To Use
Use LLMs to run map-reduce summarization tasks on large documents until a target token size is met.
- text-distillation
- text-summarization
- text-compression
- token-management
- llm-distillery
- large-language-model
- LLM
- semantic-chunking
- text-processing
- AI-text-reduction
- openai-api
- tokenization
Tokenization Service's Smart Contracts
Salient is a natural language processing and sentiment analysis library
Here is a README generated from the code snippet:
- vectorizer
- machine-learning
- code-vectorization
- embedding
- ai
- artificial-intelligence
- nlp
- natural-language-processing
- code-embedding
- programming
- code-processing
- software-engineering
- tokenizer
- code-tokenizer
- View more
Bedrock Tokenization Engine
Tools for manipulating sentences
Basic English tokenizer
A tokenizer and lemmatizer for canonical terms in text
Taiwanese Hokkien Transliterator and Tokeniser
Official AstroPay client for easy frontend integration to Tokenizer
a wrapper around the LunaSec CLI enabling it to be used as an NPM package
SmartCal is a lightweight and powerful library for evaluating complex mathematical expressions dynamically. It supports tokenization, parsing, and interpreting mathematical formulas seamlessly, offering flexibility to integrate custom data objects for dyn
- smartcal
- math
- calculation
- expression-parser
- expression-evaluator
- math-parser
- dynamic-calculations
- tokenization
- formula-interpreter
- typescript
- javascript
- math-engine
- nutriwell
A powerful and flexible text search library for JavaScript that enables you to build a simple text search engine.
- search
- text-search
- query
- query-parser
- parser
- tokenizer
- interpreter
- AST
- abstract-syntax-tree
- boolean-search
- evaluation
- normalization
- factory
- search-engine
- View more
Tools for working with human language data.
- natural
- language
- processing
- nlp
- machine
- learning
- speech
- token
- sentence
- tag
- extraction
- chunking
- parsing
- coreference
- View more
Basis Theory utility for decrypting Google Pay Tokens
Well typed, string transmutation via splicing, driven by definitions of regular expressions, priority match and recursive clauses