keywords:tokenization

skyflow-node

Skyflow SDK for Node.js

skyflowtech

published version 1.14.1, a month ago0 dependents licensed under $MIT

9,792

@cybersource/flex-sdk-web

Easily create payment tokens using Flex API

jadonald

published version 0.3.1, 5 years ago4 dependents licensed under $SEE LICENSE IN LICENSE.md

3,967

gpt-token-utils

Isomorphic utilities for GPT-3 tokenization and prompt building.

teffenellis

published version 1.2.0, 2 years ago0 dependents licensed under $MIT

2,469

creditcards-tokenizer

Wix Restaurants credit-cards tokenizer

wix-ci-publisher

published version 1.2.902, 5 years ago1 dependents licensed under $Apache-2.0

2,224

az

A NLP library for Russian language

denull

published version 0.2.3, 9 years ago3 dependents licensed under $MIT

1,758

@diplodoc/sentenizer

text segmentation into sentences

3y3

published version 0.0.8, 10 months ago2 dependents licensed under $MIT

1,522

@basis-theory/apple-pay-js

Basis Theory utility for decrypting Apple Pay Tokens

bt-platform

published version 2.0.2, a year ago0 dependents licensed under $Apache-2.0

903

html-tokenizer

Small, fast, event-driven, fault-tolerant html tokenizer. Works in node or browsers.

greim

published version 4.0.1, 4 years ago5 dependents licensed under $MIT

1,066

@cybersource/flex-sdk-node

Easily create keys for use in a Flex API integration and verifying Flex token contents

jadonald

published version 0.1.0, 7 years ago0 dependents licensed under $SEE LICENSE IN LICENSE.md

542

lex-es6

es6 version of aaditmshah/lexer

disjukr

published version 1.1.0, 9 years ago1 dependents

564

Tokenizing strings of text. Extracting arrays of words and optionally number, emojis, tags, usernames and email addresses from strings. For Node.js and the browser. When you need more than just [a-z] regular expressions.

eklem

published version 9.1.2, a year ago2 dependents licensed under $MIT

485

niubiz-pay

Library that allows you to make payments and tokenize debit and credit cards with Niubiz.

zzmaxzz

published version 0.0.9, a month ago0 dependents licensed under $MIT

337

@bedrock/tokenizer

A Bedrock module to provide auto-rotating tokenizers

mattcollier

published version 10.0.0, a year ago0 dependents

213

@bedrock/tokenization

Bedrock Tokenization Engine

dlongley

published version 22.1.0, 2 months ago0 dependents

151

@bbreukelen/textsplitter

#### Description I needed the SentenceSplitter from llamaindex but had to import the entire llamaindex package which is 1GB. I pulled it out and had GPT make a standalone version. It's not exactly the same but close.