Lib0

Monorepo of isomorphic utility functions

This library is meant to replace all global JavaScript functions with isomorphic module imports. Additionally, it implements several performance-oriented utility modules. Most noteworthy are the binary encoding/decoding modules [lib0/encoding] / [lib0/decoding], the randomized testing framework [lib0/testing], the fast Pseudo Random Number Generator [lib0/PRNG], the small socket.io alternative [lib0/websocket], and the logging module [lib0/logging] that allows colorized logging in all environments. Lib0 has only one dependency, which is also from the author of lib0. If lib0 is transpiled with rollup or webpack, very little code is produced because of the way that it is written. All exports are pure and are removed by transpilers that support dead code elimination. Here is an example of how dead code elemination and mangling optimizes code from lib0:

// How the code is optimized by transpilers:

// lib0/json.js
export const stringify = JSON.stringify
export const parse = JSON.parse

// index.js
import * as json from 'lib0/json.js'
export const f = (arg1, arg2) => json.stringify(arg1) + json.stringify(arg2)

// compiled with rollup and uglifyjs:
const s=JSON.stringify,f=(a,b)=>s(a)+s(b)
export {f}

Performance resources

Each function in this library is tested thoroughly and is not deoptimized by v8 (except some logging and comparison functions that can't be implemented without deoptimizations). This library implements its own test suite that is designed for randomized testing and inspecting performance issues.

node --trace-deop and node --trace-opt
https://youtu.be/IFWulQnM5E0 Good intro video
https://github.com/thlorenz/v8-perf
https://github.com/thlorenz/deoptigate - A great tool for investigating deoptimizations
https://github.com/vhf/v8-bailout-reasons - Description of some deopt messages

Code style

The code style might be a bit different from what you are used to. Stay open. Most of the design choices have been thought through. The purpose of this code style is to create code that is optimized by the compiler and that results in small code bundles when used with common module bundlers. Keep that in mind when reading the library.

Modules should only export pure functions and constants. This way the module bundler can eliminate dead code. The statement const x = someCondition ? A : B cannot be eleminated, because it is tied to a condition.
Use Classes for structuring data. Classes are well supported by jsdoc and are immediately optimized by the compiler. I.e. prefer class Coord { constructor (x, y) { this.x = x; this.y = y} } instead of { x: x, y: y }, because the compiler needs to be assured that the order of properties does not change. { y: y, x: x } has a different hidden class than { x: x, y: y }, which will lead to code deoptimizations if their use is alternated.
The user of your module should never create data objects with the new keyword. Prefer exporting factory functions like const createCoordinate = (x, y) => new Coord(x, y).
The use of class methods is discouraged, because method names can't be mangled or removed by dead code elimination.
The only acceptable use of methods is when two objects implement functionality differently. E.g. class Duck { eat () { swallow() } } and class Cow { eat () { chew() } } have the same signature, but implement it differently.
Prefer const variable declarations. Use let only in loops. const always leads to easier code.
Keep the potential execution stack small and compartmentalized. Nobody wants to debug spaghetti code.
Give proper names to your functions and ask yourself if you would know what the function does if you saw it in the execution stack.
Avoid recursion. There is a stack limit in most browsers and not every recursive function is optimized by the compiler.
Semicolons are superfluous. Lint with https://standardjs.com/

Modules

[lib0/array] Utility module to work with Arrays.

import * as array from 'lib0/array.js'

array.last(arr: Array<L>): L

Return the last element of an array. The element must exist

array.create(): Array<C>
array.copy(a: Array<D>): Array<D>
array.appendTo(dest: Array<M>, src: Array<M>)

Append elements from src to dest

array.from(arraylike: ArrayLike<T>|Iterable<T>): T

Transforms something array-like to an actual Array.

array.every(arr: Array<ITEM>, f: function(ITEM, number, Array<ITEM>):boolean): boolean

True iff condition holds on every element in the Array.

array.some(arr: Array<S>, f: function(S, number, Array<S>):boolean): boolean

True iff condition holds on some element in the Array.

array.equalFlat(a: Array<ELEM>, b: Array<ELEM>): boolean

[lib0/binary] Binary data constants.

import * as binary from 'lib0/binary.js'

binary.BIT1: number

n-th bit activated.

binary.BIT2
binary.BIT3
binary.BIT4
binary.BIT5
binary.BIT6
binary.BIT7
binary.BIT8
binary.BIT9
binary.BIT10
binary.BIT11
binary.BIT12
binary.BIT13
binary.BIT14
binary.BIT15
binary.BIT16
binary.BIT17
binary.BIT18
binary.BIT19
binary.BIT20
binary.BIT21
binary.BIT22
binary.BIT23
binary.BIT24
binary.BIT25
binary.BIT26
binary.BIT27
binary.BIT28
binary.BIT29
binary.BIT30
binary.BIT31
binary.BIT32
binary.BITS0: number

First n bits activated.

binary.BITS1
binary.BITS2
binary.BITS3
binary.BITS4
binary.BITS5
binary.BITS6
binary.BITS7
binary.BITS8
binary.BITS9
binary.BITS10
binary.BITS11
binary.BITS12
binary.BITS13
binary.BITS14
binary.BITS15
binary.BITS16
binary.BITS17
binary.BITS18
binary.BITS19
binary.BITS20
binary.BITS21
binary.BITS22
binary.BITS23
binary.BITS24
binary.BITS25
binary.BITS26
binary.BITS27
binary.BITS28
binary.BITS29
binary.BITS30
binary.BITS31: number
binary.BITS32: number

[lib0/broadcastchannel] Helpers for cross-tab communication using broadcastchannel with LocalStorage fallback.

import * as broadcastchannel from 'lib0/broadcastchannel.js'

// In browser window A:
broadcastchannel.subscribe('my events', data => console.log(data))
broadcastchannel.publish('my events', 'Hello world!') // => A: 'Hello world!' fires synchronously in same tab

// In browser window B:
broadcastchannel.publish('my events', 'hello from tab B') // => A: 'hello from tab B'

broadcastchannel.subscribe(room: string, f: function(any):any)

Subscribe to global publish events.

broadcastchannel.unsubscribe(room: string, f: function(any):any)

Unsubscribe from publish global events.

broadcastchannel.publish(room: string, data: any)

Publish data to all subscribers (including subscribers on this tab)

[lib0/buffer] Utility functions to work with buffers (Uint8Array).

import * as buffer from 'lib0/buffer.js'

buffer.createUint8ArrayFromLen(len: number)
buffer.createUint8ArrayViewFromArrayBuffer(buffer: ArrayBuffer, byteOffset: number, length: number)

Create Uint8Array with initial content from buffer

buffer.createUint8ArrayFromArrayBuffer(buffer: ArrayBuffer)

Create Uint8Array with initial content from buffer

buffer.toBase64
buffer.fromBase64
buffer.copyUint8Array(uint8Array: Uint8Array): Uint8Array

Copy the content of an Uint8Array view to a new ArrayBuffer.

buffer.encodeAny(data: any): Uint8Array

Encode anything as a UInt8Array. It's a pun on typescripts's any type. See encoding.writeAny for more information.

buffer.decodeAny(buf: Uint8Array): any

Decode an any-encoded value.

[lib0/component] Web components.

import * as component from 'lib0/component.js'

component.registry
component.define(name: string, constr: any, opts: ElementDefinitionOptions)
component.whenDefined(name: string): Promise<void>
new component.Lib0Component(state: S)
component.Lib0Component#state: S|null
component.Lib0Component#setState(state: S)
component.Lib0Component#updateState(stateUpdate: any)
component.createComponent(name: string, cnf: module:component~CONF<T>): Class<module:component.Lib0Component>
component.createComponentDefiner(definer: function)
component.defineListComponent
component.defineLazyLoadingComponent

[lib0/conditions] Often used conditions.

import * as conditions from 'lib0/conditions.js'

conditions.undefinedToNull

[lib0/decoding] Efficient schema-less binary decoding with support for variable length encoding.

import * as decoding from 'lib0/decoding.js'

Use [lib0/decoding] with [lib0/encoding]. Every encoding function has a corresponding decoding function.

Encodes numbers in little-endian order (least to most significant byte order) and is compatible with Golang's binary encoding (https://golang.org/pkg/encoding/binary/) which is also used in Protocol Buffers.

// encoding step
const encoder = new encoding.createEncoder()
encoding.writeVarUint(encoder, 256)
encoding.writeVarString(encoder, 'Hello world!')
const buf = encoding.toUint8Array(encoder)

// decoding step
const decoder = new decoding.createDecoder(buf)
decoding.readVarUint(decoder) // => 256
decoding.readVarString(decoder) // => 'Hello world!'
decoding.hasContent(decoder) // => false - all data is read

new decoding.Decoder(uint8Array: Uint8Array)

A Decoder handles the decoding of an Uint8Array.

decoding.Decoder#arr: Uint8Array

Decoding target.

decoding.Decoder#pos: number

Current decoding position.

decoding.createDecoder(uint8Array: Uint8Array): module:decoding.Decoder
decoding.hasContent(decoder: module:decoding.Decoder): boolean
decoding.clone(decoder: module:decoding.Decoder, newPos: number): module:decoding.Decoder

Clone a decoder instance. Optionally set a new position parameter.

decoding.readUint8Array(decoder: module:decoding.Decoder, len: number): Uint8Array

Create an Uint8Array view of the next len bytes and advance the position by len.

Important: The Uint8Array still points to the underlying ArrayBuffer. Make sure to discard the result as soon as possible to prevent any memory leaks. Use buffer.copyUint8Array to copy the result into a new Uint8Array.

decoding.readVarUint8Array(decoder: module:decoding.Decoder): Uint8Array

Read variable length Uint8Array.

decoding.readTailAsUint8Array(decoder: module:decoding.Decoder): Uint8Array

Read the rest of the content as an ArrayBuffer

decoding.skip8(decoder: module:decoding.Decoder): number

Skip one byte, jump to the next position.

decoding.readUint8(decoder: module:decoding.Decoder): number

Read one byte as unsigned integer.

decoding.readUint16(decoder: module:decoding.Decoder): number

Read 2 bytes as unsigned integer.

decoding.readUint32(decoder: module:decoding.Decoder): number

Read 4 bytes as unsigned integer.

decoding.readUint32BigEndian(decoder: module:decoding.Decoder): number

Read 4 bytes as unsigned integer in big endian order. (most significant byte first)

decoding.peekUint8(decoder: module:decoding.Decoder): number

Look ahead without incrementing position. to the next byte and read it as unsigned integer.

decoding.peekUint16(decoder: module:decoding.Decoder): number

Look ahead without incrementing position. to the next byte and read it as unsigned integer.

decoding.peekUint32(decoder: module:decoding.Decoder): number

Look ahead without incrementing position. to the next byte and read it as unsigned integer.

decoding.readVarUint(decoder: module:decoding.Decoder): number

Read unsigned integer (32bit) with variable length. 1/8th of the storage is used as encoding overhead.

numbers < 2^7 is stored in one bytlength
numbers < 2^14 is stored in two bylength

decoding.readVarInt(decoder: module:decoding.Decoder): number

Read signed integer (32bit) with variable length. 1/8th of the storage is used as encoding overhead.

numbers < 2^7 is stored in one bytlength
numbers < 2^14 is stored in two bylength

decoding.peekVarUint(decoder: module:decoding.Decoder): number

Look ahead and read varUint without incrementing position

decoding.peekVarInt(decoder: module:decoding.Decoder): number

Look ahead and read varUint without incrementing position

decoding.readVarString(decoder: module:decoding.Decoder): String

Read string of variable length

varUint is used to store the length of the string

Transforming utf8 to a string is pretty expensive. The code performs 10x better when String.fromCodePoint is fed with all characters as arguments. But most environments have a maximum number of arguments per functions. For effiency reasons we apply a maximum of 10000 characters at once.

decoding.peekVarString(decoder: module:decoding.Decoder): string

Look ahead and read varString without incrementing position

decoding.readFromDataView(decoder: module:decoding.Decoder, len: number): DataView
decoding.readFloat32(decoder: module:decoding.Decoder)
decoding.readFloat64(decoder: module:decoding.Decoder)
decoding.readBigInt64(decoder: module:decoding.Decoder)
decoding.readBigUint64(decoder: module:decoding.Decoder)
decoding.readAny(decoder: module:decoding.Decoder)
new decoding.RleDecoder(uint8Array: Uint8Array, reader: function(module:decoding.Decoder):T)

T must not be null.

decoding.RleDecoder#s: T|null

Current state

decoding.RleDecoder#read()
decoding.RleDecoder#s: T
new decoding.IntDiffDecoder(uint8Array: Uint8Array, start: number)
decoding.IntDiffDecoder#s: number

Current state

decoding.IntDiffDecoder#read(): number
new decoding.RleIntDiffDecoder(uint8Array: Uint8Array, start: number)
decoding.RleIntDiffDecoder#s: number

Current state

decoding.RleIntDiffDecoder#read(): number
decoding.RleIntDiffDecoder#s: number
new decoding.UintOptRleDecoder(uint8Array: Uint8Array)
decoding.UintOptRleDecoder#s: number
decoding.UintOptRleDecoder#read()
decoding.UintOptRleDecoder#s: number
new decoding.IncUintOptRleDecoder(uint8Array: Uint8Array)
decoding.IncUintOptRleDecoder#s: number
decoding.IncUintOptRleDecoder#read()
new decoding.IntDiffOptRleDecoder(uint8Array: Uint8Array)
decoding.IntDiffOptRleDecoder#s: number
decoding.IntDiffOptRleDecoder#read(): number
new decoding.StringDecoder(uint8Array: Uint8Array)
decoding.StringDecoder#spos: number
decoding.StringDecoder#read(): string
decoding.RleDecoder#arr: Uint8Array

Decoding target.

decoding.RleDecoder#pos: number

Current decoding position.

decoding.IntDiffDecoder#arr: Uint8Array

Decoding target.

decoding.IntDiffDecoder#pos: number

Current decoding position.

decoding.RleIntDiffDecoder#arr: Uint8Array

Decoding target.

decoding.RleIntDiffDecoder#pos: number

Current decoding position.

decoding.UintOptRleDecoder#arr: Uint8Array

Decoding target.

decoding.UintOptRleDecoder#pos: number

Current decoding position.

decoding.IncUintOptRleDecoder#arr: Uint8Array

Decoding target.

decoding.IncUintOptRleDecoder#pos: number

Current decoding position.

decoding.IntDiffOptRleDecoder#arr: Uint8Array

Decoding target.

decoding.IntDiffOptRleDecoder#pos: number

Current decoding position.

[lib0/diff] Efficient diffs.

import * as diff from 'lib0/diff.js'

diff.simpleDiffString(a: string, b: string): module:diff~SimpleDiff<string>

Create a diff between two strings. This diff implementation is highly efficient, but not very sophisticated.

diff.simpleDiff
diff.simpleDiffArray(a: Array<T>, b: Array<T>, compare: function(T, T):boolean): module:diff~SimpleDiff<Array<T>>

Create a diff between two arrays. This diff implementation is highly efficient, but not very sophisticated.

Note: This is basically the same function as above. Another function was created so that the runtime can better optimize these function calls.

[lib0/dom] Utility module to work with the DOM.

import * as dom from 'lib0/dom.js'

dom.doc: Document
dom.createElement
dom.createDocumentFragment
dom.createTextNode
dom.domParser
dom.emitCustomEvent
dom.setAttributes
dom.setAttributesMap
dom.fragment
dom.append
dom.remove
dom.addEventListener
dom.removeEventListener
dom.addEventListeners
dom.removeEventListeners
dom.element
dom.canvas
dom.text
dom.pairToStyleString
dom.pairsToStyleString
dom.mapToStyleString
dom.querySelector
dom.querySelectorAll
dom.getElementById
dom.parseFragment
dom.childNodes: any
dom.parseElement
dom.replaceWith
dom.insertBefore
dom.appendChild
dom.ELEMENT_NODE
dom.TEXT_NODE
dom.CDATA_SECTION_NODE
dom.COMMENT_NODE
dom.DOCUMENT_NODE
dom.DOCUMENT_TYPE_NODE
dom.DOCUMENT_FRAGMENT_NODE
dom.checkNodeType(node: any, type: number)
dom.isParentOf(parent: Node, child: HTMLElement)

[lib0/encoding] Efficient schema-less binary encoding with support for variable length encoding.

import * as encoding from 'lib0/encoding.js'

Use [lib0/encoding] with [lib0/decoding]. Every encoding function has a corresponding decoding function.

// encoding step
const encoder = new encoding.createEncoder()
encoding.writeVarUint(encoder, 256)
encoding.writeVarString(encoder, 'Hello world!')
const buf = encoding.toUint8Array(encoder)

// decoding step
const decoder = new decoding.createDecoder(buf)
decoding.readVarUint(decoder) // => 256
decoding.readVarString(decoder) // => 'Hello world!'
decoding.hasContent(decoder) // => false - all data is read

new encoding.Encoder()

A BinaryEncoder handles the encoding to an Uint8Array.

encoding.Encoder#bufs: Array<Uint8Array>
encoding.createEncoder(): module:encoding.Encoder
encoding.length(encoder: module:encoding.Encoder): number

The current length of the encoded data.

encoding.toUint8Array(encoder: module:encoding.Encoder): Uint8Array

Transform to Uint8Array.

encoding.write(encoder: module:encoding.Encoder, num: number)

Write one byte to the encoder.

encoding.set(encoder: module:encoding.Encoder, pos: number, num: number)

Write one byte at a specific position. Position must already be written (i.e. encoder.length > pos)

encoding.writeUint8(encoder: module:encoding.Encoder, num: number)

Write one byte as an unsigned integer.

encoding.setUint8(encoder: module:encoding.Encoder, pos: number, num: number)

Write one byte as an unsigned Integer at a specific location.

encoding.writeUint16(encoder: module:encoding.Encoder, num: number)

Write two bytes as an unsigned integer.

encoding.setUint16(encoder: module:encoding.Encoder, pos: number, num: number)

Write two bytes as an unsigned integer at a specific location.

encoding.writeUint32(encoder: module:encoding.Encoder, num: number)

Write two bytes as an unsigned integer

encoding.writeUint32BigEndian(encoder: module:encoding.Encoder, num: number)

Write two bytes as an unsigned integer in big endian order. (most significant byte first)

encoding.setUint32(encoder: module:encoding.Encoder, pos: number, num: number)

Write two bytes as an unsigned integer at a specific location.

encoding.writeVarUint(encoder: module:encoding.Encoder, num: number)

Write a variable length unsigned integer.

Encodes integers in the range from [0, 4294967295] / [0, 0xffffffff]. (max 32 bit unsigned integer)

encoding.writeVarInt(encoder: module:encoding.Encoder, num: number)

Write a variable length integer.

Encodes integers in the range from [-2147483648, -2147483647].

We don't use zig-zag encoding because we want to keep the option open to use the same function for BigInt and 53bit integers (doubles).

We use the 7th bit instead for signaling that this is a negative number.

encoding.writeVarString(encoder: module:encoding.Encoder, str: String)

Write a variable length string.

encoding.writeBinaryEncoder(encoder: module:encoding.Encoder, append: module:encoding.Encoder)

Write the content of another Encoder.

TODO: can be improved!

encoding.writeUint8Array(encoder: module:encoding.Encoder, uint8Array: Uint8Array)

Append fixed-length Uint8Array to the encoder.

encoding.writeVarUint8Array(encoder: module:encoding.Encoder, uint8Array: Uint8Array)

Append an Uint8Array to Encoder.

encoding.writeOnDataView(encoder: module:encoding.Encoder, len: number): DataView

Create an DataView of the next len bytes. Use it to write data after calling this function.

// write float32 using DataView
const dv = writeOnDataView(encoder, 4)
dv.setFloat32(0, 1.1)
// read float32 using DataView
const dv = readFromDataView(encoder, 4)
dv.getFloat32(0) // => 1.100000023841858 (leaving it to the reader to find out why this is the correct result)

encoding.writeFloat32(encoder: module:encoding.Encoder, num: number)
encoding.writeFloat64(encoder: module:encoding.Encoder, num: number)
encoding.writeBigInt64(encoder: module:encoding.Encoder, num: bigint)
encoding.writeBigUint64(encoder: module:encoding.Encoder, num: bigint)
encoding.writeAny(encoder: module:encoding.Encoder, data: undefined|null|number|bigint|boolean|string|Object<string,any>|Array<any>|Uint8Array)

Encode data with efficient binary format.

Differences to JSON: • Transforms data to a binary format (not to a string) • Encodes undefined, NaN, and ArrayBuffer (these can't be represented in JSON) • Numbers are efficiently encoded either as a variable length integer, as a 32 bit float, as a 64 bit float, or as a 64 bit bigint.

Encoding table:

Data Type	Prefix	Encoding Method	Comment
undefined	127		Functions, symbol, and everything that cannot be identified is encoded as undefined
null	126
integer	125	writeVarInt	Only encodes 32 bit signed integers
float32	124	writeFloat32
float64	123	writeFloat64
bigint	122	writeBigInt64
boolean (false)	121		True and false are different data types so we save the following byte
boolean (true)	120		- 0b01111000 so the last bit determines whether true or false
string	119	writeVarString
object<string,any>	118	custom	Writes {length} then {length} key-value pairs
array	117	custom	Writes {length} then {length} json values
Uint8Array	116	writeVarUint8Array	We use Uint8Array for any kind of binary data

Reasons for the decreasing prefix: We need the first bit for extendability (later we may want to encode the prefix with writeVarUint). The remaining 7 bits are divided as follows: [0-30] the beginning of the data range is used for custom purposes (defined by the function that uses this library) [31-127] the end of the data range is used for data encoding by lib0/encoding.js

new encoding.RleEncoder(writer: function(module:encoding.Encoder, T):void)

Now come a few stateful encoder that have their own classes.

encoding.RleEncoder#s: T|null

Current state

encoding.RleEncoder#write(v: T)
new encoding.IntDiffEncoder(start: number)

Basic diff decoder using variable length encoding.

Encodes the values [3, 1100, 1101, 1050, 0] to [3, 1097, 1, -51, -1050] using writeVarInt.

encoding.IntDiffEncoder#s: number

Current state

encoding.IntDiffEncoder#write(v: number)
new encoding.RleIntDiffEncoder(start: number)

A combination of IntDiffEncoder and RleEncoder.

Basically first writes the IntDiffEncoder and then counts duplicate diffs using RleEncoding.

Encodes the values [1,1,1,2,3,4,5,6] as [1,1,0,2,1,5] (RLE([1,0,0,1,1,1,1,1]) ⇒ RleIntDiff[1,1,0,2,1,5])

encoding.RleIntDiffEncoder#s: number

Current state

encoding.RleIntDiffEncoder#write(v: number)
new encoding.UintOptRleEncoder()

Optimized Rle encoder that does not suffer from the mentioned problem of the basic Rle encoder.

Internally uses VarInt encoder to write unsigned integers. If the input occurs multiple times, we write write it as a negative number. The UintOptRleDecoder then understands that it needs to read a count.

Encodes [1,2,3,3,3] as [1,2,-3,3] (once 1, once 2, three times 3)

encoding.UintOptRleEncoder#s: number
encoding.UintOptRleEncoder#write(v: number)
encoding.UintOptRleEncoder#toUint8Array()
new encoding.IncUintOptRleEncoder()

Increasing Uint Optimized RLE Encoder

The RLE encoder counts the number of same occurences of the same value. The IncUintOptRle encoder counts if the value increases. I.e. 7, 8, 9, 10 will be encoded as [-7, 4]. 1, 3, 5 will be encoded as [1, 3, 5].

encoding.IncUintOptRleEncoder#s: number
encoding.IncUintOptRleEncoder#write(v: number)
encoding.IncUintOptRleEncoder#toUint8Array()
new encoding.IntDiffOptRleEncoder()

A combination of the IntDiffEncoder and the UintOptRleEncoder.

The count approach is similar to the UintDiffOptRleEncoder, but instead of using the negative bitflag, it encodes in the LSB whether a count is to be read. Therefore this Encoder only supports 31 bit integers!

Encodes [1, 2, 3, 2] as [3, 1, 6, -1] (more specifically [(1 << 1) | 1, (3 << 0) | 0, -1])

Internally uses variable length encoding. Contrary to normal UintVar encoding, the first byte contains:

1 bit that denotes whether the next value is a count (LSB)
1 bit that denotes whether this value is negative (MSB - 1)
1 bit that denotes whether to continue reading the variable length integer (MSB)

Therefore, only five bits remain to encode diff ranges.

Use this Encoder only when appropriate. In most cases, this is probably a bad idea.

encoding.IntDiffOptRleEncoder#s: number
encoding.IntDiffOptRleEncoder#write(v: number)
encoding.IntDiffOptRleEncoder#toUint8Array()
new encoding.StringEncoder()

Optimized String Encoder.

Encoding many small strings in a simple Encoder is not very efficient. The function call to decode a string takes some time and creates references that must be eventually deleted. In practice, when decoding several million small strings, the GC will kick in more and more often to collect orphaned string objects (or maybe there is another reason?).

This string encoder solves the above problem. All strings are concatenated and written as a single string using a single encoding call.

The lengths are encoded using a UintOptRleEncoder.

encoding.StringEncoder#sarr: Array<string>
encoding.StringEncoder#write(string: string)
encoding.StringEncoder#toUint8Array()
encoding.RleEncoder#bufs: Array<Uint8Array>
encoding.IntDiffEncoder#bufs: Array<Uint8Array>
encoding.RleIntDiffEncoder#bufs: Array<Uint8Array>

[lib0/map] Isomorphic module to work access the environment (query params, env variables).

import * as map from 'lib0/environment.js'

map.isNode
map.isBrowser
map.isMac
map.hasParam
map.getParam
map.getVariable
map.getConf(name: string): string|null
map.hasConf
map.production

[lib0/error] Error helpers.

import * as error from 'lib0/error.js'

error.create
error.methodUnimplemented
error.unexpectedCase

[lib0/eventloop] Utility module to work with EcmaScript's event loop.

import * as eventloop from 'lib0/eventloop.js'

eventloop.enqueue(f: function():void)
eventloop#destroy()
eventloop.timeout(timeout: number, callback: function): module:eventloop~TimeoutObject
eventloop.interval(timeout: number, callback: function): module:eventloop~TimeoutObject
eventloop.Animation
eventloop.animationFrame(cb: function(number):void): module:eventloop~TimeoutObject
eventloop.idleCallback(cb: function): module:eventloop~TimeoutObject

Note: this is experimental and is probably only useful in browsers.

eventloop.createDebouncer(timeout: number): function(function():void):void

[lib0/function] Common functions and function call helpers.

import * as function from 'lib0/function.js'

function.callAll(fs: Array<function>, args: Array<any>)

Calls all functions in fs with args. Only throws after all functions were called.

function.nop
function.apply(f: function():T): T
function.id(a: A): A
function.equalityStrict(a: T, b: T): boolean
function.equalityFlat(a: Array<T>|object, b: Array<T>|object): boolean
function.equalityDeep(a: any, b: any): boolean

[lib0/lib0] Experimental method to import lib0.

import * as lib0 from 'lib0/index.js'

Not recommended if the module bundler doesn't support dead code elimination.

[lib0/idb] Helpers to work with IndexedDB.

import * as idb from 'lib0/indexeddb.js'

idb.rtop
idb.openDB
idb.deleteDB
idb.createStores
idb.transact(db: IDBDatabase, stores: Array<string>, access: "readwrite"|"readonly"): Array<IDBObjectStore>
idb.count
idb.get
idb.del
idb.put
idb.add
idb.addAutoKey
idb.getAll
idb.getAllKeys
idb.queryFirst(store: IDBObjectStore, query: IDBKeyRange|null, direction: 'next'|'prev'|'nextunique'|'prevunique'): Promise<any>
idb.getLastKey(store: IDBObjectStore): Promise<any>
idb.getFirstKey(store: IDBObjectStore): Promise<any>
idb.getAllKeysValues
idb.iterate
idb.iterateKeys
idb.getStore
idb.createIDBKeyRangeBound
idb.createIDBKeyRangeUpperBound
idb.createIDBKeyRangeLowerBound

[lib0/isomorphic] Isomorphic library exports from isomorphic.js.

import * as isomorphic from 'lib0/isomorphic.js'

isomorphic.performance
isomorphic.cryptoRandomBuffer

[lib0/iterator] Utility module to create and manipulate Iterators.

import * as iterator from 'lib0/iterator.js'

iterator.mapIterator(iterator: Iterator<T>, f: function(T):R): IterableIterator<R>
iterator.createIterator(next: function():IteratorResult<T>): IterableIterator<T>
iterator.iteratorFilter(iterator: Iterator<T>, filter: function(T):boolean)
iterator.iteratorMap(iterator: Iterator<T>, fmap: function(T):M)

[lib0/json] JSON utility functions.

import * as json from 'lib0/json.js'

json.stringify(object: any): string

Transform JavaScript object to JSON.

json.parse(json: string): any

Parse JSON object.

[lib0/logging] Isomorphic logging module with support for colors!

import * as logging from 'lib0/logging.js'

logging.BOLD
logging.UNBOLD
logging.BLUE
logging.GREY
logging.GREEN
logging.RED
logging.PURPLE
logging.ORANGE
logging.UNCOLOR
logging.print(args: Array<string|Symbol|Object|number>)
logging.warn(args: Array<string|Symbol|Object|number>)
logging.printError(err: Error)
logging.printImg(url: string, height: number)
logging.printImgBase64(base64: string, height: number)
logging.group(args: Array<string|Symbol|Object|number>)
logging.groupCollapsed(args: Array<string|Symbol|Object|number>)
logging.groupEnd
logging.printDom(createNode: function():Node)
logging.printCanvas(canvas: HTMLCanvasElement, height: number)
logging.vconsoles
new logging.VConsole(dom: Element)
logging.VConsole#ccontainer: Element
logging.VConsole#group(args: Array<string|Symbol|Object|number>, collapsed: boolean)
logging.VConsole#groupCollapsed(args: Array<string|Symbol|Object|number>)
logging.VConsole#groupEnd()
logging.VConsole#print(args: Array<string|Symbol|Object|number>)
logging.VConsole#printError(err: Error)
logging.VConsole#printImg(url: string, height: number)
logging.VConsole#printDom(node: Node)
logging.VConsole#destroy()
logging.createVConsole(dom: Element)
logging.createModuleLogger(moduleName: string): function(...any)

[lib0/map] Utility module to work with key-value stores.

import * as map from 'lib0/map.js'

map.create(): Map<any, any>

Creates a new Map instance.

map.copy(m: Map<X,Y>): Map<X,Y>

Copy a Map object into a fresh Map object.

map.setIfUndefined(map: Map<K, T>, key: K, createT: function():T): T

Get map property. Create T if property is undefined and set T on map.

const listeners = map.setIfUndefined(events, 'eventName', set.create)
listeners.add(listener)

map.map(m: Map<K,V>, f: function(V,K):R): Array<R>

Creates an Array and populates it with the content of all key-value pairs using the f(value, key) function.

map.any(m: Map<K,V>, f: function(V,K):boolean): boolean

Tests whether any key-value pairs pass the test implemented by f(value, key).

map.all(m: Map<K,V>, f: function(V,K):boolean): boolean

Tests whether all key-value pairs pass the test implemented by f(value, key).

[lib0/math] Common Math expressions.

import * as math from 'lib0/math.js'

math.floor
math.ceil
math.abs
math.imul
math.round
math.log10
math.log2
math.log
math.sqrt
math.add(a: number, b: number): number
math.min(a: number, b: number): number
math.max(a: number, b: number): number
math.isNaN
math.pow
math.exp10(exp: number): number

Base 10 exponential function. Returns the value of 10 raised to the power of pow.

math.sign
math.isNegativeZero(n: number): boolean

[lib0/metric] Utility module to convert metric values.

import * as metric from 'lib0/metric.js'

metric.yotta
metric.zetta
metric.exa
metric.peta
metric.tera
metric.giga
metric.mega
metric.kilo
metric.hecto
metric.deca
metric.deci
metric.centi
metric.milli
metric.micro
metric.nano
metric.pico
metric.femto
metric.atto
metric.zepto
metric.yocto
metric.prefix(n: number, baseMultiplier: number): {n:number,prefix:string}

Calculate the metric prefix for a number. Assumes E.g. prefix(1000) = { n: 1, prefix: 'k' }

[lib0/mutex] Mutual exclude for JavaScript.

import * as mutex from 'lib0/mutex.js'

mutex.createMutex(): mutex

Creates a mutual exclude function with the following property:

const mutex = createMutex()
mutex(() => {
  // This function is immediately executed
  mutex(() => {
    // This function is not executed, as the mutex is already active.
  })
})

[lib0/number]

import * as number from 'lib0/number.js'

number.MAX_SAFE_INTEGER
number.MIN_SAFE_INTEGER
number.LOWEST_INT32
number.HIGHEST_INT32: number
number.isInteger
number.isNaN

[lib0/object] Utility functions for working with EcmaScript objects.

import * as object from 'lib0/object.js'

object.create(): Object<string,any>
object.assign

Object.assign

object.keys(obj: Object<string,any>)
object.forEach(obj: Object<string,any>, f: function(any,string):any)
object.map(obj: Object<string,any>, f: function(any,string):R): Array<R>
object.length(obj: Object<string,any>): number
object.some(obj: Object<string,any>, f: function(any,string):boolean): boolean
object.every(obj: Object<string,any>, f: function(any,string):boolean): boolean
object.hasProperty(obj: any, key: string|symbol): boolean

Calls Object.prototype.hasOwnProperty.

object.equalFlat(a: Object<string,any>, b: Object<string,any>): boolean

[lib0/observable] Observable class prototype.

import * as observable from 'lib0/observable.js'

new observable.Observable()

Handles named events.

observable.Observable#on(name: N, f: function)
observable.Observable#once(name: N, f: function)
observable.Observable#off(name: N, f: function)
observable.Observable#emit(name: N, args: Array<any>)

Emit a named event. All registered event listeners that listen to the specified name will receive the event.

observable.Observable#destroy()
websocket.WebsocketClient#on(name: N, f: function)
websocket.WebsocketClient#once(name: N, f: function)
websocket.WebsocketClient#off(name: N, f: function)
websocket.WebsocketClient#emit(name: N, args: Array<any>)

Emit a named event. All registered event listeners that listen to the specified name will receive the event.

[lib0/pair] Working with value pairs.

import * as pair from 'lib0/pair.js'

new pair.Pair(left: L, right: R)
pair.create(left: L, right: R): module:pair.Pair<L,R>
pair.createReversed(right: R, left: L): module:pair.Pair<L,R>
pair.forEach(arr: Array<module:pair.Pair<L,R>>, f: function(L, R):any)
pair.map(arr: Array<module:pair.Pair<L,R>>, f: function(L, R):X): Array<X>

[lib0/prng] Fast Pseudo Random Number Generators.

import * as prng from 'lib0/prng.js'

Given a seed a PRNG generates a sequence of numbers that cannot be reasonably predicted. Two PRNGs must generate the same random sequence of numbers if given the same seed.

prng.DefaultPRNG
prng.create(seed: number): module:prng~PRNG

Create a Xoroshiro128plus Pseudo-Random-Number-Generator. This is the fastest full-period generator passing BigCrush without systematic failures. But there are more PRNGs available in ./PRNG/.

prng.bool(gen: module:prng~PRNG): Boolean

Generates a single random bool.

prng.int53(gen: module:prng~PRNG, min: Number, max: Number): Number

Generates a random integer with 53 bit resolution.

prng.uint53(gen: module:prng~PRNG, min: Number, max: Number): Number

Generates a random integer with 53 bit resolution.

prng.int32(gen: module:prng~PRNG, min: Number, max: Number): Number

Generates a random integer with 32 bit resolution.

prng.uint32(gen: module:prng~PRNG, min: Number, max: Number): Number

Generates a random integer with 53 bit resolution.

prng.int31(gen: module:prng~PRNG, min: Number, max: Number): Number
prng.real53(gen: module:prng~PRNG): Number

Generates a random real on [0, 1) with 53 bit resolution.

prng.char(gen: module:prng~PRNG): string

Generates a random character from char code 32 - 126. I.e. Characters, Numbers, special characters, and Space:

prng.letter(gen: module:prng~PRNG): string
prng.word(gen: module:prng~PRNG, minLen: number, maxLen: number): string
prng.utf16Rune(gen: module:prng~PRNG): string

TODO: this function produces invalid runes. Does not cover all of utf16!!

prng.utf16String(gen: module:prng~PRNG, maxlen: number)
prng.oneOf(gen: module:prng~PRNG, array: Array<T>): T

Returns one element of a given array.

prng.uint8Array(gen: module:prng~PRNG, len: number): Uint8Array
prng.uint16Array(gen: module:prng~PRNG, len: number): Uint16Array
prng.uint32Array(gen: module:prng~PRNG, len: number): Uint32Array

[lib0/promise] Utility helpers to work with promises.

import * as promise from 'lib0/promise.js'

promise.create(f: function(PromiseResolve<T>,function(Error):void):any): Promise<T>
promise.createEmpty(f: function(function():void,function(Error):void):void): Promise<void>
promise.all(arrp: Array<Promise<T>>): Promise<Array<T>>

Promise.all wait for all promises in the array to resolve and return the result

promise.reject(reason: Error): Promise<never>
promise.resolve(res: T|void): Promise<T|void>
promise.until(timeout: number, check: function():boolean, intervalResolution: number): Promise<void>
promise.wait(timeout: number): Promise<undefined>
promise.isPromise(p: any): boolean

Checks if an object is a promise using ducktyping.

Promises are often polyfilled, so it makes sense to add some additional guarantees if the user of this library has some insane environment where global Promise objects are overwritten.

[lib0/queue]

import * as queue from 'lib0/queue.js'

new de#QueueNode()
de#next: module:queue.QueueNode|null
new ueue()
tart: module:queue.QueueNode | null
nd: module:queue.QueueNode | null
(): module:queue.Queue
()
(queue: module:queue.Queue)
()
(queue: module:queue.Queue, n: module:queue.QueueNode)
()
(queue: module:queue.Queue): module:queue.QueueNode | null
()

[lib0/random] Isomorphic module for true random numbers / buffers / uuids.

import * as random from 'lib0/random.js'

Attention: falls back to Math.random if the browser does not support crypto.

random.rand
random.uint32
random.oneOf(arr: Array<T>): T
random.uuidv4

[lib0/set] Utility module to work with sets.

import * as set from 'lib0/set.js'

set.create
set.toArray(set: Set<T>): Array<T>

[lib0/sort] Efficient sort implementations.

import * as sort from 'lib0/sort.js'

Note: These sort implementations were created to compare different sorting algorithms in JavaScript. Don't use them if you don't know what you are doing. Native Array.sort is almost always a better choice.

sort.insertionSort(arr: Array<T>, compare: function(T,T):number): void
sort.quicksort(arr: Array<T>, compare: function(T,T):number): void

This algorithm beats Array.prototype.sort in Chrome only with arrays with 10 million entries. In most cases [].sort will do just fine. Make sure to performance test your use-case before you integrate this algorithm.

Note that Chrome's sort is now a stable algorithm (Timsort). Quicksort is not stable.

[lib0/statistics] Utility helpers for generating statistics.

import * as statistics from 'lib0/statistics.js'

statistics.median(arr: Array<number>): number
statistics.average(arr: Array<number>): number

[lib0/storage] Isomorphic variable storage.

import * as storage from 'lib0/storage.js'

Uses LocalStorage in the browser and falls back to in-memory storage.

storage.varStorage

This is basically localStorage in browser, or a polyfill in nodejs

[lib0/string] Utility module to work with strings.

import * as string from 'lib0/string.js'

string.fromCharCode
string.fromCodePoint
string.trimLeft(s: string): string
string.fromCamelCase(s: string, separator: string): string
string.utf8ByteLength(str: string): number

Compute the utf8ByteLength

string.utf8TextEncoder
string.encodeUtf8
string.decodeUtf8

[lib0/symbol] Utility module to work with EcmaScript Symbols.

import * as symbol from 'lib0/symbol.js'

symbol.create(): Symbol

Return fresh symbol.

symbol.isSymbol(s: any): boolean

[lib0/testing] Testing framework with support for generating tests.

import * as testing from 'lib0/testing.js'

// test.js template for creating a test executable
import { runTests } from 'lib0/testing.js'
import * as log from 'lib0/logging.js'
import * as mod1 from './mod1.test.js'
import * as mod2 from './mod2.test.js'
import { isBrowser, isNode } from 'lib0/environment.js'

if (isBrowser) {
  // optional: if this is ran in the browser, attach a virtual console to the dom
  log.createVConsole(document.body)
}

runTests({
 mod1,
 mod2,
}).then(success => {
  if (isNode) {
    process.exit(success ? 0 : 1)
  }
})

// mod1.test.js
/**
 * runTests automatically tests all exported functions that start with "test".
 * The name of the function should be in camelCase and is used for the logging output.
 *
 * @param {t.TestCase} tc
 *\/
export const testMyFirstTest = tc => {
  t.compare({ a: 4 }, { a: 4 }, 'objects are equal')
}

Now you can simply run node test.js to run your test or run test.js in the browser.

testing.extensive
testing.envSeed
new testing.TestCase(moduleName: string, testName: string)
testing.TestCase#moduleName: string
testing.TestCase#testName: string
testing.TestCase#resetSeed()
testing.TestCase#prng: prng.PRNG

A PRNG for this test case. Use only this PRNG for randomness to make the test case reproducible.

testing.repititionTime
testing.run(moduleName: string, name: string, f: function(module:testing.TestCase):void|Promise<any>, i: number, numberOfTests: number)
testing.describe(description: string, info: string)

Describe what you are currently testing. The message will be logged.

export const testMyFirstTest = tc => {
  t.describe('crunching numbers', 'already crunched 4 numbers!') // the optional second argument can describe the state.
}

testing.info(info: string)

Describe the state of the current computation.

export const testMyFirstTest = tc => {
  t.info(already crunched 4 numbers!') // the optional second argument can describe the state.
}

testing.printDom
testing.printCanvas
testing.group(description: string, f: function(void):void)

Group outputs in a collapsible category.

export const testMyFirstTest = tc => {
  t.group('subtest 1', () => {
    t.describe('this message is part of a collapsible section')
  })
  await t.groupAsync('subtest async 2', async () => {
    await someaction()
    t.describe('this message is part of a collapsible section')
  })
}

testing.groupAsync(description: string, f: function(void):Promise<any>)

Group outputs in a collapsible category.

export const testMyFirstTest = async tc => {
  t.group('subtest 1', () => {
    t.describe('this message is part of a collapsible section')
  })
  await t.groupAsync('subtest async 2', async () => {
    await someaction()
    t.describe('this message is part of a collapsible section')
  })
}

testing.measureTime(message: string, f: function():void): number

Measure the time that it takes to calculate something.

export const testMyFirstTest = async tc => {
  t.measureTime('measurement', () => {
    heavyCalculation()
  })
  await t.groupAsync('async measurement', async () => {
    await heavyAsyncCalculation()
  })
}

testing.measureTimeAsync(message: string, f: function():Promise<any>): Promise<number>

Measure the time that it takes to calculate something.

export const testMyFirstTest = async tc => {
  t.measureTimeAsync('measurement', async () => {
    await heavyCalculation()
  })
  await t.groupAsync('async measurement', async () => {
    await heavyAsyncCalculation()
  })
}

testing.compareArrays(as: Array<T>, bs: Array<T>, m: string): boolean
testing.compareStrings(a: string, b: string, m: string)
testing.compareObjects(a: Object<K,V>, b: Object<K,V>, m: string)
testing.compare(a: T, b: T, message: string?, customCompare: function(any,T,T,string,any):boolean)
testing.assert(condition: boolean, message: string?)
testing.fails(f: function():void)
testing.runTests(tests: Object<string, Object<string, function(module:testing.TestCase):void|Promise<any>>>)
testing.fail(reason: string)
testing.skip(cond: boolean)

[lib0/time] Utility module to work with time.

import * as time from 'lib0/time.js'

time.getDate(): Date

Return current time.

time.getUnixTime(): number

Return current unix time.

time.humanizeDuration(d: number): string

Transform time (in ms) to a human readable format. E.g. 1100 => 1.1s. 60s => 1min. .001 => 10μs.

[lib0/tree] Red-black-tree implementation.

import * as tree from 'lib0/tree.js'

new tree.Tree()

This is a Red Black Tree implementation

tree.Tree#findNext(id: K)
tree.Tree#findPrev(id: K)
tree.Tree#findNodeWithLowerBound(from: K)
tree.Tree#findNodeWithUpperBound(to: K)
tree.Tree#findSmallestNode(): V
tree.Tree#findWithLowerBound(from: K): V
tree.Tree#findWithUpperBound(to: K): V
tree.Tree#iterate(from: K, from: K, f: K)
tree.Tree#find(id: K): V|null
tree.Tree#findNode(id: K): module:tree~N<V>|null
tree.Tree#delete(id: K)
tree.Tree#put()

[lib0/url] Utility module to work with urls.

import * as url from 'lib0/url.js'

url.decodeQueryParams(url: string): Object<string,string>

Parse query parameters from an url.

url.encodeQueryParams(params: Object<string,string>): string

[lib0/websocket] Tiny websocket connection handler.

import * as websocket from 'lib0/websocket.js'

Implements exponential backoff reconnects, ping/pong, and a nice event system using [lib0/observable].

new websocket.WebsocketClient(url: string, opts: object, opts.binaryType: 'arraybuffer' | 'blob' | null)
websocket.WebsocketClient#ws: WebSocket?
websocket.WebsocketClient#shouldConnect: boolean

Whether to connect to other peers or not

websocket.WebsocketClient#send(message: any)
websocket.WebsocketClient#destroy()
websocket.WebsocketClient#disconnect()
websocket.WebsocketClient#connect()

@raedle/lib0

Lib0

Performance resources

Code style

Modules

License

Dependents (1)

Package Sidebar

Install

Repository

Homepage

Weekly Downloads

Version

License

Unpacked Size

Total Files

Last publish

Collaborators

@raedle/lib0

Lib0

Performance resources

Code style

Modules

License

Dependents (1)

Package Sidebar

Install

Repository

Homepage

DownloadsWeekly Downloads

Version

License

Unpacked Size

Total Files

Last publish

Collaborators

Weekly Downloads