TypeScript Binary

Powerful, lightweight binary messages in TypeScript.

🔌 Compatible with geckos.io, socket.io and peer.js.
🗜️ Smaller than FlatBuffers or Protocol Buffers.
🏋️‍♀️ Support advanced features like property mangling and 16-bit floats.
⚡️ Based on the lightning-fast sitegui/js-binary library, written by Guilherme Souza.

📣 When to use?

TypeScript Binary is an optimal choice for real-time HTML5 and Node.js applications and games.

	TypeScript Binary	FlatBuffers	Protocol Buffers	Raw JSON
Serialization format	Binary	Binary	Binary	String
Schema definition	Native	.fbs files	.proto files	Native
TypeScript Types	Native	Code generation	Code generation	Native
External tooling dependencies	None	cmake and flatc	None^*	N/A
Reference data size^†	34 bytes	68 bytes	72 bytes	175 bytes (minified)
Fast & efficient	🟢	🟢	🟢	🔴
16-bit floats	🟢	🔴	🔴	🔴
Boolean-packing	🟢	🔴	🔴	🔴
Arbitrary JSON	🟢	🔴	🔴	🟢
Property mangling	🟢	🔴	🔴	🔴
Suitable for real-time data	🟢	🟢	🔴	🔴
Suitable for web APIs	🔴	🔴	🟢	🟢
Supports HTML5 / Node.js	🟢	🟢	🟢	🟢
Cross-language (Java, C++, Python, etc.)	🔴	🟢	🟢	🟢

^†Based on the Reference data formats and schemas

^*When using protobufjs

See Reference data

Sample data (Minified JSON):

{
  "players": [
    {
      "id": 123,
      "position": {
        "x": 1.0,
        "y": 2.0,
        "z": 3.0
      },
      "velocity": {
        "x": 1.0,
        "y": 2.0,
        "z": 3.0
      },
      "health": 1.00
    },
    {
      "id": 456,
      "position": {
        "x": 1.0,
        "y": 2.0,
        "z": 3.0
      },
      "velocity": {
        "x": 1.0,
        "y": 2.0,
        "y": 3.0
      },
      "health": 0.50
    }
  ]
}

TypeScript Binary

const ExampleMessage = new BinaryCoder({
  players: [
    {
      id: Type.UInt,
      position: {
        x: Type.Float16,
        y: Type.Float16,
        z: Type.Float16
      },
      velocity: {
        x: Type.Float16,
        y: Type.Float16,
        y: Type.Float16
      },
      health: Type.UScalar
    },
  ],
});

FlatBuffers

// ExampleMessage.fbs

namespace ExampleNamespace;

table Vec3 {
  x: float;
  y: float;
  z: float;
}

table Player {
  id: uint;
  position: Vec3;
  velocity: Vec3;
  health: float;
}

table ExampleMessage {
  players: [Player];
}

root_type ExampleMessage;

Protocol Buffers (Proto3)

syntax = "proto3";

package example;

message Vec3 {
  float x = 1;
  float y = 2;
  float z = 3;
}

message Player {
  uint32 id = 1;
  Vec3 position = 2;
  Vec3 velocity = 3;
  float health = 4;
}

message ExampleMessage {
  repeated Player players = 1;
}

Install

npm install typescript-binary

yarn add typescript-binary

Usage

Define formats

Create a BinaryCoder like so:

import { BinaryCoder, Type } from "typescript-binary";

// Define your format:
const GameWorldData = new BinaryCoder({
  time: Type.UInt,
  players: [{
    id: Type.UInt,
    isJumping: Type.Boolean,
    position: {
      x: Type.Float,
      y: Type.Float
    }
  }]
});

// Encode:
const bytes = GameWorldData.encode({
  time: 123,
  players: [
    {
       id: 44,
       isJumping: true,  
       position: {
         x: 110.57345,
         y: -93.5366
       }
    }
  ]
});

bytes.byteLength
// 14

// Decode:
const data = GameWorldData.decode(bytes);

Inferred types

BinaryCoder will automatically infer the types for encode() and decode() from the schema provided (see the Types section below).

For example, the type T for GameWorldData.decode(...): T would be inferred as:

{
  timeRemaining: number,
  players: {
    id: string,
    health: number,
    isJumping: boolean,
    position?: {
      x: number,
      y: number
    }
  }[]
}

You can also use the Infer<T> helper type to use inferred types in any custom method/handler:

import { Infer } from "typescript-binary";

function updateGameWorld(data: Infer<typeof GameWorldData>) {
  // e.g. Access `data.players[0].position?.x`
}

✨ Parsing formats

By default, each BinaryCoder encodes a 2-byte identifier based on the shape of the data.

You can explicitly set Id in the BinaryCoder constructor to any 2-byte string or unsigned integer (or disable entirely by passing false).

Use BinaryFormatHandler

Handle multiple binary formats at once using a BinaryFormatHandler:

import { BinaryFormatHandler } from "typescript-binary";

const binaryHandler = new BinaryFormatHandler()
  .on(MyFormatA, (data) => handleMyFormatA(data))
  .on(MyFormatB, (data) => handleMyFormatB(data));

// Trigger handler (or throw UnhandledBinaryDecodeError)
binaryHandler.processBuffer(binary);

Note: Cannot be used with formats where Id is disabled.

Manual handling

You can manually read message identifers from incoming buffers with the static function BinaryCoder.peekIntId(...) (or BinaryCoder.peekStrId(...)):

if (BinaryCoder.peekStrId(incomingBinary) === MyMessageFormat.Id) {
  // Do something special.
}

💥 Id Collisions

By default Id is based on a hash code of the encoding format. So the following two messages would have identical Ids:

const Person = new BinaryCoder({
  firstName: Type.String,
  lastName: Type.String
});

const FavoriteColor = new BinaryCoder({
  fullName: Type.String,
  color: Type.String
});

NameCoder.Id === ColorCoder.Id
  // true

If two identical formats with different handlers is a requirement, you can explicitly set unique identifiers.

const Person = new BinaryCoder({
  firstName: Type.String,
  lastName: Type.String
}, "PE");

const FavoriteColor = new BinaryCoder({
  fullName: Type.String,
  color: Type.String
}, "FC");

✨ Validation / Transforms

Validation

The great thing about binary encoders is that data is implicitly type-validated, however, you can also add custom validation rules using setValidation():

const UserMessage = new BinaryCoder({
  uuid: Type.String,
  // ...
})
.setValidation({
  uuid: (x) => {
    if (!isValidUUIDv4(x)) {
      throw new Error('Invalid UUIDv4: ' + x);
    }
  }
});

Transforms

You can also apply additional encode/decode transforms.

Here is an example where we're stripping out all whitespace:

const PositionMessage = new BinaryCoder({ name: Type.String })
  .setTransforms({ name: a => a.replace(/\s+/g, '') });

let binary = PositionMessage.encode({ name: 'Hello  There' })
let data = PositionMessage.decode(binary);

data.name
  // "HelloThere"

Unlike validation, transforms are applied asymmetrically.

The transform function is only applied on encode(), but you can provide two transform functions.

Here is an example which cuts the number of bytes required from 10 to 5:

const PercentMessage = new BinaryCoder({ value: Type.String }, false)
  .setTransforms({
    value: [
      (before) => before.replace(/\$|USD/g, '').trim(),
      (after) => '$' + after + ' USD'
    ]
  });

let binary = PercentMessage.encode({ value: ' $45.53 USD' })
let data = PercentMessage.decode(binary);

binary.byteLength
  // 5

data.value
  // "$45.53 USD"

Types

Here are all the ready-to-use types:

Type	JavaScript Type	Bytes	About
`Type.Int`	`number`	1-8^*	Integer between `-Number.MAX_SAFE_INTEGER` and `Number.MAX_SAFE_INTEGER`.
`Type.Int8`	`number`	1	Integer between -127 to 128.
`Type.Int16`	`number`	2	Integer between -32,767 to 32,767.
`Type.Int32`	`number`	4	Integer between -2,147,483,647 to 2,147,483,647.
`Type.UInt`	`number`	1-8^#	Unsigned integer between 0 and `Number.MAX_SAFE_INTEGER`.
`Type.UInt8`	`number`	1	Unsigned integer between 0 and 255.
`Type.UInt16`	`number`	2	Unsigned integer between 0 and 65,535.
`Type.UInt32`	`number`	4	Unsigned integer between 0 and 4,294,967,295.
`Type.Scalar`	`number`	1	Signed scalar between -1.0 and 1.0.
`Type.UScalar`	`number`	1	Unsigned scalar between 0.0 and 1.0.
`Type.Float16`	`number`	2	A 16-bit "half-precision" floating point. Important Note: Low decimal precision. Max. large values ±65,500.
`Type.Float32`	`number`	4	A 32-bit "single-precision" floating point.
`Type.Float64`	`number`	8	Default JavaScript `number` type. A 64-bit "double-precision" floating point.
`Type.String`	`string`	1^† + n	A UTF-8 string.
`Type.Boolean`	`boolean`	1	A single boolean.
`Type.BooleanTuple`	`boolean[]`	1^¶	Variable-length array/tuple of boolean values packed into 1^¶ byte.
`Type.Bitmask8`	`boolean[]`	1	8 booleans.
`Type.Bitmask16`	`boolean[]`	2	16 booleans.
`Type.Bitmask32`	`boolean[]`	4	32 booleans.
`Type.JSON`	`any`	1^† + n	JSON format data, encoded as a UTF-8 string.
`Type.Binary`	`ArrayBuffer`	1^† + n	JavaScript `ArrayBuffer` data.
`Type.RegExp`	`RegExp`	1^† + n + 1	JavaScript `RegExp` object.
`Type.Date`	`Date`	8	JavaScript `Date` object.
`Optional(T)`	`T \| undefined`	1	Any optional field. Use the `Optional(...)` helper. Array elements cannot be optional.
`[T]`	`Array<T>`	1^† + n	Use array syntax. Any array.
`{}`	`object`	none	Use object syntax. No overhead to using object types. Buffers are ordered, flattened structures.

^*Int is a variable-length integer ("varint") which encodes <±64 = 1 byte, <±8,192 = 2 bytes, <±268,435,456 = 4 bytes, otherwise = 8 bytes.

^#UInt is a variable-length unsigned integer ("varuint") which encodes <128 = 1 byte, <16,384 = 2 bytes, <536,870,912 = 4 bytes, otherwise = 8 bytes.

^†Length of payload bytes as a UInt. Typically 1 byte, but could be 2-8 bytes for very large payloads.

^¶2-bit overhead: 6 booleans per byte (i.e. 9 booleans would require 2 bytes).

Encoding guide

See docs/ENCODING.md for an overview on how most formats are encoded (including the dynamically sized integer types).

typescript-binary

TypeScript Binary

📣 When to use?

Install

Usage

Define formats

Inferred types

✨ Parsing formats

Use BinaryFormatHandler

Manual handling

💥 Id Collisions

✨ Validation / Transforms

Validation

Transforms

Types

Encoding guide

Readme

Keywords

Package Sidebar

Install

Repository

Homepage

Weekly Downloads

Version

License

Unpacked Size

Total Files

Last publish

Collaborators

typescript-binary

TypeScript Binary

📣 When to use?

Install

Usage

Define formats

Inferred types

✨ Parsing formats

Use BinaryFormatHandler

Manual handling

💥 Id Collisions

✨ Validation / Transforms

Validation

Transforms

Types

Encoding guide

Readme

Keywords

Package Sidebar

Install

Repository

Homepage

DownloadsWeekly Downloads

Version

License

Unpacked Size

Total Files

Last publish

Collaborators

Weekly Downloads