ProtoScript
A Protobuf runtime and code generation tool for JavaScript and TypeScript
What is this? 🧐
ProtoScript is a protocol buffers runtime and code generation tool for JavaScript and TypeScript.
ProtoScript consists of two parts:
-
A runtime. The interface is nearly identical to google-protobuf, but under the hood it is significantly smaller (9.6KB vs 46KB gzipped) and written in ESM to enable dead-code elimination. If you only use the generated JSON serializers, the protobuf runtime will be eliminated from the final build, resulting in an even smaller runtime (less than 1 KB).
-
A code generator. This is a replacement of
protoc
's JavaScript generation that generates idiomatic JavaScript code, JSON serializers/deserializers, and includes TSDoc comments.
If you're looking for an RPC framework, you may be interested in TwirpScript, which uses ProtoScript under the hood to implement Twirp.
Use cases
- Network communication: Protobuf can be used to define APIs and data contracts in a language-independent manner. This makes it easier to share and maintain APIs among different services and platforms. Services written in various programming languages can be autogenerated and exchange data in a language-agnostic manner, using
protobuf
orJSON
as the wire format. - Data storage: store and retrieve structured data in a compact binary format. protobuf's binary format is more space-efficient than JSON, XML and other formats.
Highlights 🛠
-
Idiomatic JavaScript and TypeScript code. ProtoScript generates and consumes plain JavaScript objects over classes. It does not generate any of Java idioms from
protoc --js_out
such as theList
suffix naming for repeated fields,Map
suffix for maps, or the various getter and setter methods. Compare the generated code from ProtoScript's TypeScript example to the protoc example. -
In-editor API documentation. Comments in your
.proto
files become TSDoc comments in the generated code and will show inline documentation in supported editors. -
JSON Serialization/Deserialization. Unlike
protoc
, ProtoScript's code generation also generates JSON serialization and deserialization methods. -
Small. ProtoScript's runtime and generated code are built to support tree shaking to minimize bundle sizes. This results in a significantly smaller bundle size than google-protobuf. ProtoScript's runtime is 67KB (9.6KB gzipped) compared to google-protobuf's 231KB (46KB gzipped). If you only use the generated JSON serializers, the protobuf runtime will be eliminated from the final build, resulting in a substantially smaller runtime (less than 1 KB).
-
Isomorphic. ProtoScript's generated serializers/deserializers can be consumed in the browser or Node.js runtimes.
-
No additional runtime dependencies.
Example
If you have the following protobuf defined in user.proto
:
syntax = "proto3";
message User {
string first_name = 1;
string last_name = 2;
bool active = 3;
User manager = 4;
repeated string locations = 5;
map<string, string> projects = 6;
}
Using the User
generated by ProtoScript will look like this:
import { User, UserJSON } from "./user.pb.js";
const user = {
firstName: "Harry",
lastName: "Potter",
active: true,
locations: ["Hogwarts"],
projects: { DA: "Dumbledore's Army },
manager: {
firstName: "Albus",
lastName: "Dumbledore",
}
}
// protocol buffers
const bytes = User.encode(user);
console.log(bytes); // Uint8Array
const userFromBytes = User.decode(bytes);
console.log(userFromBytes); // our user above
// json
const json = UserJSON.encode(user)
console.log(json); // json string
const userFromJSON = UserJSON.decode(json);
console.log(userFromJSON); // our user above
// ProtoScript generates and consumes plain JavaScript objects (POJOs) over classes. If you want to generate a full message
// with default fields, you can use the #initialize method on the generated message class:
const user = User.initialize();
console.log(user);
// the generated object is just a POJO. You can encode any POJO as protobuf or json as long as it conforms to a subset of the message interface defined in your .proto:
console.log(User.encode(user))
console.log(UserJSON.encode(user))
Installation 📦
-
Install the protocol buffers compiler:
MacOS:
brew install protobuf
Linux:
apt install -y protobuf-compiler
Windows:
choco install protoc
Or install from a precompiled binary.
-
Add this package to your project:
npm install protoscript
oryarn add protoscript
Requirements ⚠️
- Node.js v16 or greater
- TypeScript v4.7 or greater when using TypeScript
Examples 🚀
npx
npx protoscript
cli
protoc \
--plugin=protoc-gen-protoscript=./node_modules/protoscript/compiler.js
--protoscript_out=. \
--protoscript_opt=language=typescript \
Note: Windows users replace ./node_modules/protoscript/compiler.js
above with ./node_modules/protoscript/compiler.cmd
Buf
ProtoScript can be used with Buf.
buf.gen.yaml
version: v1
plugins:
- name: protoc-gen-protoscript
path: ./node_modules/protoscript/dist/compiler.js
out: .
opt:
- language=typescript
strategy: all
Working with other tools
TypeScript
As a design goal, ProtoScript should always work with the strictest TypeScript compiler settings. If your generated ProtoScript files are failing type checking, please open an issue.
Linters
ProtoScript does not make any guarantees for tools like linters and formatters such as prettier or eslint. It generally does not make sense to run these tools against code generation artifacts, like the .pb.ts
or .pb.js
files generated by ProtoScript. This is because:
- The permutations of preferences and plugins for these tools quickly explode beyond what is feasible for a single tool to target. There are always new tools that could be added as well.
- The code is generated automatically, and not all rules are auto fixable. This means there are cases that would always require manual intervention by the user.
- Autogenerated code is readonly, and expected to be correct. Autogenerated code has a much difference maintenance cycle than code written by hand, and should generally be treated as a binary or a dependency. You don't lint your node_modules!
Configuration 🛠
ProtoScript aims to be zero config, but can be configured via the cli interface, or when using the npx protoscript
command, by creating a proto.config.mjs
(or .js
or .cjs
) file in your project root.
Name | Description | Type |
---|---|---|
root |
The root directory. `.proto` files will be searched under this directory, and `proto` import paths will be resolved relative to this directory. ProtoScript will recursively search all subdirectories for `.proto` files.
Defaults to the project root. Example: If we have the following project structure:
Default: A.proto would import "src/B.proto"; Setting // proto.config.mjs /** @type {import('protoscript').Config} */
export default {
root: "src",
}; A.proto would import "B.proto"; TypeScript projects will generally want to set this value to match their |
string (filepath) |
exclude |
An array of patterns that should be skipped when searching for `.proto` files.
Example: If we have the following project structure: /src /foo A.proto /bar B.proto Setting // proto.config.mjs /** @type {import('protoscript').Config} */
export default {
exclude: ["/bar/"]
} Will only process A.proto (B.proto) will be excluded from ProtoScript's code generation. |
string[] (RegExp pattern) |
dest |
The destination folder for generated files.
Defaults to colocating generated files with the corresponding If we have the following project structure:
ProtoScript will generate the following:
Setting // proto.config.mjs /** @type {import('protoscript').Config} */
export default {
dest: "out",
}
Note that the generated directory structure will mirror the Setting // proto.config.mjs /** @type {import('protoscript').Config} */
export default {
root: "src",
dest: "out",
}
|
string (filepath) |
language |
Whether to generate JavaScript or TypeScript.
If omitted, ProtoScript will attempt to autodetect the language by looking for a |
javascript | typescript |
json |
JSON serializer options.
See https://developers.google.com/protocol-buffers/docs/proto3#json for more context. |
{ emitFieldsWithDefaultValues?: boolean, useProtoFieldName?: boolean } |
typecript |
TypeScript options.
|
{ emitDeclarationOnly?: boolean } |
JSON
ProtoScript's JSON serialization/deserialization implements the proto3 specification. If you find a discrepancy, please open an issue.
ProtoScript will serialize JSON keys as lowerCamelCase
versions of the proto field. Per the proto3 spec, the runtime will accept both lowerCamelCase
and the original proto field name when deserializing. You can provide the json_name
field option to specify an alternate key name. When doing so, the runtime will encode JSON messages using the the json_name
as the key, and will decode JSON messages using the json_name
if present, otherwise falling back to the lowerCamelCase
name and finally to the original proto field name.
Contributing 👫
PR's and issues welcomed! For more guidance check out CONTRIBUTING.md
Licensing 📃
See the project's MIT License.