Raft Consensus Algorithm over Coaty in TypeScript/JavaScript
Table of Contents
Overview
This project contains a TypeScript implementation of the Raft Consensus Algorithm. Raft is a protocol with which a cluster of nodes can maintain a replicated state machine. The state machine is kept in sync through the use of a replicated log. It was originally proposed in "In Search of an Understandable Consensus Algorithm" by Diego Ongaro and John Ousterhout.
Many different implementations of this library are in existence, though none of them that were originally developed for TypeScript/Javascript are being actively maintained. The most popular implementation of the Raft algorithm can be found inside the etcd project, which is written entirely in Go. The authors of this library have decided to use the etcd Raft implementation to create a custom port from Go to TypeScript.
This library includes the ported code together with an additional layer on top to provide better programmability to the user. The additional layer handles aspects like persistency, communication between nodes, cluster configuration as well as client interaction with reproposed inputs.
The additional programmability layer uses the Coaty Framework for communication between nodes as well as some aspects of persistency and cluster configuration which requires Node.js JavaScript runtime (version 14 LTS or higher).
This project comes with complete documentation of its public API including all public and protected type and member definitions.
This project includes a complete example that demonstrates best practices and typical usage patterns of the library.
Installation
You can install the latest version of this library in your application as follows:
npm install @coaty/consensus.raftThis npm package uses ECMAScript version es2019 and module format commonjs.
Getting started
A RaftStateMachine stores some form of state and defines inputs that can be
used to modify this state. Our library replicates an implementation of the
RaftStateMachine interface on multiple nodes in a cluster using Raft. This
means that each node has their own RaftStateMachine instance and applies the
same inputs in the same order to replicate the same shared state. We therefore
end up with one consistent state that can be accessed by all nodes in the
cluster.
Defining a custom replicated state machine
The first step of getting started is to implement your own RaftStateMachine.
The implementation will depend on your use case. In this example we will create
a simple key value store that stores string key value pairs. Objects of type
KVSet and KVDelete will later be used as state machine inputs. getState()
and setState() should serialize and deserialize the stored state. For further
info have a look at the API
documentation
of the RaftStateMachine interface. The implementation of the string key value
store looks like this:
class KVStateMachine implements RaftStateMachine {
private _state = new Map<string, string>();
processInput(input: RaftData) {
if (input.type === "set") {
// Add or update key value pair
this._state.set(input.key, input.value);
} else if (input.type === "delete") {
// Delete key value pair
this._state.delete(input.key);
}
}
getState(): RaftData {
// Convert from Map to JSON compatible object
return Array.from(this._state);
}
setState(state: RaftData): void {
// Convert from JSON compatible object to Map
this._state = new Map(state);
}
}
// Use an input object of this type to set or update a key value pair
type KVSet = { type: "set"; key: string; value: string; }
// Use an input object of this type to delete a key value pair
type KVDelete = { type: "delete"; key: string }Bootstrapping a new node
After you have implemented the RaftStateMachine interface you're ready to
bootstrap your first node. What you will end up with is a running
RaftController that can connect() to the Raft cluster and propose() inputs
of type KVSet and KVDelete as well as read the state with getState().
You will need to specify a couple of options when bootstrapping a new
RaftController. Have a look at the API
documentation
of the RaftControllerOptions interface for further info. The following code
will start a new controller, that will create a new Raft cluster on connect():
// URL to the MQTT broker used by Coaty for the communication between nodes
const brokerUrl = "mqtt://localhost:1883";
// Path to the database file for persistent storage that will be created
const databaseFilePath = "raft-agent-1.db"
// Id that uniquely identifies the node in the cluster
const id = "1";
// Instance of your RaftStateMachine implementation
const stateMachine = new KVStateMachine();
// Create a new cluster on connect
const shouldCreateCluster = true;
// RaftControllerOptions
const controllerOptions: RaftControllerOptions = { id, stateMachine, shouldCreateCluster };
const components: Components = {
controllers: {
RaftController
}
};
const configuration: Configuration = {
communication: {
brokerUrl: brokerUrl,
shouldAutoStart: true,
},
controllers: {
RaftController: controllerOptions
},
databases: {
// Database key to persist Raft data must equal the one specified in
// RaftControllerOptions.databaseKey (if not specified, defaults to "raftdb").
// Database adapter must support local database operations, e.g. "SqLiteNodeAdapter".
// Ensure path to database file exists and is accessible.
raftdb: {
adapter: "SqLiteNodeAdapter",
connectionString: databaseFilePath,
},
},
};
const container = Container.resolve(components, configuration);
// Represents the new node and can be used to access and modify the replicated state
const raftController = container.getController<RaftController>("RaftController");Note: The code above doesn't define the RaftControllerOptions.cluster
property. It therefore defaults to the empty string. Define this property if you
want to start multiple different clusters.
Accessing and modifying the replicated state
Once you have bootstrapped the RaftController you can use it as follows:
// Connect the new node to the Raft cluster
await raftController.connect();
// Read the current replicated state
const currentState = await raftController.getState();
console.log("Current state: %s", JSON.stringify(currentState));
// > Current state: []
// Read the current cluster configuration
const currentConfiguration = await raftController.getClusterConfiguration();
console.log("Current configuration: %s", JSON.stringify(currentConfiguration));
// > Current configuration: ["1"]
// Subscribe to replicated state updates
const observable1 = raftController.observeState();
observable1.subscribe((state) => console.log("New state: %s", JSON.stringify(state)));
// > New state: [["meaning of life","42"]]
// > New state: [["meaning of life","42"],["coaty","io"]]
// > New state: [["coaty","io"]]
// To modify the replicated state use KVSet and KVDelete inputs
const setInput1: KVSet = { type: "set", key: "meaning of life", value: "42" };
const setInput2: KVSet = { type: "set", key: "coaty", value: "io" };
const deleteInput: KVDelete = { type: "delete", key: "meaning of life" };
const resultingState1 = await raftController.propose(setInput1);
console.log("Resulting state: %s", JSON.stringify(resultingState1))
// > Resulting state: [["meaning of life","42"]]
const resultingState2 = await raftController.propose(setInput2);
console.log("Resulting state: %s", JSON.stringify(resultingState2))
// > Resulting state: [["meaning of life","42"],["coaty","io"]]
const resultingState3 = await raftController.propose(deleteInput);
console.log("Resulting state: %s", JSON.stringify(resultingState3))
// > Resulting state: [["coaty","io"]]
// Gracefully stop the node without disconnecting from the Raft cluster
await raftController.stop();
// Reconnect
await raftController.connect();
// Disconnect the node from the Raft cluster before shutting down
await raftController.disconnect();Logging
This package supports logging by use of the npm
debug package. Logging output can be
filtered by specifying the DEBUG environment variable on startup, like this:
# Logs all supported log levels: INFO, WARNING, ERROR, DEBUG
DEBUG="consensus.raft:*"
# Logs ERROR logs only
DEBUG="consensus.raft:ERROR"
# Logs all supported log levels except INFO
DEBUG="consensus.raft:*,-consensus.raft:INFO"Contributing
If you like this package, please consider ★ starring the project on GitHub. Contributions are welcome and appreciated. If you wish to contribute please follow the Coaty developer guidelines described here.
License
Non-ported code and documentation copyright 2023 Siemens AG. Ported code and documentation copyright 2016 The etcd Authors. @nodeguy/channel code and documentation copyright 2017 David Braun.
Non-ported code is licensed under the Apache 2.0 license.
Non-ported documentation is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
@nodeguy/channel code and documentation is licensed under the Apache 2.0 license.
Credits
Last but certainly not least, a big Thank You! to the folks who designed, implemented and contributed to this library:
- Łukasz Zalewski @lukasz-zet
- Leonard Giglhuber @LeonardGiglhuber
- Andreas Dachsberger
- Finn Capelle
- Felix Elfering