comm node

0. About

The goal of this package is to create microservices with the least amount of code regarding the setup/networking part. The configuration (network, db) of a service is declared in a file.

In the scope of this package, a network is composed of a central node (message router) and multiple service nodes. Each service node is a micro-service, and its behaviour is defined by a handler class, that contains the methods that the service will be able to process, and their logic. The service nodes can communicate between them, but only through the central node.

Each service can be called synchronously, or asynchronously. In the first case, the service will keep the connection with the client querying it, and reply with the result. In the second case, the query to the service is put into a queue, processed a message at a time, and a new message with the reply is sent back, through the central node.

1. Installation

npm i comm-node

2. Configuration

2.1 Main configuration

# ./node_config.yaml
nodes:
  central: # name of the node
    type: central # node type : central/service
    identifier: central # unique identifier
    address: # address of the web server that will receive the messages
      host: localhost
      port: 3001
    db: # message storage configuration
      <Message storage configuration (see 2.2) >

2.2 Message storage configuration

Persistent messages with a database connexion

# ./node_config.yaml
nodes:
  central:
    db:
      type: postgres # for now, only postgres is available
      host: localhost
      port: 5432
      username: postgres
      password: pass
      database: message_db

Example of a docker postgres configuration

# ./docker-compose.yml
db:
  image: postgres
  environment:
    POSTGRES_USER: postgres
    POSTGRES_PASSWORD: pass
    POSTGRES_DB: message_db
  ports:
    - 5432:5432

Non-persistent messages (the messages queue is stored in a run-time variable)

# ./node_config.yaml
nodes:
  central:
    db:
      type: mock

2.3 Client node specific configuration

# ./node_config.yaml
nodes:
  greeter:
    centralNodeAddress:
      host: localhost
      port: 3001

3. Usage

3.1 Central node

import {ComCentralNode} from 'comm-node'

ComCentralNode.constructFromEnvFile('central')
    .setup()
    .then((node: ComCentralNode) => node.start())

There should be only one central node in a network. The node configuration is loaded from the configuration file. The node name is passed to constructFromEnvFile(). A central node is only responsible for receiving, routing and sending messages.

3.2 Client nodes

3.2.1 Overview

import {ClientNodeConfigParser, ComServiceNode, MockHandler} from 'comm-node'

let clientNodeConfig = ClientNodeConfigParser.loadFromEnvFile('greeter')
let handlerInstance = new MockHandler(clientNodeConfig)
new ComServiceNode(clientNodeConfig, handlerInstance)
    .setup()
    .then((node: ComServiceNode) => node.start())

There can be as many client nodes as needed in a network, and they need to have a different name and identifier. The node name is passed to constructFromEnvFile(). Each node is built around a handler instance. This object dictates what functions the node can respond to.

3.2.2 Custom Handlers

3.2.2.a. Overview

import {
    ClientNodeConfigParser, ComServiceNode, BaseHandlerInstance,
} from 'comm-node'

class CustomHandler extends BaseHandlerInstance {
    public async sayHi(): Promise<string> {
        return 'hi'
    }
}

let clientNodeConfig = ClientNodeConfigParser.loadFromEnvFile('greeter')
let handlerInstance = new CustomHandler(clientNodeConfig)
new ComServiceNode(clientNodeConfig, handlerInstance)
    .setup()
    .then((node: ComServiceNode) => node.start())

Any handler must extend the class BaseHandlerInstance, and the config must be passed to it when it is instantiated.

3.2.2.b. Function parameters

import {BaseHandlerInstance} from 'comm-node'

class CalculatorHandler extends BaseHandlerInstance {
    public async sum(a: number, b: number): Promise<number> {
        return a + b
    }
}

There is at this time no way to check the type of parameters ar run time, or to have optional parameters

3.2.2.c. Call another client

import {BaseHandlerInstance} from 'comm-node'

class CustomHandler extends BaseHandlerInstance {
    public async doStuffAndCall(callerName: string, ...internalArgs: any[]): Promise<void> {
        doSomething()
        this.triggerFunctionCall('greeter', 'greet', {name: callerName}, internalArgs)
        return
    }
}

Use this.triggerFunctionCall (inherited from BaseHandlerInstance ) to call another client node. The parameters are :

• node name
• function name
• parameters
• internalArgs (optional)

The parameter ...internalArgs: any[] is an optional argument and can be omitted from the function declaration and when calling triggerFunctionCall, but is used to link messages together (see 5.1)

3.2.2.d. Trigger an event

import {BaseHandlerInstance} from 'comm-node'

class DataFetchHandler extends BaseHandlerInstance {
    public async fetchData(...internalArgs: any[]): Promise<void> {
        const foundData = doSomething()
        this.triggerEvent('data.fetched', {data: foundData}, internalArgs)
        return
    }
}

Use this.triggerEvent (inherited from BaseHandlerInstance ) to trigger an event. The parameters are :

• event name
• parameters
• internalArgs (optional)

The parameter ...internalArgs: any[] is an optional argument and can be omitted from the function declaration and when calling triggerEvent, but is used to link messages together (see 5.1)

3.3 Event listeners

clientNodeConfig.addEventListener('bell.ring', handlerInstance.ring)

To add an event listener to a node, use the addEventListener function of the config object with the event name, and the method to call.

4. Complete example

Ping/Pong

In this example, two service nodes will communicate indefinitely between them. The first node pinger will trigger a ping event when its sendPing() method is called. The second node, ponger will call the sendPing() method from pinger when a ping event is triggered.

              /-pingEvent-->  [ central node ]   --replyToPing()---\
[ pinger ] --/                                                       \-> [ ponger ]
           <-\                                                       /--
              \-sendPing()--  [ central node ]  <-pinger.sendPing()-/

Main code :

// app.ts

import {
    ComCentralNode, BaseHandler,
    ClientNodeConfigParser, ComClientNode,
} from 'comm-node'

/* First Handler : trigger a ping event when sendPing() called */
class PingHandler extends BaseHandler {

    // _start() is called on node startup
    public async _start(): Promise<this> {
        await this._wait(1000) // _wait() is a method from BaseHandler
        this.sendPing(0)
        return this
    }

    public async sendPing(count: number): Promise<void> {
        await this._wait(250)
        console.log('ping #' + count)
        this._triggerEvent('ping', {count: count})
    }
}

/* Second Handler : calls sendPing() from the first node when a ping event is triggered */
class PongHandler extends BaseHandler {

    // _registerEvents() is called on node setup, events observation needs to be declared here
    public _registerEvents(): this {
        super._registerEvent('ping', this.replyToPing)
        return this
    }

    public async replyToPing(count: number): Promise<void> {
        await this._wait(250)
        console.log('pong #' + count)
        this._triggerFunctionCall('pinger', 'sendPing', {count: count + 1})
    }
}

/* Service nodes instantation from configuration */
let pingerConfig = ClientNodeConfigParser.loadFromEnvFile('pinger')
let pongerConfig = ClientNodeConfigParser.loadFromEnvFile('ponger')
let pingHandler = new PingHandler(pingerConfig)
let pongHandler = new PongHandler(pongerConfig)

/* Central node instantation from configuration and start */
ComCentralNode.constructFromEnvFile('central')
    .setup()
    .then((node: ComCentralNode) => node.start())

/* Service nodes start */
new ComClientNode(pingerConfig, pingHandler)
    .setup()
    .then((node: ComClientNode) => node.start())
new ComClientNode(pongerConfig, pongHandler)
    .setup()
    .then((node: ComClientNode) => node.start())

Configuration file :

nodes:
  central:
    type: central
    identifier: central
    address:
      host: localhost
      port: 30010
    db:
      type: mock
  pinger:
    type: client
    identifier: pinger
    address:
      host: localhost
      port: 30020
    db:
      type: mock
    centralNodeAddress:
      host: localhost
      port: 30010
  ponger:
    type: client
    identifier: ponger
    address:
      host: localhost
      port: 30021
    db:
      type: mock
    centralNodeAddress:
      host: localhost
      port: 30010

Expected output

  2023-12-09 16:13:00.548	INFO		SYS central	 node is running 1.15.0 (central)  # central node started
  2023-12-09 16:13:00.552	INFO		SYS central	 web server running on port 30010

  2023-12-09 16:13:00.576	INFO		SYS central	 registered node <pinger> <localhost:30020> # pinger node registered
  2023-12-09 16:13:00.585	INFO		SYS pinger	 config registered to central node
  2023-12-09 16:13:00.586	INFO		SYS pinger	 node is running 1.15.0 (pinger) # pinger node started
  2023-12-09 16:13:00.587	INFO		SYS pinger	 web server running on port 30020

  2023-12-09 16:13:00.589	INFO		SYS central	 registered node <ponger> <localhost:30021> # ponger node registered
  2023-12-09 16:13:00.592	INFO		SYS ponger	 config registered to central node
  2023-12-09 16:13:00.593	INFO		SYS ponger	 node is running 1.15.0 (ponger) # ponger node started
  2023-12-09 16:13:00.593	INFO		SYS ponger	 web server running on port 30021
  ping #0
  2023-12-09 16:13:04.746	INFO		MSG pinger	 a23d0c74-4c4b-4a18-865f-fd91879cd8fd	 SND EVT OK # pinger sends event (ping)
  2023-12-09 16:13:08.681	INFO		MSG central	 a23d0c74-4c4b-4a18-865f-fd91879cd8fd	 RCV EVT OK # central node receives it
  2023-12-09 16:13:08.889	INFO		MSG central	 2c67d831-f6f3-4925-8f2b-f0b684921f1c	 SND FCL OK # send function call (replyToPing)
  pong #0
  2023-12-09 16:13:12.999	INFO		MSG ponger	 2c67d831-f6f3-4925-8f2b-f0b684921f1c	 RCV FCL OK # function call received by ponger
  2023-12-09 16:13:13.207	INFO		MSG ponger	 a2b693b3-bbf8-451e-b571-05e6ce36e7b5	 SND FCL OK # ponger sends function call (sendPing)
  2023-12-09 16:13:13.242	INFO		MSG central	 a2b693b3-bbf8-451e-b571-05e6ce36e7b5	 RCV FCL OK # central node receives it
  2023-12-09 16:13:13.414	INFO		MSG ponger	 aaf6be74-57f8-4e57-88ff-46d73e10950c	 SND RES OK # ponger sends response message in reply of the function call
  2023-12-09 16:13:13.448	INFO		MSG central	 aaf15858-522d-433a-b31c-f567ef12d9c3	 SND FCL OK # central node sends function call (sendPing)
  2023-12-09 16:13:13.649	INFO		MSG central	 aaf6be74-57f8-4e57-88ff-46d73e10950c	 RCV RES OK # central node receives the response message from ponger (aaf6...)
  ping #1
  2023-12-09 16:13:17.351	INFO		MSG pinger	 aaf15858-522d-433a-b31c-f567ef12d9c3	 RCV FCL OK # pinger receiveds the sendPing() call from central
  2023-12-09 16:13:17.558	INFO		MSG pinger	 0208db52-aefe-42e9-9eca-4a775caa9577	 SND EVT OK # pinger sends event ping
  2023-12-09 16:13:17.763	INFO		MSG pinger	 6004c948-859a-4423-b0b8-f95503f42d19	 SND RES OK # pinger sends response message in reply of the function call
  2023-12-09 16:13:18.001	INFO		MSG central	 0208db52-aefe-42e9-9eca-4a775caa9577	 RCV EVT OK # central receives the event, and will trigger the function call to ponger etc...
  ...

5. Technical information

5.1. Messages

The information sent to and between nodes is sent in messages that are structured like this :

Function call / Event trigger messages

{
  "uuid": "50181820-13a4-4016-bdc2-cef831db4c0e",
  "origin_uuid": "10447723-7d51-4a1e-b4fc-83f2bfb5ee0a",
  "parent_uuid": "04d31eab-3e13-4aa8-9aa1-7e6fc1457844",
  "type": "functionCall",
  "source": "bruno",
  "destination": "my-service",
  "content": {
    "name": "parseResource",
    "params": {
      "a": 1,
      "b": "value"
    }
  },
  "libVersion": "1.15.0"
}

• uuid : unique identifier, uuidv4
• origin_uuid : optionnal, uuid of the first message in the response chain
• parent_uuid : optionnal, uuid of the parent message in the response chain
• type : any between functionCall, eventTrigger
• source : the name of the node that sent the message
• destination : name of the service that should receive this message
• content :
  • name : name of the event/function that is called
  • params : object, the keys are the name of the function parameters (in the handler class of the service node)
• libVersion : version of comm-node for the service that sent the message. If major/minor version numbers do not match, the message will be refused

Response messages

{
  "uuid": "50181820-13a4-4016-bdc2-cef831db4c0e",
  "origin_uuid": "10447723-7d51-4a1e-b4fc-83f2bfb5ee0a",
  "parent_uuid": "04d31eab-3e13-4aa8-9aa1-7e6fc1457844",
  "response_to_id": "f3ca7f50-74f7-4405-8825-a624e34ac604",
  "type": "response",
  "source": "my-service",
  "destination": "central",
  "content": {
    "code": 200,
    "response": null,
    "success": true
  },
  "libVersion": "1.15.0"
}

• uuid : unique identifier, uuidv4
• origin_uuid : optionnal, uuid of the first message in the response chain
• parent_uuid : optionnal, uuid of the parent message in the response chain
• response_to_id : required only for response messages, uuid of the message that this one responds to
• type : response
• source : the name of the node that sent the message
• destination : name of the service that should receive this message
• content :
  • code : code of the response, in the same idea as HTML status codes
  • response : can be any type
  • success : true/false
• libVersion : version of comm-node for the service that sent the message. If major/minor version numbers do not match, the message will be refused

5.2 Routes

5.2.1 Common routes

Method	Path	Payload	Notes
POST	/api/direct	see 5.1	Send a synchronous message, the response will be sent directly to this request
POST	/api/queue	see 5.1	Send an asynchronous message that will be queued, a response message will be sent once the message has been processed
GET	/api/queue	-	Show the content of the queue

5.2.2 Central node specific routes

Method	Path	Payload	Notes
GET	/api/config/nodes	-	Show the list of registered¹ nodes
DELETE	/api/config/nodes/{nodeName}	-	Unregister¹ a specific node
POST	/api/config/nodes	see below	Register¹ a new node

¹ - Messages received by the central nodes will be refused unless the node is registered beforehand. A service node will automatically send a registration call on start to the central node.

More info and payload for node registration

Payload for nodes that can only send messages to other nodes :

{
  "name": "bruno",
  "canReceiveMessages": false,
  "libVersion": "1.15.0"
}

• name : name of the service, unique on the network. This is the value in the source/destination field of messages
• canReceiveMessages : false
• libVersion : version of comm-node for the service that is registering itself. If major/minor version numbers do not match, the message will be refused

Payload for nodes that can send and receive messages :

{
  "name": "my-service",
  "canReceiveMessages": true,
  "libVersion": "1.15.0",
  "address": {
    "host": "localhost",
    "port": 3004
  },
  "serverConfiguration": {},
  "events": [
    {
      "eventName": "resource.fetched",
      "functionName": "processResourceFetched"
    }
  ]
}

• name : name of the service, unique on the network. This is the value in the source/destination field of messages
• canReceiveMessages : true
• libVersion : version of comm-node for the service that is registering itself. If major/minor version numbers do not match, the message will be refused
• address :
  • host : web server host of the node
  • port : web server port of the node
• serverConfiguration : if the service can receive sync/async messages
• events: list of events that the service will respond to
  • eventName : name of the event
  • functionName : name of the function that will be called by the central server on the service if this event is triggered