@sagebots/mermaid

Mermaid

Mermaid is full-featured Chatbot messaging server for rapid development & consistent cross-platform user experiences.

var mermaid = require('@sagebots/mermaid');

mermaid();

Installation

$ npm install @sage-bots/mermaid

Overview

Defining a bot

We define bots using pure configuration files.

The Metadata is a declarative JSON Psuedo-Language.

Each stage in the workflow is defined using one .json document.

These are defined in the project root's meta folder.

All files in the meta directory are traversed and read at runtime and put into a JavaScript object hash between uri -> object.

Each property in each .json file is used to collectively define the behavior of the interaction.

Here's an explanation of the 10 base properties:

1. uri - This is the ID (String) for the interaction.

2. info - This is an array of string or objects that is the leading texts leading up to the prompt.

3. prompt - The prompt is a special text and should ultimately be the message that the user is going to be responding to. It's special functionality is it will repeat itself if the user's input is invalid.

4. next-uri - The next-uri parameter defines the next stage in the workflow.

5. type - This is possibly the most important property. This defines the template that this stage will use.

6. query - You can make an API call to another REST service and use the data from the response as contextual data for the current stage.

7. storage-property - This is a template-specific property, specifically for the 'store-information' template chosen. This defines where to store the information received in the response.

8. validators - This is an array of objects defining what type of validation this stage needs to run the responses through to maintain high data integrity. The validator objects also have an optional error.message property that defines the error response.

9. before-hooks - Can setup custom functionality before the stage.

10. after-hooks - Can setup custom functionality after the stage.

And here's an example of most of those properties used in one example document:

The following example is extremely verbose, and not realistic, but aims to show all the different types of properties / functionality you can leverage in one data document.

{
    "uri": "/intro/end-working",
    "prompt" : "When do you end working?",
    "next-uri" : "/intro/end",
    "query": {
        "endpoint": "<%=external_service_url%>/api/v1/cart",
        "method": "POST",
        "data": {
            "vendor_code": "<%=user.session.cart.vendor_code%>",
            "qty": "<%=user.session.cart.qty%>",
            "item_name": "<%=user.session.cart.item_name%>",
            "item_price": "<%=user.session.cart.item_price%>",
            "discount_amount": "<%=user.session.cart.discount_amount%>",
            "tax_rate": "<%=user.session.cart.tax_rate%>"
        },
        "store": {
            "session.cart.description": {
                "template": "<%=description%>"
            },
            "session.cart.tax": {
                "template": "<%=tax_amount%>"
            },
            "session.cart.tax_formatted": {
                "template": "<%=tax_amount%>",
                "formatters": ["CentsToDollars"]
            }
        }
    },
    "memory": {
        "receipt": {
            "name": {
                "template": "<%=user.first_name%> <%=user.last_name%>"
            },
            "title": {
                "field": "user.session.cart.item_name"
            },
            "subtitle": {
                "field": "user.session.cart.description"
            },
            "image_url": {
                "field": "user.session.cart.image_url"
            },
            "quantity": {
                "field": "user.session.cart.qty"
            },
            "item_price": {
                "field": "user.session.cart.item_price",
                "formatters": ["CentsToDollars"]
            },
            "price": {
                "field": "user.session.cart.total_formatted"
            }
        }
    },
    "type": "store-information",
    "storage-property": "session.end-working",
    "validators": [{
        "type": "time",
        "options": {
            "error": {
                "message" : "Sorry, i don't understand your entry, please tell me the time you start working in the format AM/PM format, for example 09:30am"
            }
        }
    }],
    "after-hooks": [{
        "type": "set-active"
    }]
}

Extending the Metadata

Proprietary Type System

Each type has 4 functions that it needs to define:

1. getMessages - This function is a message data pre-processor

2. getPatternCatcher - This function is an array of response handlers with REGEX matching.

3. getURIForResponse - This function determines what endpoint to go to next.

4. getEnd - This function is a response post-processor

These four functions make up the behavior for every particular stage in the workflow.

Validators

• Quickly add custom validation for incoming messages
• Error message handling
• The best place to define new validators is in the root of the project; in a ${PROJECT_HOME}/validators directory.

Hooks

• Quickly add code / functionality before or after a stage.

Botflow Engine

• Route-based - Using a http-based routing system to make it easy to manage state and traverse the graph.

• Stateful - the botflow engine is designed similarly to a workflow engine that performs some type of interaction based on a state change.

• Is designed as a tree that starts from a single root node, and has linked lists (in the form of JSON documents with pointers driven from the next-uri property)

• This allows us to defined a conversation, or a series of steps / interactions in a simple way.

• Using this data structure, we can build systems that generate these documents and link them on the fly. Otherwise known as "training" the bots brain.

Web Server with Full-Featured REST API

• The foundation of the mermaid framework is an Feathers server which wraps Express under the hood.

• We use feathers in order to quickly spin up a REST API to easily integrate with other systems, including our own UI (Dolphin)

• The REST API can be found at [PROTOCOL]:[HOSTNAME]:[PORT]/api/v1/{{resource}}

• For example for a local install on port 3000:

  • You can view user data here: http://localhost:3000/v1/users
  • You can view messages data here: http://localhost:3000/v1/messages

• Additionally, you can also use all the HTTP verbs:

  • GET
  • POST
  • PUT
  • DELETE

Cross-platform Messenger

• Seamless message handling support for downstream platforms.

Universal Commands

• e.g: "start", "restart", "goto", "back", "quit", "help"

Tests

A number of tests have been added under /tests.

Types of Test

Unit Tests

See /tests/unit for unit tests have been written to cover the workflow validators.

Functional Tests

See /tests/functional for functional tests that cover various workflows, bot memory storage, and restarting mid-conversation.

Load Test

See /tests/load for a rudimentary test that to mimics a number of users connecting at once and having differing conversations.

Warning!

There is a bug in Mermaid that prevents the load test from passing successfully (see limitations).

Methodology

We generate a number of test users (TOTAL_USERS) which will converse with a single instance of the chatbot. Each user is instantiated with its own response server so we can identify if the bot is sending the correct messages back to the correct user. Each user will traverse the entire test conversation, with a limited number of concurrent users (MAX_CONCURRENT_USERS) sending messages simultaneously.

We track how long the bot takes to respond at each step and display the statistics at the end if the test completes successfully. You can set the thresholds for slow and failed response times by changing the constants RESPONSE_THRESHOLD_SLOW and RESPONSE_THRESHOLD_FAIL respectively.

Limitations

1. At the time of writing Mermaid appears to have a bug with the Web API which we use for communicating with the bot whilst testing. This bug prevents us testing with multiple concurrent users because the bot will send the reply to the wrong user causing the test to abort. For this test to run correctly this Mermaid bug needs to be fixed. You can see more information on this (and a test case) in the /tests/load/bug directory.

2. Time constraints mean we are using one response server per test user which does not scale well. This should be refactored to only require a single test sever for all the users to reduce memory usage and the chance of finding a port that is already in used by another process.

Configuring the Load Test

There are several constants in the load test spec file which can be altered to simulate different loads:

• TOTAL_USERS - The total number of test users to generate. Each test user will have a different conversation with the bot.
• MAX_CONCURRENT_USERS - The maximum number of users who will be having a conversation with the bot simultaneously.
• RESPONSE_TIMEOUT - The maximum number of milliseconds to wait for the bot to respond to any message from any user. The test will be terminated if the bot takes this long to reply.
• RESPONSE_THRESHOLD_SLOW - Used for the output statistics if the test completes successfully. If the bot takes this long to respond the request will be marked as "slow".
• RESPONSE_THRESHOLD_FAIL - Used for the output statistics if the test completes successfully. If the bot takes this long to respond the request will be marked as a "fail".

How to Run the Tests

Scripts have been added to the package file to allow quick execution of the various tests.

Please Note: Some of the functional tests (storageBot and restartedBot) require a REAL MongoDB server to be running or they will fail.

• To run all the tests at once: npm run test.
• To run just the unit tests: npm run test-unit.
• To run just the functional tests: npm run test-functional.
• To run just the load test: npm run test-load.

You can also run an individual test using Mocha directly, assuming your terminal is open in the root Mermaid directory: mocha ./tests/functional/workflowBot.js.

Debugging Tests

If you need to debug what the chatbot is doing when running a test you can set the constant OUTPUT_BOT_LOGS to true in any of the test spec files.

Unfortunately errors are not currently handled correctly by Mermaid so it may be necessary to turn this on in order to find out why a test is failing. If Mermaid encounters an error it will most likely continue running and may (or quite often not) output something to the console. The test will simply see this as a request timing out.

Include Files

There is some common functionality used by all the tests which can be found in the /tests/includes directory.

• databaseUtilities.js - Utilities to help us work with a REAL MongoDB server instance for the functional tests that require this instead of Mockgoose.
• testChildBot.js - A class that handles instantiating a chatbot app as a child process and waiting for it to become ready for communication.
• testServer.js - A class that handles sending messages to a chatbot instance, waiting on replies from the bot, and handling timeouts, etc.

Each file is well commented, and there are examples in the functional and load tests on how to use the various methods.