Bespoken DataDog Plugin
This plugin makes it easy to send your voice app's end-to-end test results to a DataDog instance for reporting and monitoring.
It leverages Bespoken's filters to report test results to DataDog.
Getting Started
Installation and Usage
To use the Bespoken Datadog Plugin, just include it in your test project dependencies with:
npm install bespoken-datadog-plugin --save
Then include it as your filter like this in the testing.json:
{
"filter": "./node_modules/bespoken-datadog-plugin/index.js"
}
Alternatively, you can call it from an existing filter like this:
const DatadogPlugin = require('bespoken-datadog-plugin')
module.exports = {
onTestEnd: async (test, testResult) => {
await DatadogPlugin.sendToDataDog(test, testResult)
},
onTestSuiteEnd: async(testResults) => {
await DatadogPlugin.sendSuiteResultsToDataDog(testResults);
},
}
Environment Variables
Variable | Description | Default |
---|---|---|
DATADOG_API_KEY | API key to access DATADOG | |
DATADOG_CUSTOMER | Customer Tag for data points in DATADOG | bespoken |
DATADOG_JOB_NAME | JobName Tag for data points in DATADOG | EndToEndTests |
DATADOG_RUN_NAME | RunName Tag for data point in DATADOG | Current date in ISO format |
The DATADOG_API_KEY must be set as an environment variable.
For local running, we typically recommend using the package dotenv
and setting it in a file labeled .env
. We have provided a file example.env you can use as a template (just copy and rename to .env).
It can also be set manually like so:
export DATADOG_API_KEY=<DATA_DOG_API_KEY>
(Use set
instead of export
if using Windows Command Prompt).
DataDog Configuration
- Create a DataDog account.
- Take the API key from the Integrations -> API section
- Add it to the
.env
file
DataDog Metrics
DataDog captures metrics related to how all the tests have performed. Each time we run the tests, we push the result of each test to DataDog.
We use next metrics:
utterance.success
utterance.failure
test.success
test.failure
testsuite.success_percentage
testsuite.failure_percentage
testsuite.skipped_percentage
The metrics can be easily reported on through a DataDog Dashboard. They also can be used to set up notifications when certain conditions are triggered.
Read more about configuring DataDog in our walkthrough.
DataDog Tags
By default we report the results at utterance, test and test suite level:
Reporting Element | Tag | Description |
---|---|---|
Utterance & Test & TestSuite | jobName | The name of the testing job for the current execution (defaults to EndToEndTests ). You can define it here. |
Utterance & Test & TestSuite | runName | The name of the test execution. By default, it is the timestamp when the test was executed. |
Utterance & Test & TestSuite | testSuiteName | The name of the test suite we are executing. In general, it is the YAML file name containing the test cases. |
Utterance & Test | testName | The test description. Usually written after the test: keyword of the test script. |
Utterance | customer | The name of the customer running the test scripts. You can change it here. |
Utterance | utterance | The interaction sent to the voice app. |
Utterance | voiceId | The voice used to do TTS when sending the text utterance to the voice service. We can use Amazon Polly or Google Wavenet voices. |