cypress-slack-reporter
A Slack Reporting tool built for Cypress but should work with any mocha based framework that is using mochawesome
- Slack reporter with integration with CircleCI
- Reports Github/BitBucket Triggering Commit Details
- Reports CirleCI Build Logs / Status / Artefacts
- Reports Test Status & Provides Report Links
- Takes the output of Mochawesome JSON output to determine test result & corresponding slack message
- Provides a URL link to the Test Artefacts (Mochawesome HTML Test Report / Cypress Video & Screenshots)
- Programatically run Cypress via a script file with full report generation and Slack Reporting.
For users who are not using CircleCi, you can get a simple report
- pass
--ci-provider none
provider flag to provide a simple slack message based on the mochawesome report status - Pass
--custom-url
along with--ci-provider custom
to set custom report page url. that will be sent to slack.
For jenkins users
- pass
--ci-provider jenkins
provider flag.
Reporting Features
It provides the following distinct message types
- Build Failure / Cypress error
- Test Failure
- Test Success
It provides the following information
- CircleCI Build Status
- Test Stats (Total Tests / Passes / Failures)
- Author with link to Github commit
- Branch name
- Pull Request number and link to PR (only if PR)
And the following build/test artefacts
- CircleCI Build Log button
- HTML Test report button (only on build success)
- Videos of test runs (one link per test)
- Screenshots of failed tests (one link per failing test)
Screenshots showing Slack Alert Format
Installation
Note Please see the pre-requisites folder to current neccessary pre-requisites
-
Install the app
$ yarn add cypress-slack-reporter --dev
or
$ npm install cypress-slack-reporter --save-dev
-
Create a Slack app & create an incoming webhook
Set the following environment variables in your localhost or CI configuration.
-
SLACK_WEBHOOK_URL
- The full URL you created in the last stepeg.
export SLACK_WEBHOOK_URL=yourWebhookUrlHere
You can optionally set one or more of the following env vars, to utilise a different slack webhook, dependent on the status of your build or test run.
-
SLACK_WEBHOOK_ERROR_URL
- For failing CI runs -
SLACK_WEBHOOK_FAILED_URL
- For failing test runs -
SLACK_WEBHOOK_PASSED_URL
- For passing test runs
Any of the 4 env vars above with accept a comma seperated list of webhooks, if you wish to post your slack message to multiple webhooks (one webhook per channel in slack).
eg. export SLACK_WEBHOOK_URL=your1stWebhookUrlHere,your2ndWebhookUrlHere
Execution
$ npx cypress-slack-reporter --help
Usage: index.ts [options]
Options:
-v, --version output the version number
--vcs-provider [type] VCS Provider [github|bitbucket|none] (default: "github")
--ci-provider [type] CI Provider [circleci|jenkins|bitbucket|none|custom] (default: "circleci")
--custom-url [type] On selected --ci-provider=custom this link will be set to Test Report (default: "")
--report-dir [type] mochawesome json & html test report directory, relative to your package.json (default: "mochareports")
--screenshot-dir [type] cypress screenshot directory, relative to your package.json (default: "cypress/screenshots")
--video-dir [type] cypress video directory, relative to your package.json (default: "cypress/videos")
--verbose show log output
--only-failed only send message for failed tests
--custom-text [type] add additional text to message, wrap message in quotes
-h, --help display help for command
Pre-Requisites
- A test tool capable of utilising mochawesome to report results
- mochawesome for json test result generation
- mochawesome-merge to combine multiple mochawesome reports
- mochawesome-report-generator to generate a HTML report, from your mochawesome json test results
- cypress-multi-reporters to allow you to use multple reporters, in case you require other outputs (junit/spec etc)
Yarn installation Instructions
yarn add mochawesome --dev
yarn add mochawesome-merge --dev
yarn add mochawesome-report-generator --dev
yarn add cypress-multi-reporters --dev
NPM installation Instructions
npm install mochawesome --save-dev
npm install mochawesome-merge --save-dev
npm install mochawesome-report-generator --save-dev
npm install cypress-multi-reporters --save-dev
- Add the following in the base of your project
cypress.json
{
...
"reporter": "cypress-multi-reporters",
"reporterOptions": {
"configFile": "reporterOpts.json"
}
}
reporterOpts.json
{
"reporterEnabled": "mochawesome",
"mochawesomeReporterOptions": {
"reportDir": "cypress/reports/mocha",
"quiet": true,
"overwrite": false,
"html": false,
"json": true
}
}
CircleCI
This project is building in CircleCI and can be viewed at the following link
See the .circleci
folder
-
config.yml
- Contains the CircleCI build configuration
The following env vars are read for CircleCI users.
-
CIRCLE_SHA1
- The SHA1 hash of the last commit of the current build -
CIRCLE_BRANCH
- The name of the Git branch currently being built. -
CIRCLE_USERNAME
- The GitHub or Bitbucket username of the user who triggered the build. -
CIRCLE_BUILD_URL
- The URL for the current build. -
CIRCLE_BUILD_NUM
- The number of the CircleCI build. -
CIRCLE_PULL_REQUEST
- Comma-separated list of URLs of the current build’s associated pull requests. -
CIRCLE_PROJECT_REPONAME
- The name of the repository of the current project. -
CIRCLE_PROJECT_USERNAME
- The GitHub or Bitbucket username of the current project. CI_URL="https://circleci.com/api/v1.1/project"
-
CIRCLE_PROJECT_ID
- This project ID used in artefact URLS
If you wish to use another CI provider, you can pass any name other than circleci
into the CLI flag --ci-provider
, which will allow you to enter your own environment variables for CI.
CI_URL
-
CI_SHA1
, -
CI_BRANCH
, -
CI_USERNAME
, -
CI_BUILD_URL
, -
CI_BUILD_NUM
, -
CI_PULL_REQUEST
, CI_PROJECT_REPONAME
CI_PROJECT_USERNAME
CircleCI Artifact Notes
CircleCI have recently changed the API for retrieving API's. A URL is generated for artifacts in the format
https://${CI_BUILD_NUM}-${CI_PROJECT_ID}-gh.circle-artifacts.com/0
You can get the CIRCLE_PROJECT_ID
by checking https://circleci.com/docs/api/#artifacts-of-a-build
For example. the ID for this project is 177880476
, you can see it in the following URL
https://circleci.com/api/v1.1/project/github/YOU54F/cypress-slack-reporter/1/artifacts
which will return
[ {
"path" : "root/app/mochareports/.gitignore",
"pretty_path" : "root/app/mochareports/.gitignore",
"node_index" : 0,
"url" : "https://1-177880476-gh.circle-artifacts.com/0/root/app/mochareports/.gitignore"
},
...
]
In order to correctly construct your artifact URL, you will need to manually retrieve this ID and set it as an env var titled CIRCLE_PROJECT_ID
EXPORT CIRCLE_PROJECT_ID=177880476
in windows
SET CIRCLE_PROJECT_ID=177880476
or in your CircleCI project's environment page.
There is also another workaround by setting a destination option in store_artifacts job in config.yml (CircleCI).
- store_artifacts:
path: ~/path/to/cypress/videos
destination: cypress/videos
will allow you to access artifacts through https://${CI_BUILD_NUM}-${CI_PROJECT_ID}-gh.circle-artifacts.com/0/cypress/videos/some_test_result.mp4
This is what it says on CircleCI Documentation:
Currently, store_artifacts has two keys: path and destination.
path is a path to the file or directory to be uploaded as artifacts.
destination (Optional) is a prefix added to the artifact paths in the artifacts API. The directory of the file specified in path is used as the default.
Scripted Runner
An example script is here as cli/spec/ts
A example of how you can use this script in your project to:-
- Run Cypress with Mochawesome & junit reporters
- Merge mochawesome reports with
mochawesome-merge
- Construct a slack alert with the merged report, screenshots and videos
Either with the cli
./node_modules/.bin/cypress-slack-reporter-full
Or with your own script
rm -rf ./cypress/reports/mocha && npx ts-node script.ts
It can be called with the following options
interface SlackRunnerOptions {
ciProvider: string;
vcsRoot: string;
reportDir: string;
videoDir: string;
screenshotDir: string;
customUrl?: string;
onlyFailed?: boolean;
}
And will return a Slack IncomingWebhookResult.
TODO
- [x] provide user ability to provide own CI artefact paths
- [ ] typescript s3 uploader scripts and add to CLI
- [x] tsified
- [x] able to run in isolation
- [x] mock aws-sdk s3 upload function
- [x] tests
- [x] retrieve s3 links for test report/artefacts and inject into the slack report
- [x] uploading artefacts to s3
- [ ] add to CLI
- [ ] programatically run
- [ ] Add into main slack-reporter script
- [ ] provide CLI options to provide paths/credentials
- [x] Programatically run
- [x] provide ability to be programatically run via a script
- [x] provide example
- [x] add usage instructions to readme
- [x] test example
- [x] compile
- [x] Migrate Slack mock to seperate module available at npm - slack-mock-typed
- Additional CI providers
- [x] Jenkins
Contributors
- With thanks to mikepsinn for Jenkins support.
- With thanks to samswartz13 for bug fixes
- With thanks to ayushipatel126 for dotenv support.
- With thanks to saikatharryc for custom url support.