TestRail Reporter for posting TestRun for Cypress based on Post true OR False flag
![version] ![downloads] ![MIT License]
Publishes Cypress runs on TestRail if the Flag "postToTestrail" is set to "true"
Cloned from the original package cypress-testrail-reporter by Milutin Savovic
Core features:
- Allows you to set statuses for test cases in a specific TestRun.
- Multi project support (provide
testrail-projects.json
config file in pathcypress/fixtures/
following format explained below) and generate test reports coverage for each project during time execution.
Important note: Keep your tests under the one of the following folder structure:
-
cypress/e2e
- default path for Cypress v10 and above -
cypress/integration
- default path for Cypress v9 and bellow cypress/tests
Install
$ npm i testrail-reporter-with-flag-for-cypress
$ yarn add testrail-reporter-with-flag-for-cypress
Usage
Add reporter to your cypress.json
:
...
{
"reporter": "testrail-reporter-with-flag-for-cypress",
"reporterOptions": {
"host": "https://yourdomain.testrail.io",
"username": "username",
"password": "password / API token",
"postToTestrail": true OR false,
"projectId": idNumber,
"runId": testRunNumber,
"jsonConfig": true OR false,
}
Your Cypress tests should include the ID of your TestRail test case. Make sure your test case IDs are distinct from your test titles:
// Good:
it("C123 C124 Can authenticate a valid user", ...
it("Can authenticate a valid user C321", ...
// Bad:
it("C123Can authenticate a valid user", ...
it("Can authenticate a valid userC123", ...
Reporter Options
-
host: string domain name of your TestRail instance (e.g. for a hosted instance https://instance.testrail.com).
-
username: string email of the user under which the test run will be created.
-
password: string password or the API key for the aforementioned user.
-
postToTestrail: boolean true to post results to sepcified run or false to not post any resluts
-
projectId: number project with which the tests are associated.
-
runId: number a specific test run id number.
-
jsonConfig: boolean true to run with multiple projects and you must provide
testrail-projects.json
in path/cypress/fixtures/
or false to skip this functionality.
Your testrail-projects.json
file should look like this format:
{
"data": [{
"projects": [{
"projectName": "App Lifecycle (Contact : Bumblebee)",
"projectId": projectIdNumber,
"runId": runIdNumber,
"milestoneId": milestoneIdNumber OR null,
"reportIds": [
reportIdNumber01,
reportIdNumber02,
reportIdNumber03
],
"reportCoverage": true OR false,
"postToTestrail": true OR false,
"specs_name": [
"cypress/e2e/app-life-cycle/**/*"
]
},
{
"projectName": "Course Builder",
"projectId": projectIdNumber,
"runId": runIdNumber,
"milestoneId": milestoneIdNumber OR null,
"reportIds": [
reportIdNumber01
],
"reportCoverage": true OR false,
"postToTestrail": true OR false,
"specs_name": [
"cypress/e2e/course-builder/tenant-experience/**/*",
"cypress/e2e/thinkific/student-experience/student-certificates/**/*",
"cypress/e2e/site-scripts/**/*"
]
},
{
"projectName": "Open ID Connect SSO into Thinkific (Contact: Reboot)",
"projectId": projectIdNumber,
"runId": runIdNumber,
"milestoneId": milestoneIdNumber OR null,
"reportIds": [],
"reportCoverage": true OR false,
"postToTestrail": true OR false,
"specs_name": [
"cypress/e2e/oidc/**/*",
"cypress/e2e/thinkific/tenant-experience/general/create-tenant.spec.js"
]
}
]}
]
}
Note: if you set jsonConfig: true
the package will ignore postToTestrail
, projectId
and runId
options as these values should be provided in testrail-projects.json
TestRail Settings
To generate test coverage reports, the testrail community recommends 3 report types:
-
Coverage for References (Cases) report. Follow these instructions to set up this report and set the report id(s) value(s) in
testrail-projects.json
. - Summary for References (Defects) report. Follow these instructions to set up this report and and set the report id(s) value(s) in
testrail-projects.json
. - Comparison for References (Results) report. Follow these instructions to set up this report and set the report id(s) value(s) in
testrail-projects.json
.
Remember to mark On demand via the API
and Everyone (with project access)
on section Access and Scheduling when creating the reports, so they can be created during run execution.
Also, reportId
can be found at the url of the report after being created:
Note: if you set "reportCoverage": true
for each specific project in your testrail-projects.json
the package will pick up the values inside reportIds
to generate those reports automatically in TestRail, otherwise if you set "reportCoverage": false
those reports generations will be skipped.
To increase security, the TestRail team suggests using an API key instead of a password. You can see how to generate an API key here.
If you maintain your own TestRail instance on your own server, it is recommended to enable HTTPS for your TestRail installation.
For TestRail hosted accounts maintained by Gurock, all accounts will automatically use HTTPS.
You can read the whole TestRail documentation here.
Author
Author: Shruti Rathee - github
License
This project is licensed under the MIT license.
Acknowledgments
- Milutin Savovic, author of the cypress-testrail-reporter repository that was cloned.
- Pierre Awaragi, owner of the mocha-testrail-reporter repository that was forked.