testarmada-performance-reporter

Magellan extension for calculating performance metrics

This repo is an extension of Magellan framework to display/report performance metrics for pages as the E2E tests are running. We have made use of browser's Navigation Timing to get basic metrics on the pages or any url. Here is how the metrics will display :

Running 1 tests with 3 workers with firefox

--> Worker 1, mock port: 12000, running test: tests/walmart-book.js @firefox 
(1 / 1) <-- Worker 1 PASS  tests/walmart-book.js @firefox  

============= Suite Complete =============

     Status: PASSED
    Runtime: 14.6s
Total tests: 1
 Successful: 1 / 1

---------------------------------------
Gathered performance metrics: 
---------------------------------------

url: http://www.walmart.com/search/?query=sam%20walton%20made%20in%20america
metrics:
[
  "DomContentLoad: 0.585 seconds",
  "Page Load (onLoad): 5.639 seconds",
  "Full Page Load: 3.912 seconds",
  "Number of Requests: 94",
  "First Paint: 0 seconds"
]

What's in the Box

Included in this boilerplate:

• Command to get performance metrics
• Example base class for tests (lib/example-base-test-class.js)
• Example tests which use the example base class (tests/*)
• A .gitignore file which ignores Magellan-generated artifacts (logs, etc)
• Stock configuration (conf/nightwatch.json)
• A place to put custom commands (lib/custom_commands)

Prerequisites

Check that you have at least npm version 2.7.1:

npm --version

If not, run:

npm install -g npm

Installation Example

1. Clone magellan-boilerplate into an isolated place outside of your project:

cd ~
git clone [e-mail]:TestArmada/boilerplate.git

2. Copy the contents of boilerplate into your project test folder, remove .git

cd ~/myproject
mkdir automated-tests
cd automated-tests
cp -R ~/boilerplate ./
rm -rf .git

3. Install npm modules and run example tests

npm install
npm test

4. Try the testing workflow

To run the tests included in this boiler plate, simply type:

./node_modules/.bin/magellan --serial

If you already have ./node_modules/.bin in your PATH, you can simply type this instead:

magellan --serial

without ./ or any path prefix.

In the above example, the --serial option runs all of your tests one at a time with live output. To get help on command options, type:

magellan --help

For more information on how to run tests, including in different browsers, in parallel, and with filters and tags, see: https://github.com/TestArmada/magellan

5. Modify example tests and base class

To start developing your own tests, look at tests/* and lib/example-base-test-class.js.

Notes

• Magellan may support multiple test frameworks, but this boilerplate is based on Nightwatch.js (by using the magellan-nightwatch adapter).
• This boilerplate project is suitable for use with tests that run against a live server, a QA server, or even a mock (i.e. any use case Magellan supports).

testarmada-performance-reporter