eol-ft-runner

FUNCTIONAL TEST RUNNER (eol-ft-runner)

This framework uses Selenium-webdriver, wd, Appium & CucumberJS to run UI tests written in BDD format. This project is a work in progress. As we build towards v1.0, you can checkout the "Projects" section of GitHub to keep track of our progress.

How to install

npm install eol-ft-runner -D

Browser Support status

Browser	macOS/Linux	Windows
Chrome	✅	✅
Chrome Headless	✅	✅
Firefox	✅	✅
Firefox Headless	✅	✅
Safari	✅	N/A
Edge	✅	✅
Edge Headless	🛠 WIP	🛠 WIP
Android Chrome	✅	✅
iOS Safari	✅	N/A
Internet Explorer	N/A	🛠 WIP
Opera	❌	❌
Brave	❌	❌

Mobile App status

Platform	macOS/Linux	Windows
Android App	✅	✅
iOS App	✅	N/A

Note on browser drivers

Browser drivers are not bundled with this package. You will need to install the browser drivers (Chromedriver or Gecko Driver or Edge Driver) - in your project using the below command:

npm install chromedriver --save-dev

(or)

npm install geckodriver --save-dev

(or)

npm install @sitespeed.io/edgedriver --save-dev

This will allow you to control your browser driver version, based on the browser version you are testing against.

Run GUI test(s) on your local environment:

Run your test with a configuration file:

./node_modules/eol-ft-runner/bin/ft-runner --config config.json

Run your test with CLI arguments:

./node_modules/eol-ft-runner/bin/ft-runner --config config.json --browser chrome --tags @sanity --cores 2

Run headless using --headless (only supported on chrome/firefox):

./node_modules/eol-ft-runner/bin/ft-runner --config config.json --browser chrome --tags @sanity --cores 2 --headless

Run using npm scripts:

npm run test -- --config config.json --browser chrome --tags @sanity --cores 2

In your package.json, the npm script test should point to the ft-runner executable: ./node_modules/eol-ft-runner/bin/ft-runner.

browser can be 'chrome', 'firefox', 'safari', 'edge', 'android', 'ios'.

tags are cucumber tags found on the top of a scenario inside a feature file. Tags are optional, and will execute all scenarios if not provided. To run multiple tags, use --tags "@sanity @smoke @etc".

cores are the number of parallel threads of execution specified in the format "--cores 3" or "--cores 10". Cores are optional, and will run on 2 cores if not specified. For Safari & Edge browsers, cores will be defaulted to 1 due to their respective browser instance restrictions.

Configuration file attributes

Attribute name	Mandatory?	Type	Example
featurePath	Yes	String	"./path/to/feature/dir"
stepDefinitionPath	Yes	String	"./path/to/step_def/dir"
supportFolderPath	Yes	String	"./path/to/support/dir"
reportFolderPath	Yes	String	"./path/to/report/dir"
browser	No	Array of Strings	[ "chrome", "firefox" ]
tags	No	String	"@sanity" "@sanity @smoke"
parallelType	No	String	"features"
rerun	No	Boolean	true
remoteAppiumHub	No	Object	{ "address": "https://www.example-cloud.com/wd/hub", "port": 8081 } port is optional
desiredCapabilities	No	Object	View below sample

SAMPLE CONFIGURATION FILE (click to expand)

{
  "configurations": {
    "browser": [ "chrome" ],
    "tags": "@sanity",
    "featurePath": [ "test/features/" ],
    "stepDefinitionPath": [ "test/step_definitions/" ],
    "supportFolderPath": [ "test/support" ],
    "reportFolderPath": "test/reports",
    "rerun": true,
    "remoteAppiumHub": {
      "address": "https://www.example-cloud.com/wd/hub",
      "port": 8081
    },
    "desiredCapabilities": {
      "chrome": {
        "browserName": "chrome",
        "unhandledPromptBehavior": "accept"
      },
      "ios": {
        "browserName": "Safari",
        "platformName": "iOS",
        "platformVersion": "14.1",
        "deviceName": "iPhone 12",
        "automationName": "XCUITest",
        "startIWDP": true,
        "newCommandTimeout": 30,
        "safariAllowPopups": true
      }
    }
  }
}

Create the config.json file anywhere in your project, and provide its relative path as a command-line argument: --config <relative_path_of_config.json>. For Windows, replace all instances of forward-slashes(/) each with 2 backslashes(\).

CLI arguments

CLI argument	Description	Expected value type	Example
--config /path/to/config.json	Mandatory configuration file that defines the location of feature, step-definition and hook files	String	--config ./config.json
--browser [browsers]	Specify the browser name for the session (can be 'chrome', 'firefox', 'safari', 'edge', 'android', 'ios')	String	--browser chrome --browser chrome firefox (for parallel execution)
--headless	Attempt to run a headless session (applicable for Chrome, Firefox)	Boolean (optional)
--tags <tags>	Provide select cucumber tags to be executed	String	e.g. --tags @sanity e.g. --tags "@smoke @sanity" (for multiple tags)
--parallelType features	Provide this flag to run scenarios from a single feature within the same thread	String	e.g. --parallelType features
--remoteAppiumHub <url>	Provide this flag to override the appium url even if one is provided in the config file	String	e.g. --remoteAppiumHub https://appium.samplehuburl.com/
--addDesiredCaps <desiredCapObject>	Add an entirely new desired capability object or add new attributes to an existing desired capability object	Stringified JSON	e.g. --addDesiredCap "{\"ios\":{\"secretKey\":\"shhd0ntte11any1\"}}"
--rerun	Re-execute all failing tests one more time	Boolean (optional)
--cores <n>	Number of threads in execution during parallel run	Number (default: 2)	--cores 4
--retry <n>	Cucumber-js native retry mechanism to run any failed test scenario(s)	Number	--retry 3
--webhookUrl <url>	Slack webhook url for Slack notifications	String	--webhookUrl https://webhookurl.slack.com/blah/
--ciLinkTitle <title>	Set title of the Slack message forwarded using Slack webhook URL	String	--ciLinkTitle "Build 14"
--ciLinkUrl <url>	Set hyperlink URL for the slack title	String	--ciLinkUrl https://jenkins/url

Important: key-value pairs passed via CLI arguments will override the same key-value pairs provided in the configuration json file. For example, --browser chrome CLI argument will override any browser attribute setting in the configuration file.

Read more:

• Driver class methods
• Differentiating desktop browser and mobile browser tests
• Running tests on Android chrome browser
• Running tests on iOS Safari browser
• Desired capabilities for Android & iOS using Appium
• Debugging tests in VSCode

Sample test project

Sample node project using eol-ft-runner: https://github.com/moneesh2011/test-eol-ft-runner