QualWeb core

The core allows you to perform automatic accessibility evaluations on web pages. By itself, core does not contain any evaluation modules, but are added separately. The monorepo contains several modules that we maintain:

• @qualweb/act-rules
• @qualweb/wcag-techniques
• @qualweb/best-practices
• @qualweb/cui-checks
• @qualweb/counter

You can also perform evaluations at http://qualweb.di.fc.ul.pt/evaluator/, or by installing the chrome extension.

How to install

As a base example, here's how to install core and the ACT rules module. The procedure for all modules is generally the same (but do check their READMEs for specifics):

  $ npm i --save @qualweb/core @qualweb/act-rules

How to run

import { QualWeb } from '@qualweb/core';
import { ACTRules } from '@qualweb/act-rules';

async function main() {
  // First, initialize QualWeb. In this step, you can also enable the use of
  // some Puppeteer plugins that may help in the evaluation process.
  const qualweb = new QualWeb({
    // Check https://github.com/berstend/puppeteer-extra/tree/master/packages/puppeteer-extra-plugin-adblocker
    adBlock: true, // Default value = false
    // Check https://github.com/berstend/puppeteer-extra/tree/master/packages/puppeteer-extra-plugin-stealth
    stealth: true // Default value = false
  });

  // Also create an instance of the evaluation module(s) you want to run.
  const actRulesModule = new ACTRules({
    // See @qualweb/act-rules for options.
  });

  // Then, start the QualWeb core proper. This starts puppeteer and makes core
  // ready to run evaluations.

  await qualweb.start(
    {
      // Concurrently runs more evaluations. Default = 1
      maxConcurrency: 5,
      // Timeout for reaching remote URLs. Default = 30 seconds.
      timeout: 60 * 1000,
      // Displays diagnost information in stdout/terminal. Default = false
      monitor: true,
    },
    {
      // Disable headless mode (i.e. show a browser window) for evaluations.
      // By default, the browser window is not visible when running.
      headless: false,
    },
  );

  const urlToEvaluate = 'https://www.w3.org/WAI/standards-guidelines/act/rules/';

  // QualWeb stores resulting reports in a POJO, keyed by the URL.
  const reports = qualweb.evaluate({
    url: urlToEvaluate,
    // Set the modules to be run here. Each module will be run once for each URL
    // that QualWeb visits. In this example, just `urlToEvaluate`.
    modules: [
      actRulesModule,
    ],
  });

  const urlReport = reports[urlToEvaluate];

  console.debug(urlReport.metadata);

  // Remember to stop QualWeb once you're done. This closes the Puppeteer
  // instance.
  await qualweb.stop();
}

EARL reports

The package @qualweb/earl-reporter can convert a report from QualWeb's own format to the Evaluation And Reporting Language report format:

import { generateEARLReport } from '@qualweb/earl-reporter';

// Reports that were previously generated by QualWeb.evaluate()
const reports = { /** ... */ };

const earlReports = generateEARLReport(reports);

// EARL reports are also keyed by the evaluated URL.

Plugins

QualWeb allows for middleware-style plugins to run at set times during evaluation. Use this if you need to set special values prior to loading a page or right before the evaluation itself runs, such as authorization cookies to access a page that requires login.

import { QualWeb } from '@qualweb/core';

async function main() {
  const qw = new QualWeb();

  let cookies: any = {};

  qw.use({
    beforePageLoad(page, url) {
      // This code runs after a new Puppeteer page has been created but before
      // it navigates to the target URL.
    },
    afterPageLoad(page, url) {
      // This code is called right after a page has been loaded by Puppeteer
      // but before QualWeb runs.

      cookies = await page.cookies();
    },
  });

  await qw.start();

  const reports = await qw.evaluate({
    url: 'https://www.google.com',
  });

  // reports will now contain the evaluation for google.com while cookies will
  // contain any cookies that were stored as a result of navigating to the URL.

  await qw.stop();
}

The available options fot the evaluate() function are documented here

Report details

In this section it's explained the evaluation report in detail. For a detailed version of the EARL report check @qualweb/earl-reporter.

  {
    "type": "evaluation",
    "system": {
      "name": "QualWeb",
      "description": "QualWeb is an automatic accessibility evaluator for webpages.",
      "version": "QualWeb version",
      "homepage": "http://www.qualweb.di.fc.ul.pt/",
      "date": "date of the evaluation",
      "hash": "unique hash",
      "url": {
        "inputUrl": "inserted url",
        "protocol": "protocol of the url",
        "domainName": "domain name of the url",
        "domain": "domain of the url",
        "uri": "uri of the url",
        "completeUrl": "complete url after all redirects"
      },
      "page": {
        "viewport": {
          "mobile": "was evaluated on a mobile context or not",
          "landscape": "was evaluated on a landscape context or not",
          "userAgent": "user agent used",
          "resolution": {
            "width": "window's width used",
            "height": "window's height used",
          }
        },
        "dom": {
          "html": "html code as a string",
          "title": "Title of the webpage",
          "elementCount": "Element count of the webpage"
        }
      }
    },
    "metadata": {
      "passed": "number of passed rules/techniques/best practices",
      "warning": "number of warning rules/techniques/best practices",
      "failed": "number of failed rules/techniques/best practices",
      "inapplicable": "number of inapplicable rules/techniques/best practices",
    },
    "modules": {
      "act-rules": {
        "type": "act-rules",
        "metadata": {
          "passed": "number of passed rules",
          "warning": "number of warning rules",
          "failed": "number of failed rules",
          "inapplicable": "number of inapplicable rules",
        },
        "assertions": {
          "QW_ACT_R1": {
            "name": "Name of the rule",
            "code": "QualWeb rule id",
            "mapping": "ACT rule id mapping",
            "description": "Description of the rule",
            "metadata": {
              "target": "Any target, can be one element, multiple elements, attributes, a relation between elements",
              "success-criteria?": [
                {
                  "name": "Name of the success criteria",
                  "level": "Level of conformance of the success criteria",
                  "principle": "Principle of the success criteria",
                  "url": "Url of the success criteria"
                }
              ],
              "related?": [], // related WCAG 2.1 techniques
              "url?": "Url of the rule",
              "passed": "Number of passed results",
              "warning": "Number of warning results",
              "failed": "Number ff failed results",
              "type?": [], // usually "ACTRule" or "TestCase"
              "a11yReq?": [], // WCAG 2.1 relation - something like "WCAG21:language"
              "outcome": "Outcome of the rule",
              "description": "Description of the outcome";
            },
            "results": [
              {
                "verdict": "Outcome of the test",
                "description": "Description of the test",
                "resultCode": "Test identifier",
                "pointer?": "Element pointer in CSS notation",
                "htmlCode?": "Element html code",
                "attributes?": "Attributes of the element",
                "accessibleName?": "Accessible name of the test target"
              },
              { ... }
            ]
          },
          "...": { ... }
        }
      },
      "wcag-techniques": {
        "type": "wcag-techniques",
        "metadata": {
          "passed": "number of passed techniques",
          "warning": "number of warning techniques",
          "failed": "number of failed techniques",
          "inapplicable": "number of inapplicable techniques",
        },
        "assertions": {
          "QW_WCAG_T1": {
            "name": "Name of the technique",
            "code": "QualWeb technique id",
            "mapping": "WCAG techniques code mapping",
            "description": "Description of the technique",
            "metadata": {
              "target": "Any target, can be one element, multiple elements, attributes, a relation between elements",
              "success-criteria?": [
                {
                  "name": "Name of the success criteria",
                  "level": "Level of conformance of the success criteria",
                  "principle": "Principle of the success criteria",
                  "url": "Url of the success criteria"
                }
              ],
              "related?": [], // related WCAG 2.1 techniques
              "url?": "Url of the technique",
              "passed": "Number of passed results",
              "warning": "Number of warning results",
              "failed": "Number ff failed results",
              "type?": [], // usually "ACTRule" or "TestCase"
              "a11yReq?": [], // WCAG 2.1 relation - something like "WCAG21:language"
              "outcome": "Outcome of the technique",
              "description": "Description of the outcome";
            },
            "results": [
              {
                "verdict": "Outcome of the test",
                "description": "Description of the test",
                "resultCode": "Test identifier",
                "pointer?": "Element pointer in CSS notation",
                "htmlCode?": "Element html code",
                "attributes?": "Attributes of the element" // if available
              },
              { ... }
            ]
          },
          "...": { ... }
        }
      },
      "best-practices": {
        "type": "best-practices",
        "metadata": {
          "passed": "number of passed best practices",
          "warning": "number of warning best practices",
          "failed": "number of failed best practices",
          "inapplicable": "number of inapplicable best practices",
        },
        "assertions": {
          "QW_BP1": {
            "name": "Name of the technique",
            "code": "QualWeb best practices id",
            "description": "Description of the best practices",
            "metadata": {
              "target": "Any target, can be one element, multiple elements, attributes, a relation between elements",
              "success-criteria?": [
                {
                  "name": "Name of the success criteria",
                  "level": "Level of conformance of the success criteria",
                  "principle": "Principle of the success criteria",
                  "url": "Url of the success criteria"
                }
              ],
              "related?": [], // related WCAG 2.1 techniques
              "passed": "Number of passed results",
              "warning": "Number of warning results",
              "failed": "Number ff failed results",
              "type?": [], // usually "ACTRule" or "TestCase"
              "a11yReq?": [], // WCAG 2.1 relation - something like "WCAG21:language"
              "outcome": "Outcome of the best practices",
              "description": "Description of the outcome";
            },
            "results": [
              {
                "verdict": "Outcome of the test",
                "description": "Description of the test",
                "resultCode": "Test identifier",
                "pointer?": "Element pointer in CSS notation",
                "htmlCode?": "Element html code",
                "attributes?": "Attributes of the element" // if available
              },
              { ... }
            ]
          },
          "...": { ... }
        }
      },
      "counter": {
        "type": "counter",
        "data": {
          "roles": {
            "button": 2,
            "link": 4,
            ...
          },
          "tags": {
            "div": 10,
            "table": 3,
            ...
          }
        }
      }
    }
  }

Evaluation problems

Sometimes, some webpages fail to evaluate, or the evaluation may take a really long time. Before creating an issue check the error.log file and verify that:

• The URL is correct, and it uses http or https, or www, or both;
• The webpage exists;
• If using https, that the certificate is valid;
  • If you really want to evaluate the page with an invalid certificate, add "--ignore-certificate-errors" to the args in qualweb puppeteer launch options.
• The webpage is not password protected. Consider adding a small plugin to set authorization cookies as an option.
• The webpage is an HTML Document.

License

ISC