assistive-playwright-client
Presentation
This package contains a node.js library that extends playwright to allow end-to-end testing of web applications with a screen reader (such as NVDA or JAWS) and checking that the screen reader says what is expected.
This requires two main features that are not natively supported by playwright:
- being able to send keystrokes at a low level so that the screen reader can receive them. This is achieved by using either Virtual Box or QEMU and sending low level events with their API.
- being able to capture the text read by the screen reader. This is achieved by using text-to-socket-engine.
So, assistive-playwright-client allows to easily clone and start a virtual machine (with the vm-providers component) and it provides access to the following functions (through the assistive-playwright-server component that is supposed to be running inside the virtual machine):
- the standard playwright API
- an implementation of the playwright Keyboard and Mouse interfaces that send low-level events to the virtual machine
- an API to access screen-reader messages
Here is a schema describing the architecture of Assistive-Playwright:
Getting started
-
Make sure you have the following software installed on the host machine:
- nodejs
- VirtualBox or QEMU
-
Make sure you have a VirtualBox or QEMU virtual machine properly configured. To configure the virtual machine, you can follow this step-by-step guide. The virtual machine should be configured with:
- The NVDA or JAWS screen reader
- text-to-socket-engine and assistive-playwright-server that are configured to work together, with assistive-playwright-server listening on http port 7779
- A snapshot of the virtual machine should be saved in the running state with all these programs running.
-
Install assistive-playwright-client in your project:
npm install assistive-playwright-client
- Make sure to start
vboxwebsrv
in order to be able to start virtual machines of typevirtualbox
:
vboxwebsrv --authentication null
- Here is an example
gettingStarted.js
file that shows how to useassistive-playwright-client
:
const { createVM } = require("assistive-playwright-client");
(async () => {
console.log("Creating VM...");
const {
chromium /* can be replaced with firefox or webkit */,
screenReader,
calibrateMouse,
keyboard,
vm
} = await createVM({
vmSettings: {
type: "virtualbox",
vm: "win10-chromium-nvda",
snapshot: "nvda"
}
});
try {
console.log("Launching browser...");
const browser = await chromium.launch({ headless: false });
const page = await browser.newPage({ viewport: null });
const mouse = await calibrateMouse(page);
screenReader.on("message", msg => console.log(`sr> ${msg}`));
await page.goto("https://duckduckgo.com/");
await mouse.click(0, 0, { origin: await page.$("input[type=text]") });
await screenReader.waitForMessage("Search the web");
await keyboard.type("assistive-playwright-client");
await keyboard.press("Enter");
await screenReader.waitForMessage(
"assistive playwright client at Duck Duck Go"
);
await keyboard.press("Tab");
await screenReader.waitForMessage("edit");
} finally {
console.log("Destroying VM...");
await vm.destroy();
console.log("Done!");
}
})().catch(error => {
console.log(`Error: ${error}`);
process.exit(1);
});
The API documentation is available here
Note that in order to run tests with a screen reader, instead of directly depending on this package, it is easier and recommended to use the assistive-playwright-test package along with @playwright/test.