Playwright Selector Finder

A powerful tool for finding and interacting with elements using Playwright's vision locators. This package makes web automation more resilient by using visual recognition and natural language descriptions.

Features

👁️ Vision-based element location
🗣️ Natural language element descriptions
⚡ Simple and intuitive API
🎯 Configurable vision settings
🔄 Common browser automation actions
📝 Full TypeScript support

Installation

npm install playwright-selector-finder @playwright/test playwright

Usage

import MCPServer from 'playwright-selector-finder';

async function example() {
  // Create a new instance
  const server = new MCPServer({
    headless: false,
    visionTimeout: 5000,
    visionConfidence: 0.7
  });

  try {
    // Start the server
    await server.start();

    // Navigate to a website
    await server.callTool('browser_navigate', {
      url: 'https://example.com'
    });

    // Click using natural language description
    await server.callTool('browser_click', {
      element: 'Login button in the top right corner'
    });

    // Type text
    await server.callTool('browser_type', {
      element: 'Username field',
      text: 'myusername',
      submit: true
    });

  } finally {
    // Always close the server
    await server.stop();
  }
}

Available Tools

Navigation

browser_navigate: Navigate to a URL
browser_go_back: Go back to previous page
browser_go_forward: Go forward to next page

Interactions

browser_click: Click on an element
browser_hover: Hover over an element
browser_type: Type text into an element

Keyboard

browser_press_key: Press a keyboard key

Utilities

browser_wait: Wait for specified time
browser_save_as_pdf: Save page as PDF
browser_close: Close the browser

Configuration Options

interface MCPServerOptions {
  headless?: boolean;        // Run in headless mode
  visionTimeout?: number;    // Vision locator timeout (ms)
  visionConfidence?: number; // Vision confidence threshold (0-1)
}

Using with Test Frameworks

Jest Example

import MCPServer from 'playwright-selector-finder';

describe('Website Tests', () => {
  let server: MCPServer;

  beforeAll(async () => {
    server = new MCPServer({ headless: true });
    await server.start();
  });

  afterAll(async () => {
    await server.stop();
  });

  test('can login', async () => {
    await server.callTool('browser_navigate', {
      url: 'https://example.com'
    });

    await server.callTool('browser_click', {
      element: 'Login button'
    });
  });
});

Best Practices

Always use try/finally to ensure server cleanup
Configure appropriate timeouts for your use case
Use descriptive element descriptions
Handle errors appropriately
Consider using headless mode in CI/CD

Requirements

Node.js >= 18.0.0
Playwright >= 1.42.0

License

Apache-2.0 - see LICENSE for more details.

Playwright MCP

A Model Context Protocol (MCP) server that provides browser automation capabilities using Playwright. This server enables LLMs to interact with web pages through structured accessibility snapshots, bypassing the need for screenshots or visually-tuned models.

Key Features

Fast and lightweight: Uses Playwright's accessibility tree, not pixel-based input.
LLM-friendly: No vision models needed, operates purely on structured data.
Deterministic tool application: Avoids ambiguity common with screenshot-based approaches.

Use Cases

Web navigation and form-filling
Data extraction from structured content
Automated testing driven by LLMs
General-purpose browser interaction for agents

Example config

{
  "mcpServers": {
    "playwright": {
      "command": "npx",
      "args": [
        "@playwright/mcp@latest"
      ]
    }
  }
}

Running headless browser (Browser without GUI).

This mode is useful for background or batch operations.

{
  "mcpServers": {
    "playwright": {
      "command": "npx",
      "args": [
        "@playwright/mcp@latest",
        "--headless"
      ]
    }
  }
}

Running headed browser on Linux w/o DISPLAY

When running headed browser on system w/o display or from worker processes of the IDEs, you can run Playwright in a client-server manner. You'll run the Playwright server from environment with the DISPLAY

npx playwright run-server

And then in MCP config, add following to the env:

{
  "mcpServers": {
    "playwright": {
      "command": "npx",
      "args": [
        "@playwright/mcp@latest"
      ],
      "env": {
        // Use the endpoint from the output of the server above.
        "PLAYWRIGHT_WS_ENDPOINT": "ws://localhost:<port>/"
      }
    }
  }
}

Tool Modes

The tools are available in two modes:

Snapshot Mode (default): Uses accessibility snapshots for better performance and reliability
Vision Mode: Uses screenshots for visual-based interactions

To use Vision Mode, add the --vision flag when starting the server:

{
  "mcpServers": {
    "playwright": {
      "command": "npx",
      "args": [
        "@playwright/mcp@latest",
        "--vision"
      ]
    }
  }
}

Vision Mode works best with the computer use models that are able to interact with elements using X Y coordinate space, based on the provided screenshot.

Snapshot Mode

The Playwright MCP provides a set of tools for browser automation. Here are all available tools:

browser_navigate
- Description: Navigate to a URL
- Parameters:
  - url (string): The URL to navigate to
browser_go_back
- Description: Go back to the previous page
- Parameters: None
browser_go_forward
- Description: Go forward to the next page
- Parameters: None
browser_click
- Description: Perform click on a web page
- Parameters:
  - element (string): Human-readable element description used to obtain permission to interact with the element
  - ref (string): Exact target element reference from the page snapshot
browser_hover
- Description: Hover over element on page
- Parameters:
  - element (string): Human-readable element description used to obtain permission to interact with the element
  - ref (string): Exact target element reference from the page snapshot
browser_drag
- Description: Perform drag and drop between two elements
- Parameters:
  - startElement (string): Human-readable source element description used to obtain permission to interact with the element
  - startRef (string): Exact source element reference from the page snapshot
  - endElement (string): Human-readable target element description used to obtain permission to interact with the element
  - endRef (string): Exact target element reference from the page snapshot
browser_type
- Description: Type text into editable element
- Parameters:
  - element (string): Human-readable element description used to obtain permission to interact with the element
  - ref (string): Exact target element reference from the page snapshot
  - text (string): Text to type into the element
  - submit (boolean): Whether to submit entered text (press Enter after)
browser_press_key
- Description: Press a key on the keyboard
- Parameters:
  - key (string): Name of the key to press or a character to generate, such as ArrowLeft or a
browser_snapshot
- Description: Capture accessibility snapshot of the current page (better than screenshot)
- Parameters: None
browser_save_as_pdf
- Description: Save page as PDF
- Parameters: None
browser_wait
- Description: Wait for a specified time in seconds
- Parameters:
  - time (number): The time to wait in seconds (capped at 10 seconds)
browser_close
- Description: Close the page
- Parameters: None

Vision Mode

Vision Mode provides tools for visual-based interactions using screenshots. Here are all available tools:

browser_navigate
- Description: Navigate to a URL
- Parameters:
  - url (string): The URL to navigate to
browser_go_back
- Description: Go back to the previous page
- Parameters: None
browser_go_forward
- Description: Go forward to the next page
- Parameters: None
browser_screenshot
- Description: Capture screenshot of the current page
- Parameters: None
browser_move_mouse
- Description: Move mouse to specified coordinates
- Parameters:
  - x (number): X coordinate
  - y (number): Y coordinate
browser_click
- Description: Click at specified coordinates
- Parameters:
  - x (number): X coordinate to click at
  - y (number): Y coordinate to click at
browser_drag
- Description: Perform drag and drop operation
- Parameters:
  - startX (number): Start X coordinate
  - startY (number): Start Y coordinate
  - endX (number): End X coordinate
  - endY (number): End Y coordinate
browser_type
- Description: Type text at specified coordinates
- Parameters:
  - text (string): Text to type
  - submit (boolean): Whether to submit entered text (press Enter after)
browser_press_key
- Description: Press a key on the keyboard
- Parameters:
  - key (string): Name of the key to press or a character to generate, such as ArrowLeft or a
browser_save_as_pdf
- Description: Save page as PDF
- Parameters: None
browser_wait
- Description: Wait for a specified time in seconds
- Parameters:
  - time (number): The time to wait in seconds (capped at 10 seconds)
browser_close
- Description: Close the page
- Parameters: None