stealthy-scraper

Extra stealthy web scraper in Typescript

Overview

stealthy-scraper is a wrapper around puppeteer-extra that adds additional stealth functionality and other helpful features.

Why to use

• If puppeteer's Page.goto and Browser.newPage is being detected. stealthy-scraper has a newBrowser function as an alternative way to navigate to a new url which is more reliable.
• If puppeteer's default word typing is being detected. stealthy-scraper has a safeType function that better mimicks human typing behavior.
• When you want to more neatly centralize all of the puppeteer, puppeteer-extra, and puppeteer-extra's plugin dependencies into one package.

Usage Overview

npm i --save stealthy-scraper

import { createScraper } from 'stealthy-scraper'
const scraper = await createScraper({
  puppeteerOptions: {
    headless: true,
    ...
  },
  snapshotsDirPath: './scraper-snapshots',
})
await scraper.page.goto('difficultoscrape.com')
const searchTextInput = await scraper.page.waitForSelector('...')
await scraper.safeType(searchTextInput, 'my search term')
// ...
await scraper.newBrowser(newUrlFromSearchResults)
await scraper.close()

Development

See ./contributing/development.md

Disclaimer

I do not condone the usage of this package for malevolent purposes. Please be very curtious and a good citizen when using it. I do not take any responsibility for any damages you incur on yourself (e.g. IP blacklisted) or others (e.g. DoS) through any use of this package.

---

If you found this package delightful, feel free to buy me a coffee ✨