@vostersc/crawler

2.0.9 • Public • Published

Crawler

This package is a wrapper around Puppeteer. The package provides faster out of the box use by taking care of things like proxy urls, ips, user agents, and others.

Quick Start

const C = new Crawler();

const config = [
    {selector: C.exampleActionFunction, urls: ['http://shop.harmonsgrocery.com/search?search_term=peach']},
    {selector: C.exampleActionFunction, urls: ['http://shop.harmonsgrocery.com/search?search_term=avocado']},
    {selector: C.exampleActionFunction, urls: ['http://shop.harmonsgrocery.com/search?search_term=steak']}
];
const anon = false;
const logger = false;
const keepOpen = true;

C.performAction(config, anon, logger, keepOpen); //ignore the await

setTimeout(() => {
    console.log('status', C.getStatus());
}, 30000);

Readme

Keywords

none

Package Sidebar

Install

npm i @vostersc/crawler

Weekly Downloads

19

Version

2.0.9

License

ISC

Unpacked Size

235 kB

Total Files

6

Last publish

Collaborators

  • vostersc