krabber
TypeScript icon, indicating that this package has built-in type declarations

1.1.7 • Public • Published

krabber

Easy to use webscraper using Axios and JSDOM.

Codecov branch Gitlab pipeline status (branch) Libraries.io dependency status for latest release npm

Example

import { Scrape } from 'krabber';
 
const scrapeConfig = {
    url: 'https://github.com/axios/axios',
    options: {
        headers: { 'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:64.0) Gecko/20100101 Firefox/64.0' }
    },
    select: {
        numberOfCommits: ({ dom, res, prev, url }) => {
            return dom.querySelector('.commits span.text-emphasized.num').innerHTML.trim();
        }
    }
}
 
Scrape(scrapeConfig)
    .then((res) => {
        console.log(res.numberOfCommits);
    })

More examples, including pagination handling

Please see the test suite in src/lib/scrape.spec.ts

Package Sidebar

Install

npm i krabber

Weekly Downloads

1

Version

1.1.7

License

MIT

Unpacked Size

19.5 kB

Total Files

14

Last publish

Collaborators

  • ledgit