scraping-ninja-toolkit
All the goodies you'll ever need to scrape the web
Documentation
In-browser Playground
You can try the library on codesandbox, it uses a cors proxy fetcher to let you grab contents from any website inside your browser.
- CodeSandbox: https://codesandbox.io/s/pkyv3n2xym
Installation
yarn add scraping-ninja-toolkit# or npm i scraping-ninja-toolkit
Features
- All in one package
- Nodejs / Browsers compatibility
- Blazingly fast
- Extensible
Overview
The library is articulated around two main components:
- the
fetcher
let you grab contents from any url, - the
scraper
let you extract data from webpages.
There is also some additional tools like an enhanced axios client.
Quick Example
const fetcher = ; // Fetch the given url and return a page scraperconst page = await fetcher; // Scrape an objectconst quote = page;
<!-- quote -->
Advanced real world example
const fetcher = ;const fs = ; async { // Get categories urls const categories = await fetcher links'.menu-aside__a'; // For each category // => frontend // => backend ... const results = await fetcher; fs;};
Credits
• FB55: his work is the core of this library.
• Matt Mueller and cheerio contributors : A good portion of the code and concepts are copied/derived from the cheerio and x-ray scraper libraries.
License
MIT © 2019 Jimmy Laurent