Remote web content crawler done right.
Sometimes I want to grab some nice images from a url like http://bbs.005.tv/thread-492392-1-1.html, so I made this little program to combine node-fetch and cheerio to make my attempt fulfilled.
$ npm install --save recrawlerFor Single Page Apps please head to recrawler-nightmare
const recrawler = require('recrawler')
recrawler('http://some-url.com/a/b/c')
.then($ => {
$('img.nice-images').each(function () {
const url = $(this).attr('src')
console.log(url)
})
})cheerio options. Except decodeEntities is false by default here.
MIT © EGOIST