sooty

A bit of oofle dust to scrape websites

it useful ★ ★ ★

Features

• Scrape by query selector
• Built on headless chrome

Installation

npm install --save sooty

Dependencies

• NodeJS
• Chrome

Usage

import Sooty from 'sooty';

const scrapper = new Sooty({
  url: 'https://google.com',              // go to google search page
  interactions: {
    search: [
      {                                   // fill out Google search
        fields: { q: 'funny cat videos' }
      },
      {                                   // press "Google Search" button
        click: 'input[name=btnK]',
        waitUntil: 'networkidle0'
      }
    ]
  },
  queries: {
    catVideos: {                          // query google search results
      selector: '.rc .r a',
      requires: ['search']
    }
  }
});
scrapper.run().then(results => {
  results.catVideos.forEach(catVideo => {
    console.log(`${catVideo}\n`);
  });
});

Support

Submit an issue

Screenshots

Contribute a screenshot

Contributing

Review the guidelines for contributing

License

MIT License

Jam Risser © 2018

Changelog

Review the changelog

Credits

• Jam Risser - Author

Support on Beerpay (actually, I drink coffee)

A ridiculous amount of coffee ☕ ☕ ☕ was consumed in the process of building this project.

Add some fuel if you'd like to keep me going!