Search results
15 packages found
Very straightforward, event driven web crawler. Features a flexible queue interface and a basic cache mechanism with extensible backend.
Very straightforward, event driven web crawler. Features a flexible queue interface and a basic cache mechanism with extensible backend.
MongoDB queue implementation for simplecrawler
Very straightforward, event driven web crawler. Features a flexible queue interface and a basic cache mechanism with extensible backend.
SQLite FetchQueue Implementation for Simplecrawler
Forked version of https://github.com/ciffard/node-simplecrawler.
A web crawler forked from simplecrawler.
A web crawler forked from simplecrawler.
Priority based Semantic Web Crawler.
- semantic-crawler
- pdf-crawler
- text-crawler
- priority
- priority-crawler
- scraper
- crawling
- spider
- scraping
- simplecrawler
- crawler
- osmosis
- js-crawler
- supercrawler
- View more
Very straigntforward web crawler. Uses EventEmitter. Generates queue statistics and has a basic cache mechanism with extensible backend. This version is forked in order to add the ability to filter by referrer url
Very straightforward web crawler. Uses EventEmitter. Based on Simplecrawler but using a distributed queue system.
- simple
- crawler
- spider
- cache
- queue
- simplecrawler
- eventemitter
- spiderhunt
- distributed
- redis
- rabbitmq
- rabbit
- scalable
iSpider is a simple and fast web spider written with Nodejs!
web-spider is a simple and fast web spider written with Nodejs!
MongoDB queue for Node Simple Crawler
MongoDB FetchQueue Implementation for Simplecrawler