more convenient scraper for node.js
var scrapm = require('scrapm');
scrapm([options], callback);
options
are request options:
- url - (required) target url you want to scrape
-
blocked - default
false
,true
if the server is blocking scraping by machine. it cheats the server by changhing 'User-Agent' in header. -
encoding - default
false
,true
if the encoding of the webpage is notutf-8
. -
sourceEncoding - default
''
, encoding of the webpage ifoptions.encoding
istrue
-
targetEncoding - default
''
, encoding you want to get ifoptions.encoding
istrue
callback
is called with (err, $)
. You can manipulate DOM with $
as you want. $
is implemented with jsdom.
- iconv
- jsdom
- request