turkish-columnist-crawler

Welcome to Turkish Columnist Crawler 👋

Crawl the columns on newspaper websites.

🏠 Homepage

Install

npm i turkish-columnist-crawler

Usage

This is a simple use case. See the Documentation section for details.

const crawler = require('turkish-columnist-crawler')

let urls = SAMPLE_URL

crawler(url, {
    date: '23 Nisan 1920', // Activate if you want to scan articles written after this date.
    last: false, // Activate only if you want the last article.
    externalParams: null, //Allows you to add things that need to be added to the return data.
    saveDisk: true, //Saves outputs to disk
    limit: -1, //Determines the maximum count of articles to be scanned. If you give -1, it scans a maximum of 100 articles.
    filePath: __dirname + '/texts', // If you set the saveDisk parameter to true, you have to give the entire Path of the folder where the files will be saved.
    strOp: false // If you use this parameter, you will see that there are spaces between all punctuation and words.
})

Supported Newspapers

1. www.milliyet.com.tr
2. www.hurriyet.com.tr
3. www.sozcu.com.tr
4. www.haberturk.com
5. www.evrensel.net
6. www.yenicaggazetesi.com.tr
7. www.turkiyegazetesi.com.tr
8. www.fanatik.com.tr
9. www.takvim.com.tr
10. www.sabah.com.tr

Author

👤 Omer Hamid Kamisli

• Website: ohkamisli.com
• Twitter: @ohkamisli
• Github: @ohkamisli
• LinkedIn: @ohkamisli

🤝 Contributing

Contributions, issues and feature requests are welcome!
Feel free to check issues page.

Show your support

Give a ⭐️ if this project helped you!

📝 License

Copyright © 2020 Omer Hamid Kamisli.
This project is MIT licensed.

---

This README was generated with ❤️ by readme-md-generator