gotsentimental

GoT Twitter Sentiment Analysis

As seen on www.got.show

Installing

$ npm install gotsentimental --save

Dependencies:

• recent node.js + npm
• MongoDB

Usage

You need to create a Twitter API key for the crawler.

See example/app for an advanced example.

Example:

const gotsent = require('gotsentimental');
 
// adjust config
gotsent.cfg.extend({
    "mongodb": {
        "uri": "mongodb://example/gotsentimental"
    },
    "twitter": {
        "access_token": "xxx",
        "access_token_secret": "xxx",
        "consumer_key": "xxx",
        "consumer_secret": "xxx"
    }
});
 
// initialize
gotsent.init();
 
// update DB - this might take a few hours
gotsent.update().then(function(res) {
    // print some update stats
    console.log(res);
 
    // get top5 most popular characters
    gotsent.mostPopular(5).then(function(res) {
        res.forEach(function(character) {
            console.log(character.name);
        });
    }, console.error);
 
    // gracefully shut down
    gotsent.shutdown();
}, function(err) {
    console.error(err);
    gotsent.shutdown();
});

API

Generated CSV files

The crawler generates static CSV files in the directory set in the config ("csvpath"). These files have to be made available, e.g. by using express.static(__dirname + '/csv');. See example/app for an example.

The following files are generated and required by chart.js:

Episodes:

/csv/episodes.csv

Data per Character:

/csv/Character_Name.csv               (complete overview, grouped per day)
/csv/Character_Name/2016-03.csv       (monthly overview, grouped per hour)

Types

Character

Name	Type	Description
name	string	name of the character
slug	string	human-readale URL-identifier for the character
total	number	total number of tweets in database
positive	number	total number of positive tweets in database
negative	number	total number of negative tweets in database
heat	number	how controverse is the character
popularity	number	how much is the character is discussed
updated	Date	date when the document was last updated

Methods and Attributes

• .cfg : Object
• .cfg.extend(json)
• .css : string
• .js : string
• .init()
• .shutdown()
• .update([full]) ⇒ Promise.<Object>
• .updateCharacter(name, [full]) ⇒ Promise.<Object>
• .startUpdateLoop()
• .stopUpdateLoop() ⇒ Promise
• .character(name) ⇒ Promise.<Character>
• .mostPopular([n]) ⇒ Promise.<Array.<Character>>
• .mostHated([n]) ⇒ Promise.<Array.<Character>>
• .mostDiscussed([n]) ⇒ Promise.<Array.<Character>>
• .stats() ⇒ Promise.<Object>

gotsentimental.cfg : Object

Object containing the package configuration. The config can be changed by directly overwriting attributes or using .cfg.extend(json). The default values are stored in defaults.json. Every value can be overwritten.

gotsentimental.cfg.extend(json)

Merges the given Object into the config by overwriting attributes. Arrays are concatenated.

Param	Type	Description
json	Object	Config Object

gotsentimental.init()

Initilaize the package. Opens the MongoDB connection and initializes the Twitter client.

gotsentimental.shutdown()

Close any open resources like the database connection.

gotsentimental.update([full]) ⇒ Promise.<Object>

Update data by crawling for new tweets and generating new CSV files.

Param	Type	Default	Description
[full]	boolean	false	full rebuild or incremental update

Returns: Promise.<Object> - A promise to the update results.

gotsentimental.updateCharacter(name, [full]) ⇒ Promise.<Object>

Update data for given character by crawling for new tweets and generating new CSV files.

Param	Type	Default	Description
name	string		Name of the character
[full]	boolean	false	full rebuild or incremental update

Returns: Promise.<Object> - A promise to the update results.

gotsentimental.startUpdateLoop()

Start the update loop. Waits the amount of secunds set in the config after completing one iteration before starting the next incremental update.

gotsentimental.stopUpdateLoop() ⇒ Promise

Waits for the current update to complete, if one is running.

Returns: Promise - A promise which resolves when the loop is stopped.

gotsentimental.character(name) ⇒ Promise.<Character>

Get a character by name.

Returns: Promise.<Character> - A promise to the character.

Param	Type	Description
name	string	Name of the character

gotsentimental.mostPopular([n]) ⇒ Promise.<Array.<Character>>

Get the most popular characters.

Returns: Promise.<Array.<Character>> - A promise to the array of characters

Param	Type	Default	Description
[n]	number	10	Number of Characters to return

gotsentimental.mostHated([n]) ⇒ Promise.<Array.<Character>>

Get the most hated characters.

Returns: Promise.<Array.<Character>> - A promise to the array of characters

Param	Type	Default	Description
[n]	number	10	Number of characters to return

gotsentimental.mostDiscussed([n]) ⇒ Promise.<Array.<Character>>

Get the most discussed characters.

Returns: Promise.<Array.<Character>> - A promise to the array of characters

Param	Type	Default	Description
[n]	number	10	Number of Characters to return

gotsentimental.css : string

Absolute path to the Chart CSS file. It should be served with e.g. express' sendFile.

gotsentimental.js : string

Absolute path to the Chart JS file. It should be served with e.g. express' sendFile.

gotsentimental.stats() ⇒ Promise.<Object>

Get stats about tweets in database. The returned Object has the following attributes:

• total (total number of tweets),
• positive (total number of positive tweets),
• negative (total number of negative tweets).

Returns: Promise.<Object> - A promise to the stats Object

Testing

Install Gulp:

npm install -g gulp

npm test

Hook up npm and git

To run npm test automatically before every git commit, install a git pre-commit hook:

npm run hookup

git aborts the commit if the tests fail. You can (but shouldn't) bypass it with git commit --no-verify ....