A Limitedly parallel queue for NodeJS
Usage
var LimitingQueue = require('limiting-queue');
var queue = new LimitingQueue(options);
Options
All of these options can be modified after intitalization by setting them ine the queue.opts object.
So, for instance:
queue.opts.maxWorkers = 3;
or, to reset an option back to its default:
delete opts.maxWorkers;
maxWorkers
The maximum numbers of simultaneous workers. An integer greater than zero. Any negative number will be interpreted as "no limit." Defaults to -1.
maxRetries
The maximum number of retries for a queue entry. An integer greater than zero. Any negative number will be interpreted as "no limit." Defaults to -1.
maxWait
The number of milliseconds to wait before declaring a job failed by timeout. An integer greater than zero. Any negative number will be interpreted as "no limit." Defaults to -1.
maxQueueSize
The maximum allowed number of items waiting in the queue. Integer greater than zero. Any negative number will be interpreted as no limit. Defaults to -1.
callback
The function to call to do work on a queue entry. May optionally return a function to be called in case of a timeout (to do any necessary cleanup). Defaults to a function which trivially declares the object complete.
Takes three parameters:
payload
The information previously passed into the "push" function.
previousAttempts
The number of times this payload has previously been pushed to this function.
deferred
The deferred object to declare success or failure on the queue item. Call deferred.fulfill() to
declare success, and deferred.reject(error) to declare a failure.
failure
The function to call when a queue entry flunks out of the queue entirely.
Defaults to an empty function.
Takes three parameters:
payload
The information previously passed into the "push" function.
totalAttempts
The total number of attempts made to process this information.
errors
The array of errors that caused this to error out - in order.
progress
A progress callback to be called every time a new worker starts.
Takes two paramters:
queueSize
The number of jobs in the queue. (Not being worked on.)
workers
The number of workers currently working.
autoStart
Pass true to start the queue immediately, false to let it wait.
retryImmediately
Pass true to push failed jobs to the head of the queue, false to push them to the back.
Methods
queue.push(payload)
Push a new job to the front of the queue. Returns true if the job is added, false otherwise.
queue.append(payload)
Append to the back of the queue. Returns true if the job is added, false otherwise.
queue.prioritize(payload, priority)
Adds an item into the queue after all the items with higher or equal priority, but before any items with lower priority. Default priority is 0. Returns true if the job is added, false otherwise.
It is NOT recommended to mix and match prioritize and push/append, as the first is a relative insertion method, and the latter are absolute. This can lead to unexpected behaviour.
queue.size()
Returns the number of objects waiting in the queue.
queue.workers()
Returns the number of active workers.
queue.start()
Start the queue.
queue.stop()
Stop the queue. This does not cancel working workers - it just ceases the creation of new workers.
queue.forEach()
Execute a function on each member currently in the queue without removing it from the queue. This function is sequential in nature.
Contributing
Please maintain the existing style - but do open a pull request if you have a bugfix or a cool feature. Be sure to document your change. Pull requests are preferred to bug reports (though if you submit a bug report, you're welcome to fix your own bug).
Please be sure to update all documentation to reflect your changes - add to the Readme files and the in code commenting.