quantify
Stability: 1 - Experimental
Yet another Node.js metrics module.
This one was created and inspired by the measured module. Much as the creator of that one, I wanted to better understand the underlying algorithms, as well as provide for a different interface for interacting with metrics informed by the metric, entry, reporter, subscription ideas of @bltroutwine from Ten Billion a Day, One-Hundred Milliseconds Per - Monitoring Real-Time Bidding At AdRoll presentation.
The subscription concept is elaborated in the quantify.subscribe(config) section.
The histogram implementation uses weighted sampling and exponentially decaying reservoir described here.
Contributors
Usage
To run the below example run:
npm run readme
"use strict"; const Quantify = ; const metrics = ; // create a counterconst counter = metrics;// create a gaugeconst gauge = metrics;// create a histogramconst histogram = metrics;// create a meterconst meter = metrics;// create a timerconst timer = metrics; // create a counter with metadataconst counterWithMetadata = metrics; counter; // incrementcounter; // decrement counterWithMetadata; // increment gauge; // set histogram; // updatehistogram; // update meter; // markmeter; // 10 "simultaneous" marks const stopwatch = timerstart; // start a timerstopwatch; // stop a timertimer; // explicitly update the timer with given value console;consoledirmetrics; // get metrics synchronously const subscriptionName = metrics;metrics;metrics; // invoke a specific subscription every 5 seconds;
Tests
npm test
Documentation
Quantify
Public API
- Quantify.COUNTER_FIELDS
- Quantify.GAUGE_FIELDS
- Quantify.HISTOGRAM_FIELDS
- Quantify.HISTOGRAM_MEASURE_FIELDS
- Quantify.METER_FIELDS
- Quantify.METER_RATE_FIELDS
- Quantify.METRICS
- Quantify.METRIC_TYPES
- Quantify.TIMER_FIELDS
- Quantify.TIMER_MEASURE_FIELDS
- Quantify.TIMER_RATE_FIELDS
- new Quantify(name)
- quantify.counter(name, unit, [metadata])
- quantify.gauge(name, unit, [metadata])
- quantify.getMetrics(filters)
- quantify.histogram(name, units, [metadata])
- quantify.meter(name, units, [metadata])
- quantify.subscribe(config)
- quantify.timer(name, units, [metadata])
- quantify.unsubscribe(subscriptionName)
- Event '<subscriptionName>'
Quantify.COUNTER_FIELDS
- ['unit', 'value']
Counter only has value
field.
Quantify.GAUGE_FIELDS
- ['unit', 'value']
Gauge only has value
field.
Quantify.HISTOGRAM_FIELDS
- ['updateCount', 'max', 'mean', 'median', 'min', 'percentile75', 'percentile95', 'percentile98', 'percentile99', 'percentile999', 'standardDeviation', 'sampleSize', 'measureUnit', 'sampleSizeUnit']
All histogram fields.
Quantify.HISTOGRAM_MEASURE_FIELDS
- ['max', 'mean', 'median', 'min', 'percentile75', 'percentile95', 'percentile98', 'percentile99', 'percentile999', 'standardDeviation']
HISTOGRAM_MEASURE_FIELDS
are the HISTOGRAM_FIELDS
that share the measure unit. The measure unit is the unit associated with the value given to histogram.update(<value>)
.
Quantify.METER_FIELDS
- ['updateCount', 'meanRate', 'oneMinuteRate','fiveMinuteRate', 'fifteenMinuteRate', 'rateUnit', 'updateCountUnit']
All meter fields.
Quantify.METER_RATE_FIELDS
- ['meanRate', 'oneMinuteRate', 'fiveMinuteRate', 'fifteenMinuteRate']
METER_RATE_FIELDS
are the METER_FIELDS
that are per second rates.
Quantify.METRICS
- ['counters', 'gauges', 'histograms', 'meters', 'timers']
Quantify.METRIC_TYPES
- ['counter', 'gauge', 'histogram', 'meter', 'timer']
Quantify.TIMER_FIELDS
- ['updateCount', 'meanRate', 'oneMinuteRate', 'fiveMinuteRate', 'fifteenMinuteRate', 'max', 'mean', 'median', 'min', 'percentile75', 'percentile95', 'percentile98', 'percentile99', 'percentile999', 'standardDeviation', 'sampleSize', 'measureUnit', 'rateUnit', 'sampleSizeUnit']
All timer fields.
Quantify.TIMER_MEASURE_FIELDS
- ['max', 'mean', 'median', 'min', 'percentile75', 'percentile95', 'percentile98', 'percentile99', 'percentile999', 'standardDeviation']
TIMER_MEASURE_FIELDS
are the TIMER_FIELDS
that share the measure unit. The measure unit is the unit associated with the value given to timer.update(<value>)
.
Quantify.TIMER_RATE_FIELDS
- ['meanRate', 'oneMinuteRate', 'fiveMinuteRate', 'fifteenMinuteRate'];
TIMER_RATE_FIELDS
are the TIMER_FIELDS
that are per second rates.
new Quantify(name)
name
: String Quantify instance name.
quantify.counter(name, unit, [metadata])
name
: String Counter name.unit
: String What is counted.metadata
: Object Optional metadata.- Return: Counter Instance of a Counter entry.
Get or create a counter with provided name.
var Quantify = ;var metrics = ;var counter = metrics;var counterWithMetadata = metrics;counter; // incrementcounter; // incrementcounter; // decrement
quantify.gauge(name, unit, [metadata])
name
: String Gauge name.unit
: String What is measured.metadata
: Object Optional metadata.- Return: Gauge Instance of a Gauge entry.
Get or create a gauge with provided name.
var Quantify = ;var metrics = ;var gauge = metrics;var gaugeWithMetadata = metrics;gauge; // set to 17gauge; // set to 10gauge; // set to 122
quantify.getMetrics(filters)
filters
: Object (Default: undefined)counters
: RegExp (Default: undefined) If specified, subscription will only return counters with names that match the RegExp.gauges
: RegExp (Default: undefined) If specified, subscription will only return gauges with names that match the RegExp.histograms
: RegExp (Default: undefined) If specified, subscription will only return histograms with names that match the RegExp.meters
: RegExp (Default: undefined) If specified, subscription will only return meters with names that match the RegExp.timers
: RegExp (Default: undefined) If specified, subscription will only return timers with names that match the RegExp.
- Return: Object Snapshot of metrics.
Synchronously calculate the snapshot of all metrics and return it.
With filters, one can specify what should be returned:
var metrics = ; var fooCounter = metrics;var barCounter = metrics; var fooSnasphot = metrics;console; // 0console; // undefined var barSnapshot = metrics;console; // undefinedconsole; // 0
quantify.histogram(name, units, [metadata])
name
: String Histogram name.units
: Object Units to use.measureUnit
: String What specific feature/property/aspect is being measured (ex: request latency).sampleSizeUnit
: String What is being measured (ex: web request).
metadata
: Object Optional metadata.- Return: Histogram Instance of a Histogram entry.
Get or create a histogram with provided name.
var Quantify = ;var metrics = ;var histogram = metrics;var histogramWithMetadata = metrics;histogram;histogram;histogram;
quantify.meter(name, units, [metadata])
name
: String Meter name.units
: Object Units to use.rateUnit
: String The rate of what is being measured per second (ex: web requests per second).updateCountUnit
: String What is being measured (ex: web request).
metadata
: Object Optional metadata.- Return: Meter Instance of a Meter entry.
Get or create a meter with provided name.
var Quantify = ;var metrics = ;var meter = metrics;var meterWithMetadata = metrics;meter;meter;meter;
quantify.subscribe(config)
config
: _Objectfilters
: Object (Default: undefined)counters
: RegExp (Default: undefined) If specified, subscription will only return counters with names that match the RegExp.gauges
: RegExp (Default: undefined) If specified, subscription will only return gauges with names that match the RegExp.histograms
: RegExp (Default: undefined) If specified, subscription will only return histograms with names that match the RegExp.meters
: RegExp (Default: undefined) If specified, subscription will only return meters with names that match the RegExp.timers
: RegExp (Default: undefined) If specified, subscription will only return timers with names that match the RegExp.
label
: String (Default: undefined) Optional label for human readibility.
- Return: String Unique subscription name.
Creates a new subscription. The subscription name returned has two uses. First, it is a name of a method to call when this subscription should emit current data. Second, it is a name of an event that will be emitted when the method with this subscription name is called.
var metrics = ; var subscriptionName = metrics;metrics; metricssubscriptionName; // trigger the subscription
Why? This is in order to decouple the timing of metric reporting from Quantify itself. Additionally, we decouple the timing of metric reporting from whatever reporters are interested in the particular subscription.
Another module can handle scheduling of when various metrics should be reported. If you want some things to be reported once a minute, but others once every five minutes, a way to achieve that in Quantify is to setup two subscriptions. For example:
var metrics = ; var everyMinute = metrics;var everyFiveMinutes = metrics; metrics;metrics; ;;
With filters, one can specify what should be reported for each subscription:
var metrics = ; var fooCounter = metrics;var barCounter = metrics; var everyMinute = metrics;var everyFiveMinutes = metrics; metrics;metrics; ;;
Lastly, Quantify enables mutliple handlers to subscribe to the emitted subscription event:
var metrics = ; var fooCounter = metrics; var everyMinute = metrics; metrics;metrics; ;
quantify.timer(name, units, [metadata])
name
: String Timer name.unit
: Object Units to use.measureUnit
: String What specific feature/property/aspect is being measured (ex: request latency).rateUnit
: String The rate of what is being measured per second (ex: web requests per second).sampleSizeUnit
: String What is being measured (ex: web request).
metadata
: Object Optional metadata.- Return: Timer Instance of a Timer entry.
Get or create a timer with provided name.
var Quantify = ;var metrics = ;var timer = metrics;var timerWithMetadata = metrics;timer; // explicitly record a time intervalvar stopwatch = timerstart; // start measuring a time interval;
If you use stopwatch
functionality, the interval is calculated in milliseconds and the value provided internally via timer.update()
will be updated in milliseconds.
If you never use stopwatch
for a specific timer instance, the unit of the timer.update()
call are whatever you want them to be.
quantify.unsubscribe(subscriptionName)
subscriptionName
: String Name of subscription to unsubscribe.- Return: Boolean
false
if subcription does not exist,true
if successfully unsubscribed.
<subscriptionName>
Event function (data) {}
data
: Object Object containing counters, gauges, histograms, and meters corresponding to the given<subscriptionName>
.latency
: Number Number of milliseconds it took to prepare this subscription.counters
: Object Object containing counters by name. Each counter having the properties:unit
,value
.gauges
: Object Object containing gauges by name. Each gauge having the properties:unit
,value
.histograms
: Object Object containing histograms by name. Each histogram having the properties:updateCount
,max
,mean
,median
,min
,percentile75
,percentile95
,percentile98
,percentile99
,percentile999
,sampleSize
,standardDeviation
,measureUnit
,sampleSizeUnit
.meters
: Object Object containing meters by name. Each meter having the properties:updateCount
,fifteenMinuteRate
,fiveMinuteRate
,meanRate
,oneMinuteRate
,rateUnit
,updateCountUnit
.timers
: Object Object containing timers by name. Each timer having the properties:updateCount
,fifteenMinuteRate
,fiveMinuteRate
,meanRate
,oneMinuteRate
,max
,mean
,median
,min
,percentile75
,percentile95
,percentile98
,percentile99
,percentile999
,sampleSize
,standardDeviation
,measureUnit
,rateUnit
,sampleSizeUnit
.
Each subscription emits an event uniquely named with a given subscriptionName
and containing the appropriate data
according to previously set subscription filters.
Counter
An incrementing and decrementing value.
Public API
counter.update(n)
n
: Integer Value to update the counter with. Use negative value to decrement.
Updates the counter value with provided integer. To decrement, use a negative integer.
counter.reset()
Resets the counter value to 0.
counter.value
Returns the current counter value.
Gauge
The instantenous value of something.
Public API
gauge.update(n)
n
: Integer Gauge value to update with.
Updates the gauge with the provided value.
gauge.value
Returns the current gauge value.
Histogram
The statistical distribution of values in a stream of data.
This implementation uses weighted sampling and exponentially decaying reservoir described here.
Public API
histogram.snapshot()
Returns the snapshot of the histogram at the present time. Snapshot is necessary because the Histogram implementation uses weighted sampling and exponentially decaying reservoir in order to give percentile and other statistical estimates while maintaining a fixed sample size and being responsive to changes. These estimates are time-dependent on when the histogram is updated and the time the histogram snapshot is taken. For more on snapshots see the pull request discussed here. For more on the statistics involved see Metrics Metrics Everywhere (Slides) and Metrics Metrics Everywhere (Video).
The returned snapshot has the following available:
snapshot.max()
- Returns the maximum value.snapshot.mean()
- Returns the mean value.snapshot.median()
- Returns the median value.snapshot.min()
- Returns the minimum value.snapshot.percentile75()
- Returns the 75th percentile value.snapshot.percentile95()
- Returns the 95th percentile value.snapshot.percentile98()
- Returns the 98th percentile value.snapshot.percentile99()
- Returns the 99th percentile value.snapshot.percentile999()
- Returns the 99.9th percentile value.snapshot.quantile(q)
- Returns theq
th percentile value (q
should be between 0.0 and 1.0).snapshot.size()
- Returns the number of values in the snapshot.snapshot.standardDeviation()
- Return the standard deviation.
histogram.update(n)
n
: Integer Value to update the histogram with.
Updates the histogram with the provided value.
histogram.updateCount()
Returns the count of total updates to the histogram.
Meter
Measures the rate at which events occur.
Public API
- meter.fifteenMinuteRate()
- meter.fiveMinuteRate()
- meter.meanRate()
- meter.oneMinuteRate()
- meter.update(n)
- meter.updateCount()
meter.fifteenMinuteRate()
Returns the fifteen minute rate in updates per second.
meter.fiveMinuteRate()
Returns the five minute rate in updates per second.
meter.meanRate()
Returns the mean rate since meter creation in updates per second.
meter.oneMinuteRate()
Returns the one minute rate in updates per second.
meter.update(n)
n
: Integer Value to update the meter with.
Updates the meter n
times.
meter.updateCount()
Returns the current count of meter updates.
Timer
A combination of a histogram of the duration of an event and a meter of the rate of its occurence.
Public API
- timer.fifteenMinuteRate()
- timer.fiveMinuteRate()
- timer.meanRate()
- timer.oneMinuteRate()
- timer.snapshot()
- timer.update(n)
- timer.updateCount()
timer.fifteenMinuteRate()
Returns the fifteen minute rate in updates per second.
timer.fiveMinuteRate()
Returns the five minute rate in updates per second.
timer.meanRate()
Returns the mean rate since timer creation in updates per second.
timer.oneMinuteRate()
Returns the one minute rate in updates per second.
timer.snapshot()
Returns the snapshot of the histogram corresponding to the timer at the present time. See histogram.snapshot().
timer.update(n)
n
: Integer Value to update the timer with.
Updates the timer with the provided value.
timer.updateCount()
Returns the current count of timer updates.