X-Hook Service

A concurrency-safe "track and retry" service for Webhooks (and other things?).

You provide the event storage and webhook request mechanism.
We provide the logic and sane defaults to retry failed webhooks with exponential backoff.
(and a bit of random jitter to prevent duplicate events on multiple servers)

Lightweight, zero-dependency.

Install

npm install --save x-hook-service@1

Usage

To run the service which retries failed webhooks in the background:

let XHookService = require("x-hook-service");

let xhookService = XHookService.create({
  store: store,
  retryWindow: 1 * 60 * 1000,
  retryOffset: 15 * 1000,
  // backoffMap: XHooks._backoffMap,
  random: Math.random(),
});

// fetches webhook events every minute
xhookService.run();

// for graceful shutdowns
// (not always necessary since weak timeout references are used)
await xhookService.stop();

Run a webhook immediately (typically the first time):

// schedules a webhook immediately
let date = new Date();
let event = { ulid: "01H9Y080000000000000000000", retries: 0, retry_at: null };
await xhookService.immediate(date, event);

Store & Event Implementation

You must provide a Store which can retrieve and update Webhook Events and Attempts (however you choose to track those).

They will be called by the service like this (with all the logic omitted):

let events = await store.anyIncomplete(nearFutureDate);
let event = await store.oneEvent(eventId);
eventId = store.getEventId(event);

let attempt = await store.addAttempt(attemptDate, { retry_at }, event);
let result = await store.runAttempt(attemptDate, attempt, event);
await store.updateAttempt(attemptDate, attempt, event, result);

The Store Interface

let store = {};

These are used to get a list of webhooks that should be attempted again, as well as the most up-to-date version of a specific event (it is checked just before a retry).

store.anyIncomplete = async function (nearFutureDate) {
  // return events for which the webhook should be tried again

  // Example:
  //     SELECT
  //         *
  //     FROM
  //         events
  //     WHERE
  //         completed_at is NULL
  //         AND
  //         retry_at <= :near_future_date

  return [event];
};

store.oneEvent = async function (eventId) {
  // return the event by the given id
  return event;
};

store.getEventId = function (event) {
  return event.ulid;
};

The actual running and reporting of the webhook is broken into 3 steps to ensure data consistency:

store.addAttempt = async function (attemptDate, { retry_at }, event) {
  // create and return the data representing a new webhook attempt

  // Example:
  //
  //     UPDATE
  //         events
  //     SET
  //         retries = :retries,
  //         retry_at = :retry_at
  //     WHERE
  //         event.ulid = :ulid

  return attempt;
};

store.runAttempt = async function (attemptDate, attempt, event) {
  // make the webhook request and return the result

  // Example:
  //
  //     let payload = JSON.stringify(event.details)
  //     let xHubSig = xHubSign(payload);
  //     fetch(event.webhook_url, {
  //         headers: { x-hub-signature-256: xHubSig },
  //         body: payload
  //     })

  return result;
};

store.updateAttempt = async function (attemptDate, attempt, event, result) {
  // update the record of attempt with the result

  // Example:
  //
  //     UPDATE
  //         events
  //     SET
  //         completed_at = CURRENT_TIMESTAMP,
  //         retry_at = null
  //     WHERE
  //         event.ulid = :ulid

  return;
};

The Event Interface

The event must have

• some form of id (retrieved via getEventId())
• a retries property indicating how many times the webhook has failed
• retry_at, indicate the next time a webhook request should be tried
• whatever details you need to attempt the webhook request

{
  ulid: '',
  retries: 0,
  retry_at: new Date(),
  // the rest is up to you
}