@specprotected/spec-proxy-fastly-worker

Spec Proxy Fastly Compute@Edge API Integration

This document describes a method of integrating with Spec Proxy through a Fastly Compute@Edge worker.

We won't cover in detail how to use a Fastly worker and will instead direct you to follow their documentation to get a worker project set up and deployed. Once deployed, all you need do is copy our sample code into your edge worker function and communicate with the Spec team to coordinate the Spec Proxy deployment.

Fastly Compute@Edge Documentation

The Fastly Backend documentation talks about backends and how they work. You will need a backend that points to your origin server and another one that points to your Spec Proxy deployment which this library will use to properly route traffic. We can support complex deployment scenarios as well, we go into this in detail later in this document.

Implementation Example

Using our library is simple. We require only a single function call if we are the only library you are using.

Here are some of Fastly's documents on implementing a JavaScript worker:

• JavaScript Worker Documentation
• JavaScript Code Examples

Static Backends

Note: You MUST configure Fastly Backends in order for this deployment method to work.

---

In this example, customers are responsible for configuring the Compute Service with the appropriate backends. These backends are then registered with the Compute function. We translate the incoming hostname to the name of the backend that should be used. This assures proper routing of traffic out of the Compute Service.

In addition to your Backends that serve your application, you will need to create a Backend configuration for your Spec Proxy deployment. Work with your Spec representative to get the hostname of your Spec Proxy deployment which you will use when defining the spec-proxy Backend in your Fastly configuration.

Configuration Options are defined below.

/// <reference types="@fastly/js-compute" />

import {
  specProxyProcess,
  setFastlyBackends,
} from "@specprotected/spec-proxy-fastly-worker";

// This function sets the backends that are used by the fastly service worker
// library to properly route incoming traffic to its ultimate destination.
// You must supply a map of url hostname to Fastly backend, which this library will
// use to automatically direct traffic.
setFastlyBackends({
  // This is your origin destination backend
  "www.example.com": "origin",
  // This is your Spec Proxy deployment backend
  "www.example.com.spec-internal.com": "spec-proxy",
});

addEventListener("fetch", (event) => {
  event.respondWith(
    specProxyProcess(event, {
      /* Configuration Options */
    }),
  );
});

Dynamic Backends

If you have a dynamic number of backends or the number of backends is quite large, you can use the Dynamic Backends configuration.

---

Note: Dynamic Backends must be enabled on the Compute Service by Fastly, please contact your representative to use this feature or email [e-mail].

---

This feature is enabled in the worker through the property dynamicBackends in your SpecConfiguration object.

The implementation is quite similar to static backends, but relies on some information provided to the worker of the intended destination. This can occur through a header that contains the destination host, or through hostname replacement, or both. By supplying the configuration option alternateHostHeader, the Spec Proxy library will infer the destination host from this header. Additionally, you can use the hostReplacements property to supply an array of replacements. We use the destination host as the target of replacement, which comes from the Host header or the alternateHostHeader if that is supplied, and then replace part of the hostname with an alternate hostname that points to the destination origin. We will set the Host header of the outgoing request to the destination host value after all replacements are made.

To reiterate, we first acquire the destination host value. This value comes from the Host header unless alternateHostHeader is specified, in which case we will use the header you provide. We then take the destination host and perform any hostReplacements on it to create the final destination host value.

If no match is found, and Static Backends are also set using the technique above, then we will attempt to match a Static Backend configuration with the incoming host.

The dynamic backend mechanism essentially allows a prior service to inform the Edge Worker the location of the origin dynamically. This allows you to add and remove hosts without re-configuring the worker itself. If you change any hosts, you should contact your Spec representative to ensure that the new traffic is properly analyzed.

/// <reference types="@fastly/js-compute" />

import { specProxyProcess } from "@specprotected/spec-proxy-fastly-worker";

import { allowDynamicBackends } from "fastly:experimental";

// configure these values as necessary
allowDynamicBackends({
  connectTimeout: 1000,
  firstByteTimeout: 15000,
  betweenBytesTimeout: 10000,
});

addEventListener("fetch", (event) => {
  event.respondWith(
    specProxyProcess(event, {
      dynamicBackends: true,
      hostReplacements: [
        {
          find: ".customer.com",
          replace: ".origin-customer.com",
        },
        {
          find: "(.*).customer.com",
          replace: "origin-$1.customer.com",
          regex: true,
        },
      ],
    }),
  );
});

In addition to the Configuration Options below, the Spec Proxy Fastly Library provides these additional configuration options to support the Dynamic Backends feature.

Variable	Type	Default	Description
dynamicBackends	Boolean	false	Set this flag to true to enable Dynamic Backends support in the Spec Proxy Library. Dynamic Backends is a feature that allows the Compute@Edge worker to dynamically route traffic to Fastly Backends without having to pre-define them in the UI.
alternateHostHeader	String	null	When provided, the Spec Porxy Library will use the header specified in this value to acquire the destination hostname. This hostname is used when the Dynamic Backend is constructed.
hostReplacements	Array	null	When provided, the Spec Proxy Library will attempt to replace the hostname using the supplied find/replace values. This will use the value of the incoming Host header unless alternateHostHeader (above) is supplied. This feature can be used to algorithmically replace incoming hostnames to form the destination hostname. This is useful if your origin servers share a common pattern, such as prefixing the hostname with origin, as in www.customer.com becomes www.origin.customer.com. This supports a standard string find/replace as well as regex find/replace with support for capture groups.

You can find the definition of the HostReplacement type in our library's typings definition, but we'll also show this here. The replacement occurs from the two properties find and replace. We attempt to find the text using the find pattern and replace it with the replace pattern. When regex is set to true then you may use regular expressions, including capture groups, to perform the find/replace.

export interface HostReplacement {
  /**
   * The pattern to find within the hostname
   */
  find: string;
  /**
   * The text to replace the found pattern within the hostname.
   * This can contain capture group references when regex = true.
   */
  replace: string;
  /**
   * Set to true to enable a regular expression search. Since regex
   * searches are slightly slower, this is not enabled by default.
   */
  regex: boolean;
}

Configuration Options

We provide a few configuration options for how traffic should be handled by the Cloudflare Worker.

Variable	Type	Default	Description
disableSpecProxy	Boolean	false	Toggle between enabling or disabling Spec processing. When disabled (true), all traffic is routed directly to the customer's downstream origin, bypassing Spec completely. This setting causes all of the following settings to be ignored.
inlineMode	Boolean	false	Toggle between two available processing modes. Inline mode (true) works by forwarding traffic through the Spec Trust Cloud for processing. This mode enables inline mitigations. Mirror mode (false) creates a copy of traffic to send to the Spec Trust Cloud for processing while the original message is forwarded directly to the customer's downstream origin. This mode does not allow for inline mitigations. Note: Do not adjust this setting without contacting your Spec representative.
percentageOfIPs	Number	100	Number representing the percentage of all IP addresses which should have traffic routed through Spec. The remaining percentage of IPs will be routed directly to the customer's downstream origin. This can be used for progressive onboarding / rollout.
customerKey	String	none	A key provided by Spec to validate that traffic came from a customer-controlled service worker
disableSpecTraffic	Boolean	none	When set to true, disables routing traffic to Spec Proxy only through the /spec_traffic path prefix. Generally, you do not want disable this feature, but it's provided so customers can control routing to this prefix.

The inlineMode configuration option is the only option that changes how Spec Proxy itself behaves. For more details on what inline mode means and what features of Spec Proxy are available to you when running in inline mode, please contact your Spec representative.

The customerKey option provides extra validation that we are only processing traffic that originated from your service workers. In general, this is redundant for inline processing, since we are processing traffic destined for the customer origin and validating it with a customer-provided SSL certificate. For mirror mode configurations, while we only allow traffic into Spec Proxy from your edge platform's IP address ranges and do not return any data in the responses to mirrored traffic, using the customerKey option is recommended. If this option is provided, we will validate this key prior to processing any mirrored traffic. The key is encrypted in transit with the rest of your mirrored traffic.

Runtime Configuration

You can use a Fastly ConfigStore to configure the Compute@Edge worker in realtime without deploying a new version of worker code. Integrating it into the Spec Proxy configuration object is quite simple. ConfigStore values are configured in your Compute service configuration in the Fastly web UI. Below is an example of how the addEventListener call from the above example changes to use the ConfigStore.

import { ConfigStore } from "fastly:config-store";

addEventListener("fetch", (event) => {
  // load up a fastly store for easy runtime configuration
  let store = new ConfigStore("SpecProxy");

  // Note: null is returned if the key isn't present, so the value
  // of inlineMode and disableSpecProxy would default to false
  let inlineMode = store.get("inline_mode") === "true";
  let disableSpecProxy = store.get("disable_spec_proxy") === "true";

  // configure our use of the spec library
  let config = {
    inlineMode,
    disableSpecProxy,
  };

  // only assign percentage of IPs if it's valid
  let percentageIPs = parseInt(store.get("percentage_of_ips"));
  if (!isNaN(percentageIPs)) {
    config.percentageOfIPs = percentageIPs;
  }

  event.respondWith(specProxyProcess(event, config));
});

Configuring Fastly Backends

---

Note: You do not need to configure backends when using the Dynamic Backends feature.

---

When you create your Compute service in your Fastly account, you will be presented with the service configuration UI. Within this UI are options to configure your Domain, e.g. www.example.com, and then your Origins. The Hosts menu option under Origins is where you configure your Fastly Backends in the UI for your Compute service. You will create two Hosts here, one for your origin server and the other for your Spec Proxy deployment.

Create the host for your origin server by clicking the Create a host button. Take note of the name you supply to the host configuration as this is the name of the backend that you'll use in the implementation example above when calling setFastlyBackends. Give this host the location of your origin server, for example this could be a load balancer that has an IP whitelist that only allows Fastly IPs to connect to it and is located at load-balancer.example.com. Most of the other options here you can configure as you would like, but the options that are particularly important are:

• TLS from Fastly to your host
• Verify certificate
• Certificate hostname
• SNI hostname
• Override host

These options configure secure connections to your origin server. Set TLS from Fastly to your host and Verify certificate to Yes and set the other three options to their appropriate value for your origin. Likely, this will be the value of your Domain configuration, e.g. www.example.com.

Next, you will create the Spec Proxy host. Click Create a host to create a new entry and again take note of the name you provide as this will be the backend you use in the implementation example above when calling setFastlyBackends. You'll set the same options in this configuration as the ones listed above and you can set them them to the same values as the host for your origin server with the exception of Certificate hostname and SNI hostname. These two values will have .spec-internal.com appended to your hostname above, e.g. www.example.com becomes www.example.com.spec-internal.com. This will inform Spec Proxy with the correct location of your origin server for processing.

When you are done, you will have two Hosts, one for your origin server and the other for your Spec Proxy deployment. Fill in the setFastlyBackends function call in the example above with the names of these hosts. When providing the hostname of Spec Proxy, it will be the value of your origin server with .spec-internal.com appeneded to the end. For example, the host www.example.com will become www.example.com.spec-internal.com.

Summary:

Origin

Configuration Name	Configuration Value
Name	origin
Address	www.example.com
Enable TLS	Yes
Verify Certificate	Yes
Certificate hostname	www.example.com
SNI hostname	www.example.com (or check "match certificate hostname")
Override host	www.example.com

Spec Proxy

Configuration Name	Configuration Value
Name	spec-proxy
Address	www.example.com.spec-internal.com
Enable TLS	Yes
Verify Certificate	Yes
Certificate hostname	www.example.com.spec-internal.com
SNI hostname	www.example.com.spec-internal.com (or check "match certificate hostname")
Override host	www.example.com