Edge Side Includes (ESI) for Fastly Compute JavaScript
Run Edge Side Includes (ESI) at the edge in your Fastly Compute JavaScript application.
Usage
npm install @fastly/esi
This is designed to be very easy to use:
/// <reference types="@fastly/js-compute" />
import { EsiTransformStream } from "@fastly/esi";
addEventListener("fetch", (event) => event.respondWith(handleRequest(event)));
async function handleRequest(event) {
const req = event.request;
// Perform a backend request
const url = new URL(req.url);
const headers = new Headers(req.headers);
const beresp = await fetch(url, {
headers,
backend: 'origin_0'
});
// Initialize the ESI Transformation
const esiTransformStream = new EsiTransformStream(url, headers, {
fetch(input, init) {
return fetch(input, { ...init, backend: 'origin_0' });
},
});
// Simply use pipeThrough
const transformedResponse = beresp.body.pipeThrough(esiTransformStream);
return new Response(transformedResponse, {
status: beresp.status,
headers: beresp.headers,
});
}
NOTE:
@fastly/esi
is provided as a Fastly Labs product. Visit the Fastly Labs site for terms of use.
API
EsiTransformStream
constructor
const esiTransformStream = new EsiTransformStream(url, headers, options);
Initializes the ESI transformation stream using the following values. These values are used in handling
esi:include
tags as well as providing values for ESI variables.
For predictable results, pass the same values as you used to make the backend request.
In most cases, the only value you need to pass for options
is fetch
— to provide the
backend
used by fetch()
resulting from esi:include
tags.
-
url
- the absolute URL where the resource was fetched from, used to resolve relative URLs, as well as used by ESI variables. -
headers
- the headers, which will be sent along with the subrequests caused byesi:include
, as well as used by ESI variables. -
options
- an object with the following keys:-
fetch(input, init)
(optional)The
fetch()
function called whenever anesi:include
tag is encountered. If not specified, the globalfetch()
function is used.- It will be called with the resolved, absolute URL of the resource being requested, along with
the
headers
passed in to the constructor, with the exception that if the resource is being requested from a different host, then thehost
header is set to that of the resource being requested. - A common use of providing this function in Fastly Compute is to add the
backend
value as a fetch is being made. This is not needed when you're using dynamic backends.const esiTransformStream = new EsiTransformStream(url, headers, { fetch(input, init) { return fetch(input, { ...init, backend: 'origin_0' }); }, });
- It will be called with the resolved, absolute URL of the resource being requested, along with
the
-
processIncludeResponse(esiIncludeResult)
(optional)A function called after a
fetch()
fromesi:include
has succeeded. It is called with an object that contains the following keys:-
url
- the resolved, absolute URL of the document obtained by theesi:include
tag. -
headers
- the HTTP headers used when fetching the resource requested by theesi:include
tag.Note: The above values are as they were when the
fetch
function was called. If you provided an overridefetch
function that caused the resource to be obtained from another location, or using modified headers, those are not reflected in the above values. -
response
- theResponse
object returned by thefetch()
call.
This function is expected to return a
string
, or aPromise
resolving to astring
, which is used by the transformer to replace the entireesi:include
tag in the stream. This value is optional, and should only be used in advanced cases, with care.The default functionality recursively passes the response through another
EsiTransformationStream
such that templates may call into additional templates. -
-
handleIncludeError(xmlElement)
(optional)A function called when resources requested by both the
src
andalt
(if provided) values of anesi:include
tag have been tried, and have resulted in errors. The function can return astring
or aPromise
that resolves to astring
that will be used to replace the entireesi:include
tag, ornull
.If this is not specified, or if this returns
null
or aPromise
that resolves tonull
, then the default behavior is to throw anEsiIncludeError
error. -
esiPrefix
(optional, advanced)A string value or
null
, used to specify a default XML prefix identifier to interpret as an ESI tag. If you wish to suppress this functionality altogether, set this value tonull
. See Custom Namespace Prefix below for details.The default value is
'esi'
.
-
Notes
Supported tags
At the current time, the following tags are supported as described in the specification:
- esi:include
- esi:comment
- esi:remove
- esi:try / esi:attempt / esi:except
- esi:choose / esi:when / esi:otherwise
- esi:vars
ESI Variables are supported in the attributes of ESI tags.
ESI Expressions are supported in the test
attribute of esi:when
.
Additionally, the <!--esi ...--> comment is supported.
The following tags are not supported:
- esi:inline
Errors
If an error from esi:include
is not handled, or you handle it and return null
(via the
handleIncludeError
option), the stream will throw a EsiIncludeError
. Note that if this happens
on a stream that is being read by event.respondWith()
, then the platform has already sent the
status code and headers, and is already streaming to the client. Therefore, it is too late to
respond to this error or send alternate status codes or headers.
If you wish to return an alternate status code, then you must stream the entire response to memory, make sure there are no errors, and then return that buffer.
const value = new Response(beresp.body.pipeThrough(esiTransformStream));
let buffer;
try {
buffer = await value.arrayBuffer();
} catch(ex) {
if(!(ex instanceof EsiIncludeError)) {
throw ex;
}
return new Response(
'esi:include error',
{
status: 500,
},
);
}
return new Response(
buffer,
{
status: beresp.status,
headers: beresp.headers,
},
);
Because this could result in a longer TTFB, it is ideal to handle the errors from esi:include
.
Backend requests
Keep in mind that esi:include
tags will cause a backend request, so they are subject to
constraints
at the platform level.
Max inclusion depth
The templates that are fetched by esi:include
tags can recursively include additional esi:include
tags.
This library imposes a limit of 10 levels deep, after which an EsiIncludeError
will be thrown.
XML Namespacing
ESI tags are defined as XML tags that live in the http://www.edge-delivery.org/esi/1.0
namespace.
This means that formally, the namespace needs to be declared on the tag itself or on a parent element, e.g.:
<esi:include src="/bar" xmlns:esi="http://www.edge-delivery.org/esi/1.0"/>
or:
<html xmlns:esi="http://www.edge-delivery.org/esi/1.0">
<esi:include src="/bar" />
</html>
However, implementations of ESI have had a history of not explicitly requiring this declaration. Additionally, because of the way we must handle XML tags in HTML, it's not always possible to reliably find the parent of an XML tag. For these reasons, this library implies this declaration by default and makes it available everywhere.
<!-- Works, because xmlns:esi="http://www.edge-delivery.org/esi/1.0" is available by default. -->
<esi:include src="/bar" />
Custom Namespace Prefix
If you'd like to make ESI available under a prefix other than esi:
instead, provide it as the esiPrefix
value when
constructing EsiTransformStream
. This may be useful if you are using this library in conjunction with other
processing that uses the 'esi'
prefix for its own use.
For example, you may set up your transform stream like this:
const esiTransformStream = new EsiTransformStream(url, headers, {
esiPrefix: 'my-esi'
});
Then, if you have a document like this, its various tags will be handled as described:
<!-- EsiTransformStream knows about my-esi, so this is treated as an ESI include -->
<my-esi:include src="/foo" />
<!-- EsiTransform does not know about esi, so this is not handled -->
<esi:include src="/bar" />
<!-- Attribute on the tag explicitly sets namespace, so this is treated as an ESI include -->
<esi:include src="/bar" xmlns:esi="http://www.edge-delivery.org/esi/1.0" />
If you wish to disable the automatic prefix declaration altogether, it's also possible to set the value to null
:
const esiTransformStream = new EsiTransformStream(url, headers, {
esiPrefix: null
});
Note: If you do this, then
EsiTransformStream
will not know about any ESI namespaces, so you will need to specify the namespace in your document.
XML tags in HTML
ESI tags are defined as an XML-based language. This means they must follow the rules of XML, such as attribute quoting and matching closing tags. However, this library is designed to work in an HTML context, so it operates in the following way with respect to HTML and XML tags:
-
HTML and XML tags that appear with the default namespace (no XML prefix) are treated as plain text by the transformation. This includes the opening tag and any attributes, as well as any closing tags.
-
Tags that appear with an XML prefix are classified as XML and must meet the rules of XML. Then:
- Tags that belong to the ESI namespace are handled and processed by this library.
- Tags that belong to other namespaces are passed through without processing.
If the text content of either type of tag contain any nested tags, then they are also processed according to the same rules, that is, only tags that appear with an XML prefix are treated as XML.
There are several reasons we do this:
- HTML employs a looser set of rules. For example, HTML defines tags that do not require closing tags (such as
input
), and tags whose closing tags may be implied (such asp
). This makes it very challenging to quickly and accurately determine the structure of the HTML document. - Some ESI templates are authored with partial HTML elements. For example, a "header" template may contain the opening tag of an HTML element, whose matching closing tag exists in a "footer" template. To be compatible with such scenarios, we do not enforce XML rules on HTML tags.
This has a few additional implications relating to advanced namespace use:
- This library won't find ESI tags from the default namespace, even when it's set as the default namespace. For example, while formally valid, the following won't work in this library:
<div>
<!-- include tag in default namespace will be ignored. -->
<include src="/bar" xmlns="http://www.edge-delivery.org/esi/1.0"/>
</div>
- If you're declaring
xmlns:<prefix>
yourself, that declaration will be ignored if it's placed on a tag from the default namespace. For example, while formally valid, the following won't work in this library:
<!-- xmlns:esi is ignored because it's on div which is in the default namespace. -->
<div xmlns:esi="http://www.edge-delivery.org/esi/1.0">
<esi:include src="/foo" /> <!-- Won't be recognized! -->
</div>
In both of the cases above, do this instead:
<div>
<esi:include src="/foo" xmlns:esi="http://www.edge-delivery.org/esi/1.0" />
</div>
Issues
If you encounter any non-security-related bug or unexpected behavior, please file an issue using the bug report template.
Security issues
Please see our SECURITY.md for guidance on reporting security-related issues.
License
MIT.