Wrender
High-performance image compression and transformation reverse-proxy for Node.js Express apps.
This library can be used to serve up compressed and transformed images from a high-resolution origin (e.g. Amazon S3) suitable for caching and delivery by a CDN.
It provides features comparable to Imgix and Cloudinary for environments where you want much more customisation for how you source and handle your images. You will of course need your own CDN!
It is an open-source re-implementation of Car Throttle's image delivery service, which (running behind Cloudfront) handles hundreds of GBs and tens of millions of requests every day.
The image processing is extremely fast and is handled by Sharp, which implements the libvips library as a native module. As such, Node.js Streams are used to abstract the handling of image data.
The recommended usage is part of a larger express-based application although a simple server is provided for example, testing and non-production environments. Rate-limiting, authentication, logging, and other such features are best implemented alongside with relevant packages and therefore are not provided here, although we do present a few examples to better demonstrate certain use-cases.
Use version 3.x.x for Node 12+
Use version 2.x.x for Node 10
Breaking changes:
-
Recipe handler functions are treated as
async
functions, so they can now return a promise. This lets you use the Sharp metadata api inside recipes. -
Recipe handler functions must return the Sharp
image
object for the pipeline to work.
Compression Defaults
-
All images are converted to JPEG and compressed at quality level 85.
-
All EXIF data is stripped (including colour profiles).
-
All images are converted to sRGB colour space.
-
If include EXIF is set to true, all metadata is preserved, and an sRGB ICC colour profile is assigned.
-
Cache headers are set expire 1 year in the future. Set your web server or CDN to respect the headers.
-
Source images larger than 3000px in each dimension are not transformed and an error response is sent.
Usage
const express = ;const wrender = ; const app = ; const instance = ; app;
For a complete example with full configuration object and defaults, see below.
Recipes
Different strategies for image handling are defined as the first parameter of the URL path. All recipe paths contain /:origin
, which refers to the specific origin the client wishes to use. Failure to end your recipe with /:origin
will result in an error being thrown, so ideally you should configure recipes on boot. For example, a recipe of /hello/:origin
would match:
/hello/https://static.carthrottle.com/workspace/uploads/articles/dsc_6267-56ead06f7fda8.jpg
# Note the protocol, that's important to allow HTTPS origins
# If the origin contains a query string, you must encode the URL first:
`/hello/http%3A%2F%2Fstatic.carthrottle.com%2Fworkspace%2Fuploads%2Farticles%2F%3Ffilename%3Ddsc_6267-56ead06f7fda8.jpg`
Built-in recipes
These are the recipes that are attached if you omit recipes
from the config object you supply to wrender
.
- Proxy
- Exposed at
wrender.recipes.proxy
- The default path is
/proxy/:origin
- Applies compression to the source image, but no other transformations.
- Exposed at
- Resize
- Exposed at
wrender.recipes.resize
- The default path is
/resize/:width/:height/:origin
- You can also resize by
:width
or:height
, whilst maintaining the aspect ratio, by setting either:width
or:height
to0
.
- Exposed at
- Crop
- Exposed at
wrender.recipes.crop
- The default path is
/crop/:width/:height/:origin
- Resizes the source image to the desired dimensions (maintaining aspect ratio), then performs a crop from the centre.
- Exposed at
Origins
Origins describe where the original image content is coming from. They append the path in the recipe, replacing /:origin
with their path, and can be used to obfuscate the original source of the images.
Built-in origins
If you omit origins
from the config object you supply to wrender, the default HTTP origin will be used.
- HTTP
- Fetch source images from an external HTTP(S) source
- Exposed at
wrender.origins.http()
- Function taking entirely optional
opts
:prefix
- add a prefix to the origin to avoid catch-all usagedefaults
- pass a set of default options torequest.defaults
whitelist
- pass a whitelist in micromatch format for hostnames to allow (see examples)blacklist
- pass a whitelist in micromatch format for hostnames to deny (see examples)
- The default path is
/:source
, which makes this origin act as a catch-all - If you require a query string then you must url-encode the entire
:source
, otherwise Express will strip the query string
- FS
- Loads an image from the filesystem
- Exposed at
wrender.origins.fs()
- Function taking opts
prefix
- optionally add a prefix to the origin to avoid catch-all usagemount
- optionally define the start mount for the source, e.g./data
- The default path is
/:source
, which makes this origin act as a catch-all
- Identicon
- Generates an image from the hash of an input token similar to GitHub
- Exposed at
wrender.origins.identicon()
- URL params
/identicon/:token
token
- input token which is hashed to generate the background colour (e.g. user id)
- Function taking opts
prefix
- optionally add a prefix to the origin to avoid catch-all usage (default 'identicon')size
- size of the generated image (note: increasing this may impact memory usage). If:size
or:width
is used in the recipe params, the recipe params will overwrite the options here.gridsize
- odd-numbered-integer to divide the image into pixelssaturation
- intensity of the foreground colour[0, 1]
lightness
- white/black level of the foreground colour[0, 1]
background
- rgb array for the background colour[ r, g, b ] [0, 255]
invert
- swaps the foreground and background colours (i.e. pixels are white on a coloured background)
- Initials
- Generates an plain background colour with a text overlay
- Exposed at
wrender.origins.initials()
- URL params
/initials/:token/:text
token
- input token which is hashed to generate the background colour (e.g. user id)text
- text to overlay (keep to 1 or 2 characters for best results)
- Function taking opts
prefix
- optionally add a prefix to the origin to avoid catch-all usage (default 'initials')size
- size of the generated image (note: increasing this may impact memory usage). If:size
or:width
is used in the recipe params, the recipe params will overwrite the options here.saturation
- intensity of the background colour[0, 1]
lightness
- white/black level of the background colour[0, 1]
color
- text colour (default 'white')font
- font family for the test (default 'sans-serif')
API
const wrender = ; const instance = ; app;/** * Available recipes: * - /proxy/:origin * - /resize/:width/:height/:origin * - /crop/:width/:height/:origin * * Available origins: * - /http/:url * - /fs/:path * - /s3/:Bucket/:Key * * All together, available routes are, noting that the instance is mounted to "/images": * - /images/proxy/http/:url * - /images/proxy/fs/:path * - /images/proxy/s3/:Bucket/:Key * - /images/resize/:width/:height/http/:url * - /images/resize/:width/:height/fs/:path * - /images/resize/:width/:height/s3/:Bucket/:Key * - /images/crop/:width/:height/http/:url * - /images/crop/:width/:height/fs/:path * - /images/crop/:width/:height/s3/:Bucket/:Key */
Error Handling
If an error is caught inside wrender's route handler, a blank 1x1 PNG is served as a response along with an appropriate error code (usually 404 or perhaps 500).
It is advised (but not required) to add a onError
callback function to the constructor. This callback takes one argument (error
) and is fired after the response is sent. You can use this callback to log errors wherever you like.
Custom Recipes
Custom recipes are designed to allow you complete customisation of how images are transformed before being served to the client, by using the Sharp API.
Recipes are created using the wrender.createRecipe
method with the following arguments:
wrender// Where `path` is a string defining the first part of the mount point, ending in /:origin// Where `handler` is function, with the arguments (image, params)// `image` is the Sharp instance, for you to instruct the transformation// `params` is the req.params, which contain the variables in the route that you set with `path`, plus...// `params.query` is req.query// `params.path` is req.path// `params.originalUrl` is req.originalUrl// - it is always executed asynchronously (i.e. always treats the return value as a promise)// - you *must* return the image (sharp object) at the end of the handler function// Where `config` is a plain object containing overrides to the Wrender instance config scoped to the this recipe only. Useful for customising quality or GIF conversion on a URL-basis.
Recipe examples
Asynchronous recipes are not supported. If you're looking to do an asynchronous operation with your recipe, consider using the underlying sharp
package.
wrender
This recipe will resize the image using the built-in resize recipe, to 200x200, then flop the image about the horizontal X axis, as discussed in the Sharp API operation docs.
wrender
By using wrender.invokeRecipe(recipe, image, [params])
you can call existing recipes with pre-defined values. This is useful if you wish to hide options from the URLs to prevent undesired costs or DoS attacks:
const watermark = 'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mP8/5+hHgAHggJ/PchI7wAAAABJRU5ErkJggg==' 'base64'; wrender
Following the composite docs, we can see how we would implement a watermark recipe.
Custom Origins
Not every use-case involves fetching information from a public-facing image endpoint. Therefore wrender support custom origins, which can also be used to obfuscate the source of your images.
Origins are created using the wrender.createOrigin
method with the following arguments:
wrender// Where `path` is a string defining the last part of the mount point// Where `handler` is a function, optionally async, with the arguments (params)// `params` is the req.params, which contain the variables in the route that you set with `path`, plus...// `params.query` is req.query// `params.path` is req.path// `params.originalUrl` is req.originalUrl
Ensure params in your origin paths are unique to your origin, as conflicting params with recipes will lead to unexpected behaviours. For example, a recipe with /resize/:width/:height/:origin
and an origin with /fb/:width/:profile_id
will lead to /resize/:width/:height/fb/:width/:profile_id
. Not good!
handler
expects a readable stream to be returned. Origin functions can be async, allowing you to perform (hopefully) simple async operations, to a database or an external source.
Origin examples
Whitelist/blacklist HTTP origins
It's likely you will want to run your HTTP(S) origins through a whitelist/blacklist, to ensure only origins you allow (or prevent origins you disallow) from being hit by your wrender instance. This is supported by default, and the micromatch syntax is supported:
app; // => /images/proxy/https://s3.amazonaws.com/user-uploads.someimportantcompany.com/profiles/1505c30c51bb93545db48919b3cce7f9.jpg// => Will succeed, since s3.amazonaws.com is in the whitelist// => /images/proxy/https://i.imgur.com/cl4Bu.gif// => Will fail, since i.imgur.com isn't in the whitelist// => /images/proxy/https://hack.thepla.net/evilcorp.exe// => Hasn't got a chance, since it's not in the whitelist, and irrelevantly isn't in the blacklist// => In this example, you would need to remove the whitelist array in order to only use the blacklist
Private S3 Buckets
const wrender = ;const AWS = ; const s3 = region secretAccessKey accessKeyId ; // Load these from environment variables moduleexports = wrender; // => /images/proxy/s3/user-uploads.someimportantcompany.com/profiles/1505c30c51bb93545db48919b3cce7f9.jpg// => Streams from S3, as long as the s3 instance has the correct permissions// => Super-effective with EC2 instance roles & ECS task roles
Facebook Profile Pictures
const request = ;const wrender = ; moduleexports = wrender; // => /images/proxy/fb/113741208636938// => https://graph.facebook.com/113741208636938/picture?width=1024&height=1024
This is also a good example for using custom origins to rewrite URLs.
Lookup image data from a model
const wrender = ;const AWS = ;const images = ;const s3 = region secretAccessKey accessKeyId ; moduleexports = wrender; // => /images/resize/200/200/users/9ff4a3cf5fe1a735ec96f142a2081f3e// => s3://user-uploads.someimportantcompany.com/profiles/9ff4a3cf5fe1a735ec96f142a2081f3e.jpg
Hopefully, images.findById
will be nicely cached or easy to pull up.
Docker
$ docker build -t g-wilson/wrender:dev .$ docker run -it -p 3010:3010 g-wilson/wrender:dev