Polyfill service
Makes web development less frustrating by selectively polyfilling just what the browser needs. Use it on your own site, or as a service.
For usage information see the hosted service, which formats and displays the service API documentation located in the docs folder.
- Requirements
- Running locally
- Configuration
- Testing
- Real user monitoring
- polyfill.io CDN
- Library
- License
Requirements
Running the polyfill service requires a few tools:
- Node.js 6.x and npm 3.x: For installing the dependencies and running the application (npm is installed with Node.js)
Running locally
Clone the project to your system:
git clone git@github.com:Financial-Times/polyfill-service.git
Change into the project directory:
cd polyfill-service
Install the dependencies:
npm install
Build the polyfill sources and start the server, rebuilding and restarting whenever any changes are made to the project:
npm run dev
Configuration
You can configure the Polyfill service using environment variables. In development, configurations are set in .env
. In production, these are set through Heroku config.
PORT
: The port on which to listen for HTTP requests (default 3000).NODE_ENV
: Name of environment.dev
,production
,ci
orqa
. Just used for logging.FASTLY_SERVICE_ID
,FASTLY_SERVICE_ID_QA
,FASTLY_API_KEY
: Used to fetch and render cache hit stats on the usage page of the hosted documentation, and to deploy VCL. If not specified, no stats will be shown and VCL deploy will fail.PINGDOM_CHECK_ID
,PINGDOM_API_KEY
,PINGDOM_ACCOUNT
,PINGDOM_USERNAME
,PINGDOM_PASSWORD
: Used to fetch and render uptime and response time stats on the usage page of the hosted documentation. If not specified, no stats will be shown.GRAPHITE_HOST
: Host to which to send Carbon metrics. If not set, no metrics will be sent.GRAPHITE_PORT
: Port on theGRAPHITE_HOST
to which to send Carbon metrics (default 2002).BROWSERSTACK_USERNAME
andBROWSERSTACK_ACCESS_KEY
: BrowserStack credentials for test tasks (not used by the service itself)ENABLE_ACCESS_LOG
: Any truthy value will enable writing an HTTP access log to STDOUT from Node. Useful if you are not running node behind a routing layer like nginx or heroku.RUM_MYSQL_DSN
: DSN URL for a MySQL database with the schema documented in db-schema.sql. If present, RUM reporting routes will be exposed. See Real User MonitoringRUM_BEACON_HOST
: Hostname of the server to which RUM beacon requests should be sent. See Real User MonitoringSURROGATE_KEY
: The surrogate key assigned to all responses. Useful for purging cached responses from a CDN. (defaultpolyfill-service
);SENTRY_DSN
: The data source name (DSN) to be used for sending error logs to SentrySENTRY_RELEASE
: An identifier for the current release to be sent to Sentry
Testing
The tests are split into tests for the service and tests for the polyfills. The polyfill tests require BROWSERSTACK_USERNAME
and BROWSERSTACK_ACCESS_KEY
to be configured, view the configuration section for more information.
npm run test # run service tests and polyfill tests on a small set of browsers npm run test-node # run the service tests npm run ci # run the service tests and polyfills tests on a large set of browsers
We run the tests on CircleCI. npm run ci
must pass before we merge a pull request.
Real User Monitoring
We have shipped experimental support for using RUM to monitor feature support and performance in browsers. This involves a number of parts, all activated by the presence RUM_MYSQL_DSN
and RUM_BEACON_HOST
env vars:
- RUM client code: a small snippet of legacy-compatible code that will evaluate feature detects on the client, sample resource timing data, and beacon the results back to the service. This is shipped as part of the Node app.
- Beacon endpoint: an endpoint to collect RUM data, terminated at the CDN, logging query data out to Amazon S3 to avoid overloading the backend. To clarify, the backend node server does not provide a route handler for the RUM data collection URL, so this is shipped when we deploy VCL to Fastly. It also requires log streaming to be configured in the Fastly UI.
- Lambda processing function: an AWS Lambda function is used to move the data from the raw log files on S3 into the MySQL backend. This is shipped using a dedicated process with Apex, see below.
- Reporting endpoints: API routes that deliver useful analysis of the RUM data are provided in the node server. These return CSV data intended to populate a spreadsheet. This is shipped as part of the Node app.
Because this requires a fair amount of orchestration, we recommend only enabling it for the FT hosted version. If you want to run the service yourself, you can opt out of this RUM feature by not setting a RUM_MYSQL_DSN
or RUM_BEACON_HOST
.
Routes
/v2/getRumPerfData
: Return stats for resource timing metrics, grouped by CDN POP and country of request origin/v2/getRumCompatData
: Return stats for differences between current targeting configuration and live feature detect results from browers, grouped by feature name, browser family and version.
Deploying Lambda
All the bits of the RUM solution are deployed as part of our existing deployment workflow except the Lambda functions, which require Apex. To deploy the Lambda functions:
- Create the following 7 environment variables in your local environment or the
.env
file in the project root:RUM_MYSQL_DSN
,RUM_AWS_ACCESS_KEY
,RUM_AWS_SECRET_KEY
, a second copy of each of these suffixed with_QA
, andRUM_AWS_REGION
. FT devs can get the correct values for these variables from Heroku config or Lastpass. - Run
npm run deploy-lambda
ornpm run deploy-lambda -- --env=prod
as appropriate - If this is the first time you've deployed the function to this AWS profile, you then need to configure the function in the AWS UI:
- Set up a trigger to invoke the function whenever a file is written to the appropriate S3 bucket
polyfill.io CDN
The Financial Times and Fastly host a public version of this service on polyfill.io.
Release process
- Test the release candidate with the npm run compatgen task to generate an updated compatibility table. -
npm run compatgen && git commit docs/assets/compat.json -m 'update compat.json'
- Tag the commit using npm's version command. -
npm version {premajor | preminor | prepatch}
if creating a new RC, ornpm version prerelease
if you already have an activepremajor
,preminor
orprepatch
. - Publish to npm under the
next
dist-tag. -npm publish --tag next
- Push the commits and tags to the git remote. -
git push origin master --tags
- Deploy to QA. -
npm run deploy
- Announce the release on twitter
- Wait some number of days for feedback (usually 7 days). If necessary, make fixes and return to step 1
- Tag the commit/package using npm's version command, using the same semver level as you used for the
pre
versions. -npm version {major | minor | patch}
- Publish to npm under the
latest
dist-tag. -npm publish
- Push the commits and tags to the git remote. -
git push origin master --tags
- Deploy to production. -
npm run promote
Monitoring
We use Graphite and Grafana to keep track of application metrics. You can view requests, bundle build duration, cache hit ratios, and memory usage. It's important after a deploy to make sure we haven't unexpectedly had an impact on these.
We also use Pingdom to track uptime. You should get notifications if you're a member of the Origami team.
Library
API reference
The Polyfill service can also be used as a library in NodeJS projects. To do this:
- Add this repo as a dependency in your package.json
e.g.
npm install polyfill-service --save
- Rebuild your project using
npm install
- Use the API from your code
getPolyfillString(options)
(method)
Returns a promise of a polyfill bundle string. Options is an object with the following keys:
uaString
: String, required. The user agent to evaluate for polyfills that should be included conditionallyminify
: Boolean, optional. Whether to minify the bundlefeatures
: Object, optional. An object with the features that are to be considered for polyfill inclusion. If not supplied, no features will be considered and the output will be blank. To load the default feature set, set features to{default:{}}
. Each feature must be an entry in the features object with the key corresponding to the name of the feature and the value an object with the following properties:flags
: Array, optional. Array of flags to apply to this feature (see below)
excludes
: Array, optional. Array of features to exclude from the final bundle.unknown
: String, optional. What to do when the user agent is not recognised. Set topolyfill
to return polyfills for all qualifying features,ignore
to return nothing. Defaults toignore
.
Flags that may be applied to polyfills are:
gated
: Wrap this polyfill in a feature-detect, to avoid overwriting the native implementationalways
: Include this polyfill regardless of the user-agent
Example:
;
getPolyfills(options)
(method)
Returns a promise of a set of features which are configured to be applied in browsers with the specified user agent string. Options is an object with the following keys:
uaString
: String, required. The user agent to evaluate for features that should be included conditionallyfeatures
: Object, optional. An object with the features that are to be considered for polyfill inclusion. If not supplied, all default features will be considered. Each feature must be an entry in the features object with the key corresponding to the name of the feature and the value an object with the following properties:flags
: Array, optional. Array of flags to apply to this feature (see below)
excludes
: Array, optional. Array of features to exclude from the final bundle.
Example:
;
listAllPolyfills()
(method)
Return a promise of an array all the polyfills as an array of strings. This list corresponds to directories and subdirectories in the /polyfills
directory.
Example:
;
License
Except where indicated in selected polyfill config files, the polyfill service codebase is licensed under the terms of the MIT license. Contributors must accept our contribution terms.