Library of common functions needed for Node.js Backtrace services.
$ npm install
$ npm run build
$ npm run test
Then in your code you can include by using
import * as backtrace-service from 'backtrace-service'or use require instead
const backtraceService = require('backtrace-service');To start using Service Layer Integration you have to create new object of a IdentityManager class.
After that you can use serviceRequest and authenticateRequest methods.
For an example service implementation, see here.
Services are expected to configure one or more authentication tokens, which will be distributed to
coronerd instances using the service. While a service is running, it may receive registration
requests at ${url_prefix}/service, which enable the service to integrate new coronerd instances on
the fly.
Coronerd URLs provided as the url parameter to coronerdCallback are the base URLs, for example,
https://backtrace.sp.backtrace.io/. Services may expect to append the appropriate resource for
their needs, e.g. /api/config, to reach that resource on the coronerd instance.
What data coronerd will send to service to register? Coronerd via HTTP POST will send in a request body:
- action - action name (for example register),
- url - coroner url (for example
https://yolo.sp.backtrace.io/) - service_to_coronerd_url - where the service should make requests to coronerd (default to url if omitted)
- nonce - auth token
- hmac - Hash Message Authentication Code
TypeScript:
import { ICoronerRequestOption, IdentityManager } from 'backtrace-service';
...
const idm = new IdentityManager(serviceName, serviceSecret);
app.post(
'/api/{serviceName}/service',
idm.serviceRequest({
name: serviceName,
secret: serviceSecret,
coronerdCallback: (url: string) => {
//coroner callback
},
logger: logger
} as ICoronerRequestOption),
);JavaScript:
const btservice = require('backtrace-service');
const idm = new btservice.IdentityManager(serviceName, serviceSecret);
app.post(
'${url_prefix}/service',
idm.serviceRequest({
name: 'simple',
secret: 'asdfghjk',
coronerdCallback: (url) => {
console.log('heard from coronerd at: ' + url);
},
}),
);The following options are accepted as the sole argument for the call:
- name: Name of the service (usually its type).
- secret: The shared secret that the service will use to authenticate incoming requests and config replies from a coronerd.
- coronerdCallback: A callback that takes a URL parameter, and performs any service specific setup associated with integrating a new coronerd instance.
- logger (optional): An object that has a
logfunction which can be logged to. For example, winston logger instances.
This function is intended as an additional middleware which may be used in application routes to
validate requests that involve a session token issued by a remote coronerd. The actual call can
reuse the same options argument used for serviceRequest, although it does not use
coronerdCallback.
In the route middleware list, prior to authenticateRequest, the application must attach a
req.coronerAuth object which contains:
- url: The full URL to the remote coronerd instance.
- token: The user's session token to be validated.
For example:
req.coronerAuth = {
url: "https://backtrace.sp.backtrace.io/",
token: "f5af46b8eb32adb860ef46a9e714cfde",
}
If req.coronerAuth object is undefined, authenticateRequest method will try to retrieve token
and location information from headers. If you prefer to use headers instead of extending request
object please set X-Coroner-Token and X-Coroner-Location
This normalized form is used due to the fact that different services take these parameters from clients in different ways.
Middlewares that come after authenticateRequest will have access to the validated coronerd
/api/config response in req.coronerAuth.cfg.
Example of sample middleware
TypeScript:
import { ICoronerRequestOption, IdentityManager } from 'backtrace-service';
...
@Middleware({ type: 'before' })
export class AuthMiddleware implements ExpressMiddlewareInterface {
private readonly _identityManager = new IdentityManager(serviceName, serviceSecret);
public use(req: express.Request, res: express.Response, next: express.NextFunction): any {
return this._identityManager.authenticateRequest()(req, res, next);
}
}In example above middleware will try to retrieve X-Coroner-Token and X-Coroner-Location from
request headers.
JavaScript:
const idm = new btservice.IdentityManager();
// create express app...
// prepare utility method
function prepAuth(req, res, next) {
const auth = {
token: req.get('X-Coroner-Token'),
url: req.get('X-Coroner-Location'),
};
req.coronerAuth = auth;
next();
}
// create application options
const svcOpts = {
name: 'simple',
secret: 'asdfghjk',
coronerdCallback: (url) => {
log('info', 'heard: ' + url);
},
logger: logger,
};
app.post('/api/{name}/{action}', prepAuth, idm.authenticateRequest(svcOpts), (req, res) => {
res.json(req.coronerAuth);
});Project Validation supports gating sensitive information in services on a project-basis for Backtrace Teams. The projectValidation middleware ensures all Express response.send invocations procceed successfully only if the projectValidated flag is true. This is intentionally not middleware in order to support the case where the route handler has to do some work before determining projects in scope (e.g. backtrace-sca needs to fetch correlated defects before it is aware of the projects that these defects come from). To set the projectValidation flag to true, two methods are available for use in service route handlers.
-
This is to be used in routes that use the authenticateRequest middleware. It relies on /api/config to validate the request's list of scoped projects. If the scoped list is valid, the res.send will be successful. If invalidated, the service will return a 403.
scopedProjectscan be a list of project names or pids.
service.app.get(
'/api/my-service/use-projects',
service.identityManager.authenticateRequest(),
async (req: express.Request, res: express.Response, next: express.NextFunction) => {
// ... service-related work
const scopedProjects = ["my-project"]; // Projects needed to fulfill request.
service.identityManager.checkProjectAccess(req, res, scopedProjects);
res.send({ message: "Expected result." });
}
);- For service route handler's that do not care about project access at all, you can intentionally bypass the project validation step.
service.app.get(
'/api/my-service/get-version',
async (req: express.Request, res: express.Response, next: express.NextFunction) => {
// ... service-related work
service.identityManager.bypassProjectAccessCheck(res);
res.send({ message: "Expected result." });
}
);Backtrace-Service tools offers few utility methods. By using them we want to standarize how we read application configurations/log application messages/create custom auth methods.
Service Layer Integration using IdentityManager class to provide integrations methods with
coronerd. If you want to write auth method to coronerd or check if token is valid you can use
methods available in aIdentityManager instance.
import { IdentityManager } from 'backtrace-service';
const token = //auth token
const universeUrl = //universe url: https://yolo.sp.backtrace.io
const identityManager = new IdentityManager(serviceName, serviceSecret);
// get configuration from coronerd
const configuration = await identityManager.getConfiguration(universeUrl, token);
// validate token
const validToken = await identityManager.isValidToken(universeUrl, token)A service may login to a coronerd via its base URL using this method. This performs a login using the service HMAC exchange, returning a configuration that includes a token usable for service purposes.
const cfg = await identityManager.loginCoronerd("https://foo.sp.backtrace.io");IdentityManager method that allows you to prepare a request to coronerd to retrieve configuration.
Please keep in mind, if you're using SL integration, configuration should be available in
request.coronerAuth.cfg. GetConfiguration method using Promise API to get data from a server.
IdentityManager method that allows you to validate token with coronerd. In this case
IdentityManager will return boolean information - true if coronerd returns valid configuration,
otherwise false. IsValidToken method using Promise API. Please keep in mind to use await or
then/catch syntax to get a result from method.
Backtrace-Service package allows you to read service configuration file. To standarize paths used
in your service to configuration files we suggest to use these methods to easily integrate service
with other services.
A service can obtain its service integration settings by using
getDescriptor, which returns an IServiceDescriptor. It must specify a
default port number, to be used in on-premise configurations. It may
specify additional default parameters as IDescriptorOpts.
The values returned in the IServiceDescriptor must be used by the service in order to properly integrate. This function generates a descriptor file, if necessary, otherwise it uses what's provided.
const descriptor = getDescriptor('service-name', 12345);This is a helper function that reduces boilerplate by allowing services to listen on the descriptor-specified port in the descriptor-specified manner.
The service must provide its SSL credentials (if it has any) as well as the app to process requests. It is valid for no SSL credentials to pass, although in this case, only descriptors with protocol 'http' will work.
// fill in from config file, parameter is as used by https.createServer().
const credentials = { key: "", cert: "" };
listenDescriptor(descr, app, credentials, () => {
console.log(`Listening on ${descr.port}`);
});getConfig fetches configuration from the expected place on the machine (outside the service). If
configuration doesn't exists then getConfig method will try to fetch configuration from internal
service path Algorithm:
- Try to fetch configuration from:
/etc/backtrace/${serviceName}/${serviceName}.conf - If configuration from #1 doesn't exists try to read configuration frile from path:
${process.cwd()}/${serviceName}.conf - Return config or undefined
example:
import { getBackupConfig, getProperConfig, getConfig } from 'backtrace-service';
//read backup configuration - service internal configuration
const serviceConf = getBackupConfig('service-name');
//read production configuration - outside service
const prodConf = getProperConfig('service-name');
//get configuration - depends on preferences
const conf = getConfig('service-name');Utility methods that allows you to get more information about Backtrace universe from a url.
GetBacktraceHostName method allows you to get Backtrace host name from provided urlString. For
example - method will return https://yolo.sp.backtrace.io from url
https://yolo.sp.backtrace.io/blah/poop/doop?query=123&anotherQuery=nbgaubakgb
GetBacktaceUniverseName method allows you to retrieve a Backtrace universe name. For example:
method will return yolo from url https://yolo.sp.backtrace.io or
https://submit.backtrace.io/yolo/....
example:
import { UniverseHelper } from 'backtrace-service';
const url = 'https://yolo.sp.backtrace.io/blah?a=1&b=bgj';
const url2 = 'https://submit.backtrace.io/bolo/token/json';
// method below will print a string 'https://yolo.sp.backtrace.io
console.log(UniverseHelper.getBacktraceHostName(url));
//method below will print a string 'bolo
console.log(UnvierseHelper.getBacktraceUniverseName(url2));Backtrace-service contains a lot useful methods that you can use to generate your express application with all Backtrace best practices. By default backtrace-service package allows you to:
- setup auth,
- setup service layer configuration,
- setup metrics,
- setup integration endpoints,
- setup Backtrace (logger),
- read configuration files (app conf + service descriptor),
- and even more...
We recommend to use BacktraceService.bootstrap method to generate Backtrace api, but if you want to call each methods separately, feel free to use BacktraceService static methods.
Quick sample:
const backtraceService = BacktraceService.bootstrap<IConfiguration>({
name: 'service-name',
port: 9999,
});
backtraceService.start();In TypeScript you can use generic attribute to provide configuration class. In JavaScript, you don't need to provide any generic attribute!
BacktraceService in application root directory will read configuration file. By doing that bootstrap method or BacktraceService constructor will read configuration and set configuration data in BacktraceService configuration variable.
By default BacktraceService will read from configuration file:
- metrics object,
- ssl object,
- logger object,
- backtrace object,
In addition to that BacktraceService will extend default configuration with values:
- isProduction - determine environment type,
- descriptor - service descriptor,
Example configuration file:
{
"api": {
"prefix": "/api/symbold"
},
"db": {
"type": "sqlite",
"port": 5432,
"database": "./symbol-server.db"
},
"log": {
"level": "debug",
"output": "logs",
},
"metrics": {
"enabled": false,
"secret": "fake secret",
"uuid": "fake uuid"
},
"backtrace": {
"enabled": true,
"backtraceUrl": "https://submit.backtrace.io/sample/99999fb147eccb0c1aa91f03499c96238ceaa3199dbb5cbdbb0a8ed57ae37191/json"
},
"ssl": {
"key": "path/to/key",
"cert": "path/to/cert"
}
}In example above you can see api and db object that will be readed by BacktraceService, but default method won't use these objects to generate your application.
Configuration file allows you to define remote metrics, that backtrace-service will send to Circonus/Backtrace. In case if you would like to disable them, you can use enabled property. If you will set this property to false, backtrace-service will ignore these properties and won't start Backtrace/Circonus integration.
BacktraceService.setupServiceLayer method will generate default service layer configuration for your application. By default this method will read from service/use service descriptor, setup identityManager and create application endpoint.
Usage:
const app = express();
BacktraceService.setupServiceLayer(app, descriptor);
Backtrace.setupAuth method add all security middlewares to your express application - helmet, cors.
Usage:
const app = express();
BacktraceService.setupAuth(app)
Backtrace.setupMiddlewares method will add all used middlewares/utilities to your express application - json, compression, pingdom.
Usage:
const app = express();
BacktraceService.setupMiddlewares(app);
BacktraceService.readConfiguration method allows you to read service configuration files. In addition to that (only in TypeScript), this method allows you to use strongly defined configuration file. By using generic attribute you can pass to readConfiguration method, expected configuration definition. If you want to ues default configuration object or you're JavaScript user, you can ignore generic attribute.
Usage:
const configuraiton = BacktraceService.readCofniguration('backtrace-service-name', 9999);All methods above will be used by bootstrap method to extend your express application. Usage:
const backtraceService = BacktraceService.bootstrap<IConfiguration>({
name: 'service-name',
port: 9999,
});You can start express applicaiton by using BacktraceService object. By doing that BacktraceService will use or not use ssl certs, descriptor port and more.
backtraceService.start();
BacktraceService constructor will create new MetricsStorage instance class. By doing that our application can send data to circonus. If you bootstrap your application by using bootstrap method you can access metricsStorage by retrieving backtraceService.metrics. If you don't want to use bootstrap method, you can still create MetricsStorage object by using MetricsStorage construtor.
const metrics = new MetricsStorage({uuid: 'uuid', secret: 'secret'});In addition to that, you can define sending interval. By doing that backtrace-service will send data to Circonus by using your time interval, instead of default value (1000).
If you pass to MetricsStorage construtor express application, metricsStorage will add request latency middleware to express application. By doing that MetricsStorage will send execution time of each action to Circonus.
If you want to get more details from metricsStorage object, you can use logger object to provide default logger methods.
Full sample:
const app = express();
const metrics = new MetricsStorage({uuid: 'uuid', secret: 'secret', interval: 10000}, app, logger);