Lightgate

Boomi Lightgate is a containerized JavaScript engine designed to support the execution of batch and real-time automation tasks. This technology serves as the runtime engine and a flexible and extensible framework for defining and executing automations. Lightgate simplifies the complexities involved in developing automation and integration workloads and provides functionality to normalize adapter endpoints, handle complex mapping, instrument schedule, scale leveraging core Kubernetes features, and seamlessly implement strong security, etc. Lightgate allows an automation developer to focus on writing automation tasks in a single, normalized, pipeline.

Key Features

Adapter SDK and Abstraction

Lightgate provides a Software Development Kit (SDK) for adapters, which acts as a bridge between the automation engine and various endpoint systems. This SDK abstracts away the complexities associated with interacting with different endpoints by adapting them into a user-friendly JavaScript Object-Relational Mapping (ORM). This abstraction simplifies the integration process, making it easier for developers to work with diverse endpoint systems.

Data Mapping and Aggregation

The automation engine within Lightgate incorporates advanced mapping capabilities. It allows users to map endpoint models seamlessly, providing the flexibility to aggregate data from different sources into a canonical data model. This approach enables efficient data manipulation and transformation, facilitating a unified representation of information across connected systems.

Cloud Runtime Environment

Lightgate offers a cloud runtime environment where automation tasks can be packaged and deployed effortlessly. Leveraging Kubernetes, Lightgate ensures compatibility with various cloud environments, providing users with the freedom to choose their preferred cloud infrastructure. This cloud-based approach enhances scalability, reliability, and overall performance.

Package Marketplace

Developers using Lightgate can create profiles in the cloud runtime environment. They have the option to package their automation tasks and contribute them to the Lightgate Package Marketplace. This marketplace serves as a centralized hub where developers can share their automation packages, making it easy for other users to discover, install, and utilize these pre-built solutions.

Citizen Integrator-Friendly

Lightgate promotes collaboration by empowering citizen integrators. With the ability to install packages, provide necessary credentials, and deploy fully functional code deployments with no coding or configuration skill required. This user-friendly approach ensures a seamless collaboration between developers and non-technical users. Developers can be contacted directly to modify or extend packages, perform consulting to build new packages, etc. The marketplace connects developers to business users seamlessly.

Installation and Usage

Install Lightgate using npm:

npm install lightgate

Creating an Automation

To create an automation, extend the Automation class provided by Lightgate. This base class includes methods for loading annotations and executing tasks. Lightgate leverages decorators to define tasks and their properties within your automation class. Annotations include information such as whether a method is the starting point, the next step on success or failure, how the task should be executed (async or worker threads), etc.

/**
 * This class extends the Lightgate Automation class and defines the tasks to be executed
 * @file: testAutomation.js
 */
class TestAutomation extends Automation {
  @LightgateAnnotation({
    isStart: true,
    onSuccess: 'stepOne',
    onFailure: 'logError',
  })
  async start() {
    // Implementation
    console.log('Console log from start method');
    return true;
  }

  @LightgateAnnotation({
    onSuccess: 'end',
    onFailure: 'logError',
  })
  async stepOne() {
    // Implementation
    console.log('Console log from stepOne method');
    return true
  }

  @LightgateAnnotation({
    onSuccess: 'end',
    onFailure: 'end',
  })
  async logError() {
    // Implementation
    console.log('Console log from logError method');
    return true
  }

  @LightgateAnnotation({
    isEnd: true,
  })
  async end() {
    // Implementation
    console.log('Console log from end method');
  }
}

export default TestAutomation;

Levergage an Adapter

Import an adapter into your automation and leverage an async function, I/O intensive, to perform a function.

Install the adapter via npm:

npm i lightgate-stripe-adapter

Import the adapter into your automation

import {LightgateStripeAdapter} from 'lightgate-stripe-adapter';

implement the adapter in an async function. Note: you can also create a Lightgate Task which is reusable component that can be used in many automations.

  @LightgateAnnotation({
    onSuccess: 'end',
    onFailure: 'logError',
  })
  async stepOne() {
    try {
      // Create an instance of Lightgate with options
      let lightgateInstance = new LightgateStripeAdapter({ apiKey: process.env.STRIPE_SECRET_KEY});

      // Ensure that the constructor and initialization have completed
      await lightgateInstance.initialize();

      // retrieve the customers from the stripe adapter
      const customers = await lightgateInstance.Customer.listAll();

      // Log the customers
      console.log('customers:', customers);
      return true
    } catch (error) {
      console.error('Error:', error.stack);
      return false
    }
  }

Executing Automation

Instantiate your automation class, initialize it, and execute the automation tasks. The run method automates the execution based on the defined tasks.

    try {
      // Create an instance of Lightgate with options
      testAutomationInstance = new TestAutomation();
      await testAutomationInstance.initialize();
      const response = await testAutomationInstance.run();
      if (!response) {
        throw new Error('Automation failed');
      }
      console.log('response:', response); // json response object with stats
    } catch (error) {
      console.error('Error running test automation:', error);
      expect(true).toBe(false); 
    }

Building Adapters

Lightgate supports building any kind of connected endpoint as an adapter. By extending the lightgate/adapter class, providing a connection, and models for each supported connected system type. The adapters provide the ability to abstract endpoints for lightgate automation developers. Adapters can support several authentication mechanisms including OAuth, OAuth2, OpenID Connect, and Basic auth. The adapter configuration and packing allows for these to be based to the automation developer and then further to any potential citizen integrator via the Lightgate Cloud Runtime.

Install the Lightgate SDK

npm i lightgate

Import the Adapter from lightgate

import {Adapter} from "lightgate";

class LightgateStripeAdapter extends Adapter {
  constructor(options) {
    // Add a default value for modelsDir or use an empty string if not provided
    const { packageDir = '../../lightgate-stripe-adapter', ...otherOptions } = options;
        
    // Pass the modified options to the super constructor
    super({ ...otherOptions, config: { packageDir } });
  }
}
export default LightgateStripeAdapter;

Create a connection class

This simple stripe connector leverages another open source stripe sdk which even further abstracts the complexities of the stripe endpoint for the lower code automation developer as needed. This is just an example.

import {Connection} from "lightgate";
import Stripe from 'stripe';

class StripeConnection extends Connection {
  constructor(options) {
    super(options)
    this.apiKey = options.apiKey || null;
  }

  /**
   * Connect to stripe and return the stripe instance
   * @returns 
   */
  async connect() {
    console.log('Connecting to stripe: ', this.apiKey);
    if (!this.stripe) {
      console.log('Creating new stripe instance');
      this.stripe = new Stripe(this.apiKey);
    }
  }
}

export default StripeConnection;

Build a Model

The model class maps the adapter endpoint to a javascript Object. The object can then be easily acted upon by an automation developer to perform complex functions on endpoint. These methods might start off as rudamentary CRUD operations but more mature adapters will provide more complex features. There are a number of abstract methods that the models must override such as populate(data), listAll(), findOne(id), create(data), removeOne(id), etc. The Lightgate Model class provides the abstract methods required to be implemented. These can always be extended on an adapter basis but these are the baase level requirements.

Models can be acted upon directly via the adapter. Example used within a Lightgate Automation:

const customers = await lightgateAdapterInstance.Customer.listAll();

Extend the Lightgate Model and Overried the methods above with the implementation. Note the connection is passed to the model, this is handled by the Adapter automatically and the underlying engine.

import {Model} from 'lightgate';

class Customer extends Model {
  constructor(connection) {
    super(connection);
  }

  /**
   * Retrieve all customers
   * @returns {Promise}
   */
  async listAll() {
    try {
      const response = await this.connection.stripe.customers.list();
      const mappedCustomers = await this.parseResponse(response, 'listAll');
      return mappedCustomers;
    } catch (error) {
      console.error(`Error fetching stripe customers: ${error.stack}`);
      throw error;
    }
  }
}
export default Customer