Some of the page links in this document and links to other GitLab files do not work in Confluence however, the information is available in other sections of the Confluence material.
Sample Properties can be used to help you configure the adapter in the Itential Automation Platform. You will need to update connectivity information such as the host, port, protocol and credentials.
Itential Product and opensource adapters utilize SemVer for versioning. The current version of the adapter can be found in the package.json file or viewed in the IAP GUI on the System page. All Itential opensource adapters can be found in the Itential OpenSource Repository.
Any release prior to 1.0.0 is a pre-release. Initial builds of adapters are generally set up as pre-releases as there is often work that needs to be done to configure the adapter and make sure the authentication process to Apache_airflow works appropriately.
Release notes can be viewed in CHANGELOG.md.
Itential Product adapters are built for particular versions of IAP and packaged with the versions they work with.
Itential opensource adapter as well as custom adapters built with the Itential Adapter Builder work acoss many releases of IAP. As a result, it is not often necessary to modify an adapter when upgrading IAP. If IAP has changes that impact the pronghorn.json, like adding a new required section, this will most likely require changes to all adapters when upgrading IAP.
Many of the scripts that come with all adapters built using the Itential Adapter Builder do have some dependencies on IAP or the IAP database schema and so it is possible these scripts could stop working in different versions of IAP. If you notify Itential of any issues, the Adapter Team will attempt to fix the scripts for newer releases of IAP.
These instructions will help you get a copy of the project on your local machine for development and testing. Reading this section is also helpful for deployments as it provides you with pertinent information on prerequisites and properties.
There is Adapter documentation available on the Itential Documentation Site. This documentation includes information and examples that are helpful for:
Authentication
IAP Service Instance Configuration
Code Files
Endpoint Configuration (Action & Schema)
Mock Data
Adapter Generic Methods
Headers
Security
Linting and Testing
Build an Adapter
Troubleshooting an Adapter
Others will be added over time. Want to build a new adapter? Use the Itential Adapter Builder
The following is a list of required packages for installation on the system the adapter will run on:
Node.js
npm
Git
The following list of packages are required for Itential opensource adapters or custom adapters that have been built utilizing the Itential Adapter Builder. You can install these packages by running npm install inside the adapter directory.
| Package | Description |
|---|---|
| @itentialopensource/adapter-utils | Runtime library classes for all adapters; includes request handling, connection, authentication throttling, and translation. |
| ajv | Required for validation of adapter properties to integrate with Apache_airflow. |
| axios | Utilized by the node scripts that are included with the adapter; helps to build and extend the functionality. |
| commander | Utilized by the node scripts that are included with the adapter; helps to build and extend the functionality. |
| dns-lookup-promise | Utilized by the node scripts that are included with the adapter; helps to build and extend the functionality. |
| fs-extra | Utilized by the node scripts that are included with the adapter; helps to build and extend the functionality. |
| mocha | Testing library that is utilized by some of the node scripts that are included with the adapter. |
| mocha-param | Testing library that is utilized by some of the node scripts that are included with the adapter. |
| mongodb | Utilized by the node scripts that are included with the adapter; helps to build and extend the functionality. |
| nyc | Testing coverage library that is utilized by some of the node scripts that are included with the adapter. |
| ping | Utilized by the node scripts that are included with the adapter; helps to build and extend the functionality. |
| readline-sync | Utilized by the node script that comes with the adapter; helps to test unit and integration functionality. |
| semver | Utilized by the node scripts that are included with the adapter; helps to build and extend the functionality. |
| winston | Utilized by the node scripts that are included with the adapter; helps to build and extend the functionality. |
If you are developing and testing a custom adapter, or have testing capabilities on an Itential opensource adapter, you will need to install these packages as well.
chai
eslint
eslint-config-airbnb-base
eslint-plugin-import
eslint-plugin-json
testdouble
- Set up the name space location in your IAP node_modules.
cd /opt/pronghorn/current/node_modules (* could be in a different place)
if the @itentialopensource directory does not exist, create it:
mkdir @itentialopensource- Clone/unzip/tar the adapter into your IAP environment.
cd @itentialopensource
git clone git@gitlab.com:@itentialopensource/adapters/adapter-apache_airflow
or
unzip adapter-apache_airflow.zip
or
tar -xvf adapter-apache_airflow.tar- Run the adapter install script.
cd adapter-apache_airflow
npm install
npm run lint:errors
npm run test- Restart IAP
systemctl restart pronghorn-
Create an adapter service instance configuration in IAP Admin Essentials GUI
-
Copy the properties from the sampleProperties.json and paste them into the service instance configuration in the inner/second properties field.
-
Change the adapter service instance configuration (host, port, credentials, etc) in IAP Admin Essentials GUI
Mocha is generally used to test all Itential Opensource Adapters. There are unit tests as well as integration tests performed. Integration tests can generally be run as standalone using mock data and running the adapter in stub mode, or as integrated. When running integrated, every effort is made to prevent environmental failures, however there is still a possibility.
Unit Testing includes testing basic adapter functionality as well as error conditions that are triggered in the adapter prior to any integration. There are two ways to run unit tests. The prefered method is to use the testRunner script; however, both methods are provided here.
node utils/testRunner --unit
npm run test:unit
npm run test:baseunitTo add new unit tests, edit the test/unit/adapterTestUnit.js file. The tests that are already in this file should provide guidance for adding additional tests.
Standalone Integration Testing requires mock data to be provided with the entities. If this data is not provided, standalone integration testing will fail. When the adapter is set to run in stub mode (setting the stub property to true), the adapter will run through its code up to the point of making the request. It will then retrieve the mock data and return that as if it had received that data as the response from Apache_airflow. It will then translate the data so that the adapter can return the expected response to the rest of the Itential software. Standalone is the default integration test.
Similar to unit testing, there are two ways to run integration tests. Using the testRunner script is better because it prevents you from having to edit the test script; it will also resets information after testing is complete so that credentials are not saved in the file.
node utils/testRunner
answer no at the first prompt
npm run test:integrationTo add new integration tests, edit the test/integration/adapterTestIntegration.js file. The tests that are already in this file should provide guidance for adding additional tests.
Integration Testing requires connectivity to Apache_airflow. By using the testRunner script it prevents you from having to edit the integration test. It also resets the integration test after the test is complete so that credentials are not saved in the file.
Note: These tests have been written as a best effort to make them work in most environments. However, the Adapter Builder often does not have the necessary information that is required to set up valid integration tests. For example, the order of the requests can be very important and data is often required for
createsandupdates. Hence, integration tests may have to be enhanced before they will work (integrate) with Apache_airflow. Even after tests have been set up properly, it is possible there are environmental constraints that could result in test failures. Some examples of possible environmental issues are customizations that have been made within Apache_airflow which change order dependencies or required data.
node utils/testRunner
answer yes at the first prompt
answer all other questions on connectivity and credentialsTest should also be written to clean up after themselves. However, it is important to understand that in some cases this may not be possible. In addition, whenever exceptions occur, test execution may be stopped, which will prevent cleanup actions from running. It is recommended that tests be utilized in dev and test labs only.
Reminder: Do not check in code with actual credentials to systems.
There are several node scripts that now accompany the adapter. These scripts are provided to make several activities easier. Many of these scripts can have issues with different versions of IAP as they have dependencies on IAP and Mongo. If you have issues with the scripts please report them to the Itential Adapter Team. Each of these scripts are described below.
| Run | Description |
|---|---|
| npm run adapter:install | Provides an easier way to install the adapter. |
| npm run adapter:checkMigrate | Checks whether your adapter can and should be migrated to the latest foundation. |
| npm run adapter:findPath | Can be used to see if the adapter supports a particular API call. |
| npm run adapter:migrate | Provides an easier way to update your adapter after you download the migration zip from Itential DevSite. |
| npm run adapter:update | Provides an easier way to update your adapter after you download the update zip from Itential DevSite. |
| npm run adapter:revert | Allows you to revert after a migration or update if it resulted in issues. |
| npm run troubleshoot | Provides a way to troubleshoot the adapter - runs connectivity, healthcheck and basic get. |
| npm run connectivity | Provides a connectivity check to the Apache_airflow system. |
| npm run healthcheck | Checks whether the configured healthcheck call works to Apache_airflow. |
| npm run basicget | Checks whether the basic get calls works to Apache_airflow. |
Itential Product Adapters are maintained by the Itential Product Team.
Itential OpenSource Adapters are maintained by the Itential Adapter Team and the community at large.
Custom Adapters are maintained by other sources.