We are looking for Contributors and Maintainers!
MindSphere has evolved into Insights Hub. This library still uses the name MindSphere in class names etc. for compatibility reasons.
NodeJS Library for Insights Hub and Industrial IoT Connectivity - TypeScript SDK for Insights Hub - Insights Hub Command Line Interface - Insights Hub Development Proxy
The full documentation can be found at https://developer.siemens.com/industrial-iot-open-source/mindconnect-nodejs/index.html
There are several ways to install the library. The most common one is via npm registry:
# install the latest stable library from the npm registry
npm install @mindconnect/mindconnect-nodejs --save
# install the latest alpha library from the npm registry
npm install @mindconnect/mindconnect-nodejs@alpha --save
The easiest way to start is to use the provided command line interface to create a starter project:
# for typescript nodejs project run
npx @mindconnect/mindconnect-nodejs starter-ts
# for javascript nodejs project run
npx @mindconnect/mindconnect-nodejs starter-jsThe following steps describe the easiest way to test the library. You can of course create the required dependencies also programmatically via API calls.
Insights Hub V3 IoT model requires that you create an asset type and aspect types to describe your assets. For the example we will create an asset type of type Engine with two aspect types: Environment and Vibration. (Note that my tenant is called castidev, you will have to use your own tenant name)
More information about Insights Hub Data Model.
Create an asset (In example it is called AcmeMotor) of type Engine in AssetManager for your data.
Create an agent in Asset Manager of type core.MindConnectLib create initial JSON token and store it to file (e.g. agentconfig.json)
{
"content": {
"baseUrl": "https://southgate.eu1.mindsphere.io",
"iat": "<yourtokenishere>",
"clientCredentialProfile": [
"SHARED_SECRET"
],
"clientId": "a3ac5ae889544717b02fa8282a30d1b4",
"tenant": "<yourtenantishere>"
},
"expiration": "2018-04-06T00:47:39.000Z"
}Read the initial configuration from the config file and create the agent. If you are using the SHARED_SECRET profile there is no need to setup the local certificate for the communication (recommended for smaller devices).
const configuration = require("../../agentconfig.json");
const agent = new MindConnectAgent(configuration);If you want to use the RSA_3072 profile you must also set up the agent certificate.
// you can create the private.key for example using openssl:
// openssl genrsa -out private.key 3072
agent.SetupAgentCertificate(fs.readFileSync("private.key"));The first operation is onboarding of the agent. This creates a client secret which is used for the communication with Insights Hub.
This data is stored by default in the .mc folder in your application if you don't change the base path in the constructor of the agent.
Important: Make sure that your folder with the configurations is not reachable from the internet as it contains the client_secret for the authentication.
if (!agent.IsOnBoarded()) {
await agent.OnBoard();
}Important: From the version 3.8.0 it is possible to create the data source configuration and data mappings fully automatic via code
First you need to create a data source configuration
// create data source configuration for an asset type castidev.Engine
const generatedConfig = await agent.GenerateDataSourceConfiguration(`${agent.GetTenant()}.Engine`);
await agent.PutDataSourceConfiguration(generatedConfig);and then the data mappings:
const mappings = await agent.GenerateMappings(targetAssetId);
await agent.PutDataMappings(mappings);The agents have now access to Insights Hub TypeScript SDK (in beta)
agent.Sdk();
// the sdk gives you access to e.g. asset management client with which you can get asset types or assets from Insights Hub
// which can be used for automatic data source configuration and automatic mappings
const assetMgmt = agent.Sdk().GetAssetManagementClient();
await assetMgmt.GetAssets(...);
await assetMgmt.GetAspectTypes(...);If you take a look at the Insights Hub configuration of your agent now it should look like this:
And the data mappings should be in place
for (let index = 0; index < 5; index++) {
const values: DataPointValue[] = [
{ "dataPointId": "DP-Temperature", "qualityCode": "0", "value": (Math.sin(index) * (20 + index % 2) + 25).toString() },
{ "dataPointId": "DP-Pressure", "qualityCode": "0", "value": (Math.cos(index) * (20 + index % 25) + 25).toString() },
{ "dataPointId": "DP-Humidity", "qualityCode": "0", "value": ((index + 30) % 100).toString() },
{ "dataPointId": "DP-Acceleration", "qualityCode": "0", "value": (1000.0 + index).toString() },
{ "dataPointId": "DP-Frequency", "qualityCode": "0", "value": (60.0 + (index * 0.1)).toString() },
{ "dataPointId": "DP-Displacement", "qualityCode": "0", "value": (index % 10).toString() },
{ "dataPointId": "DP-Velocity", "qualityCode": "0", "value": (50.0 + index).toString() }
];
// there is an optional timestamp parameter if you need to use something else instead of Date.now()
const result = await agent.PostData(values);
}(If you were using UI to configure data mappings you will have long integers instead of human-readable data point Ids.)
If you don't want to send the data points one by one, you can also use the BulkPostData method
const bulk: TimeStampedDataPoint[] =
[{
"timestamp": "2018-08-23T18:38:02.135Z",
"values":
[{ "dataPointId": "DP-Temperature", "qualityCode": "0", "value": "10" },
{ "dataPointId": "DP-Pressure", "qualityCode": "0", "value": "10" }]
},
{
"timestamp": "2018-08-23T19:38:02.135Z",
"values": [{ "dataPointId": "DP-Temperature", "qualityCode": "0", "value": "10" },
{ "dataPointId": "DP-Pressure", "qualityCode": "0", "value": "10" }]
}];
await agent.BulkPostData (bulk);Events can now be created with the library. You can create events for your agent or for your entities. In order to create an event for your entity you need to know the assetId of the asset.
const configuration = require("../../agentconfig.json");
const agent = new MindConnectAgent(configuration);
if (!agent.IsOnBoarded()) {
await agent.OnBoard();
}
const event: MindsphereStandardEvent = {
"entityId": configuration.content.clientId, // use assetid if you dont want to store event in the agent :)
"sourceType": "Event",
"sourceId": "application",
"source": "Meowz",
"severity": 20, // 0-99 : 20:error, 30:warning, 40: information
"timestamp": new Date().toISOString(),
"description": "Test"
};
// send event with current timestamp
await agent.PostEvent(event);
Files can now be uploaded via the library. You can upload files for your agent or for your entities. In order to create an event for your entity you need to know the assetid of the asset.
Since version 3.5.1. the agents are using the multipart upload API of the Insights Hub. This means that the agents can upload files also bigger > 8 MB, The multipart upload must be switched on (chunk:true) if you want to activate this behavior. The parameter parallelUploads determine the maximal number of parallel uploads. You can increase this on a powerful computer to speed up the upload or decrease to prevent network congestion.
const configuration = require("../../agentconfig.json");
const agent = new MindConnectAgent(configuration);
if (!agent.IsOnBoarded()) {
await agent.OnBoard();
}
await agent.UploadFile(agent.ClientId(), "custom/mindsphere/path/package.json", "package.json", {
retry: RETRYTIMES,
description: "File uploaded with MindConnect-NodeJS Library",
parallelUploads: 5,
chunk: true
});
Here is a demo agent implementation.
You can wrap all asynchronous object calls into the retry function which will automatically retry the operation for n times before throwing an exception.
import { MindConnectAgent, MindsphereStandardEvent, retry, TimeStampedDataPoint } from "@mindconnect/mindconnect-nodejs";
// if you want to be more resillient you can wrap every async method
// in the retry function which will try to retry several times before throwing an exception
// onboarding over flaky connection
await retry (5, ()=>agent.OnBoard())
// bulk upload with 5 retries
const bulk: TimeStampedDataPoint[] =
[{
"timestamp": "2018-08-23T18:38:02.135Z",
"values":
[{ "dataPointId": "DP-Temperature", "qualityCode": "0", "value": "10" },
{ "dataPointId": "DP-Pressure", "qualityCode": "0", "value": "10" }]
},
{
"timestamp": "2018-08-23T19:38:02.135Z",
"values": [{ "dataPointId": "DP-Temperature", "qualityCode": "0", "value": "10" },
{ "dataPointId": "DP-Pressure", "qualityCode": "0", "value": "10" }]
}];
await retry(5, () => agent.BulkPostData(bulk));The data in the Insights Hub can be observed in the fleet manager.
Set the http_proxy or HTTP_PROXY environment variable if you need to connect via proxy.
# set http proxy environment variable if you are using e.g. fiddler on the localhost.
export HTTP_PROXY=http://localhost:8888The library comes with the typescript SDK which can be used to access Insights Hub APIs
It implements support for both frontend (browser e.g. angular, react...) and backend development in node.js. and different Insights Hub authentication types:
Frontend: - Browser (Session, Cookies)
Backend (node.js): - UserCredentials - AppCredentials - ServiceCredentials - Insights Hub Agents
| Name | SDK - Client | Command |
|---|---|---|
| Identity Management | ✔️ | ✔️ |
| Resource Access Management1 | ✔️ | ✔️ |
| Oauth Authorization | ✔️ | ✔️ |
| Tenant Management | ✔️ | ✔️ |
| Token Management | ✔️ | ✔️ |
| Message Broker2 (preview) | ✔️ | ✔️ |
| Usage Transparency | ✔️ |
1 In the first stage of the availability Resource Access Management must be enabled for the tenant via Insights Hub support team.
2 Message Broker is only available on preview tenants
| Name | SDK - Client | Command |
|---|---|---|
| IoT File | ✔️ | ✔️ |
| IoT Time Series | ✔️ | ✔️ |
| IoT TS Aggregates (v3, v4) | ✔️ | ✔️ |
| IoT TS Bulk | ✔️ | ✔️ |
| Integrated Data Lake | ✔️ | ✔️ |
| Name | SDK - Client | Command |
|---|---|---|
| Agent Management | ✔️ | ✔️ |
| MindConnect API | ✔️ | ✔️ |
| Commanding API (sync) | ✔️ | ✔️ |
| OPC UA PubSub | ✔️ |
| Name | SDK - Client | Command |
|---|---|---|
| Asset Management | ✔️ | ✔️ |
| Event Management | ✔️ | ✔️ |
| Notification | ✔️ | ✔️ |
| WorkOrder Management | ✔️ | ✔️ |
| Visual Flow Creator | ✔️ | ✔️ |
| Name | SDK - Client | Command |
|---|---|---|
| Anomaly Detection | ✔️ | ✔️ |
| Data Exchange | ✔️ | ✔️ |
| Event Analytics | ✔️ | ✔️ |
| Job Manager | ✔️ | ✔️ |
| KPI Calculation | ✔️ | ✔️ |
| Model Management | ✔️ | ✔️ |
| Signal Calculation | ✔️ | ✔️ |
| Signal Validation | ✔️ | ✔️ |
| Spectrum Analysis | ✔️ | ✔️ |
| Trend Prediction | ✔️ | ✔️ |
| Name | SDK - Client | Command |
|---|---|---|
| Device Management | ✔️ | ✔️ |
| Device Status | ✔️ | ✔️ |
| Deployment Workflow | ✔️ | ✔️ |
| Device Configuration | ✔️ | ✔️ |
| Edge App Deployment | ✔️ | ✔️ |
| Edge App Instance Management | ✔️ | ✔️ |
| Firmware Deployment | ✔️ | ✔️ |
| Name | SDK - Client | Command |
|---|---|---|
| SDI Data Management | ✔️ | ✔️ |
| SDI Data Query | ✔️ | ✔️ |
| SDI Semantic Modelling | ✔️ | ✔️ |
The example below shows how to use the sdk.
// The example shows how to Get Assets from Insights Hub with custom AssetType using frontend authentication
// you can pass an instance an Authorizer (BrowserAuth, UserAuth, CredentialsAuth, TokenManagerAuth, MindConnectAgent)
// to use different authorization types in Insights Hub or implement the TokenRotation interface if you want to
// provide your own authorizer.
//
// The default constructor uses frontend authorization.
const sdk = new MindSphereSdk ();
const am = sdk.GetAssetManagementClient();
const assets = await am.GetAssets({
filter: JSON.stringify({
typeId: {
startsWith: `${tenant}`,
},
}),
});
// you will get fully typed assets in responseIf an API is missing and you would like to contribute a Client for it take a look at CONTRIBUTING.md.
The full documentation for the command line interface can be found at
The library comes with a command line interface which can also be installed globally. You can use the command line mode to upload timeseries, files and create events in the Insights Hub.
# install the library globaly if you want to use its command line interface.
npm install -g @mindconnect/mindconnect-nodejsThe library can also be downloaded as executable from the GitHub release page.
Download the version for your system and place it in folder which is in your PATH.
-
mc.exefor windows -
mc-macosfor macOS -
mc-linuxfor Linux
Linux, macOS: Rename the file to mc and make sure that the file is marked as executable (chmod +x).
First step is to configure the CLI. For this you will need a session cookie from Insights Hub, service credentials (which have been deprecated) or application credentials from your developer cockpit.
First start the credentials configuration. This will start a web server on your local computer where you can enter the credentials.
# run mc service-credentials --help for full information
$ mc service-credentials
navigate to http://localhost:4994 to configure the CLI
press CTRL + C to exit
Navigate to http://localhost:4994 to configure the CLI. (see full documentation for XSRF-TOKEN and SESSION)
The image below shows the dialog for adding new credentials (press on the + sign in the upper left corner)
You can get the application credentials from your developer or operator cockpit in Insights Hub. (if you don't have any application you can register a dummy one just for CLI)
Once configured you can press CTRL + C to stop the configuration server and start using the CLI. Remember the passkey you have created as you will be using it with almost all CLI commands.
The CLI can be used to create starter projects, upload timeseries, events and files, read agent diagnostics etc. The CLI commands should only be used in secure environments! (e.g on your working station, not on the agents).
Here is an overview of CLI commands:
# run mc --help to get a full list of the commands
mc --helpUsage: mc [options] [command]
Options:
-V, --version output the version number
-h, --help display help for command
Commands:
onboard|ob [options] onboard the agent with configuration stored in the config file
configure-agent|co [options] create data source configuration and mappings (optional: passkey) *
agent-token|atk [options] displays the agent token for use in other tools (e.g. postman)
upload-timeseries|ts [options] parse .csv file with timeseriesdata and upload the timeseries data to mindsphere
upload-file|uf [options] upload the file to the mindsphere file service (optional: passkey) *
create-event|ce [options] create an event in the mindsphere (optional: passkey) *
agent-status|as [options] displays the agent status and agent onboarding status *
create-agent|ca [options] create an agent in the mindsphere *
offboard-agent|of [options] offboards the agent in the mindsphere *
renew-agent|rn [options] renews the agent secrets *
service-credentials|sc [options] provide login for commands which require technical user credentials *
service-token|stk [options] displays the service token for use in other tools (e.g. postman) *
register-diagnostic|rd [options] register agent for diagnostic *
get-diagnostic|gd [options] get diagnostic information *
unregister-diagnostic|ud [options] unregister agent from diagnostic *
prepare-bulk|pb [options] creates a template directory for timeseries (bulk) upload *
run-bulk|rb [options] runs the timeseries (bulk) upload job from <directoryname> directory *
check-bulk|cb [options] checks the progress of the upload jobs from <directoryname> directory *
download-bulk|db [options] download the timeseries data in bulk from mindsphere *
asset-lock|lck [options] lock/unlock asset model modifications *
asset-info|ai [options] get infos about asset *
assets|ast [options] list, create or delete assets *
asset-types|at [options] list, create or delete asset types *
aspects|as [options] list, create or delete aspects *
event-types|et [options] list, create or delete event types *
events|ev [options] list, create or delete events *
events-bulk|dn [options] download or delete the events in bulk *
aggregates|ag [options] list timeseries aggregates *
notifications|nt [options] send email, sms and push notifications *
oe-device-types|oedt [options] list, create or delete device types (open edge) *
oe-devices|oed [options] list, create or delete (open edge) devices *
oe-device-status|oeds [options] list, get, or update (open edge) device status information *
oe-app-inst|oeai [options] list, create, configure or delete app instance (open edge) *
oe-app-deploy|oead [options] list, create, update app installation task(s) (open edge) *
oe-deploy-workflow|oedw [options] list, create/instantiate, update or delete/cancel workflow deployment model or instance(s) (open edge) *
oe-firm-deploy|oefd [options] list, create, update firmware deployment task(s) (open edge) *
tenant|ti [options] create or delete tenant legal configuration and logo *
subtenants|st [options] list, create or delete subtenants *
list-assets|la [options] list assets in the tenant *
delete-asset|da [options] delete asset with id <assetid> from mindsphere *
list-files|ls [options] list files stored with the asset *
download-file|df [options] download the file from mindsphere file service *
delete-file|de [options] delete the file from mindsphere file service *
identity-management|iam [options] manage mindsphere users and groups *
data-lake|dlk [options] manage data lake, data lake access permissions and STS tokens *
sdi-data-lakes|sdl [options] manage data lakes for SDI *
sdi-data-registries|sdr [options] manage data registries for SDI *
sdi-iot-registries|sdt [options] manage iot data registries for SDI *
sdi-data-types|sdy [options] manage data types for SDI *
sdi-file-upload|sdu [options] upload file to SDI *
sdi-ingest-jobs|sdj [options] manage ingest jobs for SDI *
sdi-search-schemas|sds [options] search SDI schemas *
sdi-data-queries|sdq [options] manage data queries for SDI *
sdi-execution-jobs|sdx [options] manage data execution jobs for SDI *
sdi-ontologies|sdo [options] manage ontologies for SDI *
sdi-ontology-jobs|sdb [options] manage ontology jobs for SDI *
mobile-apps|mb [options] list, create or delete mobile apps *
mobile-app-instances|mbi [options] list, create or delete mobile app instances *
signal-calculation|cal [options] process timeseries data *
kpi-calculation|kp [options] calculate kpi states or compute kpis @
event-analytics|ea [options] analyze mindsphere events @
models|ml [options] list, create or delete analytic models *
jobs|jb [options] list, create or stop jobs *
schedules|js [options] list, create, start, stop or delete job schedules *
data-exchange|dx [options] list, upload, download and manage data exchange files and directories *
anomaly-detection|ad [options] train anomaly detection models and detect timeseries anomalies *
dev-proxy|px [options] starts mindsphere development proxy & (optional passkey) *
mqtt-createjwt|jw [options] creates a signed token for opcua pub sub authentication #
starter-ts|st [options] creates a starter project in typescript #
starter-js|sj [options] creates a starter project in javascript #
help [command] display help for command
Documentation:
the magenta colored commands * use app or service credentials or borrowed mindsphere cookies
the cyan colored commands require mindconnectlib (agent) credentials
the blue colored commands @ use analytical functions of Insights Hub
the green colored commands # are used as setup and utility commands
the yellow colored commands & use borrowed mindsphere application cookies
the credentials and cookies should only be used in secure environments
Full documentation: developer.siemens.com/industrial-iot-open-source/index.html
The CLI comes with a development proxy which can be used to kickstart your Insights Hub development. It provides an endpoint at your local machine at
which will authenticate all requests using either a borrowed SESSION and XSRF-TOKEN cookie from Insights Hub or the the configured app credentials or service credentials.
The command below will start your development proxy without any installation and configuration (you just need the cookies from an existing app):
npx @mindconnect/mindconnect-nodejs dev-proxy --session <SESSION-TOKEN> --xsrftoken <XSRF-TOKEN> --host <appname>.<tenant>.<region>.mindsphere.ioFor more complex tasks install and configure the CLI
Usage: mc dev-proxy|px [options]
starts mindsphere development proxy (optional passkey) *
Options:
-m, --mode [credentials|session] service/app credentials authentication of
session authentication (default: "session")
-o, --port <port> port for web server (default: "7707")
-r, --norewrite don't rewrite hal+json urls
-w, --nowarn don't warn for missing headers
-d, --dontkeepalive don't keep the session alive
-v, --verbose verbose output
-s, --session <session> borrowed SESSION cookie from brower
-x, --xsrftoken <xsrftoken> borrowed XSRF-TOKEN cookie from browser
-h, --host <host> the address where SESSION and XSRF-TOKEN
have been borrowed from
-t, --timeout <timeout> keep alive timeout in seconds (default:
"60")
-k, --passkey <passkey> passkey
--help display help for command
Examples:
mc dev-proxy runs on default port (7707) using cookies
mc dev-proxy --port 7777 --passkey passkey runs on port 7777 using app/service credentials
Configuration:
- create environment variables: MDSP_HOST, MDSP_SESSION and MDSP_XSRF_TOKEN using borrowed cookies
see more documentation at https://developer.siemens.com/industrial-iot-open-source/mindconnect-nodejs/development-proxy.html
This project has been released under an Open Source license. The release may include and/or use APIs to Siemens’ or third parties’ products or services. In no event shall the project’s Open Source license grant any rights in or to these APIs, products or services that would alter, expand, be inconsistent with, or supersede any terms of separate license agreements applicable to those APIs. “API” means application programming interfaces and their specifications and implementing code that allows other software to communicate with or call on Siemens’ or third parties’ products or services and may be made available through Siemens’ or third parties’ products, documentations or otherwise.