Parametric Connector Framework (PCF) follows a declarative paradigm and aims to make modern IDE the new user interface for modeling & synchronizing data with digital twins. It allows you to define your iModel as code then it takes care of the steps to synchronize it with external data to your desired state. It requires minimal programming experience and domain knowledge so any developer can quickly write a Connector that just works and scales.
# make sure your npm version is < 7.0.0
npm --version
# run this command if you npm version is > 7.0.0
npm install -g npm@6.x# 1. install global pcf command line utility
npm install -g @itwin/pcf-cli
# 2. initialize a connector template. Change "MyProject" to your desired project name.
pcf init MyProject
cd MyProject
# 3. install the latest version of pcf core in your project
npm install @itwin/pcf
# 4. add application specific info in ./src/App.ts
# 5. execute your connector (compilation is included in this command)
npm run startYou usually do not need to install any iTwin.js related dependencies besides domain schema npm packages (`@bentley/${schemaName}-schema`) since most iTwin.js libraries are automatically installed as you install @itwin/pcf. If you do, you must make sure that they share the same version string as the ones installed by PCF, otherwise you may encounter unknown errors.
Currently, all the documentations and API references of this project are embedded in source files. Use your IDE/language server to look them up through go-to-definitions.
You will be using a set of constructs to build your connector.
| Name | Definition |
|---|---|
| IR Model | An intermediate representation virtual Entity-Relationship store generated by a Loader from your source data. It is made of three classes: IR Entity = External Class (e.g. database table). IR Relationship = External Relationship Class (e.g. database link table or table that contains foreign key relationships). IR Instance = the instances of an External Class (e.g. a row in database table) |
| Loader | An accessor to a data source. Responsible for converting the source data into an IR Model. You may use an existing Loader or write your own. |
| DMO | A DMO (Dynamic Mapping Object) defines the mappings between IR Model and iModel. |
| Node | A Node corresponds to an EC Entity and some Nodes use DMO to populate multiple EC Instances. An iModel is synchronized based on user-defined Nodes. |
You may have difficulty understanding this tutorial without a basic understanding of BIS.
It's important to first see an overall picture of what a Connector does:
- Read source data
- Define mappings (and transform geometries)
- Attach mappings to the iModel elements
Next, we will go over them in order and show how they are handled by PCF constructs.
X could be any data source. (e.g., database, spreadsheet, API, etc.)
Similar to a keyboard driver for an operating system, Loader makes your data available to be consumed by a universal connector that is configured through Node & DMO definitions. It is a lightweight object that's responsible for converting a type of source data into an intermediate representation model (IR Model) in memory, which can be interfaced and optimized independently of the source data.
You may need to write your own Loader if you need to customize the way of accessing source data. But before deciding to write one yourself, check out the existing ones or consider extending them. All loaders must extend the base class Loader.
IR Model is meant to be generic to represent all types of external data models. It is an in-memory store that consists of two types of object, IR Entity and IR Relationship, whose instances are called IR Instances.
A collection of DMO's is the Single Source of Truth of the mappings from source data to an iModel. Each DMO controls the one-to-one mapping from an IR class to an EC class in iModel. PCF provides a default property mapping from IR Instance to iModel Element, in addition, DMO's could attach callbacks on each visit to override element property values.
export const dmoA: ElementDMO = {
irEntity: "ExternalClassA",
// use an existing class from BisCore schema
ecElement: "BisCore:SpatialCategory",
};
export const dmoB: pcf.ElementDMO = {
irEntity: "ExternalClassB",
// create a dynamic class in iModel
ecElement: {
name: "ExternalClassB",
baseClass: PhysicalElement.classFullName,
properties: [
{
name: "BuildingNumber",
type: primitiveTypeToString(PrimitiveType.String),
},
{
name: "RoomNumber",
type: primitiveTypeToString(PrimitiveType.String),
},
],
},
// callback to override property values or attach geometry streams to current element
modifyProps(props: any, instance: IRInstance) {
props.buildingNumber = instance.get("building_id");
props.roomNumber = instance.get("room_number");
// props.geom = <build geometry stream>
},
// callback to skip syncing an element
doSyncInstance(instance: IRInstance) {
return isValidId(instance.get("id")) ? true : false;
},
// find class B instances with this foreign key column
categoryAttr: "external_class_b_id",
};Note:
- Only Primitive EC Properties can be added to DMO.ecElement/ecRelationship. They cannot be deleted once added.
A collection of Nodes is the Single Source of Truth of the hierarchy of a subject tree in iModel. You now gain the freedom to organize the content of your iModel as if it's a file system by passing around Nodes. It's important to know that the ordering of Nodes matters as they are synchronized in the same order as defined. Since the dependencies between Nodes are constrained by the fact that a variable cannot be referenced until it's defined in a programming language, we can guarantee that the elements inside an iModel are always synchronized in the correct order without hardcoding the logic anywhere.
ElementNode & RelationshipNode must attach a DMO so that they can populate multiple instances of EC Elements & Relationships in iModel based on the instances of external data.
import * as pcf from "@itwin/pcf";
import { dmoA, dmoB } from "./dmos/Elements.ts";
import { dmoC } from "./dmos/RelatedElements.ts";
export class SampleConnector extends pcf.PConnector {
public async form() {
// skip some config ...
const subjectA = new pcf.SubjectNode(this, { key: "SubjectA" });
const defModel = new pcf.ModelNode(this, { key: "ModelA", subject: subjectA, modelClass: DefinitionModel, partitionClass: DefinitionPartition });
const phyModel = new pcf.ModelNode(this, { key: "ModelB", subject: subjectA, modelClass: PhysicalModel, partitionClass: PhysicalPartition });
const sptCategory = new pcf.ElementNode(this, { key: "CategoryA", model: defModel, dmo: dmoA });
const phyElement = new pcf.ElementNode(this, { key: "ElementB", model: phyModel, dmo: dmoB, category: sptCategory });
const assembly = new pcf.RelatedElementNode(this, {
key: "PhysicalElementAssemblesElementsA",
subject: subjectA,
dmo: dmoC,
source: phyElement,
target: phyElement,
});
}
}Read the following material if you're not familiar with concepts such as Element, Model and Relationship in iModel:
Great articles to learn some background information to help you organize Nodes
Note:
- The following entity class cannot be deleted from your iModel once created: Subject, Partition, Model.
- Modifying the key of SubjectNode or ModelNode would cause new Subject, Model, and Partition to be created.
- Parent-child Modeling is not supported yet. Only the top models and their elements are synchronized.
Everything doesn't have to be static. You still have the freedom to dynamically generate DMO's & Nodes based on a set of rules and data. Since your connector is represented purely by objects (DMOs & Nodes), you can programmatically generate them based on external source data. See a sample below.
// inside XYZConnector.ts where Nodes are defined:
import * as pcf from "@itwin/pcf";
export class XYZConnector extends pcf.PConnector {
public async form() {
...
const model = new pcf.ModelNode(...);
const data: any[] = // Define any logics to get the data necessary to generate PCF construct instances
for (const item of data) {
// use data stored in "item" to populate the fields of DMO and/or Node
const dmo: pcf.ElementDMO = ...
const node = new pcf.ElementNode(this, { ..., model, dmo } );
}
}
}"Wait… don't we need to write tests to ensure that the elements are correctly inserted/updated/deleted in all kinds of scenarios?"
PCF aims to eliminate the need for end applications to write tests. Looking back at what we did, we defined a bunch of objects as the inputs to PCF, which would handle the rest to synchronize the target iModel to our desired state through the objects. So long as their definitions are correct, PCF promises a successful synchronization.
"Okay… people make mistakes in configuration files all the time, how can I be confident that my definitions are correct for the objects?"
PCF enforces strict typing on objects through TypeScript so that functionalities such as code completion and code-refactoring available in most modern IDE's (e.g. Visual Studio Code) will help you to write the correct definitions for them.
Though runtime errors are minimized, there are still a few types of runtime errors that could not be discovered at compile time. For example, you may accidentally assign a PhysicalElement to a FunctionalModel this will fail because they are not of the same type. You should still have a very basic understanding of how information is organized in an iModel.
"Where did the API documentation for PCF go?"
They are all embedded in code. You will be working in a single context, your modern IDE. Why? You tend to do this anyway as you code and you always see the correct version of the documentation.
# clone PCF repo
git clone https://github.com/iTwin/pcf.git
# install PCF core dependencies
cd core
npm ci
# build
npm run build# create global symlink
npm link
# use @itwin/pcf package locally without installing it
cd <your connector project dir>
npm link @itwin/pcfnpm run test