@deeptrekker/api-common

DeepTrekker API

💡 Read the service disclaimer before using the API.

Intro

This documentation defines the core types, schemas, enumerations, and constructs used by client-side applications and server side devices to communicate with each other. Together, this forms the entirety of the DeepTrekker API. This API is powered by TypeScript and the documentation and samples are generated directly from these types.

Getting Started

Refer to the documentation on how to integrate the DeepTrekker API within a client-side solution for communicating with DeepTrekker related hardware.

NPM Packages

• @deeptrekker/api-common

  To conform with our api standard, developers should incorporate this package in their client-side JavaScript/Typescript applications. This package contains a schema that defines the precise structure that payloads should be created in that will be understood when parsed by vehicle servers. It also contains well-defined types and constructs that can be directly imported when developing using the same language to streamline the development process.

  💡 Since the documentation, and its associated samples, stem from this package, it should be referred to as the single source of truth for all clients that implement our API.

  Contents

  • Version
  • API Schema
  • Samples
  • Generated Samples
    • Requests
    • States

  Set up

  This package can be installed using the following methods

  1. Install NPM package manager.
  2. Navigate to the root of your client application directory.
  3. In the terminal, run npm init to initialize.
  4. Run npm install @deeptrekker/api-common.
  5. If you're developing a JS/TS application, you can import the necessary types into your project. Otherwise, you can reference the json schema also provided for validating payloads and conforming to our API.

• @deeptrekker/sdk

  👷‍♂️ Under construction

Communication

This API supports powerful bi-directional communication between servers and clients for relaying vehicle data. Under the hood, we are using the most versatile technologies and standards available in order to accommodate communication across a variety of different technological stacks.

WebRTC

Realtime communication can be achieved via WebRTC. This is a modern web technology that powers peer-peer communication which can endure varying degrees of latency and interruptions throughout session. To start a session, clients initially have to negotiate via a https/websocket handshake with the signal server. Afterwards, they are free to communicate and exchange data in real time via numerous parallel data channels. In the temporary absence of the signalling server, existing client sessions can continue to live on.

Expected Architecture

It is expected that a 1:1 relationship exists between a peer connection and a front-end client. Clients with multiple peer connections are not supported. It is expected that one or more data and video channels are to be used within a single peer connection. A data channel named "DATA_CHANNEL_API" will be opened on connection which facilitates api communications. Through the api the client may operate vehicle controls, receive live data, and request video streams.

Upon request of a video stream, the peer connection will be renegotiated to support the requested configuration. Depending on the hardware in use this has the potential to require too many resources from the backend and cause a crash. It is suggested to keep requested video to 2 1080p streams at a maximum (on currently released hardware as of Feb 2023). If more streams are required, the lower resolution substreams may be used at the cost of visual quality.

Signal Server

To initialize a WebRTC communication room, a client would first need to contact the signalling server and negotiate a connection. This is done by performing a sequence of handshakes between a client and the signalling server. Afterwards, both the vehicle client and the controller clients can freely communicate with one another and sent/receive data.

💡 In the context of the DeepTrekker (WebRTC) API, we refer to the vehicle client of the Signal Server as the vehicle server.

SignalR

Clients can optionally integrate SignalR in client-side code to handle signalling negotiation and subsequent receive/send messages with a vehicle server. We use SignalR to encapsulate some of the complexity of WebRTC connections and use its protocols where applicable.

Documentation on how to integrate client-side SignalR code can be found here: https://docs.microsoft.com/en-us/aspnet/core/signalr/client-features?view=aspnetcore-3.1

💡 SignalR has "hubs" instead of "rooms" as a space for client communication. We will use the terms room and hub interchangeably throughout the documentation.

Manual Negotiation (No library)

For client side implementations that do not have access to a SignalR library, the following language agnostic handshake steps can be performed to establish a wss connection for communication with SignalR endpoints.

Please note that when populating the arguments field within websocket messages the value is a stringified json field matching the indicated object definitions (ie session_check, join_session, etc).

💡 Here is some documentation on websockets

Refer to the following steps that also include pictures illustrating how to perform the ceremony using Postman.

1. Send an http post request to signalling server (replace 'localhost' with the IP address of the BridgeBox)

   • URL: https://localhost:5001/sessionHub/negotiate?negotiateVersion=1
     • Query item: negotiateVersion=1 (included in URL above)

   

2. Get the "connectionToken" from the reply of the post request and use it to open a web socket to connect to signalling server (replace 'localhost' with the IP address of the BridgeBox)

   • URL: wss://localhost:5001/sessionHub?id= + connectionToken
   • Open web socket with the above URL (click "New" button beside "My workspace" bar in top left corner)

   

3. Send either of the following json objects as an initial payload via WSS protocol to connect to the HUB.

   • payload (in plain text)

     ⚠️ Ensure you copy this entire line including the invisible Record Separator "RS" ASCII character (\x1E) immediately following the "}" bracket:

     {"protocol":"json", "version":1}
     

   • base64 encoded payload (includes the "RS" character)

     eyJwcm90b2NvbCI6Impzb24iLCAidmVyc2lvbiI6MX0e
     

   💡 No other wss payloads will work until this one is sent first as the initial payload. You must include the invisible "RS" ascii character at the end of every payload thereafter as well. This is used to determine the end and start of different payloads. Read more about SignalR's use of the Record Separator.

   

4. After sending the initial json protocol payload, a payload response with an empty JSON object will be returned by the WSS endpoint.

   • The empty JSON message from server through web socket signals a successful connection to the signalling server

5. Send a session_check message. Be sure to include the "RS" ascii character at the end of any payload sent. After constructing your arguments json, stringify it and then construct the rest of the message as shown. This pattern will be followed in the rest of the sent messages.

   • Client requests list of sessions that are available to join

     {
         "arguments": ["{\"client_id\": \"test\"}"],
         "invocationId": "0",
         "streamIds": [],
         "target": "session_check",
         "type": 1
      }

6. Await a session_list message.

   • Client receives list of sessions that are available to them

     {
         "type":1,
         "target":"session_info",
         "arguments":[{
             "session_id":"33555648",
             "connection_id":"ispG-PRx-G6BiFKrjGMn1A",
             "clients":[{}]
         }]
     }

7. Send a join_session message.

   • Client requests to join an available session

     {
         "arguments": ["{\"client_id\": \"test\", session_id:\"33555648\"}"],
         "invocationId": "0",
         "streamIds": [],
         "target": "join_session",
         "type": 1
     }

8. WebRTC SDP negotiation will be initiated by the backend by sending an offer and ice candidates

WebRTC SDP Negotiation

The next part of the negotiation involves negotiating the SDP. A good video example can be found here.

💡 Ensure the JSON payloads are structured with correctly formatted values for their types. Even if it looks like a number but the interface has it as type string, it should still be wrapped in double quotes when sent in a payload to ensure error-free deserialization by strongly-typed languages.

1. Await a session_info message.

   • Client receives requested session information, should see themselves in the session_info

     {
         "type":1,
         "target":"session_info",
         "arguments":[{
             "session_id":"33555648",
             "connection_id":"ispG-PRx-G6BiFKrjGMn1A",
             "clients":[{
                 "client_id":"test",
                 "connection_id":"odQS6Kiei5y0AaaoZdO5Pw"
             }]
         }]
     }

2. Await an offer message.

   • Client receives the offer from the vehicle

     {
         "type":1,
         "target":"offer",
         "arguments":[
             "{\n  \"target\": \"odQS6Kiei5y0AaaoZdO5Pw\",\n  \"caller\": \"ispG-PRx-G6BiFKrjGMn1A\",\n  \"sessionId\": \"33555648\",\n  \"sdp\": {\n    \"type\": \"offer\",\n    \"sdp\": \"v=0\\r\\no=- 272599448504607653 2 IN IP4 127.0.0.1\\r\\ns=-\\r\\nt=0 0\\r\\na=msid-semantic: WMS\\r\\nm=application 9 UDP/DTLS/SCTP webrtc-datachannel\\r\\nc=IN IP4 0.0.0.0\\r\\na=ice-ufrag:H9aZ\\r\\na=ice-pwd:aGo/p9pm1eUS1Aw2qzbgYuQy\\r\\na=ice-options:trickle\\r\\na=fingerprint:sha-256 2D:85:37:C8:CD:64:56:11:7E:A3:65:0D:E4:76:DB:70:AA:05:5B:D9:63:4C:04:24:F2:31:83:DB:BD:C6:6D:EF\\r\\na=setup:actpass\\r\\na=mid:0\\r\\na=sctp-port:5000\\r\\na=max-message-size:262144\\r\\n\"\n  }\n}"
         ]
     }

3. Send an answer message.

   • Client sends their answer to the signaling server

     {
         "type":1,
         "target":"answer",
         "invocationId":"3",
         "arguments":[
             "{\"caller\":\"odQS6Kiei5y0AaaoZdO5Pw\",\"sdp\":{\"sdp\":\"v=0\\r\\no=- 6398202607803557374 2 IN IP4 127.0.0.1\\r\\ns=-\\r\\nt=0 0\\r\\na=msid-semantic: WMS\\r\\nm=application 9 UDP/DTLS/SCTP webrtc-datachannel\\r\\nc=IN IP4 0.0.0.0\\r\\na=ice-ufrag:heL6\\r\\na=ice-pwd:VasvuVpgy6Alpi9q4np1OGWX\\r\\na=ice-options:trickle\\r\\na=fingerprint:sha-256 66:D8:E7:9F:35:2C:68:4E:D8:01:30:F2:49:A4:D4:81:55:AB:55:8E:15:69:C2:4D:71:48:78:52:22:57:12:98\\r\\na=setup:active\\r\\na=mid:0\\r\\na=sctp-port:5000\\r\\na=max-message-size:262144\\r\\n\",\"type\":\"answer\"},\"sessionId\":\"33555648\",\"target\":\"ispG-PRx-G6BiFKrjGMn1A\"}\n"
         ]
     }

4. Send and Receive a ice_candidate messages.

   • Client and Vehicle sends their ice candidates to each other and responds with their ice-candidates

     • Receiving

       {
           "type":1,
           "target":"ice_candidate",
           "arguments":[
               "{\n  \"target\": \"odQS6Kiei5y0AaaoZdO5Pw\",\n  \"caller\": \"ispG-PRx-G6BiFKrjGMn1A\",\n  \"sessionId\": \"33555648\",\n  \"candidate\": {\n    \"sdpMid\": \"0\",\n    \"sdpMlineIndex\": \"0\",\n    \"content\": \"candidate:557811690 1 udp 2122260223 10.77.100.1 34423 typ host generation 0 ufrag H9aZ network-id 1 network-cost 50\"\n  }\n}"
           ]
       }

     • Sending

       {
           "type":1,
           "target":"ice_candidate",
           "invocationId":"4",
           "arguments":[
               "{\"caller\":\"odQS6Kiei5y0AaaoZdO5Pw\",\"candidate\":{\"content\":\"candidate:1 1 UDP 2122317823 192.168.88.254 56130 typ host\",\"sdpMid\":\"0\",\"sdpMlineIndex\":\"0\"},\"sessionId\":\"33555648\",\"target\":\"ispG-PRx-G6BiFKrjGMn1A\"}\n"
           ]
       }

5. Run application logic and utilize established Webrtc peer connection to communicate through the API (data channel DATA_CHANNEL_API).

6. Send a leave_session message when application wants to disconnect.

   • Client requests to leave the session, other clients in the session and vehicle will receive updated session info

     {
         "type":1,
         "target":"leave_session",
         "invocationId":"0",
         "arguments":["{\"client_id\":\"test\",    \"session_id\":\"33555648\"}\n"
         ]
     }

7. A session_closed message will be broadcast to all clients if the vehicle server disconnects.

Video Channel

Video streams are transmitted via a dedicated WebRTC data channel between the vehicle server and client.

💡 We refer to this channel as the video channel.

Data Channel

Another data channel exists which conveys our API messages between the vehicle server and client bidirectionally via a WebRTC connection. The payloads transmitted on this channel are structured based on the Deep Trekker API constructs shown in this document. The following sections describe these schemas and payloads in detail.

Payload

A payload is a chunk of data that can be transferred during a peer-peer WebRTC connection (vehicle server - client). This is conveyed via the data channel. For a payload to be valid, it must be a subset of the vehicle state and nested inside a TransferPayload object. We use JSON as our data-interchange format and adhere to our generated API JSON Schema for validating payloads.

JSON Schema

A JSON schema is a structure that defines how payloads should be created. It can also be used by clients and servers alike to validate incoming payloads. The schema for the DeepTrekker API can be accessed and imported from the api-common package under dist/schemas/schema.json.

💡 We use the Json Schema Validator to help construct payloads. To use this tool, simply copy and paste the DeepTrekker API JSON Schema on the left hand side, then proceed to write a TransferPayload on the right hand side. You can use this documentation as a human readable reference to help aid you in constructing the payload. As you write, the validator will provide feedback on the correctness of the input based on the schema you provided.

Structure

A valid payload is essentially a tree structure with the TransferPayload as the wrapper. The TransferPayload contains various meta data that goes along with a payload such as information about the API version compatibility and the status of the transfer.

💡 If a payload is wrapped in the TransferPayload, we refer to it as a request or response.

There are 5 different types of transferred payloads:

• Requests

  • GET
  • SET
  • UPDATE
  • DELETE

• Response

  • ERROR

Sample GET request:

{@codeblock ~~/dist/samples/generated/requests/transferGetRequest.json}

Errors are a type of response that gets returned, either as a result of a bad payload request sent or for reasons unprovoked depending on the error code supplied.

The full list of errors and their details can be seen here.

Refer to the individual sections in the table of contents for further examples on various different kinds of payloads.

💡 Samples in these sections do not contain any real mac addresses or IDs, and so, they should not be used to refer to any real device. These are only there to guide you on how to implement the correct structure. However, properties that represent enumerations or constants are an exception this. In these cases, we've explicitly linked them in the documentation (e.g. model > DeviceModel)

Parameters

Some parameters in payloads exist solely to convey information in one direction. These parameters will have one of the following attributes labelled in their documentation:

• Readonly (Server -> Client)

  Parameters labeled as readonly have values written by the server with the intention of being read by clients.

• Writeonly (Client -> Server)

  Parameters labeled as writeonly have values that are written by a client with the intention of being read by the server.

💡 The server will ignore any readonly parameters that exist in paylaods sent by clients (it will not throw away the entire payload though).

Lists

In most cases, lists of objects are organized as dictionaries in payloads and the API. This is because these objects are semi-immutable and have a unique ID associated with them. With this structure in place, accessing and merging payloads becomes a simpler process.

For example, vehicle devices do not typically change without a reboot a system. Therefore, it is reasonably safe for client to cache their IDs in memory and then utilize them throughout their code for subsequent access:

    // Typescript

    // using a previously cached device id during a session.

    state.device[cachedDeviceId]

Transfer Frequency

Payload transfers can occur very quickly. There is no limit on how fast payloads can be sent between clients and servers. It is the responsibility of the client to throttle any requests that are received depending on their use case. The vehicle server is able to handle any number of requests at any frequency by calling timeouts or throttling mechanisms accordingly without any noticeable effect to the client.

Rate Property

The rate property exists in certain payloads objects. It is a writeonly property used to continuously update a particular property in which there is no deterministic state. Each time this is sent, it will incrementally adjust the value associated nearby it in the payload.

Read more on the rate property here.

Maintaining the Vehicle state

In some cases, decoupling the state from the UI can be a useful design pattern to employ when building an application. This is especially true when utilizing declarative style libraries such as ReactJS and Redux. Since we create subsets of a vehicle state when forming payloads, it naturally makes sense to maintain the entire vehicle state (in memory or cache) and just merge incoming payloads into it. Managing an isolated state of the vehicle that UI logic can be wired up to would encourage a clean separation of concerns and should facilitate the rapid development of a reliable application that can communicate with DeepTrekker systems.

The following examples show a snapshot of vehicle states that could hypothetically exist in memory at some point during the operation of a vehicle:

{@codeblock ~~/dist/samples/generated/states/a200RandomState.json}

{@codeblock ~~/dist/samples/generated/states/a150RandomState.json}

{@codeblock ~~/dist/samples/generated/states/revolutionRandomState.json}

{@codeblock ~~/dist/samples/generated/states/pivotRandomState.json}

---

⚖️ Service Disclaimer

The Deep Trekker API ("our API library") is provided as is, without warranty of any kind, either express or implied, including but not limited to the implied warranties of merchantability and fitness for a particular purpose. We make no representations or warranties that the API library will meet your requirements or that its operation will be uninterrupted or error-free. We will not be liable for any damages of any kind arising from the use of the API library, including but not limited to direct, indirect, incidental, punitive, and consequential damages.

We do not provide technical support for integrating our API library into your software application. While we strive to make our API library easy to use and provide documentation and sample code, it is your responsibility to ensure that your software is compatible with our API library and that the integration is implemented correctly. We will not be liable for any issues or errors that arise from the integration of our API library into your software application.