An edge library for deploying computer vision applications built with Roboflow to JS environments.
This library is designed to be used within the browser, using a bundler such as vite, webpack, parcel, etc. Assuming your bundler is set up, you can install by executing:
npm install inferencejs
Begin by initializing the InferenceEngine
. This will start a background worker which is able to download and
execute models without blocking the user interface.
import { InferenceEngine } from "inferencejs";
const PUBLISHABLE_KEY = "rf_a6cd..."; // replace with your own publishable key from Roboflow
const inferEngine = new InferenceEngine();
const workerId = await inferEngine.startWorker("[PROJECT URL SLUG]", [VERSION NUMBER], PUBLISHABLE_KEY);
//make inferences against the model
const result = await inferEngine.infer(workerId, img);
Creates a new InferenceEngine instance.
Starts a new worker for the given model and returns the workerId
. Important- publishableKey
is required and can be obtained from Roboflow in your project settings folder.
Infer on n image using the worker with the given workerId
. img
can be created using new CVImage(HTMLImageElement | HTMLVideoElement | ImageBitmap | TFJS.Tensor)
or createImageBitmap
Stops the worker with the given workerId
.
The result of making an inference using the InferenceEngine
on a YOLOv8 or YOLOv5 object detection model is an array of the following type:
type RFObjectDetectionPrediction = {
class?: string;
confidence?: number;
bbox?: {
x: number;
y: number;
width: number;
height: number;
};
color?: string;
};
The result of making an inference using the InferenceEngine
on a Gaze model. An array with the following type:
type GazeDetections = {
leftEye: { x: number; y: number };
rightEye: { x: number; y: number };
yaw: number;
pitch: number;
}[];
The x position of the left eye as a floating point number between 0 and 1, measured in percentage of the input image width.
The y position of the left eye as a floating point number between 0 and 1, measured in percentage of the input image height.
The x position of the right eye as a floating point number between 0 and 1, measured in percentage of the input image width.
The y position of the right eye as a floating point number between 0 and 1, measured in percentage of the input image height.
The yaw of the visual gaze, measured in radians.
The pitch of the visual gaze, measured in radians.
A class representing an image that can be used for computer vision tasks. It provides various methods to manipulate and convert the image.
The CVImage(image)
class constructor initializes a new instance of the class. It accepts one image of one of the following types:
-
ImageBitmap
: An optionalImageBitmap
representation of the image. -
HTMLImageElement
: An optionalHTMLImageElement
representation of the image. -
tf.Tensor
: An optionaltf.Tensor
representation of the image. -
tf.Tensor4D
: An optional 4Dtf.Tensor
representation of the image.
Returns a promise that resolves to an ImageBitmap
representation of the image. If the image is already a bitmap, it returns the cached bitmap.
Returns a tf.Tensor
representation of the image. If the image is already a tensor, it returns the cached tensor.
Returns a promise that resolves to a 4D tf.Tensor
representation of the image. If the image is already a 4D tensor, it returns the cached 4D tensor.
Returns a promise that resolves to a JavaScript array representation of the image. If the image is already a tensor, it converts the tensor to an array.
Returns an array containing the dimensions of the image. If the image is a bitmap, it returns [width, height]
. If the image is a tensor, it returns the shape of the tensor. If the image is an HTML image element, it returns [width, height]
.
Disposes of the tensor representations of the image to free up memory.
Creates a new CVImage
instance from a given tensor-like array.
A fully functional example can be found in demo/demo.ts
To start the local development server, execute npm run dev
to run the demo in development mode, with hot reloading.
To run library tests execute npm run test
. To run the index.html file in dev mode with vite packaging run npm run dev
.