@techolution-ai/computer-vision

A JavaScript/TypeScript library for computer vision applications, providing tools for image processing, scanning, and MQTT-based messaging.

Features
Usage
API Reference
Requirements
License
Contributing
Support

Features

The library provides several modules:

Core computer vision functionality
Scanner module for image/video scanning
MQTT-based messaging system for real-time communication

Usage

Stream the video

Use scanner to stream the camera's live recording to web using useScanner hook.

import { useMemo } from 'react';

import { useScanner } from '@techolution-ai/computer-vision/scanner';

function App(){

  const scanner = useScanner({ baseUrl: 'http://0.0.0.0:8501' })
  const streamUrl = useMemo(()=>scanner.getVideoStreamUrl(),[scanner]);

  // render the image with steaming url
  return <ScanImage src={streamUrl} />
}

Listening to MQTT messages

We can use useMessages to connect, subscribe to certain topics, and can receive the messages on onMessage callback.

We must use the MessagesProvider to use useMessages hook. It can be put at the root (or one level up) to the component's tree.

import {
  MessagesProvider,
  useMessages,
} from '@techolution-ai/computer-vision/messages'


function MessagesListener() {

  const handleReceiveMessage = (topic: string, message: any) => {
    // your topic and message is here
    // add your logic here
  }

  useMessages({
    topics: ['current_status', 'products_scanned'], // subscribing to topics
    onMessage: handleReceiveMessage, // message handler
  })
}

function App(){
  <MessagesProvider url="ws://0.0.0.0:9001">
    <MessagesListener />
  </MessagesProvider>
}

Scanner with MQTT with UI

We can combine above examples to do a full integration.

We also have a ready-to-use react component: ScannerStatus with default options added which can be used to show different statuses in the UI.

import { useMemo, useState } from 'react';

import {
  ScanImage,
  useScanner,
  ScannerStatus,
  defaultStatusMap,
  TStatusMap,
} from '@techolution-ai/computer-vision/scanner';

import {
  MessagesProvider,
  useMessages,
} from '@techolution-ai/computer-vision/messages';

function MessagesListener() {
  const scanner = useScanner({ baseUrl: 'http://0.0.0.0:8501' })

  const streamUrl = useMemo(() => scanner.getVideoStreamUrl(), [scanner])

  const [statusMap, setStatusMap] = useState<TStatusMap | null>(
    defaultStatusMap,
  )

  const [status, setStatus] = useState('idle')
  const [messages, setMessages] = useState<{ topic: string; message: any }[]>([])

  const handleReceiveMessage = (topic: string, message: any) => {
    try {
      const value = JSON.parse(message.toString())

      if (topic === 'current_status') {
        setStatus(value.status)
      } else if (topic === 'item_scanned') {
        setMessages((prev) => [...prev, { topic, message: value }])
      }
    } catch (e) {
      console.error('Error parsing message:', e)
    }
  }

  useMessages({
    topics: ['current_status', 'item_scanned'],
    onMessage: handleReceiveMessage,
  })

  return (
    <ScannerStatus
      style={{
        border: '4px solid #4b2f99',
        borderRadius: 16,
        overflow: 'hidden',
        position: 'relative',
        width: '100%',
        height: '100%',
      }}
      status={status}
      statusMap={statusMap}
    >
      <ScanImage
        src={streamUrl}
        style={{
          width: '100%',
          height: '100%',
          objectFit: 'cover',
          position: 'absolute',
          top: 0,
          left: 0,
          bottom: 0,
          right: 0,
        }}
      />
    </ScannerStatus>
  )
}

function App() {
  return (
    <MessagesProvider url="ws://0.0.0.0:9001">
      <MessagesListener />
    </MessagesProvider>
  )
}

API Reference

The library exposes three main modules:

Scanner Module
- Specialized scanning capabilities
- Import via @techolution-ai/computer-vision/scanner
Messages Module
- MQTT-based messaging system
- Import via @techolution-ai/computer-vision/messages

MessagesProvider (component)

Provider for MQTT connection.

Props

Prop	Type	Default Value	Remarks
url	string	--	Required
enableDebugging	boolean	false	Optional, to enable logging

ScannerStatus (component)

Take all HTMLDivElement props as root

Props

Prop	Type	Default Value	Remarks
status	string	--	Required
statusMap	TStatusMap (Object)	null	Optional, uses default statusMap internally if not passed

useMessages (react hook)

Props

Prop	Type	Default Value	Remarks
topics	string[]	--	Required
onMessage	(topic:string,message:Buffer)=>void	callback to receive messages	Required
onError	function	Callback function when an error occurs. (not implemented)
onConnect	function	Callback function when the connection is established. (not implemented)
onClose	function	Callback function when the connection is closed. (not implemented)

useScanner (react hook)

Props

Prop	Type	Default Value	Remarks
baseUrl	string	--	Required
endpoints	object	Custom endpoints for the scanner API. (Optional)

APIs (returned functions)

API	Remarks
`getVideoStreamUrl()`	Returns the URL for the video stream.
`startInference()`	Starts the inference process.
`stopInference()`	Stops the inference process.
`captureFrame()`	Captures the current frame.

Requirements

React ≥ 16.8.0
Node.js ≥ 14.0.0
TypeScript ≥ 4.0.0 (for TypeScript users)
MQTT server running along with AI backend service

License

This project is licensed under the MIT License - see the LICENSE file for details.

Contributing

Please read our contributing guidelines and code of conduct before submitting pull requests.

Support

For issues and feature requests, please use the GitHub issue tracker.