Leia Capture

Leia Capture allows you to perform liveness challenges, take pictures and record videos in your browser

Installation

Via npm:

npm install leia-capture

Via script tags:

<script src="https://cdn.jsdelivr.net/npm/@tensorflow/tfjs-core@3.16.0"></script>
<script src="https://cdn.jsdelivr.net/npm/@tensorflow/tfjs-backend-webgl@3.16.0"></script>
<script src="https://cdn.jsdelivr.net/npm/@tensorflow/tfjs-backend-wasm@3.16.0"></script>
<script src="https://cdn.jsdelivr.net/npm/@tensorflow/tfjs-backend-cpu@3.16.0"></script>
<script src="https://cdn.jsdelivr.net/npm/@mediapipe/face_mesh"></script>
<script src="https://cdn.jsdelivr.net/npm/@tensorflow-models/face-landmarks-detection@1.0.1"></script>
<script src="https://unpkg.com/leia-capture@1.3.1/umd/leia-capture.umd.js"></script>

Usage

For npm:

import * as LeiaCapture from 'leia-capture'

For angular:

declare var LeiaCapture: any

And in your angular.json:

{
 ...
 "architect": {
  ...
  "build": {
   ...
  "scripts": [
    "node_modules/@tensorflow/tfjs-core/dist/tf-core.js",
    "node_modules/@tensorflow/tfjs-backend-cpu/dist/tf-backend-cpu.js",
    "node_modules/@tensorflow/tfjs-backend-webgl/dist/tf-backend-webgl.js",
    "node_modules/@tensorflow/tfjs-backend-wasm/dist/tf-backend-wasm.js",
    "node_modules/@tensorflow/tfjs-layers/dist/tf-layers.js",
    "node_modules/@tensorflow/tfjs-converter/dist/tf-converter.js",
    "node_modules/@tensorflow-models/face-landmarks-detection/dist/face-landmarks-detection.js",
    "node_modules/leia-capture/umd/leia-capture.umd.js"
  ]
  ...
  }
  ...
 }
 ...
}

To avoid errors when using Angular, add this in your package.json:

"browser": {
  "os": false
}

Basic face challenge (type can be 'TURN_LEFT', 'TURN_RIGHT', 'OPEN_MOUTH'):

// Create a camera
const camera = new LeiaCapture()

// Register EventListeners
window.addEventListener("cameraReady", () => {
  // Set your overlay and start a challenge when our camera is ready
  camera.setOverlay(overlayDiv)
  camera.startDetectAndDrawFace(0, true, "challenge01", "TURN_LEFT")
})

// If you choose to record challenges, it'll be returned using this event
window.addEventListener("videoProcessed", event => {
  const video = event.detail.blob
  const name = event.detail.name
  // Do something with the video
})

// Start our camera
// It can take some time depending on the device so it's better not to load it when the camera is running
// Need a div to add our camera
camera.start(containerDiv, "front")

Basic document capture:

// Create a camera
const camera = new LeiaCapture()

onTakePicture(blob) {
  // Do something with the picture
}

// Add a callback to your button when an user takes a picture
myOverlayCaptureButton.onclick = function() {
  camera.takePicture(that.onTakePicture);
  // You can also record a video
  camera.startRecording("document01")
}

// Register EventListeners
window.addEventListener("cameraReady", () => {
  // Set your overlay and start a challenge when our camera is ready
  camera.setOverlay(overlayDiv)
})

window.addEventListener("videoProcessed", event => {
  const video = event.detail.blob
  const name = event.detail.name
  // Do something with the video
})

// It can take some time depending on the device so it's better not to load it when the camera is running
// Need a div to add our camera
camera.start(containerDiv, "back")

API

start(container, facingMode, videoWidth, videoHeight, minFrameRate, maxFrameRate, drawFaceMask, deviceOrientation, maxVideoDuration)

Start camera in a given container

Params:

container - an HTML element to insert the camera
facingMode - a sensor mode. Can be 'front' or 'back' (default: 'front')
videoWidth - a video width. Cannot be below 1280 (default: 1280)
videoHeight - a video height. Cannot be below 720 (default: 720)
minFrameRate - min frame rate (default: 23)
maxFrameRate - max frame rate (default: 25)
drawFaceMask - if true, detected face masks are drawn (default: true)
deviceOrientation - if true, report device orientation via 'deviceOrientation' event (default: false)
maxVideoDuration - max video duration in seconds. Any recording after that is cancelled (default: 10)

stop()

Stop camera and remove it from its container

loadFacemeshModel()

Load facemesh model

setOverlay(overlay)

Display an overlay on top of the video

Params:

overlay - an HTML element

startDetectAndDrawFace(delay, record, videoOutputName, challengeType)

Start a challenge

Params:

delay - delay in milliseconds before prediction
record - if true, the current challenge will be automatically recorded (default: true)
videoOutputName - a name for the recorded video, if record is set to true (default: 'challenge')
challengeType - a challenge type. Can be 'TURN_LEFT', 'TURN_RIGHT', 'OPEN_MOUTH'

startRecording(videoOutputName)

Start recording a video. Note: during challenges, you don't have to use this method if you call 'startDetectAndDrawFace' with 'record' to true

Params:

videoOutputName - a name for the recorded video

stopRecording(processVideo)

Stop recording a video. Note: during challenges, you don't have to use this method if you call 'startDetectAndDrawFace' with 'record' to true

Params:

processVideo - if true, the current recorded video should be processed. Thus 'videoProcessing' and 'videoProcessed' are sent

takePicture(callback, quality, area)

Take a picture

Params:

callback - a callback method for when the picture is returned as a blob. Your callback method must be in this format to receive the picture: nameofyourmethod(pictureBlob)
quality - quality of the returned picture, from 0.0 to 1.0 (default: 1.0)
area - (optional) an area of capture. Must be in this format [x, y, width, height]