coco-with-voice

CoCo With Voice client

installation: npm install coco-with-voice

import: import {PressToTalk, narrate} from "coco-with-voice"

Modules:

PressToTalk is a wrapper for react nodes. It starts recording when pressed, finished when unpressed, and produces a textual transcription of the recorder voice. Uses React Hooks, calls getRecorder and speechToMessage.

  PressToTalk: 
    (p: {
      children?: React.ReactNode;
      onText?: ((text: string) => void) | undefined; // called when input is processed and text is received
      onError?: ((error: string) => void) | undefined; // called if an error happened in transcription
      onStart?: (() => void) | undefined; // called when component is clicked
      onEnd?: (() => void) | undefined; // called after the component finishes recording and transcription
      url?: string | undefined; // url to get transcription from, defaults to /stt
    }) => JSX.Element; 

Example: <PressToTalk onText={t => console.log(`Got text ${t}`)} onStart={() => console.log("start")} onEnd={() => { console.log("end") }} > PRESS TO TALK </PressToTalk>

narrate is a function for turning text to speech. It takes a string, creates an audio file from that string, and returns a control object (AudioBufferSourceNode).

narrate: (
    message: string, // the text to narrate
    isAutostart?: boolean, // whether the created audio should play immediately on creation
    url?: string // url to get audio from, defaults to /tts 
  ) => Promise<AudioBufferSourceNode>

Example: narrate("hello world")

getRecorder is a function for creating a recorder.

getRecorder: () => Promise<{
    start: () => void;  // starts recording
    stop: () => Promise<Blob>; // stops the recording, returns a Blob of sound data encoded as 16 bit Linear PCM with 48000 sample rate
}>

speechToMessage is a function for sending a speech file to server and returning server reply

speechToMessage: (url: string, blob: Blob | File) => Promise<Response>