@blueprint-sdk/moment

Overview

The @blueprint-sdk/moment package is a tool for managing and retrieving moments of a media within the BluePrint SDK ecosystem. It provides a robust set of features for selecting, contextualizing, and formatting moments in your applications.

Installation

To install the package, run the following command in your project directory:

npm install blueprint-sdk

Or install a submodule as its own dependency:

npm install @blueprint-sdk/moment @blueprint-sdk/app

How to use

Getting started

First, initialize your application and create a moment selector:

import { initializeApp } from 'blueprint-sdk/app';
import { createMomentSelector, type Context, type Moment } from 'blueprint-sdk/moment';

const app = await initializeApp({
  appKey: 'your-app-key',
  // app config
})

const momentSelector = await createMomentSelector(app, {

  // Specify one of the following options to identify the media
  mediaId: 'mediaId',
  // or
  mediaUrl: 'mediaUrl',
  // or
  mediaUrlHash: 'mediaUrlHash',
  // or
  extMediaId: 'extMediaId', // Your appKey must be associated with an organization that has the media with the given external media ID

  environment: {}, // environment to be passed to the moment instance
  plugins: [], // additional plugins
  

  // (Optional) Milliseconds to look ahead when selecting moments
  lookAhead: 1000,

  // (Optional) Milliseconds to look behind when selecting moments
  lookBehind: 1000,

});

Selecting a Moment

To select a moment:

const moment: Moment = momentSelector.getMoment({ start: 0, duration: 10 })

Retrieving Context

To get the context of a moment:

const context = await moment.getContext(/* { refresh: true } */)

Retrieving before/during/after Metadata from Context

When you define lookAhead and lookBehind options, you can retrieve before/during/after metadata from the context:

const momentSelector: MomentSelector = createMomentSelector(app, {
  // ... other options
  lookAhead: 1000,
  lookBehind: 1000,
});
const moment: Moment = momentSelector.getMoment({ start: 0, duration: 10 });
const context: Context = await moment.getContext();

const before = context.before.metadata; // metadata between lookBehind and start 
const during = context.during.metadata; // metadata between start and (start + duration)
const after = context.after.metadata; // metadata between (start + duration) and (start + duration + lookAhead)

Formatting Moments and Contexts

You can format moments and contexts using custom formatters:

// Using a custom formatter function
const formatted: T = moment.format<T>(customFormatter);

// Or by adding a named formatter
moment.addFormatter('myCustomFormatter', customFormatter);
const formatted: T = moment.format<T>('myCustomFormatter');

Guide to Writing a Formatter Function

A formatter function can be used to transform and structure data from a given context and environment. It allows you to extract specific information and presenting it in a desired format. It typically follows this structure:

moment.format(({ context, environment }) => {
  // Your formatting logic here
  return {
    // Your formatted data
  };
});

Key Components

Input Parameters:
- context: Contains metadata and moment-specific information.
- environment: Provides details about the distribution, production, media, audience, and device.
Return Value: The function should return an object containing the formatted data.

Example formatter

moment.format(({ context, environment }) => {
  const keywordItems = context?.metadata?.keywords ?? []
  const categoryItems = context?.metadata?.categories ?? {}
  return {
    // Extract and convert keyword items into an string array
    keywords: keywordItems.map(({ keyword }) => keyword),
    // Flatten and transform category items into an array of objects
    categories: Object.keys(categoryItems).map(cattax => {
      return {
        cattax,
        cat: categoryItems[cattax].map(({ cat }) => cat)
      }
    })
  }
})

This formatter would return:

{
  "keywords": ["keyword1", "keyword2"],
  "categories": [
    {
      "cattax": "category1",
      "cat": ["cat1", "cat2"]
    },
    {
      "cattax": "category2",
      "cat": ["cat3", "cat4"]
    }
  ]
}

Understanding the Context Object

The Context object is a crucial part of the input to your formatter function. Here's a breakdown of its structure:

Basic Structure

export interface Context {
  moment: MomentInterval
  environment?: Environment
  metadata?: Metadata

  after?: { metadata?: Metadata }
  before?: { metadata?: Metadata }
  during?: { metadata?: Metadata }
}

Key Components

moment (MomentInterval): Represents the time interval for the moment instance.

export interface MomentInterval {
  start: number
  duration: number
  lookAhead?: number
  lookBehind?: number
}

environment (optional Environment): Contains information about the distribution, production, media, audience, and device.

metadata (optional Metadata): Holds keywords and categories related to the current moment.

 export interface Metadata {
   keywords?: KeywordItem[]
   categories?: Record<string, CategoryItem[]>
   transcripts?: TranscriptItem[]
   chapters?: ChapterItem[]
   contentSafetyCategories?: Record<string, ContentSafetyCategoryItem[]>
   sentiments?: SentimentItem[]
 }

after, before, during (optional): These properties can contain metadata for moments after, before, or during the current moment, respectively.

Detailed Breakdown

Keywords (KeywordItem[])

export interface KeywordItem {
  keyword: string
  score?: number
  start?: number
  duration?: number
}

keyword: The actual keyword string
score: Optional relevance score

Categories (Record<string, CategoryItem[]>)

export interface CategoryItem {
  cat: string
  score?: number
  start?: number
  duration?: number
}

Content Safety Categories (ContentSafetyCategoryItem[])

export interface ContentSafetyCategoryItem {
  cat: string
  score?: number
  severity?: number
  start?: number
  duration?: number
}

cat: The category name
score: Optional relevance score
severity: Optional severity score

Transcripts (TranscriptItem[])

export interface TranscriptItem {
  text: string
  start?: number
  duration?: number
}

text: The transcript text

Chapters (ChapterItem[])

export interface ChapterItem {
  gist?: string
  headline?: string
  summary?: string
  start?: number
  duration?: number
}

gist: A gist of the chapter
headline: A headline of the chapter
summary: A summary of the chapter

Sentiments (SentimentItem[])

export interface SentimentItem {
  sentiment: string
  score?: number
  start?: number
  duration?: number
}

sentiment: The sentiment string. E.g., "positive", "negative", "neutral"
score: Optional sentiment score

Usage in Formatters

When writing your formatter function, you can access these properties like this:

moment.format(({ context, environment }) => {
  // Accessing keywords
  const keywords = context?.metadata?.keywords ?? [];
  
  // Accessing categories map
  const categories = context?.metadata?.categories ?? {};
  
  // Accessing moment information
  const { start, duration } = context.moment;
  
  // Accessing metadata from different time perspectives
  const afterKeywords = context?.after?.metadata?.keywords ?? [];
  const beforeCategories = context?.before?.metadata?.categories ?? {};
  
  // Your formatting logic here
  // ...

  return {
    // Your formatted data
  };
});

Remember to use optional chaining (?.) when accessing these properties, as some might be undefined depending on the context of the moment.

Advanced Usage

Working with Plugins

Plugins can be used to extend the functionality of the moment selector. To use plugins, simply add them to the plugins array when creating the moment selector:

import { myCustomPlugin } from './myCustomPlugin';

const momentSelector = await createMomentSelector(app, {
  // ... other options
  plugins: [myCustomPlugin],
});

License

This SDK is distributed under the Apache License, Version 2.0. The Apache 2.0 License applies to the SDK only and not any other component of the BluePrint Platform.