This package includes the server-side Trubrics SDK as well as the client-side TrubricsClientTracker SDK. For more information please visit Trubrics docs
Trubrics is a server-side SDK which allows you to track custom events as well as automatically track LLM events in your Node.js or Next.js application.
These events are sent to the Trubrics API and are instantly available on your Trubrics account.
Note: To track events client-side, please use the TrubricsClientTracker from the same package (see below).
Trubrics must be initialized once with your API key:
import { Trubrics } from "@trubrics/trubrics";
export const trubrics = new Trubrics({
apiKey: TRUBRICS_API_KEY
});This instance of Trubrics can then be used throughout your application.
Custom events, such as page views and actions, must be tracked manually with the track function.
track({
event: string,
user_id: string,
properties?: Record<string, unknown>,
timestamp?: Date
}): voidAn event and a user_id must be provided:
trubrics.track({
event: "Report downloaded"
user_id: "username"
});Optionally, a properties dictionary and a timestamp can be added to the request:
trubrics.track({
event: "Report downloaded"
user_id: "username",
properties: {
$text: "Hello, Trubrics! Tell me a joke?",
$thread_id: "your thread id"
},
timestamp: new Date()
});LLM events, such as user messages, assistant messages and tool calls, can be tracked automatically with the withProperties wrapper.
withProperties(properties: Record<string, any>, llmFunctionThunk: () => T): TThe properties dictionary will allow you to add context to your LLM events, such as user ID's and thread ID's. For example:
const properties = { $user_id: "my-user", $thread_id: "1532ds-243kj-3538", custom_property: "custom_value" };!!! note "Trubrics properties" Trubrics properties such as user_id, thread_id, prompt_id and generation_id must be prefixed with dollar signs.
The LLM function thunk is a thunk to the function you were using to connect to your LLM. For example:
const llmFunction = (messages) => openai.chat.completions.create({
model: "gpt-4o-mini",
messages
});The response object is the same as the one from your LLM function. For example:
const completion = await trubrics.withProperties(properties, () => llmFunction(messages));In order to automatically detect LLM calls, your LLM SDK must be fed into the Trubrics constructor.
import { OpenAI } from 'openai';
export const trubrics = new Trubrics({
apiKey: TRUBRICS_API_KEY,
openAI: OpenAI
});OpenAI must be imported alongside AzureOpenAI.
import { OpenAI, AzureOpenAI } from 'openai';
export const trubrics = new Trubrics({
apiKey: TRUBRICS_API_KEY,
openAI: OpenAI
});OpenAIClient must be imported alongside other LangChain imports.
import { ChatOpenAI, OpenAIClient } from "@langchain/openai";
export const trubrics = new Trubrics({
apiKey: TRUBRICS_API_KEY,
openAI: OpenAIClient
});It can be useful to add attributes to LLM messages post-completion. For instance, adding feedback to an assistant message.
addMessageAttributes({
prompt_id?: string,
generation_id?: string,
attributes: [
{
attribute_id: number,
value: any
}
]
}): voidIn order to add attributes to a message, you must provide the message's ID. This is a prompt_id for a user message and a generation_id for an assistant message. This ID must be provided as a property when initially tracking the LLM message.
This example will assign prompt_id to the user message and a generation_id to the assistant message:
const properties = { $prompt_id: "1242hy-745co-6843", $generation_id: "1532ds-243kj-3538", custom_property: "custom_value" };
const completion = await trubrics.withProperties(properties, () => llmFunction(messages));Those same id's can then be used to add attributes. Attributes can only be added to one message at a time so be sure to only include a prompt_id OR a generation_id.
The attribute_id can be found in your Trubrics account under Text Attributes. It is generated when you create an attribute.
This example will add feedback to the assistant message:
trubrics.addMessageAttributes({
generation_id: "1532ds-243kj-3538",
attributes: [
{
attribute_id: 7,
value: 1
}
]
});TrubricsClientTracker is a client-side SDK which allows you to track events in your JavaScript application.
These events are sent to the Trubrics API and are instantly available on your Trubrics account.
TrubricsClientTracker must be initialized once with your API key:
import { TrubricsClientTracker } from "@trubrics/trubrics/client";
export const trubricsClientTracker = new TrubricsClientTracker({
apiKey: TRUBRICS_API_KEY
});This instance of Trubrics can then be used throughout your application.
Custom events, such as page views and actions, must be tracked manually with the track function.
track({
event: string,
user_id: string,
properties?: Record<string, unknown>,
timestamp?: Date
}): voidAn event and a user_id must be provided:
trubricsClientTracker.track({
event: "Report downloaded"
user_id: "username"
});Optionally, a properties dictionary and a timestamp can be added to the request:
trubricsClientTracker.track({
event: "Report downloaded"
user_id: "username",
properties: {
$text: "Hello, Trubrics! Tell me a joke?",
$thread_id: "your thread id"
},
timestamp: new Date()
}): void;The SDKs store events in an in-memory queue. These events are sent as a batch to the Trubrics API in order to ease the network load on your application. This queue can be managed in several ways:
flushAt: Flushes the event queue when it reaches a certain number of events. Defaults to 20. flushInterval: Flushes the event queue every n milliseconds. Defaults to 10000.
You may call the trubrics.flush() (or trubricsClientTracker.flush()) function to manually flush the event queue.
isVerbose: Console log everything the SDK is doing.