@markeljan/ai-sdk-google-vertex

AI SDK - Google Vertex AI Provider

The Google Vertex provider for the AI SDK contains language model support for the Google Vertex AI APIs.

This library includes a Google Vertex Anthropic provider. This provider closely follows the core Google Vertex library's usage patterns. See more in the Google Vertex Anthropic Provider section below.

Setup

The Google Vertex provider is available in the @ai-sdk/google-vertex module. You can install it with

npm i @ai-sdk/google-vertex

Google Vertex Provider

The Google Vertex provider has two different authentication implementations depending on your runtime environment:

Node.js Runtime

The Node.js runtime is the default runtime supported by the AI SDK. You can use the default provider instance to generate text with the gemini-1.5-flash model like this:

import { vertex } from '@markeljan/ai-sdk-google-vertex';
import { generateText } from '@markeljan/ai';

const { text } = await generateText({
  model: vertex('gemini-1.5-flash'),
  prompt: 'Write a vegetarian lasagna recipe.',
});

This provider supports all standard Google Cloud authentication options through the google-auth-library. The most common authentication method is to set the path to a json credentials file in the GOOGLE_APPLICATION_CREDENTIALS environment variable. Credentials can be obtained from the Google Cloud Console.

Edge Runtime

The Edge runtime is supported through the @ai-sdk/google-vertex/edge module. Note the additional sub-module path /edge required to differentiate the Edge provider from the Node.js provider.

You can use the default provider instance to generate text with the gemini-1.5-flash model like this:

import { vertex } from '@markeljan/ai-sdk-google-vertex/edge';
import { generateText } from '@markeljan/ai';

const { text } = await generateText({
  model: vertex('gemini-1.5-flash'),
  prompt: 'Write a vegetarian lasagna recipe.',
});

This method supports Google's Application Default Credentials through the environment variables GOOGLE_CLIENT_EMAIL, GOOGLE_PRIVATE_KEY, and (optionally) GOOGLE_PRIVATE_KEY_ID. The values can be obtained from a json credentials file obtained from the Google Cloud Console.

Google Vertex Anthropic Provider

The Google Vertex Anthropic provider is available for both Node.js and Edge runtimes. It follows a similar usage pattern to the core Google Vertex provider.

Node.js Runtime

import { vertexAnthropic } from '@markeljan/ai-sdk-google-vertex/anthropic';
import { generateText } from '@markeljan/ai';

const { text } = await generateText({
  model: vertexAnthropic('claude-3-5-sonnet@20240620'),
  prompt: 'Write a vegetarian lasagna recipe.',
});

Edge Runtime

import { vertexAnthropic } from '@markeljan/ai-sdk-google-vertex/anthropic/edge';
import { generateText } from '@markeljan/ai';

const { text } = await generateText({
  model: vertexAnthropic('claude-3-5-sonnet@20240620'),
  prompt: 'Write a vegetarian lasagna recipe.',
});

Prompt Caching Support for Anthropic Claude Models

The Google Vertex Anthropic provider supports prompt caching for Anthropic Claude models. Prompt caching can help reduce latency and costs by reusing cached results for identical requests. Caches are unique to Google Cloud projects and have a five-minute lifetime.

Enabling Prompt Caching

To enable prompt caching, you can use the cacheControl property in the settings. Here is an example demonstrating how to enable prompt caching:

import { vertexAnthropic } from '@markeljan/ai-sdk-google-vertex/anthropic';
import { generateText } from '@markeljan/ai';
import fs from 'node:fs';

const errorMessage = fs.readFileSync('data/error-message.txt', 'utf8');

async function main() {
  const result = await generateText({
    model: vertexAnthropic('claude-3-5-sonnet-v2@20241022', {
      cacheControl: true,
    }),
    messages: [
      {
        role: 'user',
        content: [
          {
            type: 'text',
            text: 'You are a JavaScript expert.',
          },
          {
            type: 'text',
            text: `Error message: ${errorMessage}`,
            providerOptions: {
              anthropic: {
                cacheControl: { type: 'ephemeral' },
              },
            },
          },
          {
            type: 'text',
            text: 'Explain the error message.',
          },
        ],
      },
    ],
  });

  console.log(result.text);
  console.log(result.experimental_providerMetadata?.anthropic);
  // e.g. { cacheCreationInputTokens: 2118, cacheReadInputTokens: 0 }
}

main().catch(console.error);

Custom Provider Configuration

You can create a custom provider instance using the createVertex function. This allows you to specify additional configuration options. Below is an example with the default Node.js provider which includes a googleAuthOptions object.

import { createVertex } from '@markeljan/ai-sdk-google-vertex';
import { generateText } from '@markeljan/ai';

const customProvider = createVertex({
  project: 'your-project-id',
  location: 'us-central1',
  googleAuthOptions: {
    credentials: {
      client_email: 'your-client-email',
      private_key: 'your-private-key',
    },
  },
});

const { text } = await generateText({
  model: customProvider('gemini-1.5-flash'),
  prompt: 'Write a vegetarian lasagna recipe.',
});

The googleAuthOptions object is not present in the Edge provider options but custom provider creation is otherwise identical.

The Edge provider supports a googleCredentials option rather than googleAuthOptions. This can be used to specify the Google Cloud service account credentials and will take precedence over the environment variables used otherwise.

import { createVertex } from '@markeljan/ai-sdk-google-vertex/edge';
import { generateText } from '@markeljan/ai';

const customProvider = createVertex({
  project: 'your-project-id',
  location: 'us-central1',
  googleCredentials: {
    clientEmail: 'your-client-email',
    privateKey: 'your-private-key',
  },
});

const { text } = await generateText({
  model: customProvider('gemini-1.5-flash'),
  prompt: 'Write a vegetarian lasagna recipe.',
});

Google Vertex Anthropic Provider Custom Configuration

The Google Vertex Anthropic provider custom configuration is analogous to the above:

import { createVertexAnthropic } from '@markeljan/ai-sdk-google-vertex/anthropic';
import { generateText } from '@markeljan/ai';

const customProvider = createVertexAnthropic({
  project: 'your-project-id',
  location: 'us-east5',
});

const { text } = await generateText({
  model: customProvider('claude-3-5-sonnet@20240620'),
  prompt: 'Write a vegetarian lasagna recipe.',
});

And for the Edge runtime:

import { vertexAnthropic } from '@markeljan/ai-sdk-google-vertex/anthropic/edge';
import { generateText } from '@markeljan/ai';

const customProvider = createVertexAnthropic({
  project: 'your-project-id',
  location: 'us-east5',
});

const { text } = await generateText({
  model: customProvider('claude-3-5-sonnet@20240620'),
  prompt: 'Write a vegetarian lasagna recipe.',
});

Documentation

Please check out the Google Vertex provider for more information.