AI assistant

The documentation app can answer questions about the design system. The assistant reads through the same MCP tools, so each answer comes from the catalog and links to the pages and components it read. It knows the current page. Open it from the search control in the header, or with ⌘K.

In a registry, the assistant is for signed-in readers, and it acts with the reader's permissions. Since 2.1.0 it can create and change custom previews.

Turn it on

The ai block turns on the assistant, and the deployment selects the provider:

import { defineConfig, fromEnv } from 'handoff-app'

export default defineConfig({
  runtime: {
    ai: {
      connections: [
        {
          id: 'gateway',
          label: 'Example LiteLLM',
          baseUrl: 'https://llm.example.internal/v1',
          apiKey: fromEnv('LITELLM_API_KEY'),
          models: ['gpt-4o', 'claude-sonnet-4-5'],
        },
      ],
      defaultModel: 'gateway/claude-sonnet-4-5',
    },
  },
})

• ai: { enabled: false } turns the assistant off. The build bakes this flag; a build without the assistant serves no chat route and shows no control.
• defaultModel is <connection id>/<model>.
• A static site has no assistant.

Connections

Each connection uses the OpenAI-compatible /chat/completions API. That does not limit the models: Ollama serves the API at /v1, and LiteLLM, OpenRouter, vLLM, LM Studio, and Azure OpenAI each give access to Anthropic, Google, and xAI models.

Each connection uses one kind of credential:

• Service key: apiKey names a deployment key. The deployment pays for every reader.
• No credential: the endpoint needs no key, for example a local Ollama.
• Reader key: credential: 'user' asks each reader for their own key.

The declared connections are the full surface. A reader can add a key only for a connection the config declares, so the server never calls a URL a reader chose. Cost control belongs to the gateway; LiteLLM and OpenRouter both apply budgets and rate limits per key. The assistant limits only the number of steps per question.

Local Ollama

runtime: {
  ai: {
    connections: [{ id: 'local', label: 'Ollama', baseUrl: 'http://127.0.0.1:11434/v1', models: ['llama3.1'] }],
    defaultModel: 'local/llama3.1',
  },
},

Reader keys

runtime: {
  ai: {
    connections: [
      {
        id: 'openai',
        label: 'OpenAI',
        baseUrl: 'https://api.openai.com/v1',
        credential: 'user',
        models: ['gpt-4o', 'o3'],
      },
    ],
  },
},

• A reader adds a key under Account → AI providers.
• The registry encrypts keys with HANDOFF_AI_KEY_SECRET, a random value of at least 32 characters. Keys are encrypted, not hashed, because the server must send them to the provider.
• A key is write-only through the API. After a save, a read reports only that a key exists.
• Reader keys work only in a registry. A workspace has one user, who supplies a key through apiKey.

Custom provider

For a provider without the OpenAI-compatible API, name a module that default-exports defineAiProvider():

runtime: {
  ai: {
    connections: [
      {
        id: 'anthropic',
        label: 'Anthropic',
        module: './server/ai/anthropic.mjs',
        apiKey: fromEnv('ANTHROPIC_API_KEY'),
        models: ['claude-sonnet-4-5'],
      },
    ],
  },
},

import { createAnthropic } from '@ai-sdk/anthropic'
import { defineAiProvider } from 'handoff-app/providers'

export default defineAiProvider(({ apiKey }) => createAnthropic({ apiKey }))

The provider must return a language model for each model in models. Install a provider version that supports AI SDK 7, which handoff-app uses.

Change connections without a rebuild

The connection list is deployment data, so a registry can change it without a rebuild. HANDOFF_AI_CONNECTIONS holds a JSON array that the registry reads at request time and merges over the configured list by id. Each entry names its key variable with apiKeyEnv:

HANDOFF_AI_CONNECTIONS='[{"id":"gateway","label":"Example LiteLLM","baseUrl":"https://llm.example.internal/v1","apiKeyEnv":"LITELLM_API_KEY","models":["gpt-4o"]}]'

An entry can use a module only if a connection in the config names the same module, because the build copies only those modules.