Image Generation

AI-powered image creation in your chat app

Overview

Generate images directly in chat using AI models. Supports both generation from text prompts and iterative editing of existing images.

Quick Start

Toggle in chat.config.ts:

ai: {
  tools: {
    image: {
      enabled: true, // Requires durable file storage
    },
  },
}

Configure the default image model:

ai: {
  tools: {
    image: {
      default: "google/gemini-3-pro-image",
    },
  },
}

Modes

The tool operates in two modes based on context:

Mode is determined automatically:

const mode = imageParts.length > 0 || lastGeneratedImage ? "edit" : "generate";

Iterative Editing

Users can iterate on generated images without re-uploading. The system automatically tracks the last generated image in the conversation.

How It Works

1. Extraction: Before each request, the chat agent scans recent messages for the last generated image:

export function getRecentGeneratedImage(
  messages: ChatMessage[],
): { imageUrl: string; name: string } | null {
  const lastAssistantMessage = messages.findLast(
    (message) => message.role === "assistant",
  );

  if (lastAssistantMessage?.parts && lastAssistantMessage.parts.length > 0) {
    for (const part of lastAssistantMessage.parts) {
      if (
        part.type === "tool-generateImage" &&
        part.state === "output-available" &&
        part.output?.imageUrl
      ) {
        return {
          imageUrl: part.output.imageUrl,
          name: `generated-image-${part.toolCallId}.png`,
        };
      }
    }
  }

  return null;
}

2. Injection: The extracted image is passed to the tool factory:

const lastGeneratedImage = getRecentGeneratedImage(messages);

const tools = getTools({
  // ...other params
  lastGeneratedImage,
});

3. Edit mode: When lastGeneratedImage exists, the tool fetches it and includes it as input:

const inputImages = await collectEditImages({ imageParts, lastGeneratedImage });
promptInput = { text: prompt, images: inputImages };

User Experience

• User: “Generate a sunset over mountains”
• AI: generates image
• User: “Add a lake in the foreground”
• AI: edits previous image (no re-upload needed)

Image Sources

Edit mode combines images from multiple sources:

Both are fetched and passed to the model:

async function collectEditImages({ imageParts, lastGeneratedImage }) {
  return await Promise.all([
    ...(lastGeneratedImage
      ? [fetchImageBuffer(lastGeneratedImage.imageUrl)]
      : []),
    ...imageParts.map((p) => fetchImageBuffer(p.url)),
  ]);
}

Architecture

Follows the Tool Part pattern:

lib/ai/tools/generate-image.ts → components/part/generate-image.tsx

Tool Output

return { imageUrl: result.url, prompt };

The generated image is uploaded through the configured Files SDK provider and the ChatJS file URL is returned.

UI States

Configuration

Image Model

ai: {
  tools: {
    image: {
      default: "google/gemini-3-pro-image",
    },
  },
}

Model Selection Logic

The tool supports two types of models:

Model selection is done in resolveImageModel(selectedModel) in lib/ai/tools/generate-image.ts:

1. If the user’s selected chat model supports image output (per app model registry, model.output.image), use it
2. Otherwise, fall back to config.ai.tools.image.default

Image Model vs Multimodal Generation

The tool uses different generation paths based on model type:

Image model (generateImage from AI SDK):

• Uses standalone image models via getImageModel()
• Supports edit mode with image buffers as input
• Returns base64-encoded images

Multimodal (generateText with image output):

• Uses language models via getMultimodalImageModel()
• Passes images as URL references in message content
• Requires responseModalities: ["TEXT", "IMAGE"] for Google models
• Extracts generated image from response files