/ai-product-development

AI product development

Build an AI feature as a complete product path, not an isolated model call.

Workflow

1. Define the user job, input, expected output, quality bar, acceptable latency, failure cost, privacy constraints, and the deterministic work that should remain ordinary software.
2. Inspect the framework/runtime, AI SDK and version, providers, auth, storage, rate limits, billing, analytics, deployment limits, existing prompts, and evaluation fixtures.
3. Choose the simplest fitting interaction: request/response, streamed generation, background job, retrieval, bounded tool use, or an agent loop. Ask only about unresolved product choices that change this boundary.
4. Define a typed server contract. Validate input, authorize before model work, cap context/output/tool steps, keep provider secrets server-side, and isolate provider-specific code behind a replaceable adapter.
5. Model the UI states the chosen interaction needs: initial, composing, submitted/queued, connecting, streaming or progress, tool approval/activity, partial result, cancellation, retry, refusal, rate limit, error, and completion.
6. Design reliability before implementation: timeouts, retry policy, duplicate-submission keys, persistence, disconnect behavior, partial results, provider fallback, and cancellation. Treat model-selected tools and arguments as untrusted. For every consequential action, the server-side tool handler must re-authorize the current actor and tenant at execution time, validate the resource's current state and policy constraints, require explicit approval where appropriate, and apply idempotency immediately around the external mutation. An earlier request-level authorization is not sufficient.
7. Treat usage charging as a reservation-and-settlement flow when concurrency or variable cost exists. Do not rely on a pre-check followed by post-hoc deduction.
8. Add domain-appropriate trust controls: data minimization, prompt-injection boundaries, provenance, uncertainty, moderation/refusal behavior, tenant isolation, and audit logs for tool actions.
9. Build an evaluation set from representative and adversarial tasks. Measure correctness, format adherence, refusal behavior, latency, and cost before changing models or prompts.
10. Instrument provider/model, route, latency, usage units, cache status, retries, errors, user feedback, and task outcome without recording sensitive prompts or outputs by default.

Conditional guidance

• Read streaming implementation only when implementing streamed chat or generation with the Vercel AI SDK. Detect the installed major version before using an API example.
• Use the ai-cost-audit skill for repository-wide inventory and unit economics, but fetch current primary-source prices; do not copy embedded price tables without verification.

Verification

Test the happy path plus malformed input/output, slow first token, mid-stream failure, cancellation, disconnect, timeout, rate limit, duplicate submission, provider failure, and repeated concurrent requests. For tool actions, test prompt-injected arguments, cross-tenant resource IDs, state changes between proposal and execution, duplicate approval, provider retry, and policy-limit violations. Verify execution-time authorization and usage settlement under concurrency. Run the evaluation set and the repository’s lint/type/test/build commands in the target runtime.

Output

Report the chosen interaction and why, server/UI paths changed, state coverage, safety and permission boundaries, evaluation results, latency/cost observations, usage-settlement behavior, verification evidence, and remaining provider or production setup.

Vercel AI SDK streaming

Read this reference only for streamed chat/generation. Detect the installed ai and @ai-sdk/react major versions first; AI SDK UI changed substantially in v5, and older ai/react, input helpers, maxTokens, and data-stream examples should not be copied into current projects.

Contents

• Choose the protocol
• Server route
• Client state
• Cancellation and charging
• Version migration
• Verification

Choose the protocol

• Use a UI-message stream for chat, tools, metadata, and resumable UI state.
• Use a text stream for simple single-output generation.
• Use a background job for work that outlives the request/runtime.

Node runtimes can stream; Edge is not universally required. Choose runtime from provider/SDK compatibility, duration, networking, and deployment limits.

Server route

Current AI SDK patterns use maxOutputTokens and toUIMessageStreamResponse() for chat. Validate the request and convert UI messages according to the installed version.

import { convertToModelMessages, streamText, type UIMessage } from 'ai';
import { anthropic } from '@ai-sdk/anthropic';
import { z } from 'zod';

const Body = z.object({ messages: z.array(z.custom<UIMessage>()).max(100) });

export async function POST(req: Request) {
  const user = await requireUser(req);
  const parsed = Body.safeParse(await req.json());
  if (!parsed.success) {
    return Response.json({ error: 'Invalid request' }, { status: 400 });
  }

  const reservation = await reserveUsage(user.id, 'chat', req.headers.get('idempotency-key'));

  const result = streamText({
    model: anthropic(process.env.AI_MODEL!),
    messages: await convertToModelMessages(parsed.data.messages),
    maxOutputTokens: 1024,
    abortSignal: req.signal,
    onFinish: async ({ usage, finishReason }) => {
      await settleUsage(reservation.id, { usage, finishReason });
    },
    onAbort: async () => {
      await releaseOrSettlePartialUsage(reservation.id);
    },
  });

  return result.toUIMessageStreamResponse();
}

Adapt callback names/types to the installed SDK. Ensure settlement failures are observable and recoverable; provider callbacks must not be the only durable accounting record.

Client state

Current useChat comes from @ai-sdk/react, uses transport-based configuration, does not own text-input state, and returns status such as submitted, streaming, ready, and error.

'use client';

import { useChat } from '@ai-sdk/react';
import { DefaultChatTransport } from 'ai';
import { useState } from 'react';

export function Chat() {
  const [input, setInput] = useState('');
  const { messages, sendMessage, status, error, stop, regenerate } = useChat({
    transport: new DefaultChatTransport({ api: '/api/chat' }),
  });

  const busy = status === 'submitted' || status === 'streaming';
  // Render message parts for the installed SDK, including tool and error parts.
  // Keep the Stop action available while busy and Retry/Regenerate after failure.
  return null;
}

Render parts, not a legacy flat message.content, when using current UI messages. Distinguish submitted/no-token state from active streaming and partial completion.

Cancellation and charging

Pass the request abort signal upstream and expose Stop. Navigation/unmount alone may not provide durable accounting. For fixed-cost work, atomically reserve before starting and capture/release afterward. For metered work, settle actual usage, including partial/cancelled output, according to product policy. Concurrent requests must not overspend.

Version migration

When an existing app uses ai/react, handleInputChange, handleSubmit, isLoading, toDataStreamResponse, or maxTokens, consult the installed-major migration guide. Do not perform a partial API migration that changes only imports.

Verification

• Authorized and unauthorized requests are decided before stream creation.
• Invalid message shapes and oversized context are rejected.
• First-token delay, partial tokens, tool parts, finish, refusal, and error states render.
• Stop aborts upstream work; disconnect behavior is observed in server logs.
• Duplicate/concurrent requests respect idempotency and usage reservations.
• Settlement handles success, length, tool completion, cancellation, and provider error.
• Target runtime streams without buffering and passes lint/type/test/build.