Server Components and Streaming AI Interfaces

The first version of every AI interface has the same shape. Someone adds 'use client' to the page so they can hold the streaming response in state. The page now ships as a client bundle, the model client is one careless import away from the browser, and the static two-thirds of the layout re-renders on every token.

None of that is necessary. The streaming part of an AI interface is genuinely small — usually one element — and everything around it can stay on the server. Getting that boundary right is most of the work, and it is a rendering problem before it is an AI problem.

Two kinds of streaming, routinely confused

They are unrelated mechanisms and they solve different problems.

HTML streaming is a server rendering technique. React renders the page and flushes it in pieces as data resolves; a <Suspense> boundary marks a hole, the fallback ships immediately, and the real content replaces it when ready. The browser gets useful markup before the slowest data source has responded. This happens once per navigation.

Token streaming is a data-transfer technique. The model emits text incrementally, your server relays it, and a client component appends to state as it arrives. This is a long-lived response body being consumed in a loop, and it needs a client.

The confusion is costly because it leads people to make the whole page a Client Component in order to get the second one. You want both, at different scopes: HTML streaming for the page, token streaming for one element inside it.

Where the boundary belongs

A useful test: a component must be a Client Component only if it needs an event handler, a browser API, or state that changes after hydration. Everything else — data fetching, layout, headings, prior conversation history, the empty state — is server work.

For a typical assistant page that means:

app/assistant/page.jsx           Server  — layout, metadata, history fetch
  components/ConversationList    Server  — renders stored messages
  components/AssistantPanel      Client  — the streaming leaf, ~80 lines
  components/SuggestedPrompts    Server  — static list, no interactivity

The client bundle is one file. The API key, the database client and the model wrapper are imported only from server files, so there is no path by which they can be bundled.

HTML streaming: the shell first

Conversation history often comes from a database, and there is no reason for the page header and the input box to wait for it.

import { Suspense } from 'react';
import { getConversation } from '@/lib/db/conversations';
import AssistantPanel from '@/components/assistant/AssistantPanel';
import ConversationList from '@/components/assistant/ConversationList';

export const metadata = {
  title: 'Assistant',
  description: 'Ask questions about your workspace.',
};

export default async function AssistantPage({ params }) {
  const { id } = await params;

  return (
    <main>
      <h1>Assistant</h1>

      <Suspense fallback={<ConversationSkeleton />}>
        <ConversationHistory id={id} />
      </Suspense>

      <AssistantPanel conversationId={id} />
    </main>
  );
}

async function ConversationHistory({ id }) {
  const messages = await getConversation(id);
  return <ConversationList messages={messages} />;
}

function ConversationSkeleton() {
  return <div aria-hidden="true" role="presentation" />;
}

Line 14 marks the hole. Everything outside it — the heading, the panel, the layout — reaches the browser without waiting on the database. Line 17 keeps the async work in a separate component, which is what makes the boundary meaningful: await inside the page component itself would delay the whole page, <Suspense> or not.

Two details that catch people. params is awaited in Next.js 16, not read synchronously. And the fallback should reserve approximately the height of the real content — a zero-height fallback produces a layout shift the moment data lands, which costs you on Cumulative Layout Shift for no benefit.

Token streaming into the leaf

The server side is a route handler that relays the provider's stream. Newline-delimited JSON keeps the client parser trivial and lets you send structured events, not just text.

Define the event shape once. Even in a JavaScript project it is worth writing down as types, because this contract crosses the network:

export type AssistantEvent =
  | { type: 'delta'; text: string }
  | { type: 'sources'; items: { title: string; url: string }[] }
  | { type: 'done'; messageId: string }
  | { type: 'error'; message: string };

The handler:

import { streamAnswer } from '@/lib/assistant/stream';

export const runtime = 'nodejs';
export const maxDuration = 60;

export async function POST(request) {
  const { conversationId, question } = await request.json();

  if (typeof question !== 'string' || !question.trim()) {
    return Response.json({ error: 'Question required' }, { status: 400 });
  }

  const encoder = new TextEncoder();
  const upstream = new AbortController();
  request.signal.addEventListener('abort', () => upstream.abort());

  const body = new ReadableStream({
    async start(controller) {
      const send = (event) =>
        controller.enqueue(encoder.encode(`${JSON.stringify(event)}\n`));

      try {
        for await (const event of streamAnswer({
          conversationId,
          question,
          signal: upstream.signal,
        })) {
          send(event);
        }
      } catch (error) {
        if (error.name !== 'AbortError') {
          send({ type: 'error', message: 'The assistant could not answer.' });
        }
      } finally {
        controller.close();
      }
    },
  });

  return new Response(body, {
    headers: {
      'content-type': 'application/x-ndjson; charset=utf-8',
      'cache-control': 'no-store',
      'x-accel-buffering': 'no',
    },
  });
}

Line 10 validates before spending anything. Line 24 forwards the abort signal to the provider, so a closed tab stops the generation instead of leaving it running and billing. Line 33 distinguishes a genuine failure from a cancellation — an aborted stream is not an error and should not render as one.

The x-accel-buffering: no header matters behind reverse proxies that buffer responses by default. A stream that arrives all at once at the end is indistinguishable, to a user, from no streaming at all.

The client leaf, and the bug everyone hits

'use client';

import { useRef, useState } from 'react';

export default function AssistantPanel({ conversationId }) {
  const [question, setQuestion] = useState('');
  const [answer, setAnswer] = useState('');
  const [sources, setSources] = useState([]);
  const [status, setStatus] = useState('idle');
  const abortRef = useRef(null);

  async function handleSubmit(event) {
    event.preventDefault();
    setAnswer('');
    setSources([]);
    setStatus('streaming');

    const controller = new AbortController();
    abortRef.current = controller;

    try {
      const response = await fetch('/api/assistant', {
        method: 'POST',
        signal: controller.signal,
        headers: { 'content-type': 'application/json' },
        body: JSON.stringify({ conversationId, question }),
      });

      const reader = response.body.pipeThrough(new TextDecoderStream()).getReader();
      let buffer = '';

      while (true) {
        const { value, done } = await reader.read();
        if (done) break;

        buffer += value;
        const lines = buffer.split('\n');
        buffer = lines.pop() ?? '';

        for (const line of lines) {
          if (!line.trim()) continue;
          const event = JSON.parse(line);
          if (event.type === 'delta') setAnswer((prev) => prev + event.text);
          if (event.type === 'sources') setSources(event.items);
          if (event.type === 'error') setStatus('error');
        }
      }
      setStatus((prev) => (prev === 'error' ? prev : 'done'));
    } catch (error) {
      if (error.name !== 'AbortError') setStatus('error');
    } finally {
      abortRef.current = null;
    }
  }

  return (
    <form onSubmit={handleSubmit}>
      <label htmlFor="assistant-question">Your question</label>
      <textarea
        id="assistant-question"
        value={question}
        onChange={(event) => setQuestion(event.target.value)}
        rows={3}
        required
      />

      <button type="submit" disabled={status === 'streaming'}>
        Ask
      </button>
      {status === 'streaming' && (
        <button type="button" onClick={() => abortRef.current?.abort()}>
          Stop
        </button>
      )}

      <output htmlFor="assistant-question" aria-live="polite" aria-busy={status === 'streaming'}>
        {answer}
      </output>

      {sources.length > 0 && (
        <ul>
          {sources.map((source) => (
            <li key={source.url}>
              <a href={source.url} target="_blank" rel="noopener noreferrer">
                {source.title}
              </a>
            </li>
          ))}
        </ul>
      )}
    </form>
  );
}

Line 23 passes the abort signal into fetch, which is what makes the Stop button real rather than decorative. Lines 31 and 40 are the chunk-boundary fix: a network chunk can split a JSON line anywhere, so the trailing fragment must be carried into the next read. Omitting this produces the classic bug where everything works locally — small responses arrive in one chunk — and JSON.parse throws in production.

Note setAnswer((prev) => prev + event.text) rather than setAnswer(answer + …). Deltas arrive faster than renders, and reading answer from the closure drops text.

Keeping the transcript on the server

The client holds only the in-flight answer. Persisting belongs on the server, where the database credentials are, and a Server Action is the least ceremonious way to do it:

'use server';

import { revalidatePath } from 'next/cache';
import { appendMessage } from '@/lib/db/conversations';

export async function saveExchange(conversationId, question, answer) {
  await appendMessage(conversationId, { role: 'user', content: question });
  await appendMessage(conversationId, { role: 'assistant', content: answer });
  revalidatePath(`/assistant/${conversationId}`);
}

Call it from the leaf once the stream reports done. History then re-renders as a Server Component with no client-side state to keep in sync — which removes the entire category of bug where the visible transcript and the stored transcript disagree.

Rendering streamed markdown without thrash

Model output is usually markdown, and parsing a partial document on every delta is both wasteful and visually unstable: an unclosed code fence flips the rest of the answer in and out of a <pre> as tokens arrive.

Two mitigations. Render plain text while streaming and parse markdown once on done — simple, and users rarely notice. Or, if incremental formatting matters, buffer at paragraph boundaries and only parse completed blocks, holding the trailing partial block as plain text.

Either way, pin the container so growth does not shove the page around, and respect motion preferences on any caret or fade you add:

.assistant-answer {
  min-height: 6rem;
  overflow-anchor: none;
}

.assistant-caret {
  animation: assistant-blink 1s steps(2, start) infinite;
}

@media (prefers-reduced-motion: reduce) {
  .assistant-caret {
    animation: none;
    opacity: 1;
  }
}

overflow-anchor: none prevents the browser's scroll anchoring from fighting your own auto-scroll as content grows — a small line that fixes a jitter that is otherwise very hard to diagnose.

Accessibility of something that arrives over time

Streaming text is hostile to screen readers by default: an aria-live="assertive" region announces every delta and produces unusable noise.

Use aria-live="polite" so announcements queue, and prefer a native <output> element, which carries the right role without extra attributes. Set aria-busy while streaming so assistive technology knows the content is still settling. Never move focus to the answer mid-stream — it interrupts whatever the user is doing and steals it from the input.

The Stop button deserves specific attention. It must be a real <button>, reachable by keyboard, present in the DOM while streaming and removed when not — not disabled, not visually hidden.

What to measure

Time to first token is the number users feel; total generation time is not. A response that starts in a moment and takes a while to finish reads as fast. One that takes the same total time with nothing on screen for the first half reads as broken.

Then check the boring things: the client bundle for this route (it should be the leaf and its dependencies, nothing more), Cumulative Layout Shift when the answer replaces the fallback, and the abort rate — a high one means users are giving up, which is a latency signal, not a quality signal.

The server side of this — bounded loops, budgets, degradation paths — is covered in building production-ready AI applications, and a complete worked agent using this exact streaming pattern is in how to build an AI agent with Next.js. If you are choosing the tools you will build it with, best AI coding tools for developers covers that layer.