Implement streaming responses from an LLM API to display tokens as they're generated.

Code Playground
JavaScript (V8)
Loading...
Console Output
// Output will appear here...
// For React: ReactDOM.createRoot(rootElement).render(<App />);
Recommended Answer

Implementation:

import OpenAI from 'openai';

const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

async function streamChat(userMessage) {
  const stream = await openai.chat.completions.create({
    model: 'gpt-4',
    messages: [{ role: 'user', content: userMessage }],
    stream: true,
  });

  console.log('Assistant: ');
  
  for await (const chunk of stream) {
    const content = chunk.choices[0]?.delta?.content || '';
    process.stdout.write(content); // Print without newline
  }
  
  console.log('\n'); // End with newline
}

// Usage
await streamChat("Tell me a short story about AI.");

For web applications (React example):

async function streamToUI(userMessage, onToken) {
  const stream = await openai.chat.completions.create({
    model: 'gpt-4',
    messages: [{ role: 'user', content: userMessage }],
    stream: true,
  });

  for await (const chunk of stream) {
    const content = chunk.choices[0]?.delta?.content || '';
    if (content) {
      onToken(content); // Callback to update UI
    }
  }
}

// In React component:
// streamToUI(message, (token) => setResponse(prev => prev + token));