Implement streaming responses from an LLM API to display tokens as they're generated.
Code Playground
JavaScript (V8)
Loading...
Console Output
// Output will appear here...
// For React: ReactDOM.createRoot(rootElement).render(<App />);
// For React: ReactDOM.createRoot(rootElement).render(<App />);
Recommended Answer
Implementation:
import OpenAI from 'openai';
const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });
async function streamChat(userMessage) {
const stream = await openai.chat.completions.create({
model: 'gpt-4',
messages: [{ role: 'user', content: userMessage }],
stream: true,
});
console.log('Assistant: ');
for await (const chunk of stream) {
const content = chunk.choices[0]?.delta?.content || '';
process.stdout.write(content); // Print without newline
}
console.log('\n'); // End with newline
}
// Usage
await streamChat("Tell me a short story about AI.");For web applications (React example):
async function streamToUI(userMessage, onToken) {
const stream = await openai.chat.completions.create({
model: 'gpt-4',
messages: [{ role: 'user', content: userMessage }],
stream: true,
});
for await (const chunk of stream) {
const content = chunk.choices[0]?.delta?.content || '';
if (content) {
onToken(content); // Callback to update UI
}
}
}
// In React component:
// streamToUI(message, (token) => setResponse(prev => prev + token));