OpenAI TTS Streaming Response in FastAPI: Setup Guide (2026)
- 8 hours ago
- 8 min read
Your voice assistant pipeline is fast in isolation. Whisper transcribes in under a second. The LLM replies in a second. OpenAI TTS synthesizes in another second. But end-to-end, the user clicks the mic button, speaks, releases — and waits four to five seconds of silence before any audio plays back. The pipeline is fast; the experience is broken. The fix isn't a faster model — it's the OpenAI TTS streaming response in FastAPI, so audio starts playing while the rest is still being synthesized.
TL;DR — the OpenAI TTS streaming response FastAPI pattern
To stream OpenAI TTS audio response through FastAPI: use
AsyncOpenAI.audio.speech.with_streaming_response.createwith response_format="mp3", wrap the byte stream in an async generator, and return it as a FastAPI StreamingResponse with media_type="audio/mpeg". Set the X-Accel-Buffering: no header to defeat reverse-proxy buffering and Cache-Control: no-cache so chunks aren't held back. On replies longer than ~100 words this cuts perceived latency from 3-5 seconds to under 1 second — the browser starts playing audio while the API call is still in flight.
Why streaming changes the user experience
Without streaming, the full audio buffer must be synthesized server-side, written to disk or memory, and only then sent to the browser. The user hears nothing for the entire duration of the OpenAI API round trip plus your own backend overhead. For a 100-word reply, that's typically 2-3 seconds of dead air.
With OpenAI TTS streaming response in FastAPI, the OpenAI API begins returning audio bytes as soon as the first phoneme is synthesized — usually within 300-500 ms of the request. FastAPI forwards those bytes through chunked HTTP transfer encoding. The browser's <audio> element starts decoding and playing the first chunk while the rest of the audio is still being generated. End-to-end perceived latency drops by 60-75% for any reply longer than a couple of sentences.
The math:
Reply length | Non-streaming wait | Streaming wait | Improvement |
1 sentence (~15 words) | ~1.0 s | ~0.5 s | ~50% |
1 short paragraph (~50 words) | ~2.0 s | ~0.6 s | ~70% |
1 long paragraph (~150 words) | ~4.0 s | ~0.7 s | ~83% |
Multi-paragraph (~300 words) | ~7.0 s | ~0.8 s | ~89% |
The longer the reply, the bigger the win. For voice assistants that occasionally produce longer answers (explanations, multi-step instructions), streaming is the single highest-leverage latency optimization.
How the streaming pipeline flows
LLM text answer
│
▼
AsyncOpenAI.audio.speech.with_streaming_response.create() ← async streaming
│ iter_bytes(chunk_size=4096)
▼
async generator (yield each chunk)
│
▼
FastAPI StreamingResponse (media_type="audio/mpeg")
│ Transfer-Encoding: chunked
│ X-Accel-Buffering: no
▼
Browser <audio> element
│
▼
Playback starts on the first chunk (~300-500 ms after request)
Step 1 - The streaming TTS service
Use the async OpenAI client. The blocking sync client doesn't compose cleanly with FastAPI's async streaming - you'd end up offloading to a thread and losing some of the perceived-latency win.
# app/services/tts_stream.py
from typing import AsyncIterator
from openai import AsyncOpenAI
aclient = AsyncOpenAI() # reads OPENAI_API_KEY from env
async def stream_speech(
text: str,
voice: str = "alloy",
model: str = "gpt-4o-mini-tts",
chunk_size: int = 4096,
) -> AsyncIterator[bytes]:
"""
Stream OpenAI TTS audio as chunked MP3 bytes.
Designed to be wired directly into a FastAPI StreamingResponse.
"""
async with aclient.audio.speech.with_streaming_response.create(
model=model,
voice=voice,
input=text,
response_format="mp3",
) as response:
async for chunk in response.iter_bytes(chunk_size=chunk_size):
yield chunk
Three deliberate choices:
AsyncOpenAI, not OpenAI. The sync client returns a context manager you have to wrap in anyio.to_thread.run_sync — that buffers chunks in the worker thread before they hit FastAPI's event loop, which partially defeats streaming. The async client streams natively.
response_format="mp3". MP3 (audio/mpeg) is the only format every modern browser's <audio> element decodes reliably as a chunked stream. Opus has lower latency in theory but inconsistent browser support for streamed playback.
chunk_size=4096. Small enough that the first chunk arrives quickly, large enough to avoid syscall overhead. Don't go below 1024 — many proxies will silently coalesce or drop tiny chunks.
Step 2 — The FastAPI streaming endpoint
# app/main.py
from fastapi import FastAPI
from fastapi.responses import StreamingResponse
from pydantic import BaseModel
from app.services.tts_stream import stream_speech
app = FastAPI()
class SpeakRequest(BaseModel):
text: str
voice: str = "alloy"
@app.post("/api/speak")
async def speak(req: SpeakRequest):
return StreamingResponse(
stream_speech(text=req.text, voice=req.voice),
media_type="audio/mpeg",
headers={
"Cache-Control": "no-cache",
"X-Accel-Buffering": "no",
"Content-Disposition": 'inline; filename="speech.mp3"',
},
)
The headers matter as much as the code:
media_type="audio/mpeg" — required for the browser to invoke the audio decoder. Without it some browsers will treat the response as a download.
Cache-Control: no-cache — without this, some CDNs and intermediate caches will buffer the full response before forwarding, which kills streaming. This single header has saved more streaming setups than any other tweak.
X-Accel-Buffering: no — Nginx-specific. If you reverse-proxy through Nginx (or many PaaS providers that do), this header tells Nginx not to buffer the response body. Without it, Nginx may hold chunks until it has the full response.
Content-Disposition: inline — tells browsers to play the audio, not download it.
Step 3 — The minimum viable browser test
You can test the streaming endpoint without writing any frontend code. Open the endpoint URL directly in Chrome with a POST tool (Postman, Insomnia, or curl) and pipe it to mpv or ffplay:
curl -N -X POST http://localhost:8000/api/speak \
-H "Content-Type: application/json" \
-d '{"text": "This is a streaming test. You should hear me start almost immediately, well before the full response has finished generating."}' \
| ffplay -nodisp -autoexit -
The -N flag disables curl's response buffering. If you hear audio within ~500 ms, the streaming chain is working. If you hear silence followed by the full clip, something in the chain is buffering — work backwards from the browser/CLI through Nginx to your FastAPI app.
Step 4 — Wiring it to a browser <audio> element
For most voice assistants the simplest pattern is: POST to the streaming endpoint, set the response stream as the src of an <audio> element via a blob URL.
// browser-side, simplified
async function speak(text) {
const res = await fetch("/api/speak", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ text }),
});
const audioEl = document.querySelector("audio");
audioEl.src = URL.createObjectURL(await res.blob());
audioEl.play();
}
This pattern is simple but it waits for the full blob before playback — you lose the streaming benefit. For true streaming playback in the browser, you need the MediaSource API (more complex, ~30 lines) or a streaming-aware library. For most voice-assistant use cases the perceived-latency win comes from the server side; the browser-side win is smaller and rarely worth the complexity.
When NOT to stream
Streaming has overhead. Use the non-streaming TTS endpoint when:
Replies are very short (< 30 words). The first chunk arrives in ~400 ms whether you stream or not — there's no perceptual difference for one-sentence replies.
You need to know the audio duration upfront (e.g., for synchronized animations or progress bars). A streamed response doesn't expose total length until it finishes.
You're caching synthesized audio (e.g., for FAQ-style replies that get spoken repeatedly). Cache the full MP3 once and serve it as a static asset.
Network conditions are unreliable (mobile-first apps). A dropped connection mid-stream is harder to recover from than a single failed batch request.
For a voice assistant that mostly produces 50-300 word replies on reasonably stable connections, streaming wins. Outside that envelope, batch may be simpler.
Common pitfalls in OpenAI TTS streaming with FastAPI
Why does my streaming response not actually stream - the audio plays all at once?
The most common cause is reverse-proxy buffering. Nginx, Cloudflare, or your hosting provider's edge layer is holding the response until it has the full body, then forwarding it in one piece. Fix: set X-Accel-Buffering: no and Cache-Control: no-cache response headers. If you control the proxy config directly, also disable proxy_buffering for the streaming route.
Should I use AsyncOpenAI or the sync client with anyio.to_thread.run_sync for streaming?
For streaming, use AsyncOpenAI. The sync client returns a context manager whose chunks are produced in a worker thread, and bridging that to FastAPI's async event loop adds buffering you don't want. The async client streams chunks natively into your async def generator, so they arrive in the FastAPI response with minimal added latency.
What audio format should I stream - MP3, Opus, or AAC?
MP3 (audio/mpeg). It's the only format that decodes reliably as a chunked stream in every modern browser's <audio> element. Opus has lower theoretical latency but browser support for streamed Opus playback is inconsistent. AAC works but adds licensing surface area. Stick with MP3 unless you have a specific reason not to.
How do I disable Nginx buffering for the streaming endpoint?
Two ways. (1) Send the X-Accel-Buffering: no response header from FastAPI - Nginx reads this header and disables buffering for that response. (2) Add proxy_buffering off; to the relevant location block in your Nginx config. The header approach is preferred because it's per-request and doesn't require redeploying the proxy config.
Can I stream the LLM and TTS together - token by token?
Yes, but it's significantly more complex. The pattern is: stream the LLM response, split on sentence boundaries, and trigger a TTS request per sentence. The first TTS request can start before the LLM finishes generating. This is the right architecture for true real-time voice assistants but adds ~150-200 lines of coordination code. For most use cases, streaming TTS alone gets you 80% of the perceived-latency win for 10% of the complexity.
Where this fits in a full voice assistant
This OpenAI TTS streaming response in FastAPI pattern is the synthesis half of a full voice-assistant pipeline. The complete flow is:
Browser records audio with MediaRecorder.
Audio is uploaded to FastAPI and transcribed by OpenAI Whisper — covered in our OpenAI Whisper + FastAPI integration example.
The transcript goes to an LLM agent (OpenAI Agents SDK) that may search a vector store for context.
The agent's text answer is streamed to the browser as OpenAI TTS audio (this post).
The browser plays the audio as soon as the first chunk arrives.
For the full architecture, all five layers wired together with browser code, Docker Compose, and tested cross-browser, read the AI Voice Assistant architecture deep-dive.
Grab the free PRD template
We've packaged the complete Product Requirements Document for this voice assistant — architecture, API spec, sprint plan, system prompt, the full STT/LLM/TTS contracts — as a downloadable PDF. Use it as a starting point for your own build.
Download the AI Voice Assistant PRD → (free, 362 KB)
Get the complete project
If you want the working repo - with OpenAI TTS streaming response in FastAPI already wired up, the Whisper STT side integrated, an LLM agent on top, and a Next.js frontend that plays the streamed audio - the AI Voice Assistant course on Codersarts Labs ships it all:
Complete commented source code for backend and frontend.
Step-by-step lessons across 5 modules / 20 lessons.
Docker Compose setup that runs the full stack with one command.
Production CORS, streaming headers, and reverse-proxy configuration.
Browser playback patterns for <audio> and MediaSource API.
Redis session store upgrade path for multi-worker deployments.
$29.99 self-paced. Everything above.
Closing
A correct OpenAI TTS streaming response in FastAPI setup is short - about 30 lines of Python - but the perceived-latency improvement is dramatic for any reply longer than a sentence or two. The three pieces that have to be right: use the async OpenAI client, return a StreamingResponse with the right media type, and defeat reverse-proxy buffering with explicit headers. Skip any one of them and your "streaming" endpoint will quietly batch in the background. With all three in place, the audio starts before the full response is generated - which is the difference between a voice assistant that feels real-time and one that feels broken.
Comments