Skip to main content
Streamstraight provides reliable, robust LLM streams from a server source to your frontend client. It connects to both your server and client via WebSockets and socket.io for a reliable connection and temporarily persists each stream to Redis for replayability.

​
Streamstraight Server

Your server/async worker is where the LLM stream begins. Connect to Streamstraight using our server SDK, and pass an [async] iterable of chunks to our SDK. Any data type serializable to JSON can be passed in. The SDK handles opening up a WebSocket connection to Streamstraight and persisting each chunk to Redis. Once each chunk is persisted to Redis, your frontend client can connect to Streamstraight and request them in order.

​
Streamstraight Client

Your frontend client can directly connect to Streamstraight using our Client SDK. When the client connects, Streamstraight will replay the stream from the beginning and continue tailing until the stream from your server ends. Streams are uniquely identified by a streamId parameter and live in Redis for at least 60 minutes. Do not treat Streamstraight as your database—the temporary storage is solely for replayability.

​
Build with confidence

Streamstraight solves the biggest problems with streaming LLM generations from your server to client:
  • Reliable, buffered streams over flaky connections
  • Streams originating from any server, including async queues and background jobs
  • Replayability for clients connecting or reconnecting to the stream
  • Multiple clients connecting to the same stream
  • Multiple streams to a single frontend client
  • Easy-to-use server and client SDKs