AI & agent infrastructure

The problem

AI sessions make high-frequency traffic with low latency. Each token from a model is a small payload. But JSON encoding for each token, HTTP connection churn, or a poll loop adds up fast when users expect instant and continuous delivery.

The designers of most realtime platforms did not plan for this pattern. Those platforms are optimized for occasional messages between humans, not for hundreds of small binary frames per second from a model inference server.

How Celeris solves it

Celeris is a persistent WebSocket transport. When the session is open, it stays open. There is no reconnection overhead per token and no HTTP round-trip per chunk. With a binary-first transport, you can keep each token frame small. MessagePack encodes it efficiently, with no required JSON wrapper.

Each agent session maps naturally to a channel segment. Many users can see the same session, and many agents can share one context. They write to and read from the same segment with no extra coordination logic.

In a measured test, the p95 round-trip latency was 32 ms. At that latency, the transport layer is not the bottleneck between the model and the user.

Open a streaming session

const client = createClient({ credentialProvider });
const channel = client.channel("agent-sessions");

const session = channel.segment(`session-${sessionId}`);
session.onMessage((payload) => {
  appendToken(readJson(payload).token);
});
session.subscribe();

await channel.connect();

// From your inference server, publish each token as it arrives
await session.publish({
  payload: jsonPayload({ token: nextToken, done: false }),
});

The session stays live until you close it. The SDK manages the reconnection logic.

Relevant features

• Binary-first messaging: MessagePack token frames, with no JSON overhead per chunk
• Channels and segments: one segment per session, with support for many observers
• p95 32ms latency: a measured single-node result. In that test, the transport added far less delay than a model needs to produce its tokens
• Persistent WebSocket: no reconnect overhead between tokens