Why I Built Celeris

A specific kind of frustration comes when you know that your architecture is wrong and you cannot repair it cleanly. It is not a bug that you can find. It is not a performance problem that you can profile and optimize. It is a structural problem in a platform that you do not control, and it appears under load, at the worst possible times.

I lived with that feeling for too long. This is the story of what I did about it.

The Firebase problem

Some years ago, I worked on a product that needed realtime functions at a serious scale. It had high concurrency, high message throughput, and many simultaneous connections. In that product, the realtime layer was not a feature. It carried the load.

We used Firebase, specifically the Firebase Realtime Database.

RTDB is genuinely good for many use cases. It works for small teams, moderate concurrency, and quick prototypes. The problem is that it has no concept of horizontal sharding. There is one database. When you fill it, or when too many concurrent connections hit it, there is no gradual degradation path. You get a ceiling.

We hit that ceiling.

The simple solution (one RTDB instance for everything) was not possible. Thus, I designed a workaround. My approach used Firestore as the primary datastore. It replicated the data that needed realtime access into multiple RTDB instances. The routing logic sent each user to the RTDB shard that held their data. In theory, this distributed the load.

In practice, the data lived in two places at the same time. The routing logic had to agree with the location of the data. The state was scattered across Firestore and the active RTDB shards. If the shard of a user was under load, their experience degraded, and the architecture did not show that clearly. We stitched together two products that were not designed for this, and every seam showed.

Each traffic spike found a new failure point. The architecture worked until it failed, and it failed too frequently. My time went to shard assignments and dropped connections, not to product features.

The memory that stays with me is not the technical embarrassment of the workaround. It is the feeling of responsibility for a system with a foundation that you do not trust. You examine your dashboards too frequently. You fear traffic spikes. You do not hope for them.

The search for something better

After that experience, I examined the alternatives properly.

The requirements were clear: high throughput, high concurrency, and multi-region support. The critical requirement was realtime delivery that did not make the database a liability under load.

That last requirement needs an explanation, because it shaped everything that came after.

Most teams use this pattern: receive a realtime message, then write it to Postgres or to another central relational database. This works at a modest scale. At a serious scale, a burst of inbound messages becomes a burst of write pressure directly on the database. Relational databases cannot absorb that load without careful protection. Connection pools fill, and latency climbs. With bad luck, the failures cascade.

I was close enough to this problem to know that it was real. I wanted a delivery layer that kept the database off the delivery path. The layer receives the message and delivers it to subscribers. The application persists the message at its own speed, not at the speed of the traffic spike.

I examined Ably, Pusher, and PubNub. Each is a solid product in a different way. But none of them solves this at the architecture level. They are managed convenience layers across WebSocket delivery. They move your messages to clients reliably, with good tools.

They do not give you a Kafka-backed delivery layer with a write-only export connector. With that connector, your own queue receives the messages, and you persist to your database when you are ready. That design decision was not there, and it was the one that my previous job needed.

So I decided to build it.

The honest origin story

Here is the part that I will not decorate. I selected Rust initially because I wanted to learn Rust.

Before that, I wrote mostly TypeScript and Go. Rust was on my list for a long time because of its reputation for correctness, its memory model, and the absence of a garbage collector. I wanted to build something non-trivial in it, not a toy project. I wanted real concurrency, real IO, and real pressure. A WebSocket server seemed to be the correct shape of problem.

The realtime infrastructure problem gave me a reason to make the project real. Thus, I built a WebSocket platform in Rust, on Actix Web and Tokio, with Kafka behind it and Redis for the connection state.

The learning-project frame did not survive contact with the results.

The number that changed things

During development, I ran a proper load test. It was not a quick smoke test. It was a sustained benchmark. I pointed k6 at a single AWS instance of the type c8g.large and let the test run for 25 minutes.

The numbers were 96,114 messages per second, 102 MB of memory, and 56% CPU.

I looked at that output for a while.

I knew that Rust was efficient in theory. But the theory is different from the sight of 96,000 messages per second at 102 megabytes of memory on one mid-range EC2 instance. This was not a tuned result from ideal conditions. This was a sustained 25-minute test, and it included the overhead of the k6 clients that made the load.

This told me that the economics of the platform were different from my assumption. A single node could do the work of a cluster of less efficient alternatives. With that memory footprint, many more instances can operate on the same hardware budget. With that CPU headroom, you can scale vertically before you must scale horizontally.

At that point, this was not a learning project anymore. If a single node could do this, developers deserved access to it as a managed service. They did not need to build the platform themselves.

The binary gap

During the build, I saw the same thing in each of the existing platforms. They all use JSON as their base.

This is not exactly a criticism. JSON is universal, and it is easy to read and use. But it is also expensive. It is verbose on the wire and slow to parse at a high volume. For some applications (games, trade infrastructure, IoT), the payload size and the parse latency are genuine constraints, and the cost is real. Bandwidth costs money, CPU cycles at scale cost money, and the serialization overhead on each message adds up.

MessagePack and Protobuf exist. They are genuinely better formats for high-frequency binary data. But the existing managed platforms ignore them or treat them as extras. Ably documents MessagePack support, but it is not the default or a first-class design decision. You manage Protobuf yourself. The transport layer uses text as its base, and you always pay that tax.

I wanted a platform with first-class binary formats from the start, not a plugin or a compatibility mode. I wanted the default path for teams that know their data shape and do not want a string conversion first.

The principles

Between the first prototype that worked and the multitenant version, a set of principles emerged. I was not willing to compromise on them.

Pure WebSocket underneath. The SDK is a convenience layer, not a proprietary protocol. If you want raw WebSocket without the SDK, you can connect that way. The protocol is documented. You are not trapped.

Your data goes to infrastructure that you own. BYOD means that you bring your own destination, and it will arrive soon. Write-only export connectors will send data to Kafka, RabbitMQ, S3, and HTTP endpoints. When Celeris accepts a message, the message will flow into your own queue. You consume from that queue at your own speed. Your database stays off the spike path.

Honest pricing. Billing has three dimensions: messages, connection minutes, and channel minutes. The prices are on the website. You do not need a sales call to know if you can afford it. The 10-year option bills the base plan fee at the start, and it renews automatically every 10 years unless you cancel. Usage charges stay separate.

A transport layer, not an application layer. Celeris does not want to be your chat SDK or your collaborative editor. It wants to be the layer that you build those products on. It has one job, and it does that job well.

What it is now

Celeris is live. It has two regions, US and EU. Celeris routes connections through AWS Global Accelerator. This network has 130 Points of Presence in 95 cities across 53 countries. SDKs are available for JavaScript and TypeScript, Python, Go, and Java, and Rust and Dart SDKs are in development.

The transport is binary-first, and each publisher's messages arrive in the order that it sent them. BYOD export connectors are next.

This is v1, and there is more to build. But the foundation is there, and it works. That foundation is the part that mattered, the part that my previous job needed.

This is the platform that I wanted to exist. Now it does.