1. How Discord Stores Messages in 60 Seconds
Every message lands in one wide table partitioned by channel and time bucket. Writes are trivially routable: one message, one place. Discord outgrew MongoDB, moved to Cassandra in 2017 on twelve nodes, then grew to 177 nodes holding trillions of messages by 2022. Then it migrated everything to ScyllaDB and landed on 72 nodes at 15ms p99.
The drama was never writes. Opening a popular channel fires thousands of reads at the same partition within seconds. Replicas queue, quorum consistency spreads the delay to neighboring queries and JVM pauses stack on top. The cluster did not run out of disk. It ran out of schema.
2. The Components of Discord Message Storage
Clients open channels and poll history over persistent connections. Rust data services sit in front of the database, route each channel to one instance by consistent hashing and merge overlapping reads into single queries over gRPC. ScyllaDB stores the messages with Cassandra-compatible protocol and no garbage-collection pauses. Denser hybrid storage pairs persistent disks with NVMe SSDs. Everything else, presence and media and notifications, lives outside the message path.
The ordering matters. Coalescing came before the engine swap and bought years of runway alone. Never ask the database the same question twice is a routing decision, not a hardware purchase.
3. Hot Partitions and Quorum Amplification
Partition by channel and time: in this illustrative scenario one hot channel concentrates 5,000 reads per second onto a single partition. At replication factor 3 with quorum reads, that becomes 10,000 replica reads against a handful of nodes. Double the hot channels on event night and the saturated set grows faster than the fleet, because every hot read multiplies across replicas while new nodes only dilute cold load.
Discord reads and writes at quorum, so saturated nodes slow down strangers too: cold channels sharing hardware with a hot partition pay the hot partition's latency. Add Cassandra's garbage-collection pauses and compaction backlogs and three queues stack. Historical-message reads ran at a p99 between 40 ms and 125 ms on Cassandra while medians looked healthy; message inserts ranged from 5 ms to 70 ms. Medians lie about chat. Everyone notices the stuck message.
4. Request Coalescing: Ask Once
The fix routes one channel to one service instance, then merges every overlapping read for the same message into a single database query. Thousands of readers, one query. Consistent hashing matters here: random load balancing would scatter the crowd past the merger, while hash routing concentrates hot keys exactly where coalescing can reach them.
Then the engine swap removed the pause tax. ScyllaDB reimplements Cassandra storage in C++, keeping protocol and queries while deleting stop-the-world garbage collection. Denser disks cut utilization by a modeled 53 percent. Together: 105 nodes retired, a 59 percent smaller fleet, historical-read p99 at 15 ms and insert p99 at 5 ms. The migration itself staged recent data first and backfilled history behind it. When the stock migrator projected three months, engineers rewrote it in Rust in about a day. The rebuild moved data at up to 3.2 million messages per second and finished in nine days, pausing at 99.9999% because the last token ranges held tombstones Cassandra had never compacted away.
5. What It Costs
Model fleet cost from public cloud list prices for memory-heavy database machines and attached SSDs and the 105 retired nodes land in the low millions per year, before counting the eliminated firefighting toil. The overkill was compound interest on a schema and an engine and an operating model, priced monthly.
Price your own read path with the RPS envelope calculator: a few million chat events per second at a blended price, sustained all month, dwarfs the database line. Every duplicate read eliminated upstream deletes its share of that envelope.
6. The Verdict: What to Steal
Steal coalescing before hardware. Merging duplicate reads is a routing decision that costs one service layer and retires whole fleets. Most read-heavy products I look at run a smaller version of this incident: a write-modeled schema, a hot key, a fleet sized for the average.
Steal the hot-set dashboards next. Track per-partition heat, per-replica queue depth, coalescing hit rate. The cluster is healthy when its hottest partition is bored. Steal the compatible swap last. Same protocol, no pause tax, evenings preserved.
Do not steal 72 nodes. That number belongs to Discord's crowd. What transfers is the order of operations: merge first, measure second, migrate third. Hardware is what you buy after the clever is exhausted, not instead of it.
177 nodes. 72 nodes. One coalescing layer. The crowd was the capacity all along.
7. Frequently Asked Questions
How does Discord store messages?
In a wide-column store partitioned by channel and time bucket. Discord moved from MongoDB to Cassandra in 2017, then migrated trillions of messages to ScyllaDB, cutting 177 nodes to 72.
Why did Discord leave Cassandra for ScyllaDB?
Hot partitions and JVM garbage-collection pauses pushed p99 latency toward 125ms across 177 nodes. ScyllaDB speaks the same protocol without the pause tax. A Rust coalescing layer removed duplicate reads first.
What is request coalescing?
Merging overlapping reads into one database query. Discord routes each channel to one service instance by consistent hashing, so thousands of readers opening the same channel cost a single query.
What database does Discord use now?
ScyllaDB for message storage at 15ms p99, with Rust data services in front. Relational stores were never the message path.
Sources and Method
Storage facts follow Discord Engineering publications and ScyllaDB case coverage. Node counts, latencies and migration steps come from the published accounts. Cost framing is modeled from public cloud list prices.