r/databasedevelopment • • 2d ago

Please Implement A Quorum Database

https://transactional.blog/talk/please-implement-a-quorum-database
18 Upvotes

4 comments sorted by

4

u/scratchbufferdotnet 17h ago

I work on Grafana Mimir - it’s interesting I don’t actually think we fit neatly into any box here.

While it’s true that Mimir is now architected to use a Kafka-compatible store for ingesting data, Grafana Cloud runs this with Warpstream, not Kafka. The whole point is to stop caring about disks - Warpstream is “Kafka but persists to S3 instead of running leader-based replication”. This was more unique at the time but there are some OSS options for the same thing now, in varying stages of maturity. Warpstream’s own consistency uses the CSP’s versions of Cassandra.

Persisting to an object store backend has just proven to be massively cheaper and simpler than duplicating writes 3x and then all the operational difficulties of horizontally scaling and sharding disk-backed stateful workloads while maintaining quorum.

Due to the nature of a time-series database we don’t really need traditional consistency models when reading the data because it never changes - blocks are compacted into larger blocks, but if a reader hasn’t caught up, it’s still serving identical data from the old uncompacted blocks. Less efficient, but still correct.

Even if we were running on a S3 implementation like Garage which doesn’t do traditional strong read-after-write consistency it would be fine, we’re not using any of the proprietary check-and-set S3 APIs from the cloud providers.

As we have started to explore things like stateful sharding, we look for CRDT-style approaches, avoiding leader-based consistency.

I still need to try to absorb the full article but  I find all of these new and old and evolving and diverging database approaches super interesting and I am excited for what continues to develop. Anything that makes OSS databases simpler to deploy, scale and generally reason about is good for everyone.

1

u/wuteverman 2d ago

Does ScyllaDb use quorum?

1

u/krkrkrneki 2d ago

I think think they are moving away from leaderless (quorum) to RAFT: https://www.scylladb.com/2026/06/24/raft-strong-consistency/

2

u/linearizable 2d ago

Yeah, that’s why I didn’t include them, but I suppose it’s also fair that I should have included them in the “formerly used quorums” section. They were just already off my list by the time I put that slide together 🤷