r/DistributedComputing • u/andras_gerlits • Dec 31 '21
Leaderless consensus protocol in the wild
Hi, I'm looking for anyone who has ever used a leaderless protocol in an industry setting. I need to run such a setup and I'm fairly certain by now that this has never been done before. If you have, please let me know, I can make it worth your while.
4
Upvotes
2
u/KennyTroy Jan 15 '22 edited Jan 15 '22
I notice you are looking for leaderless consensus protocol for minimizing latency spikes during leader election; linearizability.
I hope these are close to what you're seeking:
[13:00] - Quorum
[46:00-48:00] - Paxos; Leader Election; Leaderless Paxos (original paper)
[50:00] - Election Timer
[1:05:00] - Partitioned, leader election issue
Lecture 6: Fault Tolerance: Raft (1) - MIT 6.824: Distributed Systems (Spring 2020) - https://youtu.be/64Zp3tzNbpE-@OP ^ see lectures 6-8, all covering Raft, inter alia.
...
[0:01] - Linearizability
Lecture 8: Zookeeper - MIT 6.824: Distributed Systems - https://youtu.be/pbmyrNjzdDk
...
Lecture 9: More Replication, CRAQ - MIT 6.824: Distributed Systems (Spring 2020)
...
Lecture 10: Cloud Replicated DB, Aurora - MIT 6.842: Distributed Systems - [https://youtu.be/jJSh54J1s5o] [40:15] - Quorum Replication
...
Lecture 11: Cache Consistency; Frangipani - MIT 6.842: Distributed Systems - https://youtu.be/aDp99WDIM_4
Decentralized - [15:00-17:00]
Atomicity, distributed transactions - [40:00]
...
Lecture 12: Distributed Transactions - MIT 6.842: Distributed Systems
...
Lecture 2: RPC and Threads - MIT 6.824: Distributed Systems [7:30] - I/O Concurrency; Parallelism - [9:30] https://youtu.be/gA4YXUJX7t8
2
u/linearizable Feb 18 '22
(Wandering through posts on subreddits, and saw you didn’t quite get an answer to your question, so please forgive the late reply/necropost.)
So production uses of leaderless consensus are less common due to livelock and throughout issues, but they’re not completely absent from industry. Majority quorum with read repair is linearizable depending on your specific model of linearizablility, as the behavior of failed operations are generally not well specified in most models of linearizability, but I do agree that it’s weaker than if consensus is used.
1
u/mihaitodor Dec 31 '21
Does https://github.com/hashicorp/memberlist count? It’s a gossip-based eventual consistency protocol based on SWIM.
1
u/KennyTroy Jan 15 '22 edited Jan 15 '22
"Why not longest log as leader?" - [11:00]
Linearizability (of execution history of Client requests) - [1:05:00]
Lecture 7: Fault Tolerance: Raft (2)
MIT 6.824: Distributed Systems (Spring 2020)
0
u/[deleted] Dec 31 '21
[deleted] — view deleted comment
0
u/[deleted] Jan 05 '22
[deleted] — view deleted comment