Steve,
There are other groups claimed they work on HA solution.
We had discussions about it not so long ago in this list.
Is it possible that your colleagues present their design?
As you point out the issue gets fairly complex fast,
particularly because of the split-brain problem you describe.
There are several jiras dedicated to the problem already.
You can post your design there or create a new one.
> Looking at the facebook/google "multi-master" solution, I think they
> don't worry about consistency, just let the masters drift apart.
Not sure I follow this.
What facebook/google "multi-master" solution?
Why would they not worry about consistency?
Consistency of what?
Thanks,
--Konstantin
Steve Loughran wrote:
Konstantin Shvachko wrote:
And the only remaining step is to implement fail-over mechanism.
:)
Colleagues of mine work on HA stuff; I try and steer clear of it as it
gets complex fast. Test case: what happens when a network failure
splits the datacentre in two, you now have two clusters each with half
the data and possibly a primary/2ary master in each one. Then leave the
partition up for a while, do inconsistent operations on each then have
the network come back up. Then work out how to merge the state
Looking at the facebook/google "multi-master" solution, I think they
don't worry about consistency, just let the masters drift apart.
see also Johan's recent talk on HDFS:
http://www.slideshare.net/steve_l/hdfs
