Re: [infinispan-dev] ISPN-263 and handling partitions

2013-04-25 Thread Sanne Grinovero
Simple solution: each node works on power-via-ethernet: when the cable is unplugged, it's down. Then we think of a redundant design at the physical network level to make it unlikely. Sanne On 24 April 2013 16:38, Manik Surtani wrote: > > On 24 Apr 2013, at 16:08, Mircea Markus wrote: > >> >>

Re: [infinispan-dev] ISPN-263 and handling partitions

2013-04-24 Thread Manik Surtani
On 24 Apr 2013, at 16:08, Mircea Markus wrote: > > On 22 Apr 2013, at 17:43, Manik Surtani wrote: > >> On 22 Apr 2013, at 14:47, Bela Ban wrote: >> >>> >>> >>> On 4/19/13 11:51 AM, Sanne Grinovero wrote: TBH I'm not understanding which problem this thread is about :) Surely

Re: [infinispan-dev] ISPN-263 and handling partitions

2013-04-24 Thread Mircea Markus
On 22 Apr 2013, at 17:43, Manik Surtani wrote: > On 22 Apr 2013, at 14:47, Bela Ban wrote: > >> >> >> On 4/19/13 11:51 AM, Sanne Grinovero wrote: >>> TBH I'm not understanding which problem this thread is about :) >>> >>> Surely network partitions are a problem, but there are many forms of >

Re: [infinispan-dev] ISPN-263 and handling partitions

2013-04-24 Thread Mircea Markus
On 17 Apr 2013, at 13:23, Dan Berindei wrote: > I like the idea of always clearing the state in members of the minority > partition(s), but one problem with that is that there may be some keys that > only had owners in the minority partition(s). If we wiped the state of the > minority partitio

Re: [infinispan-dev] ISPN-263 and handling partitions

2013-04-24 Thread Mircea Markus
On 17 Apr 2013, at 16:53, Manik Surtani wrote: >> Of course, you could argue that the cluster already lost those keys when we >> allowed the majority partition to continue working without having those >> keys... We could also rely on the topology information, and say that we only >> support pa

Re: [infinispan-dev] ISPN-263 and handling partitions

2013-04-23 Thread Bela Ban
right On 4/23/13 7:26 PM, Manik Surtani wrote: > > On 23 Apr 2013, at 16:37, Bela Ban wrote: > >> >> >> On 4/22/13 6:46 PM, Manik Surtani wrote: >>> >>> On 22 Apr 2013, at 16:46, Mircea Markus wrote: >>> would the read only partition be wiped out and repopulated during merge? >>> >>> N

Re: [infinispan-dev] ISPN-263 and handling partitions

2013-04-23 Thread Manik Surtani
On 23 Apr 2013, at 16:37, Bela Ban wrote: > > > On 4/22/13 6:46 PM, Manik Surtani wrote: >> >> On 22 Apr 2013, at 16:46, Mircea Markus wrote: >> >>> would the read only partition be wiped out and repopulated during >>> merge? >> >> Naively, yes. But as Dan pointed out, if a key K only exi

Re: [infinispan-dev] ISPN-263 and handling partitions

2013-04-23 Thread Bela Ban
On 4/22/13 6:46 PM, Manik Surtani wrote: > > On 22 Apr 2013, at 16:46, Mircea Markus wrote: > >> would the read only partition be wiped out and repopulated during >> merge? > > Naively, yes. But as Dan pointed out, if a key K only exists in the > minor partition (primary and backups), then just

Re: [infinispan-dev] ISPN-263 and handling partitions

2013-04-22 Thread Manik Surtani
On 22 Apr 2013, at 16:46, Mircea Markus wrote: > would the read only partition be wiped out and repopulated during merge? Naively, yes. But as Dan pointed out, if a key K only exists in the minor partition (primary and backups), then just wiping the minor partition nodes will result in data

Re: [infinispan-dev] ISPN-263 and handling partitions

2013-04-22 Thread Manik Surtani
On 22 Apr 2013, at 14:47, Bela Ban wrote: > > > On 4/19/13 11:51 AM, Sanne Grinovero wrote: >> TBH I'm not understanding which problem this thread is about :) >> >> Surely network partitions are a problem, but there are many forms of >> "partition", and many different opinions of what an "acc

Re: [infinispan-dev] ISPN-263 and handling partitions

2013-04-22 Thread Manik Surtani
On 22 Apr 2013, at 14:24, Bela Ban wrote: > Yes, that's what we decided to do a long time ago... see policy based > merge approach in the mailing lists. > > But the point of *this* discussion is not data reconciliation after a > merge, but how to *avoid* that reconciliation by making conflict

Re: [infinispan-dev] ISPN-263 and handling partitions

2013-04-22 Thread Mircea Markus
On 5 Apr 2013, at 15:53, Manik Surtani wrote: > 1. We write a SplitBrainListener which is registered when the channel > connects. The aim of this listener is to identify when we have a partition. > This can be identified when a view change is detected, and the new view is > significantly sm

Re: [infinispan-dev] ISPN-263 and handling partitions

2013-04-22 Thread Bela Ban
On 4/19/13 11:51 AM, Sanne Grinovero wrote: > TBH I'm not understanding which problem this thread is about :) > > Surely network partitions are a problem, but there are many forms of > "partition", and many different opinions of what an "acceptable" > behaviour is that the grid should implement,

Re: [infinispan-dev] ISPN-263 and handling partitions

2013-04-22 Thread Bela Ban
Yes, that's what we decided to do a long time ago... see policy based merge approach in the mailing lists. But the point of *this* discussion is not data reconciliation after a merge, but how to *avoid* that reconciliation by making conflicting minority partitions read only, such that a merge a

Re: [infinispan-dev] ISPN-263 and handling partitions

2013-04-22 Thread Bela Ban
On 4/19/13 11:04 AM, Pedro Ruivo wrote: > I have another suggestion that can allow all the partitions to > process write commands. > > I think we can rely on the primary owners to decide which partition > has the write permission on a key. For example, if the primary owner > of key1 is on partit

Re: [infinispan-dev] ISPN-263 and handling partitions

2013-04-19 Thread Pedro Ruivo
After think a bit about it, we can conclude that the behaviour when we have partition totally depends on the application. So, why not create an generic interface (like, execute(Operation, Key, isPrimaryPartition)) that is invoked when a partition occurs? Maybe I'm losing the focus of the first

Re: [infinispan-dev] ISPN-263 and handling partitions

2013-04-19 Thread Sanne Grinovero
TBH I'm not understanding which problem this thread is about :) Surely network partitions are a problem, but there are many forms of "partition", and many different opinions of what an "acceptable" behaviour is that the grid should implement, which largely depend on assumptions the client applicat

Re: [infinispan-dev] ISPN-263 and handling partitions

2013-04-19 Thread Pedro Ruivo
On 04/05/2013 02:53 PM, Manik Surtani wrote: > 2. The SBL flips a switch in an interceptor (SplitBrainHandlerInterceptor?) > which switches the node to be read-only (reject invocations that change the > state of the local node) if it is in the smaller partition (newView.size < > oldView.size

Re: [infinispan-dev] ISPN-263 and handling partitions

2013-04-19 Thread Pedro Ruivo
On 04/18/2013 02:26 PM, Manik Surtani wrote: > I presume versioning (based on Vector Clocks) is now in master, as a > part of Pedro's work on total order? No, I didn't add vector clocks. ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org ht

Re: [infinispan-dev] ISPN-263 and handling partitions

2013-04-18 Thread Manik Surtani
On 18 Apr 2013, at 06:44, Dan Berindei wrote: > > > > On Wed, Apr 17, 2013 at 5:53 PM, Manik Surtani wrote: > > On 17 Apr 2013, at 08:23, Dan Berindei wrote: > >> I like the idea of always clearing the state in members of the minority >> partition(s), but one problem with that is that th

Re: [infinispan-dev] ISPN-263 and handling partitions

2013-04-18 Thread Dan Berindei
On Wed, Apr 17, 2013 at 5:53 PM, Manik Surtani wrote: > > On 17 Apr 2013, at 08:23, Dan Berindei wrote: > > I like the idea of always clearing the state in members of the minority > partition(s), but one problem with that is that there may be some keys that > only had owners in the minority part

Re: [infinispan-dev] ISPN-263 and handling partitions

2013-04-17 Thread Manik Surtani
On 17 Apr 2013, at 08:23, Dan Berindei wrote: > I like the idea of always clearing the state in members of the minority > partition(s), but one problem with that is that there may be some keys that > only had owners in the minority partition(s). If we wiped the state of the > minority partiti

Re: [infinispan-dev] ISPN-263 and handling partitions

2013-04-17 Thread Manik Surtani
On 17 Apr 2013, at 07:24, Bela Ban wrote: > If we go with a primary partition approach, then only the primary > partition will be allowed to make progress (a.k.a. accept changes), we > therefore won't have any conflicts. > > The partition approach must be chosen so there can only be 1 primary

Re: [infinispan-dev] ISPN-263 and handling partitions

2013-04-17 Thread Dan Berindei
ectly (assuming which one is newer or > older) which won't end up well. > > > > Radim > > > > - Original Message - > > | From: "Adrian Nistor" > > | To: "infinispan -Dev List" > > | Cc: "Manik Surtani" > &

Re: [infinispan-dev] ISPN-263 and handling partitions

2013-04-17 Thread Bela Ban
these topology ids directly (assuming which one is newer or older) > which won't end up well. > > Radim > > - Original Message - > | From: "Adrian Nistor" > | To: "infinispan -Dev List" > | Cc: "Manik Surtani" > | Sent: Wedne

Re: [infinispan-dev] ISPN-263 and handling partitions

2013-04-17 Thread Bela Ban
If we go with a primary partition approach, then only the primary partition will be allowed to make progress (a.k.a. accept changes), we therefore won't have any conflicts. The partition approach must be chosen so there can only be 1 primary partition max, and the minority partitions shut down

Re: [infinispan-dev] ISPN-263 and handling partitions

2013-04-17 Thread Radim Vansa
er or older) which won't end up well. Radim - Original Message - | From: "Adrian Nistor" | To: "infinispan -Dev List" | Cc: "Manik Surtani" | Sent: Wednesday, April 17, 2013 10:31:39 AM | Subject: Re: [infinispan-dev] ISPN-263 and handling partitio

Re: [infinispan-dev] ISPN-263 and handling partitions

2013-04-17 Thread Adrian Nistor
In case of MergeView the cluster topology manager running on (the new) coordinator will request the current cache topology from all members and will compute a new topology as the union of all. The new topology id is computed as the max + 2 of the existing topology ids. Any currently pending reb

Re: [infinispan-dev] ISPN-263 and handling partitions

2013-04-16 Thread Manik Surtani
Guys - I've started documenting this here [1] and will put together a prototype this week. One question though, perhaps one for Dan/Adrian - is there any special handling for state transfer if a MergeView is detected? - M [1] https://community.jboss.org/wiki/DesignDealingWithNetworkPartitions

Re: [infinispan-dev] ISPN-263 and handling partitions

2013-04-06 Thread Bela Ban
On 4/5/13 11:02 PM, Dennis Reed wrote: > Some other alternatives for detecting a split: > > - a hard-coded number of nodes required for a quorum configured by > the customer. This is what we recommend for HASIngletons when the > customer wants to guarantee that something is running at most once i

Re: [infinispan-dev] ISPN-263 and handling partitions

2013-04-06 Thread Bela Ban
On 4/5/13 3:53 PM, Manik Surtani wrote: > Guys, > > So this is what I have in mind for this, looking for opinions. > > 1. We write a SplitBrainListener which is registered when the > channel connects. The aim of this listener is to identify when we > have a partition. This can be identified wh

Re: [infinispan-dev] ISPN-263 and handling partitions

2013-04-05 Thread Dennis Reed
Some other alternatives for detecting a split: - a hard-coded number of nodes required for a quorum configured by the customer. This is what we recommend for HASIngletons when the customer wants to guarantee that something is running at most once in the cluster. - an alternative communicatio

[infinispan-dev] ISPN-263 and handling partitions

2013-04-05 Thread Manik Surtani
Guys, So this is what I have in mind for this, looking for opinions. 1. We write a SplitBrainListener which is registered when the channel connects. The aim of this listener is to identify when we have a partition. This can be identified when a view change is detected, and the new view is s