Thanks Ivan and Utkarsh. I will open a Jira ticket and produce a patch. Regards, Aniruddha.
On Wed, May 9, 2012 at 2:07 PM, Utkarsh Srivastava <[email protected]>wrote: > We trying to cater for non all-to-all topologies, i.e., region A > subscribes to B which subscribes to C but there is no subscription > from A to C directly. > > That being said, I don't think the non all-to-all case was ever > fleshed out completely, and is quite complex. So, we could just change > the policy as you suggest to make it simpler and better. > > Utkarsh > > On Wed, May 9, 2012 at 12:46 PM, Ivan Kelly <[email protected]> wrote: > > Yes, I think what we have not is broken causal consistency, and the > > modification you suggest would fix it. > > > > Again, with regions A, B & C and topic t. > > > > Each region has a remote component = [A:0, B:0, C:0] > > A publishes Ua1 to t, which has rc [A:1,B:0,C:0], which is sent to B, > but delayed for C. > > > > A(rc) = [A:1, B:0, C:0] > > B(rc) = [A:1, B:0, C:0] > > C(rc) = [A:0, B:0, C:0] > > > > B publishes Ub1 to t, which has rc [A:1,B:1,C:0], which is sent to A > > and C. > > > > * This is where the schemes diverge > > 1. With the current scheme, the lastSeqIdPushed would be updated to > > [A:1,B:1,C:0] and a subscriber at C would recieve this as the remote > > component for both Ub1 and Ua1 when it eventually arrived. It could > > not tell which was first. > > > > 2. With the new scheme, Ub1 would be delivered to a subscriber at C > > with [A:1,B:1,C:0], and lastSeqIdPushed would be updated to > > [A:0,B:1,C:0]. Then when Ua1 arrives, it would be delivered to the > > subscriber with [A:1,B:0,C:0] and lastSeqIdPushed would be updated to > > [A:1,B:1,C:0]. With this, the subscriber could see that Ub1 was > > causally dependent on Ua1. > > > > -Ivan > > > > On Wed, May 09, 2012 at 09:45:20AM -0700, Aniruddha Laud wrote: > >> Even if B sees that the remote component for A received from C is not > >> monotonically increasing, would it make any difference in how region B > >> orders messages? What if B updated it's remote component for region A > only > >> when the message it received has srcRegion as A ? That way we could > still > >> guarantee that the remote components internally are monotonically > >> increasing and it would give us information on exactly how many > messages a > >> region has seen from any of the other regions. > >> > >> In the current scheme, after the merging, the remote component for > region X > >> gives the maximum messageID that any of the regions have seen from X for > >> that topic. > >> > >> Regards, > >> Aniruddha. > >> > >> On Wed, May 9, 2012 at 4:19 AM, Ivan Kelly <[email protected]> wrote: > >> > >> > On Tue, May 08, 2012 at 08:07:11PM -0700, Aniruddha Laud wrote: > >> > > When the RegionManager subscribes to topics from a remote hub, it > gets > >> > > merges the remote component part of the received message with that > of the > >> > > last persisted local message. The merge policy takes the maximum of > the > >> > two > >> > > values for any region X in these two messages. Why exactly is this > being > >> > > done? How is the remote component section from one region useful > for hubs > >> > > in another region? > >> > My understanding of this code is that it is trying to ensure that the > >> > remote components for a topic are monotonically increasing in the view > >> > of a single region. > >> > > >> > Take the example of 3 regions, A,B&C, with topic t. > >> > > >> > A publishes message 1, which B and C receives with remote components > >> > [A:1,B:null,C:null]. > >> > > >> > Now A publishs message 2, so which arrives quickly at B but with a > >> > delay at C. > >> > > >> > The last pushedSeqId for the topic is now > >> > at A: [A:2,B:null,C:null] > >> > at B: [A:2,B:null,C:null] > >> > at C: [A:1,B:null,C:null] > >> > > >> > C now publishes to topic t. The message has remote components of > >> > [A:1,B:null,C:1] > >> > > >> > When this arrives at B, if B does not do the merging, the client will > >> > see that the remote component for A is not increasing monotonically. > >> > > >> > Utkarsh can probably give you more detail on this. > >> > > >> > Regards > >> > Ivan > >> > >
