+1 Any sort of overly nice infra-software ends up being either - an disaster integration and operation disaster - a maintainer nightmares because of complex internal logic - a dangerously silent data-corruption contraption or any combination of the above.
Hadoop RPC versioning is somewhat close to the compatibility level concept here. And ot even simpler, I believe. Yet, it makes downstream development to be complex, bulky, hard to test, and error prone _even_ with right tools like Bigtop. Perhaps, it would make sense to reconsider the use case that lead to the proposal and think of different ways to solve it. Cos On September 30, 2015 1:28:52 AM PDT, "Branko Čibej" <br...@apache.org> wrote: >My point is that is has to be just one extensible protocol. So in your >case, either: in step 6, the node A would be able to read the >A-compatible bits of data; or: in step 5, the B-nodes will know that >the >persistent store was made with A. > >Doing it any other way will force you to dumb down your whole grid, >caches, etc. to the lowest common denominator; which makes >heterogeneous >grids impossible, and therefore makes rolling upgrades on a live grid >impossible. > >Of course you may not care about these scenarios. > >-- Brane > >On 30.09.2015 10:12, Vladimir Ozerov wrote: >> Brane, >> >> I see you point, but I do not see how we can implement it in our >> distributed environment. Very weird situations could appear. E.g. if >there >> are two versions A and B: >> 1) Node A starts. >> 2) Node B starts and agrees with A to use old A-protocol. >> 3) Some data is persisted on disk using A-protocol; >> 4) Cluster shuts down. >> 5) Several new B-nodes appear, but have no clue that something was >> persisted using A-protocol. They decide to use B-protocol. >> 6) Node A restarts and meets unknown B-protocol. >> >> On Wed, Sep 30, 2015 at 11:01 AM, Branko Čibej <br...@apache.org> >wrote: >> >>> On 30.09.2015 09:51, Vladimir Ozerov wrote: >>>> Igniters, >>>> >>>> Normally we are trying to maintain backward compatibility with >previous >>>> versions. But it is not always possible. >>>> >>>> E.g. we are about to release portable protocol. There are lots >>> suggestions >>>> how to optimize it, but all of them are relatively hard to >implement. It >>>> would not be a problem if are able to improve it iteratively from >release >>>> to release while still allowing for different versions (e.g. 1.5 >and 1.6) >>>> to communicate. >>>> >>>> What if we add a top-level property "*compatibility level*" >allowing user >>>> to "downgrade" some parts of the system to communicate with earlier >>>> versions? >>> That's likely to be a huge pain to maintain. The problem is that it >>> doesn't force you to think about compatibility too much, and you'll >end >>> up having any number of incompatible protocols that you'll have to >>> maintain simultaneously. >>> >>> There's an IMO better way: define the protocol to be natrually >>> extensible, and introduce a capability exchange step. Map protocol >>> extensions to capabilities (these can represented as simple tokens, >or >>> whatever), then make the client and server use any capabilities that >are >>> common to both. >>> >>> We do this in Subversion; that's how a 1.9 server can work with a >1.0 >>> client but the same server will work much better (with more features >>> available) with a 1.9 client. And the other way around, of course. >>> >>> -- Brane >>>
