That is a fair concern and I think eventually we might want to have java clients backwards compatible. However, blocking KIP-35 on that might not be the best idea. The reason I say so is due to following reasons.
1. Backwards compatibility in java clients is a larger discussion, as we have already seen. Having a separate KIP focussing exactly on that will help in reducing moving pieces in one KIP. 2. It probably will fall out of 0.10 due to tight timeline, I am assuming we do not want to delay 0.10 a lot. 3. Even if we make java clients backward compatible in 0.10, I do not think it will be able to work with older releases as older broker versions still won't provide info on supported api versions. If we add api versions req/resp in 0.10, and add backwards compatibility in java clients in later versions, they will probably work with 0.10 and we will be able to test that. On Fri, Apr 1, 2016 at 10:28 AM, Ewen Cheslack-Postava <e...@confluent.io> wrote: > On Fri, Apr 1, 2016 at 10:22 AM, Gwen Shapira <g...@confluent.io> wrote: > > > I have two concerns with the proposal as it is: > > > > 1. Having an API that publishes protocol is useless for clients if > > developers don't bump the API when they should. > > I would like to see good documentation on when protocols are bumped and a > > proposal on how this will be automatically validated to the extent > possible > > (and to what extent are we still at risk of accidental breakage). > > Even if we don't implement all of this at first pass, I want to know > which > > direction we are going in order to solve the client compatibility issue. > > > > 2. In addition to third-party clients, there are stream processing > > frameworks who use the Java client we publish as their client and would > > also like to enjoy the same compatibility benefits C and Python clients > > will enjoy. It will be very silly if Apache Kafka clients are the worst > > clients out there from compatibility POV. The KIP can be implemented in > > parts, but I really want to see an effort to build the Java client > > compatibility into the KIP and if possible into the release too. > > > > I wouldn't conflate those two things. Changing the compatibility approach > of the Java clients could easily be another KIP (and probably a large one, > too). There are already high quality, well maintained clients trying to use > a different approach and this addresses their needs. Blocking the entire > ecosystem on the Java client seems problematic. > > That said, agreed that it would be bad for the Java client to have the > worst compatibility story. > > -Ewen > > > > > > Gwen > > > > On Thu, Mar 31, 2016 at 2:51 PM, Jason Gustafson <ja...@confluent.io> > > wrote: > > > > > Bumping this thread. I talked with Ashish and Magnus about this KIP > > offline > > > and I'm gradually coming over. The new API actually stands by itself > > > outside of the discussion about whether the client should support > > backwards > > > compatibility or not. For the Java client, we could continue to support > > the > > > current compatibility approach in which the client supports only > brokers > > > with the same version or greater. In that case, we would use this API > > only > > > to assert that the current API versions are all supported, and raise an > > > exception if they are not. This gives us the capability going forward > to > > > detect when the client is talking to an older broker, which we don't > have > > > right now. This check should be straightforward, so we could do it now, > > > which would resolve some of the uneasiness about having an unused > feature > > > which we depended on other clients to test for us. Does that make sense > > or > > > not? > > > > > > -Jason > > > > > > On Thu, Mar 17, 2016 at 4:06 PM, Ashish Singh <asi...@cloudera.com> > > wrote: > > > > > > > We have proposed and discussed majorly three approaches so far, there > > > were > > > > many minor versions with small variations. Comparing them really > > > requires a > > > > side by side proposal and their pros/cons, and I agree with others > that > > > > this has been lacking in the KIP. We just updated the KIP with > > following > > > > details. > > > > > > > > 1. Provide proposed changes in all the three proposals we have > > discussed > > > so > > > > far. Except the current proposal, these proposals are in rejected > > > > alternatives. > > > > 2. Provide reasoning on why the rejected proposals were rejected. > > > > 3. Add scenarios for all of these proposals from a client developer > and > > > > core Kafka developer point of view. > > > > > > > > As we are really close to 0.10 deadline, a quick round of voting will > > > > really help. If you really do not like the idea, please feel free to > > say > > > > so. If the vote fails for the current proposal, it can at lease > provide > > > > recommendations that we should consider for next version of proposal > > and > > > > put it up for vote again for next release. However, as stated earlier > > by > > > > multiple people having this ASAP will be awesome. > > > > > > > > On Thu, Mar 17, 2016 at 3:29 PM, Dana Powers <dana.pow...@gmail.com> > > > > wrote: > > > > > > > > > On Thu, Mar 17, 2016 at 1:42 PM, Gwen Shapira <g...@confluent.io> > > > wrote: > > > > > > > > > > > "I think I would make this approach work by looking at the > released > > > > > server > > > > > > version documentation for each version that I am trying to > support > > > and > > > > > test > > > > > > against*, manually identify the expected "protocol vectors" each > > > > > supports, > > > > > > store that as a map of vectors to "broker versions", check each > > > vector > > > > at > > > > > > runtime until I find a match, and write code compatibility checks > > > from > > > > > > there." > > > > > > > > > > > > How is this better than a global version ID? > > > > > > > > > > > > > > > As a client developer, it seems roughly the same. I think it > probably > > > > > avoids the server development workflow issues, and possibly the > need > > to > > > > > agree on semantics of the global version ID? But others surely are > > more > > > > > qualified than me to comment on that part. > > > > > > > > > > -Dana > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > Regards, > > > > Ashish > > > > > > > > > > > > > -- > Thanks, > Ewen > -- Regards, Ashish
