On Mon, Mar 14, 2016 at 4:51 PM, Jason Gustafson <ja...@confluent.io> wrote:
> >
> > Are you suggesting this as a solution for the problem where a KIP-35
> aware
> > client sends a higher version of metadata req, say v2, to a KIP-35 aware
> > broker that only supports up to v1.
>
>
> Yes, that's right. In that case, the client first sends v1, finds out that
> the broker supports v2, and then sends v2 (if it has any reason to do so).
>
> We had a bit of discussion on such scenarios, and it seemed to be a chicken
> > and egg problem that is hard to avoid. Your suggestion definitely makes
> > sense, however it falls under the purview of clients.
>
>
> That basically means clients should figure it out for themselves? Might be
> nice to have a better answer.
>
You mean to provide guidance for clients? I think that would makes sense.
However, I do not see how server can alleviate this issue.
>
> KIP-35 only aims on adding support for getting the version info from a
> > broker. This definitely can be utilized by our clients. However, that can
> > follow KIP-35 changes. Does this sound reasonable to you?
>
>
> It may be OK, but I'm a little concerned about offering a feature that we
> don't support ourselves. Sometimes it's not until implementation that we
> find out whether it really works as expected. And if we're eventually
> planning to support it, I feel we should think through some of the cases a
> bit more. For example, the upgrade and downgrade cases that Becket
> mentioned earlier. It doesn't feel too great to support this feature unless
> we can offer guidance on how to use it.
>
> Totally agreed with implementation is required to make sure the feature is
working as expected. Magnus and I had some offline chat and he is willing
to test this out with librdkafka.
Making our client utilize KIP-35 changes will be super awesome. My only
concern is that if we decide to make those changes as part of this KIP, we
will have to move KIP-35 out of 0.10 release scope.
If testing is the only concern, would librdkafka validation be good enough?
> -Jason
>
>
>
> On Mon, Mar 14, 2016 at 4:20 PM, Ashish Singh <asi...@cloudera.com> wrote:
>
> > Hi Jason,
> >
> > On Mon, Mar 14, 2016 at 4:04 PM, Jason Gustafson <ja...@confluent.io>
> > wrote:
> >
> > > Perhaps clients should always send the oldest version of the metadata
> > > request which supports KIP-35 when initially connecting to the cluster.
> > > Depending on the versions in the response, it can upgrade to a more
> > recent
> > > version. Then maybe we don't need the empty response hack?
> > >
> > Are you suggesting this as a solution for the problem where a KIP-35
> aware
> > client sends a higher version of metadata req, say v2, to a KIP-35 aware
> > broker that only supports up to v1.
> >
> > We had a bit of discussion on such scenarios, and it seemed to be a
> chicken
> > and egg problem that is hard to avoid. Your suggestion definitely makes
> > sense, however it falls under the purview of clients.
> >
> > >
> > > One thing that's not clear to me is whether the ultimate goal of this
> KIP
> > > is to have our clients support multiple broker versions. It would be a
> > > little weird to have this feature if our own clients don't use it.
> > >
> > KIP-35 only aims on adding support for getting the version info from a
> > broker. This definitely can be utilized by our clients. However, that can
> > follow KIP-35 changes. Does this sound reasonable to you?
> >
> > >
> > > -Jason
> > >
> > > On Mon, Mar 14, 2016 at 3:34 PM, Ashish Singh <asi...@cloudera.com>
> > wrote:
> > >
> > > > On Mon, Mar 14, 2016 at 2:12 PM, Ismael Juma <ism...@juma.me.uk>
> > wrote:
> > > >
> > > > > On Mon, Mar 14, 2016 at 8:45 PM, Gwen Shapira <g...@confluent.io>
> > > wrote:
> > > > > >
> > > > > > > I don't see how it helps. If the client is communicating with a
> > > > broker
> > > > > > that
> > > > > > > does not support KIP-35, that broker will simply close the
> > > > connection.
> > > > > If
> > > > > > > the broker supports KIP-35, then it will provide the broker
> > > version.
> > > > I
> > > > > > > don't envisage a scenario where a broker does not support
> KIP-35,
> > > but
> > > > > > > implements the new behaviour of sending an empty response. Do
> > you?
> > > > > > >
> > > > > > > Are you sure about that? Per KIP-35, the broker supplies the
> > > version
> > > > in
> > > > > > response to Metadata request, not in response to anything else.
> > > > > > If the client sends producer request version 42 (accidentally or
> > due
> > > to
> > > > > > premature upgrade) to KIP-35-compactible broker - we want to see
> an
> > > > empty
> > > > > > packet and not a connection close.
> > > > > > Sending a broker version was deemed impractical IIRC.
> > > > > >
> > > > >
> > > > > OK, so this is a different case than the one Ashish described
> > ("client
> > > > that
> > > > > wants to support broker versions that do not provide broker version
> > in
> > > > > metadata and broker versions that provides version info in
> > metadata").
> > > > So,
> > > > > you are suggesting that if a client is communicating with a broker
> > that
> > > > > implements KIP-35 and it receives an empty response, it will assume
> > > that
> > > > > the broker doesn't support the request version and it won't try to
> > > parse
> > > > > the response? I think it would be good to explain this kind of
> thing
> > in
> > > > > detail in the KIP.
> > > > >
> > > > Actually even in this case and the case I mentioned, closing
> connection
> > > > should be fine. Lets think about possible reasons that could lead to
> > this
> > > > issue.
> > > >
> > > > 1. Client has incorrect mapping of supported protocols for a broker
> > > > version.
> > > > 2. Client misread broker version from metadata response.
> > > > 3. Client constructed unsupported protocol version by mistake.
> > > >
> > > > In all the above cases irrespective of what broker does, client will
> > keep
> > > > sending wrong request version.
> > > >
> > > > At this point, I think sending an empty packet instead of closing
> > > > connection is a nice to have and not mandatory requirement. Like in
> the
> > > > above case, a client can catch parsing error and be sure that there
> is
> > > > something wrong in the protocol version it is sending. However, a
> > generic
> > > > connection close does not really provide any information on probable
> > > cause.
> > > >
> > > > What do you guys suggest?
> > > >
> > > > >
> > > > > Ismael
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > >
> > > > Regards,
> > > > Ashish
> > > >
> > >
> >
> >
> >
> > --
> >
> > Regards,
> > Ashish
> >
>
--
Regards,
Ashish
