Don't know if this would be an option:
If we defined and used a Java annotation which defines what Kafka-Version a
feature is available from (or up to which version it is supported) and then we
could do quick checks that compare the current version with the annotations on
the methods we call. I think this type of check should be quite easy to
understand and we wouldn't have to build, maintain, test, document etc. loads
of separate modules.
Chris
Am 12.03.18, 13:30 schrieb "vino yang" <yanghua1...@gmail.com>:
Hi Chris,
OK, Hope for listening someone's opinion.
Vino yang.
2018-03-12 20:23 GMT+08:00 Christofer Dutz <christofer.d...@c-ware.de>:
> Hi Vino,
>
> please don't interpret my opinion as some official project decision.
> For discussions like this I would definitely prefer to hear the opinions
> of others in the project.
> Perhaps having a new client API and having compatibility layers inside the
> connector would be another option.
> So per default the compatibility level of the Kafka client lib would be
> used but a developer could explicitly choose
> older compatibility levels, where we have taken care of the work to decide
> what works and what doesn't.
>
> Chris
>
>
>
> Am 12.03.18, 13:07 schrieb "vino yang" <yanghua1...@gmail.com>:
>
> Hi Chris,
>
> In some ways, I argee with you. Though kafka API has the
> compatibility. But
>
>
> - old API + higher server version : this mode would miss some key
> new
> feature.
> - new API + older server version : this mode, users are in a puzzle
> about which feature they could use and which could not. Also, new
> API will
> do more logic judgement and something else (which cause performance
> cost)
> for backward compatibility.
>
> I think it's the main reason that other framework split different
kafka
> connector with versions.
>
> Anyway, I will respect your decision. Can I claim this task about
> upgrading
> the kafka client's version to 1.x?
>
>
> 2018-03-12 16:30 GMT+08:00 Christofer Dutz <christofer.d...@c-ware.de
> >:
>
> > Hi Vino,
> >
> > I would rather go a different path. I talked to some Kafka pros and
> they
> > sort of confirmed my gut-feeling.
> > The greatest changes to Kafka have been in the layers behind the API
> > itself. The API seems to have been designed with backward
> compatibility in
> > mind.
> > That means you can generally use a newer API with an older broker as
> well
> > as use a new broker with an older API (This is probably even the
> safer way
> > around). As soon as you try to do something with the API which your
> broker
> > doesn't support, you get error messages.
> >
> > https://cwiki.apache.org/confluence/display/KAFKA/
> Compatibility+Matrix
> >
> > I would rather update the existing connector to a newer Kafka
> version ...
> > 0.8.2.2 is quite old and we should update to a version of at least
> 0.10.0
> > (I would prefer a 1.x) and stick with that. I doubt many will be
> using an
> > ancient 0.8.2 version (09.09.2015). And everything starting with
> 0.10.x
> > should be interchangeable.
> >
> > I wouldn't like to have yet another project maintaining a Zoo of
> adapters
> > for Kafka.
> >
> > Eventually a Kafka-Streams client would make sense though ... to
> sort of
> > extend the Edgent streams from the edge to the Kafka cluster.
> >
> > Chris
> >
> >
> >
> > Am 12.03.18, 03:41 schrieb "vino yang" <yanghua1...@gmail.com>:
> >
> > Hi guys,
> >
> > How about this idea, I think we should support kafka's new
> client API.
> >
> > 2018-03-04 15:10 GMT+08:00 vino yang <yanghua1...@gmail.com>:
> >
> > > The reason is that Kafka 0.9+ provided a new consumer API
> which has
> > more
> > > features and better performance.
> > >
> > > Just like Flink's implementation : https://github.com/apache/
> > > flink/tree/master/flink-connectors.
> > >
> > > vinoyang
> > > Thanks.
> > >
> > >
> >
> >
> >
>
>
>
