Oh I see, I thought that our tests are testing different Kafka versions right now. There is a test suite for 2.1.0 version in addition to 1.0.0 if I understand that gradle.build file correctly.
I don't mean to drop the support to earlier Kafka versions. I just mean to set the recent versions as default and add tests verifying that earlier versions are also still supported. On 2020/10/30 16:24:45, Alexey Romanenko <[email protected]> wrote: > We have a Jira task about testing KafkaIO against different Kafka versions > [1]. Once it will be implemented, then I think it should solve a potential > problem with new Kafka client versions. > > In the same time, imho, until there are Beam users, that use old Kafka client > versions, we need to support them. > > [1] https://issues.apache.org/jira/browse/BEAM-7003 > > > On 30 Oct 2020, at 10:05, Piotr Szuberski <[email protected]> > > wrote: > > > > In my opinion it would be good to keep Beam's dependencies as close to the > > recent stable versions as possible and, if needed, keep the support for > > earlier versions. > > > > For now we keep the old dependency as the base and test whether it works > > for some newer versions. That way we may always ignore the dependency check > > report that will get resolved > > at 1.0.0 and miss out that some important change has been released because > > every dependency check points at the Kafka dependency's deprecation. If we > > used newer version as the base then the Dependency check will do its job in > > this case. > > > > I've run Kafka both local and performance tests using kafka-clients:2.6.0 > > of Kafka clients and they passed - I'm not sure whether it answers your > > question about the implementation with the usage of ConsumerSpEL class. > > > > > > On 2020/10/28 17:06:56, Alexey Romanenko <[email protected]> wrote: > >> Piotr, thank you for tasing this question. Let me ask some questions > >> before. > >> > >> What will give us this dependencies update? What are the pros and cons? > >> Can users use recent versions of Kafka client with current implementation > >> based on ConsumerSpEL class? > >> > >> > >>> On 22 Oct 2020, at 10:47, Piotr Szuberski <[email protected]> > >>> wrote: > >>> > >>> Should we update Kafka dependencies to the recent ones (Kafka clients to > >>> 2.6.0 and Kafka_2.11 to 2.4.1)? > >>> > >>> What would have to be done to keep the backwards compatibility and which > >>> previous versions would we want to support? > >>> > >>> Kafka's backward compatibility is quite good so maybe there wouldn't be > >>> anything to do? > >>> > >>> Let's vote/discuss. > >> > >> > >
