Yeah the intent is to support 0.8 and 0.9. Will figure something out. Thanks Joe On Feb 21, 2016 4:47 PM, "West, Joshua" <josh_w...@bose.com> wrote:
> Hi Oleg, > > Hmm -- from what I can tell, this isn't a Zookeeper communication issue. > Nifi is able to connect into the Kafka brokers' Zookeeper cluster and > retrieve the list of the kafka brokers to connect to. Seems, from the > logs, to be a problem when attempting to consume from Kafka itself. > > I'm guessing that the Kafka 0.9.0 client libraries just are not compatible > with Kafka 0.8.2.1 so in order to use Nifi 0.5.0 with Kafka, the Kafka > version must be >= 0.9.0. > > Any change Nifi could add backwards compatible support for Kafka 0.8.2.1 > too? Let you choose which client library version, when setting up the > GetKafka processor? > > -- > Josh West <josh_w...@bose.com> > Bose Corporation > > > > On Sun, 2016-02-21 at 15:02 +0000, Oleg Zhurakousky wrote: > > Josh > > Also, keep in mind that there are incompatible property names in Kafka > between the 0.7 and 0.8 releases. One of the change that went it was > replacing “zk.connectiontimeout.ms” with “zookeeper.connection.timeout.ms”. > Not sure if it’s related though, but realizing that 0.4.1 was relying on > this property it’s value was completely ignored with 0.8 client libraries > (you could actually see the WARN message to that effect) and now it is not > ignored, so take a look and see if tinkering with its value changes > something. > > Cheers > Oleg > > On Feb 20, 2016, at 6:47 PM, Oleg Zhurakousky < > ozhurakou...@hortonworks.com> wrote: > > Josh > > The only change that ’s went and relevant to your issue is the fact that > we’ve upgraded client libraries to Kafka 0.9 and between 0.8 and 0.9 Kafka > introduced wire protocol changes that break compatibility. > I am still digging so stay tuned. > > Oleg > > On Feb 20, 2016, at 4:10 PM, West, Joshua <josh_w...@bose.com> wrote: > > Hi Oleg and Joe, > > Kafka 0.8.2.1 > > Attached is the app log with hostnames scrubbed. > > Thanks for your help. Much appreciated. > > -- > Josh West <josh_w...@bose.com> > Bose Corporation > > > > On Sat, 2016-02-20 at 15:46 -0500, Joe Witt wrote: > > And also what version of Kafka are you using? > On Feb 20, 2016 3:37 PM, "Oleg Zhurakousky" <ozhurakou...@hortonworks.com> > wrote: > > Josh > > Any chance to attache the app-log or relevant stack trace? > > Thanks > Oleg > > On Feb 20, 2016, at 3:30 PM, West, Joshua <josh_w...@bose.com> wrote: > > Hi folks, > > I've upgraded from Nifi 0.4.1 to 0.5.0 and I am no longer able to use the > GetKafka processor. I'm seeing errors like so: > > 2016-02-20 20:10:14,953 WARN > [ConsumerFetcherThread-NiFi-sldjflkdsjflksjf_**SCRUBBED**-1455999008728-5b8c7108-0-0] > kafka.consumer.ConsumerFetcherThread > [ConsumerFetcherThread-NiFi-sldjflkdsjflksjf_**SCRUBBED**-1455999008728-5b8c7108-0-0], > Error in fetchkafka.consumer.ConsumerFetcherThread$FetchRequest@7b49a642. > Possible cause: java.lang.IllegalArgumentException > > ^ Note the hostname of the server has been scrubbed. > > My configuration is pretty generic, except that with Zookeeper we use a > different root path, so our Zookeeper connect string looks like so: > > zookeeper-node1:2181,zookeeper-node2:2181,zookeeper-node3:2181/kafka > > Is anybody else experiencing issues? > > Thanks. > > -- > Josh West <josh_w...@bose.com> > > Cloud Architect > Bose Corporation > > > > > > <nifi-app.log.kafkaissues.bz2> > > > >
