Re: issues with Kafka ConsumeKafka_0_10 (1.0.0)

2017-06-23 Thread Jeremy Farbota
Haha, well put. Switching from .6 to 1.0.0 was quite a mountain for my dev ops guy who was completely new to NiFi. I expect this next upgrade to be a lot smoother. Thanks for the tip btw. It was definitely a back pressure issue. The weird thing is the back pressure seemed to be shared within the p

Re: issues with Kafka ConsumeKafka_0_10 (1.0.0)

2017-06-23 Thread Joe Witt
I get it. If there are curveballs with upgrades that you run into and ideas you have to make it easier please let us know. We are moving toward the extension registry concept which will help a ton in that regard and we've done a lot with backward/forward compatible bits but upgrades can still mak

Re: issues with Kafka ConsumeKafka_0_10 (1.0.0)

2017-06-23 Thread Jeremy Farbota
I see. That makes some sense. I've been reading about the improvements and it is good to reaffirm the new version will likely help us out a lot. We do not plan on getting this far behind again. [image: Payoff, Inc.] *Jeremy Farbota* Software Engineer, Data Payoff, Inc. jfarb...@payoff.com (217)

Re: issues with Kafka ConsumeKafka_0_10 (1.0.0)

2017-06-23 Thread Joe Witt
Jeremy It is possible that backpressure was being engaged in NiFi and causing our consumer code to handle it poorly. We did fix that a while ago and I think it ended up in NiFi 1.2.0 (off top of mind anyway). Between your current release and the latest 1.3.0 release a few bugs with those process