Re: PutKafka won't Put! "Failed while waiting for acks from Kafka"

[Andrew Grande]

Pat, which NiFi version is that? Which Kafka broker version on the other
end?

Andrew

On Sat, Jun 11, 2016 at 12:15 PM Pat Trainor  wrote:

> Users,
>
> Has anyone come across PutKafka simply not wanting to send to Kafka?
>
> In this troubleshooting, I have replicated symptoms in the simplest way I
> can-with a GenerateFlowFile and a PutKafka:
>
>
> With a known good Kafka running (other /producers using the same Topic)...
>
> ...yields:
> Auto refresh started
> 11:17:19 EDT WARNING
> 2b76634c-6354-4c3c-b977-86f73d64efbcPutKafka[id=2b76634c-6354-4c3c-b977-86f73d64efbc]
> Processor Administratively Yielded for 1 sec due to processing failure
> 11:18:00 EDT ERROR
> 2b76634c-6354-4c3c-b977-86f73d64efbcPutKafka[id=2b76634c-6354-4c3c-b977-86f73d64efbc]
> Failed while waiting for acks from Kafka
> 11:18:00 EDT ERROR
> 2b76634c-6354-4c3c-b977-86f73d64efbcPutKafka[id=2b76634c-6354-4c3c-b977-86f73d64efbc]
> PutKafka[id=2b76634c-6354-4c3c-b977-86f73d64efbc] failed to process due to
> java.lang.NullPointerException; rolling back session:
> java.lang.NullPointerException
> 11:18:00 EDT ERROR 2b76634c-6354-4c3c-b977-86f73d64efbc
>
> PutKafka[id=2b76634c-6354-4c3c-b977-86f73d64efbc]
> PutKafka[id=2b76634c-6354-4c3c-b977-86f73d64efbc] failed to process session
> due to java.lang.NullPointerException: java.lang.NullPointerException
>
> Everything is super-stock, out of the box, and not customized. In fact, I
> tested Kafka with the Apache test consumer/producer pair, and am trying to
> send to the same topic, same consumer, still running, after it worked for
> the test producer.
>
> Funny thing is, I see this error in forums/emails lately, but nobody nails
> it down as to why it is occurring...
>
> ==> nifi-app.log <==
> 2016-06-11 12:04:24,172 WARN [kafka-producer-network-thread |
> SecondPutKafka2k] org.apache.kafka.common.network.Selector Error in I/O
> with localhost/127.0.0.1
> java.io.EOFException: null
> at
> org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:62)
> ~[kafka-clients-0.8.2.2.jar:na]
> at org.apache.kafka.common.network.Selector.poll(Selector.java:248)
> ~[kafka-clients-0.8.2.2.jar:na]
> at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:192)
> [kafka-clients-0.8.2.2.jar:na]
> at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:191)
> [kafka-clients-0.8.2.2.jar:na]
> at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:122)
> [kafka-clients-0.8.2.2.jar:na]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_91]
>
> attached has screenshots...
>
> I've slowed down the 'schedules', messes with Message Delimiter, and what
> seems like all the obvious things... This makes me think it is obvious, so
> on day 2, I'm reaching out to see if any of you have seen/fixed this...
>
> Thanks!
>
> pat 
> ( ͡° ͜ʖ ͡°)
>
>

Re: PutKafka won't Put! "Failed while waiting for acks from Kafka"

[Pat Trainor]

Andrew,

  

Thanks for writing.

  

_Nifi:_

nifi-0.6.1

_Kafka:_

kafka_2.11-0.10.0.0

  

startup with no changes to configs:

  

#

Kafka:

bin/zookeeper-server-start.sh config/zookeeper.properties &  
bin/kafka-server-start.sh config/server.properties &  
#  
# create a topic  
#  
bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1
--partitions 1 --topic test  

[...]

  

_Nifi's zookeeper.properties:_

  

  

clientPort=2181  
initLimit=10  
autopurge.purgeInterval=24  
syncLimit=5  
tickTime=2000  
dataDir=./state/zookeeper  
autopurge.snapRetainCount=30  
  
#  
# server.1=nifi-node1-hostname:2888:3888  
# server.2=nifi-node2-hostname:2888:3888  
# server.3=nifi-node3-hostname:2888:3888  
#  
**server.1=  
**

  

^^

Even though I have:

  

_nifi.state.management.embedded.zookeeper.start=**false**  
_

  

I filled in the above setting... changing it to:

  

**server.1=localhost:2888:3888  
**

  

...which made no difference, of course...

  

Any thoughts?

  

Thanks!  
  

[pat](http://about.me/PatTrainor)  

( ͡° ͜ʖ ͡°)  
  

On Jun 11 2016, at 12:17 pm, Andrew Grande  wrote:  

> Pat, which NiFi version is that? Which Kafka broker version on the other
end?

>

>  

>

> Andrew

>

>  

>

> On Sat, Jun 11, 2016 at 12:15 PM Pat Trainor
<[pat.trai...@gmail.com](mailto:pat.trai...@gmail.com)> wrote:  

>

>> Users,

>>

>>  

>>

>> Has anyone come across PutKafka simply not wanting to send to Kafka?

>>

>>  

>>

>> In this troubleshooting, I have replicated symptoms in the simplest way I
can-with a GenerateFlowFile and a PutKafka:

>>

>>  

>>

>> ![](https://community.hortonworks.com/storage/temp/4972-screenshot-
from-2016-06-11-11-34-25.png)

>>

>> With a known good Kafka running (other /producers using the same Topic)...

>>

>> ...yields:

>>

>> Auto refresh started

>>

>> 11:17:19 EDT WARNING 2b76634c-6354-4c3c-b977-86f73d64efbcPutKafka[id=2b7663
4c-6354-4c3c-b977-86f73d64efbc] Processor Administratively Yielded for 1 sec
due to processing failure

>>

>> 11:18:00 EDT ERROR 2b76634c-6354-4c3c-b977-86f73d64efbcPutKafka[id=2b76634c
-6354-4c3c-b977-86f73d64efbc] Failed while waiting for acks from Kafka

>>

>> 11:18:00 EDT ERROR 2b76634c-6354-4c3c-b977-86f73d64efbcPutKafka[id=2b76634c
-6354-4c3c-b977-86f73d64efbc]
PutKafka[id=2b76634c-6354-4c3c-b977-86f73d64efbc] failed to process due to
java.lang.NullPointerException; rolling back session:
java.lang.NullPointerException

>>

>> 11:18:00 EDT ERROR 2b76634c-6354-4c3c-b977-86f73d64efbc

>>

>> PutKafka[id=2b76634c-6354-4c3c-b977-86f73d64efbc]
PutKafka[id=2b76634c-6354-4c3c-b977-86f73d64efbc] failed to process session
due to java.lang.NullPointerException: java.lang.NullPointerException

>>

>>  

>>

>> Everything is super-stock, out of the box, and not customized. In fact, I
tested Kafka with the Apache test consumer/producer pair, and am trying to
send to the same topic, same consumer, still running, after it worked for the
test producer.

>>

>>  

>>

>> Funny thing is, I see this error in forums/emails lately, but nobody nails
it down as to why it is occurring...

>>

>>  

>>

>> ==> nifi-app.log <==  
2016-06-11 12:04:24,172 WARN [kafka-producer-network-thread |
SecondPutKafka2k] org.apache.kafka.common.network.Selector Error in I/O with
localhost/[127.0.0.1](http://127.0.0.1)  
java.io.EOFException: null  
at org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java
:62) ~[kafka-clients-0.8.2.2.jar:na]  
at org.apache.kafka.common.network.Selector.poll(Selector.java:248) ~[kafka-
clients-0.8.2.2.jar:na]  
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:192) [kafka-
clients-0.8.2.2.jar:na]  
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:191)
[kafka-clients-0.8.2.2.jar:na]  
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:122)
[kafka-clients-0.8.2.2.jar:na]  
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_91]  

>>

>>  

>>

>> attached has screenshots...

>>

>>  

>>

>> I've slowed down the 'schedules', messes with Message Delimiter, and what
seems like all the obvious things... This makes me think it is obvious, so on
day 2, I'm reaching out to see if any of you have seen/fixed this...

>>

>>  

>>

>> Thanks!  
  

>>

>> [pat](http://about.me/PatTrainor)  

>>

>> ( ͡° ͜ʖ ͡°)  
  

Re: PutKafka won't Put! "Failed while waiting for acks from Kafka"

[Joe Witt]

Pat

Could you please make sure we are seeing the full stacktrace from the app
log?  If we could get a full log that would he even better.

Thanks
Joe
On Jun 11, 2016 1:17 PM, "Pat Trainor"  wrote:

> Andrew,
>
> Thanks for writing.
>
> *Nifi:*
> nifi-0.6.1
> *Kafka:*
> kafka_2.11-0.10.0.0
>
> startup with no changes to configs:
>
> #
> Kafka:
> bin/zookeeper-server-start.sh config/zookeeper.properties &
> bin/kafka-server-start.sh config/server.properties &
> #
> # create a topic
> #
> bin/kafka-topics.sh --create --zookeeper localhost:2181
> --replication-factor 1 --partitions 1 --topic test
> [...]
>
> *Nifi's zookeeper.properties:*
>
>
> clientPort=2181
> initLimit=10
> autopurge.purgeInterval=24
> syncLimit=5
> tickTime=2000
> dataDir=./state/zookeeper
> autopurge.snapRetainCount=30
>
> #
> # server.1=nifi-node1-hostname:2888:3888
> # server.2=nifi-node2-hostname:2888:3888
> # server.3=nifi-node3-hostname:2888:3888
> #
>
> *server.1=*
>
> ^^
> Even though I have:
>
>
> *nifi.state.management.embedded.zookeeper.start=false*
>
> I filled in the above setting... changing it to:
>
>
> *server.1=localhost:2888:3888*
>
> ...which made no difference, of course...
>
> Any thoughts?
>
> Thanks!
>
> pat 
> ( ͡° ͜ʖ ͡°)
>
> On Jun 11 2016, at 12:17 pm, Andrew Grande  wrote:
>
>> Pat, which NiFi version is that? Which Kafka broker version on the other
>> end?
>>
>> Andrew
>>
>> On Sat, Jun 11, 2016 at 12:15 PM Pat Trainor 
>> wrote:
>>
>> Users,
>>
>> Has anyone come across PutKafka simply not wanting to send to Kafka?
>>
>> In this troubleshooting, I have replicated symptoms in the simplest way I
>> can-with a GenerateFlowFile and a PutKafka:
>>
>>
>> With a known good Kafka running (other /producers using the same Topic)...
>>
>> ...yields:
>> Auto refresh started
>> 11:17:19 EDT WARNING
>> 2b76634c-6354-4c3c-b977-86f73d64efbcPutKafka[id=2b76634c-6354-4c3c-b977-86f73d64efbc]
>> Processor Administratively Yielded for 1 sec due to processing failure
>> 11:18:00 EDT ERROR
>> 2b76634c-6354-4c3c-b977-86f73d64efbcPutKafka[id=2b76634c-6354-4c3c-b977-86f73d64efbc]
>> Failed while waiting for acks from Kafka
>> 11:18:00 EDT ERROR
>> 2b76634c-6354-4c3c-b977-86f73d64efbcPutKafka[id=2b76634c-6354-4c3c-b977-86f73d64efbc]
>> PutKafka[id=2b76634c-6354-4c3c-b977-86f73d64efbc] failed to process due to
>> java.lang.NullPointerException; rolling back session:
>> java.lang.NullPointerException
>> 11:18:00 EDT ERROR 2b76634c-6354-4c3c-b977-86f73d64efbc
>>
>> PutKafka[id=2b76634c-6354-4c3c-b977-86f73d64efbc]
>> PutKafka[id=2b76634c-6354-4c3c-b977-86f73d64efbc] failed to process session
>> due to java.lang.NullPointerException: java.lang.NullPointerException
>>
>> Everything is super-stock, out of the box, and not customized. In fact, I
>> tested Kafka with the Apache test consumer/producer pair, and am trying to
>> send to the same topic, same consumer, still running, after it worked for
>> the test producer.
>>
>> Funny thing is, I see this error in forums/emails lately, but nobody
>> nails it down as to why it is occurring...
>>
>> ==> nifi-app.log <==
>> 2016-06-11 12:04:24,172 WARN [kafka-producer-network-thread |
>> SecondPutKafka2k] org.apache.kafka.common.network.Selector Error in I/O
>> with localhost/127.0.0.1
>> java.io.EOFException: null
>> at
>> org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:62)
>> ~[kafka-clients-0.8.2.2.jar:na]
>> at org.apache.kafka.common.network.Selector.poll(Selector.java:248)
>> ~[kafka-clients-0.8.2.2.jar:na]
>> at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:192)
>> [kafka-clients-0.8.2.2.jar:na]
>> at
>> org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:191)
>> [kafka-clients-0.8.2.2.jar:na]
>> at
>> org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:122)
>> [kafka-clients-0.8.2.2.jar:na]
>> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_91]
>>
>> attached has screenshots...
>>
>> I've slowed down the 'schedules', messes with Message Delimiter, and what
>> seems like all the obvious things... This makes me think it is obvious, so
>> on day 2, I'm reaching out to see if any of you have seen/fixed this...
>>
>> Thanks!
>>
>> pat 
>> ( ͡° ͜ʖ ͡°)
>>
>>

Re: PutKafka won't Put! "Failed while waiting for acks from Kafka"

[Pat Trainor]

Really?! Sure! Why don't I clear everything out, and run just the generator
& PutKafka, then I'll clip the logs from that so it's clean (with app
startup, etc.)...  
  
Gimme a minute to do that...  

  

Thanks a bunch for writing back  
  

[pat](http://about.me/PatTrainor)  

( ͡° ͜ʖ ͡°)  
  

On Jun 11 2016, at 1:38 pm, Joe Witt  wrote:  

> Pat

>

> Could you please make sure we are seeing the full stacktrace from the app
log?  If we could get a full log that would he even better.

>

> Thanks  
Joe

>

> On Jun 11, 2016 1:17 PM, "Pat Trainor"
<[pat.trai...@gmail.com](mailto:pat.trai...@gmail.com)> wrote:  

>

>> Andrew,

>>

>>  

>>

>> Thanks for writing.

>>

>>  

>>

>> _Nifi:_

>>

>> nifi-0.6.1

>>

>> _Kafka:_

>>

>> kafka_2.11-0.10.0.0

>>

>>  

>>

>> startup with no changes to configs:

>>

>>  

>>

>> #

>>

>> Kafka:

>>

>> bin/zookeeper-server-start.sh config/zookeeper.properties &  
bin/kafka-server-start.sh config/server.properties &  
#  
# create a topic  
#  
bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1
--partitions 1 --topic test  

>>

>> [...]

>>

>>  

>>

>> _Nifi's zookeeper.properties:_

>>

>>  

>>

>>  

>>

>> clientPort=2181  
initLimit=10  
autopurge.purgeInterval=24  
syncLimit=5  
tickTime=2000  
dataDir=./state/zookeeper  
autopurge.snapRetainCount=30  
  
#  
# server.1=nifi-node1-hostname:2888:3888  
# server.2=nifi-node2-hostname:2888:3888  
# server.3=nifi-node3-hostname:2888:3888  
#  
**server.1=  
**
>>

>>  

>>

>> ^^

>>

>> Even though I have:

>>

>>  

>>

>> _nifi.state.management.embedded.zookeeper.start=**false**  
_

>>

>>  

>>

>> I filled in the above setting... changing it to:

>>

>>  

>>

>> **server.1=localhost:2888:3888  
**
>>

>>  

>>

>> ...which made no difference, of course...

>>

>>  

>>

>> Any thoughts?

>>

>>  

>>

>> Thanks!  
  

>>

>> [pat](http://about.me/PatTrainor)  

>>

>> ( ͡° ͜ʖ ͡°)  
  

>>

>> On Jun 11 2016, at 12:17 pm, Andrew Grande
<[apere...@gmail.com](mailto:apere...@gmail.com)> wrote:  

>>

>>> Pat, which NiFi version is that? Which Kafka broker version on the other
end?

>>>

>>>  

>>>

>>> Andrew

>>>

>>>  

>>>

>>> On Sat, Jun 11, 2016 at 12:15 PM Pat Trainor
<[pat.trai...@gmail.com](mailto:pat.trai...@gmail.com)> wrote:  

>>>

 Users,



  



 Has anyone come across PutKafka simply not wanting to send to Kafka?



  



 In this troubleshooting, I have replicated symptoms in the simplest way I
can-with a GenerateFlowFile and a PutKafka:



  



 ![](https://community.hortonworks.com/storage/temp/4972-screenshot-
from-2016-06-11-11-34-25.png)



 With a known good Kafka running (other /producers using the same
Topic)...



 ...yields:



 Auto refresh started



 11:17:19 EDT WARNING 2b76634c-6354-4c3c-b977-86f73d64efbcPutKafka[id=2b76
634c-6354-4c3c-b977-86f73d64efbc] Processor Administratively Yielded for 1 sec
due to processing failure



 11:18:00 EDT ERROR 2b76634c-6354-4c3c-b977-86f73d64efbcPutKafka[id=2b7663
4c-6354-4c3c-b977-86f73d64efbc] Failed while waiting for acks from Kafka



 11:18:00 EDT ERROR 2b76634c-6354-4c3c-b977-86f73d64efbcPutKafka[id=2b7663
4c-6354-4c3c-b977-86f73d64efbc]
PutKafka[id=2b76634c-6354-4c3c-b977-86f73d64efbc] failed to process due to
java.lang.NullPointerException; rolling back session:
java.lang.NullPointerException



 11:18:00 EDT ERROR 2b76634c-6354-4c3c-b977-86f73d64efbc



 PutKafka[id=2b76634c-6354-4c3c-b977-86f73d64efbc]
PutKafka[id=2b76634c-6354-4c3c-b977-86f73d64efbc] failed to process session
due to java.lang.NullPointerException: java.lang.NullPointerException



  



 Everything is super-stock, out of the box, and not customized. In fact, I
tested Kafka with the Apache test consumer/producer pair, and am trying to
send to the same topic, same consumer, still running, after it worked for the
test producer.



  



 Funny thing is, I see this error in forums/emails lately, but nobody
nails it down as to why it is occurring...



  



 ==> nifi-app.log <==  
2016-06-11 12:04:24,172 WARN [kafka-producer-network-thread |
SecondPutKafka2k] org.apache.kafka.common.network.Selector Error in I/O with
localhost/[127.0.0.1](http://127.0.0.1)  
java.io.EOFException: null  
at org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java
:62) ~[kafka-clients-0.8.2.2.jar:na]  
at org.apache.kafka.common.network.Selector.poll(Selector.java:248) ~[kafka-
clients-0.8.2.2.jar:na]  
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:192) [kafka-
clients-0.8.2.2.jar:na]  
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:191)
[kafka-clients-0.8.2.2.jar:na]  
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:122)
[kafka-clients-0.8.2.2.jar:na]  
at java.lang.Thread.run(T

Re: PutKafka won't Put! "Failed while waiting for acks from Kafka"

[Juan Sequeiros]

Hi Pat,

Looks to me that on your PutKafka you are pointing it to your zookeeper
instance port 2181 you should point it to your kafka broker. Port 9092 ( I
think )
Not home to verify ...

On Sat, Jun 11, 2016 at 3:31 PM, Pat Trainor  wrote:

> *OK... Prep'ing kafka:*
>
> pat@wopr:/opt/kafka$ bin/kafka-server-stop.sh
> pat@wopr:/opt/kafka$ bin/zookeeper-server-stop.sh
> pat@wopr:/opt/kafka/logs$ rm log-cleaner.log controller.log server.log
> kafkaServer-gc.log zookeeper-gc.log
>
> *Prep'ing nifi:*
>
> pat@wopr:/opt/nifi$ bin/nifi.sh stop
> pat@wopr:/opt/nifi$ rm logs/nifi-bootstrap.log logs/nifi-app.log
> logs/nifi-user.log
>
> *Starting kafka:*
>
> pat@wopr:/opt/kafka$ bin/zookeeper-server-start.sh
> config/zookeeper.properties &
> pat@wopr:/opt/kafka$ bin/kafka-server-start.sh config/server.properties &
>
> bin/kafka-topics.sh --create --zookeeper localhost:2181
> --replication-factor 1 --partitions 1 --topic test
> *bin/kafka-topics.sh --create --zookeeper localhost:2181
> --replication-factor 1 --partitions 1 --topic fast-messages*
> bin/kafka-topics.sh --create --zookeeper localhost:2181
> --replication-factor 1 --partitions 1 --topic inNLP
> bin/kafka-topics.sh --create --zookeeper localhost:2181
> --replication-factor 1 --partitions 1 --topic outNLP
>
> The above topics were preserved.
>
> kafka.logs.tar.gz attached
>
> *Starting Nifi:*
>
> pat@wopr:/opt/nifi$ bin/nifi.sh start
>
> nifi.logs.tar.gz attached.
>
> Now to log onto Nifi GUI and start the 2 processes (GenerateFlowfile &
> PutKafka ...to fast-messages topic):
>
> I really hope this helps... I can't believe I'm going to spend my entire
> weekend onthis... :-(
>
> Thanks!
>
> pat 
> ( ͡° ͜ʖ ͡°)
>
>
> "A wise man can learn more from a foolish *question *than a fool can
> learn from a wise *answer*". ~ Bruce Lee.
>
> On Jun 11 2016, at 1:38 pm, Joe Witt  wrote:
>
>> Pat
>>
>> Could you please make sure we are seeing the full stacktrace from the app
>> log?  If we could get a full log that would he even better.
>>
>> Thanks
>> Joe
>> On Jun 11, 2016 1:17 PM, "Pat Trainor"  wrote:
>>
>> Andrew,
>>
>> Thanks for writing.
>>
>> *Nifi:*
>> nifi-0.6.1
>> *Kafka:*
>> kafka_2.11-0.10.0.0
>>
>> startup with no changes to configs:
>>
>> #
>> Kafka:
>> bin/zookeeper-server-start.sh config/zookeeper.properties &
>> bin/kafka-server-start.sh config/server.properties &
>> #
>> # create a topic
>> #
>> bin/kafka-topics.sh --create --zookeeper localhost:2181
>> --replication-factor 1 --partitions 1 --topic test
>> [...]
>>
>> *Nifi's zookeeper.properties:*
>>
>>
>> clientPort=2181
>> initLimit=10
>> autopurge.purgeInterval=24
>> syncLimit=5
>> tickTime=2000
>> dataDir=./state/zookeeper
>> autopurge.snapRetainCount=30
>>
>> #
>> # server.1=nifi-node1-hostname:2888:3888
>> # server.2=nifi-node2-hostname:2888:3888
>> # server.3=nifi-node3-hostname:2888:3888
>> #
>>
>> *server.1=*
>>
>> ^^
>> Even though I have:
>>
>>
>> *nifi.state.management.embedded.zookeeper.start=false*
>>
>> I filled in the above setting... changing it to:
>>
>>
>> *server.1=localhost:2888:3888*
>>
>> ...which made no difference, of course...
>>
>> Any thoughts?
>>
>> Thanks!
>>
>> pat 
>> ( ͡° ͜ʖ ͡°)
>>
>> On Jun 11 2016, at 12:17 pm, Andrew Grande  wrote:
>>
>> Pat, which NiFi version is that? Which Kafka broker version on the other
>> end?
>>
>> Andrew
>>
>> On Sat, Jun 11, 2016 at 12:15 PM Pat Trainor 
>> wrote:
>>
>> Users,
>>
>> Has anyone come across PutKafka simply not wanting to send to Kafka?
>>
>> In this troubleshooting, I have replicated symptoms in the simplest way I
>> can-with a GenerateFlowFile and a PutKafka:
>>
>>
>> With a known good Kafka running (other /producers using the same Topic)...
>>
>> ...yields:
>> Auto refresh started
>> 11:17:19 EDT WARNING
>> 2b76634c-6354-4c3c-b977-86f73d64efbcPutKafka[id=2b76634c-6354-4c3c-b977-86f73d64efbc]
>> Processor Administratively Yielded for 1 sec due to processing failure
>> 11:18:00 EDT ERROR
>> 2b76634c-6354-4c3c-b977-86f73d64efbcPutKafka[id=2b76634c-6354-4c3c-b977-86f73d64efbc]
>> Failed while waiting for acks from Kafka
>> 11:18:00 EDT ERROR
>> 2b76634c-6354-4c3c-b977-86f73d64efbcPutKafka[id=2b76634c-6354-4c3c-b977-86f73d64efbc]
>> PutKafka[id=2b76634c-6354-4c3c-b977-86f73d64efbc] failed to process due to
>> java.lang.NullPointerException; rolling back session:
>> java.lang.NullPointerException
>> 11:18:00 EDT ERROR 2b76634c-6354-4c3c-b977-86f73d64efbc
>>
>> PutKafka[id=2b76634c-6354-4c3c-b977-86f73d64efbc]
>> PutKafka[id=2b76634c-6354-4c3c-b977-86f73d64efbc] failed to process session
>> due to java.lang.NullPointerException: java.lang.NullPointerException
>>
>> Everything is super-stock, out of the box, and not customized. In fact, I
>> tested Kafka with the Apache test consumer/producer pair, and am trying to
>> send to the same topic, same consumer, still running, after it worked for
>> the test producer.
>>
>> Funny thing is, I see this error in forums/emails lat

Re: PutKafka won't Put! "Failed while waiting for acks from Kafka"

[Oleg Zhurakousky]

Also, the NPE was a bug that was fixed a while back and will be available on 
O.7.
Any chance you can build from 0.x branch?

Cheers
Oleg

Sent from my iPhone

On Jun 11, 2016, at 16:58, Juan Sequeiros 
mailto:helloj...@gmail.com>> wrote:

Hi Pat,

Looks to me that on your PutKafka you are pointing it to your zookeeper 
instance port 2181 you should point it to your kafka broker. Port 9092 ( I 
think )
Not home to verify ...

On Sat, Jun 11, 2016 at 3:31 PM, Pat Trainor 
mailto:pat.trai...@gmail.com>> wrote:
OK... Prep'ing kafka:

pat@wopr:/opt/kafka$ bin/kafka-server-stop.sh
pat@wopr:/opt/kafka$ bin/zookeeper-server-stop.sh
pat@wopr:/opt/kafka/logs$ rm log-cleaner.log controller.log server.log 
kafkaServer-gc.log zookeeper-gc.log

Prep'ing nifi:

pat@wopr:/opt/nifi$ bin/nifi.sh stop
pat@wopr:/opt/nifi$ rm logs/nifi-bootstrap.log logs/nifi-app.log 
logs/nifi-user.log

Starting kafka:

pat@wopr:/opt/kafka$ bin/zookeeper-server-start.sh config/zookeeper.properties &
pat@wopr:/opt/kafka$ bin/kafka-server-start.sh config/server.properties &

bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 
--partitions 1 --topic test
bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 
--partitions 1 --topic fast-messages
bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 
--partitions 1 --topic inNLP
bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 
--partitions 1 --topic outNLP

The above topics were preserved.

kafka.logs.tar.gz attached

Starting Nifi:

pat@wopr:/opt/nifi$ bin/nifi.sh start

nifi.logs.tar.gz attached.

Now to log onto Nifi GUI and start the 2 processes (GenerateFlowfile & PutKafka 
...to fast-messages topic):

I really hope this helps... I can't believe I'm going to spend my entire 
weekend onthis... :-(

Thanks!

pat
( ?? ?? ??)


"A wise man can learn more from a foolish question than a fool can learn from a 
wise answer". ~ Bruce Lee.

On Jun 11 2016, at 1:38 pm, Joe Witt 
mailto:joe.w...@gmail.com>> wrote:

Pat

Could you please make sure we are seeing the full stacktrace from the app log?  
If we could get a full log that would he even better.

Thanks
Joe

On Jun 11, 2016 1:17 PM, "Pat Trainor" 
mailto:pat.trai...@gmail.com>> wrote:
Andrew,

Thanks for writing.

Nifi:
nifi-0.6.1
Kafka:
kafka_2.11-0.10.0.0

startup with no changes to configs:

#
Kafka:
bin/zookeeper-server-start.sh config/zookeeper.properties &
bin/kafka-server-start.sh config/server.properties &
#
# create a topic
#
bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 
--partitions 1 --topic test
[...]

Nifi's zookeeper.properties:


clientPort=2181
initLimit=10
autopurge.purgeInterval=24
syncLimit=5
tickTime=2000
dataDir=./state/zookeeper
autopurge.snapRetainCount=30

#
# server.1=nifi-node1-hostname:2888:3888
# server.2=nifi-node2-hostname:2888:3888
# server.3=nifi-node3-hostname:2888:3888
#
server.1=

^^
Even though I have:

nifi.state.management.embedded.zookeeper.start=false

I filled in the above setting... changing it to:

server.1=localhost:2888:3888

...which made no difference, of course...

Any thoughts?

Thanks!

pat
( ?? ?? ??)

On Jun 11 2016, at 12:17 pm, Andrew Grande 
mailto:apere...@gmail.com>> wrote:
Pat, which NiFi version is that? Which Kafka broker version on the other end?

Andrew

On Sat, Jun 11, 2016 at 12:15 PM Pat Trainor 
mailto:pat.trai...@gmail.com>> wrote:
Users,

Has anyone come across PutKafka simply not wanting to send to Kafka?

In this troubleshooting, I have replicated symptoms in the simplest way I 
can-with a GenerateFlowFile and a PutKafka:

[https://community.hortonworks.com/storage/temp/4972-screenshot-from-2016-06-11-11-34-25.png]

With a known good Kafka running (other /producers using the same Topic)...

...yields:

Auto refresh started
11:17:19 EDT WARNING 
2b76634c-6354-4c3c-b977-86f73d64efbcPutKafka[id=2b76634c-6354-4c3c-b977-86f73d64efbc]
 Processor Administratively Yielded for 1 sec due to processing failure
11:18:00 EDT ERROR 
2b76634c-6354-4c3c-b977-86f73d64efbcPutKafka[id=2b76634c-6354-4c3c-b977-86f73d64efbc]
 Failed while waiting for acks from Kafka
11:18:00 EDT ERROR 
2b76634c-6354-4c3c-b977-86f73d64efbcPutKafka[id=2b76634c-6354-4c3c-b977-86f73d64efbc]
 PutKafka[id=2b76634c-6354-4c3c-b977-86f73d64efbc] failed to process due to 
java.lang.NullPointerException; rolling back session: 
java.lang.NullPointerException
11:18:00 EDT ERROR 2b76634c-6354-4c3c-b977-86f73d64efbc

PutKafka[id=2b76634c-6354-4c3c-b977-86f73d64efbc] 
PutKafka[id=2b76634c-6354-4c3c-b977-86f73d64efbc] failed to process session due 
to java.lang.NullPointerException: java.lang.NullPointerException

Everything is super-stock, out of the box, and not customized. In fact, I 
tested Kafka with the Apache test consumer/producer pair, and am trying to send 
to the same topic, same consumer, still running, after it worked fo

Re: PutKafka won't Put! "Failed while waiting for acks from Kafka"

[Oleg Zhurakousky]

Also, just looked ta the logs and what I see is that you have some type of 
connection problem to Kafka.

2016-06-11 15:23:06,193 WARN [kafka-producer-network-thread | SecondPutKafka2k] 
org.apache.kafka.common.network.Selector Error in I/O with localhost/127.0.0.1
java.io.EOFException: null
at 
org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:62) 
~[kafka-clients-0.8.2.2.jar:na]
at org.apache.kafka.common.network.Selector.poll(Selector.java:248) 
~[kafka-clients-0.8.2.2.jar:na]
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:192) 
[kafka-clients-0.8.2.2.jar:na]
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:191) 
[kafka-clients-0.8.2.2.jar:na]
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:122) 
[kafka-clients-0.8.2.2.jar:na]

The above is coming from Kafka (not NIFi). Are you sure that you have the 
correct IP:PORT in your producer config? One quick diagnostic you can do is to 
try to send message to Kafka from the command line on the same machine as NiFi. 
Basically this:

$> bin/kafka-console-producer.sh --broker-list : --topic 

Let me know
Cheers
Oleg

On Jun 11, 2016, at 5:03 PM, Oleg Zhurakousky 
mailto:ozhurakou...@hortonworks.com>> wrote:

Also, the NPE was a bug that was fixed a while back and will be available on 
O.7.
Any chance you can build from 0.x branch?

Cheers
Oleg

Sent from my iPhone

On Jun 11, 2016, at 16:58, Juan Sequeiros 
mailto:helloj...@gmail.com>> wrote:

Hi Pat,

Looks to me that on your PutKafka you are pointing it to your zookeeper 
instance port 2181 you should point it to your kafka broker. Port 9092 ( I 
think )
Not home to verify ...

On Sat, Jun 11, 2016 at 3:31 PM, Pat Trainor 
mailto:pat.trai...@gmail.com>> wrote:
OK... Prep'ing kafka:

pat@wopr:/opt/kafka$ bin/kafka-server-stop.sh
pat@wopr:/opt/kafka$ bin/zookeeper-server-stop.sh
pat@wopr:/opt/kafka/logs$ rm log-cleaner.log controller.log server.log 
kafkaServer-gc.log zookeeper-gc.log

Prep'ing nifi:

pat@wopr:/opt/nifi$ bin/nifi.sh stop
pat@wopr:/opt/nifi$ rm logs/nifi-bootstrap.log logs/nifi-app.log 
logs/nifi-user.log

Starting kafka:

pat@wopr:/opt/kafka$ bin/zookeeper-server-start.sh config/zookeeper.properties &
pat@wopr:/opt/kafka$ bin/kafka-server-start.sh config/server.properties &

bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 
--partitions 1 --topic test
bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 
--partitions 1 --topic fast-messages
bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 
--partitions 1 --topic inNLP
bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 
--partitions 1 --topic outNLP

The above topics were preserved.

kafka.logs.tar.gz attached

Starting Nifi:

pat@wopr:/opt/nifi$ bin/nifi.sh start

nifi.logs.tar.gz attached.

Now to log onto Nifi GUI and start the 2 processes (GenerateFlowfile & PutKafka 
...to fast-messages topic):

I really hope this helps... I can't believe I'm going to spend my entire 
weekend onthis... :-(

Thanks!

pat
( ͡° ͜ʖ ͡°)


"A wise man can learn more from a foolish question than a fool can learn from a 
wise answer". ~ Bruce Lee.

On Jun 11 2016, at 1:38 pm, Joe Witt 
mailto:joe.w...@gmail.com>> wrote:

Pat

Could you please make sure we are seeing the full stacktrace from the app log?  
If we could get a full log that would he even better.

Thanks
Joe

On Jun 11, 2016 1:17 PM, "Pat Trainor" 
mailto:pat.trai...@gmail.com>> wrote:
Andrew,

Thanks for writing.

Nifi:
nifi-0.6.1
Kafka:
kafka_2.11-0.10.0.0

startup with no changes to configs:

#
Kafka:
bin/zookeeper-server-start.sh config/zookeeper.properties &
bin/kafka-server-start.sh config/server.properties &
#
# create a topic
#
bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 
--partitions 1 --topic test
[...]

Nifi's zookeeper.properties:


clientPort=2181
initLimit=10
autopurge.purgeInterval=24
syncLimit=5
tickTime=2000
dataDir=./state/zookeeper
autopurge.snapRetainCount=30

#
# server.1=nifi-node1-hostname:2888:3888
# server.2=nifi-node2-hostname:2888:3888
# server.3=nifi-node3-hostname:2888:3888
#
server.1=

^^
Even though I have:

nifi.state.management.embedded.zookeeper.start=false

I filled in the above setting... changing it to:

server.1=localhost:2888:3888

...which made no difference, of course...

Any thoughts?

Thanks!

pat
( ͡° ͜ʖ ͡°)

On Jun 11 2016, at 12:17 pm, Andrew Grande 
mailto:apere...@gmail.com>> wrote:
Pat, which NiFi version is that? Which Kafka broker version on the other end?

Andrew

On Sat, Jun 11, 2016 at 12:15 PM Pat Trainor 
mailto:pat.trai...@gmail.com>> wrote:
Users,

Has anyone come across PutKafka simply not wanting to send to Kafka?

In this troubleshooting, I have replicated symptoms in the simplest way I 
can-with a GenerateFlowFile and a PutKafka:


[https://community.h

Re: PutKafka won't Put! "Failed while waiting for acks from Kafka"

[Pat Trainor]

Yes. As soon as I fire up zookeeper & PutKafka, I get zillions of these
streaming by in the log:

  

[2016-06-12 10:24:52,267] INFO Closed socket connection for client
/127.0.0.1:59958 (no session established for client)
(org.apache.zookeeper.server.NIOServerCnxn)  
[2016-06-12 10:24:52,369] INFO Accepted socket connection from
/127.0.0.1:59960 (org.apache.zookeeper.server.NIOServerCnxnFactory)  
[2016-06-12 10:24:52,369] WARN Exception causing close of session 0x0 due to
java.io.IOException: Unreasonable length = 845873152
(org.apache.zookeeper.server.NIOServerCnxn)  

  

I will do as Juan Sequeiros  (Hello Juan!) ... ugh... I'm hanging my head the
way we programmers do when we've gone a day or more trying to figure out what
is stopping us from continuing, and then find out it is not the cosmos that is
against us, but out own naivete'/stupidite'...

  

When I put my hostname in (saw that it worked better than localhost in
archives), and changed my PutKafka port from 2181:

  

localhost:2181 ---> wopr:9092

  

**Everything chirped up and gave me a raspberry...** My only defense is that I 
see port 2181 in all the examples... And that isn't a very good excuse...

  

Thank you to everyone who pitched in and threw ideas out.  

  

Thanks!  
  

[pat](http://about.me/PatTrainor)  

( ͡° ͜ʖ ͡°)  
  

  

"A wise man can learn more from a foolish _question _than a fool can learn
from a wise _answer_". ~ Bruce Lee.  

  

On Jun 11 2016, at 5:17 pm, Oleg Zhurakousky
 wrote:  

> Also, just looked ta the logs and what I see is that you have some type of
connection problem to Kafka.

>

>  

>

> 2016-06-11 15:23:06,193 WARN [kafka-producer-network-thread |
SecondPutKafka2k] org.apache.kafka.common.network.Selector Error in I/O with
localhost/127.0.0.1

>

> java.io.EOFException: null

>

> at org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.ja
va:62) ~[kafka-clients-0.8.2.2.jar:na]

>

> at org.apache.kafka.common.network.Selector.poll(Selector.java:248) ~[kafka-
clients-0.8.2.2.jar:na]

>

> at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:192)
[kafka-clients-0.8.2.2.jar:na]

>

> at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:191)
[kafka-clients-0.8.2.2.jar:na]

>

> at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:122)
[kafka-clients-0.8.2.2.jar:na]

>

>  

>

> The above is coming from Kafka (not NIFi). Are you sure that you have the
correct IP:PORT in your producer config? One quick diagnostic you can do is to
try to send message to Kafka from the command line on the same machine as
NiFi. Basically this:

>

>  

>

> $> bin/kafka-console-producer.sh --broker-list :
\--topic 

>

>  

>

> Let me know

>

> Cheers

>

> Oleg

>

>  

>

>> On Jun 11, 2016, at 5:03 PM, Oleg Zhurakousky
<[ozhurakou...@hortonworks.com](mailto:ozhurakou...@hortonworks.com)>
wrote:

>>

>>  

>>

>> Also, the NPE was a bug that was fixed a while back and will be available
on O.7.

>>

>> Any chance you can build from 0.x branch?

>>

>>  

>>

>> Cheers

>>

>> Oleg  
  
Sent from my iPhone

>>

>>  
On Jun 11, 2016, at 16:58, Juan Sequeiros
<[helloj...@gmail.com](mailto:helloj...@gmail.com)> wrote:  
  

>>

>>> Hi Pat,

>>>

>>>  

>>>

>>> Looks to me that on your PutKafka you are pointing it to your zookeeper
instance port 2181 you should point it to your kafka broker. Port 9092 ( I
think )

>>>

>>> Not home to verify ...

>>>

>>>  

>>>

>>> On Sat, Jun 11, 2016 at 3:31 PM, Pat Trainor
<[pat.trai...@gmail.com](mailto:pat.trai...@gmail.com)> wrote:  

>>>

 _OK... Prep'ing kafka:_



  



 pat@wopr:/opt/kafka$ bin/kafka-server-stop.sh  



 pat@wopr:/opt/kafka$ bin/zookeeper-server-stop.sh  



 pat@wopr:/opt/kafka/logs$ rm log-cleaner.log controller.log server.log
kafkaServer-gc.log zookeeper-gc.log  



  



 _Prep'ing nifi:_



  



 pat@wopr:/opt/nifi$ bin/nifi.sh stop  



 pat@wopr:/opt/nifi$ rm logs/nifi-bootstrap.log logs/nifi-app.log logs
/nifi-user.log  



  



 _Starting kafka:_



  



 pat@wopr:/opt/kafka$ bin/zookeeper-server-start.sh
config/zookeeper.properties &  



 pat@wopr:/opt/kafka$ bin/kafka-server-start.sh config/server.properties
&  



  



 bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-
factor 1 --partitions 1 --topic test  
**bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 
1 --partitions 1 --topic fast-messages**  
bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1
--partitions 1 --topic inNLP  
bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1
--partitions 1 --topic outNLP  



  



 The above topics were preserved.



  



 kafka.logs.tar.gz attached  



  



 _