[
https://issues.apache.org/jira/browse/NIFI-4682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
clearlove updated NIFI-4682:
----------------------------
Attachment: KafkaTemple.xml
> when i start consumerKafka processor more than eleven,all of the processors
> get into an indefinite stuck state
> --------------------------------------------------------------------------------------------------------------
>
> Key: NIFI-4682
> URL: https://issues.apache.org/jira/browse/NIFI-4682
> Project: Apache NiFi
> Issue Type: Bug
> Affects Versions: 1.3.0
> Environment: Linux
> Reporter: clearlove
> Priority: Blocker
> Attachments: 2.png, KafkaTemple.xml, nifi-bootstrap.log
>
>
> hello,
> When i start one consumeKafka_0_10 processor ,all of the processors are all
> right.
> but when i start consumeKafka_0_10 processor more than eleven,all of the
> processors get into an indefinite stuck state.
> !1.png|thumbnail!
> like in the picture ,look like everything is all right, but all
> processor(even processor is not consumeKafka_0_10 ) are stop work.All of the
> flowfile don't move.
> And when i stop them , some of consumeKafka_0_10 processor are warning
> {{Was interrupted while trying to communicate with Kafka with lease {}. Will
> roll back session and discard any partially received data.}}
> in the nifi-app.log
> {noformat}
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.standard.EvaluateJsonPath
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.kafka.pubsub.ConsumeKafka_0_10
> 2017-12-05 10:12:02,128 INFO [StandardProcessScheduler Thread-4]
> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
> ReplaceText[id=099a83f6-7ddc-3df8-fe07-5b901799ade3] to run
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.standard.SplitJson
> 2017-12-05 10:12:02,128 INFO [StandardProcessScheduler Thread-2]
> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
> EvaluateJsonPath[id=fdd4850d-a619-3f17-6287-198cac3bb52a] to run
> 2017-12-05 10:12:02,128 INFO [StandardProcessScheduler Thread-4]
> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
> ConsumeKafka_0_10[id=bed469b9-b52e-3178-f629-07f98ee6c07e] to run
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.standard.EvaluateJsonPath
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.standard.EvaluateJsonPath
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.standard.PutSQL
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.standard.EvaluateJsonPath
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.kafka.pubsub.ConsumeKafka_0_10
> 2017-12-05 10:12:02,128 INFO [StandardProcessScheduler Thread-2]
> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
> SplitJson[id=f3f85270-6bb8-3f65-4347-6700426d6a2e] to run
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.standard.ReplaceText
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.standard.EvaluateJsonPath
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.standard.EvaluateJsonPath
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.standard.PutSQL
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.kafka.pubsub.ConsumeKafka_0_10
> 2017-12-05 10:12:02,128 INFO [StandardProcessScheduler Thread-2]
> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
> EvaluateJsonPath[id=6aacc9a9-c7e8-32b0-ed3c-3f89e72c30cf] to run
> 2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.standard.ReplaceText
> 2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.kafka.pubsub.ConsumeKafka_0_10
> 2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.kafka.pubsub.ConsumeKafka_0_10
> 2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.standard.EvaluateJsonPath
> 2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.standard.ReplaceText
> 2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.standard.EvaluateJsonPath
> 2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.standard.ReplaceText
> 2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.standard.ReplaceText
> 2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.standard.PutSQL
> 2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.standard.PutSQL
> 2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.kafka.pubsub.ConsumeKafka_0_10
> 2017-12-05 10:12:02,130 INFO [StandardProcessScheduler Thread-2]
> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
> EvaluateJsonPath[id=02aaae90-5ada-343e-6c15-3cfff7cfded0] to run
> 2017-12-05 10:12:02,130 INFO [StandardProcessScheduler Thread-2]
> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
> PutSQL[id=278dc7a5-6d2e-336a-45c3-56cb7039dc7c] to run
> 2017-12-05 10:12:02,131 INFO [StandardProcessScheduler Thread-2]
> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
> EvaluateJsonPath[id=60fd28d8-b5f0-3a5b-e31a-e3ba7b73d7cb] to run
> 2017-12-05 10:12:02,131 INFO [StandardProcessScheduler Thread-2]
> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
> ConsumeKafka_0_10[id=efa69212-d0c9-3c83-0985-b22115b5e861] to run
> 2017-12-05 10:12:02,724 INFO [Flow Service Tasks Thread-1]
> o.a.nifi.controller.StandardFlowService Saved flow controller
> org.apache.nifi.controller.FlowController@a987d5e // Another save pending =
> false
> 2017-12-05 10:12:07,130 INFO [StandardProcessScheduler Thread-6]
> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
> ReplaceText[id=2fedc410-9641-3bda-cc40-59cf739a5400] to run
> 2017-12-05 10:12:07,130 WARN [Timer-Driven Process Thread-8]
> o.a.n.p.kafka.pubsub.ConsumeKafka_0_10
> ConsumeKafka_0_10[id=a2b469df-a2ef-355f-cc68-7b5dea6abb8b] Was interrupted
> while trying to communicate with Kafka with lease
> org.apache.nifi.processors.kafka.pubsub.ConsumerPool$SimpleConsumerLease@2b5a2eed.
> Will roll back session and discard any partially received data.
> 2017-12-05 10:12:07,130 INFO [StandardProcessScheduler Thread-6]
> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
> EvaluateJsonPath[id=d925dcdc-edef-3184-2185-47a0ebf3ea3b] to run
> 2017-12-05 10:12:07,131 INFO [StandardProcessScheduler Thread-6]
> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
> EvaluateJsonPath[id=8b6fef2d-3db8-3558-89a3-abce7b0e8797] to run
> 2017-12-05 10:12:07,131 INFO [StandardProcessScheduler Thread-7]
> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
> PutSQL[id=e1af6798-96ba-31ec-c135-ff2f10d1306b] to run
> 2017-12-05 10:12:07,131 WARN [Timer-Driven Process Thread-7]
> o.a.n.p.kafka.pubsub.ConsumeKafka_0_10
> ConsumeKafka_0_10[id=d4b01bbd-62a3-3584-8391-371d52b83bce] Was interrupted
> while trying to communicate with Kafka with lease
> org.apache.nifi.processors.kafka.pubsub.ConsumerPool$SimpleConsumerLease@7060d90d.
> Will roll back session and discard any partially received data.
> 2017-12-05 10:12:07,132 INFO [StandardProcessScheduler Thread-8]
> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
> ConsumeKafka_0_10[id=b88105fe-cb1e-3c4e-f91d-22f5542cecd7] to run
> 2017-12-05 10:12:07,132 WARN [Timer-Driven Process Thread-2]
> o.a.n.p.kafka.pubsub.ConsumeKafka_0_10
> ConsumeKafka_0_10[id=4c067c00-fbc2-3571-5b7a-37b4957afb87] Was interrupted
> while trying to communicate with Kafka with lease
> org.apache.nifi.processors.kafka.pubsub.ConsumerPool$SimpleConsumerLease@158837ab.
> Will roll back session and discard any partially received data.
> 2017-12-05 10:12:07,133 INFO [StandardProcessScheduler Thread-7]
> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
> ReplaceText[id=182347e6-14ef-38c2-6873-5602f0405296] to run
> 2017-12-05 10:12:07,133 WARN [Timer-Driven Process Thread-3]
> o.a.n.p.kafka.pubsub.ConsumeKafka_0_10
> ConsumeKafka_0_10[id=4035760a-8b9e-3e34-51e9-e292c58e6ed7] Was interrupted
> while trying to communicate with Kafka with lease
> org.apache.nifi.processors.kafka.pubsub.ConsumerPool$SimpleConsumerLease@4967b312.
> Will roll back session and discard any partially received data.
> 2017-12-05 10:12:07,133 INFO [StandardProcessScheduler Thread-6]
> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
> ConsumeKafka_0_10[id=f7285790-6159-3726-662e-5717235a31dc] to run
> 2017-12-05 10:12:07,133 INFO [StandardProcessScheduler Thread-1]
> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
> ConsumeKafka_0_10[id=fae14360-943d-3a81-f3b1-675f5505e265] to run
> 2017-12-05 10:12:07,134 INFO [StandardProcessScheduler Thread-5]
> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
> ReplaceText[id=86b794b6-5875-3eac-9fdb-447743459b7d] to run
> 2017-12-05 10:12:07,133 WARN [Timer-Driven Process Thread-9]
> o.a.n.p.kafka.pubsub.ConsumeKafka_0_10
> ConsumeKafka_0_10[id=716f4415-e4ec-36da-8441-375bdddc690b] Was interrupted
> while trying to communicate with Kafka with lease
> org.apache.nifi.processors.kafka.pubsub.ConsumerPool$SimpleConsumerLease@35eec0c6.
> Will roll back session and discard any partially received data.
> 2017-12-05 10:12:07,134 WARN [Timer-Driven Process Thread-6]
> o.a.n.p.kafka.pubsub.ConsumeKafka_0_10
> ConsumeKafka_0_10[id=d2e3d4a1-3257-38a0-9e7b-2e0c5c7f7395] Was interrupted
> while trying to communicate with Kafka with lease
> org.apache.nifi.processors.kafka.pubsub.ConsumerPool$SimpleConsumerLease@261dc487.
> Will roll back session and discard any partially received data.
> 2017-12-05 10:12:07,134 INFO [StandardProcessScheduler Thread-7]
> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
> EvaluateJsonPath[id=2fb57630-0d77-3947-954c-dc3b1fd6174e] to run
> 2017-12-05 10:12:07,134 INFO [StandardProcessScheduler Thread-5]
> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
> ReplaceText[id=4cdd0444-8047-3447-9f6c-f8f536b8228a] to run
> 2017-12-05 10:12:07,134 WARN [Timer-Driven Process Thread-1]
> o.a.n.p.kafka.pubsub.ConsumeKafka_0_10
> ConsumeKafka_0_10[id=bed469b9-b52e-3178-f629-07f98ee6c07e] Was interrupted
> while trying to communicate with Kafka with lease
> org.apache.nifi.processors.kafka.pubsub.ConsumerPool$SimpleConsumerLease@4f649784.
> Will roll back session and discard any partially received data.
> 2017-12-05 10:12:07,134 INFO [StandardProcessScheduler Thread-4]
> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
> ReplaceText[id=d4372710-f550-3d31-10ae-8ae7e9300b44] to run
> 2017-12-05 10:12:07,134 INFO [StandardProcessScheduler Thread-7]
> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
> PutSQL[id=df40050e-ebf1-368f-df61-43b5eb7bbdb6] to run
> 2017-12-05 10:12:07,134 INFO [StandardProcessScheduler Thread-5]
> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
> PutSQL[id=8c979a96-03de-3beb-ec3a-1c91027162c9] to run
> {noformat}
> no Kerberos involved
> All configuration is default besides 'Kafka Brokers','Topic Name(s)','Group
> ID'
> The configuration file has been uploaded in the attachment 2.png
> the nifi-bootstrap.log is result of runing 'bin/nifi.sh dump'
> one consumeKafka processor -----------one topic-----------one partition
> --------------one replication-factor
> Every consumeKafka processor has a different topic.
> And i had tried out nifi 1.4.0,but it can not resolved the issues.
> thanks
> Sorry,my english is so bad.Please forgive me.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
