I changed python kafka version from 1.0.1 to 0.9.5 works fine. Thanks
On Thu, Jun 23, 2016 at 6:04 PM, Jungtaek Lim <kabh...@gmail.com> wrote: > Could you share implementation of spout? > In multi-lang user level functions shouldn't block, so heartbeat timeout > will occur if your spout requests to Kafka and wait for response infinitely. > > - Jungtaek Lim (HeartSaVioR) > > 2016년 6월 23일 (목) 오후 7:25, cogumelosmaravilha <cogumelosmaravi...@sapo.pt>님이 > 작성: > >> Yes, but if you submit to Storm, it's ok, with or without an empty Kafka. >> >> >> On 23-06-2016 10:40, ram kumar wrote: >> >> Yes, The same topology, when I run in single node cluster, it works. And >> when there is no data to consume from kafka, it shows heartbeat timeout >> error. >> >> Here, I am testing in multi node cluster and kafka server is in different >> node >> >> On Thu, Jun 23, 2016 at 1:48 PM, cogumelosmaravilha < >> <cogumelosmaravi...@sapo.pt>cogumelosmaravi...@sapo.pt> wrote: >> >>> Hi, >>> >>> It happens to me to, but only when my kafka is empy. >>> I'm using Petrel because the generated jar file is really small like >>> 300k Petrel vs 16M Streamparse. >>> >>> >>> On 23-06-2016 08:05, ram kumar wrote: >>> >>> Hi, >>> >>> >>> *Version: * >>> Storm : 0.10.0 >>> Streamparse : 2.1.4 >>> >>> >>> I am running a storm topology with a python streamparse "sparse run". >>> >>> This topology stops executing in the middle and throw an exception >>> >>> >>> 158031 [pool-37-thread-1] ERROR b.s.s.ShellSpout - Halting process: >>>> ShellSpout died. >>>> java.lang.RuntimeException: *subprocess heartbeat timeout* >>>> at >>>> backtype.storm.spout.ShellSpout$SpoutHeartbeatTimerTask.run(ShellSpout.java:261) >>>> [storm-core-0.10.0.jar:0.10.0] >>>> at >>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) >>>> [?:1.8.0_40] >>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) >>>> [?:1.8.0_40] >>>> at >>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) >>>> [?:1.8.0_40] >>>> at >>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) >>>> [?:1.8.0_40] >>>> at >>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) >>>> [?:1.8.0_40] >>>> at >>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) >>>> [?:1.8.0_40] >>>> at java.lang.Thread.run(Thread.java:745) [?:1.8.0_40] >>>> 158036 [pool-37-thread-1] ERROR b.s.d.executor - >>>> java.lang.RuntimeException: subprocess heartbeat timeout >>>> at >>>> backtype.storm.spout.ShellSpout$SpoutHeartbeatTimerTask.run(ShellSpout.java:261) >>>> [storm-core-0.10.0.jar:0.10.0] >>>> at >>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) >>>> [?:1.8.0_40] >>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) >>>> [?:1.8.0_40] >>>> at >>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) >>>> [?:1.8.0_40] >>>> at >>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) >>>> [?:1.8.0_40] >>>> at >>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) >>>> [?:1.8.0_40] >>>> at >>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) >>>> [?:1.8.0_40] >>>> at java.lang.Thread.run(Thread.java:745) [?:1.8.0_40] >>>> >>> >>> >>> This occurs randomly >>> I can't able to trace back to the problem >>> >>> Maybe if spout takes too long to process, then streamparse can't >>> acknowledge the heartbeat in time >>> >>> changed "supervisor.worker.timeout.secs" from 30 to 600 >>> >>> Still the topology breaks. Is there any other options here? >>> >>> >>> Thanks, >>> Ram. >>> >>> >>> >> >>
