Re: Heartbeat timeout with Python spout

[cogumelosmaravilha]

Hi,

It happens to me to, but only when my kafka is empy.
I'm using Petrel because the generated jar file is really small like 
300k Petrel vs 16M Streamparse.

On 23-06-2016 08:05, ram kumar wrote:
Hi,
*
*
*Version:
*
    Storm : 0.10.0
    Streamparse : 2.1.4


I am running a storm topology with a python streamparse "sparse run".
This topology stops executing in the middle and throw an exception

    158031 [pool-37-thread-1] ERROR b.s.s.ShellSpout - Halting
    process: ShellSpout died.
    java.lang.RuntimeException: *subprocess heartbeat timeout*
        at
    
backtype.storm.spout.ShellSpout$SpoutHeartbeatTimerTask.run(ShellSpout.java:261)
    [storm-core-0.10.0.jar:0.10.0]
        at
    java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    [?:1.8.0_40]
        at
    java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
    [?:1.8.0_40]
        at
    
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
    [?:1.8.0_40]
        at
    
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
    [?:1.8.0_40]
        at
    
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    [?:1.8.0_40]
        at
    
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    [?:1.8.0_40]
        at java.lang.Thread.run(Thread.java:745) [?:1.8.0_40]
    158036 [pool-37-thread-1] ERROR b.s.d.executor -
    java.lang.RuntimeException: subprocess heartbeat timeout
        at
    
backtype.storm.spout.ShellSpout$SpoutHeartbeatTimerTask.run(ShellSpout.java:261)
    [storm-core-0.10.0.jar:0.10.0]
        at
    java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    [?:1.8.0_40]
        at
    java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
    [?:1.8.0_40]
        at
    
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
    [?:1.8.0_40]
        at
    
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
    [?:1.8.0_40]
        at
    
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    [?:1.8.0_40]
        at
    
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    [?:1.8.0_40]
        at java.lang.Thread.run(Thread.java:745) [?:1.8.0_40]



This occurs randomly
I can't able to trace back to the problem

Maybe if spout takes too long to process, then streamparse can't 
acknowledge the heartbeat in time

changed "supervisor.worker.timeout.secs" from 30 to 600

Still the topology breaks. Is there any other options here?


Thanks,
Ram.