If Bolt B’s receive queue is full the tuple will be put into an overflow 
buffer. If  back pressure is not enabled it will keep on filling the overflow 
buffer and eventually cause an OOM.

You might also want to see the proposed changes in STORM-2306 where the receive 
queues are going to be bounded and the back pressure is allowed to propagate 
backwards up to the spout at which point the framework would stop invoking the 
nextTuple.

Thanks,
Arun

From:  Lawrence Craft <lawrence.cr...@gmail.com>
Reply-To:  "user@storm.apache.org" <user@storm.apache.org>
Date:  Tuesday, September 26, 2017 at 2:01 PM
To:  "user@storm.apache.org" <user@storm.apache.org>
Subject:  Behavior of Storm when buffers fill

Hi all, 

I'm trying to get a better understanding of how Storm behaves in adverse 
conditions. I read 
http://www.michael-noll.com/blog/2013/06/21/understanding-storm-internal-message-buffers/,
 describing the various buffers between different workers and components.

My question: if bolt A emits a tuple that is routed to Bolt B, but Bolt B's 
executor buffer is full, how does the system react? I would guess it'd be kept 
in the worker's receive buffer, but what happens when that fills? Does Bolt A's 
worker send thread block and stall?

Obviously this is not an ideal situation and buffers/backpressure watermarks 
should be set precisely to avoid something like this. I'd just like to get an 
idea of what happens.

Lawrence

Reply via email to