The miguno blog post is a bit out of date, it predates the switch from ZeroMQ to Netty as the communication layer between workers.
Notably, netty has an unbounded buffer (at least in version 0.9.6): - https://github.com/apache/storm/blob/v0.9.6/storm-core/src/jvm/backtype/storm/messaging/netty/Server.java#L97 On Thu, Mar 2, 2017 at 1:28 PM, David Koch <ogd...@googlemail.com> wrote: > Hi, > > The Storm documentation mentions setting topology.max.spout.pending as a > way of preventing "queue explosion"[1]. What is meant by this? Tuples > piling up and eventually causing out of memory exceptions? If I understand > correctly, the topology's queue and buffer sizes are all limited [2] - so > at what point could something explode even without limiting the maximum > number of pending tuples and/or the back pressure mechanism activated. > > Thanks, > > David > > [1] http://storm.apache.org/releases/1.0.0/Running- > topologies-on-a-production-cluster.html > [2] http://www.michael-noll.com/blog/2013/06/21/ > understanding-storm-internal-message-buffers/ >
