Hi Jayant, The MAX_SPOUT_PENDING is applied in each of the spouts.
Each spout take count of all the tuples it sends, increasing it for each emit and decreasing it when it receive the final ack message for a “pending tuple” (= a tuple that is sent but it’s still waiting an ack). If the number of pending tuples raises above the value you set, let’s say 500, the spouts will automatically slow down, until they reach an emission rate such that the number of pending tuples stabilise under 500. This mechanism is applied per spout, meaning that each spout can be limited in different times, and the MAX_SPOUT_PENDING represents the maximum number of non-acked tuples that a single spout will allow. Alessio > On 3 Apr 2019, at 14:45, Joshua Martell <joshua.mart...@gmail.com> wrote: > > I’m pretty sure it’s per spout task. And they’re not coordinated. Each task > gets its own count. > > Joshua > On Wed, Apr 3, 2019 at 4:50 AM Jayant Sharma <sharmajayan...@gmail.com > <mailto:sharmajayan...@gmail.com>> wrote: > Hi, > > Can someone please explain at what point storm checks and applies > MAX_SPOUT_PENDING limit. Also, is this limit applied per executor(or Task if > that's the case) of spout or aggregated and applied over all the executors of > spout. > Suppose I have 3 executors and 3 tasks of spouts, each of them fetch 1 > message from input source. If my MAX_SPOUT_PENDING is 2, will only 2 > executors be able to send the tuple forward or all 3 will send and further > nextTuple calls on all will be blocked? > > Thanks, > Jayant Sharma
