if you set topology.max.spout.pending and use reliable tuples then the spout will automatically throttle itself when its output queue grows to the configured size.
On Tue, Sep 30, 2014 at 11:04 AM, Jan Sipke van der Veen <jansi...@gmail.com > wrote: > Hello, > > I am using a simple topology with a single spout and a single bolt to test > some ideas about automatically scaling the number of worker nodes in a > Storm cluster. The bolt is set up to use some processor time and the spout > sends out tuples at a rate which is slightly higher than the bolt can > process. > > The number of emitted tuples is indeed higher than the number of acked > tuples and some time later there are some failed tuples. Just what I > expected. However, after about 5 to 10 minutes, it seems that nextTuple() > isn't called as often as before and the number of emitted tuples drops to a > level that the bolt is able to keep up with. > > Is there some sort of automatic throttling of spouts that I'm not aware of? > > Best regards, > Jan Sipke van der Veen >
