Thanks for your reply Nathan.
I have experimented a bit more with this topology and have now set the
topology.max.spout.pending to a fixed value that makes sense for the
topology. I still see some strange behavior however.
The cluster consists of four Storm worker nodes with two processor cores
each. The topology is created with these settings:
config.setMaxSpoutPending(12);
config.setNumWorkers(12);
builder.setSpout("elasticspout", new ElasticSpout(), 1).setNumTasks(1);
builder.setBolt("elasticbolt", new ElasticBolt(),
24).setNumTasks(24).shuffleGrouping("elasticspout");
In the attached screenshot you can see that at first the CPU load of all
four workers is high (both cores are combined into the metric, so 1 means
that all cores are busy). But after a few minutes, the load drops on all
worker nodes because the number of emitted tuples is lower. The queue that
the spout fetches data from is filled with plenty of data, so there is no
throttling there.
Best regards,
Jan Sipke van der Veen
On Tue, Sep 30, 2014 at 5:26 PM, Nathan Leung <ncle...@gmail.com> wrote:
> if you set topology.max.spout.pending and use reliable tuples then the
> spout will automatically throttle itself when its output queue grows to the
> configured size.
>
> On Tue, Sep 30, 2014 at 11:04 AM, Jan Sipke van der Veen <
> jansi...@gmail.com> wrote:
>
>> Hello,
>>
>> I am using a simple topology with a single spout and a single bolt to
>> test some ideas about automatically scaling the number of worker nodes in a
>> Storm cluster. The bolt is set up to use some processor time and the spout
>> sends out tuples at a rate which is slightly higher than the bolt can
>> process.
>>
>> The number of emitted tuples is indeed higher than the number of acked
>> tuples and some time later there are some failed tuples. Just what I
>> expected. However, after about 5 to 10 minutes, it seems that nextTuple()
>> isn't called as often as before and the number of emitted tuples drops to a
>> level that the bolt is able to keep up with.
>>
>> Is there some sort of automatic throttling of spouts that I'm not aware
>> of?
>>
>> Best regards,
>> Jan Sipke van der Veen
>>
>
>
