How are you measuring the utilization?
On November 22, 2017 at 08:12:51, Ali Nazemian (alinazem...@gmail.com) wrote: Hi all, One of the issues that we are dealing with is the fact that not all of the Metron feeds have the same type of resource requirements. For example, we have some feeds that even a single Strom slot is way more than what it needs. We thought we could make it more utilised in total by limiting at least the amount of available heap space per feed to the parser topology worker. However, since Storm scheduler relies on available slots, it is very hard and almost impossible to utilise the cluster in the scenario that there will be lots of different topologies with different requirements running at the same time. Therefore, on a daily basis, we can see that for example one of the Storm hosts is 120% utilised and another is 20% utilised! I was wondering whether we can address this situation by using Storm Resource Aware scheduler or not. P.S: it would be very nice to have a functionality to tune Storm topology-related parameters per feed in the GUI (for example in Management UI). Regards, Ali
