Any help regarding this question would be appreciated.
On Thu, Nov 23, 2017 at 8:57 AM, Ali Nazemian <alinazem...@gmail.com> wrote: > 30 mins average of CPU load by checking Ambari. > > On 23 Nov. 2017 00:51, "Otto Fowler" <ottobackwa...@gmail.com> wrote: > > How are you measuring the utilization? > > > On November 22, 2017 at 08:12:51, Ali Nazemian (alinazem...@gmail.com) > wrote: > > Hi all, > > > One of the issues that we are dealing with is the fact that not all of > the Metron feeds have the same type of resource requirements. For example, > we have some feeds that even a single Strom slot is way more than what it > needs. We thought we could make it more utilised in total by limiting at > least the amount of available heap space per feed to the parser topology > worker. However, since Storm scheduler relies on available slots, it is > very hard and almost impossible to utilise the cluster in the scenario > that > there will be lots of different topologies with different requirements > running at the same time. Therefore, on a daily basis, we can see that for > example one of the Storm hosts is 120% utilised and another is 20% > utilised! I was wondering whether we can address this situation by using > Storm Resource Aware scheduler or not. > > P.S: it would be very nice to have a functionality to tune Storm > topology-related parameters per feed in the GUI (for example in Management > UI). > > > Regards, > Ali > > > -- A.Nazemian
