Hi Shadi,
Thanks for asking. This metric tracks the utilization of the event loop
<https://samza.apache.org/learn/documentation/0.8/container/event-loop.html>
within a samza container, which uses a single thread, that is in charge of
reading and writing messages, flushing metrics, checkpointing, and
windowing. It is important to track the utilization (aka "duty cycle") of
any event loop, which is the sum of all the timings (activeMs) divided by
the window length (totalMs).
You are right in that most of the time this value will be close to 1, which
represents complete utilization. However when the event loop starts to have
idle time, this metric will give you an idea of how much headroom you have
before the job will start to seriously fall behind.
I hope that answers your question!
ᐧ
On Thu, Jun 25, 2015 at 6:39 PM, Shadi Noghabi <
snogh...@linkedin.com.invalid> wrote:
> Hi,
>
> I was wondering what does this utilization metric in the
> SamzaContainerMetrics show? I am asking this sine in the code it is
> calculated as below:
>
> while (!shutdownNow) {
> val loopStartTime = clock();
> process
> window
> commit
> val totalMs = clock() - loopStartTime
> metrics.utilization.set(activeMs.toFloat/totalMs)
> activeMs = 0L
> }
>
> Where the totalMs is the time it takes to start calling process until
> commit is done which is equal to the time it takes to run process, window,
> and commit. And they way activeMs is calculated is by summing up the time
> it takes to call process, window and commit, which means these two values
> are going to be almost the same and the utilization is always going to be
> almost 1.
>
> I was just wondering what is the point of doing this?
>
>
>
>
