[
https://issues.apache.org/jira/browse/SAMZA-834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15046220#comment-15046220
]
Xinyu Liu commented on SAMZA-834:
---------------------------------
RB is: https://reviews.apache.org/r/41071/.
> Heap usage increases dramatically after container timer metrics change to
> nanosecond
> ------------------------------------------------------------------------------------
>
> Key: SAMZA-834
> URL: https://issues.apache.org/jira/browse/SAMZA-834
> Project: Samza
> Issue Type: Bug
> Reporter: Xinyu Liu
> Assignee: Xinyu Liu
> Attachments: SAMZA-843.0.patch
>
>
> After the change of container timer metrics (chooseNs, windowNs, processNs,
> and commitNs) from millisecond to nanosecond, we noticed a dramatic increase
> of memory heap usage in one of our production job. After investigation we
> found that the SlidingTimeWindowReservoir.update(duration) will be called
> much more frequently due to the duration is non-zero after the nanosecond
> change (In contrast, it is often zero when using millisecond). Within the
> 5-minute window, the storage inside SlidingTimeWindowReservoir increases a
> lot for a high qps job (for our job with around 10K qps, it increases the
> heap from <5M to 100M). It causes long GCs and degrades the job performance.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
