[
https://issues.apache.org/jira/browse/HADOOP-11301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224164#comment-14224164
]
stack commented on HADOOP-11301:
--------------------------------
I tried the patch and it seems like it is doing as expected, [~maysamyabandeh].
My test is basic. I just invoke the /jmx page in the UI. Before the patch,
with this simple logging inserted:
{code}
@@ -175,7 +175,9 @@ private void updateJmxCache() {
synchronized(this) {
int oldCacheSize = attrCache.size();
int newCacheSize = updateAttrCache();
+ LOG.info("Called updateAttrCache");
if (oldCacheSize < newCacheSize) {
+ LOG.info("Called updateInfoCache");
updateInfoCache();
}
jmxCacheTS = Time.now();
{code}
... I would see this output on each request in unpatched case.
{code}
2014-11-24 22:44:41,688 INFO [1851255134@qtp-1525844775-1]
impl.MetricsSourceAdapter: Called updateAttrCache
2014-11-24 22:44:41,689 INFO [1851255134@qtp-1525844775-1]
impl.MetricsSourceAdapter: Called updateAttrCache
2014-11-24 22:44:41,693 INFO [1851255134@qtp-1525844775-1]
impl.MetricsSourceAdapter: Called updateAttrCache
2014-11-24 22:44:41,694 INFO [1851255134@qtp-1525844775-1]
impl.MetricsSourceAdapter: Called updateAttrCache
2014-11-24 22:44:41,694 INFO [1851255134@qtp-1525844775-1]
impl.MetricsSourceAdapter: Called updateAttrCache
2014-11-24 22:44:41,694 INFO [1851255134@qtp-1525844775-1]
impl.MetricsSourceAdapter: Called updateInfoCache
2014-11-24 22:44:41,696 INFO [1851255134@qtp-1525844775-1]
impl.MetricsSourceAdapter: Called updateAttrCache
2014-11-24 22:44:41,699 INFO [1851255134@qtp-1525844775-1]
impl.MetricsSourceAdapter: Called updateAttrCache
2014-11-24 22:44:41,699 INFO [1851255134@qtp-1525844775-1]
impl.MetricsSourceAdapter: Called updateInfoCache
2014-11-24 22:44:41,699 INFO [1851255134@qtp-1525844775-1]
impl.MetricsSourceAdapter: Called updateAttrCache
{code}
I then applied the patch plus logging and would get this output on each page
invocation:
{code}
2014-11-24 23:20:49,145 INFO [2113243119@qtp-1525844775-0]
impl.MetricsSourceAdapter: Called updateAttrCache
2014-11-24 23:20:49,145 INFO [2113243119@qtp-1525844775-0]
impl.MetricsSourceAdapter: Called updateInfoCache
2014-11-24 23:20:49,146 INFO [2113243119@qtp-1525844775-0]
impl.MetricsSourceAdapter: Called updateAttrCache
2014-11-24 23:20:49,146 INFO [2113243119@qtp-1525844775-0]
impl.MetricsSourceAdapter: Called updateInfoCache
2014-11-24 23:20:49,154 INFO [2113243119@qtp-1525844775-0]
impl.MetricsSourceAdapter: Called updateAttrCache
2014-11-24 23:20:49,154 INFO [2113243119@qtp-1525844775-0]
impl.MetricsSourceAdapter: Called updateInfoCache
2014-11-24 23:20:49,155 INFO [2113243119@qtp-1525844775-0]
impl.MetricsSourceAdapter: Called updateAttrCache
2014-11-24 23:20:49,156 INFO [2113243119@qtp-1525844775-0]
impl.MetricsSourceAdapter: Called updateInfoCache
2014-11-24 23:20:49,157 INFO [2113243119@qtp-1525844775-0]
impl.MetricsSourceAdapter: Called updateAttrCache
2014-11-24 23:20:49,157 INFO [2113243119@qtp-1525844775-0]
impl.MetricsSourceAdapter: Called updateInfoCache
2014-11-24 23:20:49,193 INFO [2113243119@qtp-1525844775-0]
impl.MetricsSourceAdapter: Called updateAttrCache
2014-11-24 23:20:49,193 INFO [2113243119@qtp-1525844775-0]
impl.MetricsSourceAdapter: Called updateInfoCache
2014-11-24 23:20:49,203 INFO [2113243119@qtp-1525844775-0]
impl.MetricsSourceAdapter: Called updateAttrCache
2014-11-24 23:20:49,203 INFO [2113243119@qtp-1525844775-0]
impl.MetricsSourceAdapter: Called updateInfoCache
2014-11-24 23:20:49,203 INFO [2113243119@qtp-1525844775-0]
impl.MetricsSourceAdapter: Called updateAttrCache
2014-11-24 23:20:49,203 INFO [2113243119@qtp-1525844775-0]
impl.MetricsSourceAdapter: Called updateInfoCache
{code}
Its same amount of updateAttrCache updates but we are now doing a call to
updateInfoCache for each updateAttrCache call. Thats as expected I believe.
I was thinking that we were refreshing attrs and the bean on each invocation
but added this:
{code}
@@ -163,6 +163,7 @@ private void updateJmxCache() {
}
}
else {
+ LOG.info("Returned w/o updateAttrCache");
return;
}
}
{code}
... and then see this:
{code}
2014-11-24 23:45:22,002 INFO [2113243119@qtp-1525844775-0]
impl.MetricsSourceAdapter: Returned w/o updateAttrCache
2014-11-24 23:45:22,002 INFO [2113243119@qtp-1525844775-0]
impl.MetricsSourceAdapter: Returned w/o updateAttrCache
2014-11-24 23:45:22,002 INFO [2113243119@qtp-1525844775-0]
impl.MetricsSourceAdapter: Returned w/o updateAttrCache
2014-11-24 23:45:22,002 INFO [2113243119@qtp-1525844775-0]
impl.MetricsSourceAdapter: Returned w/o updateAttrCache
2014-11-24 23:45:22,002 INFO [2113243119@qtp-1525844775-0]
impl.MetricsSourceAdapter: Returned w/o updateAttrCache
2014-11-24 23:45:22,002 INFO [2113243119@qtp-1525844775-0]
impl.MetricsSourceAdapter: Returned w/o updateAttrCache
2014-11-24 23:45:22,005 INFO [2113243119@qtp-1525844775-0]
impl.MetricsSourceAdapter: Called updateAttrCache ttl=11, all=true
2014-11-24 23:45:22,005 INFO [2113243119@qtp-1525844775-0]
impl.MetricsSourceAdapter: Called updateInfoCache
2014-11-24 23:45:22,005 INFO [2113243119@qtp-1525844775-0]
impl.MetricsSourceAdapter: Returned w/o updateAttrCache
2014-11-24 23:45:22,005 INFO [2113243119@qtp-1525844775-0]
impl.MetricsSourceAdapter: Returned w/o updateAttrCache
2014-11-24 23:45:22,005 INFO [2113243119@qtp-1525844775-0]
impl.MetricsSourceAdapter: Returned w/o updateAttrCache
2014-11-24 23:45:22,005 INFO [2113243119@qtp-1525844775-0]
impl.MetricsSourceAdapter: Returned w/o updateAttrCache
2014-11-24 23:45:22,005 INFO [2113243119@qtp-1525844775-0]
impl.MetricsSourceAdapter: Returned w/o updateAttrCache
2014-11-24 23:45:22,005 INFO [2113243119@qtp-1525844775-0]
impl.MetricsSourceAdapter: Returned w/o updateAttrCache
2014-11-24 23:45:22,005 INFO [2113243119@qtp-1525844775-0]
impl.MetricsSourceAdapter: Returned w/o updateAttrCache
2014-11-24 23:45:22,005 INFO [2113243119@qtp-1525844775-0]
impl.MetricsSourceAdapter: Returned w/o updateAttrCache
2014-11-24 23:45:22,005 INFO [2113243119@qtp-1525844775-0]
impl.MetricsSourceAdapter: Returned w/o updateAttrCache
2014-11-24 23:45:22,005 INFO [2113243119@qtp-1525844775-0]
impl.MetricsSourceAdapter: Returned w/o updateAttrCache
2014-11-24 23:45:22,005 INFO [2113243119@qtp-1525844775-0]
impl.MetricsSourceAdapter: Returned w/o updateAttrCache
2014-11-24 23:45:22,005 INFO [2113243119@qtp-1525844775-0]
impl.MetricsSourceAdapter: Returned w/o updateAttrCache
2014-11-24 23:45:22,005 INFO [2113243119@qtp-1525844775-0]
impl.MetricsSourceAdapter: Returned w/o updateAttrCache
2014-11-24 23:45:22,005 INFO [2113243119@qtp-1525844775-0]
impl.MetricsSourceAdapter: Returned w/o updateAttrCache
2014-11-24 23:45:22,005 INFO [2113243119@qtp-1525844775-0]
impl.MetricsSourceAdapter: Returned w/o updateAttrCache
2014-11-24 23:45:22,047 INFO [2113243119@qtp-1525844775-0]
impl.MetricsSourceAdapter: Called updateAttrCache ttl=11, all=true
2014-11-24 23:45:22,047 INFO [2113243119@qtp-1525844775-0]
impl.MetricsSourceAdapter: Called updateInfoCache
...
{code}
So we are using cached data sometimes. Default says use cache for ten seconds.
That is not what I see above ('11'?). I should look at this more.
> [optionally] update jmx cache to drop old metrics
> -------------------------------------------------
>
> Key: HADOOP-11301
> URL: https://issues.apache.org/jira/browse/HADOOP-11301
> Project: Hadoop Common
> Issue Type: Improvement
> Reporter: Maysam Yabandeh
> Assignee: Maysam Yabandeh
> Attachments: HADOOP-11301.v01.patch, HADOOP-11301.v02.patch,
> HADOOP-11301.v03.patch, HADOOP-11301.v04.patch
>
>
> MetricsSourceAdapter::updateJmxCache() skips updating the info cache if no
> new metric is added since last time:
> {code}
> int oldCacheSize = attrCache.size();
> int newCacheSize = updateAttrCache();
> if (oldCacheSize < newCacheSize) {
> updateInfoCache();
> }
> {code}
> This behavior is not desirable in some applications. For example nntop
> (HDFS-6982) reports the top users via jmx. The list is updated after each
> report. The previously reported top users hence should be removed from the
> cache upon each report request.
> In our production run of nntop we made a change to ignore the size check and
> always perform updateInfoCache. I am planning to submit a patch including
> this change. The feature can be enabled by a configuration parameter.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
