[jira] [Created] (HADOOP-11301) [optionally] update jmx cache to drop old metrics
Maysam Yabandeh created HADOOP-11301:
Summary: [optionally] update jmx cache to drop old metrics
Key: HADOOP-11301
URL: https://issues.apache.org/jira/browse/HADOOP-11301
Project: Hadoop Common
Issue Type: Improvement
Reporter: Maysam Yabandeh
Assignee: Maysam Yabandeh
MetricsSourceAdapter::updateJmxCache() skips updating the info cache if no new
metric is added since last time:
{code}
int oldCacheSize = attrCache.size();
int newCacheSize = updateAttrCache();
if (oldCacheSize newCacheSize) {
updateInfoCache();
}
{code}
This behavior is not desirable in some applications. For example nntop
(HDFS-6982) reports the top users via jmx. The list is updated after each
report. The previously reported top users hence should be removed from the
cache upon each report request.
In our production run of nntop we made a change to ignore the size check and
always perform updateInfoCache. I am planning to submit a patch including this
change. The feature can be enabled by a configuration parameter.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HADOOP-11276) Allow setting url connection timeout in Configuration
Maysam Yabandeh created HADOOP-11276:
Summary: Allow setting url connection timeout in Configuration
Key: HADOOP-11276
URL: https://issues.apache.org/jira/browse/HADOOP-11276
Project: Hadoop Common
Issue Type: Improvement
Reporter: Maysam Yabandeh
Priority: Minor
Currently for url resources there is no way to control the http connection
opened by Configuration:
{code}
private Document parse(DocumentBuilder builder, URL url)
throws IOException, SAXException {
if (!quietmode) {
LOG.debug(parsing URL + url);
}
if (url == null) {
return null;
}
return parse(builder, url.openStream(), url.toString());
}
{code}
If we let this method call a protected method that return a stream object from
a URL, then the application can override such method and set the application
specific connection settings on the URL, like
{code}
URLConnection con = url.openConnection();
con.setConnectTimeout(connectTimeout);
InputStream in = con.getInputStream();
return in;
{code}
Our monitoring tool currently needs to retrieve the conf from many app masters
and the default timeout of 60s is too much, and no way to configure it in the
current implementation of the Configuration.
If there is any +1 for this change I can submit an initial patch.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
