[jira] Commented: (HDFS-94) The "Heap Size" in HDFS web ui may not be accurate
[
https://issues.apache.org/jira/browse/HDFS-94?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12799010#action_12799010
]
Hudson commented on HDFS-94:
Integrated in Hdfs-Patch-h2.grid.sp2.yahoo.net #94 (See
[http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/94/])
> The "Heap Size" in HDFS web ui may not be accurate
> --
>
> Key: HDFS-94
> URL: https://issues.apache.org/jira/browse/HDFS-94
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Tsz Wo (Nicholas), SZE
>Assignee: Dmytro Molkov
> Fix For: 0.22.0
>
> Attachments: HDFS-94.patch
>
>
> It seems that the Heap Size shown in HDFS web UI is not accurate. It keeps
> showing 100% of usage. e.g.
> {noformat}
> Heap Size is 10.01 GB / 10.01 GB (100%)
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-94) The "Heap Size" in HDFS web ui may not be accurate
[
https://issues.apache.org/jira/browse/HDFS-94?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12794928#action_12794928
]
Hudson commented on HDFS-94:
Integrated in Hadoop-Hdfs-trunk-Commit #158 (See
[http://hudson.zones.apache.org/hudson/job/Hadoop-Hdfs-trunk-Commit/158/])
> The "Heap Size" in HDFS web ui may not be accurate
> --
>
> Key: HDFS-94
> URL: https://issues.apache.org/jira/browse/HDFS-94
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Tsz Wo (Nicholas), SZE
>Assignee: Dmytro Molkov
> Fix For: 0.22.0
>
> Attachments: HDFS-94.patch
>
>
> It seems that the Heap Size shown in HDFS web UI is not accurate. It keeps
> showing 100% of usage. e.g.
> {noformat}
> Heap Size is 10.01 GB / 10.01 GB (100%)
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-94) The "Heap Size" in HDFS web ui may not be accurate
[
https://issues.apache.org/jira/browse/HDFS-94?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12794609#action_12794609
]
Hudson commented on HDFS-94:
Integrated in Hadoop-Hdfs-trunk #182 (See
[http://hudson.zones.apache.org/hudson/job/Hadoop-Hdfs-trunk/182/])
> The "Heap Size" in HDFS web ui may not be accurate
> --
>
> Key: HDFS-94
> URL: https://issues.apache.org/jira/browse/HDFS-94
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Tsz Wo (Nicholas), SZE
>Assignee: Dmytro Molkov
> Fix For: 0.22.0
>
> Attachments: HDFS-94.patch
>
>
> It seems that the Heap Size shown in HDFS web UI is not accurate. It keeps
> showing 100% of usage. e.g.
> {noformat}
> Heap Size is 10.01 GB / 10.01 GB (100%)
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-94) The "Heap Size" in HDFS web ui may not be accurate
[
https://issues.apache.org/jira/browse/HDFS-94?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12794347#action_12794347
]
Hudson commented on HDFS-94:
Integrated in Hdfs-Patch-h5.grid.sp2.yahoo.net #159 (See
[http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/159/])
. The Heap Size printed in the NameNode WebUI is accurate.
(Dmytro Molkov via dhruba)
> The "Heap Size" in HDFS web ui may not be accurate
> --
>
> Key: HDFS-94
> URL: https://issues.apache.org/jira/browse/HDFS-94
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Tsz Wo (Nicholas), SZE
>Assignee: Dmytro Molkov
> Fix For: 0.22.0
>
> Attachments: HDFS-94.patch
>
>
> It seems that the Heap Size shown in HDFS web UI is not accurate. It keeps
> showing 100% of usage. e.g.
> {noformat}
> Heap Size is 10.01 GB / 10.01 GB (100%)
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-94) The "Heap Size" in HDFS web ui may not be accurate
[
https://issues.apache.org/jira/browse/HDFS-94?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12793875#action_12793875
]
Hairong Kuang commented on HDFS-94:
---
I filed HDFS-849 against the first error
org.apache.hadoop.hdfs.server.datanode.TestFiDataTransferProtocol2.pipeline_Fi_18
.
> The "Heap Size" in HDFS web ui may not be accurate
> --
>
> Key: HDFS-94
> URL: https://issues.apache.org/jira/browse/HDFS-94
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Tsz Wo (Nicholas), SZE
>Assignee: Dmytro Molkov
> Attachments: HDFS-94.patch
>
>
> It seems that the Heap Size shown in HDFS web UI is not accurate. It keeps
> showing 100% of usage. e.g.
> {noformat}
> Heap Size is 10.01 GB / 10.01 GB (100%)
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-94) The "Heap Size" in HDFS web ui may not be accurate
[
https://issues.apache.org/jira/browse/HDFS-94?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12793836#action_12793836
]
dhruba borthakur commented on HDFS-94:
--
The failed tests are
org.apache.hadoop.hdfs.server.datanode.TestFiDataTransferProtocol2.pipeline_Fi_18
org.apache.hadoop.hdfs.TestReadWhileWriting.pipeline_02_03
and are not related to this patch.
i will commit this patch soon.
> The "Heap Size" in HDFS web ui may not be accurate
> --
>
> Key: HDFS-94
> URL: https://issues.apache.org/jira/browse/HDFS-94
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Tsz Wo (Nicholas), SZE
>Assignee: Dmytro Molkov
> Attachments: HDFS-94.patch
>
>
> It seems that the Heap Size shown in HDFS web UI is not accurate. It keeps
> showing 100% of usage. e.g.
> {noformat}
> Heap Size is 10.01 GB / 10.01 GB (100%)
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-94) The "Heap Size" in HDFS web ui may not be accurate
[
https://issues.apache.org/jira/browse/HDFS-94?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12793781#action_12793781
]
Hadoop QA commented on HDFS-94:
---
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12428685/HDFS-94.patch
against trunk revision 893066.
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified
tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac
compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of
release audit warnings.
-1 core tests. The patch failed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results:
http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/156/testReport/
Findbugs warnings:
http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/156/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results:
http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/156/artifact/trunk/build/test/checkstyle-errors.html
Console output:
http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/156/console
This message is automatically generated.
> The "Heap Size" in HDFS web ui may not be accurate
> --
>
> Key: HDFS-94
> URL: https://issues.apache.org/jira/browse/HDFS-94
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Tsz Wo (Nicholas), SZE
>Assignee: Dmytro Molkov
> Attachments: HDFS-94.patch
>
>
> It seems that the Heap Size shown in HDFS web UI is not accurate. It keeps
> showing 100% of usage. e.g.
> {noformat}
> Heap Size is 10.01 GB / 10.01 GB (100%)
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-94) The "Heap Size" in HDFS web ui may not be accurate
[
https://issues.apache.org/jira/browse/HDFS-94?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12793702#action_12793702
]
dhruba borthakur commented on HDFS-94:
--
Thanks Dmytro for the patch.
@Nicholas: please let me know if you agree that the additional metrics (e.g.
non heap usage, used, committed) be done via another JIRA. Also, I do not see
the requirement of a unit test for this case.
> The "Heap Size" in HDFS web ui may not be accurate
> --
>
> Key: HDFS-94
> URL: https://issues.apache.org/jira/browse/HDFS-94
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Tsz Wo (Nicholas), SZE
> Attachments: HDFS-94.patch
>
>
> It seems that the Heap Size shown in HDFS web UI is not accurate. It keeps
> showing 100% of usage. e.g.
> {noformat}
> Heap Size is 10.01 GB / 10.01 GB (100%)
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-94) The "Heap Size" in HDFS web ui may not be accurate
[
https://issues.apache.org/jira/browse/HDFS-94?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1222#action_1222
]
Tsz Wo (Nicholas), SZE commented on HDFS-94:
It makes to replace the current codes by MemoryMXBean since it provides more
information. I think it is better to show more numbers like non-heap usage,
init, used, committed, max, etc.
> The "Heap Size" in HDFS web ui may not be accurate
> --
>
> Key: HDFS-94
> URL: https://issues.apache.org/jira/browse/HDFS-94
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Tsz Wo (Nicholas), SZE
>
> It seems that the Heap Size shown in HDFS web UI is not accurate. It keeps
> showing 100% of usage. e.g.
> {noformat}
> Heap Size is 10.01 GB / 10.01 GB (100%)
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-94) The "Heap Size" in HDFS web ui may not be accurate
[
https://issues.apache.org/jira/browse/HDFS-94?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12777438#action_12777438
]
dhruba borthakur commented on HDFS-94:
--
Currently, the code uses
{quote}
long totalMemory = Runtime.getRuntime().totalMemory();
long maxMemory = Runtime.getRuntime().maxMemory();
long used = (totalMemory * 100)/maxMemory;
{quote}
Is it better to use :
{quote}
MemoryMXBean memoryMXBean = ManagementFactory.getMemoryMXBean();
MemoryUsage status = memoryMXBean.getHeapMemoryUsage();
usedMemory = status.getUsed();
maxMemory = status.getMax();
{quote}
> The "Heap Size" in HDFS web ui may not be accurate
> --
>
> Key: HDFS-94
> URL: https://issues.apache.org/jira/browse/HDFS-94
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Tsz Wo (Nicholas), SZE
>
> It seems that the Heap Size shown in HDFS web UI is not accurate. It keeps
> showing 100% of usage. e.g.
> {noformat}
> Heap Size is 10.01 GB / 10.01 GB (100%)
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-94) The "Heap Size" in HDFS web ui may not be accurate
[
https://issues.apache.org/jira/browse/HDFS-94?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12777169#action_12777169
]
dhruba borthakur commented on HDFS-94:
--
I agree. Do you plan to do anything to this JIRA to make the reporting more
accurate?
> The "Heap Size" in HDFS web ui may not be accurate
> --
>
> Key: HDFS-94
> URL: https://issues.apache.org/jira/browse/HDFS-94
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Tsz Wo (Nicholas), SZE
>
> It seems that the Heap Size shown in HDFS web UI is not accurate. It keeps
> showing 100% of usage. e.g.
> {noformat}
> Heap Size is 10.01 GB / 10.01 GB (100%)
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-94) The "Heap Size" in HDFS web ui may not be accurate
[
https://issues.apache.org/jira/browse/HDFS-94?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12776711#action_12776711
]
Tsz Wo (Nicholas), SZE commented on HDFS-94:
> I wonder why it shows a total of 17.78GB instead of 20GB
Would it be the case that you have hit some limit? The following is quoted
from [java man
page|http://java.sun.com/javase/6/docs/technotes/tools/solaris/java.html]:
{quote}
On Solaris 7 and Solaris 8 SPARC platforms, the upper limit for this value is
approximately 4000m minus overhead amounts. On Solaris 2.6 and x86 platforms,
the upper limit is approximately 2000m minus overhead amounts. On Linux
platforms, the upper limit is approximately 2000m minus overhead amounts.
{quote}
> The "Heap Size" in HDFS web ui may not be accurate
> --
>
> Key: HDFS-94
> URL: https://issues.apache.org/jira/browse/HDFS-94
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Tsz Wo (Nicholas), SZE
>
> It seems that the Heap Size shown in HDFS web UI is not accurate. It keeps
> showing 100% of usage. e.g.
> {noformat}
> Heap Size is 10.01 GB / 10.01 GB (100%)
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
