[jira] [Commented] (HBASE-9983) Lower the memory footprint of HRegionLocation
[ https://issues.apache.org/jira/browse/HBASE-9983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13825179#comment-13825179 ] Nicolas Liochon commented on HBASE-9983: Note that I've closed the issue as I committed the patch, but more work is needed... Lower the memory footprint of HRegionLocation - Key: HBASE-9983 URL: https://issues.apache.org/jira/browse/HBASE-9983 Project: HBase Issue Type: Bug Components: Client Affects Versions: 0.98.0, 0.96.0 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Priority: Minor Fix For: 0.98.0, 0.96.1 Attachments: 9983.v2.patch, unbloated.patch Its more or less a desperate cause... If we have a lot of regions, the size of this class becomes a problem. It's under optimized for this case... This patch is just about the trivial part -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-9983) Lower the memory footprint of HRegionLocation
[ https://issues.apache.org/jira/browse/HBASE-9983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13825576#comment-13825576 ] Vladimir Rodionov commented on HBASE-9983: -- Its always good to measure the improvement. Any numbers, Nicolas, you can share? Lower the memory footprint of HRegionLocation - Key: HBASE-9983 URL: https://issues.apache.org/jira/browse/HBASE-9983 Project: HBase Issue Type: Bug Components: Client Affects Versions: 0.98.0, 0.96.0 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Priority: Minor Fix For: 0.98.0, 0.96.1 Attachments: 9983.v2.patch, unbloated.patch Its more or less a desperate cause... If we have a lot of regions, the size of this class becomes a problem. It's under optimized for this case... This patch is just about the trivial part -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-9983) Lower the memory footprint of HRegionLocation
[ https://issues.apache.org/jira/browse/HBASE-9983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13825598#comment-13825598 ] Nicolas Liochon commented on HBASE-9983: I haven't done a measure before/after for this one (would have been interested, I agree). Right now, so with this jira included, I'm seeing (with Yourkit) that an entry the location skip list takes ~550 bytes. This will be lowered with HBASE-9976 and HBASE-9988. Then we have one ServerName instance by entry as well... Lower the memory footprint of HRegionLocation - Key: HBASE-9983 URL: https://issues.apache.org/jira/browse/HBASE-9983 Project: HBase Issue Type: Bug Components: Client Affects Versions: 0.98.0, 0.96.0 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Priority: Minor Fix For: 0.98.0, 0.96.1 Attachments: 9983.v2.patch, unbloated.patch Its more or less a desperate cause... If we have a lot of regions, the size of this class becomes a problem. It's under optimized for this case... This patch is just about the trivial part -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-9983) Lower the memory footprint of HRegionLocation
[ https://issues.apache.org/jira/browse/HBASE-9983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13824799#comment-13824799 ] Nicolas Liochon commented on HBASE-9983: v2 is what I committed (rebased, ClentNoCluster removed, commented added). Thanks for the review, Stack Ted. Lower the memory footprint of HRegionLocation - Key: HBASE-9983 URL: https://issues.apache.org/jira/browse/HBASE-9983 Project: HBase Issue Type: Bug Components: Client Affects Versions: 0.98.0, 0.96.0 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Priority: Minor Fix For: 0.98.0, 0.96.1 Attachments: 9983.v2.patch, unbloated.patch Its more or less a desperate cause... If we have a lot of regions, the size of this class becomes a problem. It's under optimized for this case... This patch is just about the trivial part -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-9983) Lower the memory footprint of HRegionLocation
[ https://issues.apache.org/jira/browse/HBASE-9983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13824865#comment-13824865 ] Hudson commented on HBASE-9983: --- FAILURE: Integrated in hbase-0.96-hadoop2 #121 (See [https://builds.apache.org/job/hbase-0.96-hadoop2/121/]) HBASE-9983 Lower the memory footprint of HRegionLocation (nkeywal: rev 1542696) * /hbase/branches/0.96/hbase-client/src/main/java/org/apache/hadoop/hbase/HRegionInfo.java * /hbase/branches/0.96/hbase-client/src/main/java/org/apache/hadoop/hbase/HRegionLocation.java * /hbase/branches/0.96/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncProcess.java * /hbase/branches/0.96/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java Lower the memory footprint of HRegionLocation - Key: HBASE-9983 URL: https://issues.apache.org/jira/browse/HBASE-9983 Project: HBase Issue Type: Bug Components: Client Affects Versions: 0.98.0, 0.96.0 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Priority: Minor Fix For: 0.98.0, 0.96.1 Attachments: 9983.v2.patch, unbloated.patch Its more or less a desperate cause... If we have a lot of regions, the size of this class becomes a problem. It's under optimized for this case... This patch is just about the trivial part -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-9983) Lower the memory footprint of HRegionLocation
[ https://issues.apache.org/jira/browse/HBASE-9983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13824869#comment-13824869 ] Hudson commented on HBASE-9983: --- SUCCESS: Integrated in hbase-0.96 #192 (See [https://builds.apache.org/job/hbase-0.96/192/]) HBASE-9983 Lower the memory footprint of HRegionLocation (nkeywal: rev 1542696) * /hbase/branches/0.96/hbase-client/src/main/java/org/apache/hadoop/hbase/HRegionInfo.java * /hbase/branches/0.96/hbase-client/src/main/java/org/apache/hadoop/hbase/HRegionLocation.java * /hbase/branches/0.96/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncProcess.java * /hbase/branches/0.96/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java Lower the memory footprint of HRegionLocation - Key: HBASE-9983 URL: https://issues.apache.org/jira/browse/HBASE-9983 Project: HBase Issue Type: Bug Components: Client Affects Versions: 0.98.0, 0.96.0 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Priority: Minor Fix For: 0.98.0, 0.96.1 Attachments: 9983.v2.patch, unbloated.patch Its more or less a desperate cause... If we have a lot of regions, the size of this class becomes a problem. It's under optimized for this case... This patch is just about the trivial part -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-9983) Lower the memory footprint of HRegionLocation
[ https://issues.apache.org/jira/browse/HBASE-9983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13824867#comment-13824867 ] Hudson commented on HBASE-9983: --- SUCCESS: Integrated in HBase-TRUNK #4683 (See [https://builds.apache.org/job/HBase-TRUNK/4683/]) HBASE-9983 Lower the memory footprint of HRegionLocation (nkeywal: rev 1542694) * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/HRegionInfo.java * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/HRegionLocation.java * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncProcess.java * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java Lower the memory footprint of HRegionLocation - Key: HBASE-9983 URL: https://issues.apache.org/jira/browse/HBASE-9983 Project: HBase Issue Type: Bug Components: Client Affects Versions: 0.98.0, 0.96.0 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Priority: Minor Fix For: 0.98.0, 0.96.1 Attachments: 9983.v2.patch, unbloated.patch Its more or less a desperate cause... If we have a lot of regions, the size of this class becomes a problem. It's under optimized for this case... This patch is just about the trivial part -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-9983) Lower the memory footprint of HRegionLocation
[ https://issues.apache.org/jira/browse/HBASE-9983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13824883#comment-13824883 ] Hudson commented on HBASE-9983: --- SUCCESS: Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #840 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/840/]) HBASE-9983 Lower the memory footprint of HRegionLocation (nkeywal: rev 1542694) * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/HRegionInfo.java * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/HRegionLocation.java * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncProcess.java * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java Lower the memory footprint of HRegionLocation - Key: HBASE-9983 URL: https://issues.apache.org/jira/browse/HBASE-9983 Project: HBase Issue Type: Bug Components: Client Affects Versions: 0.98.0, 0.96.0 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Priority: Minor Fix For: 0.98.0, 0.96.1 Attachments: 9983.v2.patch, unbloated.patch Its more or less a desperate cause... If we have a lot of regions, the size of this class becomes a problem. It's under optimized for this case... This patch is just about the trivial part -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-9983) Lower the memory footprint of HRegionLocation
[
https://issues.apache.org/jira/browse/HBASE-9983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13824658#comment-13824658
]
Hadoop QA commented on HBASE-9983:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12614244/unbloated.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated
10 warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/7899//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7899//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7899//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7899//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7899//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7899//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7899//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7899//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7899//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7899//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/7899//console
This message is automatically generated.
Lower the memory footprint of HRegionLocation
-
Key: HBASE-9983
URL: https://issues.apache.org/jira/browse/HBASE-9983
Project: HBase
Issue Type: Bug
Components: Client
Affects Versions: 0.98.0, 0.96.0
Reporter: Nicolas Liochon
Assignee: Nicolas Liochon
Priority: Minor
Fix For: 0.98.0, 0.96.1
Attachments: unbloated.patch
Its more or less a desperate cause...
If we have a lot of regions, the size of this class becomes a problem. It's
under optimized for this case...
This patch is just about the trivial part
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HBASE-9983) Lower the memory footprint of HRegionLocation
[
https://issues.apache.org/jira/browse/HBASE-9983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13824670#comment-13824670
]
Ted Yu commented on HBASE-9983:
---
{code}
- addresses.add(location != null ? location.getHostnamePort() : null
location);
+ addresses.add(location != null ? location.getServerName().toString() :
null location);
{code}
What if two region servers run on the same server ?
Lower the memory footprint of HRegionLocation
-
Key: HBASE-9983
URL: https://issues.apache.org/jira/browse/HBASE-9983
Project: HBase
Issue Type: Bug
Components: Client
Affects Versions: 0.98.0, 0.96.0
Reporter: Nicolas Liochon
Assignee: Nicolas Liochon
Priority: Minor
Fix For: 0.98.0, 0.96.1
Attachments: unbloated.patch
Its more or less a desperate cause...
If we have a lot of regions, the size of this class becomes a problem. It's
under optimized for this case...
This patch is just about the trivial part
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HBASE-9983) Lower the memory footprint of HRegionLocation
[ https://issues.apache.org/jira/browse/HBASE-9983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13824744#comment-13824744 ] stack commented on HBASE-9983: -- bq. What if two region servers run on the same server ? ServerName includes port. +1 Looks like another issue leaked into this one: -byte [] b = format(i); +byte [] b = format(rd.nextLong()); Lower the memory footprint of HRegionLocation - Key: HBASE-9983 URL: https://issues.apache.org/jira/browse/HBASE-9983 Project: HBase Issue Type: Bug Components: Client Affects Versions: 0.98.0, 0.96.0 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Priority: Minor Fix For: 0.98.0, 0.96.1 Attachments: unbloated.patch Its more or less a desperate cause... If we have a lot of regions, the size of this class becomes a problem. It's under optimized for this case... This patch is just about the trivial part -- This message was sent by Atlassian JIRA (v6.1#6144)
