[jira] [Commented] (HBASE-12128) Cache configuration and RpcController selection for Table in Connection
[ https://issues.apache.org/jira/browse/HBASE-12128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14242165#comment-14242165 ] Hudson commented on HBASE-12128: SUCCESS: Integrated in HBase-0.98 #734 (See [https://builds.apache.org/job/HBase-0.98/734/]) HBASE-12604 Backport HBASE-12128 (Cache configuration and RpcController selection for Table in Connection) to 0.98 (apurtell: rev 4c1a31f289ca76a3196e5456108858e519e9c53e) * hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/TableConfiguration.java Cache configuration and RpcController selection for Table in Connection --- Key: HBASE-12128 URL: https://issues.apache.org/jira/browse/HBASE-12128 Project: HBase Issue Type: Sub-task Affects Versions: 1.0.0, 2.0.0 Reporter: Andrew Purtell Assignee: Stephen Yuan Jiang Fix For: 2.0.0, 0.99.2 Attachments: HBASE-12128.v1-1.0.patch, HBASE-12128.v1-2.0.patch, HBASE-12128.v2-2.0.patch Original Estimate: 120h Time Spent: 72h Remaining Estimate: 48h Creating Table instances should be lightweight. Apps that manage their own Connections are expected to create Tables on demand for each interaction. However we look up values from Hadoop Configuration when constructing Table objects for storing to some of its fields. Configuration is a heavyweight registry that does a lot of string operations and regex matching. Method calls into Configuration account for 48.25% of CPU time when creating the HTable object in 0.98. Another ~48% of CPU is spent constructing the desired RpcController object via reflection in 0.98. Together this can account for ~20% of total on-CPU time of the client. See parent issue for more detail. We are using Connection like a factory for Table. We should cache configuration for Table in Connection. We should also create by reflection once and cache the desired RpcController object, and clone it for new Tables. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12128) Cache configuration and RpcController selection for Table in Connection
[ https://issues.apache.org/jira/browse/HBASE-12128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14242180#comment-14242180 ] Hudson commented on HBASE-12128: FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #701 (See [https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/701/]) HBASE-12604 Backport HBASE-12128 (Cache configuration and RpcController selection for Table in Connection) to 0.98 (apurtell: rev 4c1a31f289ca76a3196e5456108858e519e9c53e) * hbase-client/src/main/java/org/apache/hadoop/hbase/client/TableConfiguration.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java Cache configuration and RpcController selection for Table in Connection --- Key: HBASE-12128 URL: https://issues.apache.org/jira/browse/HBASE-12128 Project: HBase Issue Type: Sub-task Affects Versions: 1.0.0, 2.0.0 Reporter: Andrew Purtell Assignee: Stephen Yuan Jiang Fix For: 2.0.0, 0.99.2 Attachments: HBASE-12128.v1-1.0.patch, HBASE-12128.v1-2.0.patch, HBASE-12128.v2-2.0.patch Original Estimate: 120h Time Spent: 72h Remaining Estimate: 48h Creating Table instances should be lightweight. Apps that manage their own Connections are expected to create Tables on demand for each interaction. However we look up values from Hadoop Configuration when constructing Table objects for storing to some of its fields. Configuration is a heavyweight registry that does a lot of string operations and regex matching. Method calls into Configuration account for 48.25% of CPU time when creating the HTable object in 0.98. Another ~48% of CPU is spent constructing the desired RpcController object via reflection in 0.98. Together this can account for ~20% of total on-CPU time of the client. See parent issue for more detail. We are using Connection like a factory for Table. We should cache configuration for Table in Connection. We should also create by reflection once and cache the desired RpcController object, and clone it for new Tables. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12128) Cache configuration and RpcController selection for Table in Connection
[ https://issues.apache.org/jira/browse/HBASE-12128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14230683#comment-14230683 ] Enis Soztutar commented on HBASE-12128: --- Sorry, when I commit, I had to drop the 0.98 as fixVersion since it is not committed there. Next time I'll ping or open a backport jira. Cache configuration and RpcController selection for Table in Connection --- Key: HBASE-12128 URL: https://issues.apache.org/jira/browse/HBASE-12128 Project: HBase Issue Type: Sub-task Affects Versions: 1.0.0, 2.0.0 Reporter: Andrew Purtell Assignee: Stephen Yuan Jiang Fix For: 2.0.0, 0.99.2 Attachments: HBASE-12128.v1-1.0.patch, HBASE-12128.v1-2.0.patch, HBASE-12128.v2-2.0.patch Original Estimate: 120h Time Spent: 72h Remaining Estimate: 48h Creating Table instances should be lightweight. Apps that manage their own Connections are expected to create Tables on demand for each interaction. However we look up values from Hadoop Configuration when constructing Table objects for storing to some of its fields. Configuration is a heavyweight registry that does a lot of string operations and regex matching. Method calls into Configuration account for 48.25% of CPU time when creating the HTable object in 0.98. Another ~48% of CPU is spent constructing the desired RpcController object via reflection in 0.98. Together this can account for ~20% of total on-CPU time of the client. See parent issue for more detail. We are using Connection like a factory for Table. We should cache configuration for Table in Connection. We should also create by reflection once and cache the desired RpcController object, and clone it for new Tables. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12128) Cache configuration and RpcController selection for Table in Connection
[ https://issues.apache.org/jira/browse/HBASE-12128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14230692#comment-14230692 ] Andrew Purtell commented on HBASE-12128: Thanks! Cache configuration and RpcController selection for Table in Connection --- Key: HBASE-12128 URL: https://issues.apache.org/jira/browse/HBASE-12128 Project: HBase Issue Type: Sub-task Affects Versions: 1.0.0, 2.0.0 Reporter: Andrew Purtell Assignee: Stephen Yuan Jiang Fix For: 2.0.0, 0.99.2 Attachments: HBASE-12128.v1-1.0.patch, HBASE-12128.v1-2.0.patch, HBASE-12128.v2-2.0.patch Original Estimate: 120h Time Spent: 72h Remaining Estimate: 48h Creating Table instances should be lightweight. Apps that manage their own Connections are expected to create Tables on demand for each interaction. However we look up values from Hadoop Configuration when constructing Table objects for storing to some of its fields. Configuration is a heavyweight registry that does a lot of string operations and regex matching. Method calls into Configuration account for 48.25% of CPU time when creating the HTable object in 0.98. Another ~48% of CPU is spent constructing the desired RpcController object via reflection in 0.98. Together this can account for ~20% of total on-CPU time of the client. See parent issue for more detail. We are using Connection like a factory for Table. We should cache configuration for Table in Connection. We should also create by reflection once and cache the desired RpcController object, and clone it for new Tables. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12128) Cache configuration and RpcController selection for Table in Connection
[ https://issues.apache.org/jira/browse/HBASE-12128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14229408#comment-14229408 ] Andrew Purtell commented on HBASE-12128: Please don't drop 0.98 fix versions from issues. I had it set here in the hopes this change would be backported to 0.98 also. It's fine if you don't want to do that work but by changing the fix version without a ping I might have missed it in a report. I will open a backport issue. Cache configuration and RpcController selection for Table in Connection --- Key: HBASE-12128 URL: https://issues.apache.org/jira/browse/HBASE-12128 Project: HBase Issue Type: Sub-task Affects Versions: 1.0.0, 2.0.0 Reporter: Andrew Purtell Assignee: Stephen Yuan Jiang Fix For: 2.0.0, 0.99.2 Attachments: HBASE-12128.v1-1.0.patch, HBASE-12128.v1-2.0.patch, HBASE-12128.v2-2.0.patch Original Estimate: 120h Time Spent: 72h Remaining Estimate: 48h Creating Table instances should be lightweight. Apps that manage their own Connections are expected to create Tables on demand for each interaction. However we look up values from Hadoop Configuration when constructing Table objects for storing to some of its fields. Configuration is a heavyweight registry that does a lot of string operations and regex matching. Method calls into Configuration account for 48.25% of CPU time when creating the HTable object in 0.98. Another ~48% of CPU is spent constructing the desired RpcController object via reflection in 0.98. Together this can account for ~20% of total on-CPU time of the client. See parent issue for more detail. We are using Connection like a factory for Table. We should cache configuration for Table in Connection. We should also create by reflection once and cache the desired RpcController object, and clone it for new Tables. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12128) Cache configuration and RpcController selection for Table in Connection
[ https://issues.apache.org/jira/browse/HBASE-12128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14226849#comment-14226849 ] Hudson commented on HBASE-12128: FAILURE: Integrated in HBase-TRUNK #5826 (See [https://builds.apache.org/job/HBase-TRUNK/5826/]) HBASE-12128 Cache configuration and RpcController selection for Table in Connection (enis: rev 54627ea6dc2c42fe0c67e34f69e42bd38386879e) * hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionManager.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/TableConfiguration.java Cache configuration and RpcController selection for Table in Connection --- Key: HBASE-12128 URL: https://issues.apache.org/jira/browse/HBASE-12128 Project: HBase Issue Type: Sub-task Affects Versions: 1.0.0, 2.0.0 Reporter: Andrew Purtell Assignee: Stephen Yuan Jiang Fix For: 2.0.0, 0.99.2 Attachments: HBASE-12128.v1-1.0.patch, HBASE-12128.v1-2.0.patch, HBASE-12128.v2-2.0.patch Original Estimate: 120h Time Spent: 72h Remaining Estimate: 48h Creating Table instances should be lightweight. Apps that manage their own Connections are expected to create Tables on demand for each interaction. However we look up values from Hadoop Configuration when constructing Table objects for storing to some of its fields. Configuration is a heavyweight registry that does a lot of string operations and regex matching. Method calls into Configuration account for 48.25% of CPU time when creating the HTable object in 0.98. Another ~48% of CPU is spent constructing the desired RpcController object via reflection in 0.98. Together this can account for ~20% of total on-CPU time of the client. See parent issue for more detail. We are using Connection like a factory for Table. We should cache configuration for Table in Connection. We should also create by reflection once and cache the desired RpcController object, and clone it for new Tables. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12128) Cache configuration and RpcController selection for Table in Connection
[ https://issues.apache.org/jira/browse/HBASE-12128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14225408#comment-14225408 ] Hudson commented on HBASE-12128: FAILURE: Integrated in HBase-1.0 #503 (See [https://builds.apache.org/job/HBase-1.0/503/]) HBASE-12128 Cache configuration and RpcController selection for Table in Connection (enis: rev b9dfcd01b8248cecdd8fc06632c738fe863af823) * hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/TableConfiguration.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionManager.java Cache configuration and RpcController selection for Table in Connection --- Key: HBASE-12128 URL: https://issues.apache.org/jira/browse/HBASE-12128 Project: HBase Issue Type: Sub-task Affects Versions: 1.0.0, 2.0.0 Reporter: Andrew Purtell Assignee: Stephen Yuan Jiang Fix For: 2.0.0, 0.99.2 Attachments: HBASE-12128.v1-1.0.patch, HBASE-12128.v1-2.0.patch, HBASE-12128.v2-2.0.patch Original Estimate: 120h Time Spent: 72h Remaining Estimate: 48h Creating Table instances should be lightweight. Apps that manage their own Connections are expected to create Tables on demand for each interaction. However we look up values from Hadoop Configuration when constructing Table objects for storing to some of its fields. Configuration is a heavyweight registry that does a lot of string operations and regex matching. Method calls into Configuration account for 48.25% of CPU time when creating the HTable object in 0.98. Another ~48% of CPU is spent constructing the desired RpcController object via reflection in 0.98. Together this can account for ~20% of total on-CPU time of the client. See parent issue for more detail. We are using Connection like a factory for Table. We should cache configuration for Table in Connection. We should also create by reflection once and cache the desired RpcController object, and clone it for new Tables. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12128) Cache configuration and RpcController selection for Table in Connection
[
https://issues.apache.org/jira/browse/HBASE-12128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14223361#comment-14223361
]
Hadoop QA commented on HBASE-12128:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12683210/HBASE-12128.v1-2.0.patch
against master branch at commit 0df5ed2ca6ce3758b4745f63400fd81d17107038.
ATTACHMENT ID: 12683210
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:red}-1 checkstyle{color}. The applied patch generated
3807 checkstyle errors (more than the master's current 3806 errors).
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.TestInterfaceAudienceAnnotations
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/11815//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11815//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11815//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11815//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11815//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11815//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11815//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11815//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11815//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11815//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11815//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11815//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/11815//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/11815//console
This message is automatically generated.
Cache configuration and RpcController selection for Table in Connection
---
Key: HBASE-12128
URL: https://issues.apache.org/jira/browse/HBASE-12128
Project: HBase
Issue Type: Sub-task
Affects Versions: 2.0.0
Reporter: Andrew Purtell
Assignee: Stephen Yuan Jiang
Fix For: 1.0.0, 2.0.0, 0.98.9, 0.99.2
Attachments: HBASE-12128.v1-2.0.patch
Original Estimate: 120h
Time Spent: 72h
Remaining Estimate: 48h
Creating Table instances should be lightweight. Apps that manage their own
Connections are expected to create Tables on demand for each interaction.
However we look up values from Hadoop Configuration when constructing Table
objects for storing to some of its fields. Configuration is a heavyweight
registry that does a lot of string operations and regex matching. Method
calls into Configuration account for 48.25% of CPU time when creating the
HTable object in 0.98. Another ~48% of CPU is
[jira] [Commented] (HBASE-12128) Cache configuration and RpcController selection for Table in Connection
[ https://issues.apache.org/jira/browse/HBASE-12128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14223406#comment-14223406 ] Enis Soztutar commented on HBASE-12128: --- This looks good. TestInterfaceAudienceAnnotations is a new test that checks whether every public class has a InterfaceAudience annotation. I think we should make TableConfiguration a package-protected class so that users do not use it. Also we can annotate it with InterfaceAudience.Private as well. I was thinking about the TableConfiguration and the rpc factories. One alternative approach we can take is to change TableConfiguration to be a TableContext, and have the context own both the configuration and rpc factories (and possibly other table related concept). The patch as it is will do the job. Just a suggestion, it is up to you which one is better. Cache configuration and RpcController selection for Table in Connection --- Key: HBASE-12128 URL: https://issues.apache.org/jira/browse/HBASE-12128 Project: HBase Issue Type: Sub-task Affects Versions: 2.0.0 Reporter: Andrew Purtell Assignee: Stephen Yuan Jiang Fix For: 1.0.0, 2.0.0, 0.98.9, 0.99.2 Attachments: HBASE-12128.v1-2.0.patch Original Estimate: 120h Time Spent: 72h Remaining Estimate: 48h Creating Table instances should be lightweight. Apps that manage their own Connections are expected to create Tables on demand for each interaction. However we look up values from Hadoop Configuration when constructing Table objects for storing to some of its fields. Configuration is a heavyweight registry that does a lot of string operations and regex matching. Method calls into Configuration account for 48.25% of CPU time when creating the HTable object in 0.98. Another ~48% of CPU is spent constructing the desired RpcController object via reflection in 0.98. Together this can account for ~20% of total on-CPU time of the client. See parent issue for more detail. We are using Connection like a factory for Table. We should cache configuration for Table in Connection. We should also create by reflection once and cache the desired RpcController object, and clone it for new Tables. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12128) Cache configuration and RpcController selection for Table in Connection
[ https://issues.apache.org/jira/browse/HBASE-12128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14223471#comment-14223471 ] stack commented on HBASE-12128: --- Will this work? It caches a tableconfiguration on a Connection instance but a Connection can be used to go against many tables. Should your rather be caching the tableconfiguration in a map keyed by tablename on a Connection? Cache configuration and RpcController selection for Table in Connection --- Key: HBASE-12128 URL: https://issues.apache.org/jira/browse/HBASE-12128 Project: HBase Issue Type: Sub-task Affects Versions: 2.0.0 Reporter: Andrew Purtell Assignee: Stephen Yuan Jiang Fix For: 1.0.0, 2.0.0, 0.98.9, 0.99.2 Attachments: HBASE-12128.v1-2.0.patch Original Estimate: 120h Time Spent: 72h Remaining Estimate: 48h Creating Table instances should be lightweight. Apps that manage their own Connections are expected to create Tables on demand for each interaction. However we look up values from Hadoop Configuration when constructing Table objects for storing to some of its fields. Configuration is a heavyweight registry that does a lot of string operations and regex matching. Method calls into Configuration account for 48.25% of CPU time when creating the HTable object in 0.98. Another ~48% of CPU is spent constructing the desired RpcController object via reflection in 0.98. Together this can account for ~20% of total on-CPU time of the client. See parent issue for more detail. We are using Connection like a factory for Table. We should cache configuration for Table in Connection. We should also create by reflection once and cache the desired RpcController object, and clone it for new Tables. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12128) Cache configuration and RpcController selection for Table in Connection
[ https://issues.apache.org/jira/browse/HBASE-12128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14223523#comment-14223523 ] Stephen Yuan Jiang commented on HBASE-12128: [~stack] why not working? The table configuration is the same values that we retrieve for all tables in the connection. If it would be updated by the table instance of a connection, the updated values will stay only with the table; other tables from the Connection instance will not be affected. If using a map keyed by tablename on a Connection, then we have to go through configuration registry again for the same value for a different table. It reduce the improvement from cache. Cache configuration and RpcController selection for Table in Connection --- Key: HBASE-12128 URL: https://issues.apache.org/jira/browse/HBASE-12128 Project: HBase Issue Type: Sub-task Affects Versions: 2.0.0 Reporter: Andrew Purtell Assignee: Stephen Yuan Jiang Fix For: 1.0.0, 2.0.0, 0.98.9, 0.99.2 Attachments: HBASE-12128.v1-2.0.patch Original Estimate: 120h Time Spent: 72h Remaining Estimate: 48h Creating Table instances should be lightweight. Apps that manage their own Connections are expected to create Tables on demand for each interaction. However we look up values from Hadoop Configuration when constructing Table objects for storing to some of its fields. Configuration is a heavyweight registry that does a lot of string operations and regex matching. Method calls into Configuration account for 48.25% of CPU time when creating the HTable object in 0.98. Another ~48% of CPU is spent constructing the desired RpcController object via reflection in 0.98. Together this can account for ~20% of total on-CPU time of the client. See parent issue for more detail. We are using Connection like a factory for Table. We should cache configuration for Table in Connection. We should also create by reflection once and cache the desired RpcController object, and clone it for new Tables. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12128) Cache configuration and RpcController selection for Table in Connection
[
https://issues.apache.org/jira/browse/HBASE-12128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14223743#comment-14223743
]
Hadoop QA commented on HBASE-12128:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12683418/HBASE-12128.v2-2.0.patch
against master branch at commit 7893c013bc4902a5b0e4ea4371aa9232df968578.
ATTACHMENT ID: 12683418
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/11816//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11816//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11816//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11816//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11816//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11816//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11816//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11816//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11816//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11816//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11816//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11816//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/11816//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/11816//console
This message is automatically generated.
Cache configuration and RpcController selection for Table in Connection
---
Key: HBASE-12128
URL: https://issues.apache.org/jira/browse/HBASE-12128
Project: HBase
Issue Type: Sub-task
Affects Versions: 2.0.0
Reporter: Andrew Purtell
Assignee: Stephen Yuan Jiang
Fix For: 1.0.0, 2.0.0, 0.98.9, 0.99.2
Attachments: HBASE-12128.v1-2.0.patch, HBASE-12128.v2-2.0.patch
Original Estimate: 120h
Time Spent: 72h
Remaining Estimate: 48h
Creating Table instances should be lightweight. Apps that manage their own
Connections are expected to create Tables on demand for each interaction.
However we look up values from Hadoop Configuration when constructing Table
objects for storing to some of its fields. Configuration is a heavyweight
registry that does a lot of string operations and regex matching. Method
calls into Configuration account for 48.25% of CPU time when creating the
HTable object in 0.98. Another ~48% of CPU is spent constructing the desired
RpcController object via reflection in 0.98. Together this can
[jira] [Commented] (HBASE-12128) Cache configuration and RpcController selection for Table in Connection
[ https://issues.apache.org/jira/browse/HBASE-12128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14223759#comment-14223759 ] stack commented on HBASE-12128: --- I see. Then suggest renaming the configuration 'cache'; call it ConfigurationSnapshot or CachedConfiguration? And why not have Admin and RegionLocators also use this cached Configuration? Thanks for working on this. Cache configuration and RpcController selection for Table in Connection --- Key: HBASE-12128 URL: https://issues.apache.org/jira/browse/HBASE-12128 Project: HBase Issue Type: Sub-task Affects Versions: 2.0.0 Reporter: Andrew Purtell Assignee: Stephen Yuan Jiang Fix For: 1.0.0, 2.0.0, 0.98.9, 0.99.2 Attachments: HBASE-12128.v1-2.0.patch, HBASE-12128.v2-2.0.patch Original Estimate: 120h Time Spent: 72h Remaining Estimate: 48h Creating Table instances should be lightweight. Apps that manage their own Connections are expected to create Tables on demand for each interaction. However we look up values from Hadoop Configuration when constructing Table objects for storing to some of its fields. Configuration is a heavyweight registry that does a lot of string operations and regex matching. Method calls into Configuration account for 48.25% of CPU time when creating the HTable object in 0.98. Another ~48% of CPU is spent constructing the desired RpcController object via reflection in 0.98. Together this can account for ~20% of total on-CPU time of the client. See parent issue for more detail. We are using Connection like a factory for Table. We should cache configuration for Table in Connection. We should also create by reflection once and cache the desired RpcController object, and clone it for new Tables. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12128) Cache configuration and RpcController selection for Table in Connection
[ https://issues.apache.org/jira/browse/HBASE-12128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14223833#comment-14223833 ] Enis Soztutar commented on HBASE-12128: --- The configurations parsed from conf object is specific to HTable instance for now. So it is kind of tied to the HTable internals (write buffer size, etc). I think it is not a generic configuration cache or anything, just a way for faster HTable construction. Cache configuration and RpcController selection for Table in Connection --- Key: HBASE-12128 URL: https://issues.apache.org/jira/browse/HBASE-12128 Project: HBase Issue Type: Sub-task Affects Versions: 2.0.0 Reporter: Andrew Purtell Assignee: Stephen Yuan Jiang Fix For: 1.0.0, 2.0.0, 0.98.9, 0.99.2 Attachments: HBASE-12128.v1-2.0.patch, HBASE-12128.v2-2.0.patch Original Estimate: 120h Time Spent: 72h Remaining Estimate: 48h Creating Table instances should be lightweight. Apps that manage their own Connections are expected to create Tables on demand for each interaction. However we look up values from Hadoop Configuration when constructing Table objects for storing to some of its fields. Configuration is a heavyweight registry that does a lot of string operations and regex matching. Method calls into Configuration account for 48.25% of CPU time when creating the HTable object in 0.98. Another ~48% of CPU is spent constructing the desired RpcController object via reflection in 0.98. Together this can account for ~20% of total on-CPU time of the client. See parent issue for more detail. We are using Connection like a factory for Table. We should cache configuration for Table in Connection. We should also create by reflection once and cache the desired RpcController object, and clone it for new Tables. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12128) Cache configuration and RpcController selection for Table in Connection
[ https://issues.apache.org/jira/browse/HBASE-12128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14223871#comment-14223871 ] stack commented on HBASE-12128: --- bq. ... for now We don't foresee wanting to cache configs for Admin that we get off the same Connection any time soon? Looking in constructor for HBaseAdmin, it wants to use some of the configs you are caching in this patch (RegionLocator is currently an HTable instance so it will get the benefit this patch adds) Anyways, just a suggestion. +1 on the patch even as is. Cache configuration and RpcController selection for Table in Connection --- Key: HBASE-12128 URL: https://issues.apache.org/jira/browse/HBASE-12128 Project: HBase Issue Type: Sub-task Affects Versions: 2.0.0 Reporter: Andrew Purtell Assignee: Stephen Yuan Jiang Fix For: 1.0.0, 2.0.0, 0.98.9, 0.99.2 Attachments: HBASE-12128.v1-2.0.patch, HBASE-12128.v2-2.0.patch Original Estimate: 120h Time Spent: 72h Remaining Estimate: 48h Creating Table instances should be lightweight. Apps that manage their own Connections are expected to create Tables on demand for each interaction. However we look up values from Hadoop Configuration when constructing Table objects for storing to some of its fields. Configuration is a heavyweight registry that does a lot of string operations and regex matching. Method calls into Configuration account for 48.25% of CPU time when creating the HTable object in 0.98. Another ~48% of CPU is spent constructing the desired RpcController object via reflection in 0.98. Together this can account for ~20% of total on-CPU time of the client. See parent issue for more detail. We are using Connection like a factory for Table. We should cache configuration for Table in Connection. We should also create by reflection once and cache the desired RpcController object, and clone it for new Tables. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12128) Cache configuration and RpcController selection for Table in Connection
[ https://issues.apache.org/jira/browse/HBASE-12128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14224048#comment-14224048 ] Stephen Yuan Jiang commented on HBASE-12128: Patch for 1.0 release is attached. Cache configuration and RpcController selection for Table in Connection --- Key: HBASE-12128 URL: https://issues.apache.org/jira/browse/HBASE-12128 Project: HBase Issue Type: Sub-task Affects Versions: 1.0.0, 2.0.0 Reporter: Andrew Purtell Assignee: Stephen Yuan Jiang Fix For: 1.0.0, 2.0.0 Attachments: HBASE-12128.v1-1.0.patch, HBASE-12128.v1-2.0.patch, HBASE-12128.v2-2.0.patch Original Estimate: 120h Time Spent: 72h Remaining Estimate: 48h Creating Table instances should be lightweight. Apps that manage their own Connections are expected to create Tables on demand for each interaction. However we look up values from Hadoop Configuration when constructing Table objects for storing to some of its fields. Configuration is a heavyweight registry that does a lot of string operations and regex matching. Method calls into Configuration account for 48.25% of CPU time when creating the HTable object in 0.98. Another ~48% of CPU is spent constructing the desired RpcController object via reflection in 0.98. Together this can account for ~20% of total on-CPU time of the client. See parent issue for more detail. We are using Connection like a factory for Table. We should cache configuration for Table in Connection. We should also create by reflection once and cache the desired RpcController object, and clone it for new Tables. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12128) Cache configuration and RpcController selection for Table in Connection
[
https://issues.apache.org/jira/browse/HBASE-12128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14224118#comment-14224118
]
Hadoop QA commented on HBASE-12128:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12683479/HBASE-12128.v1-1.0.patch
against master branch at commit e83082a88816684714d8a563967046e582f9b8c7.
ATTACHMENT ID: 12683479
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/11822//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11822//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11822//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11822//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11822//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11822//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11822//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11822//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11822//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11822//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11822//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11822//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/11822//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/11822//console
This message is automatically generated.
Cache configuration and RpcController selection for Table in Connection
---
Key: HBASE-12128
URL: https://issues.apache.org/jira/browse/HBASE-12128
Project: HBase
Issue Type: Sub-task
Affects Versions: 1.0.0, 2.0.0
Reporter: Andrew Purtell
Assignee: Stephen Yuan Jiang
Fix For: 1.0.0, 2.0.0
Attachments: HBASE-12128.v1-1.0.patch, HBASE-12128.v1-2.0.patch,
HBASE-12128.v2-2.0.patch
Original Estimate: 120h
Time Spent: 72h
Remaining Estimate: 48h
Creating Table instances should be lightweight. Apps that manage their own
Connections are expected to create Tables on demand for each interaction.
However we look up values from Hadoop Configuration when constructing Table
objects for storing to some of its fields. Configuration is a heavyweight
registry that does a lot of string operations and regex matching. Method
calls into Configuration account for 48.25% of CPU time when creating the
HTable object in 0.98. Another ~48% of CPU is spent constructing the desired
RpcController object via reflection in 0.98.
[jira] [Commented] (HBASE-12128) Cache configuration and RpcController selection for Table in Connection
[ https://issues.apache.org/jira/browse/HBASE-12128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14222309#comment-14222309 ] Stephen Yuan Jiang commented on HBASE-12128: [~apurtell], [~enis], [~ndimiduk] Please review this patch. Cache configuration and RpcController selection for Table in Connection --- Key: HBASE-12128 URL: https://issues.apache.org/jira/browse/HBASE-12128 Project: HBase Issue Type: Sub-task Reporter: Andrew Purtell Assignee: Stephen Yuan Jiang Fix For: 2.0.0, 0.98.9, 0.99.2 Attachments: HBASE-12128.v1-2.0.patch Original Estimate: 120h Remaining Estimate: 120h Creating Table instances should be lightweight. Apps that manage their own Connections are expected to create Tables on demand for each interaction. However we look up values from Hadoop Configuration when constructing Table objects for storing to some of its fields. Configuration is a heavyweight registry that does a lot of string operations and regex matching. Method calls into Configuration account for 48.25% of CPU time when creating the HTable object in 0.98. Another ~48% of CPU is spent constructing the desired RpcController object via reflection in 0.98. Together this can account for ~20% of total on-CPU time of the client. See parent issue for more detail. We are using Connection like a factory for Table. We should cache configuration for Table in Connection. We should also create by reflection once and cache the desired RpcController object, and clone it for new Tables. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12128) Cache configuration and RpcController selection for Table in Connection
[ https://issues.apache.org/jira/browse/HBASE-12128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14168514#comment-14168514 ] Enis Soztutar commented on HBASE-12128: --- This should be relatively easy for the new style of Table creation. As suggested we can make a TableConfiguration object and instantiate once in Connection. All HTable's will get this in ctor. Cache configuration and RpcController selection for Table in Connection --- Key: HBASE-12128 URL: https://issues.apache.org/jira/browse/HBASE-12128 Project: HBase Issue Type: Sub-task Reporter: Andrew Purtell Fix For: 2.0.0, 0.98.8, 0.99.2 Creating Table instances should be lightweight. Apps that manage their own Connections are expected to create Tables on demand for each interaction. However we look up values from Hadoop Configuration when constructing Table objects for storing to some of its fields. Configuration is a heavyweight registry that does a lot of string operations and regex matching. Method calls into Configuration account for 48.25% of CPU time when creating the HTable object in 0.98. Another ~48% of CPU is spent constructing the desired RpcController object via reflection in 0.98. Together this can account for ~20% of total on-CPU time of the client. See parent issue for more detail. We are using Connection like a factory for Table. We should cache configuration for Table in Connection. We should also create by reflection once and cache the desired RpcController object, and clone it for new Tables. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
