[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow
[ https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15168370#comment-15168370 ] Heng Chen commented on HBASE-14703: --- Failed testcase due to HBASE-15222, has no relates with current patch. > not collect stats when call HTable.mutateRow > - > > Key: HBASE-14703 > URL: https://issues.apache.org/jira/browse/HBASE-14703 > Project: HBase > Issue Type: Bug >Reporter: Heng Chen >Assignee: Heng Chen > Fix For: 2.0.0 > > Attachments: HBASE-14702_v5.2_addendum-addendum.patch, > HBASE-14703-5.2-addendum.patch, HBASE-14703-async.patch, > HBASE-14703-start.patch, HBASE-14703-v4.1.patch, HBASE-14703-v4.patch, > HBASE-14703-v6_with-check-and-mutate.patch, HBASE-14703.patch, > HBASE-14703_v1.patch, HBASE-14703_v10.patch, HBASE-14703_v10.patch, > HBASE-14703_v11.patch, HBASE-14703_v12.patch, HBASE-14703_v13.patch, > HBASE-14703_v2.patch, HBASE-14703_v3.patch, HBASE-14703_v5.1.patch, > HBASE-14703_v5.2.patch, HBASE-14703_v5.patch, HBASE-14703_v6-addendum.patch, > HBASE-14703_v6.patch, HBASE-14703_v7.patch, HBASE-14703_v8.patch, > HBASE-14703_v9.patch > > > We are trying to fix the stats implementation, by moving it out of the Result > object and into an Rpc payload (but not the 'cell payload', just as part of > the values returned from the request). This change will also us use easily > switch to AsyncProcess as the executor, and support stats, for nearly all the > rpc calls. However, that means when you upgrade the client or server, you > will lose stats visibility until the other side is upgraded. We could keep > around the Result based stats storage to accommodate the old api and send > both stats back from the server (in each result and in the rpc payload). > Note that we will still be wire compatible - protobufs mean we can just ride > over the lack of information. > The other tricky part of this is that Result has a > non-InterfaceAudience.Private getStatistics() method (along with two > InterfaceAudience.Private addResults and setStatistics methods), so we might > need a release to deprecate the getStats() method before throwing it out? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow
[ https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15168349#comment-15168349 ] Hadoop QA commented on HBASE-14703: --- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s {color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 0s {color} | {color:green} Patch does not have any anti-patterns. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s {color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s {color} | {color:green} The patch appears to include 5 new or modified test files. {color} | | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 25s {color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m 19s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 15s {color} | {color:green} master passed with JDK v1.8.0_72 {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 18s {color} | {color:green} master passed with JDK v1.7.0_95 {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 11m 20s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 38s {color} | {color:green} master passed {color} | | {color:red}-1{color} | {color:red} hbaseprotoc {color} | {color:red} 0m 7s {color} | {color:red} hbase-protocol in the patch failed. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 50s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 54s {color} | {color:green} master passed with JDK v1.8.0_72 {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 3s {color} | {color:green} master passed with JDK v1.7.0_95 {color} | | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 23s {color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 36s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 16s {color} | {color:green} the patch passed with JDK v1.8.0_72 {color} | | {color:green}+1{color} | {color:green} cc {color} | {color:green} 1m 16s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 16s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 20s {color} | {color:green} the patch passed with JDK v1.7.0_95 {color} | | {color:green}+1{color} | {color:green} cc {color} | {color:green} 1m 20s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 20s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 11m 14s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 37s {color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 1s {color} | {color:red} The patch has 17 line(s) that end in whitespace. Use git apply --whitespace=fix. {color} | | {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 26m 38s {color} | {color:green} Patch does not cause any errors with Hadoop 2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} | | {color:red}-1{color} | {color:red} hbaseprotoc {color} | {color:red} 0m 7s {color} | {color:red} hbase-protocol in the patch failed. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m 34s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 51s {color} | {color:green} the patch passed with JDK v1.8.0_72 {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 1s {color} | {color:green} the patch passed with JDK v1.7.0_95 {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 22s {color} | {color:green} hbase-protocol in the patch passed with JDK v1.8.0_72. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 50s {color} | {color:green}
[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow
[ https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15167541#comment-15167541 ] Hadoop QA commented on HBASE-14703: --- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s {color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 0s {color} | {color:green} Patch does not have any anti-patterns. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s {color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s {color} | {color:green} The patch appears to include 5 new or modified test files. {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m 43s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 12s {color} | {color:green} master passed with JDK v1.8.0_72 {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 20s {color} | {color:green} master passed with JDK v1.7.0_95 {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 10m 3s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 34s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m 46s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 33s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 50s {color} | {color:green} master passed with JDK v1.8.0_72 {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 1s {color} | {color:green} master passed with JDK v1.7.0_95 {color} | | {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m 16s {color} | {color:red} hbase-client in the patch failed. {color} | | {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m 27s {color} | {color:red} hbase-server in the patch failed. {color} | | {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m 29s {color} | {color:red} hbase-server in the patch failed with JDK v1.8.0_72. {color} | | {color:red}-1{color} | {color:red} cc {color} | {color:red} 0m 29s {color} | {color:red} hbase-server in the patch failed with JDK v1.8.0_72. {color} | | {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 29s {color} | {color:red} hbase-server in the patch failed with JDK v1.8.0_72. {color} | | {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m 27s {color} | {color:red} hbase-server in the patch failed with JDK v1.7.0_95. {color} | | {color:red}-1{color} | {color:red} cc {color} | {color:red} 0m 27s {color} | {color:red} hbase-server in the patch failed with JDK v1.7.0_95. {color} | | {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 27s {color} | {color:red} hbase-server in the patch failed with JDK v1.7.0_95. {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 9m 39s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 37s {color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s {color} | {color:red} The patch has 17 line(s) that end in whitespace. Use git apply --whitespace=fix. {color} | | {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 24m 52s {color} | {color:green} Patch does not cause any errors with Hadoop 2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} | | {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m 46s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 44s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 49s {color} | {color:green} the patch passed with JDK v1.8.0_72 {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 59s {color} | {color:green} the patch passed with JDK v1.7.0_95 {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 48s {color} | {color:green} hbase-client in the patch passed with JDK v1.8.0_72. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 20s {color} | {color:green} hbase-protocol in the patch
[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow
[ https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15163430#comment-15163430 ] Jesse Yates commented on HBASE-14703: - Mind taking a look [~chenheng]? > not collect stats when call HTable.mutateRow > - > > Key: HBASE-14703 > URL: https://issues.apache.org/jira/browse/HBASE-14703 > Project: HBase > Issue Type: Bug >Reporter: Heng Chen >Assignee: Heng Chen > Fix For: 2.0.0 > > Attachments: HBASE-14702_v5.2_addendum-addendum.patch, > HBASE-14703-5.2-addendum.patch, HBASE-14703-async.patch, > HBASE-14703-start.patch, HBASE-14703-v4.1.patch, HBASE-14703-v4.patch, > HBASE-14703-v6_with-check-and-mutate.patch, HBASE-14703.patch, > HBASE-14703_v1.patch, HBASE-14703_v10.patch, HBASE-14703_v10.patch, > HBASE-14703_v11.patch, HBASE-14703_v12.patch, HBASE-14703_v2.patch, > HBASE-14703_v3.patch, HBASE-14703_v5.1.patch, HBASE-14703_v5.2.patch, > HBASE-14703_v5.patch, HBASE-14703_v6-addendum.patch, HBASE-14703_v6.patch, > HBASE-14703_v7.patch, HBASE-14703_v8.patch, HBASE-14703_v9.patch > > > We are trying to fix the stats implementation, by moving it out of the Result > object and into an Rpc payload (but not the 'cell payload', just as part of > the values returned from the request). This change will also us use easily > switch to AsyncProcess as the executor, and support stats, for nearly all the > rpc calls. However, that means when you upgrade the client or server, you > will lose stats visibility until the other side is upgraded. We could keep > around the Result based stats storage to accommodate the old api and send > both stats back from the server (in each result and in the rpc payload). > Note that we will still be wire compatible - protobufs mean we can just ride > over the lack of information. > The other tricky part of this is that Result has a > non-InterfaceAudience.Private getStatistics() method (along with two > InterfaceAudience.Private addResults and setStatistics methods), so we might > need a release to deprecate the getStats() method before throwing it out? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow
[ https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15163293#comment-15163293 ] Hadoop QA commented on HBASE-14703: --- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s {color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 0s {color} | {color:green} Patch does not have any anti-patterns. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s {color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s {color} | {color:green} The patch appears to include 5 new or modified test files. {color} | | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 28s {color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m 55s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 15s {color} | {color:green} master passed with JDK v1.8.0_72 {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 10s {color} | {color:green} master passed with JDK v1.7.0_95 {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 9m 32s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 36s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m 47s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 38s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 1s {color} | {color:green} master passed with JDK v1.8.0_72 {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 4s {color} | {color:green} master passed with JDK v1.7.0_95 {color} | | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 22s {color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 31s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 24s {color} | {color:green} the patch passed with JDK v1.8.0_72 {color} | | {color:green}+1{color} | {color:green} cc {color} | {color:green} 1m 24s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 24s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 13s {color} | {color:green} the patch passed with JDK v1.7.0_95 {color} | | {color:green}+1{color} | {color:green} cc {color} | {color:green} 1m 13s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 13s {color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m 51s {color} | {color:red} Patch generated 2 new checkstyle issues in hbase-client (total was 340, now 307). {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 38s {color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 1s {color} | {color:red} The patch has 17 line(s) that end in whitespace. Use git apply --whitespace=fix. {color} | | {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 23m 53s {color} | {color:green} Patch does not cause any errors with Hadoop 2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} | | {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m 44s {color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 9s {color} | {color:red} hbase-client introduced 1 new FindBugs issues. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 52s {color} | {color:green} the patch passed with JDK v1.8.0_72 {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 1s {color} | {color:green} the patch passed with JDK v1.7.0_95 {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 22s {color} | {color:green} hbase-protocol in the patch passed with JDK v1.8.0_72. {color} | | {color:green}+1{color} | {color:green} unit
[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow
[ https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15154147#comment-15154147 ] Hadoop QA commented on HBASE-14703: --- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s {color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 0s {color} | {color:green} Patch does not have any anti-patterns. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s {color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s {color} | {color:green} The patch appears to include 5 new or modified test files. {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m 54s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 14s {color} | {color:green} master passed with JDK v1.8.0_72 {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 18s {color} | {color:green} master passed with JDK v1.7.0_95 {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 10m 17s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 39s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m 54s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 50s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 58s {color} | {color:green} master passed with JDK v1.8.0_72 {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 10s {color} | {color:green} master passed with JDK v1.7.0_95 {color} | | {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m 18s {color} | {color:red} hbase-client in the patch failed. {color} | | {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m 29s {color} | {color:red} hbase-server in the patch failed. {color} | | {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m 25s {color} | {color:red} hbase-server in the patch failed with JDK v1.8.0_72. {color} | | {color:red}-1{color} | {color:red} cc {color} | {color:red} 0m 25s {color} | {color:red} hbase-server in the patch failed with JDK v1.8.0_72. {color} | | {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 25s {color} | {color:red} hbase-server in the patch failed with JDK v1.8.0_72. {color} | | {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m 29s {color} | {color:red} hbase-server in the patch failed with JDK v1.7.0_95. {color} | | {color:red}-1{color} | {color:red} cc {color} | {color:red} 0m 29s {color} | {color:red} hbase-server in the patch failed with JDK v1.7.0_95. {color} | | {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 29s {color} | {color:red} hbase-server in the patch failed with JDK v1.7.0_95. {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 2m 3s {color} | {color:red} Patch generated 2 new checkstyle issues in hbase-client (total was 340, now 307). {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 39s {color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s {color} | {color:red} The patch has 17 line(s) that end in whitespace. Use git apply --whitespace=fix. {color} | | {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 26m 51s {color} | {color:green} Patch does not cause any errors with Hadoop 2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} | | {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m 53s {color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 13s {color} | {color:red} hbase-client introduced 1 new FindBugs issues. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 52s {color} | {color:green} the patch passed with JDK v1.8.0_72 {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 5s {color} | {color:green} the patch passed with JDK v1.7.0_95 {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 53s {color} | {color:green} hbase-client in the patch passed with JDK v1.8.0_72. {color} | | {color:green}+1{color} | {color:green} unit {col
[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow
[ https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15153602#comment-15153602 ] Heng Chen commented on HBASE-14703: --- Welcome back, i will rebase it today, :) > not collect stats when call HTable.mutateRow > - > > Key: HBASE-14703 > URL: https://issues.apache.org/jira/browse/HBASE-14703 > Project: HBase > Issue Type: Bug >Reporter: Heng Chen >Assignee: Heng Chen > Fix For: 2.0.0 > > Attachments: HBASE-14702_v5.2_addendum-addendum.patch, > HBASE-14703-5.2-addendum.patch, HBASE-14703-async.patch, > HBASE-14703-start.patch, HBASE-14703-v4.1.patch, HBASE-14703-v4.patch, > HBASE-14703-v6_with-check-and-mutate.patch, HBASE-14703.patch, > HBASE-14703_v1.patch, HBASE-14703_v10.patch, HBASE-14703_v10.patch, > HBASE-14703_v2.patch, HBASE-14703_v3.patch, HBASE-14703_v5.1.patch, > HBASE-14703_v5.2.patch, HBASE-14703_v5.patch, HBASE-14703_v6-addendum.patch, > HBASE-14703_v6.patch, HBASE-14703_v7.patch, HBASE-14703_v8.patch, > HBASE-14703_v9.patch > > > We are trying to fix the stats implementation, by moving it out of the Result > object and into an Rpc payload (but not the 'cell payload', just as part of > the values returned from the request). This change will also us use easily > switch to AsyncProcess as the executor, and support stats, for nearly all the > rpc calls. However, that means when you upgrade the client or server, you > will lose stats visibility until the other side is upgraded. We could keep > around the Result based stats storage to accommodate the old api and send > both stats back from the server (in each result and in the rpc payload). > Note that we will still be wire compatible - protobufs mean we can just ride > over the lack of information. > The other tricky part of this is that Result has a > non-InterfaceAudience.Private getStatistics() method (along with two > InterfaceAudience.Private addResults and setStatistics methods), so we might > need a release to deprecate the getStats() method before throwing it out? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow
[ https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15153591#comment-15153591 ] Jesse Yates commented on HBASE-14703: - Sorry [~chenheng] - fell off my queue. Thanks for the reminder. Promise I will look at it this weekend. > not collect stats when call HTable.mutateRow > - > > Key: HBASE-14703 > URL: https://issues.apache.org/jira/browse/HBASE-14703 > Project: HBase > Issue Type: Bug >Reporter: Heng Chen >Assignee: Heng Chen > Fix For: 2.0.0 > > Attachments: HBASE-14702_v5.2_addendum-addendum.patch, > HBASE-14703-5.2-addendum.patch, HBASE-14703-async.patch, > HBASE-14703-start.patch, HBASE-14703-v4.1.patch, HBASE-14703-v4.patch, > HBASE-14703-v6_with-check-and-mutate.patch, HBASE-14703.patch, > HBASE-14703_v1.patch, HBASE-14703_v10.patch, HBASE-14703_v10.patch, > HBASE-14703_v2.patch, HBASE-14703_v3.patch, HBASE-14703_v5.1.patch, > HBASE-14703_v5.2.patch, HBASE-14703_v5.patch, HBASE-14703_v6-addendum.patch, > HBASE-14703_v6.patch, HBASE-14703_v7.patch, HBASE-14703_v8.patch, > HBASE-14703_v9.patch > > > We are trying to fix the stats implementation, by moving it out of the Result > object and into an Rpc payload (but not the 'cell payload', just as part of > the values returned from the request). This change will also us use easily > switch to AsyncProcess as the executor, and support stats, for nearly all the > rpc calls. However, that means when you upgrade the client or server, you > will lose stats visibility until the other side is upgraded. We could keep > around the Result based stats storage to accommodate the old api and send > both stats back from the server (in each result and in the rpc payload). > Note that we will still be wire compatible - protobufs mean we can just ride > over the lack of information. > The other tricky part of this is that Result has a > non-InterfaceAudience.Private getStatistics() method (along with two > InterfaceAudience.Private addResults and setStatistics methods), so we might > need a release to deprecate the getStats() method before throwing it out? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow
[ https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15151885#comment-15151885 ] Heng Chen commented on HBASE-14703: --- [~jesse_yates] It is pending for some days, shall we go on with it ? > not collect stats when call HTable.mutateRow > - > > Key: HBASE-14703 > URL: https://issues.apache.org/jira/browse/HBASE-14703 > Project: HBase > Issue Type: Bug >Reporter: Heng Chen >Assignee: Heng Chen > Fix For: 2.0.0 > > Attachments: HBASE-14702_v5.2_addendum-addendum.patch, > HBASE-14703-5.2-addendum.patch, HBASE-14703-async.patch, > HBASE-14703-start.patch, HBASE-14703-v4.1.patch, HBASE-14703-v4.patch, > HBASE-14703-v6_with-check-and-mutate.patch, HBASE-14703.patch, > HBASE-14703_v1.patch, HBASE-14703_v10.patch, HBASE-14703_v10.patch, > HBASE-14703_v2.patch, HBASE-14703_v3.patch, HBASE-14703_v5.1.patch, > HBASE-14703_v5.2.patch, HBASE-14703_v5.patch, HBASE-14703_v6-addendum.patch, > HBASE-14703_v6.patch, HBASE-14703_v7.patch, HBASE-14703_v8.patch, > HBASE-14703_v9.patch > > > We are trying to fix the stats implementation, by moving it out of the Result > object and into an Rpc payload (but not the 'cell payload', just as part of > the values returned from the request). This change will also us use easily > switch to AsyncProcess as the executor, and support stats, for nearly all the > rpc calls. However, that means when you upgrade the client or server, you > will lose stats visibility until the other side is upgraded. We could keep > around the Result based stats storage to accommodate the old api and send > both stats back from the server (in each result and in the rpc payload). > Note that we will still be wire compatible - protobufs mean we can just ride > over the lack of information. > The other tricky part of this is that Result has a > non-InterfaceAudience.Private getStatistics() method (along with two > InterfaceAudience.Private addResults and setStatistics methods), so we might > need a release to deprecate the getStats() method before throwing it out? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow
[ https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15061268#comment-15061268 ] Heng Chen commented on HBASE-14703: --- [~apurtell] [~stack] Do you have time to help me review patch v10. I also upload it to review board. Thanks. > not collect stats when call HTable.mutateRow > - > > Key: HBASE-14703 > URL: https://issues.apache.org/jira/browse/HBASE-14703 > Project: HBase > Issue Type: Bug >Reporter: Heng Chen >Assignee: Heng Chen > Fix For: 2.0.0 > > Attachments: HBASE-14702_v5.2_addendum-addendum.patch, > HBASE-14703-5.2-addendum.patch, HBASE-14703-async.patch, > HBASE-14703-start.patch, HBASE-14703-v4.1.patch, HBASE-14703-v4.patch, > HBASE-14703-v6_with-check-and-mutate.patch, HBASE-14703.patch, > HBASE-14703_v1.patch, HBASE-14703_v10.patch, HBASE-14703_v10.patch, > HBASE-14703_v2.patch, HBASE-14703_v3.patch, HBASE-14703_v5.1.patch, > HBASE-14703_v5.2.patch, HBASE-14703_v5.patch, HBASE-14703_v6-addendum.patch, > HBASE-14703_v6.patch, HBASE-14703_v7.patch, HBASE-14703_v8.patch, > HBASE-14703_v9.patch > > > We are trying to fix the stats implementation, by moving it out of the Result > object and into an Rpc payload (but not the 'cell payload', just as part of > the values returned from the request). This change will also us use easily > switch to AsyncProcess as the executor, and support stats, for nearly all the > rpc calls. However, that means when you upgrade the client or server, you > will lose stats visibility until the other side is upgraded. We could keep > around the Result based stats storage to accommodate the old api and send > both stats back from the server (in each result and in the rpc payload). > Note that we will still be wire compatible - protobufs mean we can just ride > over the lack of information. > The other tricky part of this is that Result has a > non-InterfaceAudience.Private getStatistics() method (along with two > InterfaceAudience.Private addResults and setStatistics methods), so we might > need a release to deprecate the getStats() method before throwing it out? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow
[ https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15061264#comment-15061264 ] Heng Chen commented on HBASE-14703: --- OK, Thanks [~jesse_yates] for your help during this issue. Let me update the description first to make other guys understand this issue easily, the outline you made above is good for it. Thanks again. > not collect stats when call HTable.mutateRow > - > > Key: HBASE-14703 > URL: https://issues.apache.org/jira/browse/HBASE-14703 > Project: HBase > Issue Type: Bug >Reporter: Heng Chen >Assignee: Heng Chen > Fix For: 2.0.0 > > Attachments: HBASE-14702_v5.2_addendum-addendum.patch, > HBASE-14703-5.2-addendum.patch, HBASE-14703-async.patch, > HBASE-14703-start.patch, HBASE-14703-v4.1.patch, HBASE-14703-v4.patch, > HBASE-14703-v6_with-check-and-mutate.patch, HBASE-14703.patch, > HBASE-14703_v1.patch, HBASE-14703_v10.patch, HBASE-14703_v10.patch, > HBASE-14703_v2.patch, HBASE-14703_v3.patch, HBASE-14703_v5.1.patch, > HBASE-14703_v5.2.patch, HBASE-14703_v5.patch, HBASE-14703_v6-addendum.patch, > HBASE-14703_v6.patch, HBASE-14703_v7.patch, HBASE-14703_v8.patch, > HBASE-14703_v9.patch > > > In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update > serverStatistics twice. > The first one is that we wrapper {{RetryingCallable}} by > {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we > call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below: > {code} > @Override > public T callWithRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > @Override > public T callWithoutRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > {code} > The secondary one is after we get response, in {{receiveMultiAction}}, we do > update again. > {code} > // update the stats about the region, if its a user table. We don't want to > slow down > // updates to meta tables, especially from internal updates (master, etc). > if (AsyncProcess.this.connection.getStatisticsTracker() != null) { > result = ResultStatsUtil.updateStats(result, > AsyncProcess.this.connection.getStatisticsTracker(), server, regionName); > } > {code} > It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary, remove it? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow
[ https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15061245#comment-15061245 ] Jesse Yates commented on HBASE-14703: - [~chenheng] I think making the changes form RB puts this patch into the committable zone. Maybe fix the checkstyle issues? However, I've got some family stuff going on at the moment (why I haven't been responsive recently). Maybe [~apurtell] or [~stack] might be able to help push this over the goal line? I should be able to be back to working come the new year, if you can wait that long. > not collect stats when call HTable.mutateRow > - > > Key: HBASE-14703 > URL: https://issues.apache.org/jira/browse/HBASE-14703 > Project: HBase > Issue Type: Bug >Reporter: Heng Chen >Assignee: Heng Chen > Fix For: 2.0.0 > > Attachments: HBASE-14702_v5.2_addendum-addendum.patch, > HBASE-14703-5.2-addendum.patch, HBASE-14703-async.patch, > HBASE-14703-start.patch, HBASE-14703-v4.1.patch, HBASE-14703-v4.patch, > HBASE-14703-v6_with-check-and-mutate.patch, HBASE-14703.patch, > HBASE-14703_v1.patch, HBASE-14703_v10.patch, HBASE-14703_v10.patch, > HBASE-14703_v2.patch, HBASE-14703_v3.patch, HBASE-14703_v5.1.patch, > HBASE-14703_v5.2.patch, HBASE-14703_v5.patch, HBASE-14703_v6-addendum.patch, > HBASE-14703_v6.patch, HBASE-14703_v7.patch, HBASE-14703_v8.patch, > HBASE-14703_v9.patch > > > In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update > serverStatistics twice. > The first one is that we wrapper {{RetryingCallable}} by > {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we > call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below: > {code} > @Override > public T callWithRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > @Override > public T callWithoutRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > {code} > The secondary one is after we get response, in {{receiveMultiAction}}, we do > update again. > {code} > // update the stats about the region, if its a user table. We don't want to > slow down > // updates to meta tables, especially from internal updates (master, etc). > if (AsyncProcess.this.connection.getStatisticsTracker() != null) { > result = ResultStatsUtil.updateStats(result, > AsyncProcess.this.connection.getStatisticsTracker(), server, regionName); > } > {code} > It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary, remove it? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow
[ https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15046616#comment-15046616 ] Hadoop QA commented on HBASE-14703: --- {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12776245/HBASE-14703_v10.patch against master branch at commit 1f999c1e2bba62fda0fb426a168afa338b31c251. ATTACHMENT ID: 12776245 {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 20 new or modified tests. {color:green}+1 hadoop versions{color}. The patch compiles with all supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0 2.7.1) {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 protoc{color}. The applied patch does not increase the total number of protoc compiler warnings. {color:green}+1 javadoc{color}. The javadoc tool did not generate any warning messages. {color:red}-1 checkstyle{color}. The applied patch generated new checkstyle errors. Check build console for list of new errors. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 2.0.3) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:green}+1 lineLengths{color}. The patch does not introduce lines longer than 100 {color:green}+1 site{color}. The mvn post-site goal succeeds with this patch. {color:green}+1 core tests{color}. The patch passed unit tests in . {color:green}+1 zombies{color}. No zombie tests found running at the end of the build. Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/16790//testReport/ Release Findbugs (version 2.0.3)warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/16790//artifact/patchprocess/newFindbugsWarnings.html Checkstyle Errors: https://builds.apache.org/job/PreCommit-HBASE-Build/16790//artifact/patchprocess/checkstyle-aggregate.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/16790//console This message is automatically generated. > not collect stats when call HTable.mutateRow > - > > Key: HBASE-14703 > URL: https://issues.apache.org/jira/browse/HBASE-14703 > Project: HBase > Issue Type: Bug >Reporter: Heng Chen >Assignee: Heng Chen > Fix For: 2.0.0 > > Attachments: HBASE-14702_v5.2_addendum-addendum.patch, > HBASE-14703-5.2-addendum.patch, HBASE-14703-async.patch, > HBASE-14703-start.patch, HBASE-14703-v4.1.patch, HBASE-14703-v4.patch, > HBASE-14703-v6_with-check-and-mutate.patch, HBASE-14703.patch, > HBASE-14703_v1.patch, HBASE-14703_v10.patch, HBASE-14703_v10.patch, > HBASE-14703_v2.patch, HBASE-14703_v3.patch, HBASE-14703_v5.1.patch, > HBASE-14703_v5.2.patch, HBASE-14703_v5.patch, HBASE-14703_v6-addendum.patch, > HBASE-14703_v6.patch, HBASE-14703_v7.patch, HBASE-14703_v8.patch, > HBASE-14703_v9.patch > > > In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update > serverStatistics twice. > The first one is that we wrapper {{RetryingCallable}} by > {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we > call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below: > {code} > @Override > public T callWithRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > @Override > public T callWithoutRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > {code} > The secondary one is after we get response, in {{receiveMultiAction}}, we do > update again. > {code} > // update the stats about the region, if its a user table. We don't want to > slow down > // updates to meta tables, especially from internal updates (master, etc). > if (AsyncProcess.this.connection.getStatisticsTracker() != null) { > result = ResultStatsUtil.updateStats(result, > AsyncProcess.this.connection.getStatisticsTracker(), server, regionName); > } > {code} > It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary, remove it? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow
[ https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15034461#comment-15034461 ] Jesse Yates commented on HBASE-14703: - Didn't really need to, but now that it is, lets rev from RB > not collect stats when call HTable.mutateRow > - > > Key: HBASE-14703 > URL: https://issues.apache.org/jira/browse/HBASE-14703 > Project: HBase > Issue Type: Bug >Reporter: Heng Chen >Assignee: Heng Chen > Fix For: 2.0.0 > > Attachments: HBASE-14702_v5.2_addendum-addendum.patch, > HBASE-14703-5.2-addendum.patch, HBASE-14703-async.patch, > HBASE-14703-start.patch, HBASE-14703-v4.1.patch, HBASE-14703-v4.patch, > HBASE-14703-v6_with-check-and-mutate.patch, HBASE-14703.patch, > HBASE-14703_v1.patch, HBASE-14703_v2.patch, HBASE-14703_v3.patch, > HBASE-14703_v5.1.patch, HBASE-14703_v5.2.patch, HBASE-14703_v5.patch, > HBASE-14703_v6-addendum.patch, HBASE-14703_v6.patch, HBASE-14703_v7.patch, > HBASE-14703_v8.patch, HBASE-14703_v9.patch > > > In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update > serverStatistics twice. > The first one is that we wrapper {{RetryingCallable}} by > {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we > call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below: > {code} > @Override > public T callWithRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > @Override > public T callWithoutRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > {code} > The secondary one is after we get response, in {{receiveMultiAction}}, we do > update again. > {code} > // update the stats about the region, if its a user table. We don't want to > slow down > // updates to meta tables, especially from internal updates (master, etc). > if (AsyncProcess.this.connection.getStatisticsTracker() != null) { > result = ResultStatsUtil.updateStats(result, > AsyncProcess.this.connection.getStatisticsTracker(), server, regionName); > } > {code} > It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary, remove it? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow
[ https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15029448#comment-15029448 ] Heng Chen commented on HBASE-14703: --- How about patch_v9? Do i need to upload it to RB? [~jesse_yates] > not collect stats when call HTable.mutateRow > - > > Key: HBASE-14703 > URL: https://issues.apache.org/jira/browse/HBASE-14703 > Project: HBase > Issue Type: Bug >Reporter: Heng Chen >Assignee: Heng Chen > Fix For: 2.0.0 > > Attachments: HBASE-14702_v5.2_addendum-addendum.patch, > HBASE-14703-5.2-addendum.patch, HBASE-14703-async.patch, > HBASE-14703-start.patch, HBASE-14703-v4.1.patch, HBASE-14703-v4.patch, > HBASE-14703-v6_with-check-and-mutate.patch, HBASE-14703.patch, > HBASE-14703_v1.patch, HBASE-14703_v2.patch, HBASE-14703_v3.patch, > HBASE-14703_v5.1.patch, HBASE-14703_v5.2.patch, HBASE-14703_v5.patch, > HBASE-14703_v6-addendum.patch, HBASE-14703_v6.patch, HBASE-14703_v7.patch, > HBASE-14703_v8.patch, HBASE-14703_v9.patch > > > In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update > serverStatistics twice. > The first one is that we wrapper {{RetryingCallable}} by > {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we > call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below: > {code} > @Override > public T callWithRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > @Override > public T callWithoutRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > {code} > The secondary one is after we get response, in {{receiveMultiAction}}, we do > update again. > {code} > // update the stats about the region, if its a user table. We don't want to > slow down > // updates to meta tables, especially from internal updates (master, etc). > if (AsyncProcess.this.connection.getStatisticsTracker() != null) { > result = ResultStatsUtil.updateStats(result, > AsyncProcess.this.connection.getStatisticsTracker(), server, regionName); > } > {code} > It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary, remove it? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow
[ https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15028545#comment-15028545 ] Hadoop QA commented on HBASE-14703: --- {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12774479/HBASE-14703_v9.patch against master branch at commit 0d72849581c1e5a50a2aa7757b909e5531b8d12c. ATTACHMENT ID: 12774479 {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 20 new or modified tests. {color:green}+1 hadoop versions{color}. The patch compiles with all supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0 2.7.1) {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 protoc{color}. The applied patch does not increase the total number of protoc compiler warnings. {color:green}+1 javadoc{color}. The javadoc tool did not generate any warning messages. {color:red}-1 checkstyle{color}. The applied patch generated new checkstyle errors. Check build console for list of new errors. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 2.0.3) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:green}+1 lineLengths{color}. The patch does not introduce lines longer than 100 {color:green}+1 site{color}. The mvn post-site goal succeeds with this patch. {color:green}+1 core tests{color}. The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/16678//testReport/ Release Findbugs (version 2.0.3)warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/16678//artifact/patchprocess/newFindbugsWarnings.html Checkstyle Errors: https://builds.apache.org/job/PreCommit-HBASE-Build/16678//artifact/patchprocess/checkstyle-aggregate.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/16678//console This message is automatically generated. > not collect stats when call HTable.mutateRow > - > > Key: HBASE-14703 > URL: https://issues.apache.org/jira/browse/HBASE-14703 > Project: HBase > Issue Type: Bug >Reporter: Heng Chen >Assignee: Heng Chen > Fix For: 2.0.0 > > Attachments: HBASE-14702_v5.2_addendum-addendum.patch, > HBASE-14703-5.2-addendum.patch, HBASE-14703-async.patch, > HBASE-14703-start.patch, HBASE-14703-v4.1.patch, HBASE-14703-v4.patch, > HBASE-14703-v6_with-check-and-mutate.patch, HBASE-14703.patch, > HBASE-14703_v1.patch, HBASE-14703_v2.patch, HBASE-14703_v3.patch, > HBASE-14703_v5.1.patch, HBASE-14703_v5.2.patch, HBASE-14703_v5.patch, > HBASE-14703_v6-addendum.patch, HBASE-14703_v6.patch, HBASE-14703_v7.patch, > HBASE-14703_v8.patch, HBASE-14703_v9.patch > > > In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update > serverStatistics twice. > The first one is that we wrapper {{RetryingCallable}} by > {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we > call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below: > {code} > @Override > public T callWithRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > @Override > public T callWithoutRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > {code} > The secondary one is after we get response, in {{receiveMultiAction}}, we do > update again. > {code} > // update the stats about the region, if its a user table. We don't want to > slow down > // updates to meta tables, especially from internal updates (master, etc). > if (AsyncProcess.this.connection.getStatisticsTracker() != null) { > result = ResultStatsUtil.updateStats(result, > AsyncProcess.this.connection.getStatisticsTracker(), server, regionName); > } > {code} > It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary, remove it? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow
[ https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15028296#comment-15028296 ] Hadoop QA commented on HBASE-14703: --- {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12774461/HBASE-14703_v8.patch against master branch at commit e73a9594c218ed969a2f5b0b356d7b8d0e1474c0. ATTACHMENT ID: 12774461 {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 20 new or modified tests. {color:green}+1 hadoop versions{color}. The patch compiles with all supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0 2.7.1) {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 protoc{color}. The applied patch does not increase the total number of protoc compiler warnings. {color:green}+1 javadoc{color}. The javadoc tool did not generate any warning messages. {color:red}-1 checkstyle{color}. The applied patch generated new checkstyle errors. Check build console for list of new errors. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 2.0.3) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:red}-1 lineLengths{color}. The patch introduces the following lines longer than 100: +protected RpcRetryingCaller createCaller(PayloadCarryingServerCallable callable) { + * configuration flag, so they don't have to use it, even if its sent. This flag is only useful + * configuration flag, so they don't have to use it, even if its sent. This flag is only useful + * configuration flag, so they don't have to use it, even if its sent. This flag is only useful + * configuration flag, so they don't have to use it, even if its sent. This flag is only useful + new java.lang.String[] { "MemstoreLoad", "HeapOccupancy", "CompactionPressure", "Enabled", }); {color:green}+1 site{color}. The mvn post-site goal succeeds with this patch. {color:green}+1 core tests{color}. The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/16675//testReport/ Release Findbugs (version 2.0.3)warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/16675//artifact/patchprocess/newFindbugsWarnings.html Checkstyle Errors: https://builds.apache.org/job/PreCommit-HBASE-Build/16675//artifact/patchprocess/checkstyle-aggregate.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/16675//console This message is automatically generated. > not collect stats when call HTable.mutateRow > - > > Key: HBASE-14703 > URL: https://issues.apache.org/jira/browse/HBASE-14703 > Project: HBase > Issue Type: Bug >Reporter: Heng Chen >Assignee: Heng Chen > Fix For: 2.0.0 > > Attachments: HBASE-14702_v5.2_addendum-addendum.patch, > HBASE-14703-5.2-addendum.patch, HBASE-14703-async.patch, > HBASE-14703-start.patch, HBASE-14703-v4.1.patch, HBASE-14703-v4.patch, > HBASE-14703-v6_with-check-and-mutate.patch, HBASE-14703.patch, > HBASE-14703_v1.patch, HBASE-14703_v2.patch, HBASE-14703_v3.patch, > HBASE-14703_v5.1.patch, HBASE-14703_v5.2.patch, HBASE-14703_v5.patch, > HBASE-14703_v6-addendum.patch, HBASE-14703_v6.patch, HBASE-14703_v7.patch, > HBASE-14703_v8.patch > > > In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update > serverStatistics twice. > The first one is that we wrapper {{RetryingCallable}} by > {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we > call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below: > {code} > @Override > public T callWithRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > @Override > public T callWithoutRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > {code} > The secondary one is after we get response, in {{receiveMultiAction}}, we do > update again. > {code} > // update the stats about the region, if its a user table. We don't want to > slow down > // updates to meta tables, especially from internal updates (master, etc). > if (AsyncProcess.this.connec
[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow
[ https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15026991#comment-15026991 ] Jesse Yates commented on HBASE-14703: - I think that right now, its not the best solution. The cost comes not immediately, but suppose we don't finish changing the other interfaces. Then we have mutateRows and checkAndMutate being the only two cases where we use the new class and they way it looks is substantially different than every other method call. By keeping it simple, it makes it easier for people to understand what is happening as it is all written out and not hidden away. We should either be refactoring or changing behavior; when you combine both its hard to understand what the 'correct' solution is and makes it harder for others to grok the change. In this case, we are making a functionality change, so we shouldn't be too changing how things are organized (note PayloadCarryingCallable is a _functionality_ change as we needed a new base class to run the mutate callables). I think we should be trying for the smallest possible change to accomplish our goals. Is it so bad to just hold off on adding the new layer until we do the other methods as part of async? I bet we will actually figure out that we want a different heriarchy and mechanism for managing the callables once we look at it holistically, which will make the work here pointless. If I'm wrong, then we can just put the class from here and use it directly. Fair enough? > not collect stats when call HTable.mutateRow > - > > Key: HBASE-14703 > URL: https://issues.apache.org/jira/browse/HBASE-14703 > Project: HBase > Issue Type: Bug >Reporter: Heng Chen >Assignee: Heng Chen > Fix For: 2.0.0 > > Attachments: HBASE-14702_v5.2_addendum-addendum.patch, > HBASE-14703-5.2-addendum.patch, HBASE-14703-async.patch, > HBASE-14703-start.patch, HBASE-14703-v4.1.patch, HBASE-14703-v4.patch, > HBASE-14703-v6_with-check-and-mutate.patch, HBASE-14703.patch, > HBASE-14703_v1.patch, HBASE-14703_v2.patch, HBASE-14703_v3.patch, > HBASE-14703_v5.1.patch, HBASE-14703_v5.2.patch, HBASE-14703_v5.patch, > HBASE-14703_v6-addendum.patch, HBASE-14703_v6.patch, HBASE-14703_v7.patch > > > In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update > serverStatistics twice. > The first one is that we wrapper {{RetryingCallable}} by > {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we > call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below: > {code} > @Override > public T callWithRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > @Override > public T callWithoutRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > {code} > The secondary one is after we get response, in {{receiveMultiAction}}, we do > update again. > {code} > // update the stats about the region, if its a user table. We don't want to > slow down > // updates to meta tables, especially from internal updates (master, etc). > if (AsyncProcess.this.connection.getStatisticsTracker() != null) { > result = ResultStatsUtil.updateStats(result, > AsyncProcess.this.connection.getStatisticsTracker(), server, regionName); > } > {code} > It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary, remove it? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow
[ https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15026582#comment-15026582 ] Hadoop QA commented on HBASE-14703: --- {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12774272/HBASE-14703_v7.patch against master branch at commit 90bdb0dc7412f3ae8fc1e14ce033bf115c6332f3. ATTACHMENT ID: 12774272 {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 20 new or modified tests. {color:green}+1 hadoop versions{color}. The patch compiles with all supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0 2.7.1) {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 protoc{color}. The applied patch does not increase the total number of protoc compiler warnings. {color:green}+1 javadoc{color}. The javadoc tool did not generate any warning messages. {color:red}-1 checkstyle{color}. The applied patch generated 18763 checkstyle errors (more than the master's current 18685 errors). {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 2.0.3) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:red}-1 lineLengths{color}. The patch introduces the following lines longer than 100: +Map regionResults = results.get(regionName) == null ? null : results.get(regionName).result; +new AbstractMultiCallable(this.connection, this.tableName, rm.getRow(), this.rpcControllerFactory) { + MultiRequest.newBuilder().addRegionAction(regionMutationBuilder.build()).build(); +new AbstractMultiCallable(this.connection, this.tableName, row, this.rpcControllerFactory) { +HBaseProtos.CompareType compareType = HBaseProtos.CompareType.valueOf(compareOp.name()); +protected RpcRetryingCaller createCaller(PayloadCarryingServerCallable callable) { + private static ResultOrException getResultOrException(final ClientProtos.Result r, final int index){ +ClientProtos.ResultOrException.Builder resultOrExceptionOrBuilder = ClientProtos.ResultOrException.newBuilder(); +ClientProtos.ResultOrException.Builder resultOrExceptionOrBuilder = ClientProtos.ResultOrException.newBuilder(); +mutateRows(region, regionAction.getActionList(), cellScanner, regionActionResultBuilder); {color:green}+1 site{color}. The mvn post-site goal succeeds with this patch. {color:green}+1 core tests{color}. The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/16661//testReport/ Release Findbugs (version 2.0.3)warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/16661//artifact/patchprocess/newFindbugsWarnings.html Checkstyle Errors: https://builds.apache.org/job/PreCommit-HBASE-Build/16661//artifact/patchprocess/checkstyle-aggregate.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/16661//console This message is automatically generated. > not collect stats when call HTable.mutateRow > - > > Key: HBASE-14703 > URL: https://issues.apache.org/jira/browse/HBASE-14703 > Project: HBase > Issue Type: Bug >Reporter: Heng Chen >Assignee: Heng Chen > Fix For: 2.0.0 > > Attachments: HBASE-14702_v5.2_addendum-addendum.patch, > HBASE-14703-5.2-addendum.patch, HBASE-14703-async.patch, > HBASE-14703-start.patch, HBASE-14703-v4.1.patch, HBASE-14703-v4.patch, > HBASE-14703-v6_with-check-and-mutate.patch, HBASE-14703.patch, > HBASE-14703_v1.patch, HBASE-14703_v2.patch, HBASE-14703_v3.patch, > HBASE-14703_v5.1.patch, HBASE-14703_v5.2.patch, HBASE-14703_v5.patch, > HBASE-14703_v6-addendum.patch, HBASE-14703_v6.patch, HBASE-14703_v7.patch > > > In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update > serverStatistics twice. > The first one is that we wrapper {{RetryingCallable}} by > {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we > call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below: > {code} > @Override > public T callWithRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > @Override > public T callWithoutRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException
[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow
[ https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15026271#comment-15026271 ] Heng Chen commented on HBASE-14703: --- {quote} It makes the code change we are putting in bigger, but doesn't really add any more clarity {quote} I don't think so. Indeed, if we just want to fix this issue as title, it is easy and no need for this big change. But as you said, we should do it in right way. Currently logic is confused, there are two path to communicate with server and statistics information store in PB result, it is not reasonable. I think we should make it clear. After HBASE-14693, there are more statistics we need to collect, i think we should support an unify way to do it. wdyt? > not collect stats when call HTable.mutateRow > - > > Key: HBASE-14703 > URL: https://issues.apache.org/jira/browse/HBASE-14703 > Project: HBase > Issue Type: Bug >Reporter: Heng Chen >Assignee: Heng Chen > Fix For: 2.0.0 > > Attachments: HBASE-14702_v5.2_addendum-addendum.patch, > HBASE-14703-5.2-addendum.patch, HBASE-14703-async.patch, > HBASE-14703-start.patch, HBASE-14703-v4.1.patch, HBASE-14703-v4.patch, > HBASE-14703-v6_with-check-and-mutate.patch, HBASE-14703.patch, > HBASE-14703_v1.patch, HBASE-14703_v2.patch, HBASE-14703_v3.patch, > HBASE-14703_v5.1.patch, HBASE-14703_v5.2.patch, HBASE-14703_v5.patch, > HBASE-14703_v6-addendum.patch, HBASE-14703_v6.patch, HBASE-14703_v7.patch > > > In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update > serverStatistics twice. > The first one is that we wrapper {{RetryingCallable}} by > {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we > call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below: > {code} > @Override > public T callWithRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > @Override > public T callWithoutRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > {code} > The secondary one is after we get response, in {{receiveMultiAction}}, we do > update again. > {code} > // update the stats about the region, if its a user table. We don't want to > slow down > // updates to meta tables, especially from internal updates (master, etc). > if (AsyncProcess.this.connection.getStatisticsTracker() != null) { > result = ResultStatsUtil.updateStats(result, > AsyncProcess.this.connection.getStatisticsTracker(), server, regionName); > } > {code} > It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary, remove it? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow
[ https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15026207#comment-15026207 ] Jesse Yates commented on HBASE-14703: - That patch just seems a bit much for right now. It makes the code change we are putting in bigger, but doesn't really add any more clarity. How about coming back to that when we do the changes for the rest of the methods? I tend to shy away from adding heirarchy when its only being slightly used. WDYT? > not collect stats when call HTable.mutateRow > - > > Key: HBASE-14703 > URL: https://issues.apache.org/jira/browse/HBASE-14703 > Project: HBase > Issue Type: Bug >Reporter: Heng Chen >Assignee: Heng Chen > Fix For: 2.0.0 > > Attachments: HBASE-14702_v5.2_addendum-addendum.patch, > HBASE-14703-5.2-addendum.patch, HBASE-14703-async.patch, > HBASE-14703-start.patch, HBASE-14703-v4.1.patch, HBASE-14703-v4.patch, > HBASE-14703-v6_with-check-and-mutate.patch, HBASE-14703.patch, > HBASE-14703_v1.patch, HBASE-14703_v2.patch, HBASE-14703_v3.patch, > HBASE-14703_v5.1.patch, HBASE-14703_v5.2.patch, HBASE-14703_v5.patch, > HBASE-14703_v6-addendum.patch, HBASE-14703_v6.patch > > > In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update > serverStatistics twice. > The first one is that we wrapper {{RetryingCallable}} by > {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we > call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below: > {code} > @Override > public T callWithRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > @Override > public T callWithoutRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > {code} > The secondary one is after we get response, in {{receiveMultiAction}}, we do > update again. > {code} > // update the stats about the region, if its a user table. We don't want to > slow down > // updates to meta tables, especially from internal updates (master, etc). > if (AsyncProcess.this.connection.getStatisticsTracker() != null) { > result = ResultStatsUtil.updateStats(result, > AsyncProcess.this.connection.getStatisticsTracker(), server, regionName); > } > {code} > It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary, remove it? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow
[ https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15026202#comment-15026202 ] Jesse Yates commented on HBASE-14703: - Hey [~chenheng], can you attach that as a full patch? The way I find people using addendums is generally for two things: (1) bug fixes immediately after the fact, (2) helping to show changes. For instance, I used it previously to show how you go follow up, but its a large change and it would have been hard to keep track of how it fit onto the base patch, were it a single patch. For smaller changes, especially that aren't expected to be committed, a new patch and version is fine, and for really small changes - checkstyles, spelling, etc - doing point patches is ok (so a 7.1). The functional problem in particular here is that QA won't by able to handle addendums, so making it patch available doesn't actually run the test :( > not collect stats when call HTable.mutateRow > - > > Key: HBASE-14703 > URL: https://issues.apache.org/jira/browse/HBASE-14703 > Project: HBase > Issue Type: Bug >Reporter: Heng Chen >Assignee: Heng Chen > Fix For: 2.0.0 > > Attachments: HBASE-14702_v5.2_addendum-addendum.patch, > HBASE-14703-5.2-addendum.patch, HBASE-14703-async.patch, > HBASE-14703-start.patch, HBASE-14703-v4.1.patch, HBASE-14703-v4.patch, > HBASE-14703-v6_with-check-and-mutate.patch, HBASE-14703.patch, > HBASE-14703_v1.patch, HBASE-14703_v2.patch, HBASE-14703_v3.patch, > HBASE-14703_v5.1.patch, HBASE-14703_v5.2.patch, HBASE-14703_v5.patch, > HBASE-14703_v6-addendum.patch, HBASE-14703_v6.patch > > > In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update > serverStatistics twice. > The first one is that we wrapper {{RetryingCallable}} by > {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we > call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below: > {code} > @Override > public T callWithRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > @Override > public T callWithoutRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > {code} > The secondary one is after we get response, in {{receiveMultiAction}}, we do > update again. > {code} > // update the stats about the region, if its a user table. We don't want to > slow down > // updates to meta tables, especially from internal updates (master, etc). > if (AsyncProcess.this.connection.getStatisticsTracker() != null) { > result = ResultStatsUtil.updateStats(result, > AsyncProcess.this.connection.getStatisticsTracker(), server, regionName); > } > {code} > It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary, remove it? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow
[ https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15026145#comment-15026145 ] Heng Chen commented on HBASE-14703: --- Oh, i figure out why TestCheckAndMutate failed with patch_v6. It is because of here which you fix in HBASE-14703-v6_with-check-and-mutate.patch {code} - } catch(NoSuchColumnFamilyException e) { + } catch (RetriesExhaustedWithDetailsException e) { +try { + throw e.getCause(0); +} catch (NoSuchColumnFamilyException e1) { + // expected +} {code} So as your logic, if processed is false and no exception in ClientProtos.MultiResponse, there will no exception throw out (you remove it in ResponseConverter), and there is no retry. It is different with original logic (original logic will retry). IMO we should NOT retry in checkAndXXX if processed is false and just return false to users, but we need to confirm it. wdyt? I will extract a new callable class to avoid useless repeated code. The callable class will be useful in future when we unify other calls. btw. Takes advantage of the 'exists' flag in Result to track processed state is a good idea. Thanks [~jesse_yates] for your nice patch. > not collect stats when call HTable.mutateRow > - > > Key: HBASE-14703 > URL: https://issues.apache.org/jira/browse/HBASE-14703 > Project: HBase > Issue Type: Bug >Reporter: Heng Chen >Assignee: Heng Chen > Fix For: 2.0.0 > > Attachments: HBASE-14702_v5.2_addendum-addendum.patch, > HBASE-14703-5.2-addendum.patch, HBASE-14703-async.patch, > HBASE-14703-start.patch, HBASE-14703-v4.1.patch, HBASE-14703-v4.patch, > HBASE-14703-v6_with-check-and-mutate.patch, HBASE-14703.patch, > HBASE-14703_v1.patch, HBASE-14703_v2.patch, HBASE-14703_v3.patch, > HBASE-14703_v5.1.patch, HBASE-14703_v5.2.patch, HBASE-14703_v5.patch, > HBASE-14703_v6.patch > > > In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update > serverStatistics twice. > The first one is that we wrapper {{RetryingCallable}} by > {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we > call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below: > {code} > @Override > public T callWithRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > @Override > public T callWithoutRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > {code} > The secondary one is after we get response, in {{receiveMultiAction}}, we do > update again. > {code} > // update the stats about the region, if its a user table. We don't want to > slow down > // updates to meta tables, especially from internal updates (master, etc). > if (AsyncProcess.this.connection.getStatisticsTracker() != null) { > result = ResultStatsUtil.updateStats(result, > AsyncProcess.this.connection.getStatisticsTracker(), server, regionName); > } > {code} > It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary, remove it? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow
[ https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15025063#comment-15025063 ] Hadoop QA commented on HBASE-14703: --- {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12774115/HBASE-14703-v6_with-check-and-mutate.patch against master branch at commit 4a60c25c702a57d043ca3fd3a9996f9fe63f9343. ATTACHMENT ID: 12774115 {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 3 new or modified tests. {color:red}-1 patch{color}. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/16655//console This message is automatically generated. > not collect stats when call HTable.mutateRow > - > > Key: HBASE-14703 > URL: https://issues.apache.org/jira/browse/HBASE-14703 > Project: HBase > Issue Type: Bug >Reporter: Heng Chen >Assignee: Heng Chen > Fix For: 2.0.0 > > Attachments: HBASE-14702_v5.2_addendum-addendum.patch, > HBASE-14703-5.2-addendum.patch, HBASE-14703-async.patch, > HBASE-14703-start.patch, HBASE-14703-v4.1.patch, HBASE-14703-v4.patch, > HBASE-14703-v6_with-check-and-mutate.patch, HBASE-14703.patch, > HBASE-14703_v1.patch, HBASE-14703_v2.patch, HBASE-14703_v3.patch, > HBASE-14703_v5.1.patch, HBASE-14703_v5.2.patch, HBASE-14703_v5.patch, > HBASE-14703_v6.patch > > > In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update > serverStatistics twice. > The first one is that we wrapper {{RetryingCallable}} by > {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we > call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below: > {code} > @Override > public T callWithRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > @Override > public T callWithoutRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > {code} > The secondary one is after we get response, in {{receiveMultiAction}}, we do > update again. > {code} > // update the stats about the region, if its a user table. We don't want to > slow down > // updates to meta tables, especially from internal updates (master, etc). > if (AsyncProcess.this.connection.getStatisticsTracker() != null) { > result = ResultStatsUtil.updateStats(result, > AsyncProcess.this.connection.getStatisticsTracker(), server, regionName); > } > {code} > It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary, remove it? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow
[ https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15025023#comment-15025023 ] Jesse Yates commented on HBASE-14703: - With the amount of copied code, we will definitely want to consider making a new callable class to that does a lot of the work, but that's for a future patch. FWIW, this wasn't too complicated once I figured out how manage the processed state, which is a testament to this new style. > not collect stats when call HTable.mutateRow > - > > Key: HBASE-14703 > URL: https://issues.apache.org/jira/browse/HBASE-14703 > Project: HBase > Issue Type: Bug >Reporter: Heng Chen >Assignee: Heng Chen > Fix For: 2.0.0 > > Attachments: HBASE-14702_v5.2_addendum-addendum.patch, > HBASE-14703-5.2-addendum.patch, HBASE-14703-async.patch, > HBASE-14703-start.patch, HBASE-14703-v4.1.patch, HBASE-14703-v4.patch, > HBASE-14703-v6_with-check-and-mutate.patch, HBASE-14703.patch, > HBASE-14703_v1.patch, HBASE-14703_v2.patch, HBASE-14703_v3.patch, > HBASE-14703_v5.1.patch, HBASE-14703_v5.2.patch, HBASE-14703_v5.patch, > HBASE-14703_v6.patch > > > In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update > serverStatistics twice. > The first one is that we wrapper {{RetryingCallable}} by > {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we > call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below: > {code} > @Override > public T callWithRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > @Override > public T callWithoutRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > {code} > The secondary one is after we get response, in {{receiveMultiAction}}, we do > update again. > {code} > // update the stats about the region, if its a user table. We don't want to > slow down > // updates to meta tables, especially from internal updates (master, etc). > if (AsyncProcess.this.connection.getStatisticsTracker() != null) { > result = ResultStatsUtil.updateStats(result, > AsyncProcess.this.connection.getStatisticsTracker(), server, regionName); > } > {code} > It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary, remove it? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow
[ https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15023407#comment-15023407 ] Heng Chen commented on HBASE-14703: --- {quote} What do you mean? Its an optional field, so if its not there on the wire, PB will just ignore it (which is why PB generates #hasLoadStats() method). {quote} In PB, if we remove an field entirely, future users maybe reuse this tag number, which will cause some problems. {quote} ClientProtos.MultiResponse (the PB) or MultiResponse (o.a.h.hbase.client class)? {quote} MultiResponse (o.a.h.hbase.client class). {quote} I think by marking it as an EMPTY_RESULT we are implicitly saying 'processed'. The only other option for that value is as an exception, right? {quote} yeah, when i pass results[] in AP.submit, if exception happens, it will failed. That's why TestCheckAndMutate failed i mentioned above. > not collect stats when call HTable.mutateRow > - > > Key: HBASE-14703 > URL: https://issues.apache.org/jira/browse/HBASE-14703 > Project: HBase > Issue Type: Bug >Reporter: Heng Chen >Assignee: Heng Chen > Fix For: 2.0.0 > > Attachments: HBASE-14702_v5.2_addendum-addendum.patch, > HBASE-14703-5.2-addendum.patch, HBASE-14703-async.patch, > HBASE-14703-start.patch, HBASE-14703-v4.1.patch, HBASE-14703-v4.patch, > HBASE-14703.patch, HBASE-14703_v1.patch, HBASE-14703_v2.patch, > HBASE-14703_v3.patch, HBASE-14703_v5.1.patch, HBASE-14703_v5.2.patch, > HBASE-14703_v5.patch, HBASE-14703_v6.patch > > > In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update > serverStatistics twice. > The first one is that we wrapper {{RetryingCallable}} by > {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we > call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below: > {code} > @Override > public T callWithRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > @Override > public T callWithoutRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > {code} > The secondary one is after we get response, in {{receiveMultiAction}}, we do > update again. > {code} > // update the stats about the region, if its a user table. We don't want to > slow down > // updates to meta tables, especially from internal updates (master, etc). > if (AsyncProcess.this.connection.getStatisticsTracker() != null) { > result = ResultStatsUtil.updateStats(result, > AsyncProcess.this.connection.getStatisticsTracker(), server, regionName); > } > {code} > It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary, remove it? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow
[ https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15022668#comment-15022668 ] Jesse Yates commented on HBASE-14703: - {quote} optional bool enabled = 4 [default = false]; Excuse me, what's the purpose for this option. {quote} Its just come cruft left in my example addendum from when I was trying to figure out how to better implement stats. Unless you use it anywhere - I didn't see it - lets pull it out. bq. But not remove RegionLoadStats in ResultOrException for PB parser not crashed when user upgrade cluster. What do you mean? Its an optional field, so if its not there on the wire, PB will just ignore it (which is why PB generates #hasLoadStats() method). bq. Maybe we should add process flag back into MultiResponse. wdyt? ClientProtos.MultiResponse (the PB) or MultiResponse (o.a.h.hbase.client class)? I think by marking it as an EMPTY_RESULT we are implicitly saying 'processed'. The only other option for that value is as an exception, right? > not collect stats when call HTable.mutateRow > - > > Key: HBASE-14703 > URL: https://issues.apache.org/jira/browse/HBASE-14703 > Project: HBase > Issue Type: Bug >Reporter: Heng Chen >Assignee: Heng Chen > Fix For: 2.0.0 > > Attachments: HBASE-14702_v5.2_addendum-addendum.patch, > HBASE-14703-5.2-addendum.patch, HBASE-14703-async.patch, > HBASE-14703-start.patch, HBASE-14703-v4.1.patch, HBASE-14703-v4.patch, > HBASE-14703.patch, HBASE-14703_v1.patch, HBASE-14703_v2.patch, > HBASE-14703_v3.patch, HBASE-14703_v5.1.patch, HBASE-14703_v5.2.patch, > HBASE-14703_v5.patch, HBASE-14703_v6.patch > > > In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update > serverStatistics twice. > The first one is that we wrapper {{RetryingCallable}} by > {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we > call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below: > {code} > @Override > public T callWithRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > @Override > public T callWithoutRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > {code} > The secondary one is after we get response, in {{receiveMultiAction}}, we do > update again. > {code} > // update the stats about the region, if its a user table. We don't want to > slow down > // updates to meta tables, especially from internal updates (master, etc). > if (AsyncProcess.this.connection.getStatisticsTracker() != null) { > result = ResultStatsUtil.updateStats(result, > AsyncProcess.this.connection.getStatisticsTracker(), server, regionName); > } > {code} > It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary, remove it? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow
[ https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15022589#comment-15022589 ] Jesse Yates commented on HBASE-14703: - Maybe [~apurtell] can give us some advice on the compatibility story here (I forget :-/). I think we can just do the fix in master as that cuts a new major release. Worst case is you lose stats until both sides of the wire are upgraded. We would also remove the public Result#getStats() method as that again becomes an internal value, but I'm not sure if we can do that in a major release. Further, I don't know how valuable it is to backport except some of the cleanup from removing mutateRows statistics tracking and the statsTrackingRpcController (since they never gets used anyways). TL;DR on the JIRA so far: We are trying to fix the stats implementation, by moving it out of the Result object and into an Rpc payload (but not the 'cell payload', just as part of the values returned from the request). This change will also us use easily switch to AsyncProcess as the executor, and support stats, for nearly all the rpc calls. However, that means when you upgrade the client or server, you will lose stats visibility until the other side is upgraded. We could keep around the Result based stats storage to accommodate the old api and send both stats back from the server (in each result and in the rpc payload). Note that we will still be wire compatible - protobufs mean we can just ride over the lack of information. The other tricky part of this is that Result has a non-InterfaceAudience.Private getStatistics() method (along with two InterfaceAudience.Private addResults and setStatistics methods), so we might need a release to deprecate the getStats() method before throwing it out(?). > not collect stats when call HTable.mutateRow > - > > Key: HBASE-14703 > URL: https://issues.apache.org/jira/browse/HBASE-14703 > Project: HBase > Issue Type: Bug >Reporter: Heng Chen >Assignee: Heng Chen > Fix For: 2.0.0 > > Attachments: HBASE-14702_v5.2_addendum-addendum.patch, > HBASE-14703-5.2-addendum.patch, HBASE-14703-async.patch, > HBASE-14703-start.patch, HBASE-14703-v4.1.patch, HBASE-14703-v4.patch, > HBASE-14703.patch, HBASE-14703_v1.patch, HBASE-14703_v2.patch, > HBASE-14703_v3.patch, HBASE-14703_v5.1.patch, HBASE-14703_v5.2.patch, > HBASE-14703_v5.patch, HBASE-14703_v6.patch > > > In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update > serverStatistics twice. > The first one is that we wrapper {{RetryingCallable}} by > {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we > call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below: > {code} > @Override > public T callWithRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > @Override > public T callWithoutRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > {code} > The secondary one is after we get response, in {{receiveMultiAction}}, we do > update again. > {code} > // update the stats about the region, if its a user table. We don't want to > slow down > // updates to meta tables, especially from internal updates (master, etc). > if (AsyncProcess.this.connection.getStatisticsTracker() != null) { > result = ResultStatsUtil.updateStats(result, > AsyncProcess.this.connection.getStatisticsTracker(), server, regionName); > } > {code} > It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary, remove it? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow
[ https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15021571#comment-15021571 ] Hadoop QA commented on HBASE-14703: --- {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12773483/HBASE-14703_v6.patch against master branch at commit 55087ce8887b5be38b0fda0dda3fbf2f92c13778. ATTACHMENT ID: 12773483 {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 16 new or modified tests. {color:green}+1 hadoop versions{color}. The patch compiles with all supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0 2.7.1) {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 protoc{color}. The applied patch does not increase the total number of protoc compiler warnings. {color:green}+1 javadoc{color}. The javadoc tool did not generate any warning messages. {color:red}-1 checkstyle{color}. The applied patch generated 18707 checkstyle errors (more than the master's current 18686 errors). {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 2.0.3) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:red}-1 lineLengths{color}. The patch introduces the following lines longer than 100: +Map regionResults = results.get(regionName) == null ? null : results.get(regionName).result; +protected RpcRetryingCaller createCaller(PayloadCarryingServerCallable callable) { + * configuration flag, so they don't have to use it, even if its sent. This flag is only useful + * configuration flag, so they don't have to use it, even if its sent. This flag is only useful + * configuration flag, so they don't have to use it, even if its sent. This flag is only useful + * configuration flag, so they don't have to use it, even if its sent. This flag is only useful + new java.lang.String[] { "MemstoreLoad", "HeapOccupancy", "CompactionPressure", "Enabled", }); + private static ResultOrException getResultOrException(final ClientProtos.Result r, final int index){ +ClientProtos.ResultOrException.Builder resultOrExceptionOrBuilder = ClientProtos.ResultOrException.newBuilder(); +ClientProtos.ResultOrException.Builder resultOrExceptionOrBuilder = ClientProtos.ResultOrException.newBuilder(); {color:green}+1 site{color}. The mvn post-site goal succeeds with this patch. {color:green}+1 core tests{color}. The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/16634//testReport/ Release Findbugs (version 2.0.3)warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/16634//artifact/patchprocess/newFindbugsWarnings.html Checkstyle Errors: https://builds.apache.org/job/PreCommit-HBASE-Build/16634//artifact/patchprocess/checkstyle-aggregate.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/16634//console This message is automatically generated. > not collect stats when call HTable.mutateRow > - > > Key: HBASE-14703 > URL: https://issues.apache.org/jira/browse/HBASE-14703 > Project: HBase > Issue Type: Bug >Reporter: Heng Chen >Assignee: Heng Chen > Fix For: 2.0.0 > > Attachments: HBASE-14702_v5.2_addendum-addendum.patch, > HBASE-14703-5.2-addendum.patch, HBASE-14703-async.patch, > HBASE-14703-start.patch, HBASE-14703-v4.1.patch, HBASE-14703-v4.patch, > HBASE-14703.patch, HBASE-14703_v1.patch, HBASE-14703_v2.patch, > HBASE-14703_v3.patch, HBASE-14703_v5.1.patch, HBASE-14703_v5.2.patch, > HBASE-14703_v5.patch, HBASE-14703_v6.patch > > > In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update > serverStatistics twice. > The first one is that we wrapper {{RetryingCallable}} by > {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we > call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below: > {code} > @Override > public T callWithRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > @Override > public T callWithoutRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, call
[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow
[ https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15015515#comment-15015515 ] Heng Chen commented on HBASE-14703: --- Update a patch based on your last patch, and i create a diff base on your patch. * Remove some unused code. But not remove RegionLoadStats in ResultOrException for PB parser not crashed when user upgrade cluster. * Unify RS multi response for mutateRow and other calls which use multi interface currently (put/puts/gets). So we can unify different process path in AP. I try to unify checkAndMutate and AP, but failed. The reason is i have no idea how to get processed flag if not add one option in MultiResponse. I try to pass results[] into AP.submit, but found it can't pass test case TestCheckAndMutate. Maybe we should add process flag back into MultiResponse. wdyt? > not collect stats when call HTable.mutateRow > - > > Key: HBASE-14703 > URL: https://issues.apache.org/jira/browse/HBASE-14703 > Project: HBase > Issue Type: Bug >Reporter: Heng Chen >Assignee: Heng Chen > Attachments: HBASE-14702_v5.2_addendum-addendum.patch, > HBASE-14703-5.2-addendum.patch, HBASE-14703-async.patch, > HBASE-14703-start.patch, HBASE-14703-v4.1.patch, HBASE-14703-v4.patch, > HBASE-14703.patch, HBASE-14703_v1.patch, HBASE-14703_v2.patch, > HBASE-14703_v3.patch, HBASE-14703_v5.1.patch, HBASE-14703_v5.2.patch, > HBASE-14703_v5.patch > > > In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update > serverStatistics twice. > The first one is that we wrapper {{RetryingCallable}} by > {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we > call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below: > {code} > @Override > public T callWithRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > @Override > public T callWithoutRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > {code} > The secondary one is after we get response, in {{receiveMultiAction}}, we do > update again. > {code} > // update the stats about the region, if its a user table. We don't want to > slow down > // updates to meta tables, especially from internal updates (master, etc). > if (AsyncProcess.this.connection.getStatisticsTracker() != null) { > result = ResultStatsUtil.updateStats(result, > AsyncProcess.this.connection.getStatisticsTracker(), server, regionName); > } > {code} > It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary, remove it? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow
[ https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15015187#comment-15015187 ] Heng Chen commented on HBASE-14703: --- {code} + // if the stats are enabled. By the default above being 0, even if a statistic is sent back, + // there is no change to the local behavior because there is no load. Futher, even if the + // server sends back statistics, its up to the client to have the tracker enabled via a + // configuration flag, so they don't have to use it, even if its sent. This flag is only useful + // for understanding the + optional bool enabled = 4 [default = false]; {code} Excuse me, what's the purpose for this option. > not collect stats when call HTable.mutateRow > - > > Key: HBASE-14703 > URL: https://issues.apache.org/jira/browse/HBASE-14703 > Project: HBase > Issue Type: Bug >Reporter: Heng Chen >Assignee: Heng Chen > Attachments: HBASE-14703-5.2-addendum.patch, HBASE-14703-async.patch, > HBASE-14703-start.patch, HBASE-14703-v4.1.patch, HBASE-14703-v4.patch, > HBASE-14703.patch, HBASE-14703_v1.patch, HBASE-14703_v2.patch, > HBASE-14703_v3.patch, HBASE-14703_v5.1.patch, HBASE-14703_v5.2.patch, > HBASE-14703_v5.patch > > > In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update > serverStatistics twice. > The first one is that we wrapper {{RetryingCallable}} by > {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we > call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below: > {code} > @Override > public T callWithRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > @Override > public T callWithoutRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > {code} > The secondary one is after we get response, in {{receiveMultiAction}}, we do > update again. > {code} > // update the stats about the region, if its a user table. We don't want to > slow down > // updates to meta tables, especially from internal updates (master, etc). > if (AsyncProcess.this.connection.getStatisticsTracker() != null) { > result = ResultStatsUtil.updateStats(result, > AsyncProcess.this.connection.getStatisticsTracker(), server, regionName); > } > {code} > It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary, remove it? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow
[ https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15015125#comment-15015125 ] Heng Chen commented on HBASE-14703: --- {quote} What do you mean by this? {quote} What i mean is after this issue, the DataStructure for statistics will be ready. And AP will be suitable to accept other calls. {quote} If you are agreeing with the direction I was suggesting, do you want to crack at a patch for this one to do the cleanup? {quote} Of course, i will go on today. > not collect stats when call HTable.mutateRow > - > > Key: HBASE-14703 > URL: https://issues.apache.org/jira/browse/HBASE-14703 > Project: HBase > Issue Type: Bug >Reporter: Heng Chen >Assignee: Heng Chen > Attachments: HBASE-14703-5.2-addendum.patch, HBASE-14703-async.patch, > HBASE-14703-start.patch, HBASE-14703-v4.1.patch, HBASE-14703-v4.patch, > HBASE-14703.patch, HBASE-14703_v1.patch, HBASE-14703_v2.patch, > HBASE-14703_v3.patch, HBASE-14703_v5.1.patch, HBASE-14703_v5.2.patch, > HBASE-14703_v5.patch > > > In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update > serverStatistics twice. > The first one is that we wrapper {{RetryingCallable}} by > {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we > call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below: > {code} > @Override > public T callWithRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > @Override > public T callWithoutRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > {code} > The secondary one is after we get response, in {{receiveMultiAction}}, we do > update again. > {code} > // update the stats about the region, if its a user table. We don't want to > slow down > // updates to meta tables, especially from internal updates (master, etc). > if (AsyncProcess.this.connection.getStatisticsTracker() != null) { > result = ResultStatsUtil.updateStats(result, > AsyncProcess.this.connection.getStatisticsTracker(), server, regionName); > } > {code} > It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary, remove it? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow
[ https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15015093#comment-15015093 ] Jesse Yates commented on HBASE-14703: - bq. This issue is about mutateRow and its statistics. And after this issue, the framework will be build What do you mean by this? If you are agreeing with the direction I was suggesting, do you want to crack at a patch for this one to do the cleanup? I'll can start filling follow ups and pinging the dev list with the scope of changes and getting feedback. > not collect stats when call HTable.mutateRow > - > > Key: HBASE-14703 > URL: https://issues.apache.org/jira/browse/HBASE-14703 > Project: HBase > Issue Type: Bug >Reporter: Heng Chen >Assignee: Heng Chen > Attachments: HBASE-14703-5.2-addendum.patch, HBASE-14703-async.patch, > HBASE-14703-start.patch, HBASE-14703-v4.1.patch, HBASE-14703-v4.patch, > HBASE-14703.patch, HBASE-14703_v1.patch, HBASE-14703_v2.patch, > HBASE-14703_v3.patch, HBASE-14703_v5.1.patch, HBASE-14703_v5.2.patch, > HBASE-14703_v5.patch > > > In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update > serverStatistics twice. > The first one is that we wrapper {{RetryingCallable}} by > {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we > call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below: > {code} > @Override > public T callWithRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > @Override > public T callWithoutRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > {code} > The secondary one is after we get response, in {{receiveMultiAction}}, we do > update again. > {code} > // update the stats about the region, if its a user table. We don't want to > slow down > // updates to meta tables, especially from internal updates (master, etc). > if (AsyncProcess.this.connection.getStatisticsTracker() != null) { > result = ResultStatsUtil.updateStats(result, > AsyncProcess.this.connection.getStatisticsTracker(), server, regionName); > } > {code} > It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary, remove it? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow
[ https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15015076#comment-15015076 ] Heng Chen commented on HBASE-14703: --- {quote} How about we make this patch the cleanup of "stuff that doesn't work". {quote} Agreed! Currently, the logic in client is a little confused. Let's unify them and make it clear. {quote} Support stats tracking for all calls (Umbrella) Use async everywhere in HTable calls Move statistics to per-call unify statistic collection to an interface {quote} Sounds reasonable. Maybe we can split 'Use async everywhere in HTable calls' into more parts. Because each call need some debug and tests. This issue is about mutateRow and its statistics. And after this issue, the framework will be build. We can unify other calls more easily. wdyt? :) btw. It is cool! > not collect stats when call HTable.mutateRow > - > > Key: HBASE-14703 > URL: https://issues.apache.org/jira/browse/HBASE-14703 > Project: HBase > Issue Type: Bug >Reporter: Heng Chen >Assignee: Heng Chen > Attachments: HBASE-14703-5.2-addendum.patch, HBASE-14703-async.patch, > HBASE-14703-start.patch, HBASE-14703-v4.1.patch, HBASE-14703-v4.patch, > HBASE-14703.patch, HBASE-14703_v1.patch, HBASE-14703_v2.patch, > HBASE-14703_v3.patch, HBASE-14703_v5.1.patch, HBASE-14703_v5.2.patch, > HBASE-14703_v5.patch > > > In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update > serverStatistics twice. > The first one is that we wrapper {{RetryingCallable}} by > {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we > call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below: > {code} > @Override > public T callWithRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > @Override > public T callWithoutRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > {code} > The secondary one is after we get response, in {{receiveMultiAction}}, we do > update again. > {code} > // update the stats about the region, if its a user table. We don't want to > slow down > // updates to meta tables, especially from internal updates (master, etc). > if (AsyncProcess.this.connection.getStatisticsTracker() != null) { > result = ResultStatsUtil.updateStats(result, > AsyncProcess.this.connection.getStatisticsTracker(), server, regionName); > } > {code} > It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary, remove it? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow
[ https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15014002#comment-15014002 ] Jesse Yates commented on HBASE-14703: - Looking back to the original purpose, its actually about ripping out the StatsTrackingRpcRetryingCaller. With the new implementation envisioned, I don't think we will still need it. Since we are working in 'master' there is no reason to put in a 'hack' fix and then go back and fix it properly right after - we should just do it "right" the first time. Right just means a bigger change, which takes a bit longer How about we make this patch the cleanup of "stuff that doesn't work". That would be the statsTrackingRpcRetryingCaller and all the stats work in RsRpcServices where we attempt to attach the statistics to the result from the mutateRows computation. Then follow up JIRAs look like: - Support stats tracking for all calls (Umbrella) -- Use async everywhere in HTable calls -- Move statistics to per-call -- unify statistic collection to an interface wdyt? Thanks for being patient through all this - sometimes it takes some exploration and trial/error to figure out what works. > not collect stats when call HTable.mutateRow > - > > Key: HBASE-14703 > URL: https://issues.apache.org/jira/browse/HBASE-14703 > Project: HBase > Issue Type: Bug >Reporter: Heng Chen >Assignee: Heng Chen > Attachments: HBASE-14703-5.2-addendum.patch, HBASE-14703-async.patch, > HBASE-14703-start.patch, HBASE-14703-v4.1.patch, HBASE-14703-v4.patch, > HBASE-14703.patch, HBASE-14703_v1.patch, HBASE-14703_v2.patch, > HBASE-14703_v3.patch, HBASE-14703_v5.1.patch, HBASE-14703_v5.2.patch, > HBASE-14703_v5.patch > > > In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update > serverStatistics twice. > The first one is that we wrapper {{RetryingCallable}} by > {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we > call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below: > {code} > @Override > public T callWithRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > @Override > public T callWithoutRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > {code} > The secondary one is after we get response, in {{receiveMultiAction}}, we do > update again. > {code} > // update the stats about the region, if its a user table. We don't want to > slow down > // updates to meta tables, especially from internal updates (master, etc). > if (AsyncProcess.this.connection.getStatisticsTracker() != null) { > result = ResultStatsUtil.updateStats(result, > AsyncProcess.this.connection.getStatisticsTracker(), server, regionName); > } > {code} > It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary, remove it? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow
[ https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15013654#comment-15013654 ] Heng Chen commented on HBASE-14703: --- {quote} I'm starting to think that to support stats across all the mutation based calls needs to be reimplememented to support other calls. {quote} Oh, yeah, it is a big change, do you think we start it too early? I think we could open another issue for it after this issue finished. As for patch, i agree move statistic from ResultOrException into MultiResponse, But as you mentioned, it will break compatibility. we need further discussion about it. Currently, maybe we should focus on this issue purpose that is we should collect statistic stats when call mutateRow, and unify the call path of mutateRow and other call relates statistic stats. wdyt? dear [~jesse_yates] > not collect stats when call HTable.mutateRow > - > > Key: HBASE-14703 > URL: https://issues.apache.org/jira/browse/HBASE-14703 > Project: HBase > Issue Type: Bug >Reporter: Heng Chen >Assignee: Heng Chen > Attachments: HBASE-14703-5.2-addendum.patch, HBASE-14703-async.patch, > HBASE-14703-start.patch, HBASE-14703-v4.1.patch, HBASE-14703-v4.patch, > HBASE-14703.patch, HBASE-14703_v1.patch, HBASE-14703_v2.patch, > HBASE-14703_v3.patch, HBASE-14703_v5.1.patch, HBASE-14703_v5.2.patch, > HBASE-14703_v5.patch > > > In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update > serverStatistics twice. > The first one is that we wrapper {{RetryingCallable}} by > {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we > call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below: > {code} > @Override > public T callWithRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > @Override > public T callWithoutRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > {code} > The secondary one is after we get response, in {{receiveMultiAction}}, we do > update again. > {code} > // update the stats about the region, if its a user table. We don't want to > slow down > // updates to meta tables, especially from internal updates (master, etc). > if (AsyncProcess.this.connection.getStatisticsTracker() != null) { > result = ResultStatsUtil.updateStats(result, > AsyncProcess.this.connection.getStatisticsTracker(), server, regionName); > } > {code} > It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary, remove it? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow
[ https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15011726#comment-15011726 ] Hadoop QA commented on HBASE-14703: --- {color:green}+1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12773013/HBASE-14703_v5.2.patch against master branch at commit 5ebd7660a94bfb18e6e05b6e46195c76c099eda2. ATTACHMENT ID: 12773013 {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 16 new or modified tests. {color:green}+1 hadoop versions{color}. The patch compiles with all supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0 2.7.1) {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 protoc{color}. The applied patch does not increase the total number of protoc compiler warnings. {color:green}+1 javadoc{color}. The javadoc tool did not generate any warning messages. {color:green}+1 checkstyle{color}. The applied patch does not increase the total number of checkstyle errors {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 2.0.3) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:green}+1 lineLengths{color}. The patch does not introduce lines longer than 100 {color:green}+1 site{color}. The mvn post-site goal succeeds with this patch. {color:green}+1 core tests{color}. The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/16571//testReport/ Release Findbugs (version 2.0.3)warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/16571//artifact/patchprocess/newFindbugsWarnings.html Checkstyle Errors: https://builds.apache.org/job/PreCommit-HBASE-Build/16571//artifact/patchprocess/checkstyle-aggregate.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/16571//console This message is automatically generated. > not collect stats when call HTable.mutateRow > - > > Key: HBASE-14703 > URL: https://issues.apache.org/jira/browse/HBASE-14703 > Project: HBase > Issue Type: Bug >Reporter: Heng Chen >Assignee: Heng Chen > Attachments: HBASE-14703-async.patch, HBASE-14703-start.patch, > HBASE-14703-v4.1.patch, HBASE-14703-v4.patch, HBASE-14703.patch, > HBASE-14703_v1.patch, HBASE-14703_v2.patch, HBASE-14703_v3.patch, > HBASE-14703_v5.1.patch, HBASE-14703_v5.2.patch, HBASE-14703_v5.patch > > > In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update > serverStatistics twice. > The first one is that we wrapper {{RetryingCallable}} by > {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we > call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below: > {code} > @Override > public T callWithRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > @Override > public T callWithoutRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > {code} > The secondary one is after we get response, in {{receiveMultiAction}}, we do > update again. > {code} > // update the stats about the region, if its a user table. We don't want to > slow down > // updates to meta tables, especially from internal updates (master, etc). > if (AsyncProcess.this.connection.getStatisticsTracker() != null) { > result = ResultStatsUtil.updateStats(result, > AsyncProcess.this.connection.getStatisticsTracker(), server, regionName); > } > {code} > It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary, remove it? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow
[ https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15011480#comment-15011480 ] Jesse Yates commented on HBASE-14703: - bq. remove some code unused in RpcRetryingCallerImpl I left it, to preserve the current implementation. I don't know where else it would be used, but can't hurt to not change things. Then here, in AsyncProcess {code} 723 // As mutateRow, if statistic off, nothing will return. So let's stop. 724 actionsInProgress.set(0); {code} This doesn't feel right. In receiveMultiAction -> setResult we just decrement the actionsInProgress, rather than set it to 0. We don't have any tests that check running ops concurrently, so I think it would be better to refactor AsyncProcess a little bit more to unify the logic a bit > not collect stats when call HTable.mutateRow > - > > Key: HBASE-14703 > URL: https://issues.apache.org/jira/browse/HBASE-14703 > Project: HBase > Issue Type: Bug >Reporter: Heng Chen >Assignee: Heng Chen > Attachments: HBASE-14703-async.patch, HBASE-14703-start.patch, > HBASE-14703-v4.1.patch, HBASE-14703-v4.patch, HBASE-14703.patch, > HBASE-14703_v1.patch, HBASE-14703_v2.patch, HBASE-14703_v3.patch, > HBASE-14703_v5.1.patch, HBASE-14703_v5.2.patch, HBASE-14703_v5.patch > > > In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update > serverStatistics twice. > The first one is that we wrapper {{RetryingCallable}} by > {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we > call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below: > {code} > @Override > public T callWithRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > @Override > public T callWithoutRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > {code} > The secondary one is after we get response, in {{receiveMultiAction}}, we do > update again. > {code} > // update the stats about the region, if its a user table. We don't want to > slow down > // updates to meta tables, especially from internal updates (master, etc). > if (AsyncProcess.this.connection.getStatisticsTracker() != null) { > result = ResultStatsUtil.updateStats(result, > AsyncProcess.this.connection.getStatisticsTracker(), server, regionName); > } > {code} > It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary, remove it? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow
[ https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15011221#comment-15011221 ] Heng Chen commented on HBASE-14703: --- Update patch to fix failed testcase. It is here in AsyncProcess.run, {code} + if (currentCallable != null) { +if (res == null|| res.getResults().size() == 0) { + // As mutateRow, if statistic off, nothing will return. So let's stop. + actionsInProgress.set(0); + return; +} + } else { +if (res == null) { + return; +} + } {code} As for put, it will has a response, and res.getResults().size() is 0, we should collect exceptions in receiveMultiAction(multiAction, server, res, numAttempt), not just return. Let's see if we could unify this code for mutateRow and put, > not collect stats when call HTable.mutateRow > - > > Key: HBASE-14703 > URL: https://issues.apache.org/jira/browse/HBASE-14703 > Project: HBase > Issue Type: Bug >Reporter: Heng Chen >Assignee: Heng Chen > Attachments: HBASE-14703-async.patch, HBASE-14703-start.patch, > HBASE-14703-v4.1.patch, HBASE-14703-v4.patch, HBASE-14703.patch, > HBASE-14703_v1.patch, HBASE-14703_v2.patch, HBASE-14703_v3.patch, > HBASE-14703_v5.1.patch, HBASE-14703_v5.2.patch, HBASE-14703_v5.patch > > > In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update > serverStatistics twice. > The first one is that we wrapper {{RetryingCallable}} by > {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we > call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below: > {code} > @Override > public T callWithRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > @Override > public T callWithoutRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > {code} > The secondary one is after we get response, in {{receiveMultiAction}}, we do > update again. > {code} > // update the stats about the region, if its a user table. We don't want to > slow down > // updates to meta tables, especially from internal updates (master, etc). > if (AsyncProcess.this.connection.getStatisticsTracker() != null) { > result = ResultStatsUtil.updateStats(result, > AsyncProcess.this.connection.getStatisticsTracker(), server, regionName); > } > {code} > It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary, remove it? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow
[ https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15010856#comment-15010856 ] Hadoop QA commented on HBASE-14703: --- {color:green}+1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12772952/HBASE-14703_v5.1.patch against master branch at commit 5ebd7660a94bfb18e6e05b6e46195c76c099eda2. ATTACHMENT ID: 12772952 {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 16 new or modified tests. {color:green}+1 hadoop versions{color}. The patch compiles with all supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0 2.7.1) {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 protoc{color}. The applied patch does not increase the total number of protoc compiler warnings. {color:green}+1 javadoc{color}. The javadoc tool did not generate any warning messages. {color:green}+1 checkstyle{color}. The applied patch does not increase the total number of checkstyle errors {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 2.0.3) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:green}+1 lineLengths{color}. The patch does not introduce lines longer than 100 {color:green}+1 site{color}. The mvn post-site goal succeeds with this patch. {color:red}-1 core tests{color}. The patch failed these unit tests: org.apache.hadoop.hbase.regionserver.TestHRegionOnCluster org.apache.hadoop.hbase.client.TestHCM org.apache.hadoop.hbase.quotas.TestQuotaThrottle Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/16569//testReport/ Release Findbugs (version 2.0.3)warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/16569//artifact/patchprocess/newFindbugsWarnings.html Checkstyle Errors: https://builds.apache.org/job/PreCommit-HBASE-Build/16569//artifact/patchprocess/checkstyle-aggregate.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/16569//console This message is automatically generated. > not collect stats when call HTable.mutateRow > - > > Key: HBASE-14703 > URL: https://issues.apache.org/jira/browse/HBASE-14703 > Project: HBase > Issue Type: Bug >Reporter: Heng Chen >Assignee: Heng Chen > Attachments: HBASE-14703-async.patch, HBASE-14703-start.patch, > HBASE-14703-v4.1.patch, HBASE-14703-v4.patch, HBASE-14703.patch, > HBASE-14703_v1.patch, HBASE-14703_v2.patch, HBASE-14703_v3.patch, > HBASE-14703_v5.1.patch, HBASE-14703_v5.patch > > > In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update > serverStatistics twice. > The first one is that we wrapper {{RetryingCallable}} by > {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we > call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below: > {code} > @Override > public T callWithRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > @Override > public T callWithoutRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > {code} > The secondary one is after we get response, in {{receiveMultiAction}}, we do > update again. > {code} > // update the stats about the region, if its a user table. We don't want to > slow down > // updates to meta tables, especially from internal updates (master, etc). > if (AsyncProcess.this.connection.getStatisticsTracker() != null) { > result = ResultStatsUtil.updateStats(result, > AsyncProcess.this.connection.getStatisticsTracker(), server, regionName); > } > {code} > It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary, remove it? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow
[ https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15010546#comment-15010546 ] Hadoop QA commented on HBASE-14703: --- {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12772936/HBASE-14703_v5.patch against master branch at commit 5ebd7660a94bfb18e6e05b6e46195c76c099eda2. ATTACHMENT ID: 12772936 {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 16 new or modified tests. {color:green}+1 hadoop versions{color}. The patch compiles with all supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0 2.7.1) {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 protoc{color}. The applied patch does not increase the total number of protoc compiler warnings. {color:green}+1 javadoc{color}. The javadoc tool did not generate any warning messages. {color:red}-1 checkstyle{color}. The applied patch generated 1728 checkstyle errors (more than the master's current 1727 errors). {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 2.0.3) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:green}+1 lineLengths{color}. The patch does not introduce lines longer than 100 {color:green}+1 site{color}. The mvn post-site goal succeeds with this patch. {color:red}-1 core tests{color}. The patch failed these unit tests: org.apache.hadoop.hbase.client.TestHBaseAdminNoCluster Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/16568//testReport/ Release Findbugs (version 2.0.3)warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/16568//artifact/patchprocess/newFindbugsWarnings.html Checkstyle Errors: https://builds.apache.org/job/PreCommit-HBASE-Build/16568//artifact/patchprocess/checkstyle-aggregate.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/16568//console This message is automatically generated. > not collect stats when call HTable.mutateRow > - > > Key: HBASE-14703 > URL: https://issues.apache.org/jira/browse/HBASE-14703 > Project: HBase > Issue Type: Bug >Reporter: Heng Chen >Assignee: Heng Chen > Attachments: HBASE-14703-async.patch, HBASE-14703-start.patch, > HBASE-14703-v4.1.patch, HBASE-14703-v4.patch, HBASE-14703.patch, > HBASE-14703_v1.patch, HBASE-14703_v2.patch, HBASE-14703_v3.patch, > HBASE-14703_v5.patch > > > In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update > serverStatistics twice. > The first one is that we wrapper {{RetryingCallable}} by > {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we > call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below: > {code} > @Override > public T callWithRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > @Override > public T callWithoutRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > {code} > The secondary one is after we get response, in {{receiveMultiAction}}, we do > update again. > {code} > // update the stats about the region, if its a user table. We don't want to > slow down > // updates to meta tables, especially from internal updates (master, etc). > if (AsyncProcess.this.connection.getStatisticsTracker() != null) { > result = ResultStatsUtil.updateStats(result, > AsyncProcess.this.connection.getStatisticsTracker(), server, regionName); > } > {code} > It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary, remove it? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow
[ https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15010428#comment-15010428 ] Heng Chen commented on HBASE-14703: --- {quote} Now, the timeout tracking is encapsulated and reused in the RetryingCaller and in the callable. {quote} I like the idea, it is amazing! And unify switching code in AP looks wonderful! Base on that, i made some little changes. * Add some code in AP.run {code: title=AsyncProcess.java} res = caller.callWithoutRetries(callable, currentCallTotalTimeout); if (res == null|| res.getResults().size() == 0) { if (currentCallable != null) { // As mutateRow, if statistic off, nothing will return. So let's stop. actionsInProgress.set(0); } return; } {code} * Add some code in HTable.mutateRow {code: title=HTable.java} int remainingTime = tracker.getRemainingTime(callTimeout); if (remainingTime == 0) { throw new DoNotRetryIOException("Timeout for mutate row"); } {code} * remove some code unused in RpcRetryingCallerImpl {code} public T callWithoutRetries(RetryingCallable callable, int callTimeout) throws IOException, RuntimeException { +tracker.start(); //Remove it. {code} * fix some checkstyle errors and org.apache.hadoop.hbase.TestInterfaceAudienceAnnotations failed. Let's say how QA talks. > not collect stats when call HTable.mutateRow > - > > Key: HBASE-14703 > URL: https://issues.apache.org/jira/browse/HBASE-14703 > Project: HBase > Issue Type: Bug >Reporter: Heng Chen >Assignee: Heng Chen > Attachments: HBASE-14703-async.patch, HBASE-14703-start.patch, > HBASE-14703-v4.1.patch, HBASE-14703-v4.patch, HBASE-14703.patch, > HBASE-14703_v1.patch, HBASE-14703_v2.patch, HBASE-14703_v3.patch, > HBASE-14703_v5.patch > > > In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update > serverStatistics twice. > The first one is that we wrapper {{RetryingCallable}} by > {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we > call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below: > {code} > @Override > public T callWithRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > @Override > public T callWithoutRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > {code} > The secondary one is after we get response, in {{receiveMultiAction}}, we do > update again. > {code} > // update the stats about the region, if its a user table. We don't want to > slow down > // updates to meta tables, especially from internal updates (master, etc). > if (AsyncProcess.this.connection.getStatisticsTracker() != null) { > result = ResultStatsUtil.updateStats(result, > AsyncProcess.this.connection.getStatisticsTracker(), server, regionName); > } > {code} > It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary, remove it? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow
[ https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15010225#comment-15010225 ] Hadoop QA commented on HBASE-14703: --- {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12772908/HBASE-14703-v4.1.patch against master branch at commit 1b13bfcd43862451a807fa0f1943de781eef. ATTACHMENT ID: 12772908 {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 12 new or modified tests. {color:green}+1 hadoop versions{color}. The patch compiles with all supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0 2.7.1) {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 protoc{color}. The applied patch does not increase the total number of protoc compiler warnings. {color:green}+1 javadoc{color}. The javadoc tool did not generate any warning messages. {color:red}-1 checkstyle{color}. The applied patch generated 1734 checkstyle errors (more than the master's current 1727 errors). {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 2.0.3) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:red}-1 lineLengths{color}. The patch introduces the following lines longer than 100: + Collections.newSetFromMap(new ConcurrentHashMap()); +tableName, actions, nonceGroup, getPool(pool), needResults, results, callback, callable, curTimeout); +// Currently If there is no statistic tracker, server will return empty RegionActionResult. +protected RpcRetryingCaller createCaller(PayloadCarryingServerCallable callable) { + Set set = ((AsyncRequestFutureImpl)reqs).getCallsInProgress(); {color:green}+1 site{color}. The mvn post-site goal succeeds with this patch. {color:red}-1 core tests{color}. The patch failed these unit tests: org.apache.hadoop.hbase.TestInterfaceAudienceAnnotations Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/16562//testReport/ Release Findbugs (version 2.0.3)warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/16562//artifact/patchprocess/newFindbugsWarnings.html Checkstyle Errors: https://builds.apache.org/job/PreCommit-HBASE-Build/16562//artifact/patchprocess/checkstyle-aggregate.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/16562//console This message is automatically generated. > not collect stats when call HTable.mutateRow > - > > Key: HBASE-14703 > URL: https://issues.apache.org/jira/browse/HBASE-14703 > Project: HBase > Issue Type: Bug >Reporter: Heng Chen >Assignee: Heng Chen > Attachments: HBASE-14703-async.patch, HBASE-14703-start.patch, > HBASE-14703-v4.1.patch, HBASE-14703-v4.patch, HBASE-14703.patch, > HBASE-14703_v1.patch, HBASE-14703_v2.patch, HBASE-14703_v3.patch > > > In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update > serverStatistics twice. > The first one is that we wrapper {{RetryingCallable}} by > {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we > call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below: > {code} > @Override > public T callWithRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > @Override > public T callWithoutRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > {code} > The secondary one is after we get response, in {{receiveMultiAction}}, we do > update again. > {code} > // update the stats about the region, if its a user table. We don't want to > slow down > // updates to meta tables, especially from internal updates (master, etc). > if (AsyncProcess.this.connection.getStatisticsTracker() != null) { > result = ResultStatsUtil.updateStats(result, > AsyncProcess.this.connection.getStatisticsTracker(), server, regionName); > } > {code} > It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary, remove it? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow
[ https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15010132#comment-15010132 ] Hadoop QA commented on HBASE-14703: --- {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12772903/HBASE-14703-v4.patch against master branch at commit 1b13bfcd43862451a807fa0f1943de781eef. ATTACHMENT ID: 12772903 {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 12 new or modified tests. {color:red}-1 patch{color}. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/16561//console This message is automatically generated. > not collect stats when call HTable.mutateRow > - > > Key: HBASE-14703 > URL: https://issues.apache.org/jira/browse/HBASE-14703 > Project: HBase > Issue Type: Bug >Reporter: Heng Chen >Assignee: Heng Chen > Attachments: HBASE-14703-async.patch, HBASE-14703-start.patch, > HBASE-14703-v4.patch, HBASE-14703.patch, HBASE-14703_v1.patch, > HBASE-14703_v2.patch, HBASE-14703_v3.patch > > > In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update > serverStatistics twice. > The first one is that we wrapper {{RetryingCallable}} by > {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we > call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below: > {code} > @Override > public T callWithRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > @Override > public T callWithoutRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > {code} > The secondary one is after we get response, in {{receiveMultiAction}}, we do > update again. > {code} > // update the stats about the region, if its a user table. We don't want to > slow down > // updates to meta tables, especially from internal updates (master, etc). > if (AsyncProcess.this.connection.getStatisticsTracker() != null) { > result = ResultStatsUtil.updateStats(result, > AsyncProcess.this.connection.getStatisticsTracker(), server, regionName); > } > {code} > It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary, remove it? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow
[ https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15007720#comment-15007720 ] Jesse Yates commented on HBASE-14703: - sorry [~chenheng], got a bit caught up with stuff. Will take a look tomorrow. > not collect stats when call HTable.mutateRow > - > > Key: HBASE-14703 > URL: https://issues.apache.org/jira/browse/HBASE-14703 > Project: HBase > Issue Type: Bug >Reporter: Heng Chen >Assignee: Heng Chen > Attachments: HBASE-14703-async.patch, HBASE-14703-start.patch, > HBASE-14703.patch, HBASE-14703_v1.patch, HBASE-14703_v2.patch, > HBASE-14703_v3.patch > > > In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update > serverStatistics twice. > The first one is that we wrapper {{RetryingCallable}} by > {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we > call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below: > {code} > @Override > public T callWithRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > @Override > public T callWithoutRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > {code} > The secondary one is after we get response, in {{receiveMultiAction}}, we do > update again. > {code} > // update the stats about the region, if its a user table. We don't want to > slow down > // updates to meta tables, especially from internal updates (master, etc). > if (AsyncProcess.this.connection.getStatisticsTracker() != null) { > result = ResultStatsUtil.updateStats(result, > AsyncProcess.this.connection.getStatisticsTracker(), server, regionName); > } > {code} > It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary, remove it? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow
[ https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15006190#comment-15006190 ] Heng Chen commented on HBASE-14703: --- ping [~jesse_yates] :) :) > not collect stats when call HTable.mutateRow > - > > Key: HBASE-14703 > URL: https://issues.apache.org/jira/browse/HBASE-14703 > Project: HBase > Issue Type: Bug >Reporter: Heng Chen >Assignee: Heng Chen > Attachments: HBASE-14703-async.patch, HBASE-14703-start.patch, > HBASE-14703.patch, HBASE-14703_v1.patch, HBASE-14703_v2.patch, > HBASE-14703_v3.patch > > > In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update > serverStatistics twice. > The first one is that we wrapper {{RetryingCallable}} by > {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we > call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below: > {code} > @Override > public T callWithRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > @Override > public T callWithoutRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > {code} > The secondary one is after we get response, in {{receiveMultiAction}}, we do > update again. > {code} > // update the stats about the region, if its a user table. We don't want to > slow down > // updates to meta tables, especially from internal updates (master, etc). > if (AsyncProcess.this.connection.getStatisticsTracker() != null) { > result = ResultStatsUtil.updateStats(result, > AsyncProcess.this.connection.getStatisticsTracker(), server, regionName); > } > {code} > It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary, remove it? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow
[ https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15000188#comment-15000188 ] Hadoop QA commented on HBASE-14703: --- {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12771707/HBASE-14703_v3.patch against master branch at commit 1f62a487284b57fca505bc1b3d04c1f86b2e7d76. ATTACHMENT ID: 12771707 {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 8 new or modified tests. {color:green}+1 hadoop versions{color}. The patch compiles with all supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0 2.7.1) {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 protoc{color}. The applied patch does not increase the total number of protoc compiler warnings. {color:green}+1 javadoc{color}. The javadoc tool did not generate any warning messages. {color:red}-1 checkstyle{color}. The applied patch generated 1729 checkstyle errors (more than the master's current 1727 errors). {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 2.0.3) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:red}-1 lineLengths{color}. The patch introduces the following lines longer than 100: +tableName, actions, nonceGroup, getPool(pool), needResults, results, callback, callable, curTimeout); +// Currently If there is no statistic tracker, server will return empty RegionActionResult. {color:green}+1 site{color}. The mvn post-site goal succeeds with this patch. {color:green}+1 core tests{color}. The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/16492//testReport/ Release Findbugs (version 2.0.3)warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/16492//artifact/patchprocess/newFindbugsWarnings.html Checkstyle Errors: https://builds.apache.org/job/PreCommit-HBASE-Build/16492//artifact/patchprocess/checkstyle-aggregate.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/16492//console This message is automatically generated. > not collect stats when call HTable.mutateRow > - > > Key: HBASE-14703 > URL: https://issues.apache.org/jira/browse/HBASE-14703 > Project: HBase > Issue Type: Bug >Reporter: Heng Chen >Assignee: Heng Chen > Attachments: HBASE-14703-async.patch, HBASE-14703-start.patch, > HBASE-14703.patch, HBASE-14703_v1.patch, HBASE-14703_v2.patch, > HBASE-14703_v3.patch > > > In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update > serverStatistics twice. > The first one is that we wrapper {{RetryingCallable}} by > {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we > call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below: > {code} > @Override > public T callWithRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > @Override > public T callWithoutRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > {code} > The secondary one is after we get response, in {{receiveMultiAction}}, we do > update again. > {code} > // update the stats about the region, if its a user table. We don't want to > slow down > // updates to meta tables, especially from internal updates (master, etc). > if (AsyncProcess.this.connection.getStatisticsTracker() != null) { > result = ResultStatsUtil.updateStats(result, > AsyncProcess.this.connection.getStatisticsTracker(), server, regionName); > } > {code} > It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary, remove it? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow
[ https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1572#comment-1572 ] Heng Chen commented on HBASE-14703: --- {quote} Looking closer at the patch, I'm a bit concerned with how retries will work. Since the AsyncProcess handles retries, we shouldn't be calling the callable.callWithRetries, right? Then we would get two sets of retries with the callable nested under the AsyncProcess. Can't we just use the standard AP timeout then as well (its not something the user changes per-call)? {quote} Indeed, it is a problem. I have thought that callable.callWithRetries could handle timeout option inside, but i found that we can use the standard AP timeout to do it as you mentioned. I will do it. {quote} that's only going to be applicable for the checkAndXXX cases, right? {quote} yeah, it is only for checkAndXXX. {quote} In the meantime, lets just focus on supporting the mutateRow and then later we can fix checkAndMutate along with the other checkAnd methods. {quote} OK. Currently, we can leave checkAndMutate without AP before we can handle it more gracefully... {quote} For mutateRow, maybe its better to just check to see if it has any Results, rather than checking to see if there is a statistic tracker? It ties more into why the ResponseConverter won't work, rather than why there are no Results in the response and leave that information in a comment around the check for the count of results. {quote} Sounds reasonable, i will do it. Other tips: * currently, in mutateRow server side, if hbase.client.backpressure.enabled ON, response.getRegionActionResultCount() will be more than request.getRegionActionCount(). I fix it in server side, so we can pass check in ResponseConverter. * similarly, because of no results in response of mutateRow, we can't add loadstats into result in ResponseConverter. I fix it too. > not collect stats when call HTable.mutateRow > - > > Key: HBASE-14703 > URL: https://issues.apache.org/jira/browse/HBASE-14703 > Project: HBase > Issue Type: Bug >Reporter: Heng Chen >Assignee: Heng Chen > Attachments: HBASE-14703-async.patch, HBASE-14703-start.patch, > HBASE-14703.patch, HBASE-14703_v1.patch, HBASE-14703_v2.patch > > > In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update > serverStatistics twice. > The first one is that we wrapper {{RetryingCallable}} by > {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we > call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below: > {code} > @Override > public T callWithRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > @Override > public T callWithoutRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > {code} > The secondary one is after we get response, in {{receiveMultiAction}}, we do > update again. > {code} > // update the stats about the region, if its a user table. We don't want to > slow down > // updates to meta tables, especially from internal updates (master, etc). > if (AsyncProcess.this.connection.getStatisticsTracker() != null) { > result = ResultStatsUtil.updateStats(result, > AsyncProcess.this.connection.getStatisticsTracker(), server, regionName); > } > {code} > It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary, remove it? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow
[ https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14999564#comment-14999564 ] Jesse Yates commented on HBASE-14703: - Looking closer at the patch, I'm a bit concerned with how retries will work. Since the AsyncProcess handles retries, we shouldn't be calling the callable.callWithRetries, right? Then we would get two sets of retries with the callable nested under the AsyncProcess. Can't we just use the standard AP timeout then as well (its not something the user changes per-call)? bq. it seems that processed flag has no relates with results Ah, yes. For this, IIRC its all or nothing with the check so all you need to send back is a single 'everything worked' flag, rather than a flag per mutation, which would all be the same value. You added that with the 'processed' flag in the MultiResponse, but that's only going to be applicable for the checkAndXXX cases, right? I don't particularly like bolting on a processed flag onto the MulitResponse, just to support this one case. Looking at the checkAnd methods, they all follow the same basic logic and return a MutateResponse#processed, except for checkAndMutate. That makes me think we can do the same thing we are doing here for the other checkAndXXX methods, but fork logic a little bit in AsyncProcess to support that kind of processing instead (as a follow up JIRA, not here). In the meantime, lets just focus on supporting the mutateRow and then later we can fix checkAndMutate along with the other checkAnd methods. For mutateRow, maybe its better to just check to see if it has any Results, rather than checking to see if there is a statistic tracker? It ties more into why the ResponseConverter won't work, rather than why there are no Results in the response and leave that information in a comment around the check for the count of results. Good stuff Heng Chen > not collect stats when call HTable.mutateRow > - > > Key: HBASE-14703 > URL: https://issues.apache.org/jira/browse/HBASE-14703 > Project: HBase > Issue Type: Bug >Reporter: Heng Chen >Assignee: Heng Chen > Attachments: HBASE-14703-async.patch, HBASE-14703-start.patch, > HBASE-14703.patch, HBASE-14703_v1.patch, HBASE-14703_v2.patch > > > In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update > serverStatistics twice. > The first one is that we wrapper {{RetryingCallable}} by > {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we > call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below: > {code} > @Override > public T callWithRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > @Override > public T callWithoutRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > {code} > The secondary one is after we get response, in {{receiveMultiAction}}, we do > update again. > {code} > // update the stats about the region, if its a user table. We don't want to > slow down > // updates to meta tables, especially from internal updates (master, etc). > if (AsyncProcess.this.connection.getStatisticsTracker() != null) { > result = ResultStatsUtil.updateStats(result, > AsyncProcess.this.connection.getStatisticsTracker(), server, regionName); > } > {code} > It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary, remove it? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow
[ https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14998255#comment-14998255 ] Heng Chen commented on HBASE-14703: --- {quote} There seems to be a whole bunch of mechanics to get the 'processed' result out of the AP - why can't you just use the results[] option and use that to get the processed bit? If that works, you can throw out a ton of changes. {quote} Yeah, indeed, we can use results[] to get processed bit. But if we want to unify get/scan through AsyncProcess, it is hard to process. So i separated it with another interface in AP, and it seems that processed flag has no relates with results.. wdyt? {quote} can we add an overloaded createAsyncRequestFuture() that doesn't take a callable or a timeout? Would just call your version but with null, timeout (saves a bunch of changes w/o loss of clarity) {quote} Sounds good, i will do it. {quote} why the switch in mutateRow around if there is a statistic tracker for returning a value? seems like it simplify the code in the async process too. {quote} It is real tricky... Because i use ResponseConverter.getResults to convert ClientProtos.MultiResponse into {{org.apache.hadoop.hbase.client.MultiResponse}}. And in ResponseConverter.getResults, it has this check. {code} if (actions.getActionCount() != actionResult.getResultOrExceptionCount()) { throw new IllegalStateException("actions.getActionCount=" + actions.getActionCount() + ", actionResult.getResultOrExceptionCount=" + actionResult.getResultOrExceptionCount() + " for region " + actions.getRegion()); } {code} In mutateRow server side, if we set option {{hbase.client.backpressure.enabled}} OFF, it will return nothing to client. There is no results, no exceptions... So above check will always failed. To fix this, i add a switch in mutateRow as you mentioned May be we can write another converter to convert response... wdyt? > not collect stats when call HTable.mutateRow > - > > Key: HBASE-14703 > URL: https://issues.apache.org/jira/browse/HBASE-14703 > Project: HBase > Issue Type: Bug >Reporter: Heng Chen >Assignee: Heng Chen > Attachments: HBASE-14703-async.patch, HBASE-14703-start.patch, > HBASE-14703.patch, HBASE-14703_v1.patch, HBASE-14703_v2.patch > > > In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update > serverStatistics twice. > The first one is that we wrapper {{RetryingCallable}} by > {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we > call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below: > {code} > @Override > public T callWithRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > @Override > public T callWithoutRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > {code} > The secondary one is after we get response, in {{receiveMultiAction}}, we do > update again. > {code} > // update the stats about the region, if its a user table. We don't want to > slow down > // updates to meta tables, especially from internal updates (master, etc). > if (AsyncProcess.this.connection.getStatisticsTracker() != null) { > result = ResultStatsUtil.updateStats(result, > AsyncProcess.this.connection.getStatisticsTracker(), server, regionName); > } > {code} > It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary, remove it? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow
[ https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14997093#comment-14997093 ] Jesse Yates commented on HBASE-14703: - Getting there! There seems to be a whole bunch of mechanics to get the 'processed' result out of the AP - why can't you just use the results[] option and use that to get the processed bit? If that works, you can throw out a ton of changes. Other comments: - can we add an overloaded createAsyncRequestFuture() that doesn't take a callable or a timeout? Would just call your version but with null, timeout (saves a bunch of changes w/o loss of clarity) - why the switch in mutateRow around if there is a statistic tracker for returning a value? seems like it simplify the code in the async process too. - Extra newline in MultiServerCallable > not collect stats when call HTable.mutateRow > - > > Key: HBASE-14703 > URL: https://issues.apache.org/jira/browse/HBASE-14703 > Project: HBase > Issue Type: Bug >Reporter: Heng Chen >Assignee: Heng Chen > Attachments: HBASE-14703-async.patch, HBASE-14703-start.patch, > HBASE-14703.patch, HBASE-14703_v1.patch, HBASE-14703_v2.patch > > > In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update > serverStatistics twice. > The first one is that we wrapper {{RetryingCallable}} by > {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we > call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below: > {code} > @Override > public T callWithRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > @Override > public T callWithoutRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > {code} > The secondary one is after we get response, in {{receiveMultiAction}}, we do > update again. > {code} > // update the stats about the region, if its a user table. We don't want to > slow down > // updates to meta tables, especially from internal updates (master, etc). > if (AsyncProcess.this.connection.getStatisticsTracker() != null) { > result = ResultStatsUtil.updateStats(result, > AsyncProcess.this.connection.getStatisticsTracker(), server, regionName); > } > {code} > It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary, remove it? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow
[ https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14996284#comment-14996284 ] Heng Chen commented on HBASE-14703: --- [~jesse_yates] how do you think about this patch? > not collect stats when call HTable.mutateRow > - > > Key: HBASE-14703 > URL: https://issues.apache.org/jira/browse/HBASE-14703 > Project: HBase > Issue Type: Bug >Reporter: Heng Chen >Assignee: Heng Chen > Attachments: HBASE-14703-async.patch, HBASE-14703-start.patch, > HBASE-14703.patch, HBASE-14703_v1.patch, HBASE-14703_v2.patch > > > In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update > serverStatistics twice. > The first one is that we wrapper {{RetryingCallable}} by > {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we > call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below: > {code} > @Override > public T callWithRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > @Override > public T callWithoutRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > {code} > The secondary one is after we get response, in {{receiveMultiAction}}, we do > update again. > {code} > // update the stats about the region, if its a user table. We don't want to > slow down > // updates to meta tables, especially from internal updates (master, etc). > if (AsyncProcess.this.connection.getStatisticsTracker() != null) { > result = ResultStatsUtil.updateStats(result, > AsyncProcess.this.connection.getStatisticsTracker(), server, regionName); > } > {code} > It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary, remove it? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow
[ https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14991516#comment-14991516 ] Hadoop QA commented on HBASE-14703: --- {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12770758/HBASE-14703_v2.patch against master branch at commit 050ebe850b32057860fb94b46f955352db139db1. ATTACHMENT ID: 12770758 {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 12 new or modified tests. {color:green}+1 hadoop versions{color}. The patch compiles with all supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0 2.7.1) {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 protoc{color}. The applied patch does not increase the total number of protoc compiler warnings. {color:green}+1 javadoc{color}. The javadoc tool did not generate any warning messages. {color:red}-1 checkstyle{color}. The applied patch generated 1729 checkstyle errors (more than the master's current 1726 errors). {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 2.0.3) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:red}-1 lineLengths{color}. The patch introduces the following lines longer than 100: +tableName, actions, nonceGroup, getPool(pool), needResults, results, callback, callable, curTimeout); +// If statisitcs is off, the action number will not equals resultOrExceptoins number. +//return rpcCallerFactory. newCaller().callWithRetries(callable, this.operationTimeout); + RetryingCallable callable, int t) { {color:green}+1 site{color}. The mvn post-site goal succeeds with this patch. {color:green}+1 core tests{color}. The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/16404//testReport/ Release Findbugs (version 2.0.3)warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/16404//artifact/patchprocess/newFindbugsWarnings.html Checkstyle Errors: https://builds.apache.org/job/PreCommit-HBASE-Build/16404//artifact/patchprocess/checkstyle-aggregate.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/16404//console This message is automatically generated. > not collect stats when call HTable.mutateRow > - > > Key: HBASE-14703 > URL: https://issues.apache.org/jira/browse/HBASE-14703 > Project: HBase > Issue Type: Bug >Reporter: Heng Chen >Assignee: Heng Chen > Attachments: HBASE-14703-async.patch, HBASE-14703-start.patch, > HBASE-14703.patch, HBASE-14703_v1.patch, HBASE-14703_v2.patch > > > In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update > serverStatistics twice. > The first one is that we wrapper {{RetryingCallable}} by > {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we > call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below: > {code} > @Override > public T callWithRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > @Override > public T callWithoutRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > {code} > The secondary one is after we get response, in {{receiveMultiAction}}, we do > update again. > {code} > // update the stats about the region, if its a user table. We don't want to > slow down > // updates to meta tables, especially from internal updates (master, etc). > if (AsyncProcess.this.connection.getStatisticsTracker() != null) { > result = ResultStatsUtil.updateStats(result, > AsyncProcess.this.connection.getStatisticsTracker(), server, regionName); > } > {code} > It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary, remove it? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow
[ https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14991284#comment-14991284 ] Heng Chen commented on HBASE-14703: --- update patch, i refactor mutate/checkAndMutate code to use the same path as AsyncProcess. It is some tricky, because there are lots of kinds of server response for mutate/checkAndMutate. We can see how QA talks about this patch > not collect stats when call HTable.mutateRow > - > > Key: HBASE-14703 > URL: https://issues.apache.org/jira/browse/HBASE-14703 > Project: HBase > Issue Type: Bug >Reporter: Heng Chen >Assignee: Heng Chen > Attachments: HBASE-14703-async.patch, HBASE-14703-start.patch, > HBASE-14703.patch, HBASE-14703_v1.patch > > > In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update > serverStatistics twice. > The first one is that we wrapper {{RetryingCallable}} by > {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we > call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below: > {code} > @Override > public T callWithRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > @Override > public T callWithoutRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > {code} > The secondary one is after we get response, in {{receiveMultiAction}}, we do > update again. > {code} > // update the stats about the region, if its a user table. We don't want to > slow down > // updates to meta tables, especially from internal updates (master, etc). > if (AsyncProcess.this.connection.getStatisticsTracker() != null) { > result = ResultStatsUtil.updateStats(result, > AsyncProcess.this.connection.getStatisticsTracker(), server, regionName); > } > {code} > It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary, remove it? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow
[ https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14985952#comment-14985952 ] Jesse Yates commented on HBASE-14703: - I think we also need to cover checkAndMutate - it has the same logic as mutate. Otherwise, yeah, this seems like an improvement. Couldn't we also drop the StatsTrackingRpcRetryingCaller then as well? Minor nit, IMO its a bit 'nicer' to use static checking (especially in IDEs that don't catch strings when doing renames) for the error handling, so something like: {code} } catch(RetriesExhaustedWithDetailsException e) { for(Throwable rootCause: e.getCauses()){ if(rootCause instanceof NoSuchColumnFamilyException){ return; } } throw e; {code} bq. I found two difference point between original code and the patch. It doesn't seem to far fetch though to add support with another 2 parameters when doing the submission to the async process. If not, at the very least we could refactor the mutate/checkAndMutate code to use the same path and fix the stats collecting there > not collect stats when call HTable.mutateRow > - > > Key: HBASE-14703 > URL: https://issues.apache.org/jira/browse/HBASE-14703 > Project: HBase > Issue Type: Bug >Reporter: Heng Chen >Assignee: Heng Chen > Attachments: HBASE-14703-async.patch, HBASE-14703-start.patch, > HBASE-14703.patch, HBASE-14703_v1.patch > > > In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update > serverStatistics twice. > The first one is that we wrapper {{RetryingCallable}} by > {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we > call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below: > {code} > @Override > public T callWithRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > @Override > public T callWithoutRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > {code} > The secondary one is after we get response, in {{receiveMultiAction}}, we do > update again. > {code} > // update the stats about the region, if its a user table. We don't want to > slow down > // updates to meta tables, especially from internal updates (master, etc). > if (AsyncProcess.this.connection.getStatisticsTracker() != null) { > result = ResultStatsUtil.updateStats(result, > AsyncProcess.this.connection.getStatisticsTracker(), server, regionName); > } > {code} > It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary, remove it? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14703) not collect stats when call HTable.mutateRow
[ https://issues.apache.org/jira/browse/HBASE-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14983941#comment-14983941 ] Heng Chen commented on HBASE-14703: --- Oh.. I found two difference point between original code and the patch. * In original code, in mutateRow, it called callWithRetries; but in asyncProcess.submit, it called .callWithOutRetries * timeout setting is different. > not collect stats when call HTable.mutateRow > - > > Key: HBASE-14703 > URL: https://issues.apache.org/jira/browse/HBASE-14703 > Project: HBase > Issue Type: Bug >Reporter: Heng Chen >Assignee: Heng Chen > Attachments: HBASE-14703-async.patch, HBASE-14703-start.patch, > HBASE-14703.patch, HBASE-14703_v1.patch > > > In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update > serverStatistics twice. > The first one is that we wrapper {{RetryingCallable}} by > {{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we > call {{callWithRetries}} and {{callWithoutRetries}}. Relates code like below: > {code} > @Override > public T callWithRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > @Override > public T callWithoutRetries(RetryingCallable callable, int callTimeout) > throws IOException, RuntimeException { > T result = delegate.callWithRetries(callable, callTimeout); > return updateStatsAndUnwrap(result, callable); > } > {code} > The secondary one is after we get response, in {{receiveMultiAction}}, we do > update again. > {code} > // update the stats about the region, if its a user table. We don't want to > slow down > // updates to meta tables, especially from internal updates (master, etc). > if (AsyncProcess.this.connection.getStatisticsTracker() != null) { > result = ResultStatsUtil.updateStats(result, > AsyncProcess.this.connection.getStatisticsTracker(), server, regionName); > } > {code} > It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary, remove it? -- This message was sent by Atlassian JIRA (v6.3.4#6332)