[jira] [Commented] (HBASE-6377) HBASE-5533 metrics miss all operations submitted via MultiAction
[ https://issues.apache.org/jira/browse/HBASE-6377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14077335#comment-14077335 ] Misty Stanley-Jones commented on HBASE-6377: Wrong ticket, sorry. > HBASE-5533 metrics miss all operations submitted via MultiAction > > > Key: HBASE-6377 > URL: https://issues.apache.org/jira/browse/HBASE-6377 > Project: HBase > Issue Type: Bug > Components: metrics, regionserver >Affects Versions: 0.94.1, 0.95.2 >Reporter: Andrew Purtell >Assignee: Andrew Purtell > Fix For: 0.94.1, 0.95.0 > > Attachments: 6377-0.94-remove-get-put-delete-histograms.patch, > 6377-0.94.patch, 6377-trunk-remove-get-put-delete-histograms.patch, > 6377-trunk-simple.patch, 6377.patch > > > A client application (LoadTestTool) calls put() on HTables. Internally to the > HBase client those puts are batched into MultiActions. The total number of > put operations shown in the RegionServer's put metrics histogram never > increases from 0 even though millions of such operations are made. Needless > to say the latency for those operations are not measured either. The value of > HBASE-5533 metrics are suspect given the client will batch put and delete ops > like this. > I had a fix in progress but HBASE-6284 messed it up. Before, MultiAction > processing in HRegionServer would distingush between puts and deletes and > dispatch them separately. It was easy to account for the time for them. Now > both puts and deletes are submitted in batch together as mutations. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-6377) HBASE-5533 metrics miss all operations submitted via MultiAction
[ https://issues.apache.org/jira/browse/HBASE-6377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14077315#comment-14077315 ] Misty Stanley-Jones commented on HBASE-6377: Thanks, I will work something up. This is very helpful. > HBASE-5533 metrics miss all operations submitted via MultiAction > > > Key: HBASE-6377 > URL: https://issues.apache.org/jira/browse/HBASE-6377 > Project: HBase > Issue Type: Bug > Components: metrics, regionserver >Affects Versions: 0.94.1, 0.95.2 >Reporter: Andrew Purtell >Assignee: Andrew Purtell > Fix For: 0.94.1, 0.95.0 > > Attachments: 6377-0.94-remove-get-put-delete-histograms.patch, > 6377-0.94.patch, 6377-trunk-remove-get-put-delete-histograms.patch, > 6377-trunk-simple.patch, 6377.patch > > > A client application (LoadTestTool) calls put() on HTables. Internally to the > HBase client those puts are batched into MultiActions. The total number of > put operations shown in the RegionServer's put metrics histogram never > increases from 0 even though millions of such operations are made. Needless > to say the latency for those operations are not measured either. The value of > HBASE-5533 metrics are suspect given the client will batch put and delete ops > like this. > I had a fix in progress but HBASE-6284 messed it up. Before, MultiAction > processing in HRegionServer would distingush between puts and deletes and > dispatch them separately. It was easy to account for the time for them. Now > both puts and deletes are submitted in batch together as mutations. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-6377) HBASE-5533 metrics miss all operations submitted via MultiAction
[ https://issues.apache.org/jira/browse/HBASE-6377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13428712#comment-13428712 ] Hudson commented on HBASE-6377: --- Integrated in HBase-0.94-security-on-Hadoop-23 #6 (See [https://builds.apache.org/job/HBase-0.94-security-on-Hadoop-23/6/]) HBASE-6377. HBASE-5533 metrics miss all operations submitted via MultiAction Applied 6377-0.94-remove-get-put-delete-histograms.patch (Revision 1361028) Result = FAILURE apurtell : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/RegionServerMetrics.java > HBASE-5533 metrics miss all operations submitted via MultiAction > > > Key: HBASE-6377 > URL: https://issues.apache.org/jira/browse/HBASE-6377 > Project: HBase > Issue Type: Bug > Components: metrics, regionserver >Affects Versions: 0.96.0, 0.94.1 >Reporter: Andrew Purtell >Assignee: Andrew Purtell > Fix For: 0.96.0, 0.94.1 > > Attachments: 6377-0.94-remove-get-put-delete-histograms.patch, > 6377-0.94.patch, 6377-trunk-remove-get-put-delete-histograms.patch, > 6377-trunk-simple.patch, 6377.patch > > > A client application (LoadTestTool) calls put() on HTables. Internally to the > HBase client those puts are batched into MultiActions. The total number of > put operations shown in the RegionServer's put metrics histogram never > increases from 0 even though millions of such operations are made. Needless > to say the latency for those operations are not measured either. The value of > HBASE-5533 metrics are suspect given the client will batch put and delete ops > like this. > I had a fix in progress but HBASE-6284 messed it up. Before, MultiAction > processing in HRegionServer would distingush between puts and deletes and > dispatch them separately. It was easy to account for the time for them. Now > both puts and deletes are submitted in batch together as mutations. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6377) HBASE-5533 metrics miss all operations submitted via MultiAction
[ https://issues.apache.org/jira/browse/HBASE-6377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13413674#comment-13413674 ] Hudson commented on HBASE-6377: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #92 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/92/]) HBASE-6377. HBASE-5533 metrics miss all operations submitted via MultiAction Committed 6377-trunk-remove-get-put-delete-histograms.patch (Revision 1361026) Result = FAILURE apurtell : Files : * /hbase/trunk/hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/regionserver/ServerMetricsTmpl.jamon * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/RegionServerMetrics.java > HBASE-5533 metrics miss all operations submitted via MultiAction > > > Key: HBASE-6377 > URL: https://issues.apache.org/jira/browse/HBASE-6377 > Project: HBase > Issue Type: Bug > Components: metrics, regionserver >Affects Versions: 0.96.0, 0.94.1 >Reporter: Andrew Purtell >Assignee: Andrew Purtell > Fix For: 0.96.0, 0.94.1 > > Attachments: 6377-0.94-remove-get-put-delete-histograms.patch, > 6377-0.94.patch, 6377-trunk-remove-get-put-delete-histograms.patch, > 6377-trunk-simple.patch, 6377.patch > > > A client application (LoadTestTool) calls put() on HTables. Internally to the > HBase client those puts are batched into MultiActions. The total number of > put operations shown in the RegionServer's put metrics histogram never > increases from 0 even though millions of such operations are made. Needless > to say the latency for those operations are not measured either. The value of > HBASE-5533 metrics are suspect given the client will batch put and delete ops > like this. > I had a fix in progress but HBASE-6284 messed it up. Before, MultiAction > processing in HRegionServer would distingush between puts and deletes and > dispatch them separately. It was easy to account for the time for them. Now > both puts and deletes are submitted in batch together as mutations. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6377) HBASE-5533 metrics miss all operations submitted via MultiAction
[ https://issues.apache.org/jira/browse/HBASE-6377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13413435#comment-13413435 ] Hudson commented on HBASE-6377: --- Integrated in HBase-TRUNK #3122 (See [https://builds.apache.org/job/HBase-TRUNK/3122/]) HBASE-6377. HBASE-5533 metrics miss all operations submitted via MultiAction Committed 6377-trunk-remove-get-put-delete-histograms.patch (Revision 1361026) Result = FAILURE apurtell : Files : * /hbase/trunk/hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/regionserver/ServerMetricsTmpl.jamon * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/RegionServerMetrics.java > HBASE-5533 metrics miss all operations submitted via MultiAction > > > Key: HBASE-6377 > URL: https://issues.apache.org/jira/browse/HBASE-6377 > Project: HBase > Issue Type: Bug > Components: metrics, regionserver >Affects Versions: 0.96.0, 0.94.1 >Reporter: Andrew Purtell >Assignee: Andrew Purtell > Fix For: 0.96.0, 0.94.1 > > Attachments: 6377-0.94-remove-get-put-delete-histograms.patch, > 6377-0.94.patch, 6377-trunk-remove-get-put-delete-histograms.patch, > 6377-trunk-simple.patch, 6377.patch > > > A client application (LoadTestTool) calls put() on HTables. Internally to the > HBase client those puts are batched into MultiActions. The total number of > put operations shown in the RegionServer's put metrics histogram never > increases from 0 even though millions of such operations are made. Needless > to say the latency for those operations are not measured either. The value of > HBASE-5533 metrics are suspect given the client will batch put and delete ops > like this. > I had a fix in progress but HBASE-6284 messed it up. Before, MultiAction > processing in HRegionServer would distingush between puts and deletes and > dispatch them separately. It was easy to account for the time for them. Now > both puts and deletes are submitted in batch together as mutations. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6377) HBASE-5533 metrics miss all operations submitted via MultiAction
[ https://issues.apache.org/jira/browse/HBASE-6377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13413431#comment-13413431 ] Hudson commented on HBASE-6377: --- Integrated in HBase-0.94 #313 (See [https://builds.apache.org/job/HBase-0.94/313/]) HBASE-6377. HBASE-5533 metrics miss all operations submitted via MultiAction Applied 6377-0.94-remove-get-put-delete-histograms.patch (Revision 1361028) Result = FAILURE apurtell : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/RegionServerMetrics.java > HBASE-5533 metrics miss all operations submitted via MultiAction > > > Key: HBASE-6377 > URL: https://issues.apache.org/jira/browse/HBASE-6377 > Project: HBase > Issue Type: Bug > Components: metrics, regionserver >Affects Versions: 0.96.0, 0.94.1 >Reporter: Andrew Purtell >Assignee: Andrew Purtell > Fix For: 0.96.0, 0.94.1 > > Attachments: 6377-0.94-remove-get-put-delete-histograms.patch, > 6377-0.94.patch, 6377-trunk-remove-get-put-delete-histograms.patch, > 6377-trunk-simple.patch, 6377.patch > > > A client application (LoadTestTool) calls put() on HTables. Internally to the > HBase client those puts are batched into MultiActions. The total number of > put operations shown in the RegionServer's put metrics histogram never > increases from 0 even though millions of such operations are made. Needless > to say the latency for those operations are not measured either. The value of > HBASE-5533 metrics are suspect given the client will batch put and delete ops > like this. > I had a fix in progress but HBASE-6284 messed it up. Before, MultiAction > processing in HRegionServer would distingush between puts and deletes and > dispatch them separately. It was easy to account for the time for them. Now > both puts and deletes are submitted in batch together as mutations. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6377) HBASE-5533 metrics miss all operations submitted via MultiAction
[ https://issues.apache.org/jira/browse/HBASE-6377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13413417#comment-13413417 ] Hudson commented on HBASE-6377: --- Integrated in HBase-0.94-security #40 (See [https://builds.apache.org/job/HBase-0.94-security/40/]) HBASE-6377. HBASE-5533 metrics miss all operations submitted via MultiAction Applied 6377-0.94-remove-get-put-delete-histograms.patch (Revision 1361028) Result = SUCCESS apurtell : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/RegionServerMetrics.java > HBASE-5533 metrics miss all operations submitted via MultiAction > > > Key: HBASE-6377 > URL: https://issues.apache.org/jira/browse/HBASE-6377 > Project: HBase > Issue Type: Bug > Components: metrics, regionserver >Affects Versions: 0.96.0, 0.94.1 >Reporter: Andrew Purtell >Assignee: Andrew Purtell > Fix For: 0.96.0, 0.94.1 > > Attachments: 6377-0.94-remove-get-put-delete-histograms.patch, > 6377-0.94.patch, 6377-trunk-remove-get-put-delete-histograms.patch, > 6377-trunk-simple.patch, 6377.patch > > > A client application (LoadTestTool) calls put() on HTables. Internally to the > HBase client those puts are batched into MultiActions. The total number of > put operations shown in the RegionServer's put metrics histogram never > increases from 0 even though millions of such operations are made. Needless > to say the latency for those operations are not measured either. The value of > HBASE-5533 metrics are suspect given the client will batch put and delete ops > like this. > I had a fix in progress but HBASE-6284 messed it up. Before, MultiAction > processing in HRegionServer would distingush between puts and deletes and > dispatch them separately. It was easy to account for the time for them. Now > both puts and deletes are submitted in batch together as mutations. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6377) HBASE-5533 metrics miss all operations submitted via MultiAction
[ https://issues.apache.org/jira/browse/HBASE-6377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13413385#comment-13413385 ] Elliott Clark commented on HBASE-6377: -- @Andrew They're exposed through jmx too. All the more reason to get them correct if we're going to do them. +1 on trunk patch. After 4050 goes in I'm going to try and get to some of the metric re-writes that are needed. Looks like this will be one on the list. > HBASE-5533 metrics miss all operations submitted via MultiAction > > > Key: HBASE-6377 > URL: https://issues.apache.org/jira/browse/HBASE-6377 > Project: HBase > Issue Type: Bug > Components: metrics, regionserver >Affects Versions: 0.96.0, 0.94.1 >Reporter: Andrew Purtell >Assignee: Andrew Purtell > Fix For: 0.94.1 > > Attachments: 6377-0.94-remove-get-put-delete-histograms.patch, > 6377-0.94.patch, 6377-trunk-remove-get-put-delete-histograms.patch, > 6377-trunk-simple.patch, 6377.patch > > > A client application (LoadTestTool) calls put() on HTables. Internally to the > HBase client those puts are batched into MultiActions. The total number of > put operations shown in the RegionServer's put metrics histogram never > increases from 0 even though millions of such operations are made. Needless > to say the latency for those operations are not measured either. The value of > HBASE-5533 metrics are suspect given the client will batch put and delete ops > like this. > I had a fix in progress but HBASE-6284 messed it up. Before, MultiAction > processing in HRegionServer would distingush between puts and deletes and > dispatch them separately. It was easy to account for the time for them. Now > both puts and deletes are submitted in batch together as mutations. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6377) HBASE-5533 metrics miss all operations submitted via MultiAction
[ https://issues.apache.org/jira/browse/HBASE-6377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13413370#comment-13413370 ] Andrew Purtell commented on HBASE-6377: --- @Elliot we should do the same on trunk and fix up the UI. > HBASE-5533 metrics miss all operations submitted via MultiAction > > > Key: HBASE-6377 > URL: https://issues.apache.org/jira/browse/HBASE-6377 > Project: HBase > Issue Type: Bug > Components: metrics, regionserver >Affects Versions: 0.96.0, 0.94.1 >Reporter: Andrew Purtell >Assignee: Andrew Purtell > Fix For: 0.94.1 > > Attachments: 6377-0.94-remove-get-put-delete-histograms.patch, > 6377-0.94.patch, 6377-trunk-simple.patch, 6377.patch > > > A client application (LoadTestTool) calls put() on HTables. Internally to the > HBase client those puts are batched into MultiActions. The total number of > put operations shown in the RegionServer's put metrics histogram never > increases from 0 even though millions of such operations are made. Needless > to say the latency for those operations are not measured either. The value of > HBASE-5533 metrics are suspect given the client will batch put and delete ops > like this. > I had a fix in progress but HBASE-6284 messed it up. Before, MultiAction > processing in HRegionServer would distingush between puts and deletes and > dispatch them separately. It was easy to account for the time for them. Now > both puts and deletes are submitted in batch together as mutations. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6377) HBASE-5533 metrics miss all operations submitted via MultiAction
[ https://issues.apache.org/jira/browse/HBASE-6377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13413368#comment-13413368 ] Elliott Clark commented on HBASE-6377: -- Looks good to me. That was fast. Thanks > HBASE-5533 metrics miss all operations submitted via MultiAction > > > Key: HBASE-6377 > URL: https://issues.apache.org/jira/browse/HBASE-6377 > Project: HBase > Issue Type: Bug > Components: metrics, regionserver >Affects Versions: 0.96.0, 0.94.1 >Reporter: Andrew Purtell >Assignee: Andrew Purtell > Fix For: 0.94.1 > > Attachments: 6377-0.94-remove-get-put-delete-histograms.patch, > 6377-0.94.patch, 6377-trunk-simple.patch, 6377.patch > > > A client application (LoadTestTool) calls put() on HTables. Internally to the > HBase client those puts are batched into MultiActions. The total number of > put operations shown in the RegionServer's put metrics histogram never > increases from 0 even though millions of such operations are made. Needless > to say the latency for those operations are not measured either. The value of > HBASE-5533 metrics are suspect given the client will batch put and delete ops > like this. > I had a fix in progress but HBASE-6284 messed it up. Before, MultiAction > processing in HRegionServer would distingush between puts and deletes and > dispatch them separately. It was easy to account for the time for them. Now > both puts and deletes are submitted in batch together as mutations. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6377) HBASE-5533 metrics miss all operations submitted via MultiAction
[ https://issues.apache.org/jira/browse/HBASE-6377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13413364#comment-13413364 ] Lars Hofhansl commented on HBASE-6377: -- +1 on the "remove-get-put-delete-histograms" patch > HBASE-5533 metrics miss all operations submitted via MultiAction > > > Key: HBASE-6377 > URL: https://issues.apache.org/jira/browse/HBASE-6377 > Project: HBase > Issue Type: Bug > Components: metrics, regionserver >Affects Versions: 0.96.0, 0.94.1 >Reporter: Andrew Purtell >Assignee: Andrew Purtell > Fix For: 0.94.1 > > Attachments: 6377-0.94-remove-get-put-delete-histograms.patch, > 6377-0.94.patch, 6377-trunk-simple.patch, 6377.patch > > > A client application (LoadTestTool) calls put() on HTables. Internally to the > HBase client those puts are batched into MultiActions. The total number of > put operations shown in the RegionServer's put metrics histogram never > increases from 0 even though millions of such operations are made. Needless > to say the latency for those operations are not measured either. The value of > HBASE-5533 metrics are suspect given the client will batch put and delete ops > like this. > I had a fix in progress but HBASE-6284 messed it up. Before, MultiAction > processing in HRegionServer would distingush between puts and deletes and > dispatch them separately. It was easy to account for the time for them. Now > both puts and deletes are submitted in batch together as mutations. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6377) HBASE-5533 metrics miss all operations submitted via MultiAction
[ https://issues.apache.org/jira/browse/HBASE-6377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13413361#comment-13413361 ] Andrew Purtell commented on HBASE-6377: --- Attached. You can also disregard the trunk patch on this issue if not attempting to preserve the old stuff. > HBASE-5533 metrics miss all operations submitted via MultiAction > > > Key: HBASE-6377 > URL: https://issues.apache.org/jira/browse/HBASE-6377 > Project: HBase > Issue Type: Bug > Components: metrics, regionserver >Affects Versions: 0.96.0, 0.94.1 >Reporter: Andrew Purtell >Assignee: Andrew Purtell > Fix For: 0.94.1 > > Attachments: 6377-0.94-remove-get-put-delete-histograms.patch, > 6377-0.94.patch, 6377-trunk-simple.patch, 6377.patch > > > A client application (LoadTestTool) calls put() on HTables. Internally to the > HBase client those puts are batched into MultiActions. The total number of > put operations shown in the RegionServer's put metrics histogram never > increases from 0 even though millions of such operations are made. Needless > to say the latency for those operations are not measured either. The value of > HBASE-5533 metrics are suspect given the client will batch put and delete ops > like this. > I had a fix in progress but HBASE-6284 messed it up. Before, MultiAction > processing in HRegionServer would distingush between puts and deletes and > dispatch them separately. It was easy to account for the time for them. Now > both puts and deletes are submitted in batch together as mutations. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6377) HBASE-5533 metrics miss all operations submitted via MultiAction
[ https://issues.apache.org/jira/browse/HBASE-6377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13413357#comment-13413357 ] Andrew Purtell commented on HBASE-6377: --- Putting up a 0.94 patch that pulls the dodgy metrics in a few minutes. > HBASE-5533 metrics miss all operations submitted via MultiAction > > > Key: HBASE-6377 > URL: https://issues.apache.org/jira/browse/HBASE-6377 > Project: HBase > Issue Type: Bug > Components: metrics, regionserver >Affects Versions: 0.96.0, 0.94.1 >Reporter: Andrew Purtell >Assignee: Andrew Purtell > Fix For: 0.94.2 > > Attachments: 6377-0.94.patch, 6377-trunk-simple.patch, 6377.patch > > > A client application (LoadTestTool) calls put() on HTables. Internally to the > HBase client those puts are batched into MultiActions. The total number of > put operations shown in the RegionServer's put metrics histogram never > increases from 0 even though millions of such operations are made. Needless > to say the latency for those operations are not measured either. The value of > HBASE-5533 metrics are suspect given the client will batch put and delete ops > like this. > I had a fix in progress but HBASE-6284 messed it up. Before, MultiAction > processing in HRegionServer would distingush between puts and deletes and > dispatch them separately. It was easy to account for the time for them. Now > both puts and deletes are submitted in batch together as mutations. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6377) HBASE-5533 metrics miss all operations submitted via MultiAction
[ https://issues.apache.org/jira/browse/HBASE-6377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13413353#comment-13413353 ] Elliott Clark commented on HBASE-6377: -- That would be my vote for 0.94 at least. > HBASE-5533 metrics miss all operations submitted via MultiAction > > > Key: HBASE-6377 > URL: https://issues.apache.org/jira/browse/HBASE-6377 > Project: HBase > Issue Type: Bug > Components: metrics, regionserver >Affects Versions: 0.96.0, 0.94.1 >Reporter: Andrew Purtell >Assignee: Andrew Purtell > Fix For: 0.94.2 > > Attachments: 6377-0.94.patch, 6377-trunk-simple.patch, 6377.patch > > > A client application (LoadTestTool) calls put() on HTables. Internally to the > HBase client those puts are batched into MultiActions. The total number of > put operations shown in the RegionServer's put metrics histogram never > increases from 0 even though millions of such operations are made. Needless > to say the latency for those operations are not measured either. The value of > HBASE-5533 metrics are suspect given the client will batch put and delete ops > like this. > I had a fix in progress but HBASE-6284 messed it up. Before, MultiAction > processing in HRegionServer would distingush between puts and deletes and > dispatch them separately. It was easy to account for the time for them. Now > both puts and deletes are submitted in batch together as mutations. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6377) HBASE-5533 metrics miss all operations submitted via MultiAction
[ https://issues.apache.org/jira/browse/HBASE-6377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13413298#comment-13413298 ] Andrew Purtell commented on HBASE-6377: --- bq. The histogram could be dominated by the bulk insert operation. If you mean Put(List) or MultiAction, each dispatch down to HRegion is individually accounted. A big limitation we have with this approach of measuring at HRI is we can't drop down into the region to account for each op individually. Current 0.94 patch maintains how the HBASE-5533 patch did it. I believe such issues of larger rethinking were what Lars had in mind in comments above. TBD. > HBASE-5533 metrics miss all operations submitted via MultiAction > > > Key: HBASE-6377 > URL: https://issues.apache.org/jira/browse/HBASE-6377 > Project: HBase > Issue Type: Bug > Components: metrics, regionserver >Affects Versions: 0.96.0, 0.94.1 >Reporter: Andrew Purtell >Assignee: Andrew Purtell > Fix For: 0.94.2 > > Attachments: 6377-0.94.patch, 6377-trunk-simple.patch, 6377.patch > > > A client application (LoadTestTool) calls put() on HTables. Internally to the > HBase client those puts are batched into MultiActions. The total number of > put operations shown in the RegionServer's put metrics histogram never > increases from 0 even though millions of such operations are made. Needless > to say the latency for those operations are not measured either. The value of > HBASE-5533 metrics are suspect given the client will batch put and delete ops > like this. > I had a fix in progress but HBASE-6284 messed it up. Before, MultiAction > processing in HRegionServer would distingush between puts and deletes and > dispatch them separately. It was easy to account for the time for them. Now > both puts and deletes are submitted in batch together as mutations. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6377) HBASE-5533 metrics miss all operations submitted via MultiAction
[ https://issues.apache.org/jira/browse/HBASE-6377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13413300#comment-13413300 ] Andrew Purtell commented on HBASE-6377: --- If you guys think this is not worth it, then we should yank it out. > HBASE-5533 metrics miss all operations submitted via MultiAction > > > Key: HBASE-6377 > URL: https://issues.apache.org/jira/browse/HBASE-6377 > Project: HBase > Issue Type: Bug > Components: metrics, regionserver >Affects Versions: 0.96.0, 0.94.1 >Reporter: Andrew Purtell >Assignee: Andrew Purtell > Fix For: 0.94.2 > > Attachments: 6377-0.94.patch, 6377-trunk-simple.patch, 6377.patch > > > A client application (LoadTestTool) calls put() on HTables. Internally to the > HBase client those puts are batched into MultiActions. The total number of > put operations shown in the RegionServer's put metrics histogram never > increases from 0 even though millions of such operations are made. Needless > to say the latency for those operations are not measured either. The value of > HBASE-5533 metrics are suspect given the client will batch put and delete ops > like this. > I had a fix in progress but HBASE-6284 messed it up. Before, MultiAction > processing in HRegionServer would distingush between puts and deletes and > dispatch them separately. It was easy to account for the time for them. Now > both puts and deletes are submitted in batch together as mutations. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6377) HBASE-5533 metrics miss all operations submitted via MultiAction
[ https://issues.apache.org/jira/browse/HBASE-6377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13413299#comment-13413299 ] Andrew Purtell commented on HBASE-6377: --- Again, the 0.94 patch just puts back HBASE-5533 style measurements, not reinventing *proper* accounting. > HBASE-5533 metrics miss all operations submitted via MultiAction > > > Key: HBASE-6377 > URL: https://issues.apache.org/jira/browse/HBASE-6377 > Project: HBase > Issue Type: Bug > Components: metrics, regionserver >Affects Versions: 0.96.0, 0.94.1 >Reporter: Andrew Purtell >Assignee: Andrew Purtell > Fix For: 0.94.2 > > Attachments: 6377-0.94.patch, 6377-trunk-simple.patch, 6377.patch > > > A client application (LoadTestTool) calls put() on HTables. Internally to the > HBase client those puts are batched into MultiActions. The total number of > put operations shown in the RegionServer's put metrics histogram never > increases from 0 even though millions of such operations are made. Needless > to say the latency for those operations are not measured either. The value of > HBASE-5533 metrics are suspect given the client will batch put and delete ops > like this. > I had a fix in progress but HBASE-6284 messed it up. Before, MultiAction > processing in HRegionServer would distingush between puts and deletes and > dispatch them separately. It was easy to account for the time for them. Now > both puts and deletes are submitted in batch together as mutations. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6377) HBASE-5533 metrics miss all operations submitted via MultiAction
[ https://issues.apache.org/jira/browse/HBASE-6377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13413296#comment-13413296 ] Andrew Wang commented on HBASE-6377: Please correct my understanding on this if I'm wrong, but estimating via the average (total latency / num ops) is pretty undesirable for both normal MultiActions and the batched put MultiAction case. Latency from a slow op bleeds over to a fast one, which messes up per-op metrics. This would also affect doing per-region or per-column family metrics in the future, for the same reasons. I haven't looked at the code, but is it possible to do more accurate accounting of the latency of each op in a MultiAction? If so, I think it'd be worthwhile. > HBASE-5533 metrics miss all operations submitted via MultiAction > > > Key: HBASE-6377 > URL: https://issues.apache.org/jira/browse/HBASE-6377 > Project: HBase > Issue Type: Bug > Components: metrics, regionserver >Affects Versions: 0.96.0, 0.94.1 >Reporter: Andrew Purtell >Assignee: Andrew Purtell > Fix For: 0.94.2 > > Attachments: 6377-0.94.patch, 6377-trunk-simple.patch, 6377.patch > > > A client application (LoadTestTool) calls put() on HTables. Internally to the > HBase client those puts are batched into MultiActions. The total number of > put operations shown in the RegionServer's put metrics histogram never > increases from 0 even though millions of such operations are made. Needless > to say the latency for those operations are not measured either. The value of > HBASE-5533 metrics are suspect given the client will batch put and delete ops > like this. > I had a fix in progress but HBASE-6284 messed it up. Before, MultiAction > processing in HRegionServer would distingush between puts and deletes and > dispatch them separately. It was easy to account for the time for them. Now > both puts and deletes are submitted in batch together as mutations. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6377) HBASE-5533 metrics miss all operations submitted via MultiAction
[ https://issues.apache.org/jira/browse/HBASE-6377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13413287#comment-13413287 ] Elliott Clark commented on HBASE-6377: -- The trunk patch looks great. The 0.94 patch, Will give some really weird numbers I think. The histogram could be dominated by the bulk insert operation. Should that be switched to just a PersistantTimeVaryingRate? Better to give less information, but for it to be accurate. > HBASE-5533 metrics miss all operations submitted via MultiAction > > > Key: HBASE-6377 > URL: https://issues.apache.org/jira/browse/HBASE-6377 > Project: HBase > Issue Type: Bug > Components: metrics, regionserver >Affects Versions: 0.96.0, 0.94.1 >Reporter: Andrew Purtell >Assignee: Andrew Purtell > Fix For: 0.94.2 > > Attachments: 6377-0.94.patch, 6377-trunk-simple.patch, 6377.patch > > > A client application (LoadTestTool) calls put() on HTables. Internally to the > HBase client those puts are batched into MultiActions. The total number of > put operations shown in the RegionServer's put metrics histogram never > increases from 0 even though millions of such operations are made. Needless > to say the latency for those operations are not measured either. The value of > HBASE-5533 metrics are suspect given the client will batch put and delete ops > like this. > I had a fix in progress but HBASE-6284 messed it up. Before, MultiAction > processing in HRegionServer would distingush between puts and deletes and > dispatch them separately. It was easy to account for the time for them. Now > both puts and deletes are submitted in batch together as mutations. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6377) HBASE-5533 metrics miss all operations submitted via MultiAction
[ https://issues.apache.org/jira/browse/HBASE-6377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13413234#comment-13413234 ] Andrew Purtell commented on HBASE-6377: --- Tangentially, if we're doing all over the place a lot of this get nanos then get them again and subtract the diff, we might want to just use Guava's Stopwatch. Also, in our internal version of 0.94 we've scaled metrics to milliseconds. The Stopwatch javadocs talk about that: "Note that the overhead of measurement can be more than a microsecond, so it is generally not useful to specify TimeUnit.NANOSECONDS precision here." For another JIRA perhaps. > HBASE-5533 metrics miss all operations submitted via MultiAction > > > Key: HBASE-6377 > URL: https://issues.apache.org/jira/browse/HBASE-6377 > Project: HBase > Issue Type: Bug > Components: metrics, regionserver >Affects Versions: 0.96.0, 0.94.1 >Reporter: Andrew Purtell >Assignee: Andrew Purtell > Fix For: 0.94.2 > > Attachments: 6377-0.94.patch, 6377-trunk-simple.patch, 6377.patch > > > A client application (LoadTestTool) calls put() on HTables. Internally to the > HBase client those puts are batched into MultiActions. The total number of > put operations shown in the RegionServer's put metrics histogram never > increases from 0 even though millions of such operations are made. Needless > to say the latency for those operations are not measured either. The value of > HBASE-5533 metrics are suspect given the client will batch put and delete ops > like this. > I had a fix in progress but HBASE-6284 messed it up. Before, MultiAction > processing in HRegionServer would distingush between puts and deletes and > dispatch them separately. It was easy to account for the time for them. Now > both puts and deletes are submitted in batch together as mutations. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6377) HBASE-5533 metrics miss all operations submitted via MultiAction
[ https://issues.apache.org/jira/browse/HBASE-6377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13413232#comment-13413232 ] stack commented on HBASE-6377: -- If its wonky, it goes w/o saying that you didn't do it. But yeah, wonky. Patch seems fine to me. > HBASE-5533 metrics miss all operations submitted via MultiAction > > > Key: HBASE-6377 > URL: https://issues.apache.org/jira/browse/HBASE-6377 > Project: HBase > Issue Type: Bug > Components: metrics, regionserver >Affects Versions: 0.96.0, 0.94.1 >Reporter: Andrew Purtell >Assignee: Andrew Purtell > Fix For: 0.94.2 > > Attachments: 6377-0.94.patch, 6377-trunk-simple.patch, 6377.patch > > > A client application (LoadTestTool) calls put() on HTables. Internally to the > HBase client those puts are batched into MultiActions. The total number of > put operations shown in the RegionServer's put metrics histogram never > increases from 0 even though millions of such operations are made. Needless > to say the latency for those operations are not measured either. The value of > HBASE-5533 metrics are suspect given the client will batch put and delete ops > like this. > I had a fix in progress but HBASE-6284 messed it up. Before, MultiAction > processing in HRegionServer would distingush between puts and deletes and > dispatch them separately. It was easy to account for the time for them. Now > both puts and deletes are submitted in batch together as mutations. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6377) HBASE-5533 metrics miss all operations submitted via MultiAction
[
https://issues.apache.org/jira/browse/HBASE-6377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13413225#comment-13413225
]
Andrew Purtell commented on HBASE-6377:
---
{quote}
bq. But, as before, for N ops you have to submit (totalTime/N) N times to the
histogram.
That seems wonky Mr Purtell?
{quote}
Not my doing Mr. Stack, it's a histogram implementation detail. :-)
> HBASE-5533 metrics miss all operations submitted via MultiAction
>
>
> Key: HBASE-6377
> URL: https://issues.apache.org/jira/browse/HBASE-6377
> Project: HBase
> Issue Type: Bug
> Components: metrics, regionserver
>Affects Versions: 0.96.0, 0.94.1
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
> Fix For: 0.94.2
>
> Attachments: 6377-0.94.patch, 6377-trunk-simple.patch, 6377.patch
>
>
> A client application (LoadTestTool) calls put() on HTables. Internally to the
> HBase client those puts are batched into MultiActions. The total number of
> put operations shown in the RegionServer's put metrics histogram never
> increases from 0 even though millions of such operations are made. Needless
> to say the latency for those operations are not measured either. The value of
> HBASE-5533 metrics are suspect given the client will batch put and delete ops
> like this.
> I had a fix in progress but HBASE-6284 messed it up. Before, MultiAction
> processing in HRegionServer would distingush between puts and deletes and
> dispatch them separately. It was easy to account for the time for them. Now
> both puts and deletes are submitted in batch together as mutations.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6377) HBASE-5533 metrics miss all operations submitted via MultiAction
[ https://issues.apache.org/jira/browse/HBASE-6377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13413218#comment-13413218 ] stack commented on HBASE-6377: -- bq. Doing that. But, as before, for N ops you have to submit (totalTime/N) N times to the histogram. That seems wonky Mr Purtell? Patch seems fine to me. Mr Elliott, you have carnal knowledge of metrics, any opinion? > HBASE-5533 metrics miss all operations submitted via MultiAction > > > Key: HBASE-6377 > URL: https://issues.apache.org/jira/browse/HBASE-6377 > Project: HBase > Issue Type: Bug > Components: metrics, regionserver >Affects Versions: 0.96.0, 0.94.1 >Reporter: Andrew Purtell >Assignee: Andrew Purtell > Fix For: 0.94.2 > > Attachments: 6377-trunk-simple.patch > > > A client application (LoadTestTool) calls put() on HTables. Internally to the > HBase client those puts are batched into MultiActions. The total number of > put operations shown in the RegionServer's put metrics histogram never > increases from 0 even though millions of such operations are made. Needless > to say the latency for those operations are not measured either. The value of > HBASE-5533 metrics are suspect given the client will batch put and delete ops > like this. > I had a fix in progress but HBASE-6284 messed it up. Before, MultiAction > processing in HRegionServer would distingush between puts and deletes and > dispatch them separately. It was easy to account for the time for them. Now > both puts and deletes are submitted in batch together as mutations. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6377) HBASE-5533 metrics miss all operations submitted via MultiAction
[ https://issues.apache.org/jira/browse/HBASE-6377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13413163#comment-13413163 ] Andrew Purtell commented on HBASE-6377: --- bq. We can report per op latency just by dividing the batch latency by the batch size Doing that. But, as before, for N ops you have to submit (totalTime/N) N times to the histogram. > HBASE-5533 metrics miss all operations submitted via MultiAction > > > Key: HBASE-6377 > URL: https://issues.apache.org/jira/browse/HBASE-6377 > Project: HBase > Issue Type: Bug > Components: metrics, regionserver >Affects Versions: 0.96.0, 0.94.1 >Reporter: Andrew Purtell >Assignee: Andrew Purtell > Fix For: 0.94.2 > > Attachments: 6377-trunk-simple.patch > > > A client application (LoadTestTool) calls put() on HTables. Internally to the > HBase client those puts are batched into MultiActions. The total number of > put operations shown in the RegionServer's put metrics histogram never > increases from 0 even though millions of such operations are made. Needless > to say the latency for those operations are not measured either. The value of > HBASE-5533 metrics are suspect given the client will batch put and delete ops > like this. > I had a fix in progress but HBASE-6284 messed it up. Before, MultiAction > processing in HRegionServer would distingush between puts and deletes and > dispatch them separately. It was easy to account for the time for them. Now > both puts and deletes are submitted in batch together as mutations. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6377) HBASE-5533 metrics miss all operations submitted via MultiAction
[ https://issues.apache.org/jira/browse/HBASE-6377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13413106#comment-13413106 ] Lars Hofhansl commented on HBASE-6377: -- I think approach is good, though. Let's just do your patch and then regroup :) > HBASE-5533 metrics miss all operations submitted via MultiAction > > > Key: HBASE-6377 > URL: https://issues.apache.org/jira/browse/HBASE-6377 > Project: HBase > Issue Type: Bug > Components: metrics, regionserver >Affects Versions: 0.96.0, 0.94.1 >Reporter: Andrew Purtell >Assignee: Andrew Purtell > Fix For: 0.94.2 > > Attachments: 6377-trunk-simple.patch > > > A client application (LoadTestTool) calls put() on HTables. Internally to the > HBase client those puts are batched into MultiActions. The total number of > put operations shown in the RegionServer's put metrics histogram never > increases from 0 even though millions of such operations are made. Needless > to say the latency for those operations are not measured either. The value of > HBASE-5533 metrics are suspect given the client will batch put and delete ops > like this. > I had a fix in progress but HBASE-6284 messed it up. Before, MultiAction > processing in HRegionServer would distingush between puts and deletes and > dispatch them separately. It was easy to account for the time for them. Now > both puts and deletes are submitted in batch together as mutations. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6377) HBASE-5533 metrics miss all operations submitted via MultiAction
[ https://issues.apache.org/jira/browse/HBASE-6377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13413104#comment-13413104 ] Lars Hofhansl commented on HBASE-6377: -- Looks good to me. Reasoning makes sense. We can report per of latency just by dividing the batch latency by the batch size (still no need to deconstruct per individual op). Not sure if that more useful or less. Does this need more discussion? If so, maybe the best course of action is to revert HBASE-5533 now and regroup. > HBASE-5533 metrics miss all operations submitted via MultiAction > > > Key: HBASE-6377 > URL: https://issues.apache.org/jira/browse/HBASE-6377 > Project: HBase > Issue Type: Bug > Components: metrics, regionserver >Affects Versions: 0.96.0, 0.94.1 >Reporter: Andrew Purtell >Assignee: Andrew Purtell > Fix For: 0.94.2 > > Attachments: 6377-trunk-simple.patch > > > A client application (LoadTestTool) calls put() on HTables. Internally to the > HBase client those puts are batched into MultiActions. The total number of > put operations shown in the RegionServer's put metrics histogram never > increases from 0 even though millions of such operations are made. Needless > to say the latency for those operations are not measured either. The value of > HBASE-5533 metrics are suspect given the client will batch put and delete ops > like this. > I had a fix in progress but HBASE-6284 messed it up. Before, MultiAction > processing in HRegionServer would distingush between puts and deletes and > dispatch them separately. It was easy to account for the time for them. Now > both puts and deletes are submitted in batch together as mutations. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6377) HBASE-5533 metrics miss all operations submitted via MultiAction
[ https://issues.apache.org/jira/browse/HBASE-6377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13413098#comment-13413098 ] Andrew Purtell commented on HBASE-6377: --- Let me know if you want me to proceed with this approach; I'll backport to 0.94 and commit trunk and 0.94 branch. > HBASE-5533 metrics miss all operations submitted via MultiAction > > > Key: HBASE-6377 > URL: https://issues.apache.org/jira/browse/HBASE-6377 > Project: HBase > Issue Type: Bug > Components: metrics, regionserver >Affects Versions: 0.96.0, 0.94.1 >Reporter: Andrew Purtell >Assignee: Andrew Purtell > Fix For: 0.94.2 > > Attachments: 6377-trunk-simple.patch > > > A client application (LoadTestTool) calls put() on HTables. Internally to the > HBase client those puts are batched into MultiActions. The total number of > put operations shown in the RegionServer's put metrics histogram never > increases from 0 even though millions of such operations are made. Needless > to say the latency for those operations are not measured either. The value of > HBASE-5533 metrics are suspect given the client will batch put and delete ops > like this. > I had a fix in progress but HBASE-6284 messed it up. Before, MultiAction > processing in HRegionServer would distingush between puts and deletes and > dispatch them separately. It was easy to account for the time for them. Now > both puts and deletes are submitted in batch together as mutations. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6377) HBASE-5533 metrics miss all operations submitted via MultiAction
[ https://issues.apache.org/jira/browse/HBASE-6377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13413006#comment-13413006 ] Andrew Purtell commented on HBASE-6377: --- On trunk apparently all use of the "get", "put", and "delete" histograms have already been removed from the regionserver, except for the Jamon templates. Since an approach to put back two such HRegionInterface level measurements with a query/mutate distinction has been +1ed on this issue I'm going to put them back on trunk and port back the changes to 0.94. Calling them queryRequestLatency and mutateRequestLatency in RegionServerMetrics. > HBASE-5533 metrics miss all operations submitted via MultiAction > > > Key: HBASE-6377 > URL: https://issues.apache.org/jira/browse/HBASE-6377 > Project: HBase > Issue Type: Bug > Components: metrics, regionserver >Affects Versions: 0.96.0, 0.94.1 >Reporter: Andrew Purtell > Fix For: 0.94.2 > > > A client application (LoadTestTool) calls put() on HTables. Internally to the > HBase client those puts are batched into MultiActions. The total number of > put operations shown in the RegionServer's put metrics histogram never > increases from 0 even though millions of such operations are made. Needless > to say the latency for those operations are not measured either. The value of > HBASE-5533 metrics are suspect given the client will batch put and delete ops > like this. > I had a fix in progress but HBASE-6284 messed it up. Before, MultiAction > processing in HRegionServer would distingush between puts and deletes and > dispatch them separately. It was easy to account for the time for them. Now > both puts and deletes are submitted in batch together as mutations. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6377) HBASE-5533 metrics miss all operations submitted via MultiAction
[ https://issues.apache.org/jira/browse/HBASE-6377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13412972#comment-13412972 ] Andrew Purtell commented on HBASE-6377: --- Will put something up today. > HBASE-5533 metrics miss all operations submitted via MultiAction > > > Key: HBASE-6377 > URL: https://issues.apache.org/jira/browse/HBASE-6377 > Project: HBase > Issue Type: Bug > Components: metrics, regionserver >Affects Versions: 0.96.0, 0.94.1 >Reporter: Andrew Purtell > Fix For: 0.94.2 > > > A client application (LoadTestTool) calls put() on HTables. Internally to the > HBase client those puts are batched into MultiActions. The total number of > put operations shown in the RegionServer's put metrics histogram never > increases from 0 even though millions of such operations are made. Needless > to say the latency for those operations are not measured either. The value of > HBASE-5533 metrics are suspect given the client will batch put and delete ops > like this. > I had a fix in progress but HBASE-6284 messed it up. Before, MultiAction > processing in HRegionServer would distingush between puts and deletes and > dispatch them separately. It was easy to account for the time for them. Now > both puts and deletes are submitted in batch together as mutations. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6377) HBASE-5533 metrics miss all operations submitted via MultiAction
[ https://issues.apache.org/jira/browse/HBASE-6377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13412963#comment-13412963 ] Lars Hofhansl commented on HBASE-6377: -- +1 on such a patch Andy, if you have bandwidth. (Would need to be quick to go into 0.94.1...) > HBASE-5533 metrics miss all operations submitted via MultiAction > > > Key: HBASE-6377 > URL: https://issues.apache.org/jira/browse/HBASE-6377 > Project: HBase > Issue Type: Bug > Components: metrics, regionserver >Affects Versions: 0.96.0, 0.94.1 >Reporter: Andrew Purtell > Fix For: 0.94.2 > > > A client application (LoadTestTool) calls put() on HTables. Internally to the > HBase client those puts are batched into MultiActions. The total number of > put operations shown in the RegionServer's put metrics histogram never > increases from 0 even though millions of such operations are made. Needless > to say the latency for those operations are not measured either. The value of > HBASE-5533 metrics are suspect given the client will batch put and delete ops > like this. > I had a fix in progress but HBASE-6284 messed it up. Before, MultiAction > processing in HRegionServer would distingush between puts and deletes and > dispatch them separately. It was easy to account for the time for them. Now > both puts and deletes are submitted in batch together as mutations. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6377) HBASE-5533 metrics miss all operations submitted via MultiAction
[ https://issues.apache.org/jira/browse/HBASE-6377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13412894#comment-13412894 ] Andrew Purtell commented on HBASE-6377: --- bq. Perhaps we can have "Get" and "Update" metrics. "Updates" would include Put, Deleted, ICV, etc I can supply a patch that does that. ? > HBASE-5533 metrics miss all operations submitted via MultiAction > > > Key: HBASE-6377 > URL: https://issues.apache.org/jira/browse/HBASE-6377 > Project: HBase > Issue Type: Bug > Components: metrics, regionserver >Affects Versions: 0.96.0, 0.94.1 >Reporter: Andrew Purtell > > A client application (LoadTestTool) calls put() on HTables. Internally to the > HBase client those puts are batched into MultiActions. The total number of > put operations shown in the RegionServer's put metrics histogram never > increases from 0 even though millions of such operations are made. Needless > to say the latency for those operations are not measured either. The value of > HBASE-5533 metrics are suspect given the client will batch put and delete ops > like this. > I had a fix in progress but HBASE-6284 messed it up. Before, MultiAction > processing in HRegionServer would distingush between puts and deletes and > dispatch them separately. It was easy to account for the time for them. Now > both puts and deletes are submitted in batch together as mutations. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6377) HBASE-5533 metrics miss all operations submitted via MultiAction
[ https://issues.apache.org/jira/browse/HBASE-6377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13412791#comment-13412791 ] stack commented on HBASE-6377: -- Let me know and I'll revert hbase-5533 (the patch that keeps on giving!) > HBASE-5533 metrics miss all operations submitted via MultiAction > > > Key: HBASE-6377 > URL: https://issues.apache.org/jira/browse/HBASE-6377 > Project: HBase > Issue Type: Bug > Components: metrics, regionserver >Affects Versions: 0.96.0, 0.94.1 >Reporter: Andrew Purtell > > A client application (LoadTestTool) calls put() on HTables. Internally to the > HBase client those puts are batched into MultiActions. The total number of > put operations shown in the RegionServer's put metrics histogram never > increases from 0 even though millions of such operations are made. Needless > to say the latency for those operations are not measured either. The value of > HBASE-5533 metrics are suspect given the client will batch put and delete ops > like this. > I had a fix in progress but HBASE-6284 messed it up. Before, MultiAction > processing in HRegionServer would distingush between puts and deletes and > dispatch them separately. It was easy to account for the time for them. Now > both puts and deletes are submitted in batch together as mutations. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6377) HBASE-5533 metrics miss all operations submitted via MultiAction
[ https://issues.apache.org/jira/browse/HBASE-6377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13412538#comment-13412538 ] Lars Hofhansl commented on HBASE-6377: -- Perhaps we can have "Get" and "Update" metrics. "Updates" would include Put, Deleted, ICV, etc. But maybe that would require more discussion, so short term (0.94.1 at least), we could remove the Put/Delete/Get metrics as you suggest. > HBASE-5533 metrics miss all operations submitted via MultiAction > > > Key: HBASE-6377 > URL: https://issues.apache.org/jira/browse/HBASE-6377 > Project: HBase > Issue Type: Bug > Components: metrics, regionserver >Affects Versions: 0.96.0, 0.94.1 >Reporter: Andrew Purtell > > A client application (LoadTestTool) calls put() on HTables. Internally to the > HBase client those puts are batched into MultiActions. The total number of > put operations shown in the RegionServer's put metrics histogram never > increases from 0 even though millions of such operations are made. Needless > to say the latency for those operations are not measured either. The value of > HBASE-5533 metrics are suspect given the client will batch put and delete ops > like this. > I had a fix in progress but HBASE-6284 messed it up. Before, MultiAction > processing in HRegionServer would distingush between puts and deletes and > dispatch them separately. It was easy to account for the time for them. Now > both puts and deletes are submitted in batch together as mutations. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6377) HBASE-5533 metrics miss all operations submitted via MultiAction
[ https://issues.apache.org/jira/browse/HBASE-6377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13412519#comment-13412519 ] Andrew Purtell commented on HBASE-6377: --- Perhaps not a full revert, but the "put" and "delete" metrics are not useful in a basic LoadTestTool test scenario, so consider dropping those. The distinction is increasingly only valid client side. Perhaps also remove the "get" one as well so we're not in effect special casing a metric only into HRI.get(). > HBASE-5533 metrics miss all operations submitted via MultiAction > > > Key: HBASE-6377 > URL: https://issues.apache.org/jira/browse/HBASE-6377 > Project: HBase > Issue Type: Bug > Components: metrics, regionserver >Affects Versions: 0.96.0, 0.94.1 >Reporter: Andrew Purtell > > A client application (LoadTestTool) calls put() on HTables. Internally to the > HBase client those puts are batched into MultiActions. The total number of > put operations shown in the RegionServer's put metrics histogram never > increases from 0 even though millions of such operations are made. Needless > to say the latency for those operations are not measured either. The value of > HBASE-5533 metrics are suspect given the client will batch put and delete ops > like this. > I had a fix in progress but HBASE-6284 messed it up. Before, MultiAction > processing in HRegionServer would distingush between puts and deletes and > dispatch them separately. It was easy to account for the time for them. Now > both puts and deletes are submitted in batch together as mutations. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6377) HBASE-5533 metrics miss all operations submitted via MultiAction
[ https://issues.apache.org/jira/browse/HBASE-6377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13412492#comment-13412492 ] Lars Hofhansl commented on HBASE-6377: -- I'd be supportive of reverting HBASE-5533 until we work out a clear strategy of what we're measuring and when. > HBASE-5533 metrics miss all operations submitted via MultiAction > > > Key: HBASE-6377 > URL: https://issues.apache.org/jira/browse/HBASE-6377 > Project: HBase > Issue Type: Bug > Components: metrics, regionserver >Affects Versions: 0.96.0, 0.94.1 >Reporter: Andrew Purtell > > A client application (LoadTestTool) calls put() on HTables. Internally to the > HBase client those puts are batched into MultiActions. The total number of > put operations shown in the RegionServer's put metrics histogram never > increases from 0 even though millions of such operations are made. Needless > to say the latency for those operations are not measured either. The value of > HBASE-5533 metrics are suspect given the client will batch put and delete ops > like this. > I had a fix in progress but HBASE-6284 messed it up. Before, MultiAction > processing in HRegionServer would distingush between puts and deletes and > dispatch them separately. It was easy to account for the time for them. Now > both puts and deletes are submitted in batch together as mutations. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6377) HBASE-5533 metrics miss all operations submitted via MultiAction
[ https://issues.apache.org/jira/browse/HBASE-6377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13412491#comment-13412491 ] Lars Hofhansl commented on HBASE-6377: -- I "knew" there would be issues somewhere with HBASE-6284 :( What do you say Andrew, this seems bad enough to delay 0.94.1? Does a "Put" metric still make sense? Should it be a "Mutation" metric which includes Deletes? > HBASE-5533 metrics miss all operations submitted via MultiAction > > > Key: HBASE-6377 > URL: https://issues.apache.org/jira/browse/HBASE-6377 > Project: HBase > Issue Type: Bug > Components: metrics, regionserver >Affects Versions: 0.96.0, 0.94.1 >Reporter: Andrew Purtell > > A client application (LoadTestTool) calls put() on HTables. Internally to the > HBase client those puts are batched into MultiActions. The total number of > put operations shown in the RegionServer's put metrics histogram never > increases from 0 even though millions of such operations are made. Needless > to say the latency for those operations are not measured either. The value of > HBASE-5533 metrics are suspect given the client will batch put and delete ops > like this. > I had a fix in progress but HBASE-6284 messed it up. Before, MultiAction > processing in HRegionServer would distingush between puts and deletes and > dispatch them separately. It was easy to account for the time for them. Now > both puts and deletes are submitted in batch together as mutations. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6377) HBASE-5533 metrics miss all operations submitted via MultiAction
[ https://issues.apache.org/jira/browse/HBASE-6377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13412461#comment-13412461 ] Andrew Purtell commented on HBASE-6377: --- Other means for applying puts and deletes (RowMutations etc.) are not covered either, neither are Increments, or Appends. Perhaps this is an argument for reverting HBASE-5533. > HBASE-5533 metrics miss all operations submitted via MultiAction > > > Key: HBASE-6377 > URL: https://issues.apache.org/jira/browse/HBASE-6377 > Project: HBase > Issue Type: Bug > Components: metrics, regionserver >Affects Versions: 0.96.0, 0.94.1 >Reporter: Andrew Purtell > > A client application (LoadTestTool) calls put() on HTables. Internally to the > HBase client those puts are batched into MultiActions. The total number of > put operations shown in the RegionServer's put metrics histogram never > increases from 0 even though millions of such operations are made. Needless > to say the latency for those operations are not measured either. The value of > HBASE-5533 metrics are suspect given the client will batch put and delete ops > like this. > I had a fix in progress but HBASE-6284 messed it up. Before, MultiAction > processing in HRegionServer would distingush between puts and deletes and > dispatch them separately. It was easy to account for the time for them. Now > both puts and deletes are submitted in batch together as mutations. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
