[jira] [Commented] (HBASE-3614) Expose per-region request rate metrics
[ https://issues.apache.org/jira/browse/HBASE-3614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258394#comment-13258394 ] Elliott Clark commented on HBASE-3614: -- @Todd the time period is configurable. It's set through the hadoop-metrics hbase.period. Whenever the thread that publishes metrics comes around the averages are re-set. Expose per-region request rate metrics -- Key: HBASE-3614 URL: https://issues.apache.org/jira/browse/HBASE-3614 Project: HBase Issue Type: Improvement Components: metrics, regionserver Reporter: Gary Helmling Assignee: Elliott Clark Priority: Minor Fix For: 0.96.0 Attachments: HBASE-3614-0.patch, HBASE-3614-1.patch, HBASE-3614-2.patch, HBASE-3614-3.patch, HBASE-3614-4.patch, HBASE-3614-5.patch, HBASE-3614-6.patch, HBASE-3614-7.patch, HBASE-3614-8.patch, HBASE-3614-9.patch, Screen Shot 2012-04-17 at 2.41.27 PM.png We currently export metrics on request rates for each region server, and this can help with identifying uneven load at a high level. But once you see a given server under high load, you're forced to extrapolate based on your application patterns and the data it's serving what the likely culprit is. This can and should be much easier if we just exported request rate metrics per-region on each server. Dynamically updating the metrics keys based on assigned regions may pose some minor challenges, but this seems a very valuable diagnostic tool to have available. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3614) Expose per-region request rate metrics
[ https://issues.apache.org/jira/browse/HBASE-3614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13257797#comment-13257797 ] Elliott Clark commented on HBASE-3614: -- And I beat you to the rename :-) is opMetrics alright. Expose per-region request rate metrics -- Key: HBASE-3614 URL: https://issues.apache.org/jira/browse/HBASE-3614 Project: HBase Issue Type: Improvement Components: metrics, regionserver Reporter: Gary Helmling Assignee: Elliott Clark Priority: Minor Attachments: HBASE-3614-0.patch, HBASE-3614-1.patch, HBASE-3614-2.patch, HBASE-3614-3.patch, HBASE-3614-4.patch, HBASE-3614-5.patch, HBASE-3614-6.patch, HBASE-3614-7.patch, HBASE-3614-8.patch, Screen Shot 2012-04-17 at 2.41.27 PM.png We currently export metrics on request rates for each region server, and this can help with identifying uneven load at a high level. But once you see a given server under high load, you're forced to extrapolate based on your application patterns and the data it's serving what the likely culprit is. This can and should be much easier if we just exported request rate metrics per-region on each server. Dynamically updating the metrics keys based on assigned regions may pose some minor challenges, but this seems a very valuable diagnostic tool to have available. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5824) HRegion.incrementColumnValue is not used in trunk
[ https://issues.apache.org/jira/browse/HBASE-5824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13257848#comment-13257848 ] Elliott Clark commented on HBASE-5824: -- Looking through it everything seems to be good. I can't see anything that ICV does that increment doesn't. Should the old code get removed then ? HRegion.incrementColumnValue is not used in trunk - Key: HBASE-5824 URL: https://issues.apache.org/jira/browse/HBASE-5824 Project: HBase Issue Type: Bug Reporter: Elliott Clark Assignee: Jimmy Xiang Attachments: hbase-5824.patch on 0.94 a call to client.HTable#incrementColumnValue will cause HRegion#incrementColumnValue. On trunk all calls to HTable.incrementColumnValue got to HRegion#increment. My guess is that HTable#incrementColumnValue and HTable#increment serialize to the same thing over the wire so that the remote HRegionServer no longer knows which htable method was called. To repro I checked out trunk and put a break point in HRegion#incrementColumnValue and then ran TestFromClientSide. The breakpoint wasn't hit. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5836) Backport per region metrics from HBASE-3614 to 0.94.1
[ https://issues.apache.org/jira/browse/HBASE-5836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13257882#comment-13257882 ] Elliott Clark commented on HBASE-5836: -- This will also need HBASE-5788 to keep it as close to trunk as possible. Backport per region metrics from HBASE-3614 to 0.94.1 - Key: HBASE-5836 URL: https://issues.apache.org/jira/browse/HBASE-5836 Project: HBase Issue Type: Task Reporter: stack Assignee: Elliott Clark Fix For: 0.94.1 This would be good to have in 0.94. Can go into 0.94.1. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5824) HRegion.incrementColumnValue is not used in trunk
[ https://issues.apache.org/jira/browse/HBASE-5824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13257110#comment-13257110 ] Elliott Clark commented on HBASE-5824: -- Thanks. Looks like single puts are also only going through the multiput method. I'm not sure if this is an issue that needs to be fixed as well. HRegion.incrementColumnValue is not used in trunk - Key: HBASE-5824 URL: https://issues.apache.org/jira/browse/HBASE-5824 Project: HBase Issue Type: Bug Reporter: Elliott Clark Assignee: Jimmy Xiang on 0.94 a call to client.HTable#incrementColumnValue will cause HRegion#incrementColumnValue. On trunk all calls to HTable.incrementColumnValue got to HRegion#increment. My guess is that HTable#incrementColumnValue and HTable#increment serialize to the same thing over the wire so that the remote HRegionServer no longer knows which htable method was called. To repro I checked out trunk and put a break point in HRegion#incrementColumnValue and then ran TestFromClientSide. The breakpoint wasn't hit. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5788) Move Dynamic Metrics storage off of HRegion.
[ https://issues.apache.org/jira/browse/HBASE-5788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13255635#comment-13255635 ] Elliott Clark commented on HBASE-5788: -- RegionMetricsStorage could be a better name this class holds and mutates the numbers that will be exposed as dynamic region metrics. I'll add the comments, and check the line length. Not sure how I messed up eclipse. The data members can be package private now that they are in the regionserver.metrics package. I'll check to make sure. Everything else is accessed in the regionserver package so they need to be public :-/ TestRegionServerMetrics covers most of the functionality of the new class but I can create a new set of more explicit tests if you think that is needed. Move Dynamic Metrics storage off of HRegion. Key: HBASE-5788 URL: https://issues.apache.org/jira/browse/HBASE-5788 Project: HBase Issue Type: Improvement Components: regionserver Reporter: Elliott Clark Assignee: Elliott Clark Priority: Minor Attachments: HBASE-5788-0.patch, HBASE-5788-1.patch, HBASE-5788-2.patch HRegion right now has the responsibility of storing static counts and latency numbers for use by the metrics package. Since these maps are incremented and set from lots of places it makes adding functionality hard. So move the metrics functionality into SchemaMetrics making it more than just a class for naming. The next step will be to simplify the api exposed so that using it will be easier. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5788) Move Dynamic Metrics storage off of HRegion.
[ https://issues.apache.org/jira/browse/HBASE-5788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13256114#comment-13256114 ] Elliott Clark commented on HBASE-5788: -- @Enis I was thinking of getting things more like HBASE-3614. I moved metrics about get/put/delete/icv into a helper class that mutates data in RegionMetricsStorage. I was hoping to go that way with pretty simple helper classes that expose update methods. Then SchemaMetrics can be mutated in the same way and metric naming can be handled by static helpers only. Move Dynamic Metrics storage off of HRegion. Key: HBASE-5788 URL: https://issues.apache.org/jira/browse/HBASE-5788 Project: HBase Issue Type: Improvement Components: regionserver Reporter: Elliott Clark Assignee: Elliott Clark Priority: Minor Fix For: 0.96.0 Attachments: HBASE-5788-0.patch, HBASE-5788-1.patch, HBASE-5788-2.patch, HBASE-5788-3.patch, HBASE-5788-4.patch HRegion right now has the responsibility of storing static counts and latency numbers for use by the metrics package. Since these maps are incremented and set from lots of places it makes adding functionality hard. So move the metrics functionality into SchemaMetrics making it more than just a class for naming. The next step will be to simplify the api exposed so that using it will be easier. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3614) Expose per-region request rate metrics
[ https://issues.apache.org/jira/browse/HBASE-3614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13256149#comment-13256149 ] Elliott Clark commented on HBASE-3614: -- I think per region and per cf metrics being configurable is probably the right way to go. I'll add that in the next go around. Expose per-region request rate metrics -- Key: HBASE-3614 URL: https://issues.apache.org/jira/browse/HBASE-3614 Project: HBase Issue Type: Improvement Components: metrics, regionserver Reporter: Gary Helmling Assignee: Elliott Clark Priority: Minor Attachments: HBASE-3614-0.patch, HBASE-3614-1.patch, HBASE-3614-2.patch, Screen Shot 2012-04-17 at 2.41.27 PM.png We currently export metrics on request rates for each region server, and this can help with identifying uneven load at a high level. But once you see a given server under high load, you're forced to extrapolate based on your application patterns and the data it's serving what the likely culprit is. This can and should be much easier if we just exported request rate metrics per-region on each server. Dynamically updating the metrics keys based on assigned regions may pose some minor challenges, but this seems a very valuable diagnostic tool to have available. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5788) Move Dynamic Metrics storage off of HRegion.
[ https://issues.apache.org/jira/browse/HBASE-5788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13255014#comment-13255014 ] Elliott Clark commented on HBASE-5788: -- On trying to add more to this class it became obvious to me that it needs more love than the first try. I'll roll this into exposing per region metrics. Move Dynamic Metrics storage off of HRegion. Key: HBASE-5788 URL: https://issues.apache.org/jira/browse/HBASE-5788 Project: HBase Issue Type: Improvement Components: regionserver Reporter: Elliott Clark Assignee: Elliott Clark Priority: Minor Attachments: HBASE-5788-0.patch HRegion right now has the responsibility of storing static counts and latency numbers for use by the metrics package. Since these maps are incremented and set from lots of places it makes adding functionality hard. So move the metrics functionality into SchemaMetrics making it more than just a class for naming. The next step will be to simplify the api exposed so that using it will be easier. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5363) Add rat check to run automatically on mvn build.
[ https://issues.apache.org/jira/browse/HBASE-5363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13204728#comment-13204728 ] Elliott Clark commented on HBASE-5363: -- Though it is getting several files that need licenses. Add rat check to run automatically on mvn build. Key: HBASE-5363 URL: https://issues.apache.org/jira/browse/HBASE-5363 Project: HBase Issue Type: Improvement Components: build Affects Versions: 0.90.5, 0.92.0 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Some of the recent hbase release failed rat checks (mvn rat:check). We should add checks likely in the mvn package phase so that this becomes a non-issue in the future. Here's an example from Whirr: https://github.com/apache/whirr/blob/trunk/pom.xml line 388 for an example. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5363) Add rat check to run automatically on mvn build.
[ https://issues.apache.org/jira/browse/HBASE-5363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13204736#comment-13204736 ] Elliott Clark commented on HBASE-5363: -- Attached a patch to add headers to some files that were missing it according to rat. I couldn't get the rat plugin to run automatically on the first try so the patch doesn't have any pom changes in it. Add rat check to run automatically on mvn build. Key: HBASE-5363 URL: https://issues.apache.org/jira/browse/HBASE-5363 Project: HBase Issue Type: Improvement Components: build Affects Versions: 0.90.5, 0.92.0 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Attachments: HBASE-5363-1.patch Some of the recent hbase release failed rat checks (mvn rat:check). We should add checks likely in the mvn package phase so that this becomes a non-issue in the future. Here's an example from Whirr: https://github.com/apache/whirr/blob/trunk/pom.xml line 388 for an example. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5364) Fix source files missing licenses
[ https://issues.apache.org/jira/browse/HBASE-5364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13205002#comment-13205002 ] Elliott Clark commented on HBASE-5364: -- Attach patch to fix files on master. Fix source files missing licenses - Key: HBASE-5364 URL: https://issues.apache.org/jira/browse/HBASE-5364 Project: HBase Issue Type: Bug Affects Versions: 0.94.0, 0.90.5, 0.92.0 Reporter: Jonathan Hsieh Priority: Blocker Attachments: HBASE-5364-1.patch running 'mvn rat:check' shows that a few files have snuck in that do not have proper apache licenses. Ideally we should fix these before we cut another release/release candidate. This is a blocker for 0.94, and probably should be for the other branches as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5364) Fix source files missing licenses
[ https://issues.apache.org/jira/browse/HBASE-5364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13205135#comment-13205135 ] Elliott Clark commented on HBASE-5364: -- Thanks Fix source files missing licenses - Key: HBASE-5364 URL: https://issues.apache.org/jira/browse/HBASE-5364 Project: HBase Issue Type: Bug Affects Versions: 0.94.0, 0.90.5, 0.92.0 Reporter: Jonathan Hsieh Priority: Blocker Attachments: HBASE-5364-1.patch, hbase-5364-0.92.patch running 'mvn rat:check' shows that a few files have snuck in that do not have proper apache licenses. Ideally we should fix these before we cut another release/release candidate. This is a blocker for 0.94, and probably should be for the other branches as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
