[jira] Created: (HBASE-3631) CLONE - HBase 2984 breaks ability to specify BLOOMFILTER COMPRESSION via shell
CLONE - HBase 2984 breaks ability to specify BLOOMFILTER COMPRESSION via shell Key: HBASE-3631 URL: https://issues.apache.org/jira/browse/HBASE-3631 Project: HBase Issue Type: Bug Reporter: Sebastian Bauer Assignee: Kannan Muthukkaruppan Priority: Minor Fix For: 0.90.0 Attachments: 3173-v2.txt, HBASE-3173.txt HBase 2984 breaks ability to specify BLOOMFILTER COMPRESSION via shell -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HBASE-3631) CLONE - HBase 2984 breaks ability to specify BLOOMFILTER COMPRESSION via shell
[ https://issues.apache.org/jira/browse/HBASE-3631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sebastian Bauer updated HBASE-3631: --- Component/s: shell Description: HBase 2984 breaks ability to specify BLOOMFILTER COMPRESSION via shell 0.90 was fixed but in trunk there is still bug was:HBase 2984 breaks ability to specify BLOOMFILTER COMPRESSION via shell Affects Version/s: 0.92.0 Fix Version/s: (was: 0.90.0) 0.92.0 Hadoop Flags: (was: [Reviewed]) CLONE - HBase 2984 breaks ability to specify BLOOMFILTER COMPRESSION via shell Key: HBASE-3631 URL: https://issues.apache.org/jira/browse/HBASE-3631 Project: HBase Issue Type: Bug Components: shell Affects Versions: 0.92.0 Reporter: Sebastian Bauer Assignee: Kannan Muthukkaruppan Priority: Minor Fix For: 0.92.0 Attachments: 3173-v2.txt, HBASE-3173.txt HBase 2984 breaks ability to specify BLOOMFILTER COMPRESSION via shell 0.90 was fixed but in trunk there is still bug -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HBASE-3631) CLONE - HBase 2984 breaks ability to specify BLOOMFILTER COMPRESSION via shell
[ https://issues.apache.org/jira/browse/HBASE-3631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sebastian Bauer updated HBASE-3631: --- Attachment: (was: HBASE-3173.txt) CLONE - HBase 2984 breaks ability to specify BLOOMFILTER COMPRESSION via shell Key: HBASE-3631 URL: https://issues.apache.org/jira/browse/HBASE-3631 Project: HBase Issue Type: Bug Components: shell Affects Versions: 0.92.0 Reporter: Sebastian Bauer Assignee: Kannan Muthukkaruppan Priority: Minor Fix For: 0.92.0 HBase 2984 breaks ability to specify BLOOMFILTER COMPRESSION via shell 0.90 was fixed but in trunk there is still bug -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Created: (HBASE-3632) ability to record the block cache hit ratio for the last few minutes
ability to record the block cache hit ratio for the last few minutes Key: HBASE-3632 URL: https://issues.apache.org/jira/browse/HBASE-3632 Project: HBase Issue Type: Improvement Components: regionserver Reporter: dhruba borthakur Assignee: dhruba borthakur In the curent code, the block cache hit ratio is calculated by using the total number of cache hits since the regin server was rebooted. This means that this metric does not reflect the short term changes that occur in the workload; the longer the region reserver is alive slower is the rate of change of this metric. I propose that this metric reflect the cache hit ratio for the work processed in the most recent 10 minutes, is that reasonable? -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HBASE-1744) Thrift server to match the new java api.
[
https://issues.apache.org/jira/browse/HBASE-1744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13006375#comment-13006375
]
Lars Francke commented on HBASE-1744:
-
I'm afraid I will finally have to say that I have no idea when I'll be able to
work on this. I've promised it for a long time but if anyone else wants to take
a stab at it feel free.
I'm pretty confident in the Thrift definition file I've come up with. It needs
some minor updates for the updated API (increment for example) and the delete
method is still missing one of the possible cases (I've documented it in the
patch) and filters are not implemented yet either.
I'll gladly answer any questions about this in case anyone wants to work on it
and if I have the time I'll improve the patch but I can't promise anything,
sorry :(
Thrift server to match the new java api.
Key: HBASE-1744
URL: https://issues.apache.org/jira/browse/HBASE-1744
Project: HBase
Issue Type: Improvement
Components: thrift
Reporter: Tim Sell
Assignee: Lars Francke
Priority: Critical
Fix For: 0.92.0
Attachments: HBASE-1744.preview.1.patch, thriftexperiment.patch
This mutateRows, etc.. is a little confusing compared to the new cleaner java
client.
Thinking of ways to make a thrift client that is just as elegant. something
like:
void put(1:Bytes table, 2:TPut put) throws (1:IOError io)
with:
struct TColumn {
1:Bytes family,
2:Bytes qualifier,
3:i64 timestamp
}
struct TPut {
1:Bytes row,
2:mapTColumn, Bytes values
}
This creates more verbose rpc than if the columns in TPut were just
mapBytes, mapBytes, Bytes, but that is harder to fit timestamps into and
still be intuitive from say python.
Presumably the goal of a thrift gateway is to be easy first.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HBASE-3620) Make HBCK Faster
[ https://issues.apache.org/jira/browse/HBASE-3620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13006477#comment-13006477 ] stack commented on HBASE-3620: -- @Ted Do you observer only one thread running before your change? Then subsequent to your change, multiple threads running? Is backing queue a LinkedBlockingQueue as over in HBASE-3553? Thanks. Make HBCK Faster Key: HBASE-3620 URL: https://issues.apache.org/jira/browse/HBASE-3620 Project: HBase Issue Type: Improvement Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Priority: Minor Fix For: 0.90.2, 0.92.0 Attachments: hbase-3620-pool-size.txt, hbck_perf.patch Make the HBCK utility contact all region servers HDFS directories in parallel. This will speedup hbck processing, especially when there are lots of region servers. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Resolved: (HBASE-3618) Add to HBase book, 'schema' chapter - pre-creating regions and key types
[ https://issues.apache.org/jira/browse/HBASE-3618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-3618. -- Resolution: Fixed Committed to TRUNK. Thanks for the patch Doug. Add to HBase book, 'schema' chapter - pre-creating regions and key types Key: HBASE-3618 URL: https://issues.apache.org/jira/browse/HBASE-3618 Project: HBase Issue Type: Improvement Reporter: Doug Meil Assignee: stack Priority: Minor Fix For: 0.92.0 Attachments: book.xml.patch In the HBase book for the schema chapter. Expanded the section on why monotonically increasing keys are bad, and specifically why the OpenTSDB key approach works, which is a common question on the hbase dist list. Also, adding section on pre-creating regions (how to do it). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HBASE-3636) a bug about deciding whether this key is a new key for the ROWCOL bloomfilter
[ https://issues.apache.org/jira/browse/HBASE-3636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang updated HBASE-3636: -- Attachment: HBASE_3636[r1081520].patch Fix the problem by changing the function compareRowColumn. Add a unit test to verify it. Please review. a bug about deciding whether this key is a new key for the ROWCOL bloomfilter - Key: HBASE-3636 URL: https://issues.apache.org/jira/browse/HBASE-3636 Project: HBase Issue Type: Bug Components: regionserver Reporter: Liyin Tang Attachments: HBASE_3636[r1081520].patch When ROWCOL bloomfilter needs to decide whether this key is a new key or not, it will call the matchingRowColumn function, which will compare the timestamp offset between this kv and last kv. But when checking the timestamp offset, it didn't deduct the original offset of the keyvalue itself. For example, when 2 keyvalue objects have the same row key and col key, but from different storefiles. It is highly likely that these 2 keyvalue objects have different offset value. So the timestamp offset of these 2 objects are totally different. They will be regard as new keys to add into bloomfilters. So after compaction, the key count of bloomfilter will increase immediately, which is almost equal to the number of entries. The solution is straightforward. Just compare the relevant timestamp offset, which is the timestamp offset - key_value offset. This also may explain this jira: https://issues.apache.org/jira/browse/HBASE-3007 -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HBASE-3632) ability to record the block cache hit ratio for the last few minutes
[ https://issues.apache.org/jira/browse/HBASE-3632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13006609#comment-13006609 ] Todd Lipcon commented on HBASE-3632: Might be interesting to consider using something like jrobin for metrics like this - so we can get these metrics at various granularities. ability to record the block cache hit ratio for the last few minutes Key: HBASE-3632 URL: https://issues.apache.org/jira/browse/HBASE-3632 Project: HBase Issue Type: Improvement Components: regionserver Reporter: dhruba borthakur Assignee: dhruba borthakur In the curent code, the block cache hit ratio is calculated by using the total number of cache hits since the regin server was rebooted. This means that this metric does not reflect the short term changes that occur in the workload; the longer the region reserver is alive slower is the rate of change of this metric. I propose that this metric reflect the cache hit ratio for the work processed in the most recent 10 minutes, is that reasonable? -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Created: (HBASE-3637) Region stuck in OPENED state
Region stuck in OPENED state Key: HBASE-3637 URL: https://issues.apache.org/jira/browse/HBASE-3637 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.92.0 Reporter: Todd Lipcon Priority: Critical Fix For: 0.92.0 I don't 100% understand how this happened, but the following was observed: - META is in OPENED state in ZK, for a server which no longer exists - Handler sees that server is dead, and figures that the RIT timeout will handle it - RIT timeout sees that it's already in OPENED state, and assumes that the OPENED handler will handle it - loops in timeout state forever, never actually getting reassigned -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HBASE-3637) Region stuck in OPENED state
[ https://issues.apache.org/jira/browse/HBASE-3637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13006619#comment-13006619 ] Todd Lipcon commented on HBASE-3637: Some more context: this was from a test that was looping the following: - install HBase - wipe /hbase on HDFS - start hbase, run smoke test, kill all servers Notably, it wasn't clearing ZK between runs. So some leftover RIT data from a previous HBase incarnation may be confusing this one's master. Region stuck in OPENED state Key: HBASE-3637 URL: https://issues.apache.org/jira/browse/HBASE-3637 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.92.0 Reporter: Todd Lipcon Priority: Critical Fix For: 0.92.0 I don't 100% understand how this happened, but the following was observed: - META is in OPENED state in ZK, for a server which no longer exists - Handler sees that server is dead, and figures that the RIT timeout will handle it - RIT timeout sees that it's already in OPENED state, and assumes that the OPENED handler will handle it - loops in timeout state forever, never actually getting reassigned -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Created: (HBASE-3638) If a FS bootstrap, need to also ensure ZK is cleaned
If a FS bootstrap, need to also ensure ZK is cleaned
Key: HBASE-3638
URL: https://issues.apache.org/jira/browse/HBASE-3638
Project: HBase
Issue Type: Bug
Reporter: stack
Priority: Minor
In a test environment where a cycle of start, operation, kill hbase (repeat),
noticed that we were doing a bootstrap on startup but then we were picking up
the previous cycles zk state. It made for a mess in the test.
Last thing seen on previous cycle was:
{code}
2011-03-11 06:33:36,708 DEBUG org.apache.hadoop.hbase.master.AssignmentManager:
Handling transition=RS_ZK_REGION_OPENING, server=X.X.X.60020,1299853933073,
region=1028785192/.META.
{code}
Then, in the messed up cycle I saw:
{code}
2011-03-11 06:42:48,530 INFO org.apache.hadoop.hbase.master.MasterFileSystem:
BOOTSTRAP: creating ROOT and first META regions
.
{code}
Then after setting watcher on .META., we get a
{code}
2011-03-11 06:42:58,301 INFO org.apache.hadoop.hbase.master.AssignmentManager:
Processing region .META.,,1.1028785192 in state RS_ZK_REGION_OPENED
2011-03-11 06:42:58,302 WARN org.apache.hadoop.hbase.master.AssignmentManager:
Region in transition 1028785192 references a server no longer up X.X.X; letting
RIT timeout so will be assigned elsewhere
{code}
We're all confused.
Should at least clear our zk if a bootstrap happened.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HBASE-3637) Region stuck in OPENED state
[ https://issues.apache.org/jira/browse/HBASE-3637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13006708#comment-13006708 ] stack commented on HBASE-3637: -- OK if I close this in favor of HBASE-3638 Todd? Region stuck in OPENED state Key: HBASE-3637 URL: https://issues.apache.org/jira/browse/HBASE-3637 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.92.0 Reporter: Todd Lipcon Priority: Critical Fix For: 0.92.0 I don't 100% understand how this happened, but the following was observed: - META is in OPENED state in ZK, for a server which no longer exists - Handler sees that server is dead, and figures that the RIT timeout will handle it - RIT timeout sees that it's already in OPENED state, and assumes that the OPENED handler will handle it - loops in timeout state forever, never actually getting reassigned -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HBASE-3639) FSUtils.getRootDir should qualify path
[ https://issues.apache.org/jira/browse/HBASE-3639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon updated HBASE-3639: --- Attachment: hbase-3639.txt FSUtils.getRootDir should qualify path -- Key: HBASE-3639 URL: https://issues.apache.org/jira/browse/HBASE-3639 Project: HBase Issue Type: Bug Affects Versions: 0.90.1 Reporter: Todd Lipcon Priority: Critical Fix For: 0.90.2 Attachments: hbase-3639.txt Currently you can run into a StackOverflowError if the hbase root dir is on the non-default filesystem. Making FSUtils.getRootDir qualify its path solves this issue. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HBASE-3625) improve/fix support excluding Tests via Maven -D property
[
https://issues.apache.org/jira/browse/HBASE-3625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Todd Lipcon updated HBASE-3625:
---
Attachment: hbase-3625.txt
improve/fix support excluding Tests via Maven -D property
-
Key: HBASE-3625
URL: https://issues.apache.org/jira/browse/HBASE-3625
Project: HBase
Issue Type: Improvement
Components: build
Affects Versions: 0.90.1
Environment: all
Reporter: Alejandro Abdelnur
Labels: build
Attachments: hbase-3625.txt
Currently the surefire plugin configuration defines the following exclusion:
{code}
.
plugin
groupIdorg.apache.maven.plugins/groupId
artifactIdmaven-surefire-plugin/artifactId
configuration
forkModealways/forkMode
includes
include**/Test*.java/include
/includes
excludes
exclude**/*$*/exclude
/excludes
/configuration
/plugin
{code}
AFAICT the '{{***/***$**}}' does not resolve to anything meaningful.
Adding support to exclude one or more tests via Maven property, i.e.
'{{-Dtest.exclude=TESTCLASS}}' would be useful.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Created: (HBASE-3640) [replication] Transferring queues shouldn't be done inline with RS startup
[replication] Transferring queues shouldn't be done inline with RS startup -- Key: HBASE-3640 URL: https://issues.apache.org/jira/browse/HBASE-3640 Project: HBase Issue Type: Improvement Affects Versions: 0.90.1 Reporter: Jean-Daniel Cryans Assignee: Jean-Daniel Cryans Fix For: 0.90.2 Currently ReplicationSourceManager transfers queues of dead region servers in init(). I saw a handful of situations where it had a bunch of znodes to transfer and it delayed the whole region server startup by a few minutes. This instead should be done later, possibly in a Chore. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HBASE-3448) RegionSplitter : Utility class for manual region splitting
[ https://issues.apache.org/jira/browse/HBASE-3448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13006714#comment-13006714 ] Nicolas Spiegelberg commented on HBASE-3448: Going to put this in the 0.90 branch as well. Since this is only an HBase utility and was confirmed for the 0.90 branch earlier, I'm assuming that this is fine. RegionSplitter : Utility class for manual region splitting -- Key: HBASE-3448 URL: https://issues.apache.org/jira/browse/HBASE-3448 Project: HBase Issue Type: New Feature Components: client, scripts, util Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Priority: Minor Fix For: 0.92.0 Attachments: HBASE-3448.patch For certain use cases, there are a number of advantages to manually splitting regions instead of having the HBase split code determine this for you automatically. There are currently some API additions to HBaseAdmin and HTable that allow you to manually split on a small scale. This JIRA is about importing a RegionSplitter utility program to help pre-split and perform rolling splits on a live table when needed. Will also add documentation to answer common questions about why you would pre-split. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Created: (HBASE-3641) LruBlockCache.CacheStats.getHitCount() is not using the correct variable
LruBlockCache.CacheStats.getHitCount() is not using the correct variable
Key: HBASE-3641
URL: https://issues.apache.org/jira/browse/HBASE-3641
Project: HBase
Issue Type: Bug
Components: io
Affects Versions: 0.90.1, 0.92.0
Reporter: Jonathan Gray
Assignee: Jonathan Gray
Fix For: 0.90.2, 0.92.0
{code}
public long getHitCount() {
return hitCachingCount.get();
}
{code}
This should be {{hitCount.get()}}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HBASE-3641) LruBlockCache.CacheStats.getHitCount() is not using the correct variable
[
https://issues.apache.org/jira/browse/HBASE-3641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Gray updated HBASE-3641:
-
Status: Patch Available (was: Open)
LruBlockCache.CacheStats.getHitCount() is not using the correct variable
Key: HBASE-3641
URL: https://issues.apache.org/jira/browse/HBASE-3641
Project: HBase
Issue Type: Bug
Components: io
Affects Versions: 0.90.1, 0.92.0
Reporter: Jonathan Gray
Assignee: Jonathan Gray
Fix For: 0.90.2, 0.92.0
Attachments: HBASE-3641-v1.patch
{code}
public long getHitCount() {
return hitCachingCount.get();
}
{code}
This should be {{hitCount.get()}}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HBASE-3641) LruBlockCache.CacheStats.getHitCount() is not using the correct variable
[
https://issues.apache.org/jira/browse/HBASE-3641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Gray updated HBASE-3641:
-
Attachment: HBASE-3641-v1.patch
LruBlockCache.CacheStats.getHitCount() is not using the correct variable
Key: HBASE-3641
URL: https://issues.apache.org/jira/browse/HBASE-3641
Project: HBase
Issue Type: Bug
Components: io
Affects Versions: 0.90.1, 0.92.0
Reporter: Jonathan Gray
Assignee: Jonathan Gray
Fix For: 0.90.2, 0.92.0
Attachments: HBASE-3641-v1.patch
{code}
public long getHitCount() {
return hitCachingCount.get();
}
{code}
This should be {{hitCount.get()}}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HBASE-3639) FSUtils.getRootDir should qualify path
[ https://issues.apache.org/jira/browse/HBASE-3639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13006720#comment-13006720 ] stack commented on HBASE-3639: -- +1 FSUtils.getRootDir should qualify path -- Key: HBASE-3639 URL: https://issues.apache.org/jira/browse/HBASE-3639 Project: HBase Issue Type: Bug Affects Versions: 0.90.1 Reporter: Todd Lipcon Priority: Critical Fix For: 0.90.2 Attachments: hbase-3639.txt Currently you can run into a StackOverflowError if the hbase root dir is on the non-default filesystem. Making FSUtils.getRootDir qualify its path solves this issue. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Created: (HBASE-3642) Web UI should be available during startup
Web UI should be available during startup - Key: HBASE-3642 URL: https://issues.apache.org/jira/browse/HBASE-3642 Project: HBase Issue Type: Improvement Reporter: Dmitriy V. Ryaboy Priority: Minor Currently, HBase does not provide a web interface during its start-up period -- while it's waiting for RSes to report in, replaying logs, etc. It would be great if the Web UI was available and showed the current status. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HBASE-3625) improve/fix support excluding Tests via Maven -D property
[
https://issues.apache.org/jira/browse/HBASE-3625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13006723#comment-13006723
]
stack commented on HBASE-3625:
--
+1
improve/fix support excluding Tests via Maven -D property
-
Key: HBASE-3625
URL: https://issues.apache.org/jira/browse/HBASE-3625
Project: HBase
Issue Type: Improvement
Components: build
Affects Versions: 0.90.1
Environment: all
Reporter: Alejandro Abdelnur
Labels: build
Attachments: hbase-3625.txt
Currently the surefire plugin configuration defines the following exclusion:
{code}
.
plugin
groupIdorg.apache.maven.plugins/groupId
artifactIdmaven-surefire-plugin/artifactId
configuration
forkModealways/forkMode
includes
include**/Test*.java/include
/includes
excludes
exclude**/*$*/exclude
/excludes
/configuration
/plugin
{code}
AFAICT the '{{***/***$**}}' does not resolve to anything meaningful.
Adding support to exclude one or more tests via Maven property, i.e.
'{{-Dtest.exclude=TESTCLASS}}' would be useful.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HBASE-3625) improve/fix support excluding Tests via Maven -D property
[
https://issues.apache.org/jira/browse/HBASE-3625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-3625:
-
Status: Patch Available (was: Open)
Marking patch available. Looks like something that'd be useful in trunk and
branch.
improve/fix support excluding Tests via Maven -D property
-
Key: HBASE-3625
URL: https://issues.apache.org/jira/browse/HBASE-3625
Project: HBase
Issue Type: Improvement
Components: build
Affects Versions: 0.90.1
Environment: all
Reporter: Alejandro Abdelnur
Labels: build
Attachments: hbase-3625.txt
Currently the surefire plugin configuration defines the following exclusion:
{code}
.
plugin
groupIdorg.apache.maven.plugins/groupId
artifactIdmaven-surefire-plugin/artifactId
configuration
forkModealways/forkMode
includes
include**/Test*.java/include
/includes
excludes
exclude**/*$*/exclude
/excludes
/configuration
/plugin
{code}
AFAICT the '{{***/***$**}}' does not resolve to anything meaningful.
Adding support to exclude one or more tests via Maven property, i.e.
'{{-Dtest.exclude=TESTCLASS}}' would be useful.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HBASE-3641) LruBlockCache.CacheStats.getHitCount() is not using the correct variable
[
https://issues.apache.org/jira/browse/HBASE-3641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13006726#comment-13006726
]
stack commented on HBASE-3641:
--
Patch looks odd:
{code}
public long getHitCachingCount() {
- return hitCachingCount.get();
+ return hitCount.get();
}
{code}
The method is called getHitCachingCount and it was returning state of
hitCachingCount... which seems right?
LruBlockCache.CacheStats.getHitCount() is not using the correct variable
Key: HBASE-3641
URL: https://issues.apache.org/jira/browse/HBASE-3641
Project: HBase
Issue Type: Bug
Components: io
Affects Versions: 0.90.1, 0.92.0
Reporter: Jonathan Gray
Assignee: Jonathan Gray
Fix For: 0.90.2, 0.92.0
Attachments: HBASE-3641-v1.patch
{code}
public long getHitCount() {
return hitCachingCount.get();
}
{code}
This should be {{hitCount.get()}}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HBASE-3437) Allow Explicit Splits from the Shell
[ https://issues.apache.org/jira/browse/HBASE-3437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13006731#comment-13006731 ] stack commented on HBASE-3437: -- +1 for patch and +1 for 0.90 Allow Explicit Splits from the Shell Key: HBASE-3437 URL: https://issues.apache.org/jira/browse/HBASE-3437 Project: HBase Issue Type: Improvement Components: shell Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Priority: Trivial Fix For: 0.92.0 Attachments: HBASE-3437.patch There is no way currently to issue explicit splits from the shell. You can only accomplish it through HBaseAdmin currently. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HBASE-3637) Region stuck in OPENED state
[ https://issues.apache.org/jira/browse/HBASE-3637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13006738#comment-13006738 ] Todd Lipcon commented on HBASE-3637: sure Region stuck in OPENED state Key: HBASE-3637 URL: https://issues.apache.org/jira/browse/HBASE-3637 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.92.0 Reporter: Todd Lipcon Priority: Critical Fix For: 0.92.0 I don't 100% understand how this happened, but the following was observed: - META is in OPENED state in ZK, for a server which no longer exists - Handler sees that server is dead, and figures that the RIT timeout will handle it - RIT timeout sees that it's already in OPENED state, and assumes that the OPENED handler will handle it - loops in timeout state forever, never actually getting reassigned -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Resolved: (HBASE-3639) FSUtils.getRootDir should qualify path
[ https://issues.apache.org/jira/browse/HBASE-3639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon resolved HBASE-3639. Resolution: Fixed Hadoop Flags: [Reviewed] FSUtils.getRootDir should qualify path -- Key: HBASE-3639 URL: https://issues.apache.org/jira/browse/HBASE-3639 Project: HBase Issue Type: Bug Affects Versions: 0.90.1 Reporter: Todd Lipcon Priority: Critical Fix For: 0.90.2 Attachments: hbase-3639.txt Currently you can run into a StackOverflowError if the hbase root dir is on the non-default filesystem. Making FSUtils.getRootDir qualify its path solves this issue. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HBASE-3625) improve/fix support excluding Tests via Maven -D property
[
https://issues.apache.org/jira/browse/HBASE-3625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Todd Lipcon updated HBASE-3625:
---
Resolution: Fixed
Fix Version/s: 0.90.2
Assignee: Alejandro Abdelnur
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
improve/fix support excluding Tests via Maven -D property
-
Key: HBASE-3625
URL: https://issues.apache.org/jira/browse/HBASE-3625
Project: HBase
Issue Type: Improvement
Components: build
Affects Versions: 0.90.1
Environment: all
Reporter: Alejandro Abdelnur
Assignee: Alejandro Abdelnur
Labels: build
Fix For: 0.90.2
Attachments: hbase-3625.txt
Currently the surefire plugin configuration defines the following exclusion:
{code}
.
plugin
groupIdorg.apache.maven.plugins/groupId
artifactIdmaven-surefire-plugin/artifactId
configuration
forkModealways/forkMode
includes
include**/Test*.java/include
/includes
excludes
exclude**/*$*/exclude
/excludes
/configuration
/plugin
{code}
AFAICT the '{{***/***$**}}' does not resolve to anything meaningful.
Adding support to exclude one or more tests via Maven property, i.e.
'{{-Dtest.exclude=TESTCLASS}}' would be useful.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HBASE-3641) LruBlockCache.CacheStats.getHitCount() is not using the correct variable
[
https://issues.apache.org/jira/browse/HBASE-3641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13006752#comment-13006752
]
stack commented on HBASE-3641:
--
+1 (You must be a little busy Jon -- smile)
LruBlockCache.CacheStats.getHitCount() is not using the correct variable
Key: HBASE-3641
URL: https://issues.apache.org/jira/browse/HBASE-3641
Project: HBase
Issue Type: Bug
Components: io
Affects Versions: 0.90.1, 0.92.0
Reporter: Jonathan Gray
Assignee: Jonathan Gray
Fix For: 0.90.2, 0.92.0
Attachments: HBASE-3641-v1.patch, HBASE-3641-v2.patch
{code}
public long getHitCount() {
return hitCachingCount.get();
}
{code}
This should be {{hitCount.get()}}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HBASE-3638) If a FS bootstrap, need to also ensure ZK is cleaned
[
https://issues.apache.org/jira/browse/HBASE-3638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13006754#comment-13006754
]
stack commented on HBASE-3638:
--
HBASE-3637 has more background to this issue.
If a FS bootstrap, need to also ensure ZK is cleaned
Key: HBASE-3638
URL: https://issues.apache.org/jira/browse/HBASE-3638
Project: HBase
Issue Type: Bug
Reporter: stack
Priority: Minor
In a test environment where a cycle of start, operation, kill hbase (repeat),
noticed that we were doing a bootstrap on startup but then we were picking up
the previous cycles zk state. It made for a mess in the test.
Last thing seen on previous cycle was:
{code}
2011-03-11 06:33:36,708 DEBUG
org.apache.hadoop.hbase.master.AssignmentManager: Handling
transition=RS_ZK_REGION_OPENING, server=X.X.X.60020,1299853933073,
region=1028785192/.META.
{code}
Then, in the messed up cycle I saw:
{code}
2011-03-11 06:42:48,530 INFO org.apache.hadoop.hbase.master.MasterFileSystem:
BOOTSTRAP: creating ROOT and first META regions
.
{code}
Then after setting watcher on .META., we get a
{code}
2011-03-11 06:42:58,301 INFO
org.apache.hadoop.hbase.master.AssignmentManager: Processing region
.META.,,1.1028785192 in state RS_ZK_REGION_OPENED
2011-03-11 06:42:58,302 WARN
org.apache.hadoop.hbase.master.AssignmentManager: Region in transition
1028785192 references a server no longer up X.X.X; letting RIT timeout so
will be assigned elsewhere
{code}
We're all confused.
Should at least clear our zk if a bootstrap happened.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Reopened: (HBASE-3437) Allow Explicit Splits from the Shell
[ https://issues.apache.org/jira/browse/HBASE-3437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg reopened HBASE-3437: Allow Explicit Splits from the Shell Key: HBASE-3437 URL: https://issues.apache.org/jira/browse/HBASE-3437 Project: HBase Issue Type: Improvement Components: shell Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Priority: Trivial Fix For: 0.90.2, 0.92.0 Attachments: HBASE-3437.patch There is no way currently to issue explicit splits from the shell. You can only accomplish it through HBaseAdmin currently. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Resolved: (HBASE-3437) Allow Explicit Splits from the Shell
[ https://issues.apache.org/jira/browse/HBASE-3437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg resolved HBASE-3437. Resolution: Fixed Fix Version/s: 0.90.2 Allow Explicit Splits from the Shell Key: HBASE-3437 URL: https://issues.apache.org/jira/browse/HBASE-3437 Project: HBase Issue Type: Improvement Components: shell Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Priority: Trivial Fix For: 0.90.2, 0.92.0 Attachments: HBASE-3437.patch There is no way currently to issue explicit splits from the shell. You can only accomplish it through HBaseAdmin currently. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Resolved: (HBASE-3448) RegionSplitter : Utility class for manual region splitting
[ https://issues.apache.org/jira/browse/HBASE-3448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg resolved HBASE-3448. Resolution: Fixed Fix Version/s: 0.90.2 RegionSplitter : Utility class for manual region splitting -- Key: HBASE-3448 URL: https://issues.apache.org/jira/browse/HBASE-3448 Project: HBase Issue Type: New Feature Components: client, scripts, util Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Priority: Minor Fix For: 0.90.2, 0.92.0 Attachments: HBASE-3448.patch For certain use cases, there are a number of advantages to manually splitting regions instead of having the HBase split code determine this for you automatically. There are currently some API additions to HBaseAdmin and HTable that allow you to manually split on a small scale. This JIRA is about importing a RegionSplitter utility program to help pre-split and perform rolling splits on a live table when needed. Will also add documentation to answer common questions about why you would pre-split. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Reopened: (HBASE-3448) RegionSplitter : Utility class for manual region splitting
[ https://issues.apache.org/jira/browse/HBASE-3448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg reopened HBASE-3448: RegionSplitter : Utility class for manual region splitting -- Key: HBASE-3448 URL: https://issues.apache.org/jira/browse/HBASE-3448 Project: HBase Issue Type: New Feature Components: client, scripts, util Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Priority: Minor Fix For: 0.90.2, 0.92.0 Attachments: HBASE-3448.patch For certain use cases, there are a number of advantages to manually splitting regions instead of having the HBase split code determine this for you automatically. There are currently some API additions to HBaseAdmin and HTable that allow you to manually split on a small scale. This JIRA is about importing a RegionSplitter utility program to help pre-split and perform rolling splits on a live table when needed. Will also add documentation to answer common questions about why you would pre-split. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HBASE-2495) Allow record filtering with selected row key values in HBase Export
[
https://issues.apache.org/jira/browse/HBASE-2495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ian Knome updated HBASE-2495:
-
Attachment:
HBASE-2495_-_Allow_record_filtering_with_selected_row_key_values_in_HBase_Export.patch
Allow record filtering with selected row key values in HBase Export
---
Key: HBASE-2495
URL: https://issues.apache.org/jira/browse/HBASE-2495
Project: HBase
Issue Type: Improvement
Components: util
Affects Versions: 0.20.3
Reporter: Ted Yu
Labels: moved_from_0_20_5
Fix For: 0.92.0
Attachments:
HBASE-2495_-_Allow_record_filtering_with_selected_row_key_values_in_HBase_Export.patch
It is desirable to add record filtering capability to HBase Export.
The following code is an example (s is the Scan):
byte [] prefix = Bytes.toBytes(args[5]);
if (args[5].startsWith(^))
{
s.setFilter(new RowFilter(CompareOp.EQUAL, new
RegexStringComparator(args[5])));
}
else s.setFilter(new PrefixFilter(prefix));
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HBASE-2495) Allow record filtering with selected row key values in HBase Export
[
https://issues.apache.org/jira/browse/HBASE-2495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13006791#comment-13006791
]
Ian Knome commented on HBASE-2495:
--
First version of patch for review.
I have tested the fix on my local cluster and the export now works with an
optional filter criteria.
I am not sure about unit tests for this. Any thoughts? Suggestions?
Allow record filtering with selected row key values in HBase Export
---
Key: HBASE-2495
URL: https://issues.apache.org/jira/browse/HBASE-2495
Project: HBase
Issue Type: Improvement
Components: util
Affects Versions: 0.20.3
Reporter: Ted Yu
Labels: moved_from_0_20_5
Fix For: 0.92.0
Attachments:
HBASE-2495_-_Allow_record_filtering_with_selected_row_key_values_in_HBase_Export.patch
It is desirable to add record filtering capability to HBase Export.
The following code is an example (s is the Scan):
byte [] prefix = Bytes.toBytes(args[5]);
if (args[5].startsWith(^))
{
s.setFilter(new RowFilter(CompareOp.EQUAL, new
RegexStringComparator(args[5])));
}
else s.setFilter(new PrefixFilter(prefix));
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HBASE-3607) Cursor functionality for results generated by Coprocessors
[
https://issues.apache.org/jira/browse/HBASE-3607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13006796#comment-13006796
]
stack commented on HBASE-3607:
--
So, what happens if the region moves mid-cursor-scan?
What is CursorCallable adding over and above Scanner? Its not clear to me
(Pardon me).
You are inconsistent in your formatting:
{code}
+if(cache.size() ==0 this.closed)
+ return null;
+if(cache.size() ==0){//do a rpc and fetch results
+ Result[] res = this.htable.getConnection().getRegi
{code}
These are pretty radical additions:
{code}
+ /*
+ * get result from cp cursor
+ */
+ public Result[] nextCp(long cursorId, int cache) throws IOException;
+ /**
+ * closing the associated cursor object and release its region level
resources
+ * @param cursorId
+ * @throws IOException
+ */
+ public void closeCp(long cursorId) throws IOException;
{code}
Are they necessary? Why do we have to mod the HRegion when we have CPs now?
Yeah, same for these additions to HRegionServer.
I do not see the direct benefit to all these big changes Himanshu. Help me
understand.
Cursor functionality for results generated by Coprocessors
--
Key: HBASE-3607
URL: https://issues.apache.org/jira/browse/HBASE-3607
Project: HBase
Issue Type: New Feature
Components: coprocessors
Reporter: Himanshu Vashishtha
Attachments: patch-2.txt
I tried to come up with a scanner like functionality for results generated by
coprocessors at region level.
This is just a poc, and it will be good to have your comments on it.
It has support for both Incremental and In-memory Result sets. Attached is a
patch that has a test case for an incremental result (i.e., client receives a
cursorId from the CP core method, it instantiates a cursor object and
iterates over the result set. He can set a cache limit on the CursorCallable
object to reduce the number of rpc -- just like scanners.
In its current state, it has some limitations too :)), like, it is region
specific only, i.e., one can instantiate and use cursor at one region only
(and that region is determined by the input row while instantiating the
cursor). I will try to expand it so that it can have atleast a sequential
access to other regions, but as I said, I want the opinion of experts to know
whether this approach really makes some sense or not.
I have tested it with the inbuilt testing framework on my laptop only.
It will be good if I copy the use case here in the description too:
Test table has rows like:
/**
* The scenario is that I have these rows keys in the test table:
'aaa-123'
'aaa-456'
'abc-111'
'abd-111'
'abd-222'
I want to return:
('aaa', 2)
('abc', 1)
('abd', 2)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HBASE-2495) Allow record filtering with selected row key values in HBase Export
[
https://issues.apache.org/jira/browse/HBASE-2495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13006797#comment-13006797
]
stack commented on HBASE-2495:
--
Why did you change the comment on the license? You are making it different to
all the other licenses in the code base.
Yeah, you change the comment style elsewhere too... suggest you leave it.
Be careful w/ your formatting here:
{code}
+if (exportFilter!= null) {
+LOG.info(Setting Scan Filter for Export.);
+ s.setFilter(exportFilter);
+}
{code}
Also, we do 80 chars a line usually.
Just FYI.
Would suggest you change name of getExportFilter to getFilter since it can
return prefix or regex?
This patch looks great. Regards a unit test, that seems a bit tough to do.
What if you pasted into the issue examples of your exercising this new
functionality? That'd be good enough I'd say.
Nice one Ian.
Allow record filtering with selected row key values in HBase Export
---
Key: HBASE-2495
URL: https://issues.apache.org/jira/browse/HBASE-2495
Project: HBase
Issue Type: Improvement
Components: util
Affects Versions: 0.20.3
Reporter: Ted Yu
Labels: moved_from_0_20_5
Fix For: 0.92.0
Attachments:
HBASE-2495_-_Allow_record_filtering_with_selected_row_key_values_in_HBase_Export.patch
It is desirable to add record filtering capability to HBase Export.
The following code is an example (s is the Scan):
byte [] prefix = Bytes.toBytes(args[5]);
if (args[5].startsWith(^))
{
s.setFilter(new RowFilter(CompareOp.EQUAL, new
RegexStringComparator(args[5])));
}
else s.setFilter(new PrefixFilter(prefix));
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HBASE-3644) HServerAddress Violates Equivalence Relations
[ https://issues.apache.org/jira/browse/HBASE-3644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13006798#comment-13006798 ] stack commented on HBASE-3644: -- Does it help that HBASE-1502 is trying to strip out HServerAddress N? Its adding in a new ServerName instead, a carrier for the String hostname, the int port, and the long startcode. No DNS in the mix. HBASE-1502 is about removing heartbeats but along the way deprecating HSA and HServerInfo. HServerAddress Violates Equivalence Relations - Key: HBASE-3644 URL: https://issues.apache.org/jira/browse/HBASE-3644 Project: HBase Issue Type: Bug Affects Versions: 0.90.2, 0.92.0 Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Fix For: 0.90.2, 0.92.0 See HBASE-3387 or http://www.ibm.com/developerworks/java/library/j-jtp05273.html#N10184 . Basically, 'a' denotes HServerAddress(DNS) 'b' denotes HServerAddress(nslookup(DNS)). This is extremely common within HBase when 'conf/regionserver' contains DNS entries because ClusterStatus.getServers() is IP-based. You have a.address.equals(b.address) !a.stringValue.equals(b.stringValue). In this case, a.equals(b) while a.hashCode() != b.hashCode(). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
