[jira] [Commented] (HBASE-5843) Improve HBase MTTR - Mean Time To Recover
[ https://issues.apache.org/jira/browse/HBASE-5843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13561487#comment-13561487 ] nkeywal commented on HBASE-5843: Agreed, you need to be a kind of expert to make the difference between 'slow for the moment' and 'broken because we have a bad internal state'. This said, if the issue comes from a coprocessor bug, you just need to be an expert of this coprocessor. Likely, the number of threads/memory consumed/file descriptors/... could be considered as indicators that something is going wrong, but it could be already too late for a clean stop as well... > Improve HBase MTTR - Mean Time To Recover > - > > Key: HBASE-5843 > URL: https://issues.apache.org/jira/browse/HBASE-5843 > Project: HBase > Issue Type: Umbrella >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal > > A part of the approach is described here: > https://docs.google.com/document/d/1z03xRoZrIJmg7jsWuyKYl6zNournF_7ZHzdi0qz_B4c/edit > The ideal target is: > - failure impact client applications only by an added delay to execute a > query, whatever the failure. > - this delay is always inferior to 1 second. > We're not going to achieve that immediately... > Priority will be given to the most frequent issues. > Short term: > - software crash > - standard administrative tasks as stop/start of a cluster. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5930) Periodically flush the Memstore?
[ https://issues.apache.org/jira/browse/HBASE-5930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13561495#comment-13561495 ] nkeywal commented on HBASE-5930: bq. With this, maybe we will no longer need skipWAL if we can prove that deferred flush is as fast as skip WAL. In standard database, skipping the WAL is often used when you're doing a functional upgrade requiring some unavailability time, i.e.: - dump - run batch scripts to update your data - if anything goes wrong reload the dump For hundreds of reasons it makes much less sense with HBase, but it could happen (some companies don't need 24x24). So we should not remove the skipWAL imho, except if it really simplify something internally. On the patch itself, I have a question on adding some randomness. The scenario I'm thinking about is a massive but periodic update on a table: all the regions will be written simultaneously, hence flushed simultaneously. That's the main use case for this JIRA, and this could hammer the namenode, imho. Except if we thing there is enough randomness by having a different flusher by regionserver (which may not be the case if all regions servers are started simultaneously). As a side note, I would personally like a flush interval of 10 minutes: - it would help on .META. recovery, especially with the separate wal for .META. - this allows to have more regions: today, on average and in theory, each region takes 50% of an hdfs block size of memory. The more regions we flush early, the more empty memstore we have... > Periodically flush the Memstore? > > > Key: HBASE-5930 > URL: https://issues.apache.org/jira/browse/HBASE-5930 > Project: HBase > Issue Type: Improvement >Reporter: Lars Hofhansl >Assignee: Devaraj Das >Priority: Minor > Fix For: 0.96.0 > > Attachments: 5930-1.patch, 5930-wip.patch > > > A colleague of mine ran into an interesting issue. > He inserted some data with the WAL disabled, which happened to fit in the > aggregate Memstores memory. > Two weeks later he a had problem with the HDFS cluster, which caused the > region servers to abort. He found that his data was lost. Looking at the log > we found that the Memstores were not flushed at all during these two weeks. > Should we have an option to flush memstores periodically. There are obvious > downsides to this, like many small storefiles, etc. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7643) HFileArchiver.resolveAndArchive() race condition may lead to snapshot data loss
[
https://issues.apache.org/jira/browse/HBASE-7643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13561500#comment-13561500
]
Hadoop QA commented on HBASE-7643:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12566268/HBASE-7653-p4-v6.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to introduce 1 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/4156//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4156//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4156//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4156//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4156//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4156//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4156//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/4156//console
This message is automatically generated.
> HFileArchiver.resolveAndArchive() race condition may lead to snapshot data
> loss
> ---
>
> Key: HBASE-7643
> URL: https://issues.apache.org/jira/browse/HBASE-7643
> Project: HBase
> Issue Type: Bug
>Affects Versions: hbase-6055, 0.96.0
>Reporter: Matteo Bertozzi
>Assignee: Matteo Bertozzi
>Priority: Blocker
> Fix For: 0.96.0, 0.94.5
>
> Attachments: HBASE-7653-p4-v0.patch, HBASE-7653-p4-v1.patch,
> HBASE-7653-p4-v2.patch, HBASE-7653-p4-v3.patch, HBASE-7653-p4-v4.patch,
> HBASE-7653-p4-v5.patch, HBASE-7653-p4-v6.patch
>
>
> * The master have an hfile cleaner thread (that is responsible for cleaning
> the /hbase/.archive dir)
> ** /hbase/.archive/table/region/family/hfile
> ** if the family/region/family directory is empty the cleaner removes it
> * The master can archive files (from another thread, e.g. DeleteTableHandler)
> * The region can archive files (from another server/process, e.g. compaction)
> The simplified file archiving code looks like this:
> {code}
> HFileArchiver.resolveAndArchive(...) {
> // ensure that the archive dir exists
> fs.mkdir(archiveDir);
> // move the file to the archiver
> success = fs.rename(originalPath/fileName, archiveDir/fileName)
> // if the rename is failed, delete the file without archiving
> if (!success) fs.delete(originalPath/fileName);
> }
> {code}
> Since there's no synchronization between HFileArchiver.resolveAndArchive()
> and the cleaner run (different process, thread, ...) you can end up in the
> situation where you are moving something in a directory that doesn't exists.
> {code}
> fs.mkdir(archiveDir);
> // HFileCleaner chore starts at this point
> // and the archiveDirectory that we just ensured to be present gets removed.
> // The rename at this point will fail since the parent directory is missing.
> success = fs.rename(originalPath/fileName, archiveDir/fileName)
> {code}
> The bad thing of deleting the file without archiving is that if you've a
> snapshot that relies on the file to be present, or you've a clone table that
> relies on that file is that you're losing data.
> Possible solutions
> * Create a ZooKeeper lock, to notify the master ("Hey I'm archiving
> something, wait a bit")
> * Add a RS -> Master call to let the master removes files and avoid this
> kind of situations
> * Avoid to remove empty directorie
[jira] [Updated] (HBASE-7495) parallel seek in StoreScanner
[
https://issues.apache.org/jira/browse/HBASE-7495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
liang xie updated HBASE-7495:
-
Attachment: HBASE-7495.txt
> parallel seek in StoreScanner
> -
>
> Key: HBASE-7495
> URL: https://issues.apache.org/jira/browse/HBASE-7495
> Project: HBase
> Issue Type: Bug
> Components: Scanners
>Affects Versions: 0.94.3, 0.96.0
>Reporter: liang xie
>Assignee: liang xie
> Attachments: HBASE-7495.txt, HBASE-7495.txt, HBASE-7495.txt
>
>
> seems there's a potential improvable space before doing scanner.next:
> {code:title=StoreScanner.java|borderStyle=solid}
> if (explicitColumnQuery && lazySeekEnabledGlobally) {
> for (KeyValueScanner scanner : scanners) {
> scanner.requestSeek(matcher.getStartKey(), false, true);
> }
> } else {
> for (KeyValueScanner scanner : scanners) {
> scanner.seek(matcher.getStartKey());
> }
> }
> {code}
> we can do scanner.requestSeek or scanner.seek in parallel, instead of current
> serialization, to reduce latency for special case.
> Any ideas on it ? I'll have a try if the comments/suggestions are positive：）
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-7653) scan CF in parallel
liang xie created HBASE-7653: Summary: scan CF in parallel Key: HBASE-7653 URL: https://issues.apache.org/jira/browse/HBASE-7653 Project: HBase Issue Type: Improvement Components: regionserver Affects Versions: 0.96.0 Reporter: liang xie Assignee: liang xie just placeholder, the idea comes from HBASE-7495, per lars' comments. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-7654) Add List getCoprocessors() to HTableInterface.
Jean-Marc Spaggiari created HBASE-7654: -- Summary: Add List getCoprocessors() to HTableInterface. Key: HBASE-7654 URL: https://issues.apache.org/jira/browse/HBASE-7654 Project: HBase Issue Type: Bug Reporter: Jean-Marc Spaggiari Assignee: Jean-Marc Spaggiari Priority: Critical Add List getCoprocessors() to HTableInterface to retreive the list of coprocessors loaded into this table. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7495) parallel seek in StoreScanner
[
https://issues.apache.org/jira/browse/HBASE-7495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13561587#comment-13561587
]
Hadoop QA commented on HBASE-7495:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12566294/HBASE-7495.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to introduce 1 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.TestHRegion
org.apache.hadoop.hbase.regionserver.TestAtomicOperation
org.apache.hadoop.hbase.TestAcidGuarantees
org.apache.hadoop.hbase.regionserver.TestHRegionBusyWait
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/4157//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4157//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4157//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4157//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4157//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4157//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4157//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/4157//console
This message is automatically generated.
> parallel seek in StoreScanner
> -
>
> Key: HBASE-7495
> URL: https://issues.apache.org/jira/browse/HBASE-7495
> Project: HBase
> Issue Type: Bug
> Components: Scanners
>Affects Versions: 0.94.3, 0.96.0
>Reporter: liang xie
>Assignee: liang xie
> Attachments: HBASE-7495.txt, HBASE-7495.txt, HBASE-7495.txt
>
>
> seems there's a potential improvable space before doing scanner.next:
> {code:title=StoreScanner.java|borderStyle=solid}
> if (explicitColumnQuery && lazySeekEnabledGlobally) {
> for (KeyValueScanner scanner : scanners) {
> scanner.requestSeek(matcher.getStartKey(), false, true);
> }
> } else {
> for (KeyValueScanner scanner : scanners) {
> scanner.seek(matcher.getStartKey());
> }
> }
> {code}
> we can do scanner.requestSeek or scanner.seek in parallel, instead of current
> serialization, to reduce latency for special case.
> Any ideas on it ? I'll have a try if the comments/suggestions are positive：）
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7654) Add List getCoprocessors() to HTableInterface.
[ https://issues.apache.org/jira/browse/HBASE-7654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean-Marc Spaggiari updated HBASE-7654: --- Attachment: HBASE-7654-v0-trunk.patch > Add List getCoprocessors() to HTableInterface. > -- > > Key: HBASE-7654 > URL: https://issues.apache.org/jira/browse/HBASE-7654 > Project: HBase > Issue Type: Bug >Reporter: Jean-Marc Spaggiari >Assignee: Jean-Marc Spaggiari >Priority: Critical > Attachments: HBASE-7654-v0-trunk.patch > > > Add List getCoprocessors() to HTableInterface to retreive the list of > coprocessors loaded into this table. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7654) Add List getCoprocessors() to HTableInterface.
[ https://issues.apache.org/jira/browse/HBASE-7654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean-Marc Spaggiari updated HBASE-7654: --- Status: Patch Available (was: Open) Pretty trivial patch to add List getCoprocessors() to HTableDescriptor. Also adding the related tests, and cleaning some extra non-required spaces. > Add List getCoprocessors() to HTableInterface. > -- > > Key: HBASE-7654 > URL: https://issues.apache.org/jira/browse/HBASE-7654 > Project: HBase > Issue Type: Bug >Reporter: Jean-Marc Spaggiari >Assignee: Jean-Marc Spaggiari >Priority: Critical > Attachments: HBASE-7654-v0-trunk.patch > > > Add List getCoprocessors() to HTableInterface to retreive the list of > coprocessors loaded into this table. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7634) Replication handling of changes to peer clusters is inefficient
[ https://issues.apache.org/jira/browse/HBASE-7634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabriel Reid updated HBASE-7634: Status: Patch Available (was: Open) > Replication handling of changes to peer clusters is inefficient > --- > > Key: HBASE-7634 > URL: https://issues.apache.org/jira/browse/HBASE-7634 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 0.96.0 >Reporter: Gabriel Reid > Attachments: HBASE-7634.patch > > > The current handling of changes to the region servers in a replication peer > cluster is currently quite inefficient. The list of region servers that are > being replicated to is only updated if there are a large number of issues > encountered while replicating. > This can cause it to take quite a while to recognize that a number of the > regionserver in a peer cluster are no longer available. A potentially bigger > problem is that if a replication peer cluster is started with a small number > of regionservers, and then more region servers are added after replication > has started, the additional region servers will never be used for replication > (unless there are failures on the in-use regionservers). > Part of the current issue is that the retry code in > ReplicationSource#shipEdits checks a randomly-chosen replication peer > regionserver (in ReplicationSource#isSlaveDown) to see if it is up after a > replication write has failed on a different randonly-chosen replication peer. > If the peer is seen as not down, another randomly-chosen peer is used for > writing. > A second part of the issue is that changes to the list of region servers in a > peer cluster are not detected at all, and are only picked up if a certain > number of failures have occurred when trying to ship edits. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7654) Add List getCoprocessors() to HTableInterface.
[
https://issues.apache.org/jira/browse/HBASE-7654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13561625#comment-13561625
]
Hadoop QA commented on HBASE-7654:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12566301/HBASE-7654-v0-trunk.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to introduce 1 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/4158//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4158//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4158//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4158//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4158//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4158//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4158//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/4158//console
This message is automatically generated.
> Add List getCoprocessors() to HTableInterface.
> --
>
> Key: HBASE-7654
> URL: https://issues.apache.org/jira/browse/HBASE-7654
> Project: HBase
> Issue Type: Bug
>Reporter: Jean-Marc Spaggiari
>Assignee: Jean-Marc Spaggiari
>Priority: Critical
> Attachments: HBASE-7654-v0-trunk.patch
>
>
> Add List getCoprocessors() to HTableInterface to retreive the list of
> coprocessors loaded into this table.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7634) Replication handling of changes to peer clusters is inefficient
[
https://issues.apache.org/jira/browse/HBASE-7634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13561626#comment-13561626
]
Hadoop QA commented on HBASE-7634:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12565799/HBASE-7634.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 6 new
or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/4159//console
This message is automatically generated.
> Replication handling of changes to peer clusters is inefficient
> ---
>
> Key: HBASE-7634
> URL: https://issues.apache.org/jira/browse/HBASE-7634
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Affects Versions: 0.96.0
>Reporter: Gabriel Reid
> Attachments: HBASE-7634.patch
>
>
> The current handling of changes to the region servers in a replication peer
> cluster is currently quite inefficient. The list of region servers that are
> being replicated to is only updated if there are a large number of issues
> encountered while replicating.
> This can cause it to take quite a while to recognize that a number of the
> regionserver in a peer cluster are no longer available. A potentially bigger
> problem is that if a replication peer cluster is started with a small number
> of regionservers, and then more region servers are added after replication
> has started, the additional region servers will never be used for replication
> (unless there are failures on the in-use regionservers).
> Part of the current issue is that the retry code in
> ReplicationSource#shipEdits checks a randomly-chosen replication peer
> regionserver (in ReplicationSource#isSlaveDown) to see if it is up after a
> replication write has failed on a different randonly-chosen replication peer.
> If the peer is seen as not down, another randomly-chosen peer is used for
> writing.
> A second part of the issue is that changes to the list of region servers in a
> peer cluster are not detected at all, and are only picked up if a certain
> number of failures have occurred when trying to ship edits.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7651) RegionServerSnapshotManager fails with CancellationException if previous snapshot fails in per region task
[
https://issues.apache.org/jira/browse/HBASE-7651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hsieh updated HBASE-7651:
--
Summary: RegionServerSnapshotManager fails with CancellationException if
previous snapshot fails in per region task (was: RegionServerSnapshotManager
fails with CancellationException if previous fails with in per region task)
> RegionServerSnapshotManager fails with CancellationException if previous
> snapshot fails in per region task
> --
>
> Key: HBASE-7651
> URL: https://issues.apache.org/jira/browse/HBASE-7651
> Project: HBase
> Issue Type: Sub-task
> Components: snapshots
>Affects Versions: hbase-7290
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
>Priority: Blocker
>
> I've reproduced this problem consistently on a 20 node cluster.
> The first run fails on a node (jon-snaphots-2 in this case) to take snapshot
> due to a NotServingRegionException (this is acceptable)
> {code}
> 2013-01-23 13:32:48,631 DEBUG
> org.apache.hadoop.hbase.errorhandling.ForeignExceptionDispatcher: accepting
> received exception
> org.apache.hadoop.hbase.errorhandling.ForeignException$ProxyThrowable via
> jon-snapshots-2.ent.cloudera.com,22101,1358976524369:org.apache.hadoop.hbase.errorhandling.ForeignException$ProxyThrowable:
> org.apache.hadoop.hbase.NotServingRegionException:
> TestTable,0002493652,1358976652443.b858147ad87a7812ac9a73dd8fef36ad. is
> closing
> at
> org.apache.hadoop.hbase.errorhandling.ForeignException.deserialize(ForeignException.java:184)
> at
> org.apache.hadoop.hbase.procedure.ZKProcedureCoordinatorRpcs.abort(ZKProcedureCoordinatorRpcs.java:240)
> at
> org.apache.hadoop.hbase.procedure.ZKProcedureCoordinatorRpcs$1.nodeCreated(ZKProcedureCoordinatorRpcs.java:182)
> at
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.process(ZooKeeperWatcher.java:294)
> at
> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
> at
> org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
> Caused by:
> org.apache.hadoop.hbase.errorhandling.ForeignException$ProxyThrowable:
> org.apache.hadoop.hbase.NotServingRegionException:
> TestTable,0002493652,1358976652443.b858147ad87a7812ac9a73dd8fef36ad. is
> closing
> at
> org.apache.hadoop.hbase.regionserver.snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool.waitForOutstandingTasks(RegionServerSnapshotManager.java:343)
> at
> org.apache.hadoop.hbase.regionserver.snapshot.FlushSnapshotSubprocedure.flushSnapshot(FlushSnapshotSubprocedure.java:107)
> at
> org.apache.hadoop.hbase.regionserver.snapshot.FlushSnapshotSubprocedure.insideBarrier(FlushSnapshotSubprocedure.java:123)
> at
> org.apache.hadoop.hbase.procedure.Subprocedure.call(Subprocedure.java:181)
> at
> org.apache.hadoop.hbase.procedure.Subprocedure.call(Subprocedure.java:52)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> 2013-01-23 13:32:48,631 DEBUG
> org.apache.hadoop.hbase.errorhandling.ForeignExceptionDispatcher: Recieved
> error, notifying listeners...
> 2013-01-23 13:32:48,730 ERROR org.apache.hadoop.hbase.procedure.Procedure:
> Procedure 'pe-6' execution failed!
> org.apache.hadoop.hbase.errorhandling.ForeignException$ProxyThrowable via
> jon-snapshots-2.ent.cloudera.com,22101,1358976524369:org.apache.hadoop.hbase.errorhandling.ForeignException$ProxyThrowable:
> org.apache.hadoop.hbase.NotServingRegionException:
> TestTable,0002493652,1358976652443.b858147ad87a7812ac9a73dd8fef36ad. is
> closing
> at
> org.apache.hadoop.hbase.errorhandling.ForeignExceptionDispatcher.rethrowException(ForeignExceptionDispatcher.java:84)
> at
> org.apache.hadoop.hbase.procedure.Procedure.waitForLatch(Procedure.java:357)
> at
> org.apache.hadoop.hbase.procedure.Procedure.call(Procedure.java:203)
> at org.apache.hadoop.hbase.procedure.Procedure.call(Procedure.java:68)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by:
> o
[jira] [Updated] (HBASE-7651) RegionServerSnapshotManager fails with CancellationException if previous fails with in per region task
[
https://issues.apache.org/jira/browse/HBASE-7651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hsieh updated HBASE-7651:
--
Summary: RegionServerSnapshotManager fails with CancellationException if
previous fails with in per region task (was: RegionServerSnapshotManager does
not accept subsquent snapshots if previous fails with
NotServingRegionException.)
> RegionServerSnapshotManager fails with CancellationException if previous
> fails with in per region task
> --
>
> Key: HBASE-7651
> URL: https://issues.apache.org/jira/browse/HBASE-7651
> Project: HBase
> Issue Type: Sub-task
> Components: snapshots
>Affects Versions: hbase-7290
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
>Priority: Blocker
>
> I've reproduced this problem consistently on a 20 node cluster.
> The first run fails on a node (jon-snaphots-2 in this case) to take snapshot
> due to a NotServingRegionException (this is acceptable)
> {code}
> 2013-01-23 13:32:48,631 DEBUG
> org.apache.hadoop.hbase.errorhandling.ForeignExceptionDispatcher: accepting
> received exception
> org.apache.hadoop.hbase.errorhandling.ForeignException$ProxyThrowable via
> jon-snapshots-2.ent.cloudera.com,22101,1358976524369:org.apache.hadoop.hbase.errorhandling.ForeignException$ProxyThrowable:
> org.apache.hadoop.hbase.NotServingRegionException:
> TestTable,0002493652,1358976652443.b858147ad87a7812ac9a73dd8fef36ad. is
> closing
> at
> org.apache.hadoop.hbase.errorhandling.ForeignException.deserialize(ForeignException.java:184)
> at
> org.apache.hadoop.hbase.procedure.ZKProcedureCoordinatorRpcs.abort(ZKProcedureCoordinatorRpcs.java:240)
> at
> org.apache.hadoop.hbase.procedure.ZKProcedureCoordinatorRpcs$1.nodeCreated(ZKProcedureCoordinatorRpcs.java:182)
> at
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.process(ZooKeeperWatcher.java:294)
> at
> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
> at
> org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
> Caused by:
> org.apache.hadoop.hbase.errorhandling.ForeignException$ProxyThrowable:
> org.apache.hadoop.hbase.NotServingRegionException:
> TestTable,0002493652,1358976652443.b858147ad87a7812ac9a73dd8fef36ad. is
> closing
> at
> org.apache.hadoop.hbase.regionserver.snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool.waitForOutstandingTasks(RegionServerSnapshotManager.java:343)
> at
> org.apache.hadoop.hbase.regionserver.snapshot.FlushSnapshotSubprocedure.flushSnapshot(FlushSnapshotSubprocedure.java:107)
> at
> org.apache.hadoop.hbase.regionserver.snapshot.FlushSnapshotSubprocedure.insideBarrier(FlushSnapshotSubprocedure.java:123)
> at
> org.apache.hadoop.hbase.procedure.Subprocedure.call(Subprocedure.java:181)
> at
> org.apache.hadoop.hbase.procedure.Subprocedure.call(Subprocedure.java:52)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> 2013-01-23 13:32:48,631 DEBUG
> org.apache.hadoop.hbase.errorhandling.ForeignExceptionDispatcher: Recieved
> error, notifying listeners...
> 2013-01-23 13:32:48,730 ERROR org.apache.hadoop.hbase.procedure.Procedure:
> Procedure 'pe-6' execution failed!
> org.apache.hadoop.hbase.errorhandling.ForeignException$ProxyThrowable via
> jon-snapshots-2.ent.cloudera.com,22101,1358976524369:org.apache.hadoop.hbase.errorhandling.ForeignException$ProxyThrowable:
> org.apache.hadoop.hbase.NotServingRegionException:
> TestTable,0002493652,1358976652443.b858147ad87a7812ac9a73dd8fef36ad. is
> closing
> at
> org.apache.hadoop.hbase.errorhandling.ForeignExceptionDispatcher.rethrowException(ForeignExceptionDispatcher.java:84)
> at
> org.apache.hadoop.hbase.procedure.Procedure.waitForLatch(Procedure.java:357)
> at
> org.apache.hadoop.hbase.procedure.Procedure.call(Procedure.java:203)
> at org.apache.hadoop.hbase.procedure.Procedure.call(Procedure.java:68)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by:
> o
[jira] [Updated] (HBASE-7503) Add exists(List) in HTableInterface to allow multiple parallel exists at one time
[ https://issues.apache.org/jira/browse/HBASE-7503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean-Marc Spaggiari updated HBASE-7503: --- Status: Open (was: Patch Available) > Add exists(List) in HTableInterface to allow multiple parallel exists at one > time > - > > Key: HBASE-7503 > URL: https://issues.apache.org/jira/browse/HBASE-7503 > Project: HBase > Issue Type: Improvement >Reporter: Jean-Marc Spaggiari >Assignee: Jean-Marc Spaggiari >Priority: Minor > Fix For: 0.96.0 > > Attachments: HBASE-7503-v0-trunk.patch, HBASE-7503-v10-trunk.patch, > HBASE-7503-v11-trunk.patch, HBASE-7503-v12-trunk.patch, > HBASE-7503-v1-trunk.patch, HBASE-7503-v2-trunk.patch, > HBASE-7503-v2-trunk.patch, HBASE-7503-v3-trunk.patch, > HBASE-7503-v4-trunk.patch, HBASE-7503-v5-trunk.patch, > HBASE-7503-v7-trunk.patch, HBASE-7503-v8-trunk.patch, > HBASE-7503-v9-trunk.patch > > Original Estimate: 5m > Remaining Estimate: 5m > > We need to have a Boolean[] exists(List gets) throws IOException method > implemented in HTableInterface. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7503) Add exists(List) in HTableInterface to allow multiple parallel exists at one time
[ https://issues.apache.org/jira/browse/HBASE-7503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean-Marc Spaggiari updated HBASE-7503: --- Attachment: HBASE-7503-v13-trunk.patch > Add exists(List) in HTableInterface to allow multiple parallel exists at one > time > - > > Key: HBASE-7503 > URL: https://issues.apache.org/jira/browse/HBASE-7503 > Project: HBase > Issue Type: Improvement >Reporter: Jean-Marc Spaggiari >Assignee: Jean-Marc Spaggiari >Priority: Minor > Fix For: 0.96.0 > > Attachments: HBASE-7503-v0-trunk.patch, HBASE-7503-v10-trunk.patch, > HBASE-7503-v11-trunk.patch, HBASE-7503-v12-trunk.patch, > HBASE-7503-v13-trunk.patch, HBASE-7503-v1-trunk.patch, > HBASE-7503-v2-trunk.patch, HBASE-7503-v2-trunk.patch, > HBASE-7503-v3-trunk.patch, HBASE-7503-v4-trunk.patch, > HBASE-7503-v5-trunk.patch, HBASE-7503-v7-trunk.patch, > HBASE-7503-v8-trunk.patch, HBASE-7503-v9-trunk.patch > > Original Estimate: 5m > Remaining Estimate: 5m > > We need to have a Boolean[] exists(List gets) throws IOException method > implemented in HTableInterface. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7503) Add exists(List) in HTableInterface to allow multiple parallel exists at one time
[ https://issues.apache.org/jira/browse/HBASE-7503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean-Marc Spaggiari updated HBASE-7503: --- Status: Patch Available (was: Open) closestRowBefore added to RequestConverter parameters and ClientProtos.java back in the patch. I think I figure why I had some many troubles with adding ClientProtos.java and Protos.java. It's because eclipse don't add them on an patch if there was marked as "Merge issue", until you un-marked them manually, even if you fix the issue... Anyway. This one now should be good... > Add exists(List) in HTableInterface to allow multiple parallel exists at one > time > - > > Key: HBASE-7503 > URL: https://issues.apache.org/jira/browse/HBASE-7503 > Project: HBase > Issue Type: Improvement >Reporter: Jean-Marc Spaggiari >Assignee: Jean-Marc Spaggiari >Priority: Minor > Fix For: 0.96.0 > > Attachments: HBASE-7503-v0-trunk.patch, HBASE-7503-v10-trunk.patch, > HBASE-7503-v11-trunk.patch, HBASE-7503-v12-trunk.patch, > HBASE-7503-v13-trunk.patch, HBASE-7503-v1-trunk.patch, > HBASE-7503-v2-trunk.patch, HBASE-7503-v2-trunk.patch, > HBASE-7503-v3-trunk.patch, HBASE-7503-v4-trunk.patch, > HBASE-7503-v5-trunk.patch, HBASE-7503-v7-trunk.patch, > HBASE-7503-v8-trunk.patch, HBASE-7503-v9-trunk.patch > > Original Estimate: 5m > Remaining Estimate: 5m > > We need to have a Boolean[] exists(List gets) throws IOException method > implemented in HTableInterface. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7503) Add exists(List) in HTableInterface to allow multiple parallel exists at one time
[
https://issues.apache.org/jira/browse/HBASE-7503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13561667#comment-13561667
]
Hadoop QA commented on HBASE-7503:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12566309/HBASE-7503-v13-trunk.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 6 new
or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to introduce 2 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces lines longer than
100
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.security.access.TestZKPermissionsWatcher
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/4160//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4160//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4160//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4160//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4160//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4160//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4160//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/4160//console
This message is automatically generated.
> Add exists(List) in HTableInterface to allow multiple parallel exists at one
> time
> -
>
> Key: HBASE-7503
> URL: https://issues.apache.org/jira/browse/HBASE-7503
> Project: HBase
> Issue Type: Improvement
>Reporter: Jean-Marc Spaggiari
>Assignee: Jean-Marc Spaggiari
>Priority: Minor
> Fix For: 0.96.0
>
> Attachments: HBASE-7503-v0-trunk.patch, HBASE-7503-v10-trunk.patch,
> HBASE-7503-v11-trunk.patch, HBASE-7503-v12-trunk.patch,
> HBASE-7503-v13-trunk.patch, HBASE-7503-v1-trunk.patch,
> HBASE-7503-v2-trunk.patch, HBASE-7503-v2-trunk.patch,
> HBASE-7503-v3-trunk.patch, HBASE-7503-v4-trunk.patch,
> HBASE-7503-v5-trunk.patch, HBASE-7503-v7-trunk.patch,
> HBASE-7503-v8-trunk.patch, HBASE-7503-v9-trunk.patch
>
> Original Estimate: 5m
> Remaining Estimate: 5m
>
> We need to have a Boolean[] exists(List gets) throws IOException method
> implemented in HTableInterface.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7638) [0.94] region cache entry should only be removed on error if the error is from the server currently in cache
[ https://issues.apache.org/jira/browse/HBASE-7638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13561670#comment-13561670 ] Ted Yu commented on HBASE-7638: --- 0.94 builds #762 and #763 hung where the patch was reverted. I think there should be some other change(s) that caused the hang. > [0.94] region cache entry should only be removed on error if the error is > from the server currently in cache > > > Key: HBASE-7638 > URL: https://issues.apache.org/jira/browse/HBASE-7638 > Project: HBase > Issue Type: Bug >Affects Versions: 0.94.4 >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Minor > Fix For: 0.94.5 > > Attachments: HBASE-7638-v0.patch > > > See HBASE-7268. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7503) Add exists(List) in HTableInterface to allow multiple parallel exists at one time
[
https://issues.apache.org/jira/browse/HBASE-7503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13561671#comment-13561671
]
Jean-Marc Spaggiari commented on HBASE-7503:
I ran org.apache.hadoop.hbase.client.TestFromClientSide3 and it's working fine
locally.
Hadoop QA failed on
org.apache.hadoop.hbase.security.access.TestZKPermissionsWatcher.testPermissionsWatcher
which seems not related to this patch.
{code}
---
T E S T S
---
Running org.apache.hadoop.hbase.client.TestFromClientSide3
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 131.651 sec
Results :
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0
{code}
{code}
[INFO]
[INFO] Reactor Summary:
[INFO]
[INFO] HBase . SUCCESS [4.311s]
[INFO] HBase - Common SUCCESS [4.439s]
[INFO] HBase - Protocol .. SUCCESS [9.436s]
[INFO] HBase - Client SUCCESS [0.309s]
[INFO] HBase - Hadoop Compatibility .. SUCCESS [0.222s]
[INFO] HBase - Hadoop One Compatibility .. SUCCESS [0.678s]
[INFO] HBase - Server SUCCESS [2:26.146s]
[INFO] HBase - Integration Tests . SUCCESS [4.427s]
[INFO] HBase - Examples .. SUCCESS [2.638s]
[INFO]
[INFO] BUILD SUCCESS
[INFO]
{code}
> Add exists(List) in HTableInterface to allow multiple parallel exists at one
> time
> -
>
> Key: HBASE-7503
> URL: https://issues.apache.org/jira/browse/HBASE-7503
> Project: HBase
> Issue Type: Improvement
>Reporter: Jean-Marc Spaggiari
>Assignee: Jean-Marc Spaggiari
>Priority: Minor
> Fix For: 0.96.0
>
> Attachments: HBASE-7503-v0-trunk.patch, HBASE-7503-v10-trunk.patch,
> HBASE-7503-v11-trunk.patch, HBASE-7503-v12-trunk.patch,
> HBASE-7503-v13-trunk.patch, HBASE-7503-v1-trunk.patch,
> HBASE-7503-v2-trunk.patch, HBASE-7503-v2-trunk.patch,
> HBASE-7503-v3-trunk.patch, HBASE-7503-v4-trunk.patch,
> HBASE-7503-v5-trunk.patch, HBASE-7503-v7-trunk.patch,
> HBASE-7503-v8-trunk.patch, HBASE-7503-v9-trunk.patch
>
> Original Estimate: 5m
> Remaining Estimate: 5m
>
> We need to have a Boolean[] exists(List gets) throws IOException method
> implemented in HTableInterface.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7503) Add exists(List) in HTableInterface to allow multiple parallel exists at one time
[ https://issues.apache.org/jira/browse/HBASE-7503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13561689#comment-13561689 ] Ted Yu commented on HBASE-7503: --- @Jean-Marc: Mind attaching your patch again ? > Add exists(List) in HTableInterface to allow multiple parallel exists at one > time > - > > Key: HBASE-7503 > URL: https://issues.apache.org/jira/browse/HBASE-7503 > Project: HBase > Issue Type: Improvement >Reporter: Jean-Marc Spaggiari >Assignee: Jean-Marc Spaggiari >Priority: Minor > Fix For: 0.96.0 > > Attachments: HBASE-7503-v0-trunk.patch, HBASE-7503-v10-trunk.patch, > HBASE-7503-v11-trunk.patch, HBASE-7503-v12-trunk.patch, > HBASE-7503-v13-trunk.patch, HBASE-7503-v1-trunk.patch, > HBASE-7503-v2-trunk.patch, HBASE-7503-v2-trunk.patch, > HBASE-7503-v3-trunk.patch, HBASE-7503-v4-trunk.patch, > HBASE-7503-v5-trunk.patch, HBASE-7503-v7-trunk.patch, > HBASE-7503-v8-trunk.patch, HBASE-7503-v9-trunk.patch > > Original Estimate: 5m > Remaining Estimate: 5m > > We need to have a Boolean[] exists(List gets) throws IOException method > implemented in HTableInterface. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7503) Add exists(List) in HTableInterface to allow multiple parallel exists at one time
[ https://issues.apache.org/jira/browse/HBASE-7503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13561693#comment-13561693 ] Jean-Marc Spaggiari commented on HBASE-7503: The same version? Or should I increment the version to get it processed? > Add exists(List) in HTableInterface to allow multiple parallel exists at one > time > - > > Key: HBASE-7503 > URL: https://issues.apache.org/jira/browse/HBASE-7503 > Project: HBase > Issue Type: Improvement >Reporter: Jean-Marc Spaggiari >Assignee: Jean-Marc Spaggiari >Priority: Minor > Fix For: 0.96.0 > > Attachments: HBASE-7503-v0-trunk.patch, HBASE-7503-v10-trunk.patch, > HBASE-7503-v11-trunk.patch, HBASE-7503-v12-trunk.patch, > HBASE-7503-v13-trunk.patch, HBASE-7503-v1-trunk.patch, > HBASE-7503-v2-trunk.patch, HBASE-7503-v2-trunk.patch, > HBASE-7503-v3-trunk.patch, HBASE-7503-v4-trunk.patch, > HBASE-7503-v5-trunk.patch, HBASE-7503-v7-trunk.patch, > HBASE-7503-v8-trunk.patch, HBASE-7503-v9-trunk.patch > > Original Estimate: 5m > Remaining Estimate: 5m > > We need to have a Boolean[] exists(List gets) throws IOException method > implemented in HTableInterface. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7503) Add exists(List) in HTableInterface to allow multiple parallel exists at one time
[ https://issues.apache.org/jira/browse/HBASE-7503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13561703#comment-13561703 ] Ted Yu commented on HBASE-7503: --- Hadoop QA looks at attachment Id, not file name. Meaning, attaching same file would do. > Add exists(List) in HTableInterface to allow multiple parallel exists at one > time > - > > Key: HBASE-7503 > URL: https://issues.apache.org/jira/browse/HBASE-7503 > Project: HBase > Issue Type: Improvement >Reporter: Jean-Marc Spaggiari >Assignee: Jean-Marc Spaggiari >Priority: Minor > Fix For: 0.96.0 > > Attachments: HBASE-7503-v0-trunk.patch, HBASE-7503-v10-trunk.patch, > HBASE-7503-v11-trunk.patch, HBASE-7503-v12-trunk.patch, > HBASE-7503-v13-trunk.patch, HBASE-7503-v1-trunk.patch, > HBASE-7503-v2-trunk.patch, HBASE-7503-v2-trunk.patch, > HBASE-7503-v3-trunk.patch, HBASE-7503-v4-trunk.patch, > HBASE-7503-v5-trunk.patch, HBASE-7503-v7-trunk.patch, > HBASE-7503-v8-trunk.patch, HBASE-7503-v9-trunk.patch > > Original Estimate: 5m > Remaining Estimate: 5m > > We need to have a Boolean[] exists(List gets) throws IOException method > implemented in HTableInterface. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7638) [0.94] region cache entry should only be removed on error if the error is from the server currently in cache
[ https://issues.apache.org/jira/browse/HBASE-7638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13561704#comment-13561704 ] Hudson commented on HBASE-7638: --- Integrated in HBase-0.94-security #96 (See [https://builds.apache.org/job/HBase-0.94-security/96/]) HBASE-7638 Revert, due to concerns of failing tests. (Revision 1437865) HBASE-7638 [0.94] region cache entry should only be removed on error if the error is from the server currently in cache (Sergey) (Revision 1437254) Result = FAILURE larsh : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/client/TestHCM.java tedyu : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/client/TestHCM.java > [0.94] region cache entry should only be removed on error if the error is > from the server currently in cache > > > Key: HBASE-7638 > URL: https://issues.apache.org/jira/browse/HBASE-7638 > Project: HBase > Issue Type: Bug >Affects Versions: 0.94.4 >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Minor > Fix For: 0.94.5 > > Attachments: HBASE-7638-v0.patch > > > See HBASE-7268. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5458) Thread safety issues with Compression.Algorithm.GZ and CompressionTest
[
https://issues.apache.org/jira/browse/HBASE-5458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13561705#comment-13561705
]
Hudson commented on HBASE-5458:
---
Integrated in HBase-0.94-security #96 (See
[https://builds.apache.org/job/HBase-0.94-security/96/])
HBASE-5458 Thread safety issues with Compression.Algorithm.GZ and
CompressionTest (Revision 1435317)
Result = FAILURE
eclark :
Files :
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/io/hfile/Compression.java
> Thread safety issues with Compression.Algorithm.GZ and CompressionTest
> --
>
> Key: HBASE-5458
> URL: https://issues.apache.org/jira/browse/HBASE-5458
> Project: HBase
> Issue Type: Bug
> Components: io
>Affects Versions: 0.90.5, 0.92.2, 0.96.0, 0.94.4
>Reporter: David McIntosh
>Assignee: Elliott Clark
>Priority: Minor
> Fix For: 0.90.7, 0.92.3, 0.96.0, 0.94.5
>
> Attachments: HBASE-5458-090-0.patch, HBASE-5458-090-1.patch,
> HBASE-5458-090-2.patch, HBASE-5458-092-2.patch, HBASE-5458-094-2.patch,
> HBASE-5458-trunk-2.patch
>
>
> I've seen some occasional NullPointerExceptions in
> ZlibFactory.isNativeZlibLoaded(conf) during region server startups and the
> completebulkload process. This is being caused by a null configuration
> getting passed to the isNativeZlibLoaded method. I think this happens when 2
> or more threads call the CompressionTest.testCompression method at once. If
> the GZ algorithm has not been tested yet both threads could continue on and
> attempt to load the compressor. For GZ the getCodec method is not thread
> safe which could lead to one thread getting a reference to a GzipCodec that
> has a null configuration.
> {code}
> current:
> DefaultCodec getCodec(Configuration conf) {
> if (codec == null) {
> codec = new GzipCodec();
> codec.setConf(new Configuration(conf));
> }
> return codec;
> }
> {code}
> one possible fix would be something like this:
> {code}
> DefaultCodec getCodec(Configuration conf) {
> if (codec == null) {
> GzipCodec gzip = new GzipCodec();
> gzip.setConf(new Configuration(conf));
> codec = gzip;
> }
> return codec;
> }
> {code}
> But that may not be totally safe without some synchronization. An upstream
> fix in CompressionTest could also prevent multi thread access to
> GZ.getCodec(conf)
> exceptions:
> 12/02/21 16:11:56 ERROR handler.OpenRegionHandler: Failed open of
> region=all-monthly,,1326263896983.bf574519a95263ec23a2bad9f5b8cbf4.
> java.io.IOException: java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:89)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.checkCompressionCodecs(HRegion.java:2670)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2659)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2647)
> at
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:312)
> at
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:99)
> at
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:158)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.io.compress.zlib.ZlibFactory.isNativeZlibLoaded(ZlibFactory.java:63)
> at
> org.apache.hadoop.io.compress.GzipCodec.getCompressorType(GzipCodec.java:166)
> at
> org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:100)
> at
> org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:112)
> at
> org.apache.hadoop.hbase.io.hfile.Compression$Algorithm.getCompressor(Compression.java:236)
> at
> org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:84)
> ... 9 more
> Caused by: java.io.IOException: java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:89)
> at
> org.apache.hadoop.hbase.io.hfile.HFile$Reader.readTrailer(HFile.java:890)
> at
> org.apache.hadoop.hbase.io.hfile.HFile$Reader.loadFileInfo(HFile.java:819)
> at
> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.groupOrSplit(LoadIncrementalHFiles.java:405)
> at
> org.apache.hadoo
[jira] [Commented] (HBASE-7293) [replication] Remove dead sinks from ReplicationSource.currentPeers and pick new ones
[
https://issues.apache.org/jira/browse/HBASE-7293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13561706#comment-13561706
]
Hudson commented on HBASE-7293:
---
Integrated in HBase-0.94-security #96 (See
[https://builds.apache.org/job/HBase-0.94-security/96/])
HBASE-7293 [replication] Remove dead sinks from
ReplicationSource.currentPeers and pick new ones (Revision 1437238)
Result = FAILURE
larsh :
Files :
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java
> [replication] Remove dead sinks from ReplicationSource.currentPeers and pick
> new ones
> -
>
> Key: HBASE-7293
> URL: https://issues.apache.org/jira/browse/HBASE-7293
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.94.3, 0.96.0
>Reporter: Jean-Daniel Cryans
>Assignee: Lars Hofhansl
> Fix For: 0.96.0, 0.94.5
>
> Attachments: 7293-0.94.txt, 7293-0.94-v2.txt, 7293-0.96.txt
>
>
> I happened to look at a log today where I saw a lot lines like this:
> {noformat}
> 2012-12-06 23:29:08,318 INFO
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Slave
> cluster looks down: This server is in the failed servers list:
> sv4r20s49/10.4.20.49:10304
> 2012-12-06 23:29:15,987 WARN
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Can't
> replicate because of a local or network error:
> java.net.ConnectException: Connection refused
> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> at
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
> at
> org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
> at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:519)
> at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:484)
> at
> org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupConnection(HBaseClient.java:416)
> at
> org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupIOstreams(HBaseClient.java:462)
> at
> org.apache.hadoop.hbase.ipc.HBaseClient.getConnection(HBaseClient.java:1150)
> at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:1000)
> at
> org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:150)
> at $Proxy14.replicateLogEntries(Unknown Source)
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.shipEdits(ReplicationSource.java:627)
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.run(ReplicationSource.java:365)
> 2012-12-06 23:29:15,988 INFO
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Slave
> cluster looks down: Connection refused
> {noformat}
> What struck me as weird is this had been going on for some days, I would
> expect the RS to find new servers if it wasn't able to replicate. But the
> reality is that only a few of the chosen sink RS were down so eventually the
> source hits one that's good and is never able to refresh its list of servers.
> We should remove the dead servers, it's spammy and probably adds some slave
> lag.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6669) Add BigDecimalColumnInterpreter for doing aggregations using AggregationClient
[ https://issues.apache.org/jira/browse/HBASE-6669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13561707#comment-13561707 ] Hudson commented on HBASE-6669: --- Integrated in HBase-0.94-security #96 (See [https://builds.apache.org/job/HBase-0.94-security/96/]) HBASE-6669 Add BigDecimalColumnInterpreter for doing aggregations using AggregationClient (Anil Gupta) (Revision 1436726) Result = FAILURE tedyu : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/client/coprocessor/BigDecimalColumnInterpreter.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/coprocessor/TestBigDecimalColumnInterpreter.java > Add BigDecimalColumnInterpreter for doing aggregations using AggregationClient > -- > > Key: HBASE-6669 > URL: https://issues.apache.org/jira/browse/HBASE-6669 > Project: HBase > Issue Type: New Feature > Components: Client, Coprocessors >Affects Versions: 0.94.3 >Reporter: Anil Gupta >Priority: Minor > Labels: client, coprocessors > Fix For: 0.94.5 > > Attachments: 6669-0.94-v4.txt, 6669-0.94-v5.txt, > BigDecimalColumnInterpreter.java, BigDecimalColumnInterpreter.patch, > BigDecimalColumnInterpreter.patch, HBASE-6669.patch, HBASE-6669-v2.patch, > HBASE-6669-v3.patch, TestBDAggregateProtocol.patch, > TestBigDecimalColumnInterpreter.java > > > I recently created a Class for doing aggregations(sum,min,max,std) on values > stored as BigDecimal in HBase. I would like to commit the > BigDecimalColumnInterpreter into HBase. In my opinion this class can be used > by a wide variety of users. Please let me know if its not appropriate to add > this class in HBase. > Thanks, > Anil Gupta > Software Engineer II, Intuit, Inc -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7585) TestAccessController tests should close HTables
[ https://issues.apache.org/jira/browse/HBASE-7585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13561708#comment-13561708 ] Hudson commented on HBASE-7585: --- Integrated in HBase-0.94-security #96 (See [https://builds.apache.org/job/HBase-0.94-security/96/]) HBASE-7585. TestAccessController tests should close HTables (Revision 1434847) Result = FAILURE apurtell : Files : * /hbase/branches/0.94/security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java > TestAccessController tests should close HTables > --- > > Key: HBASE-7585 > URL: https://issues.apache.org/jira/browse/HBASE-7585 > Project: HBase > Issue Type: Improvement > Components: test >Affects Versions: 0.96.0, 0.94.5 >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Trivial > Attachments: HBASE-7585-0.94.patch, HBASE-7585-trunk.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7584) Improve TestAccessController.testAppend
[ https://issues.apache.org/jira/browse/HBASE-7584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13561710#comment-13561710 ] Hudson commented on HBASE-7584: --- Integrated in HBase-0.94-security #96 (See [https://builds.apache.org/job/HBase-0.94-security/96/]) HBASE-7584. Improve TestAccessController.testAppend (Revision 1434099) Result = FAILURE apurtell : Files : * /hbase/branches/0.94/security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java > Improve TestAccessController.testAppend > --- > > Key: HBASE-7584 > URL: https://issues.apache.org/jira/browse/HBASE-7584 > Project: HBase > Issue Type: Bug >Affects Versions: 0.96.0, 0.94.5 >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Minor > Fix For: 0.96.0, 0.94.5 > > Attachments: HBASE-7584.patch > > > TestAccessController#testAppend should call through HTable instead of > invoking the CP hook directly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7592) HConnectionManager.getHTableDescriptor() compares too much
[
https://issues.apache.org/jira/browse/HBASE-7592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13561711#comment-13561711
]
Hudson commented on HBASE-7592:
---
Integrated in HBase-0.94-security #96 (See
[https://builds.apache.org/job/HBase-0.94-security/96/])
HBASE-7592 HConnectionManager.getHTableDescriptor() compares too much
(Revision 1434532)
Result = FAILURE
mbertozzi :
Files :
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
> HConnectionManager.getHTableDescriptor() compares too much
> --
>
> Key: HBASE-7592
> URL: https://issues.apache.org/jira/browse/HBASE-7592
> Project: HBase
> Issue Type: Bug
> Components: Client
>Affects Versions: 0.94.4
>Reporter: Matteo Bertozzi
>Assignee: Matteo Bertozzi
>Priority: Trivial
> Fix For: 0.94.5
>
> Attachments: HBASE-7592-v0.patch, HBASE-7592-v1.patch
>
>
> in 0.94 getHTableDescriptor() looks at every htd instead of returning at the
> first found. (Trunks has already the early exit)
> {code}
> for (HTableDescriptor htd: htds) {
> if (Bytes.equals(tableName, htd.getName())) {
> hTableDescriptor = htd;
> }
> }
> return htableDescriptor
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7581) TestAccessController depends on the execution order
[ https://issues.apache.org/jira/browse/HBASE-7581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13561712#comment-13561712 ] Hudson commented on HBASE-7581: --- Integrated in HBase-0.94-security #96 (See [https://builds.apache.org/job/HBase-0.94-security/96/]) HBASE-7581. TestAccessController depends on the execution order (nkeywal) (Revision 1434098) Result = FAILURE apurtell : Files : * /hbase/branches/0.94/security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java > TestAccessController depends on the execution order > --- > > Key: HBASE-7581 > URL: https://issues.apache.org/jira/browse/HBASE-7581 > Project: HBase > Issue Type: Bug > Components: build >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal > Fix For: 0.96.0, 0.94.5 > > Attachments: 7581.v1.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7602) TestFromClientSide.testPoolBehavior is incorrect
[
https://issues.apache.org/jira/browse/HBASE-7602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13561709#comment-13561709
]
Hudson commented on HBASE-7602:
---
Integrated in HBase-0.94-security #96 (See
[https://builds.apache.org/job/HBase-0.94-security/96/])
HBASE-7602 TestFromClientSide.testPoolBehavior is incorrect (Revision
1434856)
Result = FAILURE
larsh :
Files :
*
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java
> TestFromClientSide.testPoolBehavior is incorrect
>
>
> Key: HBASE-7602
> URL: https://issues.apache.org/jira/browse/HBASE-7602
> Project: HBase
> Issue Type: Bug
>Reporter: Lars Hofhansl
>Assignee: Lars Hofhansl
> Fix For: 0.96.0, 0.94.5
>
> Attachments: 7602-0.94.txt, 7602-0.96.txt
>
>
> The writer of this test misunderstood ThreadPoolExecutor.
> The test adds Threads as tasks to a ThreadPoolExecutor and then calls join on
> the Thread objects. But these are not the running threads, it work by pure
> accident, because Thread happens to implement Runnable.
> {code}
> pool.submit(threads.get(0));
> ...
> threads.get(0).join();
> {code}
> The join will always return immediately, because the thread never ran.
> This should instead synchronize on the Future returned from submit instead,
> otherwise there is no guarantee that the threads in the pool actually
> finished.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4802) Disable show table metrics in bulk loader
[ https://issues.apache.org/jira/browse/HBASE-4802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13561713#comment-13561713 ] Hudson commented on HBASE-4802: --- Integrated in HBase-0.94-security #96 (See [https://builds.apache.org/job/HBase-0.94-security/96/]) HBASE-7644 Port HBASE-4802 'Disable show table metrics in bulk loader' to 0.94 (Revision 1437091) Result = FAILURE tedyu : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/SchemaConfigured.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/SchemaMetrics.java > Disable show table metrics in bulk loader > - > > Key: HBASE-4802 > URL: https://issues.apache.org/jira/browse/HBASE-4802 > Project: HBase > Issue Type: Bug >Reporter: Nicolas Spiegelberg >Assignee: Liyin Tang >Priority: Trivial > Fix For: 0.96.0 > > Attachments: HBASE-4802.patch > > > During bulk load, the Configuration object may be set to null. This caused > an NPE in per-CF metrics because it consults the Configuration to determine > whether to show the Table name. Need to add simple change to allow the conf > to be null & not specify table name in that instance. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7551) nodeChildrenChange event may happen after the transition to RS_ZK_REGION_SPLITTING in SplitTransaction causing the SPLIT event to be missed in the master side.
[
https://issues.apache.org/jira/browse/HBASE-7551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13561714#comment-13561714
]
Hudson commented on HBASE-7551:
---
Integrated in HBase-0.94-security #96 (See
[https://builds.apache.org/job/HBase-0.94-security/96/])
HBASE-7551 nodeChildrenChange event may happen after the transition to
RS_ZK_REGION_SPLITTING in SplitTransaction causing the SPLIT event to be missed
in the master side. (Ram, Ted, and Lars H) (Revision 1433700)
Result = FAILURE
larsh :
Files :
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
> nodeChildrenChange event may happen after the transition to
> RS_ZK_REGION_SPLITTING in SplitTransaction causing the SPLIT event to be
> missed in the master side.
> ---
>
> Key: HBASE-7551
> URL: https://issues.apache.org/jira/browse/HBASE-7551
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 0.94.4
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Critical
> Fix For: 0.96.0, 0.94.5
>
> Attachments: 7551-0.94-test.txt, 7551-0.94-v1.txt, 7551-trunk.txt,
> 7551-trunk-v2.txt, testSplitTransactionOnCluster-output.txt
>
>
> This came from HBASE-7468.
> I got the issue. I am able to reproduce this
> See the logs
> {code}
> 2013-01-14 14:37:21,760 INFO [main] regionserver.SplitTransaction(216):
> Starting split of region
> testShouldClearRITWhenNodeFoundInSplittingState,,1358154439514.a9e57d09c58b3ef3b949d602232fb2c2.
> 2013-01-14 14:37:21,760 DEBUG [main] regionserver.SplitTransaction(871):
> regionserver:61665-0x13c384e4e4f0002 Creating ephemeral node for
> a9e57d09c58b3ef3b949d602232fb2c2 in SPLITTING state
> 2013-01-14 14:37:21,844 DEBUG [main] zookeeper.ZKAssign(757):
> regionserver:61665-0x13c384e4e4f0002 Attempting to transition node
> a9e57d09c58b3ef3b949d602232fb2c2 from RS_ZK_REGION_SPLITTING to
> RS_ZK_REGION_SPLITTING
> 2013-01-14 14:37:21,849 DEBUG [Thread-873-EventThread]
> zookeeper.ZooKeeperWatcher(277): master:62334-0x13c384e4e4f001b Received
> ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected,
> path=/hbase/unassigned
> 2013-01-14 14:37:21,853 DEBUG [main] zookeeper.ZKUtil(1565):
> regionserver:61665-0x13c384e4e4f0002 Retrieved 140 byte(s) of data from znode
> /hbase/unassigned/a9e57d09c58b3ef3b949d602232fb2c2;
> data=region=testShouldClearRITWhenNodeFoundInSplittingState,,1358154439514.a9e57d09c58b3ef3b949d602232fb2c2.,
> origin=Ram.Home,61665,1358154325430, state=RS_ZK_REGION_SPLITTING
> 2013-01-14 14:37:21,918 DEBUG [main] zookeeper.ZKAssign(820):
> regionserver:61665-0x13c384e4e4f0002 Successfully transitioned node
> a9e57d09c58b3ef3b949d602232fb2c2 from RS_ZK_REGION_SPLITTING to
> RS_ZK_REGION_SPLITTING
> 2013-01-14 14:37:21,919 DEBUG [Thread-873-EventThread] zookeeper.ZKUtil(417):
> master:62334-0x13c384e4e4f001b Set watcher on existing znode
> /hbase/unassigned/a9e57d09c58b3ef3b949d602232fb2c2
> {code}
> Here we can observe that the SPLITTING node was first created. Then we
> transit it to SPLITTING to SPLITTING so that AM can have the nodeDataChange
> event. But for the nodeDataChange event to happen first nodeChildrenChange
> event should happen so that the master can set a watcher on the node.
> Now when this hang happens, we can see that after the transition happens only
> then the watcher is set by nodeChildrenChange event and so the SPLITTING to
> SPLITTING event itself is missed or skipped.
> Ideally the nodeChildrenChange event iterates thro the list of new znodes on
> the /hbase/assignment nodes. And then creates a watcher on that. One reason
> could be there are more than one znode and so the watch setting operation
> takes time. The order of execution is different when we try running from
> eclipse and when we run mvn tests.
> My conclusion is that the testcase actually reveals the problem but the same
> can happen in any case where the SPLITTING event can get missed out. May be
> some of the SPLIT related bugs that were raised is due to this? Need to
> analyse.
> Any suggestions welcome. We should ensure that the transition from SPLITTING
> to SPLITTING should happen only after the master has set the watch on the
> znode and we should be sure of that.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7648) TestAcidGuarantees.testMixedAtomicity hangs sometimes
[ https://issues.apache.org/jira/browse/HBASE-7648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13561715#comment-13561715 ] Hudson commented on HBASE-7648: --- Integrated in HBase-0.94-security #96 (See [https://builds.apache.org/job/HBase-0.94-security/96/]) HBASE-7648 TestAcidGuarantees.testMixedAtomicity hangs sometimes (Revision 1437539) Result = FAILURE jxiang : Files : * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/TestAcidGuarantees.java > TestAcidGuarantees.testMixedAtomicity hangs sometimes > - > > Key: HBASE-7648 > URL: https://issues.apache.org/jira/browse/HBASE-7648 > Project: HBase > Issue Type: Bug > Components: test >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang >Priority: Minor > Fix For: 0.96.0, 0.94.5 > > Attachments: 0.94-7648.patch, trunk-7648.patch > > > java.lang.RuntimeException: Deferred > at > org.apache.hadoop.hbase.MultithreadedTestUtil$TestContext.checkException(MultithreadedTestUtil.java:76) > at > org.apache.hadoop.hbase.MultithreadedTestUtil$TestContext.waitFor(MultithreadedTestUtil.java:69) > at > org.apache.hadoop.hbase.TestAcidGuarantees.runTestAtomicity(TestAcidGuarantees.java:301) > at > org.apache.hadoop.hbase.TestAcidGuarantees.runTestAtomicity(TestAcidGuarantees.java:244) > at > org.apache.hadoop.hbase.TestAcidGuarantees.testMixedAtomicity(TestAcidGuarantees.java:343) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45) > at > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) > at > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42) > at > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20) > at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:47) > at org.junit.rules.RunRules.evaluate(RunRules.java:18) > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47) > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231) > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60) > at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229) > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50) > at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222) > at org.junit.runners.ParentRunner.run(ParentRunner.java:300) > at org.junit.runners.Suite.runChild(Suite.java:128) > at org.junit.runners.Suite.runChild(Suite.java:24) > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) > at java.util.concurrent.FutureTask.run(FutureTask.java:138) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > at java.lang.Thread.run(Thread.java:662) > Caused by: > org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hbase.NotServingRegionException): > org.apache.hadoop.hbase.NotServingRegionException: Region is not online: > TestAcidGuarantees,,135776964.317288e8ca738963ca5e273fc56750fd. > at > org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:3211) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.flushRegion(HRegionServer.java:2963) > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at > org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364) > at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1400) > at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:1021) > at > org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:150) > at $Proxy23.flushRegion(Unknown Source) > at org.apache.hadoop.hbase.client.HBaseAdmin.flush(HBaseAdmin.java:1248) > at org.apache.hadoop.hbase.client.HBaseAdmin.flush(HBaseAdmin.java:1230) > at > org.apache.hadoop.hbase.TestAcidGuarantees$1.doAnAction(TestAcidGuarantees.java:272) > at > org.apache.hadoop.hbase.Multi
[jira] [Commented] (HBASE-7646) Make forkedProcessTimeoutInSeconds configurable
[ https://issues.apache.org/jira/browse/HBASE-7646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13561716#comment-13561716 ] Hudson commented on HBASE-7646: --- Integrated in HBase-0.94-security #96 (See [https://builds.apache.org/job/HBase-0.94-security/96/]) HBASE-7646 Make forkedProcessTimeoutInSeconds configurable (Revision 1437132) Result = FAILURE jxiang : Files : * /hbase/branches/0.94/pom.xml > Make forkedProcessTimeoutInSeconds configurable > --- > > Key: HBASE-7646 > URL: https://issues.apache.org/jira/browse/HBASE-7646 > Project: HBase > Issue Type: Bug > Components: build >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang >Priority: Trivial > Fix For: 0.96.0, 0.94.5 > > Attachments: 0.94-7646.patch, trunk-7646.patch > > > Command line property "surefire.timeout" somehow doesn't work. It may be > because forkedProcessTimeoutInSeconds is hard-coded to 900. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7507) Make memstore flush be able to retry after exception
[
https://issues.apache.org/jira/browse/HBASE-7507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13561717#comment-13561717
]
Hudson commented on HBASE-7507:
---
Integrated in HBase-0.94-security #96 (See
[https://builds.apache.org/job/HBase-0.94-security/96/])
HBASE-7507 Make memstore flush be able to retry after exception (Chunhui)
(Revision 1436110)
Result = FAILURE
zjushch :
Files :
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/HConstants.java
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java
> Make memstore flush be able to retry after exception
>
>
> Key: HBASE-7507
> URL: https://issues.apache.org/jira/browse/HBASE-7507
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.94.3
>Reporter: chunhui shen
>Assignee: chunhui shen
> Fix For: 0.96.0, 0.94.5
>
> Attachments: 7507-94.patch, 7507-trunk v1.patch, 7507-trunk v2.patch,
> 7507-trunkv3.patch
>
>
> We will abort regionserver if memstore flush throws exception.
> I thinks we could do retry to make regionserver more stable because file
> system may be not ok in a transient time. e.g. Switching namenode in the
> NamenodeHA environment
> {code}
> HRegion#internalFlushcache(){
> ...
> try {
> ...
> }catch(Throwable t){
> DroppedSnapshotException dse = new DroppedSnapshotException("region: " +
> Bytes.toStringBinary(getRegionName()));
> dse.initCause(t);
> throw dse;
> }
> ...
> }
> MemStoreFlusher#flushRegion(){
> ...
> region.flushcache();
> ...
> try {
> }catch(DroppedSnapshotException ex){
> server.abort("Replay of HLog required. Forcing server shutdown", ex);
> }
> ...
> }
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7617) TestHRegionOnCluster.testDataCorrectnessReplayingRecoveredEdits still fails occasionally.
[
https://issues.apache.org/jira/browse/HBASE-7617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13561718#comment-13561718
]
Hudson commented on HBASE-7617:
---
Integrated in HBase-0.94-security #96 (See
[https://builds.apache.org/job/HBase-0.94-security/96/])
HBASE-7617 TestHRegionOnCluster.testDataCorrectnessReplayingRecoveredEdits
still fails occasionally. (Revision 1434937)
Result = FAILURE
larsh :
Files :
*
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionOnCluster.java
> TestHRegionOnCluster.testDataCorrectnessReplayingRecoveredEdits still fails
> occasionally.
> -
>
> Key: HBASE-7617
> URL: https://issues.apache.org/jira/browse/HBASE-7617
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.94.4
>Reporter: Lars Hofhansl
>Assignee: Lars Hofhansl
> Fix For: 0.94.5
>
> Attachments: 7617.txt
>
>
> {code}
> java.lang.reflect.UndeclaredThrowableException
> at $Proxy20.move(Unknown Source)
> at org.apache.hadoop.hbase.client.HBaseAdmin.move(HBaseAdmin.java:1426)
> at
> org.apache.hadoop.hbase.regionserver.TestHRegionOnCluster.testDataCorrectnessReplayingRecoveredEdits(TestHRegionOnCluster.java:84)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at
> org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:62)
> Caused by: org.apache.hadoop.ipc.RemoteException:
> org.apache.hadoop.hbase.UnknownRegionException:
> 3775e6eba7718e4e4571942ae73b5851
> at org.apache.hadoop.hbase.master.HMaster.move(HMaster.java:1159)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1426)
> at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:1021)
> at
> org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:150)
> ... 12 more
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7644) Port HBASE-4802 'Disable show table metrics in bulk loader' to 0.94
[ https://issues.apache.org/jira/browse/HBASE-7644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13561719#comment-13561719 ] Hudson commented on HBASE-7644: --- Integrated in HBase-0.94-security #96 (See [https://builds.apache.org/job/HBase-0.94-security/96/]) HBASE-7644 Port HBASE-4802 'Disable show table metrics in bulk loader' to 0.94 (Revision 1437091) Result = FAILURE tedyu : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/SchemaConfigured.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/SchemaMetrics.java > Port HBASE-4802 'Disable show table metrics in bulk loader' to 0.94 > --- > > Key: HBASE-7644 > URL: https://issues.apache.org/jira/browse/HBASE-7644 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Ted Yu > Fix For: 0.94.5 > > Attachments: 7644-disable-show-table.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7562) ZKUtil: missing "else condition" in multi processing
[ https://issues.apache.org/jira/browse/HBASE-7562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13561720#comment-13561720 ] Hudson commented on HBASE-7562: --- Integrated in HBase-0.94-security #96 (See [https://builds.apache.org/job/HBase-0.94-security/96/]) HBASE-7562 ZKUtil: missing "else condition" in multi processing (Himanshu) (Revision 1433593) Result = FAILURE tedyu : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java > ZKUtil: missing "else condition" in multi processing > > > Key: HBASE-7562 > URL: https://issues.apache.org/jira/browse/HBASE-7562 > Project: HBase > Issue Type: Bug >Reporter: Himanshu Vashishtha >Assignee: Himanshu Vashishtha >Priority: Minor > Fix For: 0.94.5 > > Attachments: HBASE-7562.patch, HBASE-7562-v2.patch > > > The method multiOrSequential misses an else condition and process the list of > Ops both in multi and sequentially. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6066) some low hanging read path improvement ideas
[
https://issues.apache.org/jira/browse/HBASE-6066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13561721#comment-13561721
]
Hudson commented on HBASE-6066:
---
Integrated in HBase-0.94-security #96 (See
[https://builds.apache.org/job/HBase-0.94-security/96/])
HBASE-7599 Port HBASE-6066 (low hanging read path improvements) to 0.94
(Devaraj Das) (Revision 1437237)
Result = FAILURE
larsh :
Files :
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
> some low hanging read path improvement ideas
> -
>
> Key: HBASE-6066
> URL: https://issues.apache.org/jira/browse/HBASE-6066
> Project: HBase
> Issue Type: Sub-task
> Components: Performance
>Reporter: Kannan Muthukkaruppan
>Assignee: Devaraj Das
>Priority: Critical
> Labels: noob
> Fix For: 0.96.0
>
> Attachments:
> 0001-jira-HBASE-6066-89-fb-Some-read-performance-improvem.patch,
> 6066-rebased-1.patch, 6066-rebased-1.patch, metric-stringbuilder-fix.patch
>
>
> I was running some single threaded scan performance tests for a table with
> small sized rows that is fully cached. Some observations...
> We seem to be doing several wasteful iterations over and/or building of
> temporary lists.
> 1) One such is the following code in HRegionServer.next():
> {code}
>boolean moreRows = s.next(values, HRegion.METRIC_NEXTSIZE);
>if (!values.isEmpty()) {
> for (KeyValue kv : values) { --> wasteful in most
> cases
>currentScanResultSize += kv.heapSize();
>}
>results.add(new Result(values));
> {code}
> By default the "maxScannerResultSize" is Long.MAX_VALUE. In those cases,
> we can avoid the unnecessary iteration to compute currentScanResultSize.
> 2) An example of a wasteful temporary array, is "results" in
> RegionScanner.next().
> {code}
> results.clear();
> boolean returnResult = nextInternal(limit, metric);
> outResults.addAll(results);
> {code}
> results then gets copied over to outResults via an addAll(). Not sure why we
> can not directly collect the results in outResults.
> 3) Another almost similar exmaple of a wasteful array is "results" in
> StoreScanner.next(), which eventually also copies its results into
> "outResults".
> 4) Reduce overhead of "size metric" maintained in StoreScanner.next().
> {code}
> if (metric != null) {
> HRegion.incrNumericMetric(this.metricNamePrefix + metric,
>copyKv.getLength());
> }
> results.add(copyKv);
> {code}
> A single call to next() might fetch a lot of KVs. We can first add up the
> size of those KVs in a local variable and then in a finally clause increment
> the metric one shot, rather than updating AtomicLongs for each KV.
> 5) RegionScanner.next() calls a helper RegionScanner.next() on the same
> object. Both are synchronized methods. Synchronized methods calling nested
> synchronized methods on the same object are probably adding some small
> overhead. The inner next() calls isFilterDone() which is a also a
> synchronized method. We should factor the code to avoid these nested
> synchronized methods.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7578) TestCatalogTracker hangs occasionally
[ https://issues.apache.org/jira/browse/HBASE-7578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13561722#comment-13561722 ] Hudson commented on HBASE-7578: --- Integrated in HBase-0.94-security #96 (See [https://builds.apache.org/job/HBase-0.94-security/96/]) HBASE-7578 TestCatalogTracker hangs occasionally (Revision 1434437) Result = FAILURE larsh : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/catalog/CatalogTracker.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/HMaster.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditorNoCluster.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java > TestCatalogTracker hangs occasionally > - > > Key: HBASE-7578 > URL: https://issues.apache.org/jira/browse/HBASE-7578 > Project: HBase > Issue Type: Bug >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl > Fix For: 0.96.0, 0.94.5 > > Attachments: 7578-0.94-v1.txt, 7578-bandaid.txt, 7578-jstack.txt, > 7578-trunk-v1.txt > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7549) Make HTableInterface#batch() javadoc proper
[ https://issues.apache.org/jira/browse/HBASE-7549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13561723#comment-13561723 ] Hudson commented on HBASE-7549: --- Integrated in HBase-0.94-security #96 (See [https://builds.apache.org/job/HBase-0.94-security/96/]) HBASE-7549 Make HTableInterface#batch() javadoc proper (Anoop) (Revision 1433806) Result = FAILURE tedyu : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/client/HTableInterface.java > Make HTableInterface#batch() javadoc proper > --- > > Key: HBASE-7549 > URL: https://issues.apache.org/jira/browse/HBASE-7549 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 0.94.0, 0.96.0 >Reporter: Anoop Sam John >Assignee: Anoop Sam John >Priority: Trivial > Fix For: 0.96.0, 0.94.5 > > Attachments: HBASE-7549_94.patch, HBASE-7549_trunk.patch, > HBASE-7549_trunk_v2.patch > > > We can pass Put, Get, Delete, Append, Increment in the batch. But the > javadoc do cover Put, Delete and Get only. Better to correct this. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7575) FSUtils#getTableStoreFilePathMap should all ignore non-table folders
[ https://issues.apache.org/jira/browse/HBASE-7575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13561724#comment-13561724 ] Hudson commented on HBASE-7575: --- Integrated in HBase-0.94-security #96 (See [https://builds.apache.org/job/HBase-0.94-security/96/]) HBASE-7575 FSUtils#getTableStoreFilePathMap should all ignore non-table folders (Revision 1434021) Result = FAILURE jxiang : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/HConstants.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/FSUtils.java > FSUtils#getTableStoreFilePathMap should all ignore non-table folders > > > Key: HBASE-7575 > URL: https://issues.apache.org/jira/browse/HBASE-7575 > Project: HBase > Issue Type: Bug >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang >Priority: Minor > Fix For: 0.96.0, 0.94.5 > > Attachments: trunk_7575.patch, trunk_7575_v2.patch > > > Currently it just ignores .log. It should ignore all non-table folders. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7034) Bad version, failed OPENING to OPENED but master thinks it is open anyways
[
https://issues.apache.org/jira/browse/HBASE-7034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13561725#comment-13561725
]
Hudson commented on HBASE-7034:
---
Integrated in HBase-0.94-security #96 (See
[https://builds.apache.org/job/HBase-0.94-security/96/])
HBASE-7034 Bad version, failed OPENING to OPENED but master thinks it is
open anyways (Anoop) (Revision 1434797)
Result = FAILURE
tedyu :
Files :
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/zookeeper/RecoverableZooKeeper.java
*
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/zookeeper/TestRecoverableZooKeeper.java
> Bad version, failed OPENING to OPENED but master thinks it is open anyways
> --
>
> Key: HBASE-7034
> URL: https://issues.apache.org/jira/browse/HBASE-7034
> Project: HBase
> Issue Type: Bug
> Components: Region Assignment
>Affects Versions: 0.94.2
>Reporter: stack
>Assignee: Anoop Sam John
> Fix For: 0.96.0, 0.94.5
>
> Attachments: HBASE-7034_94.patch, HBASE-7034_94_V2.patch,
> HBASE-7034_Test_Trunk.patch, TestRecoverableZooKeeper.java
>
>
> I have this in RS log:
> {code}
> 2012-10-22 02:21:50,698 ERROR
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler: Failed
> transitioning node
> b9,\xEE\xAE\x9BiQO\x89]+a\xE0\x7F\xB7'X?,1349052737638.9af7cfc9b15910a0b3d714bf40a3248f.
> from OPENING to OPENED -- closing region
> org.apache.zookeeper.KeeperException$BadVersionException: KeeperErrorCode =
> BadVersion for /hbase/unassigned/9af7cfc9b15910a0b3d714bf40a3248f
> {code}
> Master says this (it is bulk assigning):
> {code}
>
> 2012-10-22 02:21:40,673 DEBUG org.apache.hadoop.hbase.zookeeper.ZKUtil:
> master:10302-0xb3a862e57a503ba Set watcher on existing znode
> /hbase/unassigned/9af7cfc9b15910a0b3d714bf40a3248f
> ...
> then this
>
> 2012-10-22 02:23:47,089 DEBUG org.apache.hadoop.hbase.zookeeper.ZKUtil:
> master:10302-0xb3a862e57a503ba Set watcher on existing znode
> /hbase/unassigned/9af7cfc9b15910a0b3d714bf40a3248f
>
> 2012-10-22 02:24:34,176 DEBUG org.apache.hadoop.hbase.zookeeper.ZKUtil:
> master:10302-0xb3a862e57a503ba Retrieved 112 byte(s) of data from znode
> /hbase/unassigned/9af7cfc9b15910a0b3d714bf40a3248f and set watcher;
> region=b9,\xEE\xAE\x9BiQO\x89]+a\xE0\x7F\xB7'X?,1349052737638.9af7cfc9b15910a0b3d714bf40a3248f.,
> origin=sv4r17s44,10304,1350872216778, state=RS_ZK_REGION_OPENED
> etc.
> {code}
> Disagreement as to what is going on here.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7599) Port HBASE-6066 (low hanging read path improvements) to 0.94
[ https://issues.apache.org/jira/browse/HBASE-7599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13561726#comment-13561726 ] Hudson commented on HBASE-7599: --- Integrated in HBase-0.94-security #96 (See [https://builds.apache.org/job/HBase-0.94-security/96/]) HBASE-7599 Port HBASE-6066 (low hanging read path improvements) to 0.94 (Devaraj Das) (Revision 1437237) Result = FAILURE larsh : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java > Port HBASE-6066 (low hanging read path improvements) to 0.94 > > > Key: HBASE-7599 > URL: https://issues.apache.org/jira/browse/HBASE-7599 > Project: HBase > Issue Type: Improvement >Reporter: Devaraj Das >Assignee: Devaraj Das > Fix For: 0.94.5 > > Attachments: 7599-0.94.patch, 7599-0.94.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7587) Fix two findbugs warning in RowResource
[ https://issues.apache.org/jira/browse/HBASE-7587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13561727#comment-13561727 ] Hudson commented on HBASE-7587: --- Integrated in HBase-0.94-security #96 (See [https://builds.apache.org/job/HBase-0.94-security/96/]) HBASE-7587. Fix two findbugs warnings in RowResource (Jean-Marc Spaggiari) (Revision 1434378) Result = FAILURE apurtell : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/rest/RowResource.java > Fix two findbugs warning in RowResource > --- > > Key: HBASE-7587 > URL: https://issues.apache.org/jira/browse/HBASE-7587 > Project: HBase > Issue Type: Bug >Reporter: Jean-Marc Spaggiari >Assignee: Jean-Marc Spaggiari >Priority: Minor > Fix For: 0.96.0, 0.94.5 > > Attachments: HBASE-7587-v0-trunk.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7503) Add exists(List) in HTableInterface to allow multiple parallel exists at one time
[ https://issues.apache.org/jira/browse/HBASE-7503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean-Marc Spaggiari updated HBASE-7503: --- Status: Open (was: Patch Available) > Add exists(List) in HTableInterface to allow multiple parallel exists at one > time > - > > Key: HBASE-7503 > URL: https://issues.apache.org/jira/browse/HBASE-7503 > Project: HBase > Issue Type: Improvement >Reporter: Jean-Marc Spaggiari >Assignee: Jean-Marc Spaggiari >Priority: Minor > Fix For: 0.96.0 > > Attachments: HBASE-7503-v0-trunk.patch, HBASE-7503-v10-trunk.patch, > HBASE-7503-v11-trunk.patch, HBASE-7503-v12-trunk.patch, > HBASE-7503-v13-trunk.patch, HBASE-7503-v13-trunk.patch, > HBASE-7503-v1-trunk.patch, HBASE-7503-v2-trunk.patch, > HBASE-7503-v2-trunk.patch, HBASE-7503-v3-trunk.patch, > HBASE-7503-v4-trunk.patch, HBASE-7503-v5-trunk.patch, > HBASE-7503-v7-trunk.patch, HBASE-7503-v8-trunk.patch, > HBASE-7503-v9-trunk.patch > > Original Estimate: 5m > Remaining Estimate: 5m > > We need to have a Boolean[] exists(List gets) throws IOException method > implemented in HTableInterface. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7503) Add exists(List) in HTableInterface to allow multiple parallel exists at one time
[ https://issues.apache.org/jira/browse/HBASE-7503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean-Marc Spaggiari updated HBASE-7503: --- Status: Patch Available (was: Open) Patch re-attached. > Add exists(List) in HTableInterface to allow multiple parallel exists at one > time > - > > Key: HBASE-7503 > URL: https://issues.apache.org/jira/browse/HBASE-7503 > Project: HBase > Issue Type: Improvement >Reporter: Jean-Marc Spaggiari >Assignee: Jean-Marc Spaggiari >Priority: Minor > Fix For: 0.96.0 > > Attachments: HBASE-7503-v0-trunk.patch, HBASE-7503-v10-trunk.patch, > HBASE-7503-v11-trunk.patch, HBASE-7503-v12-trunk.patch, > HBASE-7503-v13-trunk.patch, HBASE-7503-v13-trunk.patch, > HBASE-7503-v1-trunk.patch, HBASE-7503-v2-trunk.patch, > HBASE-7503-v2-trunk.patch, HBASE-7503-v3-trunk.patch, > HBASE-7503-v4-trunk.patch, HBASE-7503-v5-trunk.patch, > HBASE-7503-v7-trunk.patch, HBASE-7503-v8-trunk.patch, > HBASE-7503-v9-trunk.patch > > Original Estimate: 5m > Remaining Estimate: 5m > > We need to have a Boolean[] exists(List gets) throws IOException method > implemented in HTableInterface. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7503) Add exists(List) in HTableInterface to allow multiple parallel exists at one time
[ https://issues.apache.org/jira/browse/HBASE-7503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean-Marc Spaggiari updated HBASE-7503: --- Attachment: HBASE-7503-v13-trunk.patch > Add exists(List) in HTableInterface to allow multiple parallel exists at one > time > - > > Key: HBASE-7503 > URL: https://issues.apache.org/jira/browse/HBASE-7503 > Project: HBase > Issue Type: Improvement >Reporter: Jean-Marc Spaggiari >Assignee: Jean-Marc Spaggiari >Priority: Minor > Fix For: 0.96.0 > > Attachments: HBASE-7503-v0-trunk.patch, HBASE-7503-v10-trunk.patch, > HBASE-7503-v11-trunk.patch, HBASE-7503-v12-trunk.patch, > HBASE-7503-v13-trunk.patch, HBASE-7503-v13-trunk.patch, > HBASE-7503-v1-trunk.patch, HBASE-7503-v2-trunk.patch, > HBASE-7503-v2-trunk.patch, HBASE-7503-v3-trunk.patch, > HBASE-7503-v4-trunk.patch, HBASE-7503-v5-trunk.patch, > HBASE-7503-v7-trunk.patch, HBASE-7503-v8-trunk.patch, > HBASE-7503-v9-trunk.patch > > Original Estimate: 5m > Remaining Estimate: 5m > > We need to have a Boolean[] exists(List gets) throws IOException method > implemented in HTableInterface. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5930) Periodically flush the Memstore?
[ https://issues.apache.org/jira/browse/HBASE-5930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13561739#comment-13561739 ] Lars Hofhansl commented on HBASE-5930: -- How can deferred log flush ever be as fast as not writing the WAL at all? Considering only the latency of a single request that might be true in many cases, but it will definitely not be true on a busy cluster since all data is written to the disks twice. > Periodically flush the Memstore? > > > Key: HBASE-5930 > URL: https://issues.apache.org/jira/browse/HBASE-5930 > Project: HBase > Issue Type: Improvement >Reporter: Lars Hofhansl >Assignee: Devaraj Das >Priority: Minor > Fix For: 0.96.0 > > Attachments: 5930-1.patch, 5930-wip.patch > > > A colleague of mine ran into an interesting issue. > He inserted some data with the WAL disabled, which happened to fit in the > aggregate Memstores memory. > Two weeks later he a had problem with the HDFS cluster, which caused the > region servers to abort. He found that his data was lost. Looking at the log > we found that the Memstores were not flushed at all during these two weeks. > Should we have an option to flush memstores periodically. There are obvious > downsides to this, like many small storefiles, etc. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7638) [0.94] region cache entry should only be removed on error if the error is from the server currently in cache
[ https://issues.apache.org/jira/browse/HBASE-7638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13561742#comment-13561742 ] Lars Hofhansl commented on HBASE-7638: -- In both cases TestReplication and TestReplicationWithCompression did not finish. > [0.94] region cache entry should only be removed on error if the error is > from the server currently in cache > > > Key: HBASE-7638 > URL: https://issues.apache.org/jira/browse/HBASE-7638 > Project: HBase > Issue Type: Bug >Affects Versions: 0.94.4 >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Minor > Fix For: 0.94.5 > > Attachments: HBASE-7638-v0.patch > > > See HBASE-7268. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7652) Flag in the mutations to indicate asynchronous write to WAL
[ https://issues.apache.org/jira/browse/HBASE-7652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13561744#comment-13561744 ] Lars Hofhansl commented on HBASE-7652: -- I think that is a dup of HBASE-5954 > Flag in the mutations to indicate asynchronous write to WAL > --- > > Key: HBASE-7652 > URL: https://issues.apache.org/jira/browse/HBASE-7652 > Project: HBase > Issue Type: Improvement >Reporter: Devaraj Das > > Today the mutation APIs accepts a flag that indicates whether to skip the WAL > or not. As discussed in HBASE-5930, it'd be good to have an option to do > asynchronous write to WAL if the user so desired. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7638) [0.94] region cache entry should only be removed on error if the error is from the server currently in cache
[ https://issues.apache.org/jira/browse/HBASE-7638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13561746#comment-13561746 ] Lars Hofhansl commented on HBASE-7638: -- And both of them failed in Ubuntu2, which was known to have issues. (I added them back to the available machines just recently to see if that is still case) > [0.94] region cache entry should only be removed on error if the error is > from the server currently in cache > > > Key: HBASE-7638 > URL: https://issues.apache.org/jira/browse/HBASE-7638 > Project: HBase > Issue Type: Bug >Affects Versions: 0.94.4 >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Minor > Fix For: 0.94.5 > > Attachments: HBASE-7638-v0.patch > > > See HBASE-7268. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7628) Port HBASE-6509 fast-forwarding FuzzyRowFilter to 0.94
[ https://issues.apache.org/jira/browse/HBASE-7628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-7628: -- Attachment: HBASE-7628.patch > Port HBASE-6509 fast-forwarding FuzzyRowFilter to 0.94 > -- > > Key: HBASE-7628 > URL: https://issues.apache.org/jira/browse/HBASE-7628 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu > Fix For: 0.94.5 > > Attachments: HBASE-7628.patch > > > This is to port HBASE-6509: 'Implement fast-forwarding FuzzyRowFilter to > allow filtering rows e.g. by "???alex?b"' to 0.94 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7628) Port HBASE-6509 fast-forwarding FuzzyRowFilter to 0.94
[ https://issues.apache.org/jira/browse/HBASE-7628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13561753#comment-13561753 ] Anoop Sam John commented on HBASE-7628: --- I was trying to back port for our version > Port HBASE-6509 fast-forwarding FuzzyRowFilter to 0.94 > -- > > Key: HBASE-7628 > URL: https://issues.apache.org/jira/browse/HBASE-7628 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu > Fix For: 0.94.5 > > Attachments: HBASE-7628.patch > > > This is to port HBASE-6509: 'Implement fast-forwarding FuzzyRowFilter to > allow filtering rows e.g. by "???alex?b"' to 0.94 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7628) Port HBASE-6509 fast-forwarding FuzzyRowFilter to 0.94
[ https://issues.apache.org/jira/browse/HBASE-7628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13561771#comment-13561771 ] Ted Yu commented on HBASE-7628: --- The following tests passed locally: 1058 mt -Dtest=TestFuzzyRowFilter 1060 mt -Dtest=TestHbaseObjectWritable > Port HBASE-6509 fast-forwarding FuzzyRowFilter to 0.94 > -- > > Key: HBASE-7628 > URL: https://issues.apache.org/jira/browse/HBASE-7628 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu > Fix For: 0.94.5 > > Attachments: HBASE-7628.patch > > > This is to port HBASE-6509: 'Implement fast-forwarding FuzzyRowFilter to > allow filtering rows e.g. by "???alex?b"' to 0.94 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7268) correct local region location cache information can be overwritten (or deleted) w/stale information from an old server
[ https://issues.apache.org/jira/browse/HBASE-7268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-7268: -- Resolution: Fixed Status: Resolved (was: Patch Available) > correct local region location cache information can be overwritten (or > deleted) w/stale information from an old server > -- > > Key: HBASE-7268 > URL: https://issues.apache.org/jira/browse/HBASE-7268 > Project: HBase > Issue Type: Bug >Affects Versions: 0.96.0 >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Minor > Fix For: 0.96.0 > > Attachments: 7268-addendum-v0.patch, 7268-v6.patch, 7268-v8.patch, > HBASE-7268-addendum-v0.patch, HBASE-7268-v0.patch, HBASE-7268-v0.patch, > HBASE-7268-v1.patch, HBASE-7268-v2.patch, HBASE-7268-v2-plus-masterTs.patch, > HBASE-7268-v2-plus-masterTs.patch, HBASE-7268-v3.patch, HBASE-7268-v4.patch, > HBASE-7268-v5.patch, HBASE-7268-v6.patch, HBASE-7268-v7.patch, > HBASE-7268-v8.patch, HBASE-7268-v9.patch > > > Discovered via HBASE-7250; related to HBASE-5877. > Test is writing from multiple threads. > Server A has region R; client knows that. > R gets moved from A to server B. > B gets killed. > R gets moved by master to server C. > ~15 seconds later, client tries to write to it (on A?). > Multiple client threads report from RegionMoved exception processing logic "R > moved from C to B", even though such transition never happened (neither in > nor before the sequence described below). Not quite sure how the client > learned of the transition to C, I assume it's from meta from some other > thread... > Then, put fails (it may fail due to accumulated errors that are not logged, > which I am investigating... but the bogus cache update is there > nonwithstanding). > I have a patch but not sure if it works, test still fails locally for yet > unknown reason. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7503) Add exists(List) in HTableInterface to allow multiple parallel exists at one time
[
https://issues.apache.org/jira/browse/HBASE-7503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13561777#comment-13561777
]
Hadoop QA commented on HBASE-7503:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12566322/HBASE-7503-v13-trunk.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 6 new
or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to introduce 2 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces lines longer than
100
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/4161//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4161//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4161//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4161//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4161//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4161//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4161//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/4161//console
This message is automatically generated.
> Add exists(List) in HTableInterface to allow multiple parallel exists at one
> time
> -
>
> Key: HBASE-7503
> URL: https://issues.apache.org/jira/browse/HBASE-7503
> Project: HBase
> Issue Type: Improvement
>Reporter: Jean-Marc Spaggiari
>Assignee: Jean-Marc Spaggiari
>Priority: Minor
> Fix For: 0.96.0
>
> Attachments: HBASE-7503-v0-trunk.patch, HBASE-7503-v10-trunk.patch,
> HBASE-7503-v11-trunk.patch, HBASE-7503-v12-trunk.patch,
> HBASE-7503-v13-trunk.patch, HBASE-7503-v13-trunk.patch,
> HBASE-7503-v1-trunk.patch, HBASE-7503-v2-trunk.patch,
> HBASE-7503-v2-trunk.patch, HBASE-7503-v3-trunk.patch,
> HBASE-7503-v4-trunk.patch, HBASE-7503-v5-trunk.patch,
> HBASE-7503-v7-trunk.patch, HBASE-7503-v8-trunk.patch,
> HBASE-7503-v9-trunk.patch
>
> Original Estimate: 5m
> Remaining Estimate: 5m
>
> We need to have a Boolean[] exists(List gets) throws IOException method
> implemented in HTableInterface.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7571) add the notion of per-table or per-column family configuration
[ https://issues.apache.org/jira/browse/HBASE-7571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-7571: -- Attachment: 7571-v3.patch > add the notion of per-table or per-column family configuration > -- > > Key: HBASE-7571 > URL: https://issues.apache.org/jira/browse/HBASE-7571 > Project: HBase > Issue Type: Sub-task >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: 7571-v3.patch, HBASE-7571-v0-based-on-HBASE-7563.patch, > HBASE-7571-v0-including-HBASE-7563.patch, HBASE-7571-v1.patch, > HBASE-7571-v2.patch, HBASE-7571-v3.patch > > > Main part of split HBASE-7236. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7382) Port ZK.multi support from HBASE-6775 to 0.96
[ https://issues.apache.org/jira/browse/HBASE-7382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-7382: -- Attachment: 7382-trunk-v2.txt Patch v2 fixes the 3 findbugs warnings about HE_EQUALS_USE_HASHCODE > Port ZK.multi support from HBASE-6775 to 0.96 > - > > Key: HBASE-7382 > URL: https://issues.apache.org/jira/browse/HBASE-7382 > Project: HBase > Issue Type: Bug > Components: Zookeeper >Reporter: Gregory Chanan >Assignee: Himanshu Vashishtha >Priority: Critical > Fix For: 0.96.0 > > Attachments: 7382-trunk-v2.txt, HBASE-7382-trunk.patch > > > HBASE-6775 adds support for ZK.multi ZKUtil and uses it for the 0.92/0.94 > compatibility fix implemented in HBASE-6710. > ZK.multi support is most likely useful in 0.96, but since HBASE-6710 is not > relevant for 0.96, perhaps we should find another use case first before we > port. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7628) Port HBASE-6509 fast-forwarding FuzzyRowFilter to 0.94
[ https://issues.apache.org/jira/browse/HBASE-7628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13561805#comment-13561805 ] Lars Hofhansl commented on HBASE-7628: -- +1 ([~jmhsieh] your point is duly noted, we should make a followup jira - or I think there might be one already - to support filters better) > Port HBASE-6509 fast-forwarding FuzzyRowFilter to 0.94 > -- > > Key: HBASE-7628 > URL: https://issues.apache.org/jira/browse/HBASE-7628 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu > Fix For: 0.94.5 > > Attachments: HBASE-7628.patch > > > This is to port HBASE-6509: 'Implement fast-forwarding FuzzyRowFilter to > allow filtering rows e.g. by "???alex?b"' to 0.94 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5930) Periodically flush the Memstore?
[
https://issues.apache.org/jira/browse/HBASE-5930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13561810#comment-13561810
]
Ted Yu commented on HBASE-5930:
---
Where is PeriodicMemstoreFlusher instantiated ?
Currently MEMSTORE_PERIODIC_FLUSH_INTERVAL is read by both HRegion and
PeriodicMemstoreFlusher.
{code}
+ boolean shouldFlush() {
{code}
Can we pass the interval to the above method so that HRegion doesn't need to
introduce:
{code}
+ private long flushCheckInterval;
{code}
What value for MEMSTORE_PERIODIC_FLUSH_INTERVAL would be interpreted as
disabling the periodic flush ?
Thanks
> Periodically flush the Memstore?
>
>
> Key: HBASE-5930
> URL: https://issues.apache.org/jira/browse/HBASE-5930
> Project: HBase
> Issue Type: Improvement
>Reporter: Lars Hofhansl
>Assignee: Devaraj Das
>Priority: Minor
> Fix For: 0.96.0
>
> Attachments: 5930-1.patch, 5930-wip.patch
>
>
> A colleague of mine ran into an interesting issue.
> He inserted some data with the WAL disabled, which happened to fit in the
> aggregate Memstores memory.
> Two weeks later he a had problem with the HDFS cluster, which caused the
> region servers to abort. He found that his data was lost. Looking at the log
> we found that the Memstores were not flushed at all during these two weeks.
> Should we have an option to flush memstores periodically. There are obvious
> downsides to this, like many small storefiles, etc.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-7652) Flag in the mutations to indicate asynchronous write to WAL
[ https://issues.apache.org/jira/browse/HBASE-7652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj Das resolved HBASE-7652. Resolution: Duplicate > Flag in the mutations to indicate asynchronous write to WAL > --- > > Key: HBASE-7652 > URL: https://issues.apache.org/jira/browse/HBASE-7652 > Project: HBase > Issue Type: Improvement >Reporter: Devaraj Das > > Today the mutation APIs accepts a flag that indicates whether to skip the WAL > or not. As discussed in HBASE-5930, it'd be good to have an option to do > asynchronous write to WAL if the user so desired. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7638) [0.94] region cache entry should only be removed on error if the error is from the server currently in cache
[ https://issues.apache.org/jira/browse/HBASE-7638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13561812#comment-13561812 ] Lars Hofhansl commented on HBASE-7638: -- [~sershe] I might be a bit slow here... Could you explain where this is a problem? We're removing regions from the cache by uniquely identified by their startKey, so if any server reports any problem with a region there is some problem with the region and it seems fine to remove it from the cache in that case. Is this a condition that happens when a region was moved? Or split? > [0.94] region cache entry should only be removed on error if the error is > from the server currently in cache > > > Key: HBASE-7638 > URL: https://issues.apache.org/jira/browse/HBASE-7638 > Project: HBase > Issue Type: Bug >Affects Versions: 0.94.4 >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Minor > Fix For: 0.94.5 > > Attachments: HBASE-7638-v0.patch > > > See HBASE-7268. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7628) Port HBASE-6509 fast-forwarding FuzzyRowFilter to 0.94
[ https://issues.apache.org/jira/browse/HBASE-7628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-7628: -- Assignee: Anoop Sam John Integrated to 0.94 branch Thanks for the patch, Anoop. Thanks for the review, Lars. > Port HBASE-6509 fast-forwarding FuzzyRowFilter to 0.94 > -- > > Key: HBASE-7628 > URL: https://issues.apache.org/jira/browse/HBASE-7628 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Anoop Sam John > Fix For: 0.94.5 > > Attachments: HBASE-7628.patch > > > This is to port HBASE-6509: 'Implement fast-forwarding FuzzyRowFilter to > allow filtering rows e.g. by "???alex?b"' to 0.94 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7638) [0.94] region cache entry should only be removed on error if the error is from the server currently in cache
[ https://issues.apache.org/jira/browse/HBASE-7638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13561823#comment-13561823 ] Sergey Shelukhin commented on HBASE-7638: - The problems arises in multithreaded case (actually I only observed it in trunk, but I assumed this should also apply to 94). One thread gets A from cache and goes to A. Thread 2 does the same. Thread 1 errors out and updates the location from META to new correct location. Thread 2 errors out and removes it from cache. So it's not critical, just adds extra meta trips. In integration test where there are many threads and many errors it was relatively more prominent. I think it's made possible in no small part by using delayed callable, where we first build the request, including the destination server, then wait. > [0.94] region cache entry should only be removed on error if the error is > from the server currently in cache > > > Key: HBASE-7638 > URL: https://issues.apache.org/jira/browse/HBASE-7638 > Project: HBase > Issue Type: Bug >Affects Versions: 0.94.4 >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Minor > Fix For: 0.94.5 > > Attachments: HBASE-7638-v0.patch > > > See HBASE-7268. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-7628) Port HBASE-6509 fast-forwarding FuzzyRowFilter to 0.94
[ https://issues.apache.org/jira/browse/HBASE-7628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl resolved HBASE-7628. -- Resolution: Fixed Hadoop Flags: Reviewed > Port HBASE-6509 fast-forwarding FuzzyRowFilter to 0.94 > -- > > Key: HBASE-7628 > URL: https://issues.apache.org/jira/browse/HBASE-7628 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Anoop Sam John > Fix For: 0.94.5 > > Attachments: HBASE-7628.patch > > > This is to port HBASE-6509: 'Implement fast-forwarding FuzzyRowFilter to > allow filtering rows e.g. by "???alex?b"' to 0.94 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HBASE-6527) Make custom filters plugin
[ https://issues.apache.org/jira/browse/HBASE-6527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh reopened HBASE-6527: --- > Make custom filters plugin > -- > > Key: HBASE-6527 > URL: https://issues.apache.org/jira/browse/HBASE-6527 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu > > More and more custom Filters are created. > We should provide plugin mechanism for these custom Filters. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6527) Make custom filters plugin
[ https://issues.apache.org/jira/browse/HBASE-6527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13561828#comment-13561828 ] Jonathan Hsieh commented on HBASE-6527: --- The argument here is that we have a large number of fairly arcane, sometimes poorly named filters and this should be revisited to simplify, externalize, and improve this. > Make custom filters plugin > -- > > Key: HBASE-6527 > URL: https://issues.apache.org/jira/browse/HBASE-6527 > Project: HBase > Issue Type: New Feature >Reporter: Ted Yu > > More and more custom Filters are created. > We should provide plugin mechanism for these custom Filters. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6527) Make custom filters plugin
[ https://issues.apache.org/jira/browse/HBASE-6527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-6527: -- Issue Type: New Feature (was: Bug) > Make custom filters plugin > -- > > Key: HBASE-6527 > URL: https://issues.apache.org/jira/browse/HBASE-6527 > Project: HBase > Issue Type: New Feature >Reporter: Ted Yu > > More and more custom Filters are created. > We should provide plugin mechanism for these custom Filters. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7503) Add exists(List) in HTableInterface to allow multiple parallel exists at one time
[ https://issues.apache.org/jira/browse/HBASE-7503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13561832#comment-13561832 ] Sergey Shelukhin commented on HBASE-7503: - +1 on latest patch... > Add exists(List) in HTableInterface to allow multiple parallel exists at one > time > - > > Key: HBASE-7503 > URL: https://issues.apache.org/jira/browse/HBASE-7503 > Project: HBase > Issue Type: Improvement >Reporter: Jean-Marc Spaggiari >Assignee: Jean-Marc Spaggiari >Priority: Minor > Fix For: 0.96.0 > > Attachments: HBASE-7503-v0-trunk.patch, HBASE-7503-v10-trunk.patch, > HBASE-7503-v11-trunk.patch, HBASE-7503-v12-trunk.patch, > HBASE-7503-v13-trunk.patch, HBASE-7503-v13-trunk.patch, > HBASE-7503-v1-trunk.patch, HBASE-7503-v2-trunk.patch, > HBASE-7503-v2-trunk.patch, HBASE-7503-v3-trunk.patch, > HBASE-7503-v4-trunk.patch, HBASE-7503-v5-trunk.patch, > HBASE-7503-v7-trunk.patch, HBASE-7503-v8-trunk.patch, > HBASE-7503-v9-trunk.patch > > Original Estimate: 5m > Remaining Estimate: 5m > > We need to have a Boolean[] exists(List gets) throws IOException method > implemented in HTableInterface. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7571) add the notion of per-table or per-column family configuration
[
https://issues.apache.org/jira/browse/HBASE-7571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13561835#comment-13561835
]
Hadoop QA commented on HBASE-7571:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12566331/7571-v3.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 9 new
or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to introduce 1 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces lines longer than
100
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/4162//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4162//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4162//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4162//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4162//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4162//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4162//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/4162//console
This message is automatically generated.
> add the notion of per-table or per-column family configuration
> --
>
> Key: HBASE-7571
> URL: https://issues.apache.org/jira/browse/HBASE-7571
> Project: HBase
> Issue Type: Sub-task
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: 7571-v3.patch, HBASE-7571-v0-based-on-HBASE-7563.patch,
> HBASE-7571-v0-including-HBASE-7563.patch, HBASE-7571-v1.patch,
> HBASE-7571-v2.patch, HBASE-7571-v3.patch
>
>
> Main part of split HBASE-7236.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6527) Make custom filters plugin
[ https://issues.apache.org/jira/browse/HBASE-6527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13561836#comment-13561836 ] Lars Hofhansl commented on HBASE-6527: -- I think we'd get a lot of mileage by just allowing filters to be dynamically loaded (like coprocessors). > Make custom filters plugin > -- > > Key: HBASE-6527 > URL: https://issues.apache.org/jira/browse/HBASE-6527 > Project: HBase > Issue Type: New Feature >Reporter: Ted Yu > > More and more custom Filters are created. > We should provide plugin mechanism for these custom Filters. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5930) Periodically flush the Memstore?
[ https://issues.apache.org/jira/browse/HBASE-5930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13561837#comment-13561837 ] Devaraj Das commented on HBASE-5930: [~lhofhansl], yeah what [~enis] meant IMHO is that the latency from the client's point of view would improve when deferred flush is used for the mutations. Also, we considered the case that users would most likely not want to skip WAL if we promise them that there wouldn't be latency issues (maybe on a busy cluster). But yeah, it'd not make a difference on the overall IOPS in the cluster... [~nkeywal], generally agree with you that we should not remove the skipWal option without giving it a real good thought and before considering more use cases. And, yes the idea of randomizing the flushes across regionservers sounds good. I'll think up how to incorporate that. [~yuzhih...@gmail.com], good catch on the instantiation :) I was focusing on getting the logic right; forgot to instantiate the chore. I'd prefer to leave the shouldFlush() signature as is (it's a matter of implementation that the shouldFlush method implementation is using the same constant underneath but it could be very well a different constant or shouldFlush implementation could be different sometime when this constant is not even used..). > Periodically flush the Memstore? > > > Key: HBASE-5930 > URL: https://issues.apache.org/jira/browse/HBASE-5930 > Project: HBase > Issue Type: Improvement >Reporter: Lars Hofhansl >Assignee: Devaraj Das >Priority: Minor > Fix For: 0.96.0 > > Attachments: 5930-1.patch, 5930-wip.patch > > > A colleague of mine ran into an interesting issue. > He inserted some data with the WAL disabled, which happened to fit in the > aggregate Memstores memory. > Two weeks later he a had problem with the HDFS cluster, which caused the > region servers to abort. He found that his data was lost. Looking at the log > we found that the Memstores were not flushed at all during these two weeks. > Should we have an option to flush memstores periodically. There are obvious > downsides to this, like many small storefiles, etc. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7630) support large number of small regions
[ https://issues.apache.org/jira/browse/HBASE-7630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13561841#comment-13561841 ] Sergey Shelukhin commented on HBASE-7630: - My main goal would be improving compactions (but this can be achieved with certain level-like compaction algorithm, call it stripe-compactions :)), and improving failover (where there are many small regions, it's easier to shuffle them and preserve both locality and balance of data size between servers). Also good for reads, yes. > support large number of small regions > - > > Key: HBASE-7630 > URL: https://issues.apache.org/jira/browse/HBASE-7630 > Project: HBase > Issue Type: Brainstorming >Reporter: Sergey Shelukhin > > Supporting large number of small regions can help improve scalability in > terms of number of tables/cfs/etc., help speed up recovery, and also lessen > the impact of compactions (due to there being a large number of small > compactions). > This is an issue to attempt brainstorming current limitations. > We might want to take this into consideration when designing master tasks > such as in HBASE-5487. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-7655) Separate read latency metrics for compactions and normal read ops
Rishit Shroff created HBASE-7655: Summary: Separate read latency metrics for compactions and normal read ops Key: HBASE-7655 URL: https://issues.apache.org/jira/browse/HBASE-7655 Project: HBase Issue Type: Improvement Reporter: Rishit Shroff Priority: Minor Separate the read latency metrics for compaction ops and normal ops so that we can measure the performance of them separately -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6527) Make custom filters plugin
[ https://issues.apache.org/jira/browse/HBASE-6527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13561856#comment-13561856 ] Nick Dimiduk commented on HBASE-6527: - bq. I think we'd get a lot of mileage by just allowing filters to be dynamically loaded (like coprocessors). +1, but... can we dynamically load coprocessors? I thought a process restart was necessary after mucking with the classpath. Being able to do so for both coprocessors and filters would definitely make them easier for the user to consider in their application designs. That doesn't excuse HBase from shipping exemplary, well documented filters out of the box. > Make custom filters plugin > -- > > Key: HBASE-6527 > URL: https://issues.apache.org/jira/browse/HBASE-6527 > Project: HBase > Issue Type: New Feature >Reporter: Ted Yu > > More and more custom Filters are created. > We should provide plugin mechanism for these custom Filters. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7655) Separate read latency metrics for compactions and normal read ops
[ https://issues.apache.org/jira/browse/HBASE-7655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-7655: - Component/s: metrics > Separate read latency metrics for compactions and normal read ops > - > > Key: HBASE-7655 > URL: https://issues.apache.org/jira/browse/HBASE-7655 > Project: HBase > Issue Type: Improvement > Components: metrics >Reporter: Rishit Shroff >Priority: Minor > > Separate the read latency metrics for compaction ops and normal ops so that > we can measure the performance of them separately -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7651) RegionServerSnapshotManager fails with CancellationException if previous snapshot fails in per region task
[
https://issues.apache.org/jira/browse/HBASE-7651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hsieh updated HBASE-7651:
--
Attachment: hbase-7651.patch
First cut of patch. Need to add unit test. Something very similar to this has
been testing on 20 node cluster for several hours. Not all snapshot succeed but
the snapshotting mechanism no longer gets stuck.
> RegionServerSnapshotManager fails with CancellationException if previous
> snapshot fails in per region task
> --
>
> Key: HBASE-7651
> URL: https://issues.apache.org/jira/browse/HBASE-7651
> Project: HBase
> Issue Type: Sub-task
> Components: snapshots
>Affects Versions: hbase-7290
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
>Priority: Blocker
> Attachments: hbase-7651.patch
>
>
> I've reproduced this problem consistently on a 20 node cluster.
> The first run fails on a node (jon-snaphots-2 in this case) to take snapshot
> due to a NotServingRegionException (this is acceptable)
> {code}
> 2013-01-23 13:32:48,631 DEBUG
> org.apache.hadoop.hbase.errorhandling.ForeignExceptionDispatcher: accepting
> received exception
> org.apache.hadoop.hbase.errorhandling.ForeignException$ProxyThrowable via
> jon-snapshots-2.ent.cloudera.com,22101,1358976524369:org.apache.hadoop.hbase.errorhandling.ForeignException$ProxyThrowable:
> org.apache.hadoop.hbase.NotServingRegionException:
> TestTable,0002493652,1358976652443.b858147ad87a7812ac9a73dd8fef36ad. is
> closing
> at
> org.apache.hadoop.hbase.errorhandling.ForeignException.deserialize(ForeignException.java:184)
> at
> org.apache.hadoop.hbase.procedure.ZKProcedureCoordinatorRpcs.abort(ZKProcedureCoordinatorRpcs.java:240)
> at
> org.apache.hadoop.hbase.procedure.ZKProcedureCoordinatorRpcs$1.nodeCreated(ZKProcedureCoordinatorRpcs.java:182)
> at
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.process(ZooKeeperWatcher.java:294)
> at
> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
> at
> org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
> Caused by:
> org.apache.hadoop.hbase.errorhandling.ForeignException$ProxyThrowable:
> org.apache.hadoop.hbase.NotServingRegionException:
> TestTable,0002493652,1358976652443.b858147ad87a7812ac9a73dd8fef36ad. is
> closing
> at
> org.apache.hadoop.hbase.regionserver.snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool.waitForOutstandingTasks(RegionServerSnapshotManager.java:343)
> at
> org.apache.hadoop.hbase.regionserver.snapshot.FlushSnapshotSubprocedure.flushSnapshot(FlushSnapshotSubprocedure.java:107)
> at
> org.apache.hadoop.hbase.regionserver.snapshot.FlushSnapshotSubprocedure.insideBarrier(FlushSnapshotSubprocedure.java:123)
> at
> org.apache.hadoop.hbase.procedure.Subprocedure.call(Subprocedure.java:181)
> at
> org.apache.hadoop.hbase.procedure.Subprocedure.call(Subprocedure.java:52)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> 2013-01-23 13:32:48,631 DEBUG
> org.apache.hadoop.hbase.errorhandling.ForeignExceptionDispatcher: Recieved
> error, notifying listeners...
> 2013-01-23 13:32:48,730 ERROR org.apache.hadoop.hbase.procedure.Procedure:
> Procedure 'pe-6' execution failed!
> org.apache.hadoop.hbase.errorhandling.ForeignException$ProxyThrowable via
> jon-snapshots-2.ent.cloudera.com,22101,1358976524369:org.apache.hadoop.hbase.errorhandling.ForeignException$ProxyThrowable:
> org.apache.hadoop.hbase.NotServingRegionException:
> TestTable,0002493652,1358976652443.b858147ad87a7812ac9a73dd8fef36ad. is
> closing
> at
> org.apache.hadoop.hbase.errorhandling.ForeignExceptionDispatcher.rethrowException(ForeignExceptionDispatcher.java:84)
> at
> org.apache.hadoop.hbase.procedure.Procedure.waitForLatch(Procedure.java:357)
> at
> org.apache.hadoop.hbase.procedure.Procedure.call(Procedure.java:203)
> at org.apache.hadoop.hbase.procedure.Procedure.call(Procedure.java:68)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>
[jira] [Commented] (HBASE-6527) Make custom filters plugin
[ https://issues.apache.org/jira/browse/HBASE-6527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13561864#comment-13561864 ] Jonathan Hsieh commented on HBASE-6527: --- [~lhofhansl] does this imply moving some of the more arcane to a separate module / jar? Also if a user wants to do a truly custom filter, ideally they'ed go througth the trouble of adding jar to path, and somehow hooking it in (a la coproc). Doing it totally dynamically (a la MR) seems like overkill. > Make custom filters plugin > -- > > Key: HBASE-6527 > URL: https://issues.apache.org/jira/browse/HBASE-6527 > Project: HBase > Issue Type: New Feature >Reporter: Ted Yu > > More and more custom Filters are created. > We should provide plugin mechanism for these custom Filters. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7382) Port ZK.multi support from HBASE-6775 to 0.96
[
https://issues.apache.org/jira/browse/HBASE-7382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13561865#comment-13561865
]
Hadoop QA commented on HBASE-7382:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12566334/7382-trunk-v2.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 6 new
or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 2
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to introduce 1 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.wal.TestHLog
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/4163//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4163//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4163//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4163//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4163//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4163//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4163//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/4163//console
This message is automatically generated.
> Port ZK.multi support from HBASE-6775 to 0.96
> -
>
> Key: HBASE-7382
> URL: https://issues.apache.org/jira/browse/HBASE-7382
> Project: HBase
> Issue Type: Bug
> Components: Zookeeper
>Reporter: Gregory Chanan
>Assignee: Himanshu Vashishtha
>Priority: Critical
> Fix For: 0.96.0
>
> Attachments: 7382-trunk-v2.txt, HBASE-7382-trunk.patch
>
>
> HBASE-6775 adds support for ZK.multi ZKUtil and uses it for the 0.92/0.94
> compatibility fix implemented in HBASE-6710.
> ZK.multi support is most likely useful in 0.96, but since HBASE-6710 is not
> relevant for 0.96, perhaps we should find another use case first before we
> port.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7403) Online Merge
[ https://issues.apache.org/jira/browse/HBASE-7403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13561868#comment-13561868 ] Enis Soztutar commented on HBASE-7403: -- bq. Looks like you have 2 +1's already. I had put up some comments in RB. It is better to address them, or explicitly defer to another jira without going with commit. > Online Merge > > > Key: HBASE-7403 > URL: https://issues.apache.org/jira/browse/HBASE-7403 > Project: HBase > Issue Type: New Feature >Affects Versions: 0.94.3 >Reporter: chunhui shen >Assignee: chunhui shen > Fix For: 0.96.0, 0.94.5 > > Attachments: 7403-trunkv5.patch, 7403-trunkv6.patch, 7403v5.diff, > 7403-v5.txt, 7403v5.txt, hbase-7403-94v1.patch, hbase-7403-trunkv10.patch, > hbase-7403-trunkv11.patch, hbase-7403-trunkv1.patch, > hbase-7403-trunkv5.patch, hbase-7403-trunkv6.patch, hbase-7403-trunkv7.patch, > hbase-7403-trunkv8.patch, hbase-7403-trunkv9.patch, merge region.pdf > > > The feature of this online merge: > 1.Online,no necessary to disable table > 2.Less change for current code, could applied in trunk,0.94 or 0.92,0.90 > 3.Easy to call merege request, no need to input a long region name, only > encoded name enough > 4.No limit when operation, you don't need to tabke care the events like > Server Dead, Balance, Split, Disabing/Enabing table, no need to take care > whether you send a wrong merge request, it has alread done for you > 5.Only little offline time for two merging regions > Usage: > 1.Tool: > bin/hbase org.apache.hadoop.hbase.util.OnlineMerge [-force] [-async] [-show] > > 2.API: static void MergeManager#createMergeRequest > We need merge in the following cases： > 1.Region hole or region overlap, can’t be fix by hbck > 2.Region become empty because of TTL and not reasonable Rowkey design > 3.Region is always empty or very small because of presplit when create table > 4.Too many empty or small regions would reduce the system performance(e.g. > mslab) > Current merge tools only support offline and are not able to redo if > exception is thrown in the process of merging, causing a dirty data > For online system, we need a online merge. > This implement logic of this patch for Online Merge is : > For example, merge regionA and regionB into regionC > 1.Offline the two regions A and B > 2.Merge the two regions in the HDFS(Create regionC’s directory, move > regionA’s and regionB’s file to regionC’s directory, delete regionA’s and > regionB’s directory) > 3.Add the merged regionC to .META. > 4.Assign the merged regionC > As design of this patch , once we do the merge work in the HDFS,we could redo > it until successful if it throws exception or abort or server restart, but > couldn’t be rolled back. > It depends on > Use zookeeper to record the transaction journal state, make redo easier > Use zookeeper to send/receive merge request > Merge transaction is executed on the master > Support calling merge request through API or shell tool > About the merge process, please see the attachment and patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7626) Backport portions of HBASE-7460 to 0.94
[ https://issues.apache.org/jira/browse/HBASE-7626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-7626: - Description: Marking critical so it gets in. Priority: Critical (was: Major) > Backport portions of HBASE-7460 to 0.94 > --- > > Key: HBASE-7626 > URL: https://issues.apache.org/jira/browse/HBASE-7626 > Project: HBase > Issue Type: Sub-task > Components: Client, IPC/RPC >Reporter: Lars Hofhansl >Priority: Critical > Fix For: 0.94.5 > > > Marking critical so it gets in. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7649) client retry timeout doesn't need to do x2 fallback when going to different server
[ https://issues.apache.org/jira/browse/HBASE-7649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HBASE-7649: Status: Open (was: Patch Available) > client retry timeout doesn't need to do x2 fallback when going to different > server > -- > > Key: HBASE-7649 > URL: https://issues.apache.org/jira/browse/HBASE-7649 > Project: HBase > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HBASE-7649-v0.patch > > > See HBASE-7520. When we go to server A, get a bunch of failures, then finally > learn the region is on B it doesn't make sense to wait for 30 seconds before > going to B. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7649) client retry timeout doesn't need to do x2 fallback when going to different server
[ https://issues.apache.org/jira/browse/HBASE-7649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13561882#comment-13561882 ] Sergey Shelukhin commented on HBASE-7649: - doesn't work very well with many failures from many servers for the same region, can fail super fast (4-6sec). I will tweak it and make it configurable on/off. Probably it works better with time based failure, not retry based. > client retry timeout doesn't need to do x2 fallback when going to different > server > -- > > Key: HBASE-7649 > URL: https://issues.apache.org/jira/browse/HBASE-7649 > Project: HBase > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HBASE-7649-v0.patch > > > See HBASE-7520. When we go to server A, get a bunch of failures, then finally > learn the region is on B it doesn't make sense to wait for 30 seconds before > going to B. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7561) Display the total number of regions for a given table on the master webUI
[ https://issues.apache.org/jira/browse/HBASE-7561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13561892#comment-13561892 ] Michael Weng commented on HBASE-7561: - @Ted, what is the command line for building trunk into a tar.gz file? Used to be "mvn -DskipTests=true clean package". It's not generated the tar.gz file anymore on trunk. > Display the total number of regions for a given table on the master webUI > - > > Key: HBASE-7561 > URL: https://issues.apache.org/jira/browse/HBASE-7561 > Project: HBase > Issue Type: Improvement > Components: UI >Affects Versions: 0.94.4 >Reporter: Ming Ma >Assignee: Michael Weng >Priority: Minor > Fix For: 0.96.0 > > Attachments: HBASE-7561.txt, screenshot-1.jpg > > > This is to make it easy to find out the summary of the # of regions for each > table on one web page. Currently you need to click on each table URL to find > out the # of region of that table. We find this useful to support a shared > cluster with different clients. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6466) Enable multi-thread for memstore flush
[ https://issues.apache.org/jira/browse/HBASE-6466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13561913#comment-13561913 ] Sergey Shelukhin commented on HBASE-6466: - Sorry, missed the discussion above. Was there consensus on whether the patch is the culprit? This failure looks eerily familiar to me, but I don't remember where I saw it and how I fixed it, if at all (e.g. may have been one time failure on different patch). There was another patch (HBASE-7329) that makes locking around rolling more granular, too. > Enable multi-thread for memstore flush > -- > > Key: HBASE-6466 > URL: https://issues.apache.org/jira/browse/HBASE-6466 > Project: HBase > Issue Type: Improvement > Components: regionserver >Affects Versions: 0.96.0 >Reporter: chunhui shen >Assignee: chunhui shen >Priority: Critical > Fix For: 0.96.0 > > Attachments: 6466-v6.patch, 6466-v7.patch, HBASE-6466.patch, > HBASE-6466v2.patch, HBASE-6466v3.1.patch, HBASE-6466v3.patch, > HBASE-6466-v4.patch, HBASE-6466-v4.patch, HBASE-6466-v5.patch > > > If the KV is large or Hlog is closed with high-pressure putting, we found > memstore is often above the high water mark and block the putting. > So should we enable multi-thread for Memstore Flush? > Some performance test data for reference, > 1.test environment ： > random writting；upper memstore limit 5.6GB;lower memstore limit 4.8GB;400 > regions per regionserver；row len=50 bytes, value len=1024 bytes;5 > regionserver, 300 ipc handler per regionserver;5 client, 50 thread handler > per client for writing > 2.test results: > one cacheFlush handler, tps: 7.8k/s per regionserver, Flush:10.1MB/s per > regionserver, appears many aboveGlobalMemstoreLimit blocking > two cacheFlush handlers, tps: 10.7k/s per regionserver, Flush:12.46MB/s per > regionserver, > 200 thread handler per client & two cacheFlush handlers, tps:16.1k/s per > regionserver, Flush:18.6MB/s per regionserver -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7651) RegionServerSnapshotManager fails with CancellationException if previous snapshot fails in per region task
[
https://issues.apache.org/jira/browse/HBASE-7651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13561924#comment-13561924
]
Matteo Bertozzi commented on HBASE-7651:
+1, the core of this patch is having one SubprocedurePool per Subprocedure
{code}
switch (snapshot.getType()) {
case FLUSH:
+ SnapshotSubprocedurePool taskManager =
+new SnapshotSubprocedurePool(rss.getServerName().toString(), conf);
return new FlushSnapshotSubprocedure(member, exnDispatcher, wakeMillis,
timeoutMillis, involvedRegions, snapshot, taskManager);
{code}
> RegionServerSnapshotManager fails with CancellationException if previous
> snapshot fails in per region task
> --
>
> Key: HBASE-7651
> URL: https://issues.apache.org/jira/browse/HBASE-7651
> Project: HBase
> Issue Type: Sub-task
> Components: snapshots
>Affects Versions: hbase-7290
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
>Priority: Blocker
> Attachments: hbase-7651.patch
>
>
> I've reproduced this problem consistently on a 20 node cluster.
> The first run fails on a node (jon-snaphots-2 in this case) to take snapshot
> due to a NotServingRegionException (this is acceptable)
> {code}
> 2013-01-23 13:32:48,631 DEBUG
> org.apache.hadoop.hbase.errorhandling.ForeignExceptionDispatcher: accepting
> received exception
> org.apache.hadoop.hbase.errorhandling.ForeignException$ProxyThrowable via
> jon-snapshots-2.ent.cloudera.com,22101,1358976524369:org.apache.hadoop.hbase.errorhandling.ForeignException$ProxyThrowable:
> org.apache.hadoop.hbase.NotServingRegionException:
> TestTable,0002493652,1358976652443.b858147ad87a7812ac9a73dd8fef36ad. is
> closing
> at
> org.apache.hadoop.hbase.errorhandling.ForeignException.deserialize(ForeignException.java:184)
> at
> org.apache.hadoop.hbase.procedure.ZKProcedureCoordinatorRpcs.abort(ZKProcedureCoordinatorRpcs.java:240)
> at
> org.apache.hadoop.hbase.procedure.ZKProcedureCoordinatorRpcs$1.nodeCreated(ZKProcedureCoordinatorRpcs.java:182)
> at
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.process(ZooKeeperWatcher.java:294)
> at
> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
> at
> org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
> Caused by:
> org.apache.hadoop.hbase.errorhandling.ForeignException$ProxyThrowable:
> org.apache.hadoop.hbase.NotServingRegionException:
> TestTable,0002493652,1358976652443.b858147ad87a7812ac9a73dd8fef36ad. is
> closing
> at
> org.apache.hadoop.hbase.regionserver.snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool.waitForOutstandingTasks(RegionServerSnapshotManager.java:343)
> at
> org.apache.hadoop.hbase.regionserver.snapshot.FlushSnapshotSubprocedure.flushSnapshot(FlushSnapshotSubprocedure.java:107)
> at
> org.apache.hadoop.hbase.regionserver.snapshot.FlushSnapshotSubprocedure.insideBarrier(FlushSnapshotSubprocedure.java:123)
> at
> org.apache.hadoop.hbase.procedure.Subprocedure.call(Subprocedure.java:181)
> at
> org.apache.hadoop.hbase.procedure.Subprocedure.call(Subprocedure.java:52)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> 2013-01-23 13:32:48,631 DEBUG
> org.apache.hadoop.hbase.errorhandling.ForeignExceptionDispatcher: Recieved
> error, notifying listeners...
> 2013-01-23 13:32:48,730 ERROR org.apache.hadoop.hbase.procedure.Procedure:
> Procedure 'pe-6' execution failed!
> org.apache.hadoop.hbase.errorhandling.ForeignException$ProxyThrowable via
> jon-snapshots-2.ent.cloudera.com,22101,1358976524369:org.apache.hadoop.hbase.errorhandling.ForeignException$ProxyThrowable:
> org.apache.hadoop.hbase.NotServingRegionException:
> TestTable,0002493652,1358976652443.b858147ad87a7812ac9a73dd8fef36ad. is
> closing
> at
> org.apache.hadoop.hbase.errorhandling.ForeignExceptionDispatcher.rethrowException(ForeignExceptionDispatcher.java:84)
> at
> org.apache.hadoop.hbase.procedure.Procedure.waitForLatch(Procedure.java:357)
> at
> org.apache.hadoop.hbase.procedure.Procedure.call(Procedure.java:203)
> at org.apache.hadoop.hbase.procedure.Procedure.call(Procedure.java:68)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask
[jira] [Created] (HBASE-7656) Clean up line endings to be LF in the repo
Enis Soztutar created HBASE-7656: Summary: Clean up line endings to be LF in the repo Key: HBASE-7656 URL: https://issues.apache.org/jira/browse/HBASE-7656 Project: HBase Issue Type: Bug Reporter: Enis Soztutar Assignee: Enis Soztutar Fix For: 0.96.0 After HBASE-6816, there are still 2 files in the repo with CRLF line endings. We should change recommit them with LF line endings. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7656) Clean up line endings to be LF in the repo
[ https://issues.apache.org/jira/browse/HBASE-7656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-7656: - Priority: Trivial (was: Major) > Clean up line endings to be LF in the repo > --- > > Key: HBASE-7656 > URL: https://issues.apache.org/jira/browse/HBASE-7656 > Project: HBase > Issue Type: Bug >Reporter: Enis Soztutar >Assignee: Enis Soztutar >Priority: Trivial > Fix For: 0.96.0 > > > After HBASE-6816, there are still 2 files in the repo with CRLF line endings. > We should change recommit them with LF line endings. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-7657) Make ModifyTableHandler synchronous
Himanshu Vashishtha created HBASE-7657: -- Summary: Make ModifyTableHandler synchronous Key: HBASE-7657 URL: https://issues.apache.org/jira/browse/HBASE-7657 Project: HBase Issue Type: Bug Components: Admin, Client Affects Versions: 0.96.0 Reporter: Himanshu Vashishtha Assignee: Himanshu Vashishtha Fix For: 0.96.0 This is along the lines of other admin operations such as modifyColumnFamily, AddColumnFamily to make it a synchronous op. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7656) Clean up line endings to be LF in the repo
[
https://issues.apache.org/jira/browse/HBASE-7656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Enis Soztutar updated HBASE-7656:
-
Attachment: hbase-7656.patch
Trivial patch changing line endings for
{code}
# modified:
hbase-server/src/main/resources/hbase-webapps/static/js/jquery.min.js
# modified:
hbase-server/src/test/java/org/apache/hadoop/hbase/zookeeper/TestRecoverableZooKeeper.java
{code}
Will commit this shortly.
> Clean up line endings to be LF in the repo
> ---
>
> Key: HBASE-7656
> URL: https://issues.apache.org/jira/browse/HBASE-7656
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
>Priority: Trivial
> Fix For: 0.96.0
>
> Attachments: hbase-7656.patch
>
>
> After HBASE-6816, there are still 2 files in the repo with CRLF line endings.
> We should change recommit them with LF line endings.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-7658) grant with an empty string as permission should throw an exception
Matteo Bertozzi created HBASE-7658:
--
Summary: grant with an empty string as permission should throw an
exception
Key: HBASE-7658
URL: https://issues.apache.org/jira/browse/HBASE-7658
Project: HBase
Issue Type: Bug
Components: security
Affects Versions: 0.94.4, 0.96.0
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Priority: Trivial
If someone specify an empty permission
{code}grant 'user', ''{code}
AccessControlLists.addUserPermission() output a log message and doesn't change
the permission, but the user doesn't know about it.
{code}
if ((actions == null) || (actions.length == 0)) {
LOG.warn("No actions associated with user
'"+Bytes.toString(userPerm.getUser())+"'");
return;
}
{code}
I think we should throw an exception instead of just logging.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7657) Make ModifyTableHandler synchronous
[ https://issues.apache.org/jira/browse/HBASE-7657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Himanshu Vashishtha updated HBASE-7657: --- Attachment: HBASE-7657.patch TestAdmin pass. This basically revert 5359, and make modify table a synchronous op. > Make ModifyTableHandler synchronous > --- > > Key: HBASE-7657 > URL: https://issues.apache.org/jira/browse/HBASE-7657 > Project: HBase > Issue Type: Bug > Components: Admin, Client >Affects Versions: 0.96.0 >Reporter: Himanshu Vashishtha >Assignee: Himanshu Vashishtha > Fix For: 0.96.0 > > Attachments: HBASE-7657.patch > > > This is along the lines of other admin operations such as modifyColumnFamily, > AddColumnFamily to make it a synchronous op. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7647) 0.94 hfiles v2.1 are not backwards compatible with HFilev2.0
[ https://issues.apache.org/jira/browse/HBASE-7647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-7647: - Attachment: HBASE-7647-2.patch Changed some formatting and spacing. I tested all the things that JD wanted and they all worked. The delete family bloom filter is a different issue and does appear that it will cause errors. > 0.94 hfiles v2.1 are not backwards compatible with HFilev2.0 > > > Key: HBASE-7647 > URL: https://issues.apache.org/jira/browse/HBASE-7647 > Project: HBase > Issue Type: Bug > Components: HFile >Affects Versions: 0.94.4 >Reporter: Elliott Clark >Assignee: Elliott Clark > Attachments: HBASE-7647-2.patch > > > When doing a rolling re-start from 0.92.x to 0.94.x any hfiles written by > 0.94 are incompatibile with any of the 0.92 region servers. This is caused > by the checksums being put into 0.94. > * a minor version was added > * checksums were put into the block > * checksum meta data was added to block headers. > I propose that since these changes are only needed if using > hbase.regionserver.checksum.verify, they should be turned off if that option > is turned off. Doing so will allow rolling upgrades to go smoother. > If a user wants to go from a 0.92 cluster to a 0.94 cluster with > hbase.regionserver.checksum.verify they can: > * Roll out 0.94 > * Change hbase-site.xml > * roll restart the region servers. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7647) 0.94 hfiles v2.1 are not backwards compatible with HFilev2.0
[ https://issues.apache.org/jira/browse/HBASE-7647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-7647: - Attachment: (was: HBASE-7647-1.patch) > 0.94 hfiles v2.1 are not backwards compatible with HFilev2.0 > > > Key: HBASE-7647 > URL: https://issues.apache.org/jira/browse/HBASE-7647 > Project: HBase > Issue Type: Bug > Components: HFile >Affects Versions: 0.94.4 >Reporter: Elliott Clark >Assignee: Elliott Clark > Attachments: HBASE-7647-2.patch > > > When doing a rolling re-start from 0.92.x to 0.94.x any hfiles written by > 0.94 are incompatibile with any of the 0.92 region servers. This is caused > by the checksums being put into 0.94. > * a minor version was added > * checksums were put into the block > * checksum meta data was added to block headers. > I propose that since these changes are only needed if using > hbase.regionserver.checksum.verify, they should be turned off if that option > is turned off. Doing so will allow rolling upgrades to go smoother. > If a user wants to go from a 0.92 cluster to a 0.94 cluster with > hbase.regionserver.checksum.verify they can: > * Roll out 0.94 > * Change hbase-site.xml > * roll restart the region servers. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7657) Make ModifyTableHandler synchronous
[ https://issues.apache.org/jira/browse/HBASE-7657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13561956#comment-13561956 ] Matteo Bertozzi commented on HBASE-7657: +1 HBaseAdmin is supposed to be synchronous and we don't have a way to catch when a modifyTable() operation is completed. If you look at the admin test (before patch), there's a private modifyTable() that does the wait until the operation completed, relying on the implementation details of the master (executor event). > Make ModifyTableHandler synchronous > --- > > Key: HBASE-7657 > URL: https://issues.apache.org/jira/browse/HBASE-7657 > Project: HBase > Issue Type: Bug > Components: Admin, Client >Affects Versions: 0.96.0 >Reporter: Himanshu Vashishtha >Assignee: Himanshu Vashishtha > Fix For: 0.96.0 > > Attachments: HBASE-7657.patch > > > This is along the lines of other admin operations such as modifyColumnFamily, > AddColumnFamily to make it a synchronous op. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7647) 0.94 hfiles v2.1 are not backwards compatible with HFilev2.0
[
https://issues.apache.org/jira/browse/HBASE-7647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13561958#comment-13561958
]
Hadoop QA commented on HBASE-7647:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12566367/HBASE-7647-2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 27 new
or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/4164//console
This message is automatically generated.
> 0.94 hfiles v2.1 are not backwards compatible with HFilev2.0
>
>
> Key: HBASE-7647
> URL: https://issues.apache.org/jira/browse/HBASE-7647
> Project: HBase
> Issue Type: Bug
> Components: HFile
>Affects Versions: 0.94.4
>Reporter: Elliott Clark
>Assignee: Elliott Clark
> Attachments: HBASE-7647-2.patch
>
>
> When doing a rolling re-start from 0.92.x to 0.94.x any hfiles written by
> 0.94 are incompatibile with any of the 0.92 region servers. This is caused
> by the checksums being put into 0.94.
> * a minor version was added
> * checksums were put into the block
> * checksum meta data was added to block headers.
> I propose that since these changes are only needed if using
> hbase.regionserver.checksum.verify, they should be turned off if that option
> is turned off. Doing so will allow rolling upgrades to go smoother.
> If a user wants to go from a 0.92 cluster to a 0.94 cluster with
> hbase.regionserver.checksum.verify they can:
> * Roll out 0.94
> * Change hbase-site.xml
> * roll restart the region servers.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7657) Make ModifyTableHandler synchronous
[ https://issues.apache.org/jira/browse/HBASE-7657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matteo Bertozzi updated HBASE-7657: --- Status: Patch Available (was: Open) > Make ModifyTableHandler synchronous > --- > > Key: HBASE-7657 > URL: https://issues.apache.org/jira/browse/HBASE-7657 > Project: HBase > Issue Type: Bug > Components: Admin, Client >Affects Versions: 0.96.0 >Reporter: Himanshu Vashishtha >Assignee: Himanshu Vashishtha > Fix For: 0.96.0 > > Attachments: HBASE-7657.patch > > > This is along the lines of other admin operations such as modifyColumnFamily, > AddColumnFamily to make it a synchronous op. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
