[jira] [Commented] (HBASE-7824) Improve master start up time when there is log splitting work
[
https://issues.apache.org/jira/browse/HBASE-7824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13624757#comment-13624757
]
ramkrishna.s.vasudevan commented on HBASE-7824:
---
{code}
ServerName currentMetaServer =
this.catalogTracker.getMetaLocationOrReadLocationFromRoot();
{code}
This is read in two places. One in finishInitialization() and the other inside
assignMEta where the META RS is checked with prev ROOT server.
Can we check this only once and then make the pseudo code change as above
mentioned by Jeffrey?
> Improve master start up time when there is log splitting work
> -
>
> Key: HBASE-7824
> URL: https://issues.apache.org/jira/browse/HBASE-7824
> Project: HBase
> Issue Type: Bug
> Components: master
>Reporter: Jeffrey Zhong
>Assignee: Jeffrey Zhong
> Fix For: 0.94.8
>
> Attachments: hbase-7824.patch, hbase-7824_v2.patch,
> hbase-7824_v3.patch, hbase-7824-v7.patch, hbase-7824-v8.patch
>
>
> When there is log split work going on, master start up waits till all log
> split work completes even though the log split has nothing to do with meta
> region servers.
> It's a bad behavior considering a master node can run when log split is
> happening while its start up is blocking by log split work.
> Since master is kind of single point of failure, we should start it ASAP.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8287) TestRegionMergeTransactionOnCluster failed in trunk build #4010
[
https://issues.apache.org/jira/browse/HBASE-8287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13624754#comment-13624754
]
stack commented on HBASE-8287:
--
+1 on commit. Makes sense. The TestTableLockManager is flakey but you should
check the TableHFileOutputFormat fail. If you cannot reproduce local, go ahead
w/ commit.
> TestRegionMergeTransactionOnCluster failed in trunk build #4010
> ---
>
> Key: HBASE-8287
> URL: https://issues.apache.org/jira/browse/HBASE-8287
> Project: HBase
> Issue Type: Bug
> Components: master
>Reporter: chunhui shen
>Assignee: chunhui shen
>Priority: Minor
> Fix For: 0.95.2, 0.98.0
>
> Attachments: hbase-trunk-8287.patch
>
>
> From the log of trunk build #4010:
> {code}
> 2013-04-04 05:45:59,396 INFO
> [MASTER_TABLE_OPERATIONS-quirinus.apache.org,53514,1365054344859-0]
> handler.DispatchMergingRegionHandler(157):
> Cancel merging regions
> testCleanMergeReference,,1365054353296.bf3d60360122d6c83a246f5f96c2cdd1.,
> testCleanMergeReference,testRow0020,1365054353302.72fbc04566e78aa6732531296256a5aa.,
>
> because can't move them together after 842ms
>
> 2013-04-04 05:45:59,396 INFO [hbase-am-zkevent-worker-pool-2-thread-1]
> master.AssignmentManager$4(1164):
> The master has opened the region
> testCleanMergeReference,testRow0020,1365054353302.72fbc04566e78aa6732531296256a5aa.
> that was onlin
> e on quirinus.apache.org,45718,1365054345790
> {code}
> There's a small probability that fail to move merging regions together to
> same regionserver
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7937) Retry log rolling to support HA NN scenario
[ https://issues.apache.org/jira/browse/HBASE-7937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang Xie updated HBASE-7937: - Attachment: HBase-7937-0.94.txt Attached is a patch for 0.94 branch > Retry log rolling to support HA NN scenario > --- > > Key: HBASE-7937 > URL: https://issues.apache.org/jira/browse/HBASE-7937 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.94.5 >Reporter: Himanshu Vashishtha >Assignee: Himanshu Vashishtha > Fix For: 0.95.1 > > Attachments: HBase-7937-0.94.txt, HBASE-7937-trunk.patch, > HBASE-7937-v1.patch > > > A failure in log rolling causes regionserver abort. In case of HA NN, it will > be good if there is a retry mechanism to roll the logs. > A corresponding jira for MemStore retries is HBASE-7507. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7937) Retry log rolling to support HA NN scenario
[
https://issues.apache.org/jira/browse/HBASE-7937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13624753#comment-13624753
]
Hadoop QA commented on HBASE-7937:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12577433/HBase-7937-0.94.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/5169//console
This message is automatically generated.
> Retry log rolling to support HA NN scenario
> ---
>
> Key: HBASE-7937
> URL: https://issues.apache.org/jira/browse/HBASE-7937
> Project: HBase
> Issue Type: Bug
> Components: wal
>Affects Versions: 0.94.5
>Reporter: Himanshu Vashishtha
>Assignee: Himanshu Vashishtha
> Fix For: 0.95.1
>
> Attachments: HBase-7937-0.94.txt, HBASE-7937-trunk.patch,
> HBASE-7937-v1.patch
>
>
> A failure in log rolling causes regionserver abort. In case of HA NN, it will
> be good if there is a retry mechanism to roll the logs.
> A corresponding jira for MemStore retries is HBASE-7507.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7824) Improve master start up time when there is log splitting work
[
https://issues.apache.org/jira/browse/HBASE-7824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13624751#comment-13624751
]
Jeffrey Zhong commented on HBASE-7824:
--
It's only for 0.94 and the adjustment is guaranteed that splitLog will happen
right before {code}assignmentManager.assignMeta();{code} just like before to
deal with the possible data loss issue you mentioned.
> Improve master start up time when there is log splitting work
> -
>
> Key: HBASE-7824
> URL: https://issues.apache.org/jira/browse/HBASE-7824
> Project: HBase
> Issue Type: Bug
> Components: master
>Reporter: Jeffrey Zhong
>Assignee: Jeffrey Zhong
> Fix For: 0.94.8
>
> Attachments: hbase-7824.patch, hbase-7824_v2.patch,
> hbase-7824_v3.patch, hbase-7824-v7.patch, hbase-7824-v8.patch
>
>
> When there is log split work going on, master start up waits till all log
> split work completes even though the log split has nothing to do with meta
> region servers.
> It's a bad behavior considering a master node can run when log split is
> happening while its start up is blocking by log split work.
> Since master is kind of single point of failure, we should start it ASAP.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6286) Upgrade maven-compiler-plugin to 2.5.1
[ https://issues.apache.org/jira/browse/HBASE-6286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13624750#comment-13624750 ] Hudson commented on HBASE-6286: --- Integrated in HBase-TRUNK #4023 (See [https://builds.apache.org/job/HBase-TRUNK/4023/]) HBASE-6286 Move site back up out of hbase-assembly; bad idea (Revision 1465323) HBASE-6286 Move site back up out of hbase-assembly; bad idea (Revision 1465322) Result = FAILURE stack : Files : * /hbase/trunk/src/main/site/resources/images/hbase_logo.png stack : Files : * /hbase/trunk/hbase-assembly/pom.xml * /hbase/trunk/hbase-assembly/src/assembly * /hbase/trunk/hbase-assembly/src/docbkx * /hbase/trunk/hbase-assembly/src/main * /hbase/trunk/hbase-assembly/src/main/assembly * /hbase/trunk/hbase-assembly/src/main/assembly/components.xml * /hbase/trunk/hbase-assembly/src/main/assembly/hadoop-one-compat.xml * /hbase/trunk/hbase-assembly/src/main/assembly/hadoop-two-compat.xml * /hbase/trunk/hbase-assembly/src/main/assembly/src.xml * /hbase/trunk/hbase-assembly/src/site * /hbase/trunk/hbase-assembly/src/xslt * /hbase/trunk/hbase-common/src/main/resources/hbase-default.xml * /hbase/trunk/pom.xml * /hbase/trunk/src * /hbase/trunk/src/main * /hbase/trunk/src/main/docbkx * /hbase/trunk/src/main/docbkx/book.xml * /hbase/trunk/src/main/docbkx/case_studies.xml * /hbase/trunk/src/main/docbkx/community.xml * /hbase/trunk/src/main/docbkx/configuration.xml * /hbase/trunk/src/main/docbkx/customization.xsl * /hbase/trunk/src/main/docbkx/developer.xml * /hbase/trunk/src/main/docbkx/external_apis.xml * /hbase/trunk/src/main/docbkx/getting_started.xml * /hbase/trunk/src/main/docbkx/ops_mgt.xml * /hbase/trunk/src/main/docbkx/performance.xml * /hbase/trunk/src/main/docbkx/preface.xml * /hbase/trunk/src/main/docbkx/rpc.xml * /hbase/trunk/src/main/docbkx/schema_design.xml * /hbase/trunk/src/main/docbkx/security.xml * /hbase/trunk/src/main/docbkx/shell.xml * /hbase/trunk/src/main/docbkx/troubleshooting.xml * /hbase/trunk/src/main/docbkx/upgrading.xml * /hbase/trunk/src/main/docbkx/zookeeper.xml * /hbase/trunk/src/main/site * /hbase/trunk/src/main/site/resources * /hbase/trunk/src/main/site/resources/css * /hbase/trunk/src/main/site/resources/css/freebsd_docbook.css * /hbase/trunk/src/main/site/resources/css/site.css * /hbase/trunk/src/main/site/resources/doap_Hbase.rdf * /hbase/trunk/src/main/site/resources/images * /hbase/trunk/src/main/site/resources/images/big_h_logo.svg * /hbase/trunk/src/main/site/resources/images/hbase_logo.svg * /hbase/trunk/src/main/site/site.vm * /hbase/trunk/src/main/site/site.xml * /hbase/trunk/src/main/site/xdoc * /hbase/trunk/src/main/site/xdoc/acid-semantics.xml * /hbase/trunk/src/main/site/xdoc/bulk-loads.xml * /hbase/trunk/src/main/site/xdoc/cygwin.xml * /hbase/trunk/src/main/site/xdoc/index.xml * /hbase/trunk/src/main/site/xdoc/metrics.xml * /hbase/trunk/src/main/site/xdoc/old_news.xml * /hbase/trunk/src/main/site/xdoc/pseudo-distributed.xml * /hbase/trunk/src/main/site/xdoc/replication.xml * /hbase/trunk/src/main/site/xdoc/resources.xml * /hbase/trunk/src/main/site/xdoc/sponsors.xml * /hbase/trunk/src/main/xslt * /hbase/trunk/src/main/xslt/configuration_to_docbook_section.xsl > Upgrade maven-compiler-plugin to 2.5.1 > -- > > Key: HBASE-6286 > URL: https://issues.apache.org/jira/browse/HBASE-6286 > Project: HBase > Issue Type: Improvement > Components: build >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Minor > Fix For: 0.94.2 > > Attachments: HBASE-6286.patch > > > time mvn -PlocalTests clean install -DskipTests > With 2.5.1: > |user|1m35.634s|1m31.178s|1m31.366s| > |sys|0m06.540s|0m05.376s|0m05.488s| > With 2.0.2 (current): > |user|2m01.168s|1m54.027s|1m57.799s| > |sys|0m05.896s|0m05.912s|0m06.032s| -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8286) Move site back up out of hbase-assembly; bad idea
[ https://issues.apache.org/jira/browse/HBASE-8286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-8286: - Resolution: Fixed Fix Version/s: 0.98.0 0.95.0 Status: Resolved (was: Patch Available) Committed to 0.95 and trunk. Updated doc. on how to build site and deploy it. Its easy enough I'd say that anyone should be able to do it .. bug me if not the case. > Move site back up out of hbase-assembly; bad idea > - > > Key: HBASE-8286 > URL: https://issues.apache.org/jira/browse/HBASE-8286 > Project: HBase > Issue Type: Task >Reporter: stack >Assignee: stack > Fix For: 0.95.0, 0.98.0 > > Attachments: 8286.txt > > > Redoing the packaging, I thought it smart putting it all under the new > hbase-assembly module. A few weeks later, it is not looking like a good idea > (Enis suggested moving the site was maybe 'odd'). Here are a few issues: > + The generated reports will have mention of hbase-assembly because that is > where they are being generated (Elliott pointed out the the scm links mention > hbase-assembly instead of trunk). > + Doug Meil wants to build and deploy the site but currently deploy is a bit > awkward to explain because site presumes it is top level so staging goes to > the wrong place and you have to come along and do fixup afterward; it > shouldn't be so hard. > A possible downside is that we will break Nicks site check added to hadoopqa > because site is an aggregating build plugin... but lets see. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7824) Improve master start up time when there is log splitting work
[ https://issues.apache.org/jira/browse/HBASE-7824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13624740#comment-13624740 ] chunhui shen commented on HBASE-7824: - bq.Since ZK session timeout take a while, HMaster#splitLogAndExpireIfOnline will kick in so there won't be any issue. 1.ZK seession will timeout once java process exit. 2.I think in a complex network we shouldn't assert that ZK session timeout happen after HMaster#splitLogAndExpireIfOnline. e.g. the return of getMetaLocationOrReadLocationFromRoot is hanged for a little time. bq.are you fine with this adjustment? Sorry, why do this adjustment? In addition, is it only for 0.94, no trunk? If trunk only, I think there is no above trouble since ROOT has dropped in trunk > Improve master start up time when there is log splitting work > - > > Key: HBASE-7824 > URL: https://issues.apache.org/jira/browse/HBASE-7824 > Project: HBase > Issue Type: Bug > Components: master >Reporter: Jeffrey Zhong >Assignee: Jeffrey Zhong > Fix For: 0.94.8 > > Attachments: hbase-7824.patch, hbase-7824_v2.patch, > hbase-7824_v3.patch, hbase-7824-v7.patch, hbase-7824-v8.patch > > > When there is log split work going on, master start up waits till all log > split work completes even though the log split has nothing to do with meta > region servers. > It's a bad behavior considering a master node can run when log split is > happening while its start up is blocking by log split work. > Since master is kind of single point of failure, we should start it ASAP. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8286) Move site back up out of hbase-assembly; bad idea
[ https://issues.apache.org/jira/browse/HBASE-8286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13624736#comment-13624736 ] stack commented on HBASE-8286: -- site passed. going to commit. > Move site back up out of hbase-assembly; bad idea > - > > Key: HBASE-8286 > URL: https://issues.apache.org/jira/browse/HBASE-8286 > Project: HBase > Issue Type: Task >Reporter: stack >Assignee: stack > Attachments: 8286.txt > > > Redoing the packaging, I thought it smart putting it all under the new > hbase-assembly module. A few weeks later, it is not looking like a good idea > (Enis suggested moving the site was maybe 'odd'). Here are a few issues: > + The generated reports will have mention of hbase-assembly because that is > where they are being generated (Elliott pointed out the the scm links mention > hbase-assembly instead of trunk). > + Doug Meil wants to build and deploy the site but currently deploy is a bit > awkward to explain because site presumes it is top level so staging goes to > the wrong place and you have to come along and do fixup afterward; it > shouldn't be so hard. > A possible downside is that we will break Nicks site check added to hadoopqa > because site is an aggregating build plugin... but lets see. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8286) Move site back up out of hbase-assembly; bad idea
[
https://issues.apache.org/jira/browse/HBASE-8286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13624731#comment-13624731
]
Hadoop QA commented on HBASE-8286:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12577427/8286.txt
against trunk revision .
{color:red}-1 @author{color}. The patch appears to contain 2 @author tags
which the Hadoop community has agreed to not allow in code contributions.
{color:green}+1 tests included{color}. The patch appears to include 12 new
or modified tests.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces lines longer than
100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.zookeeper.lock.TestZKInterProcessReadWriteLock
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/5167//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5167//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5167//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5167//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5167//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5167//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5167//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5167//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5167//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/5167//console
This message is automatically generated.
> Move site back up out of hbase-assembly; bad idea
> -
>
> Key: HBASE-8286
> URL: https://issues.apache.org/jira/browse/HBASE-8286
> Project: HBase
> Issue Type: Task
>Reporter: stack
>Assignee: stack
> Attachments: 8286.txt
>
>
> Redoing the packaging, I thought it smart putting it all under the new
> hbase-assembly module. A few weeks later, it is not looking like a good idea
> (Enis suggested moving the site was maybe 'odd'). Here are a few issues:
> + The generated reports will have mention of hbase-assembly because that is
> where they are being generated (Elliott pointed out the the scm links mention
> hbase-assembly instead of trunk).
> + Doug Meil wants to build and deploy the site but currently deploy is a bit
> awkward to explain because site presumes it is top level so staging goes to
> the wrong place and you have to come along and do fixup afterward; it
> shouldn't be so hard.
> A possible downside is that we will break Nicks site check added to hadoopqa
> because site is an aggregating build plugin... but lets see.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7824) Improve master start up time when there is log splitting work
[
https://issues.apache.org/jira/browse/HBASE-7824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13624730#comment-13624730
]
Jeffrey Zhong commented on HBASE-7824:
--
{quote}
We will retry if fail to update META location in ROOT RS.
{quote}
Are you referring to HTable.put internal retries? It seems that in high level
you agreed to my pervious statements.
Let's go back to the possible scenario you mentioned above that a root RS
crashed after getMetaLocationOrReadLocationFromRoot. Since ZK session timeout
take a while, HMaster#splitLogAndExpireIfOnline will kick in so there won't be
any issue.
Let's conclude this issue. I'll change the patch to the following pesudo-code
snippet, are you fine with this adjustment?
{code}
...
fileSystemManager.splitAllLogs(sn);
if(serverManager.isServerOnline(currentMetaServer)){
expire(currentMetaServer);
}
...
{code}
> Improve master start up time when there is log splitting work
> -
>
> Key: HBASE-7824
> URL: https://issues.apache.org/jira/browse/HBASE-7824
> Project: HBase
> Issue Type: Bug
> Components: master
>Reporter: Jeffrey Zhong
>Assignee: Jeffrey Zhong
> Fix For: 0.94.8
>
> Attachments: hbase-7824.patch, hbase-7824_v2.patch,
> hbase-7824_v3.patch, hbase-7824-v7.patch, hbase-7824-v8.patch
>
>
> When there is log split work going on, master start up waits till all log
> split work completes even though the log split has nothing to do with meta
> region servers.
> It's a bad behavior considering a master node can run when log split is
> happening while its start up is blocking by log split work.
> Since master is kind of single point of failure, we should start it ASAP.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6593) TestAdmin times out sometimes
[ https://issues.apache.org/jira/browse/HBASE-6593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-6593: - Fix Version/s: (was: 0.95.0) 0.94.2 Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by Lars Hofhansl) > TestAdmin times out sometimes > - > > Key: HBASE-6593 > URL: https://issues.apache.org/jira/browse/HBASE-6593 > Project: HBase > Issue Type: Test >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang >Priority: Minor > Fix For: 0.94.2 > > Attachments: trunk-6593.patch > > > In TestAdmin#splitTest, individual put is used to prepare the test data. We > can group them together so as to avoid possible timeout. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6507) [hbck] TestHBaseFsck ran into TableNotEnabledException
[
https://issues.apache.org/jira/browse/HBASE-6507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-6507:
-
Fix Version/s: (was: 0.95.0)
0.94.2
Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by
Lars Hofhansl)
> [hbck] TestHBaseFsck ran into TableNotEnabledException
> --
>
> Key: HBASE-6507
> URL: https://issues.apache.org/jira/browse/HBASE-6507
> Project: HBase
> Issue Type: Test
> Components: hbck
>Reporter: Jimmy Xiang
>Assignee: Jimmy Xiang
>Priority: Minor
> Fix For: 0.94.2
>
> Attachments: trunk-6507.patch, trunk-6507_v2.patch,
> trunk-6507_v2.patch
>
>
> Here is one sample exception:
> {noformat}
> org.apache.hadoop.hbase.TableNotEnabledException:
> org.apache.hadoop.hbase.TableNotEnabledException: tableHDFSRegioininfoMissing
> at
> org.apache.hadoop.hbase.master.handler.DisableTableHandler.(DisableTableHandler.java:75)
> at
> org.apache.hadoop.hbase.master.HMaster.disableTable(HMaster.java:1154)
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6288) In hbase-daemons.sh, description of the default backup-master file path is wrong
[
https://issues.apache.org/jira/browse/HBASE-6288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-6288:
-
Fix Version/s: (was: 0.95.0)
0.94.2
Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by
Lars Hofhansl)
> In hbase-daemons.sh, description of the default backup-master file path is
> wrong
>
>
> Key: HBASE-6288
> URL: https://issues.apache.org/jira/browse/HBASE-6288
> Project: HBase
> Issue Type: Task
> Components: master, scripts, shell
>Affects Versions: 0.92.0, 0.92.1, 0.94.0
>Reporter: Benjamin Kim
>Assignee: Benjamin Kim
> Fix For: 0.94.2
>
> Attachments: HBASE-6288-92-1.patch, HBASE-6288-92.patch,
> HBASE-6288-94.patch, HBASE-6288-trunk.patch
>
>
> In hbase-daemons.sh, description of the default backup-master file path is
> wrong
> {code}
> # HBASE_BACKUP_MASTERS File naming remote hosts.
> # Default is ${HADOOP_CONF_DIR}/backup-masters
> {code}
> it says the default backup-masters file path is at a hadoop-conf-dir, but
> shouldn't this be HBASE_CONF_DIR?
> also adding following lines to conf/hbase-env.sh would be helpful
> {code}
> # File naming hosts on which backup HMaster will run.
> $HBASE_HOME/conf/backup-masters by default.
> export HBASE_BACKUP_MASTERS=${HBASE_HOME}/conf/backup-masters
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6505) Allow shared RegionObserver state
[ https://issues.apache.org/jira/browse/HBASE-6505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-6505: - Fix Version/s: (was: 0.95.0) 0.94.2 Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by Lars Hofhansl) > Allow shared RegionObserver state > - > > Key: HBASE-6505 > URL: https://issues.apache.org/jira/browse/HBASE-6505 > Project: HBase > Issue Type: New Feature >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl > Fix For: 0.94.2 > > Attachments: 6505-0.94.txt, 6505-trunk.txt, 6505.txt, 6505-v2.txt, > 6505-v3.txt, 6505-v4.txt > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6550) Refactoring ReplicationSink to make it more responsive of cluster health
[ https://issues.apache.org/jira/browse/HBASE-6550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-6550: - Fix Version/s: (was: 0.95.0) 0.94.2 Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by Lars Hofhansl) > Refactoring ReplicationSink to make it more responsive of cluster health > > > Key: HBASE-6550 > URL: https://issues.apache.org/jira/browse/HBASE-6550 > Project: HBase > Issue Type: New Feature > Components: Replication >Reporter: Himanshu Vashishtha >Assignee: Himanshu Vashishtha > Fix For: 0.94.2 > > Attachments: 6550-havealook.txt, HBase-6550-0.94.patch, > HBase-6550-0.94-v2.patch, HBase-6550-0.94-v3.patch, HBase-6550.patch, > HBase-6550-v1.patch, HBase-6550-v3.patch, HBase-6550-v4.patch, > HBase-6550-v5.patch, HBase-6550-v6.patch > > > ReplicationSink replicates the WALEdits in the local cluster. It uses native > HBase client to insert the mutations. Sometime, it takes a while to process > it (may be due to region splitting, gc pause, etc) and it undergoes the > retrial phase. > It has two repercussions: > a) The regionserver handler which is serving the request (till now, a > priority handler) is blocked for this period. > b) The caller may get timed out and it will retry it anyway, but the handler > serving the ReplicationSink requests is still working. > Refactoring ReplicationSink to have the following features: > a) Making it more configurable (have its own number of retrial limit, > connection timeout, etc) > b) Add a fail fast behavior so that it bails out in case caller is timedout, > or any exception in processing the mutation batch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6427) Pluggable compaction and scan policies via coprocessors
[ https://issues.apache.org/jira/browse/HBASE-6427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-6427: - Fix Version/s: (was: 0.95.0) 0.94.2 Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by Lars Hofhansl) > Pluggable compaction and scan policies via coprocessors > --- > > Key: HBASE-6427 > URL: https://issues.apache.org/jira/browse/HBASE-6427 > Project: HBase > Issue Type: New Feature >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl >Priority: Minor > Fix For: 0.94.2 > > Attachments: 6427-0.94-addendum.txt, 6427-0.94.txt, > 6427-notReady.txt, 6427-v10.txt, 6427-v1.txt, 6427-v2.txt, 6427-v3.txt, > 6427-v4.txt, 6427-v5.txt, 6427-v7.txt > > > When implementing higher level stores on top of HBase it is necessary to > allow dynamic control over how long KVs must be kept around. > Semi-static config options for ColumnFamilies (# of version or TTL) is not > sufficient. > This can be done with a few additional coprocessor hooks, or by makeing > Store.ScanInfo pluggable. > Was: > The simplest way to achieve this is to have a pluggable class to determine > the smallestReadpoint for Region. That way outside code can control what KVs > to retain. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6914) Scans/Gets/Mutations don't give a good error if the table is disabled.
[ https://issues.apache.org/jira/browse/HBASE-6914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-6914: - Fix Version/s: (was: 0.95.0) 0.94.2 Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by Lars Hofhansl) > Scans/Gets/Mutations don't give a good error if the table is disabled. > -- > > Key: HBASE-6914 > URL: https://issues.apache.org/jira/browse/HBASE-6914 > Project: HBase > Issue Type: Improvement > Components: Client >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 0.94.2 > > Attachments: HBASE-6914-092-3.patch, HBASE-6914-092-ADD.patch, > HBASE-6914-094-3.patch, HBASE-6914-0.patch, HBASE-6914-1.patch, > HBASE-6914-2.patch, HBASE-6914-3.patch > > > Scan a table that is disabled will have the client retry multiple times and > then will error out with NotServingRegionException. If the table is disabled > there's no need to re-try and the message should be more explicit. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6860) [replication] HBASE-6550 is too aggressive, DDOSes .META.
[ https://issues.apache.org/jira/browse/HBASE-6860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-6860: - Fix Version/s: (was: 0.95.0) 0.94.2 Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by Lars Hofhansl) > [replication] HBASE-6550 is too aggressive, DDOSes .META. > - > > Key: HBASE-6860 > URL: https://issues.apache.org/jira/browse/HBASE-6860 > Project: HBase > Issue Type: Improvement >Reporter: Jean-Daniel Cryans >Assignee: Jean-Daniel Cryans >Priority: Blocker > Fix For: 0.94.2 > > Attachments: HBASE-6860.patch > > > I was doing maintenance on clusters and rebooting a replication slave was > giving me a lot of issues because the master wasn't able to read .META. while > it was bombarded with getClosestRowBefore() coming from all the > ReplicationSinks. Since no regions are assigned, and it doesn't retry, it > goes straight back to the source which retries again. > Also it seems that in some cases with retries=1 we don't even recover from a > stale .META. cache. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6644) HBaseAdmin.createTable should wait more till table is enabled.
[ https://issues.apache.org/jira/browse/HBASE-6644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-6644: - Fix Version/s: (was: 0.95.0) 0.94.2 Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by Lars Hofhansl) > HBaseAdmin.createTable should wait more till table is enabled. > -- > > Key: HBASE-6644 > URL: https://issues.apache.org/jira/browse/HBASE-6644 > Project: HBase > Issue Type: Improvement > Components: Client, test >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang >Priority: Minor > Fix For: 0.94.2 > > Attachments: trunk-6644.patch > > > As mentioned by Jon in HBASE-6576, the issue is not completely fixed. > We need to wait longer. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6643) Accept encoded region name in compacting/spliting region from shell
[ https://issues.apache.org/jira/browse/HBASE-6643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-6643: - Fix Version/s: (was: 0.95.0) 0.94.2 Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by Lars Hofhansl) > Accept encoded region name in compacting/spliting region from shell > --- > > Key: HBASE-6643 > URL: https://issues.apache.org/jira/browse/HBASE-6643 > Project: HBase > Issue Type: Improvement > Components: shell >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang > Fix For: 0.94.2 > > Attachments: trunk-6643.patch, trunk-6643_v2.patch > > > Sometimes, the region name has binary characters. When compacting/splitting > it from shell, the region name is not recognized. If we can support encoded > region name, it will make things easier. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6586) Quarantine Corrupted HFiles with hbck
[
https://issues.apache.org/jira/browse/HBASE-6586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-6586:
-
Fix Version/s: (was: 0.95.0)
0.94.2
Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by
Lars Hofhansl)
> Quarantine Corrupted HFiles with hbck
> -
>
> Key: HBASE-6586
> URL: https://issues.apache.org/jira/browse/HBASE-6586
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 0.92.1
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
> Fix For: 0.94.2
>
> Attachments: 0001-hbase-6568-hbck-quarantine-v6.patch,
> hbase-6586-92-v3.patch, hbase-6586-92-v8a.patch, hbase-6586-92-v8b.patch,
> hbase-6586-92-v8.patch, hbase-6586-92-v9.patch, hbase-6586-94-v3.patch,
> hbase-6586-94-v9.patch, hbase-6586.patch, hbase-6586-trunk-v3.patch,
> hbase-6586-trunk-v4.patch, hbase-6586-trunk-v5.patch,
> hbase-6586-trunk-v6.patch, hbase-6586-trunk-v7.patch,
> hbase-6586-trunk-v8.patch, hbase-6586-trunk-v9.patch
>
>
> We've encountered a few upgrades from 0.90 hbases + 20.2/1.x hdfs to 0.92
> hbases + hdfs 2.x that get stuck. I haven't been able to duplicate the
> problem in my dev environment but we suspect this may be related to
> HDFS-3731. On the HBase side, it seems reasonable to quarantine what are
> most likely truncated hfiles, so that can could later be recovered.
> Here's an example of the exception we've encountered:
> {code}
> 2012-07-18 05:55:01,152 ERROR handler.OpenRegionHandler
> (OpenRegionHandler.java:openRegion(346)) - Failed open of
> region=user_mappings,080112102AA76EF98197605D341B9E6C5824D2BC|1001,1317824890618.eaed0e7abc6d27d28ff0e5a9b49c4c
> 0d.
> java.io.IOException: java.lang.IllegalArgumentException: Invalid HFile
> version: 842220600 (expected to be between 1 and 2)
> at
> org.apache.hadoop.hbase.io.hfile.FixedFileTrailer.readFromStream(FixedFileTrailer.java:306)
>
> at org.apache.hadoop.hbase.io.hfile.HFile.pickReaderVersion(HFile.java:371)
> at org.apache.hadoop.hbase.io.hfile.HFile.createReader(HFile.java:387)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile$Reader.(StoreFile.java:1026)
>
> at org.apache.hadoop.hbase.regionserver.StoreFile.open(StoreFile.java:485)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:566)
>
> at org.apache.hadoop.hbase.regionserver.Store.loadStoreFiles(Store.java:286)
> at org.apache.hadoop.hbase.regionserver.Store.(Store.java:223)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.instantiateHStore(HRegion.java:2534)
>
> at org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:454)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:3282)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:3230)
> at
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:331)
> at
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:107)
> at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:169)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>
> at java.lang.Thread.run(Thread.java:619)
> Caused by: java.lang.IllegalArgumentException: Invalid HFile version:
> 842220600 (expected to be between 1 and 2)
> at org.apache.hadoop.hbase.io.hfile.HFile.checkFormatVersion(HFile.java:515)
> at
> org.apache.hadoop.hbase.io.hfile.FixedFileTrailer.readFromStream(FixedFileTrailer.java:303)
>
> ... 17 more
> {code}
> Specifically -- the FixedFileTrailer are incorrect, and seemingly missing.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6522) Expose locks and leases to Coprocessors
[
https://issues.apache.org/jira/browse/HBASE-6522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-6522:
-
Fix Version/s: (was: 0.95.0)
0.94.2
Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by
Lars Hofhansl)
> Expose locks and leases to Coprocessors
> ---
>
> Key: HBASE-6522
> URL: https://issues.apache.org/jira/browse/HBASE-6522
> Project: HBase
> Issue Type: Improvement
>Reporter: Lars Hofhansl
>Assignee: Lars Hofhansl
> Fix For: 0.94.2
>
> Attachments: 6522.txt, 6522-v2.txt
>
>
> Currently it is not possible for CP to implement any of checkAndMutate type
> operations, because coprocessor have no way create a lock, because getLock is
> private HRegion (interestingly ReleaseLock is public).
> In addition it would nice if Coprocessor could hook into the RegionServers'
> Lease management.
> Here I propose two trivial changes:
> # Make HRegion.getLock public
> # Add {code}Leases getLeases(){code} to RegionServerServices (and hence to
> HRegionServer)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6444) Expose the ability to set custom HTTP Request Headers for the REST client used by RemoteHTable
[ https://issues.apache.org/jira/browse/HBASE-6444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-6444: - Fix Version/s: (was: 0.95.0) 0.94.2 Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by Lars Hofhansl) > Expose the ability to set custom HTTP Request Headers for the REST client > used by RemoteHTable > -- > > Key: HBASE-6444 > URL: https://issues.apache.org/jira/browse/HBASE-6444 > Project: HBase > Issue Type: Improvement > Components: REST >Reporter: Erich Hochmuth >Assignee: Jimmy Xiang > Fix For: 0.94.2 > > Attachments: HBASE-6444-0.94.patch, HBASE-6444.patch, > trunk-6444.patch, trunk-6444_v2.patch > > Original Estimate: 48h > Remaining Estimate: 48h > > My corporate security office (ISO) requires that all http traffic get routed > through a Web Access Management layer > (http://en.wikipedia.org/wiki/Web_access_management) > Our Hadoop cluster has been segmented by a virtual network with all access to > HBase from outside clients being managed through HBase Stargate rest server. > The corporate WAM system requires that all http clients authenticate with it > first before making any http request to any http service in the corporate > network. After the http client authenticates with the WAM system the WAM > system returns the client a set of values that must be inserted into a http > cookie and request header of all future http requests to other http clients. > This would mean that all requests through the RemoteHTable interface would > require that this cookie and request header be set as part of the http > request. org.apache.hadoop.hbase.rest.client.Client looks like the > appropriate place that this functionality would need to be plugged into. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6373) Add more context information to audit log messages
[ https://issues.apache.org/jira/browse/HBASE-6373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-6373: - Fix Version/s: (was: 0.95.0) 0.94.2 Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by Lars Hofhansl) > Add more context information to audit log messages > -- > > Key: HBASE-6373 > URL: https://issues.apache.org/jira/browse/HBASE-6373 > Project: HBase > Issue Type: Improvement > Components: security >Affects Versions: 0.94.2, 0.95.2 >Reporter: Marcelo Vanzin >Priority: Minor > Fix For: 0.94.2 > > Attachments: accesscontroller094.patch, accesscontroller.patch, > accesscontroller.patch > > > The attached patch adds more information to the audit log messages; namely, > it includes the IP address where the request originated, if it's available. > The patch is against trunk, but I've tested it against the 0.92 branch. I > didn't find any unit test for this code, please let me know if I missed > something. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6308) Coprocessors should be loaded in a custom ClassLoader to prevent dependency conflicts with HBase
[ https://issues.apache.org/jira/browse/HBASE-6308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-6308: - Fix Version/s: (was: 0.95.0) 0.94.2 Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by Lars Hofhansl) > Coprocessors should be loaded in a custom ClassLoader to prevent dependency > conflicts with HBase > > > Key: HBASE-6308 > URL: https://issues.apache.org/jira/browse/HBASE-6308 > Project: HBase > Issue Type: Improvement > Components: Coprocessors >Affects Versions: 0.92.1, 0.94.0 >Reporter: James Baldassari >Assignee: Andrew Purtell > Fix For: 0.94.2 > > Attachments: 6308-v2.txt, HBASE-6308-0.92.patch, > HBASE-6308-0.92-with-test.patch, HBASE-6308-0.94-with-test.patch, > HBASE-6308-trunk.patch, HBASE-6308-trunk-with-test.patch > > > Currently each coprocessor is loaded with a URLClassLoader that puts the > coprocessor's jar at the beginning of the classpath. The URLClassLoader > always tries to load classes from the parent ClassLoader first and only > attempts to load from its own configured URLs if the class was not found by > the parent. This class loading behavior can be problematic for coprocessors > that have common dependencies with HBase but whose versions are incompatible. > For example, I have a coprocessor that depends on a different version of > Avro than the version used by HBase. The current class loading behavior > results in NoSuchMethodErrors in my coprocessor because some Avro classes > have already been loaded by HBase, and the ClassLoader for my coprocessor > picks up HBase's loaded classes first. > My proposed solution to this problem is to use a custom ClassLoader when > instantiating coprocessor instances. This custom ClassLoader would always > attempt to load classes from the coprocessor's jar first and would only > delegate to the parent ClassLoader if the class were not found in the > coprocessor jar. However, certain classes would need to be exempt from this > behavior. As an example, if the Copcoessor interface were loaded by both the > region server's ClassLoader and the coprocessor's custom ClassLoader, then > the region server would get a ClassCastException when attempting to cast the > coprocessor instance to the Coprocessor interface. This problem can be > avoided by defining a set of class name prefixes that would be exempt from > loading by the custom ClassLoader. When loading a class, if the class starts > with any of these prefixes (e.g. "org.apache.hadoop"), then the ClassLoader > would delegate immediately to the parent ClassLoader. > I've already implemented a patch to provide this functionality which I'll > attach shortly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6291) Don't retry increments on an invalid cell
[
https://issues.apache.org/jira/browse/HBASE-6291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-6291:
-
Fix Version/s: (was: 0.95.0)
0.94.2
Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by
Lars Hofhansl)
> Don't retry increments on an invalid cell
> -
>
> Key: HBASE-6291
> URL: https://issues.apache.org/jira/browse/HBASE-6291
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 0.90.6, 0.92.1, 0.94.0
>Reporter: Jean-Daniel Cryans
>Assignee: Lars Hofhansl
> Fix For: 0.94.2
>
> Attachments: 6291-0.94.txt, 6291-0.96.txt, 6291-0.96-v2.txt,
> 6291-0.96-v3.txt, 6291-0.96-v4.txt
>
>
> This says it all:
> {noformat}
> ERROR: org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after
> attempts=7, exceptions:
> Thu Jun 28 18:34:44 UTC 2012,
> org.apache.hadoop.hbase.client.HTable$8@4eabaf8c, java.io.IOException:
> java.io.IOException: Attempted to increment field that isn't 64 bits wide
> {noformat}
> {{HRegion}} should be modified here to send a DoNotRetryIOException:
> {code}
> if (wrongLength) {
> throw new DoNotRetryIOException(
> "Attempted to increment field that isn't 64 bits wide");
> }
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6286) Upgrade maven-compiler-plugin to 2.5.1
[ https://issues.apache.org/jira/browse/HBASE-6286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-6286: - Fix Version/s: (was: 0.95.0) 0.94.2 Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by Lars Hofhansl) > Upgrade maven-compiler-plugin to 2.5.1 > -- > > Key: HBASE-6286 > URL: https://issues.apache.org/jira/browse/HBASE-6286 > Project: HBase > Issue Type: Improvement > Components: build >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Minor > Fix For: 0.94.2 > > Attachments: HBASE-6286.patch > > > time mvn -PlocalTests clean install -DskipTests > With 2.5.1: > |user|1m35.634s|1m31.178s|1m31.366s| > |sys|0m06.540s|0m05.376s|0m05.488s| > With 2.0.2 (current): > |user|2m01.168s|1m54.027s|1m57.799s| > |sys|0m05.896s|0m05.912s|0m06.032s| -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5728) Methods Missing in HTableInterface
[ https://issues.apache.org/jira/browse/HBASE-5728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-5728: - Fix Version/s: (was: 0.95.0) 0.94.2 Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by Lars Hofhansl) > Methods Missing in HTableInterface > -- > > Key: HBASE-5728 > URL: https://issues.apache.org/jira/browse/HBASE-5728 > Project: HBase > Issue Type: Improvement > Components: Client >Reporter: Bing Li >Assignee: Jimmy Xiang > Fix For: 0.94.2 > > Attachments: trunk-5728.patch, trunk-5728_v2.patch, > trunk-5728_v3.patch, trunk-5728_v4.patch > > > Dear all, > I found some methods existed in HTable were not in HTableInterface. >setAutoFlush >setWriteBufferSize >... > In most cases, I manipulate HBase through HTableInterface from HTablePool. If > I need to use the above methods, how to do that? > I am considering writing my own table pool if no proper ways. Is it fine? > Thanks so much! > Best regards, > Bing -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5714) Add write permissions check before any hbck run that modifies hdfs.
[ https://issues.apache.org/jira/browse/HBASE-5714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-5714: - Fix Version/s: (was: 0.95.0) 0.94.2 Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by Lars Hofhansl) > Add write permissions check before any hbck run that modifies hdfs. > --- > > Key: HBASE-5714 > URL: https://issues.apache.org/jira/browse/HBASE-5714 > Project: HBase > Issue Type: Improvement > Components: hbck >Affects Versions: 0.90.6, 0.92.2, 0.94.0, 0.95.2 >Reporter: Jonathan Hsieh >Assignee: Liang Xie > Fix For: 0.94.2 > > Attachments: HBASE-5628.patch, HBASE-5628.patch.v2, > hbase-5714-90.patch, hbase-5714-92.patch, hbase-5714-94.patch > > > We encoutered a situation where hbck was run by an under-privileged user that > was unable to write/modify/merge regions due to hdfs perms. Unfortunately, > this user was alerted of this after several minutes of read-only operations. > hbck should fail early by having a write perm check and providing actionable > advice to the hbase admin. > Maybe something like: "Current user yy does not have write perms to home>. Please run hbck as hdfs user xxx" -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5631) hbck should handle case where .tableinfo file is missing.
[ https://issues.apache.org/jira/browse/HBASE-5631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-5631: - Fix Version/s: (was: 0.95.0) 0.94.2 Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by Lars Hofhansl) > hbck should handle case where .tableinfo file is missing. > - > > Key: HBASE-5631 > URL: https://issues.apache.org/jira/browse/HBASE-5631 > Project: HBase > Issue Type: Improvement > Components: hbck >Affects Versions: 0.92.2, 0.94.0, 0.95.2 >Reporter: Jonathan Hsieh >Assignee: Jie Huang > Fix For: 0.94.2 > > Attachments: hbase-5631-addendum.patch, hbase-5631.patch, > hbase-5631-v1.patch, hbase-5631-v2.patch > > > 0.92+ branches have a .tableinfo file which could be missing from hdfs. hbck > should be able to detect and repair this properly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5582) "No HServerInfo found for" should be a WARNING message
[
https://issues.apache.org/jira/browse/HBASE-5582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-5582:
-
Fix Version/s: (was: 0.95.0)
0.94.2
Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by
Lars Hofhansl)
> "No HServerInfo found for" should be a WARNING message
> --
>
> Key: HBASE-5582
> URL: https://issues.apache.org/jira/browse/HBASE-5582
> Project: HBase
> Issue Type: Improvement
> Components: master
>Affects Versions: 0.90.4
>Reporter: Shrijeet Paliwal
>Assignee: Kevin Odell
>Priority: Trivial
> Labels: newbie
> Fix For: 0.94.2
>
> Attachments: HBASE-5582.patch, HBASE-5582.patch1
>
>
> The message from RegionServerTracker "No HServerInfo found for..." is easy to
> miss. It should not be INFO.
> From irc chat
> {noformat}
> jdcryans
> JohnP789: can you grep for "No HServerInfo found for" in that log?
> jdcryans
> wait I see it
> jdcryans
> ok there's your problem
> shrijeet_
> Yes it is there
> shrijeet_
> jdcryans: it should be INFO, why?
> jdcryans
> it shouldn't be INFO, it's so easy to miss
> jdcryans
> it's not the first time we have to look super closely to figure this one out
> shrijeet_
> yes , I will file a jira
> jdcryans
> in any case it's a mismatch in that machine's DNS config
> shrijeet_
> anyways JohnP789 is waiting :) go on
> JohnP789
> haha!
> JohnP789
> yes... ??? :-)
> jdcryans
> the master is expecting a RS called
> "localhost.localdomain,53875,1328924863478"
> 17:26 jdcryans
> but the RS calls itself "localhost,53875,1328924863478"
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6916) HBA logs at info level errors that won't show in the shell
[
https://issues.apache.org/jira/browse/HBASE-6916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-6916:
-
Fix Version/s: (was: 0.95.0)
0.94.2
Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by
Lars Hofhansl)
> HBA logs at info level errors that won't show in the shell
> --
>
> Key: HBASE-6916
> URL: https://issues.apache.org/jira/browse/HBASE-6916
> Project: HBase
> Issue Type: Bug
> Components: shell
>Affects Versions: 0.90.6, 0.92.1, 0.94.1, 0.95.2
>Reporter: Jean-Daniel Cryans
>Assignee: Jean-Daniel Cryans
>Priority: Minor
> Fix For: 0.94.2
>
> Attachments: HBASE-6916-0.94.patch, HBASE-6916.patch,
> HBASE-6916-v2.patch
>
>
> There is a weird interaction between the shell and HBA. When you try to close
> a region that doesn't exist, it doesn't throw any error:
> {noformat}
> hbase(main):029:0> close_region 'thisisaninvalidregion'
> 0 row(s) in 0.0580 seconds
> {noformat}
> Normally one should get UnknownRegionException. Starting the shell with "-d"
> I see what a non-shell user would see along with a ton of logging from ZK
> (skipped here):
> {noformat}
> INFO client.HBaseAdmin: No server in .META. for thisisaninvalidregion;
> pair=null
> {noformat}
> But again this is not the right message, it should have shown
> {noformat}
> INFO client.HBaseAdmin: No server in .META. for thisisaninvalidregion;
> pair=null
> {noformat}
> And this is because that part of the code treats both UnknownRegionException
> and NoServerForRegionException like if it was the same thing.
> There is also some ugliness in flush, compact, and split but it normally
> doesn't show since the code treats everything like it's a table and sends a
> TableNotFoundException.
> This jira is about making sure that the exceptions are correctly coming out.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6927) WrongFS using HRegionInfo.getTableDesc() and different fs for hbase.root and fs.defaultFS
[
https://issues.apache.org/jira/browse/HBASE-6927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-6927:
-
Fix Version/s: (was: 0.95.0)
0.94.2
Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by
Lars Hofhansl)
> WrongFS using HRegionInfo.getTableDesc() and different fs for hbase.root and
> fs.defaultFS
> -
>
> Key: HBASE-6927
> URL: https://issues.apache.org/jira/browse/HBASE-6927
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.92.2, 0.94.1, 0.95.2
>Reporter: Matteo Bertozzi
>Assignee: Matteo Bertozzi
> Fix For: 0.94.2
>
> Attachments: 6927.094.txt, HBASE-6927-v0.patch
>
>
> Calling HRegionInfo.getTableDesc() with different fs schema for hbase.root
> and fs.defaultFS raises "IllegalArgumentException: Wrong FS" exception.
> HRegionInfo.getTableDesc() is called only by bin/region_mover.rb to get the
> table name and can be easily replaced, getTableDesc() is also deprecated.
> The main problem is that getTableDesc() doesn't replace fs.defaultFS with
> hbase.root as all the other hbase code (all the code does this, except
> getTableDesc)
> {code}
> Configuration c = HBaseConfiguration.create();
> c.set("fs.defaultFS", c.get(HConstants.HBASE_DIR));
> c.set("fs.default.name", c.get(HConstants.HBASE_DIR));
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3271) Allow .META. table to be exported
[
https://issues.apache.org/jira/browse/HBASE-3271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-3271:
-
Fix Version/s: (was: 0.95.0)
0.94.2
Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by
Lars Hofhansl)
> Allow .META. table to be exported
> -
>
> Key: HBASE-3271
> URL: https://issues.apache.org/jira/browse/HBASE-3271
> Project: HBase
> Issue Type: Improvement
> Components: util
>Affects Versions: 0.20.6
>Reporter: Ted Yu
>Assignee: Liang Xie
> Fix For: 0.94.2
>
> Attachments: 3271.94, HBASE-3271.patch, HBASE-3271-v2.patch
>
>
> I tried to export .META. table in 0.20.6 and got:
> [hadoop@us01-ciqps1-name01 hbase]$ bin/hbase
> org.apache.hadoop.hbase.mapreduce.Export .META. h-meta 1 0 0
> 10/11/23 20:59:05 INFO jvm.JvmMetrics: Initializing JVM Metrics with
> processName=JobTracker, sessionId=
> 2010-11-23 20:59:05.255::INFO: Logging to STDERR via
> org.mortbay.log.StdErrLog
> 2010-11-23 20:59:05.255::INFO: verisons=1, starttime=0,
> endtime=9223372036854775807
> 10/11/23 20:59:05 INFO zookeeper.ZooKeeper: Client
> environment:zookeeper.version=3.2.2-888565, built on 12/08/2009 21:51 GMT
> 10/11/23 20:59:05 INFO zookeeper.ZooKeeper: Client
> environment:host.name=us01-ciqps1-name01.carrieriq.com
> 10/11/23 20:59:05 INFO zookeeper.ZooKeeper: Client
> environment:java.version=1.6.0_21
> 10/11/23 20:59:05 INFO zookeeper.ZooKeeper: Client
> environment:java.vendor=Sun Microsystems Inc.
> ...
> 10/11/23 20:59:05 INFO zookeeper.ClientCnxn: Server connection successful
> 10/11/23 20:59:05 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode
> /hbase/root-region-server got 10.202.50.112:60020
> 10/11/23 20:59:05 DEBUG client.HConnectionManager$TableServers: Found ROOT at
> 10.202.50.112:60020
> 10/11/23 20:59:05 DEBUG client.HConnectionManager$TableServers: Cached
> location for .META.,,1 is us01-ciqps1-grid02.carrieriq.com:60020
> Exception in thread "main" java.io.IOException: Expecting at least one region.
> at
> org.apache.hadoop.hbase.mapreduce.TableInputFormatBase.getSplits(TableInputFormatBase.java:281)
> at
> org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:885)
> at
> org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:779)
> at org.apache.hadoop.mapreduce.Job.submit(Job.java:432)
> at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:447)
> at org.apache.hadoop.hbase.mapreduce.Export.main(Export.java:146)
> Related code is:
> if (keys == null || keys.getFirst() == null ||
> keys.getFirst().length == 0) {
> throw new IOException("Expecting at least one region.");
> }
> My intention was to save the dangling rows in .META. (for future
> investigation) which prevented a table from being created.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6912) Filters are not properly applied in certain cases
[
https://issues.apache.org/jira/browse/HBASE-6912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-6912:
-
Fix Version/s: (was: 0.95.0)
0.94.2
Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by
Lars Hofhansl)
> Filters are not properly applied in certain cases
> -
>
> Key: HBASE-6912
> URL: https://issues.apache.org/jira/browse/HBASE-6912
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.94.1
>Reporter: Alex Newman
>Assignee: Lars Hofhansl
> Fix For: 0.94.2
>
> Attachments: 6912-0.94.txt, 6912-0.96.txt, minimalTest.java
>
>
> Steps to reproduce:
> Create a table, load data into it. Flush the table.
> Do a scan with
> 1. Some filter which should not match the first entry in the scan
> 2. Where one specifies a family and column.
> You will notice that the first entry is returned even though it doesn't match
> the filter.
> It looks like the when the first KeyValue of a scan in the column from the
> point of view of the code
> HRegion.java
> {code}
> } else if (kv != null && !kv.isInternal() && filterRowKey(currentRow)) {
> {code}
> Is generated by
> {code}
> public static KeyValue createLastOnRow(final byte [] row,
> final int roffset, final int rlength, final byte [] family,
> final int foffset, final int flength, final byte [] qualifier,
> final int qoffset, final int qlength) { return new KeyValue(row, roffset,
> rlength, family, foffset, flength, qualifier, qoffset, qlength,
> HConstants.OLDEST_TIMESTAMP, Type.Minimum, null, 0, 0); }
> {code}
> So it is always internal from that point of the code.
> Only later from within
> StoreScanner.java
> {code}
> public synchronized boolean next(List outResult, int limit, String
> metric) throws IOException {
>
> LOOP: while((kv = this.heap.peek()) != null) {
> {code}
> ( The second time through)
> Do we get the actual kv, with a proper type and timestamp. This seems to mess
> with filtering.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7801) Allow a deferred sync option per Mutation.
[
https://issues.apache.org/jira/browse/HBASE-7801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13624701#comment-13624701
]
Hadoop QA commented on HBASE-7801:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12577426/7801-0.96-v7.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 156
new or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 1
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces lines longer than
100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/5166//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5166//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5166//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5166//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5166//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5166//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5166//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5166//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5166//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/5166//console
This message is automatically generated.
> Allow a deferred sync option per Mutation.
> --
>
> Key: HBASE-7801
> URL: https://issues.apache.org/jira/browse/HBASE-7801
> Project: HBase
> Issue Type: Sub-task
>Affects Versions: 0.95.0, 0.94.6
>Reporter: Lars Hofhansl
>Assignee: Lars Hofhansl
> Fix For: 0.95.1, 0.98.0, 0.94.7
>
> Attachments: 7801-0.94-v1.txt, 7801-0.94-v2.txt, 7801-0.94-v3.txt,
> 7801-0.96-full-v2.txt, 7801-0.96-full-v3.txt, 7801-0.96-full-v4.txt,
> 7801-0.96-full-v5.txt, 7801-0.96-v1.txt, 7801-0.96-v6.txt, 7801-0.96-v7.txt
>
>
> Won't have time for parent. But a deferred sync option on a per operation
> basis comes up quite frequently.
> In 0.96 this can be handled cleanly via protobufs and 0.94 we can have a
> special mutation attribute.
> For batch operation we'd take the safest sync option of any of the mutations.
> I.e. if there is at least one that wants to be flushed we'd sync the batch,
> if there's none of those but at least one that wants deferred flush we defer
> flush the batch, etc.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6906) TestHBaseFsck#testQuarantine* tests are flakey due to TableNotEnabledException
[
https://issues.apache.org/jira/browse/HBASE-6906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-6906:
-
Fix Version/s: (was: 0.95.0)
0.94.2
Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by
Lars Hofhansl)
> TestHBaseFsck#testQuarantine* tests are flakey due to TableNotEnabledException
> --
>
> Key: HBASE-6906
> URL: https://issues.apache.org/jira/browse/HBASE-6906
> Project: HBase
> Issue Type: Bug
> Components: hbck, test
>Affects Versions: 0.92.3, 0.94.2, 0.95.2
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
> Fix For: 0.94.2
>
> Attachments: hbase-6906-94.patch, hbase-6906.patch
>
>
> This test fails periodically (1 out of 10) times on our internal jenkins
> instance.
> {code}
> FAILED TESTS
>
> 1 tests failed.
> REGRESSION:
> org.apache.hadoop.hbase.util.TestHBaseFsck.testQuarantineMissingRegionDir
> Error Message:
> org.apache.hadoop.hbase.TableNotEnabledException:
> testQuarantineMissingRegionDir at
> org.apache.hadoop.hbase.master.handler.DisableTableHandler.(DisableTableHandler.java:75)
> at org.apache.hadoop.hbase.master.HMaster.disableTable(HMaster.java:1170) at
> sun.reflect.GeneratedMethodAccessor68.invoke(Unknown Source) at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597) at
> org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
> at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1345)
> Stack Trace:
> org.apache.hadoop.hbase.TableNotEnabledException:
> org.apache.hadoop.hbase.TableNotEnabledException:
> testQuarantineMissingRegionDir
> at
> org.apache.hadoop.hbase.master.handler.DisableTableHandler.(DisableTableHandler.java:75)
> at org.apache.hadoop.hbase.master.HMaster.disableTable(HMaster.java:1170)
> at sun.reflect.GeneratedMethodAccessor68.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
> at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1345)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
> at
> org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:90)
> at
> org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:79)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.disableTableAsync(HBaseAdmin.java:766)
> at
> org.apache.hadoop.hbase.util.TestHBaseFsck.deleteTable(TestHBaseFsck.java:344)
> at
> org.apache.hadoop.hbase.util.TestHBaseFsck.doQuarantineTest(TestHBaseFsck.java:1351)
> at
> org.apache.hadoop.hbase.util.TestHBaseFsck.testQuarantineMissingRegionDir(TestHBaseFsck.java:1433)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at
> org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:62)
> Caused by:
> org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hbase.TableNotEnabledException):
> org.apache.hadoop.hbase.TableNotEnabledException:
> testQuarantineMissingRegionDir
> at
> org.apache.hadoop.hbase.master.handler.DisableTableHandler.(DisableTableHandler.java:75)
> at org.apache.hadoop.hbase.master.HMaster.disableTable(HMaster.java:1170)
> at sun.reflect.GeneratedMethodAccessor68.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
> at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(
[jira] [Commented] (HBASE-8287) TestRegionMergeTransactionOnCluster failed in trunk build #4010
[
https://issues.apache.org/jira/browse/HBASE-8287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13624700#comment-13624700
]
Hadoop QA commented on HBASE-8287:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12577425/hbase-trunk-8287.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.master.TestTableLockManager
org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/5165//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5165//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5165//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5165//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5165//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5165//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5165//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5165//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5165//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/5165//console
This message is automatically generated.
> TestRegionMergeTransactionOnCluster failed in trunk build #4010
> ---
>
> Key: HBASE-8287
> URL: https://issues.apache.org/jira/browse/HBASE-8287
> Project: HBase
> Issue Type: Bug
> Components: master
>Reporter: chunhui shen
>Assignee: chunhui shen
>Priority: Minor
> Fix For: 0.95.2, 0.98.0
>
> Attachments: hbase-trunk-8287.patch
>
>
> From the log of trunk build #4010:
> {code}
> 2013-04-04 05:45:59,396 INFO
> [MASTER_TABLE_OPERATIONS-quirinus.apache.org,53514,1365054344859-0]
> handler.DispatchMergingRegionHandler(157):
> Cancel merging regions
> testCleanMergeReference,,1365054353296.bf3d60360122d6c83a246f5f96c2cdd1.,
> testCleanMergeReference,testRow0020,1365054353302.72fbc04566e78aa6732531296256a5aa.,
>
> because can't move them together after 842ms
>
> 2013-04-04 05:45:59,396 INFO [hbase-am-zkevent-worker-pool-2-thread-1]
> master.AssignmentManager$4(1164):
> The master has opened the region
> testCleanMergeReference,testRow0020,1365054353302.72fbc04566e78aa6732531296256a5aa.
> that was onlin
> e on quirinus.apache.org,45718,1365054345790
> {code}
> There's a small probability that fail to move merging regions together to
> same regionserver
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6901) Store file compactSelection throws ArrayIndexOutOfBoundsException
[ https://issues.apache.org/jira/browse/HBASE-6901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-6901: - Fix Version/s: (was: 0.95.0) 0.94.2 Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by Lars Hofhansl) > Store file compactSelection throws ArrayIndexOutOfBoundsException > - > > Key: HBASE-6901 > URL: https://issues.apache.org/jira/browse/HBASE-6901 > Project: HBase > Issue Type: Bug > Components: HFile >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang > Fix For: 0.94.2 > > Attachments: trunk-6901.patch > > > When setting to true, > and run compaction to exclude bulk loaded files could cause > ArrayIndexOutOfBoundsException since all files are excluded. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6900) RegionScanner.reseek() creates NPE when a flush or compaction happens before the reseek.
[
https://issues.apache.org/jira/browse/HBASE-6900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-6900:
-
Fix Version/s: (was: 0.95.0)
0.94.2
Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by
Lars Hofhansl)
> RegionScanner.reseek() creates NPE when a flush or compaction happens before
> the reseek.
>
>
> Key: HBASE-6900
> URL: https://issues.apache.org/jira/browse/HBASE-6900
> Project: HBase
> Issue Type: Bug
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
> Fix For: 0.94.2
>
> Attachments: 6900-test.txt, HBASE-6900_1.patch, HBASE-6900.patch
>
>
> HBASE-5520 introduced reseek() on the RegionScanner.
> Now when a scanner is created we have the StoreScanner heap. After this if a
> flush or compaction happens parallely all the StoreScannerObservers are
> cleared so that whenever a new next() call happens we tend to recreate the
> scanner based on the latest store files.
> The reseek() in StoreScanner expects the heap not to be null because always
> reseek would be called from next()
> {code}
> public synchronized boolean reseek(KeyValue kv) throws IOException {
> //Heap cannot be null, because this is only called from next() which
> //guarantees that heap will never be null before this call.
> if (explicitColumnQuery && lazySeekEnabledGlobally) {
> return heap.requestSeek(kv, true, useRowColBloom);
> } else {
> return heap.reseek(kv);
> }
> }
> {code}
> Now when we call RegionScanner.reseek() directly using CPs we tend to get a
> NPE. In our case it happened when a major compaction was going on. I will
> also attach a testcase to show the problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6889) Ignore source control files with apache-rat
[ https://issues.apache.org/jira/browse/HBASE-6889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-6889: - Fix Version/s: (was: 0.95.0) 0.94.2 Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by Lars Hofhansl) > Ignore source control files with apache-rat > --- > > Key: HBASE-6889 > URL: https://issues.apache.org/jira/browse/HBASE-6889 > Project: HBase > Issue Type: Bug >Reporter: Jesse Yates >Assignee: Jesse Yates > Fix For: 0.94.2 > > Attachments: hbase-6889-mvn-v0.patch > > > Running 'mvn apache-rat:check' locally causes a failure because it finds the > source control files, making it hard to check that you didn't include a file > without a source header. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6871) HFileBlockIndex Write Error in HFile V2 due to incorrect split into intermediate index blocks
[ https://issues.apache.org/jira/browse/HBASE-6871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-6871: - Fix Version/s: (was: 0.95.0) 0.94.2 Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by Lars Hofhansl) > HFileBlockIndex Write Error in HFile V2 due to incorrect split into > intermediate index blocks > - > > Key: HBASE-6871 > URL: https://issues.apache.org/jira/browse/HBASE-6871 > Project: HBase > Issue Type: Bug > Components: HFile >Affects Versions: 0.94.1 > Environment: redhat 5u4 >Reporter: Feng Wang >Assignee: Mikhail Bautin >Priority: Critical > Fix For: 0.94.2 > > Attachments: 428a400628ae412ca45d39fce15241fd.hfile, > 6871.094.addendum2.txt, 6871.094.addendum.txt, 6871-0.94.txt, > 6871-0.94v2.txt, 6871-hfile-index-0.92.txt, 6871-hfile-index-0.92-v2.txt, > 6871.txt, 6871v2.txt, 787179746cc347ce9bb36f1989d17419.hfile, > 960a026ca370464f84903ea58114bc75.hfile, > d0026fa8d59b4df291718f59dd145aad.hfile, D5703.1.patch, D5703.2.patch, > D5703.3.patch, D5703.4.patch, D5703.5.patch, hbase-6871-0.94.patch, > ImportHFile.java, test_hfile_block_index.sh > > > After writing some data, compaction and scan operation both failure, the > exception message is below: > 2012-09-18 06:32:26,227 ERROR > org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest: > Compaction failed > regionName=hfile_test,,1347778722498.d220df43fb9d8af4633bd7f547613f9e., > storeName=page_info, fileCount=7, fileSize=1.3m (188.0k, 188.0k, 188.0k, > 188.0k, 188.0k, 185.8k, 223.3k), priority=9, > time=45826250816757428java.io.IOException: Could not reseek > StoreFileScanner[HFileScanner for reader > reader=hdfs://hadoopdev1.cm6:9000/hbase/hfile_test/d220df43fb9d8af4633bd7f547613f9e/page_info/b0f6118f58de47ad9d87cac438ee0895, > compression=lzo, cacheConf=CacheConfig:enabled [cacheDataOnRead=true] > [cacheDataOnWrite=false] [cacheIndexesOnWrite=false] > [cacheBloomsOnWrite=false] [cacheEvictOnClose=false] [cacheCompressed=false], > firstKey=http://com.truereligionbrandjeans.www/Womens_Dresses/pl/c/Womens_Shirts/pl/c/Womens_Shirts/pl/c/Womens_Shirts/pl/c/Womens_Shirts/pl/c/Womens_Shirts/pl/c/Womens_Shirts/pl/c/Womens_Shirts/pl/c/Womens_Shirts/pl/c/Womens_Shirts/pl/c/Womens_Shirts/pl/c/Womens_Shirts/pl/c/Womens_Shirts/pl/c/Womens_Shirts/pl/c/Womens_Shirts/pl/c/Womens_Shirts/pl/c/Womens_Sweaters/pl/c/Womens_Sweaters/pl/c/Womens_Sweaters/pl/c/Womens_Sweaters/pl/c/Womens_Sweaters/pl/c/Womens_Sweaters/pl/c/Womens_Shirts/pl/c/Womens_Sweaters/pl/c/Womens_Shirts/pl/c/Womens_Shirts/pl/c/Womens_Shirts/pl/c/Womens_Sweaters/pl/c/Womens_Sweaters/pl/c/Womens_Shirts/pl/c/Womens_Shirts/pl/c/Womens_Sweaters/pl/c/Womens_Sweaters/pl/c/Womens_Sweaters/pl/c/Womens_Sweaters/pl/c/Womens_Sweaters/pl/c/4010.html/page_info:anchor_sig/1347764439449/DeleteColumn, > lastKey=http://com.trura.www//page_info:page_type/1347763395089/Put, > avgKeyLen=776, avgValueLen=4, entries=12853, length=228611, > cur=http://com.truereligionbrandjeans.www/Womens_Exclusive_Details/pl/c/4970.html/page_info:is_deleted/1347764003865/Put/vlen=1/ts=0] > to key > http://com.truereligionbrandjeans.www/Womens_Exclusive_Details/pl/c/4970.html/page_info:is_deleted/OLDEST_TIMESTAMP/Minimum/vlen=0/ts=0 > at > org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseek(StoreFileScanner.java:178) > > at > org.apache.hadoop.hbase.regionserver.NonLazyKeyValueScanner.doRealSeek(NonLazyKeyValueScanner.java:54) > > at > org.apache.hadoop.hbase.regionserver.KeyValueHeap.generalizedSeek(KeyValueHeap.java:299) > at > org.apache.hadoop.hbase.regionserver.KeyValueHeap.reseek(KeyValueHeap.java:244) > > at > org.apache.hadoop.hbase.regionserver.StoreScanner.reseek(StoreScanner.java:521) > > at > org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:402) > at > org.apache.hadoop.hbase.regionserver.Store.compactStore(Store.java:1570) > > at org.apache.hadoop.hbase.regionserver.Store.compact(Store.java:997) > > at > org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:1216) > at > org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest.run(CompactionRequest.java:250) > > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > at java.lang.Thread.run(Thread.java:662) > Caused by: java.io.IOException: Expected block type LEAF_INDEX, but got > INTERMEDIATE_INDEX: blockTyp
[jira] [Updated] (HBASE-6888) HBase scripts ignore any HBASE_OPTS set in the environment
[ https://issues.apache.org/jira/browse/HBASE-6888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-6888: - Fix Version/s: (was: 0.95.0) 0.94.2 Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by Lars Hofhansl) > HBase scripts ignore any HBASE_OPTS set in the environment > -- > > Key: HBASE-6888 > URL: https://issues.apache.org/jira/browse/HBASE-6888 > Project: HBase > Issue Type: Bug > Components: scripts >Affects Versions: 0.94.0, 0.95.2 >Reporter: Aditya Kishore >Assignee: Aditya Kishore >Priority: Minor > Fix For: 0.94.2 > > Attachments: HBASE-6888_trunk.patch > > > hbase-env.sh which is sourced by hbase-config.sh which is eventually sourced > by the main 'hbase' script defines HBASE_OPTS form scratch, ignoring any > previous value set in the environment. > This prevents from passing additional JVM parameters to HBase programs > (shell, hbck, etc) launched through these scripts. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6868) Skip checksum is broke; are we double-checksumming by default?
[ https://issues.apache.org/jira/browse/HBASE-6868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-6868: - Fix Version/s: (was: 0.95.0) 0.94.2 Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by Lars Hofhansl) > Skip checksum is broke; are we double-checksumming by default? > -- > > Key: HBASE-6868 > URL: https://issues.apache.org/jira/browse/HBASE-6868 > Project: HBase > Issue Type: Bug > Components: HFile, wal >Affects Versions: 0.94.0, 0.94.1 >Reporter: LiuLei >Assignee: Lars Hofhansl >Priority: Blocker > Fix For: 0.94.2 > > Attachments: 6868-0.94.txt, 6868-0.96-idea.txt, 6868-0.96-v2.txt, > 6868-0.96-v3.txt > > > The HFile contains checksums for decrease the iops, so when Hbase read HFile > , that dont't need to read the checksum from meta file of HDFS. But HLog > file of Hbase don't contain the checksum, so when HBase read the HLog, that > must read checksum from meta file of HDFS. We could add setSkipChecksum per > file to hdfs or we could write checksums into WAL if this skip checksum > facility is enabled -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6851) Race condition in TableAuthManager.updateGlobalCache()
[
https://issues.apache.org/jira/browse/HBASE-6851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-6851:
-
Fix Version/s: (was: 0.95.0)
0.94.2
Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by
Lars Hofhansl)
> Race condition in TableAuthManager.updateGlobalCache()
> --
>
> Key: HBASE-6851
> URL: https://issues.apache.org/jira/browse/HBASE-6851
> Project: HBase
> Issue Type: Bug
> Components: security
>Affects Versions: 0.94.1, 0.95.2
>Reporter: Gary Helmling
>Assignee: Gary Helmling
>Priority: Critical
> Fix For: 0.94.2
>
> Attachments: HBASE-6851_2.patch, HBASE-6851_3.patch, HBASE-6851.patch
>
>
> When new global permissions are assigned, there is a race condition, during
> which further authorization checks relying on global permissions may fail.
> In TableAuthManager.updateGlobalCache(), we have:
> {code:java}
> USER_CACHE.clear();
> GROUP_CACHE.clear();
> try {
> initGlobal(conf);
> } catch (IOException e) {
> // Never happens
> LOG.error("Error occured while updating the user cache", e);
> }
> for (Map.Entry entry : userPerms.entries()) {
> if (AccessControlLists.isGroupPrincipal(entry.getKey())) {
> GROUP_CACHE.put(AccessControlLists.getGroupName(entry.getKey()),
> new Permission(entry.getValue().getActions()));
> } else {
> USER_CACHE.put(entry.getKey(), new
> Permission(entry.getValue().getActions()));
> }
> }
> {code}
> If authorization checks come in following the .clear() but before
> repopulating, they will fail.
> We should have some synchronization here to serialize multiple updates and
> use a COW type rebuild and reassign of the new maps.
> This particular issue crept in with the fix in HBASE-6157, so I'm flagging
> for 0.94 and 0.96.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6853) IllegalArgumentException is thrown when an empty region is splitted
[
https://issues.apache.org/jira/browse/HBASE-6853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-6853:
-
Fix Version/s: (was: 0.95.0)
0.94.2
Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by
Lars Hofhansl)
> IllegalArgumentException is thrown when an empty region is splitted
> ---
>
> Key: HBASE-6853
> URL: https://issues.apache.org/jira/browse/HBASE-6853
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.92.1, 0.94.1
>Reporter: ramkrishna.s.vasudevan
>Assignee: Priyadarshini
> Fix For: 0.94.2
>
> Attachments: HBASE-6853_0.94, HBASE-6853_2_splitsuccess.patch,
> HBASE-6853_addendum.patch, HBASE-6853.patch, HBASE-6853_splitfailure.patch
>
>
> This is w.r.t a mail sent in the dev mail list.
> Empty region split should be handled gracefully. Either we should not allow
> the split to happen if we know that the region is empty or we should allow
> the split to happen by setting the no of threads to the thread pool executor
> as 1.
> {code}
> int nbFiles = hstoreFilesToSplit.size();
> ThreadFactoryBuilder builder = new ThreadFactoryBuilder();
> builder.setNameFormat("StoreFileSplitter-%1$d");
> ThreadFactory factory = builder.build();
> ThreadPoolExecutor threadPool =
> (ThreadPoolExecutor) Executors.newFixedThreadPool(nbFiles, factory);
> List> futures = new ArrayList>(nbFiles);
> {code}
> Here the nbFiles needs to be a non zero positive value.
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6847) HBASE-6649 broke replication
[
https://issues.apache.org/jira/browse/HBASE-6847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-6847:
-
Fix Version/s: (was: 0.95.0)
0.94.2
Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by
Lars Hofhansl)
> HBASE-6649 broke replication
>
>
> Key: HBASE-6847
> URL: https://issues.apache.org/jira/browse/HBASE-6847
> Project: HBase
> Issue Type: Bug
>Reporter: Jean-Daniel Cryans
>Assignee: Devaraj Das
>Priority: Blocker
> Fix For: 0.94.2
>
> Attachments: HBASE-6847-0.94.patch, HBASE-6847.patch
>
>
> After running with HBASE-6646 and replication enabled I encountered this:
> {noformat}
> 2012-09-17 20:04:08,111 DEBUG
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Opening
> log for replication va1r3s24%2C10304%2C1347911704238.1347911706318 at 78617132
> 2012-09-17 20:04:08,120 DEBUG
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Break on
> IOE:
> hdfs://va1r5s41:10101/va1-backup/.logs/va1r3s24,10304,1347911704238/va1r3s24%2C10304%2C1347911704238.1347911706318,
> entryStart=78641557, pos=78771200, end=78771200, edit=84
> 2012-09-17 20:04:08,120 DEBUG
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
> currentNbOperations:164529 and seenEntries:84 and size: 154068
> 2012-09-17 20:04:08,120 DEBUG
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
> Replicating 84
> 2012-09-17 20:04:08,146 INFO
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager:
> Going to report log #va1r3s24%2C10304%2C1347911704238.1347911706318 for
> position 78771200 in
> hdfs://va1r5s41:10101/va1-backup/.logs/va1r3s24,10304,1347911704238/va1r3s24%2C10304%2C1347911704238.1347911706318
> 2012-09-17 20:04:08,158 INFO
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager:
> Removing 0 logs in the list: []
> 2012-09-17 20:04:08,158 DEBUG
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
> Replicated in total: 93234
> 2012-09-17 20:04:08,158 DEBUG
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Opening
> log for replication va1r3s24%2C10304%2C1347911704238.1347911706318 at 78771200
> 2012-09-17 20:04:08,163 ERROR
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
> Unexpected exception in ReplicationSource,
> currentPath=hdfs://va1r5s41:10101/va1-backup/.logs/va1r3s24,10304,1347911704238/va1r3s24%2C10304%2C1347911704238.1347911706318
> java.lang.IndexOutOfBoundsException
> at java.io.DataInputStream.readFully(DataInputStream.java:175)
> at
> org.apache.hadoop.io.DataOutputBuffer$Buffer.write(DataOutputBuffer.java:63)
> at
> org.apache.hadoop.io.DataOutputBuffer.write(DataOutputBuffer.java:101)
> at
> org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:2001)
> at
> org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1901)
> at
> org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1947)
> at
> org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader.next(SequenceFileLogReader.java:235)
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.readAllEntriesToReplicateOrNextFile(ReplicationSource.java:394)
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.run(ReplicationSource.java:307)
> {noformat}
> There's something weird at the end of the file and it's killing replication.
> We used to just retry.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6842) the jar used in coprocessor is not deleted in local which will exhaust the space of /tmp
[
https://issues.apache.org/jira/browse/HBASE-6842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-6842:
-
Fix Version/s: (was: 0.95.0)
0.94.2
Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by
Lars Hofhansl)
> the jar used in coprocessor is not deleted in local which will exhaust the
> space of /tmp
> ---
>
> Key: HBASE-6842
> URL: https://issues.apache.org/jira/browse/HBASE-6842
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors
>Affects Versions: 0.94.1
>Reporter: Zhou wenjian
>Assignee: Zhou wenjian
>Priority: Critical
> Fix For: 0.94.2
>
> Attachments: HBASE-6842-trunk.patch
>
>
> FileSystem fs = path.getFileSystem(HBaseConfiguration.create());
> Path dst = new Path(System.getProperty("java.io.tmpdir") +
> java.io.File.separator +"." + pathPrefix +
> "." + className + "." + System.currentTimeMillis() + ".jar");
> fs.copyToLocalFile(path, dst);
> fs.deleteOnExit(dst);
> change to
> File tmpLocal = new File(dst.toString());
> tmpLocal.deleteOnExit();
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6839) Operations may be executed without holding rowLock
[ https://issues.apache.org/jira/browse/HBASE-6839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-6839: - Fix Version/s: (was: 0.95.0) 0.94.2 Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by Lars Hofhansl) > Operations may be executed without holding rowLock > -- > > Key: HBASE-6839 > URL: https://issues.apache.org/jira/browse/HBASE-6839 > Project: HBase > Issue Type: Bug > Components: regionserver >Reporter: chunhui shen >Assignee: chunhui shen >Priority: Critical > Fix For: 0.94.2 > > Attachments: HBASE-6839.patch > > > HRegion#internalObtainRowLock will return null if timed out, > but many place which call this method don't handle this case > The bad result is operation will be executed even if it havn't obtained the > row lock. Such as put、delete、increment。。。 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6803) script hbase should add JAVA_LIBRARY_PATH to LD_LIBRARY_PATH
[ https://issues.apache.org/jira/browse/HBASE-6803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-6803: - Fix Version/s: (was: 0.95.0) 0.94.2 Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by Lars Hofhansl) > script hbase should add JAVA_LIBRARY_PATH to LD_LIBRARY_PATH > > > Key: HBASE-6803 > URL: https://issues.apache.org/jira/browse/HBASE-6803 > Project: HBase > Issue Type: Bug > Components: shell >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang > Fix For: 0.94.2 > > Attachments: trunk-6803.patch > > > Snappy SO fails to load properly if LD_LIBRARY_PATH does not include the path > where snappy SO is. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6784) TestCoprocessorScanPolicy is sometimes flaky when run locally
[
https://issues.apache.org/jira/browse/HBASE-6784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-6784:
-
Fix Version/s: (was: 0.95.0)
0.94.2
Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by
Lars Hofhansl)
> TestCoprocessorScanPolicy is sometimes flaky when run locally
> -
>
> Key: HBASE-6784
> URL: https://issues.apache.org/jira/browse/HBASE-6784
> Project: HBase
> Issue Type: Bug
>Reporter: ramkrishna.s.vasudevan
>Assignee: Lars Hofhansl
>Priority: Minor
> Fix For: 0.94.2
>
> Attachments: 6784.txt
>
>
> The problem is not seen in jenkins build.
> When we run TestCoprocessorScanPolicy.testBaseCases locally or in our
> internal jenkins we tend to get random failures. The reason is the 2 puts
> that we do here is sometimes getting the same timestamp. This is leading to
> improper scan results as the version check tends to skip one of the row
> seeing the timestamp to be same. Marking this as minor. As we are trying to
> solve testcase related failures just raising this incase we need to resolve
> this also.
> For eg,
> Both the puts are getting the time
> {code}
> time 1347635287360
> time 1347635287360
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6769) HRS.multi eats NoSuchColumnFamilyException since HBASE-5021
[
https://issues.apache.org/jira/browse/HBASE-6769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-6769:
-
Fix Version/s: (was: 0.95.0)
0.94.2
Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by
Lars Hofhansl)
> HRS.multi eats NoSuchColumnFamilyException since HBASE-5021
> ---
>
> Key: HBASE-6769
> URL: https://issues.apache.org/jira/browse/HBASE-6769
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.94.0, 0.94.1
>Reporter: Jean-Daniel Cryans
>Assignee: Elliott Clark
>Priority: Critical
> Fix For: 0.94.2
>
> Attachments: HBASE-6769-0.94-0.patch, HBASE-6769-0.94-1.patch,
> HBASE-6769-0.patch
>
>
> I think this is a pretty major usability regression, since HBASE-5021 this is
> what you get in the client when using a wrong family:
> {noformat}
> 2012-09-11 09:45:29,634 WARN
> org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException: Failed 1
> action: DoNotRetryIOException: 1 time, servers with issues: sfor3s44:10304,
> at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1601)
> at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1377)
> at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:916)
> at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:772)
> at org.apache.hadoop.hbase.client.HTable.put(HTable.java:747)
> {noformat}
> Then you have to log on the server to understand what failed.
> Since everything is now a multi call, even single puts in the shell fail like
> this.
> This is present since 0.94.0
> Assigning to Elliott because he asked.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6757) Very inefficient behaviour of scan using FilterList
[ https://issues.apache.org/jira/browse/HBASE-6757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-6757: - Fix Version/s: (was: 0.95.0) 0.94.2 Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by Lars Hofhansl) > Very inefficient behaviour of scan using FilterList > --- > > Key: HBASE-6757 > URL: https://issues.apache.org/jira/browse/HBASE-6757 > Project: HBase > Issue Type: Bug > Components: Filters >Affects Versions: 0.90.6 >Reporter: Jerry Lam >Assignee: Lars Hofhansl > Fix For: 0.94.2 > > Attachments: 6757.txt, CopyOfTestColumnPrefixFilter.java, > DisplayFilter.java > > > The behaviour of scan is very inefficient when using with FilterList. > The FilterList rewrites the return code from NEXT_ROW to SKIP from a filter > if Operator.MUST_PASS_ALL is used. > This happens when using ColumnPrefixFilter. Even though the > ColumnPrefixFilter indicates to jump to NEXT_ROW because no further match can > be found, the scan continues to scan all versions of a column in that row and > all columns of that row because the ReturnCode from ColumnPrefixFilter has > been rewritten by the FilterList from NEXT_ROW to SKIP. > This is particularly inefficient when there are many versions in a column > because the check is performed on all versions of the column instead of just > by checking the qualifier of the column name. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6734) Code duplication in LoadIncrementalHFiles
[
https://issues.apache.org/jira/browse/HBASE-6734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-6734:
-
Description:
This was due to the merge of two Jiras:
{code}
if (queue.isEmpty()) {
LOG.warn("Bulk load operation did not find any files to load in " +
"directory " + hfofDir.toUri() + ". Does it contain files in " +
"subdirectories that correspond to column family names?");
return;
}
if (queue.isEmpty()) {
LOG.warn("Bulk load operation did not find any files to load in " +
"directory " + hfofDir.toUri() + ". Does it contain files in " +
"subdirectories that correspond to column family names?");
}
{code}
was:
This was due to the merge of two Jiras:
{code}
if (queue.isEmpty()) {
LOG.warn("Bulk load operation did not find any files to load in " +
"directory " + hfofDir.toUri() + ". Does it contain files in " +
"subdirectories that correspond to column family names?");
return;
}
if (queue.isEmpty()) {
LOG.warn("Bulk load operation did not find any files to load in " +
"directory " + hfofDir.toUri() + ". Does it contain files in " +
"subdirectories that correspond to column family names?");
}
{code}
Fix Version/s: (was: 0.95.0)
0.94.2
Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by
Lars Hofhansl)
> Code duplication in LoadIncrementalHFiles
> -
>
> Key: HBASE-6734
> URL: https://issues.apache.org/jira/browse/HBASE-6734
> Project: HBase
> Issue Type: Bug
> Components: mapreduce
>Affects Versions: 0.94.1
>Reporter: Richard Ding
>Assignee: Richard Ding
>Priority: Minor
> Fix For: 0.94.2
>
> Attachments: HBASE-6734.patch
>
>
> This was due to the merge of two Jiras:
> {code}
> if (queue.isEmpty()) {
> LOG.warn("Bulk load operation did not find any files to load in " +
> "directory " + hfofDir.toUri() + ". Does it contain files in " +
> "subdirectories that correspond to column family names?");
> return;
> }
> if (queue.isEmpty()) {
> LOG.warn("Bulk load operation did not find any files to load in " +
> "directory " + hfofDir.toUri() + ". Does it contain files in " +
> "subdirectories that correspond to column family names?");
> }
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6714) TestMultiSlaveReplication#testMultiSlaveReplication may fail
[ https://issues.apache.org/jira/browse/HBASE-6714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-6714: - Fix Version/s: (was: 0.95.0) 0.94.2 Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by Lars Hofhansl) > TestMultiSlaveReplication#testMultiSlaveReplication may fail > > > Key: HBASE-6714 > URL: https://issues.apache.org/jira/browse/HBASE-6714 > Project: HBase > Issue Type: Bug > Components: Replication, test >Affects Versions: 0.92.0, 0.94.0 >Reporter: Himanshu Vashishtha >Assignee: Himanshu Vashishtha >Priority: Minor > Fix For: 0.94.2 > > Attachments: HBase-6714-v1.patch > > > java.lang.AssertionError: expected:<1> but was:<0> > at org.junit.Assert.fail(Assert.java:93) > at org.junit.Assert.failNotEquals(Assert.java:647) > at org.junit.Assert.assertEquals(Assert.java:128) > at org.junit.Assert.assertEquals(Assert.java:472) > at org.junit.Assert.assertEquals(Assert.java:456) > at > org.apache.hadoop.hbase.replication.TestMultiSlaveReplication.checkRow(TestMultiSlaveReplication.java:203) > at > org.apache.hadoop.hbase.replication.TestMultiSlaveReplication.testMultiSlaveReplication(TestMultiSlaveReplication.java:188) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > TestMultiSlaveReplication->testMultiSlaveReplication failed in our local > build citing that "row" was not replicated to second peer. This is because > after inserting "row", log is rolled and we look for "row2" in both the > clusters and then we check for existence of "row" in both clusters. > Meanwhile, Replication thread was sleeping for the second cluster and Row > "row2" is not present in the second cluster from the very beginning. So, the > "row2" existence check succeeds and control move on to find "row" in both > clusters where it fails for the second cluster. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6711) Avoid local results copy in StoreScanner
[ https://issues.apache.org/jira/browse/HBASE-6711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-6711: - Fix Version/s: (was: 0.95.0) 0.94.2 Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by Lars Hofhansl) > Avoid local results copy in StoreScanner > > > Key: HBASE-6711 > URL: https://issues.apache.org/jira/browse/HBASE-6711 > Project: HBase > Issue Type: Bug >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl >Priority: Minor > Fix For: 0.94.2 > > Attachments: 6711-0.94-v1.txt, 6711-0.96-v1.txt > > > In StoreScanner the number of results is limited to avoid OOMs. > However, this is done by first adding the KV into a local ArrayList and then > copying the entries in this list to the final result list. > In turns out the this temporary list is only used to keep track of the size > of the result set in this loop. Can use a simple int instead. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6713) Stopping META/ROOT RS may take 50mins when some region is splitting
[ https://issues.apache.org/jira/browse/HBASE-6713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-6713: - Fix Version/s: (was: 0.95.0) 0.94.2 Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by Lars Hofhansl) > Stopping META/ROOT RS may take 50mins when some region is splitting > --- > > Key: HBASE-6713 > URL: https://issues.apache.org/jira/browse/HBASE-6713 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 0.94.1 >Reporter: chunhui shen >Assignee: chunhui shen > Fix For: 0.94.2 > > Attachments: 6713.92-94, 6713v3.patch, HBASE-6713.patch, > HBASE-6713v2.patch > > > When we stop the RS carrying ROOT/META, if it is in the splitting for some > region, the whole stopping process may take 50 mins. > The reason is : > 1.ROOT/META region is closed when stopping the regionserver > 2.The Split Transaction failed updating META and it will retry > 3.The retry num is 100, and the total time is about 50 mins as default; > This configuration is set by > HConnectionManager#setServerSideHConnectionRetries > I think 50 mins is too long to acceptable, my suggested solution is closing > MetaTable regions after the compact/split thread is closed -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6688) folder referred by thrift demo app instructions is outdated
[ https://issues.apache.org/jira/browse/HBASE-6688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-6688: - Fix Version/s: (was: 0.95.0) 0.94.2 Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by Lars Hofhansl) > folder referred by thrift demo app instructions is outdated > --- > > Key: HBASE-6688 > URL: https://issues.apache.org/jira/browse/HBASE-6688 > Project: HBase > Issue Type: Bug >Affects Versions: 0.95.2 >Reporter: Jimmy Xiang >Assignee: stack >Priority: Minor > Fix For: 0.94.2 > > Attachments: thrift094.txt, thrift.txt > > > Due to the source tree module change for 0.96, the instructions in the thrift > demo example don't match the folder structure any more. > In the instruction, it is referring to: > ../../../src/main/resources/org/apache/hadoop/hbase/thrift/Hbase.thrift > it should be > ../../hbase-server/src/main/resources/org/apache/hadoop/hbase/thrift/Hbase.thrift -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6686) HFile Quarantine fails with missing dirs in hadoop 2.0
[
https://issues.apache.org/jira/browse/HBASE-6686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-6686:
-
Fix Version/s: (was: 0.95.0)
0.94.2
Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by
Lars Hofhansl)
> HFile Quarantine fails with missing dirs in hadoop 2.0
> ---
>
> Key: HBASE-6686
> URL: https://issues.apache.org/jira/browse/HBASE-6686
> Project: HBase
> Issue Type: Bug
> Components: hbck
>Affects Versions: 0.92.2, 0.94.2, 0.95.2
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
> Fix For: 0.94.2
>
> Attachments: hbase-6686-94-92.patch
>
>
> Two unit tests fail because listStatus's semantics change between hadoop 1.0
> and hadoop 2.0. (specifically -- hadoop 1.0 returns empty array if used on
> dir that does not exist, but hadoop 2.0 throws FileNotFoundException).
> here's the exception:
> {code}
> 2012-08-28 16:01:19,789 WARN [Thread-3155] hbck.HFileCorruptionChecker(230):
> Failed to quaratine an HFile in regiondir
> hdfs://localhost:38096/user/jenkins/hbase/testQuarantineMissingFamdir/34b2e072b33052bf4875f85513e9c669
> java.io.FileNotFoundException: File
> hdfs://localhost:38096/user/jenkins/hbase/testQuarantineMissingFamdir/34b2e072b33052bf4875f85513e9c669/fam
> does not exist.
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:406)
> at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1341)
> at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1381)
> at
> org.apache.hadoop.hbase.util.hbck.HFileCorruptionChecker.checkColFamDir(HFileCorruptionChecker.java:152)
> at
> org.apache.hadoop.hbase.util.TestHBaseFsck$2$1.checkColFamDir(TestHBaseFsck.java:1401)
> at
> org.apache.hadoop.hbase.util.hbck.HFileCorruptionChecker.checkRegionDir(HFileCorruptionChecker.java:185)
> at
> org.apache.hadoop.hbase.util.hbck.HFileCorruptionChecker$RegionDirChecker.call(HFileCorruptionChecker.java:267)
> at
> org.apache.hadoop.hbase.util.hbck.HFileCorruptionChecker$RegionDirChecker.call(HFileCorruptionChecker.java:258)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:98)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:206)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6679) RegionServer aborts due to race between compaction and split
[ https://issues.apache.org/jira/browse/HBASE-6679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-6679: - Fix Version/s: (was: 0.95.0) 0.94.2 Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by Lars Hofhansl) > RegionServer aborts due to race between compaction and split > > > Key: HBASE-6679 > URL: https://issues.apache.org/jira/browse/HBASE-6679 > Project: HBase > Issue Type: Bug >Reporter: Devaraj Das >Assignee: Devaraj Das > Fix For: 0.94.2 > > Attachments: 6679-1.094.patch, 6679-1.patch, > rs-crash-parallel-compact-split.log > > > In our nightlies, we have seen RS aborts due to compaction and split racing. > Original parent file gets deleted after the compaction, and hence, the > daughters don't find the parent data file. The RS kills itself when this > happens. Will attach a snippet of the relevant RS logs. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6685) Thrift DemoClient.pl got NullPointerException
[ https://issues.apache.org/jira/browse/HBASE-6685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-6685: - Fix Version/s: (was: 0.95.0) 0.94.2 Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by Lars Hofhansl) > Thrift DemoClient.pl got NullPointerException > - > > Key: HBASE-6685 > URL: https://issues.apache.org/jira/browse/HBASE-6685 > Project: HBase > Issue Type: Bug > Components: Client, Thrift >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang >Priority: Minor > Fix For: 0.94.2 > > Attachments: trunk-6685.patch > > > expected error: TSocket: Could not read 4 bytes from localhost:9090 > 12/06/25 13:48:21 ERROR server.TThreadPoolServer: Error occurred during > processing of message. > java.lang.NullPointerException > at org.apache.hadoop.hbase.util.Bytes.getBytes(Bytes.java:765) > at > org.apache.hadoop.hbase.thrift.ThriftServer$HBaseHandler.mutateRowTs(ThriftServer.java:591) > > at > org.apache.hadoop.hbase.thrift.ThriftServer$HBaseHandler.mutateRow(ThriftServer.java:576) > > at > org.apache.hadoop.hbase.thrift.generated.Hbase$Processor$mutateRow.getResult(Hbase.java:3630) > > at > org.apache.hadoop.hbase.thrift.generated.Hbase$Processor$mutateRow.getResult(Hbase.java:3618) > > at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:32) > at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:34) > at > org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:176) > > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > > at java.lang.Thread.run(Thread.java:662) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6671) Kerberos authenticated super user should be able to retrieve proxied delegation tokens
[ https://issues.apache.org/jira/browse/HBASE-6671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-6671: - Fix Version/s: (was: 0.95.0) 0.94.2 Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by Lars Hofhansl) > Kerberos authenticated super user should be able to retrieve proxied > delegation tokens > -- > > Key: HBASE-6671 > URL: https://issues.apache.org/jira/browse/HBASE-6671 > Project: HBase > Issue Type: Bug >Affects Versions: 0.94.1 >Reporter: Francis Liu >Assignee: Francis Liu > Fix For: 0.94.2 > > Attachments: 6671-trunk-v2.txt, proxy_fix_94.patch, > proxy_fix_94.patch, proxy_fix_trunk.patch > > > There a services such a oozie which perform actions in behalf of the user > using proxy authentication. Retrieving delegation tokens should support this > behavior. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6649) [0.92 UNIT TESTS] TestReplication.queueFailover occasionally fails [Part-1]
[ https://issues.apache.org/jira/browse/HBASE-6649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-6649: - Fix Version/s: (was: 0.95.0) 0.94.2 Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by Lars Hofhansl) > [0.92 UNIT TESTS] TestReplication.queueFailover occasionally fails [Part-1] > --- > > Key: HBASE-6649 > URL: https://issues.apache.org/jira/browse/HBASE-6649 > Project: HBase > Issue Type: Bug >Reporter: Devaraj Das >Assignee: Devaraj Das >Priority: Blocker > Fix For: 0.94.2 > > Attachments: 6649-0.92.patch, 6649-1.patch, 6649-2.txt, > 6649-fix-io-exception-handling-1.patch, > 6649-fix-io-exception-handling-1-trunk.patch, > 6649-fix-io-exception-handling.patch, 6649-trunk.patch, 6649-trunk.patch, > 6649.txt, HBase-0.92 #495 test - queueFailover [Jenkins].html, HBase-0.92 > #502 test - queueFailover [Jenkins].html > > > Have seen it twice in the recent past: http://bit.ly/MPCykB & > http://bit.ly/O79Dq7 .. > Looking briefly at the logs hints at a pattern - in both the failed test > instances, there was an RS crash while the test was running. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6647) [performance regression] appendNoSync/HBASE-4528 doesn't take deferred log flush into account
[ https://issues.apache.org/jira/browse/HBASE-6647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-6647: - Fix Version/s: (was: 0.95.0) 0.94.2 Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by Lars Hofhansl) > [performance regression] appendNoSync/HBASE-4528 doesn't take deferred log > flush into account > - > > Key: HBASE-6647 > URL: https://issues.apache.org/jira/browse/HBASE-6647 > Project: HBase > Issue Type: Bug >Affects Versions: 0.94.0 >Reporter: Jean-Daniel Cryans >Assignee: Jean-Daniel Cryans > Fix For: 0.94.2 > > Attachments: HBASE-6647-0.94.patch, HBASE-6647-0.94-v2.patch, > HBASE-6647-0.94-v3.patch, HBASE-6647.pach > > > Since we upgraded to 0.94.1 from 0.92 I saw that our ICVs are about twice as > slow as they were. jstack'ing I saw that most of the time we are waiting on > sync()... but those tables have deferred log flush turned on so they > shouldn't even be calling it. > HTD.isDeferredLogFlush is currently only called in the append() methods which > are pretty much not in use anymore. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6608) Fix for HBASE-6160, META entries from daughters can be deleted before parent entries, shouldn't compare HRegionInfo's
[
https://issues.apache.org/jira/browse/HBASE-6608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-6608:
-
Fix Version/s: (was: 0.95.0)
0.94.2
Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by
Lars Hofhansl)
> Fix for HBASE-6160, META entries from daughters can be deleted before parent
> entries, shouldn't compare HRegionInfo's
> -
>
> Key: HBASE-6608
> URL: https://issues.apache.org/jira/browse/HBASE-6608
> Project: HBase
> Issue Type: Bug
> Components: Client, regionserver
>Affects Versions: 0.92.1, 0.94.2, 0.95.2
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 0.94.2
>
> Attachments: 6608-v2.patch, hbase-6608_v1-0.92+0.94.patch,
> hbase-6608_v1.patch
>
>
> Our nightlies discovered that the patch for HBASE-6160 did not actually fix
> the issue of "META entries from daughters can be deleted before parent
> entries". Instead of reopening the HBASE-6160, it is cleaner to track it
> here.
> The original issue is:
> {quote}
> HBASE-5986 fixed and issue, where the client sees the META entry for the
> parent, but not the children. However, after the fix, we have seen the
> following issue in tests:
> Region A is split to -> B, C
> Region B is split to -> D, E
> After some time, META entry for B is deleted since it is not needed anymore,
> but META entry for Region A stays in META (C still refers it). In this case,
> the client throws RegionOfflineException for B.
> {quote}
> The problem with the fix seems to be that we keep and compare HRegionInfo's
> in the HashSet at CatalogJanitor.java#scan(), but HRI that are compared are
> not equal.
> {code}
> HashSet parentNotCleaned = new HashSet(); //regions
> whose parents are still around
> for (Map.Entry e : splitParents.entrySet()) {
> if (!parentNotCleaned.contains(e.getKey()) && cleanParent(e.getKey(),
> e.getValue())) {
> cleaned++;
> } else {
> ...
> {code}
> In the above case, Meta row for region A will contain a serialized version of
> B that is not offline. However Meta row for region B will contain a
> serialized version of B that is offline (MetaEditor.offlineParentInMeta()
> does that). So the deserialized version we put to HashSet and the
> deserialized version we query contains() from HashSet are different in the
> offline field, thus HRI.equals() fail.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6621) Reduce calls to Bytes.toInt
[ https://issues.apache.org/jira/browse/HBASE-6621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-6621: - Fix Version/s: (was: 0.95.0) 0.94.2 Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by Lars Hofhansl) > Reduce calls to Bytes.toInt > --- > > Key: HBASE-6621 > URL: https://issues.apache.org/jira/browse/HBASE-6621 > Project: HBase > Issue Type: Bug >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl >Priority: Minor > Fix For: 0.94.2 > > Attachments: 6621-0.96.txt, 6621-0.96-v2.txt, 6621-0.96-v3.txt, > 6621-0.96-v4.txt > > > Bytes.toInt shows up quite often in a profiler run. > It turns out that one source is HFileReaderV2$ScannerV2.getKeyValue(). > Notice that we call the KeyValue(byte[], int) constructor, which forces the > constructor to determine its size by reading some of the header information > and calculate the size. In this case, however, we already know the size (from > the call to readKeyValueLen), so we could just use that. > In the extreme case of 1's of columns this noticeably reduces CPU. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6603) RegionMetricsStorage.incrNumericMetric is called too often
[ https://issues.apache.org/jira/browse/HBASE-6603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-6603: - Fix Version/s: (was: 0.95.0) 0.94.2 Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by Lars Hofhansl) > RegionMetricsStorage.incrNumericMetric is called too often > -- > > Key: HBASE-6603 > URL: https://issues.apache.org/jira/browse/HBASE-6603 > Project: HBase > Issue Type: Bug > Components: Performance >Reporter: Lars Hofhansl >Assignee: M. Chen > Fix For: 0.94.2 > > Attachments: 6503-0.96.txt, 6603-0.94.txt > > > Running an HBase scan load through the profiler revealed that > RegionMetricsStorage.incrNumericMetric is called way too often. > It turns out that we make this call for *each* KV in StoreScanner.next(...). > Incrementing AtomicLong requires expensive memory barriers. > The observation here is that StoreScanner.next(...) can maintain a simple > long in its internal loop and only update the metric upon exit. Thus the > AtomicLong is not updated nearly as often. > That cuts about 10% runtime from scan only load (I'll quantify this better > soon). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6587) Region would be assigned twice in the case of all RS offline
[
https://issues.apache.org/jira/browse/HBASE-6587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-6587:
-
Fix Version/s: (was: 0.95.0)
0.94.2
Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by
Lars Hofhansl)
> Region would be assigned twice in the case of all RS offline
>
>
> Key: HBASE-6587
> URL: https://issues.apache.org/jira/browse/HBASE-6587
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.94.1
>Reporter: chunhui shen
>Assignee: chunhui shen
> Fix For: 0.94.2
>
> Attachments: 6587-0.94.patch, 6587.patch, HBASE-6587.patch
>
>
> In the TimeoutMonitor, we would act on time out for the regions if
> (this.allRegionServersOffline && !noRSAvailable)
> The code is as the following:
> {code}
> if (regionState.getStamp() + timeout <= now ||
> (this.allRegionServersOffline && !noRSAvailable)) {
> //decide on action upon timeout or, if some RSs just came back
> online, we can start the
> // the assignment
> actOnTimeOut(regionState);
> }
> {code}
> But we found it exists a bug that it would act on time out for the region
> which was assigned just now , and cause assigning the region twice.
> Master log for the region 277b9b6df6de2b9be1353b4fa25f4222:
> {code}
> 2012-08-14 20:42:54,367 DEBUG
> org.apache.hadoop.hbase.master.AssignmentManager: Unable to determine a plan
> to assign .META.,,1.1028785192 state=OFFLINE, ts=1
> 344948174367, server=null
> 2012-08-14 20:44:31,640 DEBUG
> org.apache.hadoop.hbase.master.AssignmentManager: No previous transition plan
> was found (or we are ignoring an existing plan) for writete
> st,VHXYHJN0BL48HMR4DI1L,1344925649429.277b9b6df6de2b9be1353b4fa25f4222. so
> generated a random one;
> hri=writetest,VHXYHJN0BL48HMR4DI1L,1344925649429.277b9b6df6de2b9be13
> 53b4fa25f4222., src=, dest=dw92.kgb.sqa.cm4,60020,1344948267642; 1 (online=1,
> available=1) available servers
> 2012-08-14 20:44:31,640 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
> master:6-0x438f53bbf9b0acd Creating (or updating) unassigned node for
> 277b9b6df6de2b9be13
> 53b4fa25f4222 with OFFLINE state
> 2012-08-14 20:44:31,643 DEBUG
> org.apache.hadoop.hbase.master.AssignmentManager: Assigning region
> writetest,VHXYHJN0BL48HMR4DI1L,1344925649429.277b9b6df6de2b9be1353b4fa
> 25f4222. to dw92.kgb.sqa.cm4,60020,1344948267642
> 2012-08-14 20:44:32,291 DEBUG
> org.apache.hadoop.hbase.master.AssignmentManager: Handling
> transition=RS_ZK_REGION_OPENING, server=dw92.kgb.sqa.cm4,60020,1344948267642,
> region=277b9b6df6de2b9be1353b4fa25f4222
> // 异常的超时
> 2012-08-14 20:44:32,518 INFO
> org.apache.hadoop.hbase.master.AssignmentManager: Regions in transition timed
> out: writetest,VHXYHJN0BL48HMR4DI1L,1344925649429.277b9b6df
> 6de2b9be1353b4fa25f4222. state=OPENING, ts=1344948272279,
> server=dw92.kgb.sqa.cm4,60020,1344948267642
> 2012-08-14 20:44:32,518 INFO
> org.apache.hadoop.hbase.master.AssignmentManager: Region has been OPENING for
> too long, reassigning region=writetest,VHXYHJN0BL48HMR4DI1L,
> 1344925649429.277b9b6df6de2b9be1353b4fa25f4222.
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6579) Unnecessary KV order check in StoreScanner
[
https://issues.apache.org/jira/browse/HBASE-6579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-6579:
-
Fix Version/s: (was: 0.95.0)
0.94.2
Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by
Lars Hofhansl)
> Unnecessary KV order check in StoreScanner
> --
>
> Key: HBASE-6579
> URL: https://issues.apache.org/jira/browse/HBASE-6579
> Project: HBase
> Issue Type: Bug
>Reporter: Lars Hofhansl
>Assignee: Lars Hofhansl
>Priority: Minor
> Fix For: 0.94.2
>
> Attachments: 6579.txt
>
>
> In StoreScanner.next(List, int, String) I find this code:
> {code}
> // Check that the heap gives us KVs in an increasing order.
> if (prevKV != null && comparator != null
> && comparator.compare(prevKV, kv) > 0) {
> throw new IOException("Key " + prevKV + " followed by a " +
> "smaller key " + kv + " in cf " + store);
> }
> prevKV = kv;
> {code}
> So this checks for bugs in the HFiles or the scanner code. It needs to
> compare each KVs with its predecessor. This seems unnecessary now, I propose
> that we remove this.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6576) HBaseAdmin.createTable should wait until the table is enabled
[
https://issues.apache.org/jira/browse/HBASE-6576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-6576:
-
Fix Version/s: (was: 0.95.0)
0.94.2
Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by
Lars Hofhansl)
> HBaseAdmin.createTable should wait until the table is enabled
> -
>
> Key: HBASE-6576
> URL: https://issues.apache.org/jira/browse/HBASE-6576
> Project: HBase
> Issue Type: Bug
> Components: Client, test
>Reporter: Gregory Chanan
>Assignee: Gregory Chanan
> Fix For: 0.94.2
>
> Attachments: HBASE-6576-92.patch, HBASE-6576-94.patch,
> HBASE-6576-trunk.patch
>
>
> The function:
> {code}
> public void createTable(final HTableDescriptor desc, byte [][] splitKeys)
> {code}
> in HBaseAdmin is synchronous and returns once all the regions of the table
> are online, but does not wait for the table to be enabled, which is the last
> step of table creation (see CreateTableHandler).
> This is confusing and leads to racy code because users do not realize that
> this is the case. For example, I saw the following test failure in 0.92 when
> I ran:
> mvn test
> -Dtest=org.apache.hadoop.hbase.client.TestAdmin#testEnableDisableAddColumnDeleteColumn
>
> {code}
> Error Message
> org.apache.hadoop.hbase.TableNotEnabledException: testMasterAdmin at
> org.apache.hadoop.hbase.master.handler.DisableTableHandler.(DisableTableHandler.java:75)
> at org.apache.hadoop.hbase.master.HMaster.disableTable(HMaster.java:1154) at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597) at
> org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
> at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1336)
> Stacktrace
> org.apache.hadoop.hbase.TableNotEnabledException:
> org.apache.hadoop.hbase.TableNotEnabledException: testMasterAdmin
> at
> org.apache.hadoop.hbase.master.handler.DisableTableHandler.(DisableTableHandler.java:75)
> at org.apache.hadoop.hbase.master.HMaster.disableTable(HMaster.java:1154)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
> at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1336)
> {code}
> The issue is that code will create and table and immediately disable it in
> order to do some testing, for example, to test an operation that only works
> when the table is disabled. If the table has not been enabled yet, they will
> get back a TableNotEnabledException.
> The specific test above was fixed in HBASE-5206, but other examples exist in
> the code, for example the following:
> {code}
> hbase org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat newtable asdf14
> {code}
> The code in question is:
> {code}
> byte[] tname = args[1].getBytes();
> HTable table = util.createTable(tname, FAMILIES);
> HBaseAdmin admin = new HBaseAdmin(conf);
> admin.disableTable(tname);
> {code}
> It would be better if createTable just waited until the table was enabled, or
> threw a TableNotEnabledException if it exhausted the configured number of
> retries.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6565) Coprocessor exec result Map is not thread safe
[ https://issues.apache.org/jira/browse/HBASE-6565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-6565: - Fix Version/s: (was: 0.95.0) 0.94.2 Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by Lars Hofhansl) > Coprocessor exec result Map is not thread safe > -- > > Key: HBASE-6565 > URL: https://issues.apache.org/jira/browse/HBASE-6565 > Project: HBase > Issue Type: Bug > Components: Client, Coprocessors >Affects Versions: 0.92.2, 0.94.0, 0.95.2 > Environment: hadoop1.0.2,hbase0.94,jdk1.6 >Reporter: Yuan Kang >Assignee: Yuan Kang > Labels: coprocessors, patch > Fix For: 0.94.2 > > Attachments: Coprocessor-result-thread unsafe-bug-fix.patch > > Original Estimate: 168h > Remaining Estimate: 168h > > I develop a coprocessor program ,but found some different results in repeated > tests.for example,normally,the result's size is 10.but sometimes it appears 9. > I read the HTable.java code,found a TreeMap(thread-unsafe) be used in > multithreading environment.It cause the bug happened -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6561) Gets/Puts with many columns send the RegionServer into an "endless" loop
[ https://issues.apache.org/jira/browse/HBASE-6561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-6561: - Fix Version/s: (was: 0.95.0) 0.94.2 Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by Lars Hofhansl) > Gets/Puts with many columns send the RegionServer into an "endless" loop > > > Key: HBASE-6561 > URL: https://issues.apache.org/jira/browse/HBASE-6561 > Project: HBase > Issue Type: Bug >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl > Fix For: 0.94.2 > > Attachments: 6561-0.94.txt, 6561-0.96.txt, 6561-0.96-v2.txt, > 6561-0.96-v3.txt, 6561-0.96-v4.txt, 6561-0.96-v4.txt, 6561-0.96-v5.txt > > > This came from the mailing this: > We were able to replicate this behavior in a pseudo-distributed hbase > (hbase-0.94.1) environment. We wrote a test program that creates a test > table "MyTestTable" and populates it with random rows, then it creates a > row with 60,000 columns and repeatedly updates it. Each column has a 18 > byte qualifier and a 50 byte value. In our tests, when we ran the > program, we usually never got beyond 15 updates before it would flush > for a really long time. The rows that are being updated are about 4MB > each (minues any hbase metadata). > It doesn't seem like it's caused by GC. I turned on gc logging, and > didn't see any long pauses. This is the gc log during the flush. > http://pastebin.com/vJKKXDx5 > This is the regionserver log with debug on during the same flush > http://pastebin.com/Fh5213mg > This is the test program we wrote. > http://pastebin.com/aZ0k5tx2 > You should be able to just compile it, and run it against a running > HBase cluster. > $ java TestTable > Carlos -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6552) TestAcidGuarantees system test should flush more aggressively
[ https://issues.apache.org/jira/browse/HBASE-6552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-6552: - Fix Version/s: (was: 0.95.0) 0.94.2 Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by Lars Hofhansl) > TestAcidGuarantees system test should flush more aggressively > - > > Key: HBASE-6552 > URL: https://issues.apache.org/jira/browse/HBASE-6552 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 0.92.2, 0.94.1, 0.95.2 >Reporter: Gregory Chanan >Assignee: Gregory Chanan > Fix For: 0.94.2 > > Attachments: HBASE-6552-94-92.patch, HBASE-6552-trunk.patch > > > HBASE-5887 allowed TestAcidGuarantees to be run as a system test by avoiding > the call to util.flush(). > It would be better to go through the HBaseAdmin interface to force flushes. > This would unify the code path between the unit test and the system test, as > well as forcing more frequent flushes, which have previously been the source > of ACID guarantee problems, e.g. HBASE-2856. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6529) With HFile v2, the region server will always perform an extra copy of source files
[
https://issues.apache.org/jira/browse/HBASE-6529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-6529:
-
Fix Version/s: (was: 0.95.0)
0.94.2
Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by
Lars Hofhansl)
> With HFile v2, the region server will always perform an extra copy of source
> files
> --
>
> Key: HBASE-6529
> URL: https://issues.apache.org/jira/browse/HBASE-6529
> Project: HBase
> Issue Type: Bug
> Components: Performance, regionserver
>Affects Versions: 0.94.0, 0.95.2
>Reporter: Jason Dai
>Assignee: Jie Huang
> Fix For: 0.94.2
>
> Attachments: hbase-6529.094.txt, hbase-6529.diff
>
>
> With HFile v2 implementation in HBase 0.94 & 0.96, the region server will use
> HFileSystem as its {color:blue}fs{color}. When it performs bulk load in
> Store.bulkLoadHFile(), it checks if its {color:blue}fs{color} is the same as
> {color:blue}srcFs{color}, which however will be DistributedFileSystem.
> Consequently, it will always perform an extra copy of source files.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6520) MSLab May cause the Bytes.toLong not work correctly for increment
[
https://issues.apache.org/jira/browse/HBASE-6520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-6520:
-
Fix Version/s: (was: 0.95.0)
0.94.2
Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by
Lars Hofhansl)
> MSLab May cause the Bytes.toLong not work correctly for increment
> -
>
> Key: HBASE-6520
> URL: https://issues.apache.org/jira/browse/HBASE-6520
> Project: HBase
> Issue Type: Bug
>Reporter: ShiXing
>Assignee: ShiXing
> Fix For: 0.94.2
>
> Attachments: HBASE-6520-0.94-v1.patch, HBASE-6520-trunk-v1.patch
>
>
> When use MemStoreLAB, the KeyValues will share the byte array allocated by
> the MemStoreLAB, all the KeyValues' "bytes" attributes are the same byte
> array. When use the functions such as Bytes.toLong(byte[] bytes, int offset):
> {code}
> public static long toLong(byte[] bytes, int offset) {
> return toLong(bytes, offset, SIZEOF_LONG);
> }
> public static long toLong(byte[] bytes, int offset, final int length) {
> if (length != SIZEOF_LONG || offset + length > bytes.length) {
> throw explainWrongLengthOrOffset(bytes, offset, length, SIZEOF_LONG);
> }
> long l = 0;
> for(int i = offset; i < offset + length; i++) {
> l <<= 8;
> l ^= bytes[i] & 0xFF;
> }
> return l;
> }
> {code}
> If we do not put a long value to the KeyValue, and read it as a long value in
> HRegion.increment(),the check
> {code}
> offset + length > bytes.length
> {code}
> will take no effects, because the bytes.length is not equal to
> keyLength+valueLength, indeed it is MemStoreLAB chunkSize which is default
> 2048 * 1024.
> I will paste the patch later.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6525) bin/replication/copy_tables_desc.rb references non-existent class
[ https://issues.apache.org/jira/browse/HBASE-6525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-6525: - Fix Version/s: (was: 0.95.0) 0.94.2 Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by Lars Hofhansl) > bin/replication/copy_tables_desc.rb references non-existent class > - > > Key: HBASE-6525 > URL: https://issues.apache.org/jira/browse/HBASE-6525 > Project: HBase > Issue Type: Bug > Components: scripts >Affects Versions: 0.94.0, 0.95.2 >Reporter: David S. Wang >Assignee: David S. Wang >Priority: Trivial > Labels: noob > Fix For: 0.94.2 > > Attachments: HBASE-6525.patch > > > $ hbase org.jruby.Main copy_tables_desc.rb > NameError: cannot load Java class > org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper > get_proxy_or_package_under_package at > org/jruby/javasupport/JavaUtilities.java:54 > method_missing at > file:/mnt/data/hbase/lib/jruby-complete-1.6.5.jar!/builtin/javasupport/java.rb:51 > (root) at copy_tables_desc.rb:35 > Removing the line that references the non-existent class seems to make the > script work without any visible side-effects. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6516) hbck cannot detect any IOException while ".tableinfo" file is missing
[ https://issues.apache.org/jira/browse/HBASE-6516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-6516: - Fix Version/s: (was: 0.95.0) 0.94.2 Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by Lars Hofhansl) > hbck cannot detect any IOException while ".tableinfo" file is missing > - > > Key: HBASE-6516 > URL: https://issues.apache.org/jira/browse/HBASE-6516 > Project: HBase > Issue Type: Bug > Components: hbck >Affects Versions: 0.94.0, 0.95.2 >Reporter: Jie Huang >Assignee: Jie Huang > Fix For: 0.94.2 > > Attachments: hbase-6516-94-v5.patch, hbase-6516.patch, > hbase-6516-v2.patch, hbase-6516-v3.patch, hbase-6516-v4.patch, > hbase-6516-v5a.patch, hbase-6516-v5.patch > > > HBaseFsck checks those missing .tableinfo files in loadHdfsRegionInfos() > function. However, no IoException will be catched while .tableinfo is > missing, since "FSTableDescriptors.getTableDescriptor" doesn't throw any > IoException. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6514) unknown metrics type: org.apache.hadoop.hbase.metrics.histogram.MetricsHistogram
[ https://issues.apache.org/jira/browse/HBASE-6514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-6514: - Fix Version/s: (was: 0.95.0) 0.94.2 Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by Lars Hofhansl) > unknown metrics type: > org.apache.hadoop.hbase.metrics.histogram.MetricsHistogram > > > Key: HBASE-6514 > URL: https://issues.apache.org/jira/browse/HBASE-6514 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 0.92.2, 0.94.0 > Environment: MacOS 10.8 > Oracle JDK 1.7 >Reporter: Archimedes Trajano >Assignee: Elliott Clark > Fix For: 0.94.2 > > Attachments: FrameworkTest.java, FrameworkTest.java, > HBASE-6514-94-0.patch, HBASE-6514-trunk-0.patch, out.txt > > > When trying to run a unit test that just starts up and shutdown the server > the following errors occur in System.out > 01:10:59,874 ERROR MetricsUtil:116 - unknown metrics type: > org.apache.hadoop.hbase.metrics.histogram.MetricsHistogram > 01:10:59,874 ERROR MetricsUtil:116 - unknown metrics type: > org.apache.hadoop.hbase.metrics.histogram.MetricsHistogram > 01:10:59,875 ERROR MetricsUtil:116 - unknown metrics type: > org.apache.hadoop.hbase.metrics.histogram.MetricsHistogram > 01:10:59,875 ERROR MetricsUtil:116 - unknown metrics type: > org.apache.hadoop.hbase.metrics.histogram.MetricsHistogram -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6504) Adding GC details prevents HBase from starting in non-distributed mode
[
https://issues.apache.org/jira/browse/HBASE-6504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-6504:
-
Fix Version/s: (was: 0.95.0)
0.94.2
Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by
Lars Hofhansl)
> Adding GC details prevents HBase from starting in non-distributed mode
> --
>
> Key: HBASE-6504
> URL: https://issues.apache.org/jira/browse/HBASE-6504
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.94.0
>Reporter: Benoit Sigoure
>Assignee: Michael Drzal
>Priority: Trivial
> Labels: noob
> Fix For: 0.94.2
>
> Attachments: HBASE-6504-output.txt, HBASE-6504.patch,
> HBASE-6504-v2.patch
>
>
> The {{conf/hbase-env.sh}} that ships with HBase contains a few commented out
> examples of variables that could be useful, such as adding
> {{-XX:+PrintGCDetails -XX:+PrintGCDateStamps}} to {{HBASE_OPTS}}. This has
> the annoying side effect that the JVM prints a summary of memory usage when
> it exits, and it does so on stdout:
> {code}
> $ ./bin/hbase org.apache.hadoop.hbase.util.HBaseConfTool
> hbase.cluster.distributed
> false
> Heap
> par new generation total 19136K, used 4908K [0x00073a20,
> 0x00073b6c, 0x00075186)
> eden space 17024K, 28% used [0x00073a20, 0x00073a6cb0a8,
> 0x00073b2a)
> from space 2112K, 0% used [0x00073b2a, 0x00073b2a,
> 0x00073b4b)
> to space 2112K, 0% used [0x00073b4b, 0x00073b4b,
> 0x00073b6c)
> concurrent mark-sweep generation total 63872K, used 0K [0x00075186,
> 0x0007556c, 0x0007f5a0)
> concurrent-mark-sweep perm gen total 21248K, used 6994K [0x0007f5a0,
> 0x0007f6ec, 0x0008)
> $ ./bin/hbase org.apache.hadoop.hbase.util.HBaseConfTool
> hbase.cluster.distributed >/dev/null
> (nothing printed)
> {code}
> And this confuses {{bin/start-hbase.sh}} when it does
> {{distMode=`$bin/hbase --config "$HBASE_CONF_DIR"
> org.apache.hadoop.hbase.util.HBaseConfTool hbase.cluster.distributed`}},
> because then the {{distMode}} variable is not just set to {{false}}, it also
> contains all this JVM spam.
> If you don't pay enough attention and realize that 3 processes are getting
> started (ZK, HM, RS) instead of just one (HM), then you end up with this
> confusing error message:
> {{Could not start ZK at requested port of 2181. ZK was started at port:
> 2182. Aborting as clients (e.g. shell) will not be able to find this ZK
> quorum.}}, which is even more puzzling because when you run {{netstat}} to
> see who owns that port, then you won't find any rogue process other than the
> one you just started.
> I'm wondering if the fix is not to just change the {{if [ "$distMode" ==
> 'false' ]}} to a {{switch $distMode case (false*)}} type of test, to work
> around this annoying JVM misfeature that pollutes stdout.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6512) Incorrect OfflineMetaRepair log class name
[ https://issues.apache.org/jira/browse/HBASE-6512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-6512: - Fix Version/s: (was: 0.95.0) 0.94.2 Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by Lars Hofhansl) > Incorrect OfflineMetaRepair log class name > -- > > Key: HBASE-6512 > URL: https://issues.apache.org/jira/browse/HBASE-6512 > Project: HBase > Issue Type: Bug > Components: hbck >Affects Versions: 0.94.0, 0.94.1, 0.94.2, 0.95.2 >Reporter: Liang Xie >Assignee: Liang Xie > Fix For: 0.94.2 > > Attachments: HBASE-6512.diff > > > At the beginning of OfflineMetaRepair.java, we can observe: > "private static final Log LOG = LogFactory.getLog(HBaseFsck.class.getName());" > It would be better change to : > "private static final Log LOG = > LogFactory.getLog(OfflineMetaRepair.class.getName());" -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6478) TestClassLoading.testClassLoadingFromLibDirInJar occasionally fails
[
https://issues.apache.org/jira/browse/HBASE-6478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-6478:
-
Fix Version/s: (was: 0.95.0)
0.94.2
Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by
Lars Hofhansl)
> TestClassLoading.testClassLoadingFromLibDirInJar occasionally fails
> ---
>
> Key: HBASE-6478
> URL: https://issues.apache.org/jira/browse/HBASE-6478
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 0.94.0
>Reporter: zhou wenjian
> Fix For: 0.94.2
>
> Attachments: HBASE-6478-trunk.patch, HBASE-6478-trunk-v2.patch,
> HBASE-6478-trunk-v3.patch, HBASE-6478-trunk-v4.patch
>
>
> When hudson runs for HBASE-6459, it encounters a failed testcase in
> org.apache.hadoop.hbase.coprocessor.TestClassLoading.testClassLoadingFromLibDirInJar.
> The link is
> https://builds.apache.org/job/PreCommit-HBASE-Build/2455/testReport/org.apache.hadoop.hbase.coprocessor/TestClassLoading/testClassLoadingFromLibDirInJar/
> I check the log, and find that the function waitTableAvailable will only
> check the meta table, when rs open the region and update the metalocation in
> meta, it may not be added to the onlineregions in rs.
> for (HRegion region:
> hbase.getRegionServer(0).getOnlineRegionsLocalContext()) {
> this Loop will ship, and found1 will be false altogether.
> that's why the testcase failed.
> So maybe we can hbave some strictly check when table is created
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6488) HBase wont run on IPv6 on OSes that use zone-indexes
[ https://issues.apache.org/jira/browse/HBASE-6488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-6488: - Fix Version/s: (was: 0.95.0) 0.94.2 Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by Lars Hofhansl) > HBase wont run on IPv6 on OSes that use zone-indexes > > > Key: HBASE-6488 > URL: https://issues.apache.org/jira/browse/HBASE-6488 > Project: HBase > Issue Type: Bug >Affects Versions: 0.94.0 >Reporter: ryan rawson >Assignee: ryan rawson > Fix For: 0.94.2 > > Attachments: HBASE-6488-trunk.txt, HBASE-6488.txt > > > In IPv6, an address may have a zone-index, which is specified with a percent, > eg: ...%0. This looks like a format string, and thus in a part of the code > which uses the hostname as a prefix to another string which is interpreted > with String.format, you end up with an exception: > 2012-07-31 18:21:39,848 FATAL org.apache.hadoop.hbase.master.HMaster: > Unhandled exception. Starting shutdown. > java.util.UnknownFormatConversionException: Conversion = '0' > at java.util.Formatter.checkText(Formatter.java:2503) > at java.util.Formatter.parse(Formatter.java:2467) > at java.util.Formatter.format(Formatter.java:2414) > at java.util.Formatter.format(Formatter.java:2367) > at java.lang.String.format(String.java:2769) > at > com.google.common.util.concurrent.ThreadFactoryBuilder.setNameFormat(ThreadFactoryBuilder.java:68) > at > org.apache.hadoop.hbase.executor.ExecutorService$Executor.(ExecutorService.java:299) > at > org.apache.hadoop.hbase.executor.ExecutorService.startExecutorService(ExecutorService.java:185) > at > org.apache.hadoop.hbase.executor.ExecutorService.startExecutorService(ExecutorService.java:227) > at > org.apache.hadoop.hbase.master.HMaster.startServiceThreads(HMaster.java:821) > at > org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:507) > at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:344) > at > org.apache.hadoop.hbase.master.HMasterCommandLine$LocalHMaster.run(HMasterCommandLine.java:220) > at java.lang.Thread.run(Thread.java:680) > 2012-07-31 18:21:39,908 INFO org.apache.hadoop.hbase.master.HMaster: Aborting -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6471) Performance regression caused by HBASE-4054
[ https://issues.apache.org/jira/browse/HBASE-6471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-6471: - Fix Version/s: (was: 0.95.0) 0.94.2 Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by Lars Hofhansl) > Performance regression caused by HBASE-4054 > --- > > Key: HBASE-6471 > URL: https://issues.apache.org/jira/browse/HBASE-6471 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 0.92.0 >Reporter: Lars George >Assignee: Jimmy Xiang >Priority: Critical > Fix For: 0.94.2 > > Attachments: trunk-6471.patch, trunk-6471.patch, trunk-6471_v2.patch > > > The patch in HBASE-4054 switches the PooledHTable to extend HTable as opposed > to implement HTableInterface. > Since HTable does not have an empty constructor, the patch added a call to > the super() constructor, which though does trigger the ZooKeeper and META > scan, causing a considerable delay. > With multiple threads using the pool in parallel, the first thread is holding > up all the subsequent ones, in effect it negates the whole reason we have a > HTable pool. > We should complete HBASE-5728, or alternatively add a protected, empty > constructor the HTable. I am +1 for the former. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6460) hbck "-repairHoles" usage inconsistent with "-fixHdfsOrphans"
[
https://issues.apache.org/jira/browse/HBASE-6460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-6460:
-
Fix Version/s: (was: 0.95.0)
0.94.2
Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by
Lars Hofhansl)
> hbck "-repairHoles" usage inconsistent with "-fixHdfsOrphans"
> -
>
> Key: HBASE-6460
> URL: https://issues.apache.org/jira/browse/HBASE-6460
> Project: HBase
> Issue Type: Bug
> Components: hbck
>Affects Versions: 0.94.0, 0.95.2
>Reporter: Jie Huang
>Assignee: Jie Huang
>Priority: Minor
> Fix For: 0.94.2
>
> Attachments: hbase-6460.patch
>
>
> According to the hbck's help info, shortcut - "-repairHoles" will enable
> "-fixHdfsOrphans" as below.
> {noformat}
> -repairHoles Shortcut for -fixAssignments -fixMeta -fixHdfsHoles
> -fixHdfsOrphans
> {noformat}
> However, in the implementation, the function "fsck.setFixHdfsOrphans(false);"
> is called in "-repairHoles". This is not consistent with the usage
> information.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6438) RegionAlreadyInTransitionException needs to give more info to avoid assignment inconsistencies
[ https://issues.apache.org/jira/browse/HBASE-6438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-6438: - Fix Version/s: (was: 0.95.0) 0.94.2 Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by Lars Hofhansl) > RegionAlreadyInTransitionException needs to give more info to avoid > assignment inconsistencies > -- > > Key: HBASE-6438 > URL: https://issues.apache.org/jira/browse/HBASE-6438 > Project: HBase > Issue Type: Bug >Reporter: ramkrishna.s.vasudevan >Assignee: rajeshbabu > Fix For: 0.94.2 > > Attachments: 6438-0.92.txt, 6438.addendum, 6438-addendum.94, > 6438-trunk_2.patch, HBASE-6438_2.patch, HBASE-6438_94_3.patch, > HBASE-6438_94_4.patch, HBASE-6438_94.patch, HBASE-6438-trunk_2.patch, > HBASE-6438_trunk.patch > > > Seeing some of the recent issues in region assignment, > RegionAlreadyInTransitionException is one reason after which the region > assignment may or may not happen(in the sense we need to wait for the TM to > assign). > In HBASE-6317 we got one problem due to RegionAlreadyInTransitionException on > master restart. > Consider the following case, due to some reason like master restart or > external assign call, we try to assign a region that is already getting > opened in a RS. > Now the next call to assign has already changed the state of the znode and so > the current assign that is going on the RS is affected and it fails. The > second assignment that started also fails getting RAITE exception. Finally > both assignments not carrying on. Idea is to find whether any such RAITE > exception can be retried or not. > Here again we have following cases like where > -> The znode is yet to transitioned from OFFLINE to OPENING in RS > -> RS may be in the step of openRegion. > -> RS may be trying to transition OPENING to OPENED. > -> RS is yet to add to online regions in the RS side. > Here in openRegion() and updateMeta() any failures we are moving the znode to > FAILED_OPEN. So in these cases getting an RAITE should be ok. But in other > cases the assignment is stopped. > The idea is to just add the current state of the region assignment in the RIT > map in the RS side and using that info we can determine whether the > assignment can be retried or not on getting an RAITE. > Considering the current work going on in AM, pls do share if this is needed > atleast in the 0.92/0.94 versions? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6437) Avoid admin.balance during master initialize
[ https://issues.apache.org/jira/browse/HBASE-6437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-6437: - Fix Version/s: (was: 0.95.0) 0.94.2 Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by Lars Hofhansl) > Avoid admin.balance during master initialize > > > Key: HBASE-6437 > URL: https://issues.apache.org/jira/browse/HBASE-6437 > Project: HBase > Issue Type: Bug >Reporter: ramkrishna.s.vasudevan >Assignee: rajeshbabu > Fix For: 0.94.2 > > Attachments: HBASE-6437_92.patch, HBASE-6437_92.patch, > HBASE-6437_94.patch, HBASE-6437_94.patch, HBASE-6437_trunk_2.patch, > HBASE-6437_trunk.patch, HBASE-6437_trunk.patch > > > In HBASE-5850 many of the admin operations have been blocked till the master > initializes. But the balancer is not. So this JIRA is to extend the > PleaseHoldException in case of admin.balance() call before master is > initialized. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6364) Powering down the server host holding the .META. table causes HBase Client to take excessively long to recover and connect to reassigned .META. table
[ https://issues.apache.org/jira/browse/HBASE-6364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-6364: - Fix Version/s: (was: 0.95.0) 0.94.2 Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by Lars Hofhansl) > Powering down the server host holding the .META. table causes HBase Client to > take excessively long to recover and connect to reassigned .META. table > - > > Key: HBASE-6364 > URL: https://issues.apache.org/jira/browse/HBASE-6364 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 0.90.6, 0.92.1, 0.94.0 >Reporter: Suraj Varma >Assignee: Nicolas Liochon > Labels: client > Fix For: 0.94.2 > > Attachments: 6364.94.v2.nolargetest.patch, > 6364.94.v2.nolargetest.security-addendum.patch, > 6364-host-serving-META.v1.patch, 6364.v11.nolargetest.patch, 6364.v1.patch, > 6364.v1.patch, 6364.v2.patch, 6364.v3.patch, 6364.v3.patch, 6364.v5.patch, > 6364.v5.withtests.patch, 6364.v6.patch, 6364.v6.withtests.patch, > 6364.v7.withtests.patch, 6364.v8.withtests.patch, 6364.v9.patch, > stacktrace.txt > > > When a server host with a Region Server holding the .META. table is powered > down on a live cluster, while the HBase cluster itself detects and reassigns > the .META. table, connected HBase Client's take an excessively long time to > detect this and re-discover the reassigned .META. > Workaround: Decrease the ipc.socket.timeout on HBase Client side to a low > value (default is 20s leading to 35 minute recovery time; we were able to get > acceptable results with 100ms getting a 3 minute recovery) > This was found during some hardware failure testing scenarios. > Test Case: > 1) Apply load via client app on HBase cluster for several minutes > 2) Power down the region server holding the .META. server (i.e. power off ... > and keep it off) > 3) Measure how long it takes for cluster to reassign META table and for > client threads to re-lookup and re-orient to the lesser cluster (minus the RS > and DN on that host). > Observation: > 1) Client threads spike up to maxThreads size ... and take over 35 mins to > recover (i.e. for the thread count to go back to normal) - no client calls > are serviced - they just back up on a synchronized method (see #2 below) > 2) All the client app threads queue up behind the > oahh.ipc.HBaseClient#setupIOStreams method http://tinyurl.com/7js53dj > After taking several thread dumps we found that the thread within this > synchronized method was blocked on NetUtils.connect(this.socket, > remoteId.getAddress(), getSocketTimeout(conf)); > The client thread that gets the synchronized lock would try to connect to the > dead RS (till socket times out after 20s), retries, and then the next thread > gets in and so forth in a serial manner. > Workaround: > --- > Default ipc.socket.timeout is set to 20s. We dropped this to a low number > (1000 ms, 100 ms, etc) on the client side hbase-site.xml. With this setting, > the client threads recovered in a couple of minutes by failing fast and > re-discovering the .META. table on a reassigned RS. > Assumption: This ipc.socket.timeout is only ever used during the initial > "HConnection" setup via the NetUtils.connect and should only ever be used > when connectivity to a region server is lost and needs to be re-established. > i.e it does not affect the normal "RPC" actiivity as this is just the connect > timeout. > During RS GC periods, any _new_ clients trying to connect will fail and will > require .META. table re-lookups. > This above timeout workaround is only for the HBase client side. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6432) HRegionServer doesn't properly set clusterId in conf
[ https://issues.apache.org/jira/browse/HBASE-6432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-6432: - Fix Version/s: (was: 0.95.0) 0.94.2 Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by Lars Hofhansl) > HRegionServer doesn't properly set clusterId in conf > > > Key: HBASE-6432 > URL: https://issues.apache.org/jira/browse/HBASE-6432 > Project: HBase > Issue Type: Bug >Affects Versions: 0.94.0, 0.95.2 >Reporter: Francis Liu >Assignee: Francis Liu > Fix For: 0.94.2 > > Attachments: HBASE-6432_94.patch, HBASE-6432.patch > > > ClusterId is normally set into the passed conf during instantiation of an > HTable class. In the case of a HRegionServer this is bypassed and set to > "default" since getMaster() since it uses HBaseRPC to create the proxy > directly and bypasses the class which retrieves and sets the correct > clusterId. > This becomes a problem with clients (ie within a coprocessor) using > delegation tokens for authentication. Since the token's service will be the > correct clusterId and while the TokenSelector is looking for one with service > "default". -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6359) KeyValue may return incorrect values after readFields()
[ https://issues.apache.org/jira/browse/HBASE-6359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-6359: - Fix Version/s: (was: 0.95.0) 0.94.2 Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by Lars Hofhansl) > KeyValue may return incorrect values after readFields() > --- > > Key: HBASE-6359 > URL: https://issues.apache.org/jira/browse/HBASE-6359 > Project: HBase > Issue Type: Bug >Reporter: Dave Revell >Assignee: Dave Revell > Labels: noob > Fix For: 0.94.2 > > Attachments: HBASE-6359-trunk-v1.diff > > > When the same KeyValue object is used multiple times for deserialization > using readFields, some methods may return incorrect values. Here is a > sequence of operations that will reproduce the problem: > # A KeyValue is created whose key has length 10. The private field keyLength > is initialized to 0. > # KeyValue.getKeyLength() is called. This reads the key length 10 from the > backing array and caches it in keyLength. > # KeyValue.readFields() is called to deserialize a new value. The keyLength > field is not cleared and keeps its value of 10, even though this value is > probably incorrect. > # If getKeyLength() is called, the value 10 will be returned. > For example, in a reducer with Iterable, all values after the first > one from the iterable are likely to return incorrect values from > getKeyLength(). > The solution is to clear all memoized values in KeyValue.readFields(). I'll > write a patch for this soon. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6321) ReplicationSource dies reading the peer's id
[
https://issues.apache.org/jira/browse/HBASE-6321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-6321:
-
Fix Version/s: (was: 0.95.0)
0.94.2
Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by
Lars Hofhansl)
> ReplicationSource dies reading the peer's id
>
>
> Key: HBASE-6321
> URL: https://issues.apache.org/jira/browse/HBASE-6321
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.92.1, 0.94.0
>Reporter: Jean-Daniel Cryans
>Assignee: Jean-Daniel Cryans
> Fix For: 0.94.2
>
> Attachments: HBASE-6321-0.92.patch, HBASE-6321-0.94.patch,
> HBASE-6321.patch
>
>
> This is what I saw:
> {noformat}
> 2012-07-01 05:04:01,638 ERROR
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Closing
> source 8 because an error occurred: Could not read peer's cluster id
> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode
> = Session expired for /va1-backup/hbaseid
> at
> org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
> at
> org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1021)
> at
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:154)
> at
> org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:259)
> at
> org.apache.hadoop.hbase.zookeeper.ClusterId.readClusterIdZNode(ClusterId.java:61)
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.run(ReplicationSource.java:253)
> {noformat}
> The session should just be reopened.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6299) RS starting region open while failing ack to HMaster.sendRegionOpen() causes inconsistency in HMaster's region state and a series of successive problems
[
https://issues.apache.org/jira/browse/HBASE-6299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-6299:
-
Fix Version/s: (was: 0.95.0)
0.94.2
Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by
Lars Hofhansl)
> RS starting region open while failing ack to HMaster.sendRegionOpen() causes
> inconsistency in HMaster's region state and a series of successive problems
>
>
> Key: HBASE-6299
> URL: https://issues.apache.org/jira/browse/HBASE-6299
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 0.90.6, 0.94.0
>Reporter: Maryann Xue
>Assignee: Maryann Xue
>Priority: Critical
> Fix For: 0.94.2
>
> Attachments: 6299v4.txt, 6299v4.txt, 6299v4.txt, HBASE-6299.patch,
> HBASE-6299-v2.patch, HBASE-6299-v3.patch
>
>
> 1. HMaster tries to assign a region to an RS.
> 2. HMaster creates a RegionState for this region and puts it into
> regionsInTransition.
> 3. In the first assign attempt, HMaster calls RS.openRegion(). The RS
> receives the open region request and starts to proceed, with success
> eventually. However, due to network problems, HMaster fails to receive the
> response for the openRegion() call, and the call times out.
> 4. HMaster attemps to assign for a second time, choosing another RS.
> 5. But since the HMaster's OpenedRegionHandler has been triggered by the
> region open of the previous RS, and the RegionState has already been removed
> from regionsInTransition, HMaster finds invalid and ignores the unassigned ZK
> node "RS_ZK_REGION_OPENING" updated by the second attempt.
> 6. The unassigned ZK node stays and a later unassign fails coz
> RS_ZK_REGION_CLOSING cannot be created.
> {code}
> 2012-06-29 07:03:38,870 DEBUG
> org.apache.hadoop.hbase.master.AssignmentManager: Using pre-existing plan for
> region
> CDR_STATS_TRAFFIC,13184390567|20120508|17||2|3|913,1337256975556.b713fd655fa02395496c5a6e39ddf568.;
>
> plan=hri=CDR_STATS_TRAFFIC,13184390567|20120508|17||2|3|913,1337256975556.b713fd655fa02395496c5a6e39ddf568.,
> src=swbss-hadoop-004,60020,1340890123243,
> dest=swbss-hadoop-006,60020,1340890678078
> 2012-06-29 07:03:38,870 DEBUG
> org.apache.hadoop.hbase.master.AssignmentManager: Assigning region
> CDR_STATS_TRAFFIC,13184390567|20120508|17||2|3|913,1337256975556.b713fd655fa02395496c5a6e39ddf568.
> to swbss-hadoop-006,60020,1340890678078
> 2012-06-29 07:03:38,870 DEBUG
> org.apache.hadoop.hbase.master.AssignmentManager: Handling
> transition=M_ZK_REGION_OFFLINE, server=swbss-hadoop-002:6,
> region=b713fd655fa02395496c5a6e39ddf568
> 2012-06-29 07:06:28,882 DEBUG
> org.apache.hadoop.hbase.master.AssignmentManager: Handling
> transition=RS_ZK_REGION_OPENING, server=swbss-hadoop-006,60020,1340890678078,
> region=b713fd655fa02395496c5a6e39ddf568
> 2012-06-29 07:06:32,291 DEBUG
> org.apache.hadoop.hbase.master.AssignmentManager: Handling
> transition=RS_ZK_REGION_OPENING, server=swbss-hadoop-006,60020,1340890678078,
> region=b713fd655fa02395496c5a6e39ddf568
> 2012-06-29 07:06:32,299 DEBUG
> org.apache.hadoop.hbase.master.AssignmentManager: Handling
> transition=RS_ZK_REGION_OPENED, server=swbss-hadoop-006,60020,1340890678078,
> region=b713fd655fa02395496c5a6e39ddf568
> 2012-06-29 07:06:32,299 DEBUG
> org.apache.hadoop.hbase.master.handler.OpenedRegionHandler: Handling OPENED
> event for
> CDR_STATS_TRAFFIC,13184390567|20120508|17||2|3|913,1337256975556.b713fd655fa02395496c5a6e39ddf568.
> from serverName=swbss-hadoop-006,60020,1340890678078, load=(requests=518945,
> regions=575, usedHeap=15282, maxHeap=31301); deleting unassigned node
> 2012-06-29 07:06:32,299 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
> master:6-0x2377fee2ae80007 Deleting existing unassigned node for
> b713fd655fa02395496c5a6e39ddf568 that is in expected state RS_ZK_REGION_OPENED
> 2012-06-29 07:06:32,301 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
> master:6-0x2377fee2ae80007 Successfully deleted unassigned node for
> region b713fd655fa02395496c5a6e39ddf568 in expected state RS_ZK_REGION_OPENED
> 2012-06-29 07:06:32,301 DEBUG
> org.apache.hadoop.hbase.master.handler.OpenedRegionHandler: The master has
> opened the region
> CDR_STATS_TRAFFIC,13184390567|20120508|17||2|3|913,1337256975556.b713fd655fa02395496c5a6e39ddf568.
> that was online on serverName=swbss-hadoop-006,60020,1340890678078,
> load=(requests=518945, regions=575, usedHeap=15282, maxHeap=31301)
> 2012-06-29 07:07:41,140 WARN
> org.apache.hadoop.hbase.master.AssignmentManager: Failed assignment of
> CDR_STATS_TRAFFIC,13184390567|20120508|17||2|3|913,1337256975556.b71
[jira] [Updated] (HBASE-6263) Use default mode for HBase Thrift gateway if not specified
[
https://issues.apache.org/jira/browse/HBASE-6263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-6263:
-
Fix Version/s: (was: 0.95.0)
0.94.2
Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by
Lars Hofhansl)
> Use default mode for HBase Thrift gateway if not specified
> --
>
> Key: HBASE-6263
> URL: https://issues.apache.org/jira/browse/HBASE-6263
> Project: HBase
> Issue Type: Bug
> Components: Thrift
>Affects Versions: 0.94.0, 0.95.2
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Minor
> Labels: noob
> Fix For: 0.94.2
>
> Attachments: HBASE-6263-0.94.patch, HBASE-6263.patch
>
>
> The Thrift gateway should start with a default mode if one is not selected.
> Currently, instead we see:
> {noformat}
> Exception in thread "main" java.lang.AssertionError: Exactly one option out
> of [-hsha, -nonblocking, -threadpool, -threadedselector] has to be specified
> at
> org.apache.hadoop.hbase.thrift.ThriftServerRunner$ImplType.setServerImpl(ThriftServerRunner.java:201)
> at
> org.apache.hadoop.hbase.thrift.ThriftServer.processOptions(ThriftServer.java:169)
> at
> org.apache.hadoop.hbase.thrift.ThriftServer.doMain(ThriftServer.java:85)
> at
> org.apache.hadoop.hbase.thrift.ThriftServer.main(ThriftServer.java:192)
> {noformat}
> See also BIGTOP-648.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6211) Put latencies in jmx
[ https://issues.apache.org/jira/browse/HBASE-6211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-6211: - Fix Version/s: (was: 0.95.0) 0.94.2 Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by Lars Hofhansl) > Put latencies in jmx > > > Key: HBASE-6211 > URL: https://issues.apache.org/jira/browse/HBASE-6211 > Project: HBase > Issue Type: Bug > Components: metrics, monitoring > Environment: RegionServerMetrics pushes latency histograms to hadoop > metrics, but they are not getting into jmx. >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 0.94.2 > > Attachments: 6211_092.txt, HBASE-6211-0.patch, HBASE-6211-1.patch, > HBASE-6211-2.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6165) Replication can overrun .META. scans on cluster re-start
[ https://issues.apache.org/jira/browse/HBASE-6165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-6165: - Fix Version/s: (was: 0.95.0) 0.94.2 Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by Lars Hofhansl) > Replication can overrun .META. scans on cluster re-start > > > Key: HBASE-6165 > URL: https://issues.apache.org/jira/browse/HBASE-6165 > Project: HBase > Issue Type: Bug >Reporter: Elliott Clark >Assignee: Himanshu Vashishtha > Fix For: 0.94.2 > > Attachments: 6165-v6.txt, HBase-6165-94-v1.patch, > HBase-6165-94-v2.patch, HBase-6165-v1.patch, HBase-6165-v2.patch, > HBase-6165-v3.patch, HBase-6165-v4.patch, HBase-6165-v5.patch > > > When restarting a large set of regions on a reasonably small cluster the > replication from another cluster tied up every xceiver meaning nothing could > be onlined. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5997) Fix concerns raised in HBASE-5922 related to HalfStoreFileReader
[ https://issues.apache.org/jira/browse/HBASE-5997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-5997: - Fix Version/s: (was: 0.95.0) 0.94.2 Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by Lars Hofhansl) > Fix concerns raised in HBASE-5922 related to HalfStoreFileReader > > > Key: HBASE-5997 > URL: https://issues.apache.org/jira/browse/HBASE-5997 > Project: HBase > Issue Type: Bug >Affects Versions: 0.90.6, 0.92.1, 0.94.0, 0.95.2 >Reporter: ramkrishna.s.vasudevan >Assignee: Anoop Sam John > Fix For: 0.94.2 > > Attachments: 5997v3_trunk.txt, 5997v3_trunk.txt, 5997v3_trunk.txt, > 5997v3_trunk.txt, HBASE-5997_0.94.patch, HBASE-5997_94 V2.patch, > HBASE-5997_94 V3.patch, Testcase.patch.txt > > > Pls refer to the comment > https://issues.apache.org/jira/browse/HBASE-5922?focusedCommentId=13269346&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13269346. > Raised this issue to solve that comment. Just incase we don't forget it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5549) Master can fail if ZooKeeper session expires
[ https://issues.apache.org/jira/browse/HBASE-5549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-5549: - Fix Version/s: (was: 0.95.0) 0.94.2 Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by Lars Hofhansl) > Master can fail if ZooKeeper session expires > > > Key: HBASE-5549 > URL: https://issues.apache.org/jira/browse/HBASE-5549 > Project: HBase > Issue Type: Bug > Components: master, Zookeeper >Affects Versions: 0.95.2 > Environment: all >Reporter: Nicolas Liochon >Assignee: Nicolas Liochon >Priority: Minor > Fix For: 0.94.2 > > Attachments: 5549_092.txt, 5549_094.txt, 5549.v10.patch, > 5549.v11.patch, 5549.v6.patch, 5549.v7.patch, 5549.v8.patch, 5549.v9.patch, > nochange.patch > > > There is a retry mechanism in RecoverableZooKeeper, but when the session > expires, the whole ZooKeeperWatcher is recreated, hence the retry mechanism > does not work in this case. This is why a sleep is needed in > TestZooKeeper#testMasterSessionExpired: we need to wait for ZooKeeperWatcher > to be recreated before using the connection. > This can happen in real life, it can happen when: > - master & zookeeper starts > - zookeeper connection is cut > - master enters the retry loop > - in the meantime the session expires > - the network comes back, the session is recreated > - the retries continues, but on the wrong object, hence fails. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5292) getsize per-CF metric incorrectly counts compaction related reads as well
[ https://issues.apache.org/jira/browse/HBASE-5292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-5292: - Fix Version/s: (was: 0.95.0) 0.94.2 Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by Lars Hofhansl) > getsize per-CF metric incorrectly counts compaction related reads as well > -- > > Key: HBASE-5292 > URL: https://issues.apache.org/jira/browse/HBASE-5292 > Project: HBase > Issue Type: Bug >Affects Versions: 0.89.20100924 >Reporter: Kannan Muthukkaruppan > Fix For: 0.94.2 > > Attachments: > 0001-jira-HBASE-5292-Prevent-counting-getSize-on-compacti.patch, > 5292-0.94.txt, ASF.LICENSE.NOT.GRANTED--D1527.1.patch, > ASF.LICENSE.NOT.GRANTED--D1527.2.patch, > ASF.LICENSE.NOT.GRANTED--D1527.3.patch, > ASF.LICENSE.NOT.GRANTED--D1527.4.patch, > ASF.LICENSE.NOT.GRANTED--D1617.1.patch, > jira-HBASE-5292-Prevent-counting-getSize-on-compacti-2012-03-09_13_26_52.patch > > > The per-CF "getsize" metric's intent was to track bytes returned (to HBase > clients) per-CF. [Note: We already have metrics to track # of HFileBlock's > read for compaction vs. non-compaction cases -- e.g., compactionblockreadcnt > vs. fsblockreadcnt.] > Currently, the "getsize" metric gets updated for both client initiated > Get/Scan operations as well for compaction related reads. The metric is > updated in StoreScanner.java:next() when the Scan query matcher returns an > INCLUDE* code via a: > HRegion.incrNumericMetric(this.metricNameGetsize, copyKv.getLength()); > We should not do the above in case of compactions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6496) Example ZK based scan policy
[ https://issues.apache.org/jira/browse/HBASE-6496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-6496: - Fix Version/s: (was: 0.95.0) 0.94.2 Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by Lars Hofhansl) > Example ZK based scan policy > > > Key: HBASE-6496 > URL: https://issues.apache.org/jira/browse/HBASE-6496 > Project: HBase > Issue Type: Sub-task >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl > Fix For: 0.94.2 > > Attachments: 6496-0.94.txt, 6496-0.96.txt, 6496-0.96-v2.txt, > 6496.txt, 6496-v2.txt > > > Provide an example of a RegionServer that listens to a ZK node to learn about > what set of KVs can safely be deleted during a compaction. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8286) Move site back up out of hbase-assembly; bad idea
[ https://issues.apache.org/jira/browse/HBASE-8286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-8286: - Status: Patch Available (was: Open) > Move site back up out of hbase-assembly; bad idea > - > > Key: HBASE-8286 > URL: https://issues.apache.org/jira/browse/HBASE-8286 > Project: HBase > Issue Type: Task >Reporter: stack >Assignee: stack > Attachments: 8286.txt > > > Redoing the packaging, I thought it smart putting it all under the new > hbase-assembly module. A few weeks later, it is not looking like a good idea > (Enis suggested moving the site was maybe 'odd'). Here are a few issues: > + The generated reports will have mention of hbase-assembly because that is > where they are being generated (Elliott pointed out the the scm links mention > hbase-assembly instead of trunk). > + Doug Meil wants to build and deploy the site but currently deploy is a bit > awkward to explain because site presumes it is top level so staging goes to > the wrong place and you have to come along and do fixup afterward; it > shouldn't be so hard. > A possible downside is that we will break Nicks site check added to hadoopqa > because site is an aggregating build plugin... but lets see. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
