[jira] [Updated] (HBASE-5677) The master never does balance because duplicate openhandled the one region
[ https://issues.apache.org/jira/browse/HBASE-5677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xufeng updated HBASE-5677: -- Attachment: Backport-HBASE-5454-to-92.patch Backport-HBASE-5454-to-90.patch Backport-HBASE-5454-to-92.patch(this patch for 92):All unit tests passed..and also verified this patch in real cluste Backport-HBASE-5454-to-90.patch(this patch for 90):Some test error,but those error also exist if no patched.and also verified this patch in real cluste pls review,thanks. > The master never does balance because duplicate openhandled the one region > -- > > Key: HBASE-5677 > URL: https://issues.apache.org/jira/browse/HBASE-5677 > Project: HBase > Issue Type: Bug >Affects Versions: 0.90.6 > Environment: 0.90 >Reporter: xufeng >Assignee: xufeng > Fix For: 0.90.7, 0.92.2 > > Attachments: 5677-proposal.txt, 5677-proposal.txt, > Backport-HBASE-5454-to-90.patch, Backport-HBASE-5454-to-92.patch, > HBASE-5677-90-v1.patch, surefire-report_no_patched_v1.html, > surefire-report_patched_v1.html > > > If region be assigned When the master is doing initialization(before do > processFailover),the region will be duplicate openhandled. > because the unassigned node in zookeeper will be handled again in > AssignmentManager#processFailover() > it cause the region in RIT,thus the master never does balance. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5677) The master never does balance because duplicate openhandled the one region
[ https://issues.apache.org/jira/browse/HBASE-5677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254572#comment-13254572 ] Hadoop QA commented on HBASE-5677: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12522752/Backport-HBASE-5454-to-92.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1536//console This message is automatically generated. > The master never does balance because duplicate openhandled the one region > -- > > Key: HBASE-5677 > URL: https://issues.apache.org/jira/browse/HBASE-5677 > Project: HBase > Issue Type: Bug >Affects Versions: 0.90.6 > Environment: 0.90 >Reporter: xufeng >Assignee: xufeng > Fix For: 0.90.7, 0.92.2 > > Attachments: 5677-proposal.txt, 5677-proposal.txt, > Backport-HBASE-5454-to-90.patch, Backport-HBASE-5454-to-92.patch, > HBASE-5677-90-v1.patch, surefire-report_no_patched_v1.html, > surefire-report_patched_v1.html > > > If region be assigned When the master is doing initialization(before do > processFailover),the region will be duplicate openhandled. > because the unassigned node in zookeeper will be handled again in > AssignmentManager#processFailover() > it cause the region in RIT,thus the master never does balance. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5654) [findbugs] Address dodgy bugs
[ https://issues.apache.org/jira/browse/HBASE-5654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254578#comment-13254578 ] Ashutosh Jindal commented on HBASE-5654: @Uma Thank you for your review. I will look into the remaining bugs and submit the updated patch . > [findbugs] Address dodgy bugs > - > > Key: HBASE-5654 > URL: https://issues.apache.org/jira/browse/HBASE-5654 > Project: HBase > Issue Type: Sub-task > Components: scripts >Affects Versions: 0.96.0 >Reporter: Jonathan Hsieh >Assignee: Ashutosh Jindal > Labels: patch > Fix For: 0.96.0 > > Attachments: Hbase-5654.patch > > > See > https://builds.apache.org/job/PreCommit-HBASE-Build/1313//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html#Warnings_STYLE > This may be broken down further. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5772) Unable to open the few links in http://hbase.apache.org/
[ https://issues.apache.org/jira/browse/HBASE-5772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254591#comment-13254591 ] Kiran BC commented on HBASE-5772: - Thanks a lot Stack. > Unable to open the few links in http://hbase.apache.org/ > > > Key: HBASE-5772 > URL: https://issues.apache.org/jira/browse/HBASE-5772 > Project: HBase > Issue Type: Bug > Components: documentation >Affects Versions: 0.94.0 >Reporter: Kiran BC >Assignee: stack > Fix For: 0.96.0 > > Attachments: 5772.txt > > > Few links in http://hbase.apache.org/ is not working. > For example, Ref Guide (multi-page) will actually link to > http://hbase.apache.org/book/book.html and if I try to open this, Page not > found error is coming. > If I add /book in the url, like http://hbase.apache.org/book/book/book.html, > it is taking me to the Apache HBase Reference Guide > I think the folder structure has been changed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3585) isLegalFamilyName() can throw ArrayOutOfBoundException
[
https://issues.apache.org/jira/browse/HBASE-3585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254596#comment-13254596
]
Uma Maheswara Rao G commented on HBASE-3585:
Currently in trunk, isLegalFamilyName used in below constructor and that is
deprecated constructor.
{code}
@Deprecated
public HColumnDescriptor(final byte[] familyName, final int minVersions,
final int maxVersions, final boolean keepDeletedCells,
final String compression, final boolean encodeOnDisk,
final String dataBlockEncoding, final boolean inMemory,
final boolean blockCacheEnabled, final int blocksize,
final int timeToLive, final String bloomFilter, final int scope) {
isLegalFamilyName(familyName);
{code}
And the usable constructor is making use of this deprecated constructors
internally. Also validated for familyName length in this usable constructor
already.
{code}
public HColumnDescriptor(final byte [] familyName) {
this (familyName == null || familyName.length <= 0?
HConstants.EMPTY_BYTE_ARRAY: familyName, DEFAULT_VERSIONS,
DEFAULT_COMPRESSION, DEFAULT_IN_MEMORY, DEFAULT_BLOCKCACHE,
DEFAULT_TTL, DEFAULT_BLOOMFILTER);
}
{code}
But isLegalFamilyName is marked as public and used only in HColumnDescriptor
class.
Anyway ths simple fix couldbe to move down that validation to last constructor,
where we are finally populating the values with setter metods.
{code}
public HColumnDescriptor(final byte[] familyName, final int minVersions,
final int maxVersions, final boolean keepDeletedCells,
final String compression, final boolean encodeOnDisk,
final String dataBlockEncoding, final boolean inMemory,
final boolean blockCacheEnabled, final int blocksize,
final int timeToLive, final String bloomFilter, final int scope) {
{code}
> isLegalFamilyName() can throw ArrayOutOfBoundException
> --
>
> Key: HBASE-3585
> URL: https://issues.apache.org/jira/browse/HBASE-3585
> Project: HBase
> Issue Type: Bug
> Components: client
>Affects Versions: 0.90.1
>Reporter: Prakash Khemani
>Priority: Minor
>
> org.apache.hadoop.hbase.HColumnDescriptor.isLegalFamilyName(byte[]) accesses
> byte[0] w/o first checking the array length.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5782) Not all the regions are getting assigned after the log splitting.
[ https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-5782: -- Priority: Blocker (was: Critical) > Not all the regions are getting assigned after the log splitting. > - > > Key: HBASE-5782 > URL: https://issues.apache.org/jira/browse/HBASE-5782 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.94.0 >Reporter: Gopinathan A >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 0.94.0 > > > Create a table with 1000 splits, after the region assignemnt, kill the > regionserver wich contains META table. > Here few regions are missing after the log splitting and region assigment. > HBCK report shows multiple region holes are got created. > Same scenario was verified mulitple times in 0.92.1, no issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5782) Not all the regions are getting assigned after the log splitting.
[ https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254646#comment-13254646 ] ramkrishna.s.vasudevan commented on HBASE-5782: --- Found the problem and working on soln. Infact this problem would have introduced data loss easily in simple RS going down scenario. > Not all the regions are getting assigned after the log splitting. > - > > Key: HBASE-5782 > URL: https://issues.apache.org/jira/browse/HBASE-5782 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.94.0 >Reporter: Gopinathan A >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 0.94.0 > > > Create a table with 1000 splits, after the region assignemnt, kill the > regionserver wich contains META table. > Here few regions are missing after the log splitting and region assigment. > HBCK report shows multiple region holes are got created. > Same scenario was verified mulitple times in 0.92.1, no issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5792) HLog Performance Evaluation Tool
[ https://issues.apache.org/jira/browse/HBASE-5792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254650#comment-13254650 ] Jonathan Hsieh commented on HBASE-5792: --- When I ran a previous version of this I believe it wanted a hbase cluster up, we should make sure this works without that. > HLog Performance Evaluation Tool > > > Key: HBASE-5792 > URL: https://issues.apache.org/jira/browse/HBASE-5792 > Project: HBase > Issue Type: Test > Components: wal >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi >Priority: Minor > Labels: performance, wal > Attachments: HBASE-5792-v0.patch, HBASE-5792-v1.patch > > > Related to HDFS-3280 and the HBase WAL slowdown on 0.23+ > It would be nice to have a simple tool like HFilePerformanceEvaluation, ... > to be able to check easily the HLog performance. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5792) HLog Performance Evaluation Tool
[ https://issues.apache.org/jira/browse/HBASE-5792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254654#comment-13254654 ] Matteo Bertozzi commented on HBASE-5792: @Jonathan the previous version had a HBaseAdmin call at startup because the tool contains also a Put test, but the hbase up si not needed. > HLog Performance Evaluation Tool > > > Key: HBASE-5792 > URL: https://issues.apache.org/jira/browse/HBASE-5792 > Project: HBase > Issue Type: Test > Components: wal >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi >Priority: Minor > Labels: performance, wal > Attachments: HBASE-5792-v0.patch, HBASE-5792-v1.patch > > > Related to HDFS-3280 and the HBase WAL slowdown on 0.23+ > It would be nice to have a simple tool like HFilePerformanceEvaluation, ... > to be able to check easily the HLog performance. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5782) Not all the regions are getting assigned after the log splitting.
[
https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254666#comment-13254666
]
ramkrishna.s.vasudevan commented on HBASE-5782:
---
The problem here is
{code}
HLogKey logKey = makeKey(encodedRegionName, tableName, seqNum, now, clusterId);
doWrite(info, logKey, edits, htd);
{code}
We form a key here which is unique based on the seqNum. Now as we do
appendNoSync() there is no sync called in append flow. But there is always the
LogSyncer thread which periodically syncs.
All the edits are added to
{code}
synchronized void append(Entry e) throws IOException {
pendingWrites.add(e);
}
{code}
Here the pendingWrites are obtained and then it is flushed one by one.
{code}
pending = logSyncerThread.getPendingWrites();
// First flush all the pending writes to HDFS. Then
// issue the sync to HDFS. If sync is successful, then update
// syncedTillHere to indicate that transactions till this
// number has been successfully synced.
logSyncerThread.hlogFlush(tempWriter, pending);
{code}
Here there is no synchronization and hence the pendingwrites are not in
sequence. Because it is not in sequence the recovered.edits file is also
created with no sequence in some cases.
{code}
if (key.getLogSeqNum() <= currentEditSeqId) {
LOG.debug(" key is " + key.getLogSeqNum()
+ "current edit is " +
currentEditSeqId);
skippedEdits++;
continue;
}
{code}
I just added the above log while replayRecoveredEdits and i can find
{code}
key is 175current edit is 176
key is 174current edit is 176
key is 186current edit is 187
key is 188current edit is 190
*** key is 189current edit is 190
key is 204current edit is 205
key is 206current edit is 207
key is 217current edit is 218
key is 216current edit is 218
key is 255current edit is 256
{code}
This leads to dataloss. Correct me if am wrong.
> Not all the regions are getting assigned after the log splitting.
> -
>
> Key: HBASE-5782
> URL: https://issues.apache.org/jira/browse/HBASE-5782
> Project: HBase
> Issue Type: Bug
> Components: wal
>Affects Versions: 0.94.0
>Reporter: Gopinathan A
>Assignee: ramkrishna.s.vasudevan
>Priority: Blocker
> Fix For: 0.94.0
>
>
> Create a table with 1000 splits, after the region assignemnt, kill the
> regionserver wich contains META table.
> Here few regions are missing after the log splitting and region assigment.
> HBCK report shows multiple region holes are got created.
> Same scenario was verified mulitple times in 0.92.1, no issues.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5782) Not all the regions are getting assigned after the log splitting.
[ https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254669#comment-13254669 ] ramkrishna.s.vasudevan commented on HBASE-5782: --- We can easily reproduce this problem. I think it applies to any edit that happens and if it is synced by the log syncer thread. So we must have some sync mechanism here to ensure this problem does not happen. Based on one of our requirements i was thinking that the log syncer thread should be configurable like either to use it or not use it. Please provide your suggestions. > Not all the regions are getting assigned after the log splitting. > - > > Key: HBASE-5782 > URL: https://issues.apache.org/jira/browse/HBASE-5782 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.94.0 >Reporter: Gopinathan A >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 0.94.0 > > > Create a table with 1000 splits, after the region assignemnt, kill the > regionserver wich contains META table. > Here few regions are missing after the log splitting and region assigment. > HBCK report shows multiple region holes are got created. > Same scenario was verified mulitple times in 0.92.1, no issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5654) [findbugs] Address dodgy bugs
[ https://issues.apache.org/jira/browse/HBASE-5654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254683#comment-13254683 ] Jonathan Hsieh commented on HBASE-5654: --- Ashutosh: Thanks for taking this on. I'm about half way through the excludes. Just starting the INT category. Questions: * why no fix?: WritableRcpEngine#getProxy * why no fix?: Ambiguous invocations in RounRobinPool/ReusablePool (specify scope of get() by changing to this.get() or ReusablePool.this.get()) * add bounds checks?Result of integer multiplication cast to long in org.apache.hadoop.hbase.io.hfile.LruBlockCache.calculateOverhead(long, long, int) Result of integer multiplication cast to long in org.apache.hadoop.hbase.regionserver.IncreasingToUpperBoundRegionSplitPolicy.getSizeToCheck(int) > [findbugs] Address dodgy bugs > - > > Key: HBASE-5654 > URL: https://issues.apache.org/jira/browse/HBASE-5654 > Project: HBase > Issue Type: Sub-task > Components: scripts >Affects Versions: 0.96.0 >Reporter: Jonathan Hsieh >Assignee: Ashutosh Jindal > Labels: patch > Fix For: 0.96.0 > > Attachments: Hbase-5654.patch > > > See > https://builds.apache.org/job/PreCommit-HBASE-Build/1313//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html#Warnings_STYLE > This may be broken down further. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5795) hbase-3927 breaks 0.92<->0.94 compatibility
[
https://issues.apache.org/jira/browse/HBASE-5795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254718#comment-13254718
]
Zhihong Yu commented on HBASE-5795:
---
VersionedWritable.readFields() would detect version mismatch and throw
exception.
> hbase-3927 breaks 0.92<->0.94 compatibility
> ---
>
> Key: HBASE-5795
> URL: https://issues.apache.org/jira/browse/HBASE-5795
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: stack
> Fix For: 0.94.0
>
> Attachments: 5795-v2.txt, 5795.unittest.txt
>
>
> This commit broke our 0.92/0.94 compatibility:
> {code}
>
> r1136686 | stack | 2011-06-16 14:18:08 -0700 (Thu, 16 Jun 2011) | 1 line
> HBASE-3927 display total uncompressed byte size of a region in web UI
> {code}
> I just tried the new RC for 0.94. I brought up a 0.94 master on a 0.92
> cluster and rather than just digest version 1 of the HServerLoad, I get this:
> {code}
> 2012-04-14 22:47:59,752 WARN org.apache.hadoop.ipc.HBaseServer: Unable to
> read call parameters for client 10.4.14.38
> java.io.IOException: Error in readFields
> at
> org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:684)
> at
> org.apache.hadoop.hbase.ipc.Invocation.readFields(Invocation.java:125)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.processData(HBaseServer.java:1269)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.readAndProcess(HBaseServer.java:1184)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:722)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:513)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:488)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: A record version mismatch occured. Expecting v2, found v1
> at
> org.apache.hadoop.io.VersionedWritable.readFields(VersionedWritable.java:46)
> at
> org.apache.hadoop.hbase.HServerLoad$RegionLoad.readFields(HServerLoad.java:379)
> at
> org.apache.hadoop.hbase.HServerLoad.readFields(HServerLoad.java:686)
> at
> org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:681)
> ... 9 more
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5782) Not all the regions are getting assigned after the log splitting.
[ https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254724#comment-13254724 ] Zhihong Yu commented on HBASE-5782: --- One option is to not use appendNoSync() in doMiniBatchPut(). For high write throughput, I think HBASE-5699 should be the long-term solution. > Not all the regions are getting assigned after the log splitting. > - > > Key: HBASE-5782 > URL: https://issues.apache.org/jira/browse/HBASE-5782 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.94.0 >Reporter: Gopinathan A >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 0.94.0 > > > Create a table with 1000 splits, after the region assignemnt, kill the > regionserver wich contains META table. > Here few regions are missing after the log splitting and region assigment. > HBCK report shows multiple region holes are got created. > Same scenario was verified mulitple times in 0.92.1, no issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5782) Not all the regions are getting assigned after the log splitting.
[ https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254728#comment-13254728 ] ramkrishna.s.vasudevan commented on HBASE-5782: --- @Zhihong Yes HBASE-5699 can provide high write throughput. But even there the syncing should be done for every WAL that we will be writing there we should ensure that we do not sync everytime holding some locks. So ideally this problem is coming out of parallel syncs(which may happen even if we have > 1 WAL). Correct me if am wrong. > Not all the regions are getting assigned after the log splitting. > - > > Key: HBASE-5782 > URL: https://issues.apache.org/jira/browse/HBASE-5782 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.94.0 >Reporter: Gopinathan A >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 0.94.0 > > > Create a table with 1000 splits, after the region assignemnt, kill the > regionserver wich contains META table. > Here few regions are missing after the log splitting and region assigment. > HBCK report shows multiple region holes are got created. > Same scenario was verified mulitple times in 0.92.1, no issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5733) AssignmentManager#processDeadServersAndRegionsInTransition can fail with NPE.
[ https://issues.apache.org/jira/browse/HBASE-5733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uma Maheswara Rao G updated HBASE-5733: --- Attachment: HBASE-5733.patch > AssignmentManager#processDeadServersAndRegionsInTransition can fail with NPE. > - > > Key: HBASE-5733 > URL: https://issues.apache.org/jira/browse/HBASE-5733 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 0.96.0 >Reporter: Uma Maheswara Rao G >Assignee: Uma Maheswara Rao G > Attachments: HBASE-5733.patch > > > Found while going through the code... > AssignmentManager#processDeadServersAndRegionsInTransition can fail with NPE > as this is directly iterating the nodes from > listChildrenAndWatchForNewChildren with-out checking for null. > Here also we need to handle with null check like other places. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5733) AssignmentManager#processDeadServersAndRegionsInTransition can fail with NPE.
[
https://issues.apache.org/jira/browse/HBASE-5733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254757#comment-13254757
]
Zhihong Yu commented on HBASE-5733:
---
@Uma:
Can you generate a patch for trunk ?
I got the following when I tried to apply your patch to trunk:
{code}
[ERROR]
/Users/zhihyu/trunk-hbase/src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManager.java:[495,75]
unreported exception com.google.protobuf.ServiceException; must be caught or
declared to be thrown
{code}
> AssignmentManager#processDeadServersAndRegionsInTransition can fail with NPE.
> -
>
> Key: HBASE-5733
> URL: https://issues.apache.org/jira/browse/HBASE-5733
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 0.96.0
>Reporter: Uma Maheswara Rao G
>Assignee: Uma Maheswara Rao G
> Attachments: HBASE-5733.patch
>
>
> Found while going through the code...
> AssignmentManager#processDeadServersAndRegionsInTransition can fail with NPE
> as this is directly iterating the nodes from
> listChildrenAndWatchForNewChildren with-out checking for null.
> Here also we need to handle with null check like other places.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5782) Not all the regions are getting assigned after the log splitting.
[ https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-5782: -- Attachment: HBASE-5782.patch > Not all the regions are getting assigned after the log splitting. > - > > Key: HBASE-5782 > URL: https://issues.apache.org/jira/browse/HBASE-5782 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.94.0 >Reporter: Gopinathan A >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 0.94.0 > > Attachments: HBASE-5782.patch > > > Create a table with 1000 splits, after the region assignemnt, kill the > regionserver wich contains META table. > Here few regions are missing after the log splitting and region assigment. > HBCK report shows multiple region holes are got created. > Same scenario was verified mulitple times in 0.92.1, no issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5782) Not all the regions are getting assigned after the log splitting.
[ https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254782#comment-13254782 ] ramkrishna.s.vasudevan commented on HBASE-5782: --- This patch passes TestHLog and TestHRegion. Not able to run the testsuite as at home. The idea here is the edits till the seq id already persisted can be safely ignored. Now from this issue we are sure that the edits can appear in diff seq not necessary in the ascending order. So all the edits less than the seq id from the store files are skipped and all other edits are consdiered. Please provide your comments. > Not all the regions are getting assigned after the log splitting. > - > > Key: HBASE-5782 > URL: https://issues.apache.org/jira/browse/HBASE-5782 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.94.0 >Reporter: Gopinathan A >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 0.94.0 > > Attachments: HBASE-5782.patch > > > Create a table with 1000 splits, after the region assignemnt, kill the > regionserver wich contains META table. > Here few regions are missing after the log splitting and region assigment. > HBCK report shows multiple region holes are got created. > Same scenario was verified mulitple times in 0.92.1, no issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5782) Not all the regions are getting assigned after the log splitting.
[ https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-5782: -- Status: Patch Available (was: Open) > Not all the regions are getting assigned after the log splitting. > - > > Key: HBASE-5782 > URL: https://issues.apache.org/jira/browse/HBASE-5782 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.94.0 >Reporter: Gopinathan A >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 0.94.0 > > Attachments: HBASE-5782.patch > > > Create a table with 1000 splits, after the region assignemnt, kill the > regionserver wich contains META table. > Here few regions are missing after the log splitting and region assigment. > HBCK report shows multiple region holes are got created. > Same scenario was verified mulitple times in 0.92.1, no issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5795) hbase-3927 breaks 0.92<->0.94 compatibility
[
https://issues.apache.org/jira/browse/HBASE-5795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254794#comment-13254794
]
stack commented on HBASE-5795:
--
v2 works out on a cluster for me
> hbase-3927 breaks 0.92<->0.94 compatibility
> ---
>
> Key: HBASE-5795
> URL: https://issues.apache.org/jira/browse/HBASE-5795
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: stack
> Fix For: 0.94.0
>
> Attachments: 5795-v2.txt, 5795.unittest.txt
>
>
> This commit broke our 0.92/0.94 compatibility:
> {code}
>
> r1136686 | stack | 2011-06-16 14:18:08 -0700 (Thu, 16 Jun 2011) | 1 line
> HBASE-3927 display total uncompressed byte size of a region in web UI
> {code}
> I just tried the new RC for 0.94. I brought up a 0.94 master on a 0.92
> cluster and rather than just digest version 1 of the HServerLoad, I get this:
> {code}
> 2012-04-14 22:47:59,752 WARN org.apache.hadoop.ipc.HBaseServer: Unable to
> read call parameters for client 10.4.14.38
> java.io.IOException: Error in readFields
> at
> org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:684)
> at
> org.apache.hadoop.hbase.ipc.Invocation.readFields(Invocation.java:125)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.processData(HBaseServer.java:1269)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.readAndProcess(HBaseServer.java:1184)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:722)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:513)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:488)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: A record version mismatch occured. Expecting v2, found v1
> at
> org.apache.hadoop.io.VersionedWritable.readFields(VersionedWritable.java:46)
> at
> org.apache.hadoop.hbase.HServerLoad$RegionLoad.readFields(HServerLoad.java:379)
> at
> org.apache.hadoop.hbase.HServerLoad.readFields(HServerLoad.java:686)
> at
> org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:681)
> ... 9 more
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5654) [findbugs] Address dodgy bugs
[
https://issues.apache.org/jira/browse/HBASE-5654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254802#comment-13254802
]
Jonathan Hsieh commented on HBASE-5654:
---
I'm being pretty strict with excludes -- for an excludes it should be explained
(because they will be ignored forever more.. )
{code}
+
+
+
+
+
+
+
{code}
I think this one may have been handled in anther patch -- maybe this exclude
isn't needed.
{code}
+
+
+
+
+
+
+
{code}
This one is definitely fixable in code instead of exclude.
{code}
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
...
+
+
+
+
+
+
+
+
+
...
+
+
+
+
+
+
+
+
+
...
+
+
+
+
+
+
+
{code}
Definitely fixable in code without exclude -- (add default case that does
nothing or enumrate all cases and have default throws some sort of runtime
exception such as IllegalStateException to catch cases where new enums created).
{code}
+
+
+
+
+
+
+
+
{code}
File a separate jira for this one and I'll take care of it. Please to do not
exclude this warning.
{code}
+
+
+
+
+
+
+
+
{code}
Does this do anything? metrics is a package, SchemaConfigured is a class...
{code}
+
+
+
+
+
+
+
+
+
{code}
Probably wants bitSize and maxKeys to be cased to doubles before div.
> [findbugs] Address dodgy bugs
> -
>
> Key: HBASE-5654
> URL: https://issues.apache.org/jira/browse/HBASE-5654
> Project: HBase
> Issue Type: Sub-task
> Components: scripts
>Affects Versions: 0.96.0
>Reporter: Jonathan Hsieh
>Assignee: Ashutosh Jindal
> Labels: patch
> Fix For: 0.96.0
>
> Attachments: Hbase-5654.patch
>
>
> See
> https://builds.apache.org/job/PreCommit-HBASE-Build/1313//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html#Warnings_STYLE
> This may be broken down further.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5782) Not all the regions are getting assigned after the log splitting.
[ https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254805#comment-13254805 ] Jean-Daniel Cryans commented on HBASE-5782: --- bq. Based on one of our requirements i was thinking that the log syncer thread should be configurable like either to use it or not use it. As long as it's enabled by default I'm good with that. > Not all the regions are getting assigned after the log splitting. > - > > Key: HBASE-5782 > URL: https://issues.apache.org/jira/browse/HBASE-5782 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.94.0 >Reporter: Gopinathan A >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 0.94.0 > > Attachments: HBASE-5782.patch > > > Create a table with 1000 splits, after the region assignemnt, kill the > regionserver wich contains META table. > Here few regions are missing after the log splitting and region assigment. > HBCK report shows multiple region holes are got created. > Same scenario was verified mulitple times in 0.92.1, no issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5782) Not all the regions are getting assigned after the log splitting.
[ https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254810#comment-13254810 ] Hadoop QA commented on HBASE-5782: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12522794/HBASE-5782.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 3 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.io.hfile.TestForceCacheImportantBlocks Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1537//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1537//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1537//console This message is automatically generated. > Not all the regions are getting assigned after the log splitting. > - > > Key: HBASE-5782 > URL: https://issues.apache.org/jira/browse/HBASE-5782 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.94.0 >Reporter: Gopinathan A >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 0.94.0 > > Attachments: HBASE-5782.patch > > > Create a table with 1000 splits, after the region assignemnt, kill the > regionserver wich contains META table. > Here few regions are missing after the log splitting and region assigment. > HBCK report shows multiple region holes are got created. > Same scenario was verified mulitple times in 0.92.1, no issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5798) NPE running hbck on 0.94 out of reportTablesInFlux
NPE running hbck on 0.94 out of reportTablesInFlux
--
Key: HBASE-5798
URL: https://issues.apache.org/jira/browse/HBASE-5798
Project: HBase
Issue Type: Bug
Reporter: stack
Got this playing w/ hbck going against the 0.94RC:
{code}
12/04/16 17:03:14 INFO util.HBaseFsck: getHTableDescriptors == tableNames => []
Exception in thread "main" java.lang.NullPointerException
at
org.apache.hadoop.hbase.util.HBaseFsck.reportTablesInFlux(HBaseFsck.java:553)
at
org.apache.hadoop.hbase.util.HBaseFsck.onlineConsistencyRepair(HBaseFsck.java:344)
at org.apache.hadoop.hbase.util.HBaseFsck.onlineHbck(HBaseFsck.java:380)
at org.apache.hadoop.hbase.util.HBaseFsck.main(HBaseFsck.java:3033)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5798) NPE running hbck on 0.94 out of reportTablesInFlux
[
https://issues.apache.org/jira/browse/HBASE-5798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254811#comment-13254811
]
stack commented on HBASE-5798:
--
Error is transient. Subsequent runs worked.
> NPE running hbck on 0.94 out of reportTablesInFlux
> --
>
> Key: HBASE-5798
> URL: https://issues.apache.org/jira/browse/HBASE-5798
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>
> Got this playing w/ hbck going against the 0.94RC:
> {code}
> 12/04/16 17:03:14 INFO util.HBaseFsck: getHTableDescriptors == tableNames =>
> []
> Exception in thread "main" java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.util.HBaseFsck.reportTablesInFlux(HBaseFsck.java:553)
> at
> org.apache.hadoop.hbase.util.HBaseFsck.onlineConsistencyRepair(HBaseFsck.java:344)
> at
> org.apache.hadoop.hbase.util.HBaseFsck.onlineHbck(HBaseFsck.java:380)
> at org.apache.hadoop.hbase.util.HBaseFsck.main(HBaseFsck.java:3033)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5798) NPE running hbck on 0.94 out of reportTablesInFlux
[
https://issues.apache.org/jira/browse/HBASE-5798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254813#comment-13254813
]
ramkrishna.s.vasudevan commented on HBASE-5798:
---
@Stack
We too got one NPE in hbck. Still not found the reason. Not sure if it is same
as this one.
> NPE running hbck on 0.94 out of reportTablesInFlux
> --
>
> Key: HBASE-5798
> URL: https://issues.apache.org/jira/browse/HBASE-5798
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>
> Got this playing w/ hbck going against the 0.94RC:
> {code}
> 12/04/16 17:03:14 INFO util.HBaseFsck: getHTableDescriptors == tableNames =>
> []
> Exception in thread "main" java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.util.HBaseFsck.reportTablesInFlux(HBaseFsck.java:553)
> at
> org.apache.hadoop.hbase.util.HBaseFsck.onlineConsistencyRepair(HBaseFsck.java:344)
> at
> org.apache.hadoop.hbase.util.HBaseFsck.onlineHbck(HBaseFsck.java:380)
> at org.apache.hadoop.hbase.util.HBaseFsck.main(HBaseFsck.java:3033)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5733) AssignmentManager#processDeadServersAndRegionsInTransition can fail with NPE.
[ https://issues.apache.org/jira/browse/HBASE-5733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uma Maheswara Rao G updated HBASE-5733: --- Attachment: (was: HBASE-5733.patch) > AssignmentManager#processDeadServersAndRegionsInTransition can fail with NPE. > - > > Key: HBASE-5733 > URL: https://issues.apache.org/jira/browse/HBASE-5733 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 0.96.0 >Reporter: Uma Maheswara Rao G >Assignee: Uma Maheswara Rao G > > Found while going through the code... > AssignmentManager#processDeadServersAndRegionsInTransition can fail with NPE > as this is directly iterating the nodes from > listChildrenAndWatchForNewChildren with-out checking for null. > Here also we need to handle with null check like other places. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5799) [89-fb] Multiget API may return incomplete resutls
[89-fb] Multiget API may return incomplete resutls -- Key: HBASE-5799 URL: https://issues.apache.org/jira/browse/HBASE-5799 Project: HBase Issue Type: Bug Reporter: Liyin Tang There is a serious bug in the multiget which will cause the multiget function only returns part of the results. In the process function: The initial region is set before sorting the input list. So after the input list has been sorted, the initial region may no longer be the correct region for the first row in the sorted list. So the first row in the sorted list may be sent to the wrong region server which has no result for this row. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-5799) [89-fb] Multiget API may return incomplete resutls
[ https://issues.apache.org/jira/browse/HBASE-5799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang reassigned HBASE-5799: - Assignee: Liyin Tang > [89-fb] Multiget API may return incomplete resutls > -- > > Key: HBASE-5799 > URL: https://issues.apache.org/jira/browse/HBASE-5799 > Project: HBase > Issue Type: Bug >Reporter: Liyin Tang >Assignee: Liyin Tang > > There is a serious bug in the multiget which will cause the multiget function > only returns part of the results. > In the process function: > The initial region is set before sorting the input list. > So after the input list has been sorted, the initial region may no longer be > the correct region for the first row in the sorted list. > So the first row in the sorted list may be sent to the wrong region server > which has no result for this row. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5733) AssignmentManager#processDeadServersAndRegionsInTransition can fail with NPE.
[ https://issues.apache.org/jira/browse/HBASE-5733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uma Maheswara Rao G updated HBASE-5733: --- Attachment: HBASE-5733.patch Thanks a lot, Ted for taking a look! Yep, accidentally uploaded the little older one than today's patch. Updated the latest one, which I tested with real cluster for aborting on this situation. > AssignmentManager#processDeadServersAndRegionsInTransition can fail with NPE. > - > > Key: HBASE-5733 > URL: https://issues.apache.org/jira/browse/HBASE-5733 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 0.96.0 >Reporter: Uma Maheswara Rao G >Assignee: Uma Maheswara Rao G > Attachments: HBASE-5733.patch > > > Found while going through the code... > AssignmentManager#processDeadServersAndRegionsInTransition can fail with NPE > as this is directly iterating the nodes from > listChildrenAndWatchForNewChildren with-out checking for null. > Here also we need to handle with null check like other places. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5104) Provide a reliable intra-row pagination mechanism
[
https://issues.apache.org/jira/browse/HBASE-5104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254820#comment-13254820
]
Zhihong Yu commented on HBASE-5104:
---
Patch didn't apply cleanly:
{code}
/usr/bin/patch: malformed patch at line 285: Index:
src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java
{code}
> Provide a reliable intra-row pagination mechanism
> -
>
> Key: HBASE-5104
> URL: https://issues.apache.org/jira/browse/HBASE-5104
> Project: HBase
> Issue Type: Bug
>Reporter: Kannan Muthukkaruppan
>Assignee: Madhuwanti Vaidya
> Attachments: D2799.1.patch, D2799.2.patch, D2799.3.patch,
> testFilterList.rb
>
>
> Addendum:
> Doing pagination (retrieving at most "limit" number of KVs at a particular
> "offset") is currently supported via the ColumnPaginationFilter. However, it
> is not a very clean way of supporting pagination. Some of the problems with
> it are:
> * Normally, one would expect a query with (Filter(A) AND Filter(B)) to have
> same results as (query with Filter(A)) INTERSECT (query with Filter(B)). This
> is not the case for ColumnPaginationFilter as its internal state gets updated
> depending on whether or not Filter(A) returns TRUE/FALSE for a particular
> cell.
> * When this Filter is used in combination with other filters (e.g., doing AND
> with another filter using FilterList), the behavior of the query depends on
> the order of filters in the FilterList. This is not ideal.
> * ColumnPaginationFilter is a stateful filter which ends up counting multiple
> versions of the cell as separate values even if another filter upstream or
> the ScanQueryMatcher is going to reject the value for other reasons.
> Seems like we need a reliable way to do pagination. The particular use case
> that prompted this JIRA is pagination within the same rowKey. For example,
> for a given row key R, get columns with prefix P, starting at offset X (among
> columns which have prefix P) and limit Y. Some possible fixes might be:
> 1) enhance ColumnPrefixFilter to support another constructor which supports
> limit/offset.
> 2) Support pagination (limit/offset) at the Scan/Get API level (rather than
> as a filter) [Like SQL].
> Original Post:
> Thanks Jiakai Liu for reporting this issue and doing the initial
> investigation. Email from Jiakai below:
> Assuming that we have an index column family with the following entries:
> "tag0:001:thread1"
> ...
> "tag1:001:thread1"
> "tag1:002:thread2"
> ...
> "tag1:010:thread10"
> ...
> "tag2:001:thread1"
> "tag2:005:thread5"
> ...
> To get threads with "tag1" in range [5, 10), I tried the following code:
> ColumnPrefixFilter filter1 = new
> ColumnPrefixFilter(Bytes.toBytes("tag1"));
> ColumnPaginationFilter filter2 = new ColumnPaginationFilter(5 /* limit
> */, 5 /* offset */);
> FilterList filters = new FilterList(Operator.MUST_PASS_ALL);
> filters.addFilter(filter1);
> filters.addFilter(filter2);
> Get get = new Get(USER);
> get.addFamily(COLUMN_FAMILY);
> get.setMaxVersions(1);
> get.setFilter(filters);
> Somehow it didn't work as expected. It returned the entries as if the filter1
> were not set.
> Turns out the ColumnPrefixFilter returns SEEK_NEXT_USING_HINT in some cases.
> The FilterList filter does not handle this return code properly (treat it as
> INCLUDE).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5792) HLog Performance Evaluation Tool
[
https://issues.apache.org/jira/browse/HBASE-5792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254827#comment-13254827
]
stack commented on HBASE-5792:
--
This is great Matteo. We need this. Yeah, agree, this tool will have most
value if it puts nothing but a lone region (and WAL).
Few minors below:
Missing annotatations on audience.
Do you need these? IIRC, the default exists w/ need of definition:
{code}
+ public HLogPerformanceEvaluation() {
+ }
{code}
You do it in another place at least too.
No harm adding a bit of class doc on HLogPutBenchmark
You don't want to use a command parser?
> HLog Performance Evaluation Tool
>
>
> Key: HBASE-5792
> URL: https://issues.apache.org/jira/browse/HBASE-5792
> Project: HBase
> Issue Type: Test
> Components: wal
>Reporter: Matteo Bertozzi
>Assignee: Matteo Bertozzi
>Priority: Minor
> Labels: performance, wal
> Attachments: HBASE-5792-v0.patch, HBASE-5792-v1.patch
>
>
> Related to HDFS-3280 and the HBase WAL slowdown on 0.23+
> It would be nice to have a simple tool like HFilePerformanceEvaluation, ...
> to be able to check easily the HLog performance.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5795) hbase-3927 breaks 0.92<->0.94 compatibility
[
https://issues.apache.org/jira/browse/HBASE-5795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihong Yu updated HBASE-5795:
--
Fix Version/s: 0.96.0
Hadoop Flags: Reviewed
> hbase-3927 breaks 0.92<->0.94 compatibility
> ---
>
> Key: HBASE-5795
> URL: https://issues.apache.org/jira/browse/HBASE-5795
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: stack
> Fix For: 0.94.0, 0.96.0
>
> Attachments: 5795-v2.txt, 5795.unittest.txt
>
>
> This commit broke our 0.92/0.94 compatibility:
> {code}
>
> r1136686 | stack | 2011-06-16 14:18:08 -0700 (Thu, 16 Jun 2011) | 1 line
> HBASE-3927 display total uncompressed byte size of a region in web UI
> {code}
> I just tried the new RC for 0.94. I brought up a 0.94 master on a 0.92
> cluster and rather than just digest version 1 of the HServerLoad, I get this:
> {code}
> 2012-04-14 22:47:59,752 WARN org.apache.hadoop.ipc.HBaseServer: Unable to
> read call parameters for client 10.4.14.38
> java.io.IOException: Error in readFields
> at
> org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:684)
> at
> org.apache.hadoop.hbase.ipc.Invocation.readFields(Invocation.java:125)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.processData(HBaseServer.java:1269)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.readAndProcess(HBaseServer.java:1184)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:722)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:513)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:488)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: A record version mismatch occured. Expecting v2, found v1
> at
> org.apache.hadoop.io.VersionedWritable.readFields(VersionedWritable.java:46)
> at
> org.apache.hadoop.hbase.HServerLoad$RegionLoad.readFields(HServerLoad.java:379)
> at
> org.apache.hadoop.hbase.HServerLoad.readFields(HServerLoad.java:686)
> at
> org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:681)
> ... 9 more
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5795) hbase-3927 breaks 0.92<->0.94 compatibility
[
https://issues.apache.org/jira/browse/HBASE-5795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254826#comment-13254826
]
Zhihong Yu commented on HBASE-5795:
---
Will integrate patch v2 in 4 hours if there is no objection.
> hbase-3927 breaks 0.92<->0.94 compatibility
> ---
>
> Key: HBASE-5795
> URL: https://issues.apache.org/jira/browse/HBASE-5795
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: stack
> Fix For: 0.94.0, 0.96.0
>
> Attachments: 5795-v2.txt, 5795.unittest.txt
>
>
> This commit broke our 0.92/0.94 compatibility:
> {code}
>
> r1136686 | stack | 2011-06-16 14:18:08 -0700 (Thu, 16 Jun 2011) | 1 line
> HBASE-3927 display total uncompressed byte size of a region in web UI
> {code}
> I just tried the new RC for 0.94. I brought up a 0.94 master on a 0.92
> cluster and rather than just digest version 1 of the HServerLoad, I get this:
> {code}
> 2012-04-14 22:47:59,752 WARN org.apache.hadoop.ipc.HBaseServer: Unable to
> read call parameters for client 10.4.14.38
> java.io.IOException: Error in readFields
> at
> org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:684)
> at
> org.apache.hadoop.hbase.ipc.Invocation.readFields(Invocation.java:125)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.processData(HBaseServer.java:1269)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.readAndProcess(HBaseServer.java:1184)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:722)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:513)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:488)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: A record version mismatch occured. Expecting v2, found v1
> at
> org.apache.hadoop.io.VersionedWritable.readFields(VersionedWritable.java:46)
> at
> org.apache.hadoop.hbase.HServerLoad$RegionLoad.readFields(HServerLoad.java:379)
> at
> org.apache.hadoop.hbase.HServerLoad.readFields(HServerLoad.java:686)
> at
> org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:681)
> ... 9 more
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-5798) NPE running hbck on 0.94 out of reportTablesInFlux
[
https://issues.apache.org/jira/browse/HBASE-5798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hsieh reassigned HBASE-5798:
-
Assignee: Jonathan Hsieh
> NPE running hbck on 0.94 out of reportTablesInFlux
> --
>
> Key: HBASE-5798
> URL: https://issues.apache.org/jira/browse/HBASE-5798
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: Jonathan Hsieh
>
> Got this playing w/ hbck going against the 0.94RC:
> {code}
> 12/04/16 17:03:14 INFO util.HBaseFsck: getHTableDescriptors == tableNames =>
> []
> Exception in thread "main" java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.util.HBaseFsck.reportTablesInFlux(HBaseFsck.java:553)
> at
> org.apache.hadoop.hbase.util.HBaseFsck.onlineConsistencyRepair(HBaseFsck.java:344)
> at
> org.apache.hadoop.hbase.util.HBaseFsck.onlineHbck(HBaseFsck.java:380)
> at org.apache.hadoop.hbase.util.HBaseFsck.main(HBaseFsck.java:3033)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5800) Birds of a feather link on web page doesn't work.
Birds of a feather link on web page doesn't work. - Key: HBASE-5800 URL: https://issues.apache.org/jira/browse/HBASE-5800 Project: HBase Issue Type: Bug Reporter: Elliott Clark Assignee: Elliott Clark Attachments: HBASE-5800-0.patch just missing the http:// -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5800) Birds of a feather link on web page doesn't work.
[ https://issues.apache.org/jira/browse/HBASE-5800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-5800: - Attachment: HBASE-5800-0.patch > Birds of a feather link on web page doesn't work. > - > > Key: HBASE-5800 > URL: https://issues.apache.org/jira/browse/HBASE-5800 > Project: HBase > Issue Type: Bug >Reporter: Elliott Clark >Assignee: Elliott Clark > Attachments: HBASE-5800-0.patch > > > just missing the http:// -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5798) NPE running hbck on 0.94 out of reportTablesInFlux
[
https://issues.apache.org/jira/browse/HBASE-5798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254837#comment-13254837
]
Anoop Sam John commented on HBASE-5798:
---
@Ram , Yes this is the same issue.. I got the reason.
The scenario is like this as in our test.
There is one table and there was a case of one region of that table was not
assigned with any of the RS. HBCK tool fixing this issue. After that HBCK will
run again.
At this time getHTableDescriptors () is not finding any table in the cluster
and return null and so reportTablesInFlux() -> errors.print("Number of Tables:
" + allTables.length); gives a NPE
Why at this time no tables getting out of getHTableDescriptors () [Even though
one table is there in the cluster is] this table is modified recently. HBCK
just changed the HRegionInfo of the region of the table by assigning it to one
of the RS.
For fix
1. We need null check in reportTablesInFlux() I think
2. When HBCK rerun after the fix we can set timelag =0?
> NPE running hbck on 0.94 out of reportTablesInFlux
> --
>
> Key: HBASE-5798
> URL: https://issues.apache.org/jira/browse/HBASE-5798
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: Jonathan Hsieh
>
> Got this playing w/ hbck going against the 0.94RC:
> {code}
> 12/04/16 17:03:14 INFO util.HBaseFsck: getHTableDescriptors == tableNames =>
> []
> Exception in thread "main" java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.util.HBaseFsck.reportTablesInFlux(HBaseFsck.java:553)
> at
> org.apache.hadoop.hbase.util.HBaseFsck.onlineConsistencyRepair(HBaseFsck.java:344)
> at
> org.apache.hadoop.hbase.util.HBaseFsck.onlineHbck(HBaseFsck.java:380)
> at org.apache.hadoop.hbase.util.HBaseFsck.main(HBaseFsck.java:3033)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5733) AssignmentManager#processDeadServersAndRegionsInTransition can fail with NPE.
[ https://issues.apache.org/jira/browse/HBASE-5733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5733: -- Status: Patch Available (was: Open) testProcessDeadServersAndRegionsInTransitionShouldNotFailWithNPE failed without the patch and passes with the patch. > AssignmentManager#processDeadServersAndRegionsInTransition can fail with NPE. > - > > Key: HBASE-5733 > URL: https://issues.apache.org/jira/browse/HBASE-5733 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 0.96.0 >Reporter: Uma Maheswara Rao G >Assignee: Uma Maheswara Rao G > Attachments: HBASE-5733.patch > > > Found while going through the code... > AssignmentManager#processDeadServersAndRegionsInTransition can fail with NPE > as this is directly iterating the nodes from > listChildrenAndWatchForNewChildren with-out checking for null. > Here also we need to handle with null check like other places. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5733) AssignmentManager#processDeadServersAndRegionsInTransition can fail with NPE.
[
https://issues.apache.org/jira/browse/HBASE-5733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254841#comment-13254841
]
Zhihong Yu commented on HBASE-5733:
---
Minor comment:
Similar sentence appears 3 times below:
{code}
+ LOG.fatal("Problem in getting the children from ZK. Going to abort");
+ master.abort("Problem in getting the children from ZK", new IOException(
+ "Failed to get the children from ZK"));
+ return;
{code}
Can "Failed to get the children from ZK" be shared ?
> AssignmentManager#processDeadServersAndRegionsInTransition can fail with NPE.
> -
>
> Key: HBASE-5733
> URL: https://issues.apache.org/jira/browse/HBASE-5733
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 0.96.0
>Reporter: Uma Maheswara Rao G
>Assignee: Uma Maheswara Rao G
> Attachments: HBASE-5733.patch
>
>
> Found while going through the code...
> AssignmentManager#processDeadServersAndRegionsInTransition can fail with NPE
> as this is directly iterating the nodes from
> listChildrenAndWatchForNewChildren with-out checking for null.
> Here also we need to handle with null check like other places.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5798) NPE running hbck on 0.94 out of reportTablesInFlux
[
https://issues.apache.org/jira/browse/HBASE-5798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254843#comment-13254843
]
Jonathan Hsieh commented on HBASE-5798:
---
I started a run of the unit test suite testing this fix -- for a method like
this, I prefer returning empty arrays instead of null arrays.
{code}
diff --git src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
src/main/java/org/apache/hadoop/hbase/cli
index ee16e72..44b7c11 100644
--- src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
+++ src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
@@ -1691,7 +1691,7 @@ public class HBaseAdmin implements Abortable, Closeable {
/**
* Get tableDescriptors
* @param tableNames List of table names
- * @return HTD[] the tableDescriptor
+ * @return HTD[] the tableDescriptor (never null)
* @throws IOException if a remote or network exception occurs
*/
public HTableDescriptor[] getTableDescriptors(List tableNames)
diff --git src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
src/main/java/org/apache/hadoop/h
index 820e2a9..f183b15 100644
--- src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
+++ src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
@@ -2195,7 +2195,7 @@ public class HConnectionManager {
@Override
public HTableDescriptor[] getHTableDescriptors(List tableNames)
throws IOException {
- if (tableNames == null || tableNames.isEmpty()) return null;
+ if (tableNames == null || tableNames.isEmpty()) return new
HTableDescriptor[0];
MasterKeepAliveConnection master = getKeepAliveMaster();
try {
return master.getHTableDescriptors(tableNames);
{code}
> NPE running hbck on 0.94 out of reportTablesInFlux
> --
>
> Key: HBASE-5798
> URL: https://issues.apache.org/jira/browse/HBASE-5798
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: Jonathan Hsieh
>
> Got this playing w/ hbck going against the 0.94RC:
> {code}
> 12/04/16 17:03:14 INFO util.HBaseFsck: getHTableDescriptors == tableNames =>
> []
> Exception in thread "main" java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.util.HBaseFsck.reportTablesInFlux(HBaseFsck.java:553)
> at
> org.apache.hadoop.hbase.util.HBaseFsck.onlineConsistencyRepair(HBaseFsck.java:344)
> at
> org.apache.hadoop.hbase.util.HBaseFsck.onlineHbck(HBaseFsck.java:380)
> at org.apache.hadoop.hbase.util.HBaseFsck.main(HBaseFsck.java:3033)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-5800) Birds of a feather link on web page doesn't work.
[ https://issues.apache.org/jira/browse/HBASE-5800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-5800. -- Resolution: Fixed Fix Version/s: 0.96.0 Hadoop Flags: Reviewed Committed to trunk and deployed (should see it in an hour or so). Thanks for the patch Elliott. > Birds of a feather link on web page doesn't work. > - > > Key: HBASE-5800 > URL: https://issues.apache.org/jira/browse/HBASE-5800 > Project: HBase > Issue Type: Bug >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 0.96.0 > > Attachments: HBASE-5800-0.patch > > > just missing the http:// -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5798) NPE running hbck on 0.94 out of reportTablesInFlux
[
https://issues.apache.org/jira/browse/HBASE-5798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254852#comment-13254852
]
Anoop Sam John commented on HBASE-5798:
---
@Jon
Yes null check I also dont like to put...:)
Also what about 2. When HBCK rerun after the fix we can set timelag =0?
> NPE running hbck on 0.94 out of reportTablesInFlux
> --
>
> Key: HBASE-5798
> URL: https://issues.apache.org/jira/browse/HBASE-5798
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: Jonathan Hsieh
>
> Got this playing w/ hbck going against the 0.94RC:
> {code}
> 12/04/16 17:03:14 INFO util.HBaseFsck: getHTableDescriptors == tableNames =>
> []
> Exception in thread "main" java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.util.HBaseFsck.reportTablesInFlux(HBaseFsck.java:553)
> at
> org.apache.hadoop.hbase.util.HBaseFsck.onlineConsistencyRepair(HBaseFsck.java:344)
> at
> org.apache.hadoop.hbase.util.HBaseFsck.onlineHbck(HBaseFsck.java:380)
> at org.apache.hadoop.hbase.util.HBaseFsck.main(HBaseFsck.java:3033)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5801) [hbck] Hbck should handle case where some regions have different HTD settings in .regioninfo files (0.90 specific)
[hbck] Hbck should handle case where some regions have different HTD settings in .regioninfo files (0.90 specific) --- Key: HBASE-5801 URL: https://issues.apache.org/jira/browse/HBASE-5801 Project: HBase Issue Type: Improvement Components: hbck Affects Versions: 0.90.7 Reporter: Jonathan Hsieh Assignee: Jimmy Xiang Recently, we encountered a case where some regions in a table have different HTableDescriptor settings serialized into HDFS their HRegionInfo .regioninfo file. hbck expects all HTDs within a table to be the same and currently bails out in this situation. We need to either point out a proper set of actions for the user to execute or automatically convert the region to a common HTD (likely the most common on, or possibly the "first one".) Not sure if this requires reformatting data but may require closing and restarting a region. This issue is hbase 0.90.x specific -- 0.92+ keep all table info in a single .tableinfo file. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5798) NPE running hbck on 0.94 out of reportTablesInFlux
[
https://issues.apache.org/jira/browse/HBASE-5798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anoop Sam John updated HBASE-5798:
--
Component/s: hbck
Affects Version/s: 0.96.0
0.94.0
> NPE running hbck on 0.94 out of reportTablesInFlux
> --
>
> Key: HBASE-5798
> URL: https://issues.apache.org/jira/browse/HBASE-5798
> Project: HBase
> Issue Type: Bug
> Components: hbck
>Affects Versions: 0.94.0, 0.96.0
>Reporter: stack
>Assignee: Jonathan Hsieh
>
> Got this playing w/ hbck going against the 0.94RC:
> {code}
> 12/04/16 17:03:14 INFO util.HBaseFsck: getHTableDescriptors == tableNames =>
> []
> Exception in thread "main" java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.util.HBaseFsck.reportTablesInFlux(HBaseFsck.java:553)
> at
> org.apache.hadoop.hbase.util.HBaseFsck.onlineConsistencyRepair(HBaseFsck.java:344)
> at
> org.apache.hadoop.hbase.util.HBaseFsck.onlineHbck(HBaseFsck.java:380)
> at org.apache.hadoop.hbase.util.HBaseFsck.main(HBaseFsck.java:3033)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-5799) [89-fb] Multiget API may return incomplete resutls
[ https://issues.apache.org/jira/browse/HBASE-5799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liyin Tang resolved HBASE-5799. --- Resolution: Fixed Fixed internally and will be committed in 89-fb branch. > [89-fb] Multiget API may return incomplete resutls > -- > > Key: HBASE-5799 > URL: https://issues.apache.org/jira/browse/HBASE-5799 > Project: HBase > Issue Type: Bug >Reporter: Liyin Tang >Assignee: Liyin Tang > > There is a serious bug in the multiget which will cause the multiget function > only returns part of the results. > In the process function: > The initial region is set before sorting the input list. > So after the input list has been sorted, the initial region may no longer be > the correct region for the first row in the sorted list. > So the first row in the sorted list may be sent to the wrong region server > which has no result for this row. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5798) NPE running hbck on 0.94 out of reportTablesInFlux
[
https://issues.apache.org/jira/browse/HBASE-5798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254862#comment-13254862
]
Jonathan Hsieh commented on HBASE-5798:
---
I think #2 makes sense, but would need to be tested to verify (it is a legacy
of the original hbck -- I didn't change this).
> NPE running hbck on 0.94 out of reportTablesInFlux
> --
>
> Key: HBASE-5798
> URL: https://issues.apache.org/jira/browse/HBASE-5798
> Project: HBase
> Issue Type: Bug
> Components: hbck
>Affects Versions: 0.94.0, 0.96.0
>Reporter: stack
>Assignee: Jonathan Hsieh
>
> Got this playing w/ hbck going against the 0.94RC:
> {code}
> 12/04/16 17:03:14 INFO util.HBaseFsck: getHTableDescriptors == tableNames =>
> []
> Exception in thread "main" java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.util.HBaseFsck.reportTablesInFlux(HBaseFsck.java:553)
> at
> org.apache.hadoop.hbase.util.HBaseFsck.onlineConsistencyRepair(HBaseFsck.java:344)
> at
> org.apache.hadoop.hbase.util.HBaseFsck.onlineHbck(HBaseFsck.java:380)
> at org.apache.hadoop.hbase.util.HBaseFsck.main(HBaseFsck.java:3033)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5792) HLog Performance Evaluation Tool
[ https://issues.apache.org/jira/browse/HBASE-5792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matteo Bertozzi updated HBASE-5792: --- Attachment: HBASE-5792-v2.patch > HLog Performance Evaluation Tool > > > Key: HBASE-5792 > URL: https://issues.apache.org/jira/browse/HBASE-5792 > Project: HBase > Issue Type: Test > Components: wal >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi >Priority: Minor > Labels: performance, wal > Attachments: HBASE-5792-v0.patch, HBASE-5792-v1.patch, > HBASE-5792-v2.patch > > > Related to HDFS-3280 and the HBase WAL slowdown on 0.23+ > It would be nice to have a simple tool like HFilePerformanceEvaluation, ... > to be able to check easily the HLog performance. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5795) hbase-3927 breaks 0.92<->0.94 compatibility
[
https://issues.apache.org/jira/browse/HBASE-5795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254864#comment-13254864
]
Lars Hofhansl commented on HBASE-5795:
--
+1 on v2, are you integrating v2 with Stacks test?
> hbase-3927 breaks 0.92<->0.94 compatibility
> ---
>
> Key: HBASE-5795
> URL: https://issues.apache.org/jira/browse/HBASE-5795
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: stack
> Fix For: 0.94.0, 0.96.0
>
> Attachments: 5795-v2.txt, 5795.unittest.txt
>
>
> This commit broke our 0.92/0.94 compatibility:
> {code}
>
> r1136686 | stack | 2011-06-16 14:18:08 -0700 (Thu, 16 Jun 2011) | 1 line
> HBASE-3927 display total uncompressed byte size of a region in web UI
> {code}
> I just tried the new RC for 0.94. I brought up a 0.94 master on a 0.92
> cluster and rather than just digest version 1 of the HServerLoad, I get this:
> {code}
> 2012-04-14 22:47:59,752 WARN org.apache.hadoop.ipc.HBaseServer: Unable to
> read call parameters for client 10.4.14.38
> java.io.IOException: Error in readFields
> at
> org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:684)
> at
> org.apache.hadoop.hbase.ipc.Invocation.readFields(Invocation.java:125)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.processData(HBaseServer.java:1269)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.readAndProcess(HBaseServer.java:1184)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:722)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:513)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:488)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: A record version mismatch occured. Expecting v2, found v1
> at
> org.apache.hadoop.io.VersionedWritable.readFields(VersionedWritable.java:46)
> at
> org.apache.hadoop.hbase.HServerLoad$RegionLoad.readFields(HServerLoad.java:379)
> at
> org.apache.hadoop.hbase.HServerLoad.readFields(HServerLoad.java:686)
> at
> org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:681)
> ... 9 more
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5798) NPE running hbck on 0.94 out of reportTablesInFlux
[
https://issues.apache.org/jira/browse/HBASE-5798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254867#comment-13254867
]
Jonathan Hsieh commented on HBASE-5798:
---
Anoop -- do you guys want to take this on or should I?
> NPE running hbck on 0.94 out of reportTablesInFlux
> --
>
> Key: HBASE-5798
> URL: https://issues.apache.org/jira/browse/HBASE-5798
> Project: HBase
> Issue Type: Bug
> Components: hbck
>Affects Versions: 0.94.0, 0.96.0
>Reporter: stack
>Assignee: Jonathan Hsieh
>
> Got this playing w/ hbck going against the 0.94RC:
> {code}
> 12/04/16 17:03:14 INFO util.HBaseFsck: getHTableDescriptors == tableNames =>
> []
> Exception in thread "main" java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.util.HBaseFsck.reportTablesInFlux(HBaseFsck.java:553)
> at
> org.apache.hadoop.hbase.util.HBaseFsck.onlineConsistencyRepair(HBaseFsck.java:344)
> at
> org.apache.hadoop.hbase.util.HBaseFsck.onlineHbck(HBaseFsck.java:380)
> at org.apache.hadoop.hbase.util.HBaseFsck.main(HBaseFsck.java:3033)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5792) HLog Performance Evaluation Tool
[ https://issues.apache.org/jira/browse/HBASE-5792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254869#comment-13254869 ] stack commented on HBASE-5792: -- @Matteo NVM. I want to use this tool now so I'll take care of the above. Good stuff. > HLog Performance Evaluation Tool > > > Key: HBASE-5792 > URL: https://issues.apache.org/jira/browse/HBASE-5792 > Project: HBase > Issue Type: Test > Components: wal >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi >Priority: Minor > Labels: performance, wal > Attachments: HBASE-5792-v0.patch, HBASE-5792-v1.patch, > HBASE-5792-v2.patch > > > Related to HDFS-3280 and the HBase WAL slowdown on 0.23+ > It would be nice to have a simple tool like HFilePerformanceEvaluation, ... > to be able to check easily the HLog performance. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5733) AssignmentManager#processDeadServersAndRegionsInTransition can fail with NPE.
[ https://issues.apache.org/jira/browse/HBASE-5733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uma Maheswara Rao G updated HBASE-5733: --- Attachment: HBASE-5733.patch > AssignmentManager#processDeadServersAndRegionsInTransition can fail with NPE. > - > > Key: HBASE-5733 > URL: https://issues.apache.org/jira/browse/HBASE-5733 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 0.96.0 >Reporter: Uma Maheswara Rao G >Assignee: Uma Maheswara Rao G > Attachments: HBASE-5733.patch, HBASE-5733.patch > > > Found while going through the code... > AssignmentManager#processDeadServersAndRegionsInTransition can fail with NPE > as this is directly iterating the nodes from > listChildrenAndWatchForNewChildren with-out checking for null. > Here also we need to handle with null check like other places. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5795) hbase-3927 breaks 0.92<->0.94 compatibility
[
https://issues.apache.org/jira/browse/HBASE-5795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254873#comment-13254873
]
Zhihong Yu commented on HBASE-5795:
---
I am open in this regard.
Since the 0.92 deserialization code would be stable (RegionLoad format in 0.92
shouldn't change), I wonder if manual verification is enough.
> hbase-3927 breaks 0.92<->0.94 compatibility
> ---
>
> Key: HBASE-5795
> URL: https://issues.apache.org/jira/browse/HBASE-5795
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: stack
> Fix For: 0.94.0, 0.96.0
>
> Attachments: 5795-v2.txt, 5795.unittest.txt
>
>
> This commit broke our 0.92/0.94 compatibility:
> {code}
>
> r1136686 | stack | 2011-06-16 14:18:08 -0700 (Thu, 16 Jun 2011) | 1 line
> HBASE-3927 display total uncompressed byte size of a region in web UI
> {code}
> I just tried the new RC for 0.94. I brought up a 0.94 master on a 0.92
> cluster and rather than just digest version 1 of the HServerLoad, I get this:
> {code}
> 2012-04-14 22:47:59,752 WARN org.apache.hadoop.ipc.HBaseServer: Unable to
> read call parameters for client 10.4.14.38
> java.io.IOException: Error in readFields
> at
> org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:684)
> at
> org.apache.hadoop.hbase.ipc.Invocation.readFields(Invocation.java:125)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.processData(HBaseServer.java:1269)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.readAndProcess(HBaseServer.java:1184)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:722)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:513)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:488)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: A record version mismatch occured. Expecting v2, found v1
> at
> org.apache.hadoop.io.VersionedWritable.readFields(VersionedWritable.java:46)
> at
> org.apache.hadoop.hbase.HServerLoad$RegionLoad.readFields(HServerLoad.java:379)
> at
> org.apache.hadoop.hbase.HServerLoad.readFields(HServerLoad.java:686)
> at
> org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:681)
> ... 9 more
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5798) NPE running hbck on 0.94 out of reportTablesInFlux
[
https://issues.apache.org/jira/browse/HBASE-5798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254879#comment-13254879
]
Anoop Sam John commented on HBASE-5798:
---
Jon, I can provide a patch tomorrow addressing both the points I have
mentioned.[If it is ok with you]
> NPE running hbck on 0.94 out of reportTablesInFlux
> --
>
> Key: HBASE-5798
> URL: https://issues.apache.org/jira/browse/HBASE-5798
> Project: HBase
> Issue Type: Bug
> Components: hbck
>Affects Versions: 0.94.0, 0.96.0
>Reporter: stack
>Assignee: Jonathan Hsieh
>
> Got this playing w/ hbck going against the 0.94RC:
> {code}
> 12/04/16 17:03:14 INFO util.HBaseFsck: getHTableDescriptors == tableNames =>
> []
> Exception in thread "main" java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.util.HBaseFsck.reportTablesInFlux(HBaseFsck.java:553)
> at
> org.apache.hadoop.hbase.util.HBaseFsck.onlineConsistencyRepair(HBaseFsck.java:344)
> at
> org.apache.hadoop.hbase.util.HBaseFsck.onlineHbck(HBaseFsck.java:380)
> at org.apache.hadoop.hbase.util.HBaseFsck.main(HBaseFsck.java:3033)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5733) AssignmentManager#processDeadServersAndRegionsInTransition can fail with NPE.
[ https://issues.apache.org/jira/browse/HBASE-5733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254880#comment-13254880 ] Uma Maheswara Rao G commented on HBASE-5733: Thanks a lot Ted for the reviews! Updated the patch with your suggestion. > AssignmentManager#processDeadServersAndRegionsInTransition can fail with NPE. > - > > Key: HBASE-5733 > URL: https://issues.apache.org/jira/browse/HBASE-5733 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 0.96.0 >Reporter: Uma Maheswara Rao G >Assignee: Uma Maheswara Rao G > Attachments: HBASE-5733.patch, HBASE-5733.patch > > > Found while going through the code... > AssignmentManager#processDeadServersAndRegionsInTransition can fail with NPE > as this is directly iterating the nodes from > listChildrenAndWatchForNewChildren with-out checking for null. > Here also we need to handle with null check like other places. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-5798) NPE running hbck on 0.94 out of reportTablesInFlux
[
https://issues.apache.org/jira/browse/HBASE-5798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hsieh reassigned HBASE-5798:
-
Assignee: Anoop Sam John (was: Jonathan Hsieh)
Anoop -- go for it. :)
> NPE running hbck on 0.94 out of reportTablesInFlux
> --
>
> Key: HBASE-5798
> URL: https://issues.apache.org/jira/browse/HBASE-5798
> Project: HBase
> Issue Type: Bug
> Components: hbck
>Affects Versions: 0.94.0, 0.96.0
>Reporter: stack
>Assignee: Anoop Sam John
>
> Got this playing w/ hbck going against the 0.94RC:
> {code}
> 12/04/16 17:03:14 INFO util.HBaseFsck: getHTableDescriptors == tableNames =>
> []
> Exception in thread "main" java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.util.HBaseFsck.reportTablesInFlux(HBaseFsck.java:553)
> at
> org.apache.hadoop.hbase.util.HBaseFsck.onlineConsistencyRepair(HBaseFsck.java:344)
> at
> org.apache.hadoop.hbase.util.HBaseFsck.onlineHbck(HBaseFsck.java:380)
> at org.apache.hadoop.hbase.util.HBaseFsck.main(HBaseFsck.java:3033)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5798) NPE running hbck on 0.94 out of reportTablesInFlux
[
https://issues.apache.org/jira/browse/HBASE-5798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254888#comment-13254888
]
Jonathan Hsieh commented on HBASE-5798:
---
Returning empty array is valid. I dug a little into the master side as well --
it returns an empty array in the case where an invalid set of table names is
passed.
> NPE running hbck on 0.94 out of reportTablesInFlux
> --
>
> Key: HBASE-5798
> URL: https://issues.apache.org/jira/browse/HBASE-5798
> Project: HBase
> Issue Type: Bug
> Components: hbck
>Affects Versions: 0.94.0, 0.96.0
>Reporter: stack
>Assignee: Anoop Sam John
>
> Got this playing w/ hbck going against the 0.94RC:
> {code}
> 12/04/16 17:03:14 INFO util.HBaseFsck: getHTableDescriptors == tableNames =>
> []
> Exception in thread "main" java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.util.HBaseFsck.reportTablesInFlux(HBaseFsck.java:553)
> at
> org.apache.hadoop.hbase.util.HBaseFsck.onlineConsistencyRepair(HBaseFsck.java:344)
> at
> org.apache.hadoop.hbase.util.HBaseFsck.onlineHbck(HBaseFsck.java:380)
> at org.apache.hadoop.hbase.util.HBaseFsck.main(HBaseFsck.java:3033)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5733) AssignmentManager#processDeadServersAndRegionsInTransition can fail with NPE.
[ https://issues.apache.org/jira/browse/HBASE-5733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254891#comment-13254891 ] Hadoop QA commented on HBASE-5733: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12522805/HBASE-5733.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 3 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1538//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1538//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1538//console This message is automatically generated. > AssignmentManager#processDeadServersAndRegionsInTransition can fail with NPE. > - > > Key: HBASE-5733 > URL: https://issues.apache.org/jira/browse/HBASE-5733 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 0.96.0 >Reporter: Uma Maheswara Rao G >Assignee: Uma Maheswara Rao G > Attachments: HBASE-5733.patch, HBASE-5733.patch > > > Found while going through the code... > AssignmentManager#processDeadServersAndRegionsInTransition can fail with NPE > as this is directly iterating the nodes from > listChildrenAndWatchForNewChildren with-out checking for null. > Here also we need to handle with null check like other places. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5782) Not all the regions are getting assigned after the log splitting.
[
https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254892#comment-13254892
]
Jean-Daniel Cryans commented on HBASE-5782:
---
Looking more into this, I think HBASE-4487 is the real issue. I think I can
also prove that you can get the issue even with a disabled {{LogSyncer}}.
t1 does {{appendNoSync}} of k1
t1 does {{syncer}} up to {{getPendingWrites}}
t2 does {{appendNoSync}} of k2
t2 does {{syncer}} up to the end
In the log you'd see k2 then k1 so what's really wrong to me is this:
{code}
// Done in parallel for all writer threads, thanks to HDFS-895
List pending = logSyncerThread.getPendingWrites();
{code}
Although accessing pending writes is done in sync, you can apply them in
whichever way.
Furthermore, {{logSyncerThread.hlogFlush}} can also append entries to the WAL
in any order. For example, if both t1 and t2 have multiple edits they could end
up intermingled in the WAL simply by doing {{hlogFlush}} at the same time.
If {{LogSyncer}} was really an issue then {{HRegion.put}} and
{{HRegion.delete}} would need to be disabled too since they don't use
{{appendNoSync}} and just sync everything :)
How this used to work is that threads could only append to the WAL under the
{{updateLock}} and that was done at the same time as the {{doWrite}} which
creates the key. The call to sync could be done by any number of threads at the
same time.
If this is right, then we should pull back HBASE-4487 or add more locks.
We should also change this Jira's title once we get a better understanding of
the problem because it's not a region assignment problem.
> Not all the regions are getting assigned after the log splitting.
> -
>
> Key: HBASE-5782
> URL: https://issues.apache.org/jira/browse/HBASE-5782
> Project: HBase
> Issue Type: Bug
> Components: wal
>Affects Versions: 0.94.0
>Reporter: Gopinathan A
>Assignee: ramkrishna.s.vasudevan
>Priority: Blocker
> Fix For: 0.94.0
>
> Attachments: HBASE-5782.patch
>
>
> Create a table with 1000 splits, after the region assignemnt, kill the
> regionserver wich contains META table.
> Here few regions are missing after the log splitting and region assigment.
> HBCK report shows multiple region holes are got created.
> Same scenario was verified mulitple times in 0.92.1, no issues.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5800) Birds of a feather link on web page doesn't work.
[ https://issues.apache.org/jira/browse/HBASE-5800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254893#comment-13254893 ] Hudson commented on HBASE-5800: --- Integrated in HBase-TRUNK #2768 (See [https://builds.apache.org/job/HBase-TRUNK/2768/]) HBASE-5800 Birds of a feather link on web page doesn't work (Revision 1326695) Result = SUCCESS stack : Files : * /hbase/trunk/src/site/xdoc/index.xml > Birds of a feather link on web page doesn't work. > - > > Key: HBASE-5800 > URL: https://issues.apache.org/jira/browse/HBASE-5800 > Project: HBase > Issue Type: Bug >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 0.96.0 > > Attachments: HBASE-5800-0.patch > > > just missing the http:// -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5776) HTableMultiplexer
[ https://issues.apache.org/jira/browse/HBASE-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254901#comment-13254901 ] Kannan Muthukkaruppan commented on HBASE-5776: -- Ted: I think keeping "Table" in the name does not take way from the fact that it can support multiple tables. The multiplexer supports table operations-- so imo the name is ok as is. > HTableMultiplexer > -- > > Key: HBASE-5776 > URL: https://issues.apache.org/jira/browse/HBASE-5776 > Project: HBase > Issue Type: Improvement >Reporter: Liyin Tang >Assignee: Liyin Tang > Attachments: D2775.1.patch, D2775.1.patch, D2775.2.patch, > D2775.2.patch > > > There is a known issue in HBase client that single slow/dead region server > could slow down the multiput operations across all the region servers. So the > HBase client will be as slow as the slowest region server in the cluster. > > To solve this problem, HTableMultiplexer will separate the multiput > submitting threads with the flush threads, which means the multiput operation > will be a nonblocking operation. > The submitting thread will shard all the puts into different queues based on > its destination region server and return immediately. The flush threads will > flush these puts from each queue to its destination region server. > Currently the HTableMultiplexer only supports the put operation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5747) Forward port "hbase-5708 [89-fb] Make MiniMapRedCluster directory a subdirectory of target/test"
[
https://issues.apache.org/jira/browse/HBASE-5747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254907#comment-13254907
]
Zhihong Yu commented on HBASE-5747:
---
Recent Hadoop QA results,
https://builds.apache.org/job/PreCommit-HBASE-Build/1538/console as an example,
show the following:
{code}
-1 core tests. The patch failed these unit tests:
{code}
I tried to use my script to find the hanging test but wasn't able to.
> Forward port "hbase-5708 [89-fb] Make MiniMapRedCluster directory a
> subdirectory of target/test"
>
>
> Key: HBASE-5747
> URL: https://issues.apache.org/jira/browse/HBASE-5747
> Project: HBase
> Issue Type: Task
>Reporter: stack
>Assignee: stack
>Priority: Blocker
> Fix For: 0.96.0
>
> Attachments: 5474.txt, 5474v2.txt, 5474v3 (1).txt, 5474v3.txt,
> 5708v4.txt, 5708v4.txt
>
>
> Forward port as much as we can of Mikhail's hard-won test cleanups over on
> 0.89 branch Will improve our being able to run unit tests in //. He also
> found a few bugs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5782) Not all the regions are getting assigned after the log splitting.
[ https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254917#comment-13254917 ] Zhihong Yu commented on HBASE-5782: --- bq. If this is right, then we should pull back HBASE-4487 or add more locks Adding more locks would take more time to validate / test. In order to get 0.94.0 out the door, can we pull back HBASE-4487 in 0.94 and pursue the locking approach in trunk (or separate branch) ? > Not all the regions are getting assigned after the log splitting. > - > > Key: HBASE-5782 > URL: https://issues.apache.org/jira/browse/HBASE-5782 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.94.0 >Reporter: Gopinathan A >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 0.94.0 > > Attachments: HBASE-5782.patch > > > Create a table with 1000 splits, after the region assignemnt, kill the > regionserver wich contains META table. > Here few regions are missing after the log splitting and region assigment. > HBCK report shows multiple region holes are got created. > Same scenario was verified mulitple times in 0.92.1, no issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5782) Not all the regions are getting assigned after the log splitting.
[ https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254925#comment-13254925 ] Lars Hofhansl commented on HBASE-5782: -- So the problem is that logSyncerThread keeps the edit in order but the syncer then applies the pending batches potentially out of order? We might just need a "sync lock" to prevent two threads syncing at the same time. This is different from the update lock, which also prevents writing any edits. > Not all the regions are getting assigned after the log splitting. > - > > Key: HBASE-5782 > URL: https://issues.apache.org/jira/browse/HBASE-5782 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.94.0 >Reporter: Gopinathan A >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 0.94.0 > > Attachments: HBASE-5782.patch > > > Create a table with 1000 splits, after the region assignemnt, kill the > regionserver wich contains META table. > Here few regions are missing after the log splitting and region assigment. > HBCK report shows multiple region holes are got created. > Same scenario was verified mulitple times in 0.92.1, no issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5795) hbase-3927 breaks 0.92<->0.94 compatibility
[
https://issues.apache.org/jira/browse/HBASE-5795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254928#comment-13254928
]
stack commented on HBASE-5795:
--
No. Please include the unit test on commit.
> hbase-3927 breaks 0.92<->0.94 compatibility
> ---
>
> Key: HBASE-5795
> URL: https://issues.apache.org/jira/browse/HBASE-5795
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: stack
> Fix For: 0.94.0, 0.96.0
>
> Attachments: 5795-v2.txt, 5795.unittest.txt
>
>
> This commit broke our 0.92/0.94 compatibility:
> {code}
>
> r1136686 | stack | 2011-06-16 14:18:08 -0700 (Thu, 16 Jun 2011) | 1 line
> HBASE-3927 display total uncompressed byte size of a region in web UI
> {code}
> I just tried the new RC for 0.94. I brought up a 0.94 master on a 0.92
> cluster and rather than just digest version 1 of the HServerLoad, I get this:
> {code}
> 2012-04-14 22:47:59,752 WARN org.apache.hadoop.ipc.HBaseServer: Unable to
> read call parameters for client 10.4.14.38
> java.io.IOException: Error in readFields
> at
> org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:684)
> at
> org.apache.hadoop.hbase.ipc.Invocation.readFields(Invocation.java:125)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.processData(HBaseServer.java:1269)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.readAndProcess(HBaseServer.java:1184)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:722)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:513)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:488)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: A record version mismatch occured. Expecting v2, found v1
> at
> org.apache.hadoop.io.VersionedWritable.readFields(VersionedWritable.java:46)
> at
> org.apache.hadoop.hbase.HServerLoad$RegionLoad.readFields(HServerLoad.java:379)
> at
> org.apache.hadoop.hbase.HServerLoad.readFields(HServerLoad.java:686)
> at
> org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:681)
> ... 9 more
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5782) Not all the regions are getting assigned after the log splitting.
[
https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254932#comment-13254932
]
Jean-Daniel Cryans commented on HBASE-5782:
---
bq. In order to get 0.94.0 out the door, can we pull back HBASE-4487 in 0.94
and pursue the locking approach in trunk (or separate branch) ?
+1, we might want to review HBASE-4282 too as it seems to do something similar
with the transaction ids.
bq. So the problem is that logSyncerThread keeps the edit in order but the
syncer then applies the pending batches potentially out of order?
It's sad that the pending edits live in {{LogSyncer}}, that thread is really
just suppose to call sync... but yeah so they are added there in order but then
it's a free for all in {{syncer}}. Adding a sync there could solve the issue
but in the end what it does is moving the log from appending (pre HBASE-4487)
to syncing plus a _ton_ of new complexity in HLog. I'd prefer solution that
doesn't add a lock to patch something that's broken.
> Not all the regions are getting assigned after the log splitting.
> -
>
> Key: HBASE-5782
> URL: https://issues.apache.org/jira/browse/HBASE-5782
> Project: HBase
> Issue Type: Bug
> Components: wal
>Affects Versions: 0.94.0
>Reporter: Gopinathan A
>Assignee: ramkrishna.s.vasudevan
>Priority: Blocker
> Fix For: 0.94.0
>
> Attachments: HBASE-5782.patch
>
>
> Create a table with 1000 splits, after the region assignemnt, kill the
> regionserver wich contains META table.
> Here few regions are missing after the log splitting and region assigment.
> HBCK report shows multiple region holes are got created.
> Same scenario was verified mulitple times in 0.92.1, no issues.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-3614) Expose per-region request rate metrics
[ https://issues.apache.org/jira/browse/HBASE-3614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark reassigned HBASE-3614: Assignee: Elliott Clark > Expose per-region request rate metrics > -- > > Key: HBASE-3614 > URL: https://issues.apache.org/jira/browse/HBASE-3614 > Project: HBase > Issue Type: Improvement > Components: metrics, regionserver >Reporter: Gary Helmling >Assignee: Elliott Clark >Priority: Minor > > We currently export metrics on request rates for each region server, and this > can help with identifying uneven load at a high level. But once you see a > given server under high load, you're forced to extrapolate based on your > application patterns and the data it's serving what the likely culprit is. > This can and should be much easier if we just exported request rate metrics > per-region on each server. > Dynamically updating the metrics keys based on assigned regions may pose some > minor challenges, but this seems a very valuable diagnostic tool to have > available. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5792) HLog Performance Evaluation Tool
[ https://issues.apache.org/jira/browse/HBASE-5792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254937#comment-13254937 ] Hadoop QA commented on HBASE-5792: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12522825/HBASE-5792-v2.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 3 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1539//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1539//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1539//console This message is automatically generated. > HLog Performance Evaluation Tool > > > Key: HBASE-5792 > URL: https://issues.apache.org/jira/browse/HBASE-5792 > Project: HBase > Issue Type: Test > Components: wal >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi >Priority: Minor > Labels: performance, wal > Attachments: HBASE-5792-v0.patch, HBASE-5792-v1.patch, > HBASE-5792-v2.patch > > > Related to HDFS-3280 and the HBase WAL slowdown on 0.23+ > It would be nice to have a simple tool like HFilePerformanceEvaluation, ... > to be able to check easily the HLog performance. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5733) AssignmentManager#processDeadServersAndRegionsInTransition can fail with NPE.
[ https://issues.apache.org/jira/browse/HBASE-5733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254950#comment-13254950 ] Hadoop QA commented on HBASE-5733: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12522828/HBASE-5733.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 3 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.regionserver.TestServerCustomProtocol Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1540//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1540//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1540//console This message is automatically generated. > AssignmentManager#processDeadServersAndRegionsInTransition can fail with NPE. > - > > Key: HBASE-5733 > URL: https://issues.apache.org/jira/browse/HBASE-5733 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 0.96.0 >Reporter: Uma Maheswara Rao G >Assignee: Uma Maheswara Rao G > Attachments: HBASE-5733.patch, HBASE-5733.patch > > > Found while going through the code... > AssignmentManager#processDeadServersAndRegionsInTransition can fail with NPE > as this is directly iterating the nodes from > listChildrenAndWatchForNewChildren with-out checking for null. > Here also we need to handle with null check like other places. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5795) hbase-3927 breaks 0.92<->0.94 compatibility
[
https://issues.apache.org/jira/browse/HBASE-5795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihong Yu updated HBASE-5795:
--
Attachment: 5795-v3.txt
Patch combining v2 and Stack's test.
> hbase-3927 breaks 0.92<->0.94 compatibility
> ---
>
> Key: HBASE-5795
> URL: https://issues.apache.org/jira/browse/HBASE-5795
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: stack
> Fix For: 0.94.0, 0.96.0
>
> Attachments: 5795-v2.txt, 5795-v3.txt, 5795.unittest.txt
>
>
> This commit broke our 0.92/0.94 compatibility:
> {code}
>
> r1136686 | stack | 2011-06-16 14:18:08 -0700 (Thu, 16 Jun 2011) | 1 line
> HBASE-3927 display total uncompressed byte size of a region in web UI
> {code}
> I just tried the new RC for 0.94. I brought up a 0.94 master on a 0.92
> cluster and rather than just digest version 1 of the HServerLoad, I get this:
> {code}
> 2012-04-14 22:47:59,752 WARN org.apache.hadoop.ipc.HBaseServer: Unable to
> read call parameters for client 10.4.14.38
> java.io.IOException: Error in readFields
> at
> org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:684)
> at
> org.apache.hadoop.hbase.ipc.Invocation.readFields(Invocation.java:125)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.processData(HBaseServer.java:1269)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.readAndProcess(HBaseServer.java:1184)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:722)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:513)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:488)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: A record version mismatch occured. Expecting v2, found v1
> at
> org.apache.hadoop.io.VersionedWritable.readFields(VersionedWritable.java:46)
> at
> org.apache.hadoop.hbase.HServerLoad$RegionLoad.readFields(HServerLoad.java:379)
> at
> org.apache.hadoop.hbase.HServerLoad.readFields(HServerLoad.java:686)
> at
> org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:681)
> ... 9 more
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5104) Provide a reliable intra-row pagination mechanism
[
https://issues.apache.org/jira/browse/HBASE-5104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Bautin updated HBASE-5104:
--
Attachment:
jira-HBASE-5104-Provide-a-reliable-intra-row-paginat-2012-04-16_12_39_42.patch
Manually attaching the most recent patch.
> Provide a reliable intra-row pagination mechanism
> -
>
> Key: HBASE-5104
> URL: https://issues.apache.org/jira/browse/HBASE-5104
> Project: HBase
> Issue Type: Bug
>Reporter: Kannan Muthukkaruppan
>Assignee: Madhuwanti Vaidya
> Attachments: D2799.1.patch, D2799.2.patch, D2799.3.patch,
> jira-HBASE-5104-Provide-a-reliable-intra-row-paginat-2012-04-16_12_39_42.patch,
> testFilterList.rb
>
>
> Addendum:
> Doing pagination (retrieving at most "limit" number of KVs at a particular
> "offset") is currently supported via the ColumnPaginationFilter. However, it
> is not a very clean way of supporting pagination. Some of the problems with
> it are:
> * Normally, one would expect a query with (Filter(A) AND Filter(B)) to have
> same results as (query with Filter(A)) INTERSECT (query with Filter(B)). This
> is not the case for ColumnPaginationFilter as its internal state gets updated
> depending on whether or not Filter(A) returns TRUE/FALSE for a particular
> cell.
> * When this Filter is used in combination with other filters (e.g., doing AND
> with another filter using FilterList), the behavior of the query depends on
> the order of filters in the FilterList. This is not ideal.
> * ColumnPaginationFilter is a stateful filter which ends up counting multiple
> versions of the cell as separate values even if another filter upstream or
> the ScanQueryMatcher is going to reject the value for other reasons.
> Seems like we need a reliable way to do pagination. The particular use case
> that prompted this JIRA is pagination within the same rowKey. For example,
> for a given row key R, get columns with prefix P, starting at offset X (among
> columns which have prefix P) and limit Y. Some possible fixes might be:
> 1) enhance ColumnPrefixFilter to support another constructor which supports
> limit/offset.
> 2) Support pagination (limit/offset) at the Scan/Get API level (rather than
> as a filter) [Like SQL].
> Original Post:
> Thanks Jiakai Liu for reporting this issue and doing the initial
> investigation. Email from Jiakai below:
> Assuming that we have an index column family with the following entries:
> "tag0:001:thread1"
> ...
> "tag1:001:thread1"
> "tag1:002:thread2"
> ...
> "tag1:010:thread10"
> ...
> "tag2:001:thread1"
> "tag2:005:thread5"
> ...
> To get threads with "tag1" in range [5, 10), I tried the following code:
> ColumnPrefixFilter filter1 = new
> ColumnPrefixFilter(Bytes.toBytes("tag1"));
> ColumnPaginationFilter filter2 = new ColumnPaginationFilter(5 /* limit
> */, 5 /* offset */);
> FilterList filters = new FilterList(Operator.MUST_PASS_ALL);
> filters.addFilter(filter1);
> filters.addFilter(filter2);
> Get get = new Get(USER);
> get.addFamily(COLUMN_FAMILY);
> get.setMaxVersions(1);
> get.setFilter(filters);
> Somehow it didn't work as expected. It returned the entries as if the filter1
> were not set.
> Turns out the ColumnPrefixFilter returns SEEK_NEXT_USING_HINT in some cases.
> The FilterList filter does not handle this return code properly (treat it as
> INCLUDE).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5620) Convert the client protocol of HRegionInterface to PB
[ https://issues.apache.org/jira/browse/HBASE-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254961#comment-13254961 ] Todd Lipcon commented on HBASE-5620: Did anyone do any benchmarks here? It concerns me to have this committed without even knowing how it affects performance. > Convert the client protocol of HRegionInterface to PB > - > > Key: HBASE-5620 > URL: https://issues.apache.org/jira/browse/HBASE-5620 > Project: HBase > Issue Type: Sub-task > Components: ipc, master, migration, regionserver >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang > Fix For: 0.96.0 > > Attachments: hbase-5620-sec.patch, hbase-5620_v3.patch, > hbase-5620_v4.patch, hbase-5620_v4.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5782) Not all the regions are getting assigned after the log splitting.
[ https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254959#comment-13254959 ] Zhihong Yu commented on HBASE-5782: --- bq. I'd prefer solution that doesn't add a lock to patch something that's broken. I agree. I suggest the following actions: 1. pull back HBASE-4487 in 0.94 and trunk 2. a) spend major effort on HBASE-5699 (multiple WALs per region server) 2. b) make HBASE-4487 semantically correct 2.a and 2.b can proceed in parallel. But I think HBASE-5699 is the long-term solution. > Not all the regions are getting assigned after the log splitting. > - > > Key: HBASE-5782 > URL: https://issues.apache.org/jira/browse/HBASE-5782 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.94.0 >Reporter: Gopinathan A >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 0.94.0 > > Attachments: HBASE-5782.patch > > > Create a table with 1000 splits, after the region assignemnt, kill the > regionserver wich contains META table. > Here few regions are missing after the log splitting and region assigment. > HBCK report shows multiple region holes are got created. > Same scenario was verified mulitple times in 0.92.1, no issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Work started] (HBASE-5634) document how to use uberhbck
[ https://issues.apache.org/jira/browse/HBASE-5634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-5634 started by Jonathan Hsieh. > document how to use uberhbck > > > Key: HBASE-5634 > URL: https://issues.apache.org/jira/browse/HBASE-5634 > Project: HBase > Issue Type: Improvement > Components: documentation, hbck >Affects Versions: 0.90.7, 0.92.2, 0.94.0, 0.96.0 >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > Attachments: uber hbck docs.pdf > > > The updated hbck from HBASE-5128 introduces many new repair options and, as a > side effect, offers many new opportunities to durably shoot oneself in the > foot. Docs need to be written and added to the ref guide to explain its > usage and ramifications and discuss repair strategies. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5634) document how to use uberhbck
[ https://issues.apache.org/jira/browse/HBASE-5634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5634: -- Status: Patch Available (was: In Progress) > document how to use uberhbck > > > Key: HBASE-5634 > URL: https://issues.apache.org/jira/browse/HBASE-5634 > Project: HBase > Issue Type: Improvement > Components: documentation, hbck >Affects Versions: 0.90.7, 0.92.2, 0.94.0, 0.96.0 >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > Attachments: uber hbck docs.pdf > > > The updated hbck from HBASE-5128 introduces many new repair options and, as a > side effect, offers many new opportunities to durably shoot oneself in the > foot. Docs need to be written and added to the ref guide to explain its > usage and ramifications and discuss repair strategies. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5634) document how to use uberhbck
[ https://issues.apache.org/jira/browse/HBASE-5634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5634: -- Attachment: uber hbck docs.pdf I've attached documentation how to use the new uber hbck features and options. This is not meant to be included as a pdf -- however, the content is meant to be integrated into the hbase reference guide. (I haven't spent the time to figure out docbook yet). > document how to use uberhbck > > > Key: HBASE-5634 > URL: https://issues.apache.org/jira/browse/HBASE-5634 > Project: HBase > Issue Type: Improvement > Components: documentation, hbck >Affects Versions: 0.90.7, 0.92.2, 0.94.0, 0.96.0 >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > Attachments: uber hbck docs.pdf > > > The updated hbck from HBASE-5128 introduces many new repair options and, as a > side effect, offers many new opportunities to durably shoot oneself in the > foot. Docs need to be written and added to the ref guide to explain its > usage and ramifications and discuss repair strategies. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5620) Convert the client protocol of HRegionInterface to PB
[ https://issues.apache.org/jira/browse/HBASE-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254979#comment-13254979 ] Jimmy Xiang commented on HBASE-5620: I did some testing with YCSB (mostly inserts). It gave me better performance for the patch which was a surprise to me. I will do some read-only testing with YCSB too. > Convert the client protocol of HRegionInterface to PB > - > > Key: HBASE-5620 > URL: https://issues.apache.org/jira/browse/HBASE-5620 > Project: HBase > Issue Type: Sub-task > Components: ipc, master, migration, regionserver >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang > Fix For: 0.96.0 > > Attachments: hbase-5620-sec.patch, hbase-5620_v3.patch, > hbase-5620_v4.patch, hbase-5620_v4.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5782) Not all the regions are getting assigned after the log splitting.
[
https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254994#comment-13254994
]
Zhihong Yu commented on HBASE-5782:
---
HLog.appendNoSync() is used by
HRegion.{append|doMiniBatchPut|mutateRowsWithLocks}.
Those methods would be affected when HLog.appendNoSync() is removed.
> Not all the regions are getting assigned after the log splitting.
> -
>
> Key: HBASE-5782
> URL: https://issues.apache.org/jira/browse/HBASE-5782
> Project: HBase
> Issue Type: Bug
> Components: wal
>Affects Versions: 0.94.0
>Reporter: Gopinathan A
>Assignee: ramkrishna.s.vasudevan
>Priority: Blocker
> Fix For: 0.94.0
>
> Attachments: HBASE-5782.patch
>
>
> Create a table with 1000 splits, after the region assignemnt, kill the
> regionserver wich contains META table.
> Here few regions are missing after the log splitting and region assigment.
> HBCK report shows multiple region holes are got created.
> Same scenario was verified mulitple times in 0.92.1, no issues.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5104) Provide a reliable intra-row pagination mechanism
[
https://issues.apache.org/jira/browse/HBASE-5104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255008#comment-13255008
]
Hadoop QA commented on HBASE-5104:
--
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12522843/jira-HBASE-5104-Provide-a-reliable-intra-row-paginat-2012-04-16_12_39_42.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 11 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac
compiler warnings.
-1 findbugs. The patch appears to introduce 4 new Findbugs (version 1.3.9)
warnings.
+1 release audit. The applied patch does not increase the total number of
release audit warnings.
-1 core tests. The patch failed these unit tests:
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/1541//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/1541//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/1541//console
This message is automatically generated.
> Provide a reliable intra-row pagination mechanism
> -
>
> Key: HBASE-5104
> URL: https://issues.apache.org/jira/browse/HBASE-5104
> Project: HBase
> Issue Type: Bug
>Reporter: Kannan Muthukkaruppan
>Assignee: Madhuwanti Vaidya
> Attachments: D2799.1.patch, D2799.2.patch, D2799.3.patch,
> jira-HBASE-5104-Provide-a-reliable-intra-row-paginat-2012-04-16_12_39_42.patch,
> testFilterList.rb
>
>
> Addendum:
> Doing pagination (retrieving at most "limit" number of KVs at a particular
> "offset") is currently supported via the ColumnPaginationFilter. However, it
> is not a very clean way of supporting pagination. Some of the problems with
> it are:
> * Normally, one would expect a query with (Filter(A) AND Filter(B)) to have
> same results as (query with Filter(A)) INTERSECT (query with Filter(B)). This
> is not the case for ColumnPaginationFilter as its internal state gets updated
> depending on whether or not Filter(A) returns TRUE/FALSE for a particular
> cell.
> * When this Filter is used in combination with other filters (e.g., doing AND
> with another filter using FilterList), the behavior of the query depends on
> the order of filters in the FilterList. This is not ideal.
> * ColumnPaginationFilter is a stateful filter which ends up counting multiple
> versions of the cell as separate values even if another filter upstream or
> the ScanQueryMatcher is going to reject the value for other reasons.
> Seems like we need a reliable way to do pagination. The particular use case
> that prompted this JIRA is pagination within the same rowKey. For example,
> for a given row key R, get columns with prefix P, starting at offset X (among
> columns which have prefix P) and limit Y. Some possible fixes might be:
> 1) enhance ColumnPrefixFilter to support another constructor which supports
> limit/offset.
> 2) Support pagination (limit/offset) at the Scan/Get API level (rather than
> as a filter) [Like SQL].
> Original Post:
> Thanks Jiakai Liu for reporting this issue and doing the initial
> investigation. Email from Jiakai below:
> Assuming that we have an index column family with the following entries:
> "tag0:001:thread1"
> ...
> "tag1:001:thread1"
> "tag1:002:thread2"
> ...
> "tag1:010:thread10"
> ...
> "tag2:001:thread1"
> "tag2:005:thread5"
> ...
> To get threads with "tag1" in range [5, 10), I tried the following code:
> ColumnPrefixFilter filter1 = new
> ColumnPrefixFilter(Bytes.toBytes("tag1"));
> ColumnPaginationFilter filter2 = new ColumnPaginationFilter(5 /* limit
> */, 5 /* offset */);
> FilterList filters = new FilterList(Operator.MUST_PASS_ALL);
> filters.addFilter(filter1);
> filters.addFilter(filter2);
> Get get = new Get(USER);
> get.addFamily(COLUMN_FAMILY);
> get.setMaxVersions(1);
> get.setFilter(filters);
> Somehow it didn't work as expected. It returned the entries as if the filter1
> were not set.
> Turns out the ColumnPrefixFilter returns SEEK_NEXT_USING_HINT in some cases.
> The FilterList filter does not handle this return code properly (treat it as
> INCLUDE).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5788) Move Dynamic Metrics storage off of HRegion.
[ https://issues.apache.org/jira/browse/HBASE-5788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255014#comment-13255014 ] Elliott Clark commented on HBASE-5788: -- On trying to add more to this class it became obvious to me that it needs more love than the first try. I'll roll this into exposing per region metrics. > Move Dynamic Metrics storage off of HRegion. > > > Key: HBASE-5788 > URL: https://issues.apache.org/jira/browse/HBASE-5788 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: Elliott Clark >Assignee: Elliott Clark >Priority: Minor > Attachments: HBASE-5788-0.patch > > > HRegion right now has the responsibility of storing static counts and latency > numbers for use by the metrics package. Since these maps are incremented and > set from lots of places it makes adding functionality hard. > > So move the metrics functionality into SchemaMetrics making it more than just > a class for naming. The next step will be to simplify the api exposed so > that using it will be easier. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5795) hbase-3927 breaks 0.92<->0.94 compatibility
[
https://issues.apache.org/jira/browse/HBASE-5795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255012#comment-13255012
]
Hadoop QA commented on HBASE-5795:
--
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12522841/5795-v3.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 6 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac
compiler warnings.
-1 findbugs. The patch appears to introduce 4 new Findbugs (version 1.3.9)
warnings.
+1 release audit. The applied patch does not increase the total number of
release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.replication.TestReplication
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/1542//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/1542//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/1542//console
This message is automatically generated.
> hbase-3927 breaks 0.92<->0.94 compatibility
> ---
>
> Key: HBASE-5795
> URL: https://issues.apache.org/jira/browse/HBASE-5795
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: stack
> Fix For: 0.94.0, 0.96.0
>
> Attachments: 5795-v2.txt, 5795-v3.txt, 5795.unittest.txt
>
>
> This commit broke our 0.92/0.94 compatibility:
> {code}
>
> r1136686 | stack | 2011-06-16 14:18:08 -0700 (Thu, 16 Jun 2011) | 1 line
> HBASE-3927 display total uncompressed byte size of a region in web UI
> {code}
> I just tried the new RC for 0.94. I brought up a 0.94 master on a 0.92
> cluster and rather than just digest version 1 of the HServerLoad, I get this:
> {code}
> 2012-04-14 22:47:59,752 WARN org.apache.hadoop.ipc.HBaseServer: Unable to
> read call parameters for client 10.4.14.38
> java.io.IOException: Error in readFields
> at
> org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:684)
> at
> org.apache.hadoop.hbase.ipc.Invocation.readFields(Invocation.java:125)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.processData(HBaseServer.java:1269)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.readAndProcess(HBaseServer.java:1184)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:722)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:513)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:488)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: A record version mismatch occured. Expecting v2, found v1
> at
> org.apache.hadoop.io.VersionedWritable.readFields(VersionedWritable.java:46)
> at
> org.apache.hadoop.hbase.HServerLoad$RegionLoad.readFields(HServerLoad.java:379)
> at
> org.apache.hadoop.hbase.HServerLoad.readFields(HServerLoad.java:686)
> at
> org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:681)
> ... 9 more
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5795) HServerLoad$RegionLoad breaks 0.92<->0.94 compatibility
[
https://issues.apache.org/jira/browse/HBASE-5795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihong Yu updated HBASE-5795:
--
Assignee: Zhihong Yu (was: stack)
Summary: HServerLoad$RegionLoad breaks 0.92<->0.94 compatibility (was:
hbase-3927 breaks 0.92<->0.94 compatibility)
> HServerLoad$RegionLoad breaks 0.92<->0.94 compatibility
> ---
>
> Key: HBASE-5795
> URL: https://issues.apache.org/jira/browse/HBASE-5795
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: Zhihong Yu
> Fix For: 0.94.0, 0.96.0
>
> Attachments: 5795-v2.txt, 5795-v3.txt, 5795.unittest.txt
>
>
> This commit broke our 0.92/0.94 compatibility:
> {code}
>
> r1136686 | stack | 2011-06-16 14:18:08 -0700 (Thu, 16 Jun 2011) | 1 line
> HBASE-3927 display total uncompressed byte size of a region in web UI
> {code}
> I just tried the new RC for 0.94. I brought up a 0.94 master on a 0.92
> cluster and rather than just digest version 1 of the HServerLoad, I get this:
> {code}
> 2012-04-14 22:47:59,752 WARN org.apache.hadoop.ipc.HBaseServer: Unable to
> read call parameters for client 10.4.14.38
> java.io.IOException: Error in readFields
> at
> org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:684)
> at
> org.apache.hadoop.hbase.ipc.Invocation.readFields(Invocation.java:125)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.processData(HBaseServer.java:1269)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.readAndProcess(HBaseServer.java:1184)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:722)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:513)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:488)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: A record version mismatch occured. Expecting v2, found v1
> at
> org.apache.hadoop.io.VersionedWritable.readFields(VersionedWritable.java:46)
> at
> org.apache.hadoop.hbase.HServerLoad$RegionLoad.readFields(HServerLoad.java:379)
> at
> org.apache.hadoop.hbase.HServerLoad.readFields(HServerLoad.java:686)
> at
> org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:681)
> ... 9 more
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5792) HLog Performance Evaluation Tool
[ https://issues.apache.org/jira/browse/HBASE-5792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-5792: - Attachment: verify.txt verify.txt Matteo's patch only I moved the tool down under the wal package. Also added a verify step that will look at resultant WAL and verify we are writing in sequence. Oh, I also made the tool actually write a log (Was using default constructor on region which won't create a log -- have to create the log external and pass it in) > HLog Performance Evaluation Tool > > > Key: HBASE-5792 > URL: https://issues.apache.org/jira/browse/HBASE-5792 > Project: HBase > Issue Type: Test > Components: wal >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi >Priority: Minor > Labels: performance, wal > Fix For: 0.96.0 > > Attachments: HBASE-5792-v0.patch, HBASE-5792-v1.patch, > HBASE-5792-v2.patch, verify.txt, verify.txt > > > Related to HDFS-3280 and the HBase WAL slowdown on 0.23+ > It would be nice to have a simple tool like HFilePerformanceEvaluation, ... > to be able to check easily the HLog performance. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5792) HLog Performance Evaluation Tool
[ https://issues.apache.org/jira/browse/HBASE-5792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-5792: - Resolution: Fixed Fix Version/s: 0.96.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed to trunk. Thanks for the patch Matteo. > HLog Performance Evaluation Tool > > > Key: HBASE-5792 > URL: https://issues.apache.org/jira/browse/HBASE-5792 > Project: HBase > Issue Type: Test > Components: wal >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi >Priority: Minor > Labels: performance, wal > Fix For: 0.96.0 > > Attachments: HBASE-5792-v0.patch, HBASE-5792-v1.patch, > HBASE-5792-v2.patch, verify.txt, verify.txt > > > Related to HDFS-3280 and the HBase WAL slowdown on 0.23+ > It would be nice to have a simple tool like HFilePerformanceEvaluation, ... > to be able to check easily the HLog performance. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5795) HServerLoad$RegionLoad breaks 0.92<->0.94 compatibility
[
https://issues.apache.org/jira/browse/HBASE-5795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255032#comment-13255032
]
Zhihong Yu commented on HBASE-5795:
---
TestReplication failure isn't related to the patch.
Integrated patch v3 to 0.94 and trunk.
Thanks for finding the bug and providing the test, Stack.
Thanks for the review Stack and Lars.
> HServerLoad$RegionLoad breaks 0.92<->0.94 compatibility
> ---
>
> Key: HBASE-5795
> URL: https://issues.apache.org/jira/browse/HBASE-5795
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: Zhihong Yu
> Fix For: 0.94.0, 0.96.0
>
> Attachments: 5795-v2.txt, 5795-v3.txt, 5795.unittest.txt
>
>
> This commit broke our 0.92/0.94 compatibility:
> {code}
>
> r1136686 | stack | 2011-06-16 14:18:08 -0700 (Thu, 16 Jun 2011) | 1 line
> HBASE-3927 display total uncompressed byte size of a region in web UI
> {code}
> I just tried the new RC for 0.94. I brought up a 0.94 master on a 0.92
> cluster and rather than just digest version 1 of the HServerLoad, I get this:
> {code}
> 2012-04-14 22:47:59,752 WARN org.apache.hadoop.ipc.HBaseServer: Unable to
> read call parameters for client 10.4.14.38
> java.io.IOException: Error in readFields
> at
> org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:684)
> at
> org.apache.hadoop.hbase.ipc.Invocation.readFields(Invocation.java:125)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.processData(HBaseServer.java:1269)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.readAndProcess(HBaseServer.java:1184)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:722)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:513)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:488)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: A record version mismatch occured. Expecting v2, found v1
> at
> org.apache.hadoop.io.VersionedWritable.readFields(VersionedWritable.java:46)
> at
> org.apache.hadoop.hbase.HServerLoad$RegionLoad.readFields(HServerLoad.java:379)
> at
> org.apache.hadoop.hbase.HServerLoad.readFields(HServerLoad.java:686)
> at
> org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:681)
> ... 9 more
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5782) Not all the regions are getting assigned after the log splitting.
[ https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255034#comment-13255034 ] stack commented on HBASE-5782: -- I just committed a tool over on HBASE-5792. It tests WALs. If you pass the -verify flag, you'll see that even w/ just three threads, sequence ids are out of order. Could be useful verifying whatever fix we have here. > Not all the regions are getting assigned after the log splitting. > - > > Key: HBASE-5782 > URL: https://issues.apache.org/jira/browse/HBASE-5782 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.94.0 >Reporter: Gopinathan A >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 0.94.0 > > Attachments: HBASE-5782.patch > > > Create a table with 1000 splits, after the region assignemnt, kill the > regionserver wich contains META table. > Here few regions are missing after the log splitting and region assigment. > HBCK report shows multiple region holes are got created. > Same scenario was verified mulitple times in 0.92.1, no issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5780) Fix race in HBase regionserver startup vs ZK SASL authentication
[ https://issues.apache.org/jira/browse/HBASE-5780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaneal Manek updated HBASE-5780: - Attachment: TestReplicationPeer-output.log TestReplicationPeer-Security-output.log > Fix race in HBase regionserver startup vs ZK SASL authentication > > > Key: HBASE-5780 > URL: https://issues.apache.org/jira/browse/HBASE-5780 > Project: HBase > Issue Type: Bug > Components: security >Affects Versions: 0.92.1, 0.94.0 >Reporter: Shaneal Manek >Assignee: Shaneal Manek > Fix For: 0.92.2, 0.96.0, 0.94.1 > > Attachments: HBASE-5780-v2.patch, HBASE-5780.patch, > TestReplicationPeer-Security-output.log, TestReplicationPeer-output.log, > testoutput.tar.gz > > > Secure RegionServers sometimes fail to start with the following backtrace: > 2012-03-22 17:20:16,737 FATAL > org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server > centos60-20.ent.cloudera.com,60020,1332462015929: Unexpected exception during > initialization, aborting > org.apache.zookeeper.KeeperException$NoAuthException: KeeperErrorCode = > NoAuth for /hbase/shutdown > at org.apache.zookeeper.KeeperException.create(KeeperException.java:113) > at org.apache.zookeeper.KeeperException.create(KeeperException.java:51) > at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1131) > at > org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.getData(RecoverableZooKeeper.java:295) > at org.apache.hadoop.hbase.zookeeper.ZKUtil.getDataInternal(ZKUtil.java:518) > at org.apache.hadoop.hbase.zookeeper.ZKUtil.getDataAndWatch(ZKUtil.java:494) > at > org.apache.hadoop.hbase.zookeeper.ZooKeeperNodeTracker.start(ZooKeeperNodeTracker.java:77) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.initializeZooKeeper(HRegionServer.java:569) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.preRegistrationInitialization(HRegionServer.java:532) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:634) > at java.lang.Thread.run(Thread.java:662) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5780) Fix race in HBase regionserver startup vs ZK SASL authentication
[
https://issues.apache.org/jira/browse/HBASE-5780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255033#comment-13255033
]
Shaneal Manek commented on HBASE-5780:
--
Odd, that test seems to always pass for me (tested on 0.94 and 0.92). How are
you running your tests locally?
Both of the following pass for me (output attached too):
{noformat}
mvn -PlocalTests -Dtest=TestReplicationPeer -Psecurity clean test
mvn -PlocalTests -Dtest=TestReplicationPeer clean test
{noformat}
I'm looking into it in more detail now too.
> Fix race in HBase regionserver startup vs ZK SASL authentication
>
>
> Key: HBASE-5780
> URL: https://issues.apache.org/jira/browse/HBASE-5780
> Project: HBase
> Issue Type: Bug
> Components: security
>Affects Versions: 0.92.1, 0.94.0
>Reporter: Shaneal Manek
>Assignee: Shaneal Manek
> Fix For: 0.92.2, 0.96.0, 0.94.1
>
> Attachments: HBASE-5780-v2.patch, HBASE-5780.patch,
> TestReplicationPeer-Security-output.log, TestReplicationPeer-output.log,
> testoutput.tar.gz
>
>
> Secure RegionServers sometimes fail to start with the following backtrace:
> 2012-03-22 17:20:16,737 FATAL
> org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server
> centos60-20.ent.cloudera.com,60020,1332462015929: Unexpected exception during
> initialization, aborting
> org.apache.zookeeper.KeeperException$NoAuthException: KeeperErrorCode =
> NoAuth for /hbase/shutdown
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:113)
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1131)
> at
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.getData(RecoverableZooKeeper.java:295)
> at org.apache.hadoop.hbase.zookeeper.ZKUtil.getDataInternal(ZKUtil.java:518)
> at org.apache.hadoop.hbase.zookeeper.ZKUtil.getDataAndWatch(ZKUtil.java:494)
> at
> org.apache.hadoop.hbase.zookeeper.ZooKeeperNodeTracker.start(ZooKeeperNodeTracker.java:77)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.initializeZooKeeper(HRegionServer.java:569)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.preRegistrationInitialization(HRegionServer.java:532)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:634)
> at java.lang.Thread.run(Thread.java:662)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5621) Convert admin protocol of HRegionInterface to PB
[ https://issues.apache.org/jira/browse/HBASE-5621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255036#comment-13255036 ] jirapos...@reviews.apache.org commented on HBASE-5621: -- --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4714/ --- (Updated 2012-04-16 20:49:59.450802) Review request for hbase. Changes --- Addressed some of Stack's comments: 1. moved Client/AdminProtcol.java to client package, 2. moved some sharable pb calls to ProtobufUtil to avoid scattering pb everywhere. As to move some calls from HConnection to HCM, it is not as straightforward as I think. The main reason is becuase HConnection is not just a network connection to a server. Probably it is better to handle it in a separate jira. Summary --- This is the admin part of HBase-5443. AdminProtocol part. This addresses bug HBASE-5621. https://issues.apache.org/jira/browse/HBASE-5621 Diffs (updated) - security/src/main/java/org/apache/hadoop/hbase/ipc/SecureRpcEngine.java 8219bea src/main/java/org/apache/hadoop/hbase/catalog/CatalogTracker.java 408db79 src/main/java/org/apache/hadoop/hbase/client/AdminProtocol.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/client/ClientProtocol.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java ee16e72 src/main/java/org/apache/hadoop/hbase/client/HConnection.java 23f8e5a src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java 820e2a9 src/main/java/org/apache/hadoop/hbase/client/HTable.java 2c87d50 src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java fe80fcf src/main/java/org/apache/hadoop/hbase/client/ServerCallable.java 2a9d86e src/main/java/org/apache/hadoop/hbase/ipc/ExecRPCInvoker.java d71e97e src/main/java/org/apache/hadoop/hbase/ipc/HBaseRPC.java 3a3a79f src/main/java/org/apache/hadoop/hbase/ipc/Invocation.java b7afa58 src/main/java/org/apache/hadoop/hbase/ipc/RpcEngine.java dd0a1bc src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java 9f159f2 src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java d0570b9 src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java ab33ac7 src/main/java/org/apache/hadoop/hbase/master/ServerManager.java 70901fe src/main/java/org/apache/hadoop/hbase/protobuf/AdminProtocol.java 422e865 src/main/java/org/apache/hadoop/hbase/protobuf/ClientProtocol.java 3d6a23a src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java b056830 src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java a912cc3 src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java ecaf9fe src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java e78e56d src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 1c0541f src/main/java/org/apache/hadoop/hbase/regionserver/HRegionThriftServer.java 759633d src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java 9487a1c src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java 04fe8b6 src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java 61f8128 src/main/java/org/apache/hadoop/hbase/util/HBaseFsckRepair.java 83a165c src/main/protobuf/Admin.proto 132c5dd src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java d6ae0e2 src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditorNoCluster.java 3cfc02b src/test/java/org/apache/hadoop/hbase/client/HConnectionTestingUtility.java 8af0f91 src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java aa7f51b src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide3.java 7dd60de src/test/java/org/apache/hadoop/hbase/client/TestHTableUtil.java 24f878e src/test/java/org/apache/hadoop/hbase/mapreduce/TestLoadIncrementalHFilesSplitRecovery.java 301ee27 src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java a59e152 src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManager.java b84a115 src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java cedf31e src/test/java/org/apache/hadoop/hbase/master/TestMaster.java 3122b15 src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java c0ac12c src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionServerBulkLoad.java d0cad45 src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java 6b64f10 Diff: https://reviews.apache.org/r/4714/diff Testing --- All unit tests passed. Thanks, Jimmy > Convert admin protocol of HRegionInterface to PB > > > Key: HBASE-5621 > URL: ht
[jira] [Commented] (HBASE-5634) document how to use uberhbck
[ https://issues.apache.org/jira/browse/HBASE-5634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255041#comment-13255041 ] Hadoop QA commented on HBASE-5634: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12522847/uber+hbck+docs.pdf against trunk revision . +1 @author. The patch does not contain any @author tags. +0 tests included. The patch appears to be a documentation patch that doesn't require tests. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1543//console This message is automatically generated. > document how to use uberhbck > > > Key: HBASE-5634 > URL: https://issues.apache.org/jira/browse/HBASE-5634 > Project: HBase > Issue Type: Improvement > Components: documentation, hbck >Affects Versions: 0.90.7, 0.92.2, 0.94.0, 0.96.0 >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > Attachments: uber hbck docs.pdf > > > The updated hbck from HBASE-5128 introduces many new repair options and, as a > side effect, offers many new opportunities to durably shoot oneself in the > foot. Docs need to be written and added to the ref guide to explain its > usage and ramifications and discuss repair strategies. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5780) Fix race in HBase regionserver startup vs ZK SASL authentication
[ https://issues.apache.org/jira/browse/HBASE-5780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255056#comment-13255056 ] Hadoop QA commented on HBASE-5780: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12522856/TestReplicationPeer-output.log against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1544//console This message is automatically generated. > Fix race in HBase regionserver startup vs ZK SASL authentication > > > Key: HBASE-5780 > URL: https://issues.apache.org/jira/browse/HBASE-5780 > Project: HBase > Issue Type: Bug > Components: security >Affects Versions: 0.92.1, 0.94.0 >Reporter: Shaneal Manek >Assignee: Shaneal Manek > Fix For: 0.92.2, 0.96.0, 0.94.1 > > Attachments: HBASE-5780-v2.patch, HBASE-5780.patch, > TestReplicationPeer-Security-output.log, TestReplicationPeer-output.log, > testoutput.tar.gz > > > Secure RegionServers sometimes fail to start with the following backtrace: > 2012-03-22 17:20:16,737 FATAL > org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server > centos60-20.ent.cloudera.com,60020,1332462015929: Unexpected exception during > initialization, aborting > org.apache.zookeeper.KeeperException$NoAuthException: KeeperErrorCode = > NoAuth for /hbase/shutdown > at org.apache.zookeeper.KeeperException.create(KeeperException.java:113) > at org.apache.zookeeper.KeeperException.create(KeeperException.java:51) > at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1131) > at > org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.getData(RecoverableZooKeeper.java:295) > at org.apache.hadoop.hbase.zookeeper.ZKUtil.getDataInternal(ZKUtil.java:518) > at org.apache.hadoop.hbase.zookeeper.ZKUtil.getDataAndWatch(ZKUtil.java:494) > at > org.apache.hadoop.hbase.zookeeper.ZooKeeperNodeTracker.start(ZooKeeperNodeTracker.java:77) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.initializeZooKeeper(HRegionServer.java:569) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.preRegistrationInitialization(HRegionServer.java:532) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:634) > at java.lang.Thread.run(Thread.java:662) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5788) Move Dynamic Metrics storage off of HRegion.
[ https://issues.apache.org/jira/browse/HBASE-5788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-5788: - Attachment: HBASE-5788-1.patch Better version. SchemaMetrics wasn't the right place to put it. While it does a lot of the interaction there are lots of other places so a static MetricsStorage seems better. > Move Dynamic Metrics storage off of HRegion. > > > Key: HBASE-5788 > URL: https://issues.apache.org/jira/browse/HBASE-5788 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: Elliott Clark >Assignee: Elliott Clark >Priority: Minor > Attachments: HBASE-5788-0.patch, HBASE-5788-1.patch > > > HRegion right now has the responsibility of storing static counts and latency > numbers for use by the metrics package. Since these maps are incremented and > set from lots of places it makes adding functionality hard. > > So move the metrics functionality into SchemaMetrics making it more than just > a class for naming. The next step will be to simplify the api exposed so > that using it will be easier. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5634) document how to use uberhbck
[ https://issues.apache.org/jira/browse/HBASE-5634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255089#comment-13255089 ] stack commented on HBASE-5634: -- +1 Fix "The using the -details option will report" I'm glad you don't call it uberhbck in the doc (well, you joke about it -- thats ok) > document how to use uberhbck > > > Key: HBASE-5634 > URL: https://issues.apache.org/jira/browse/HBASE-5634 > Project: HBase > Issue Type: Improvement > Components: documentation, hbck >Affects Versions: 0.90.7, 0.92.2, 0.94.0, 0.96.0 >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > Attachments: uber hbck docs.pdf > > > The updated hbck from HBASE-5128 introduces many new repair options and, as a > side effect, offers many new opportunities to durably shoot oneself in the > foot. Docs need to be written and added to the ref guide to explain its > usage and ramifications and discuss repair strategies. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5795) HServerLoad$RegionLoad breaks 0.92<->0.94 compatibility
[
https://issues.apache.org/jira/browse/HBASE-5795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255090#comment-13255090
]
Hudson commented on HBASE-5795:
---
Integrated in HBase-TRUNK #2769 (See
[https://builds.apache.org/job/HBase-TRUNK/2769/])
HBASE-5795 HServerLoad$RegionLoad breaks 0.92<->0.94 compatibility
(Revision 1326794)
Result = SUCCESS
tedyu :
Files :
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/HServerLoad.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/HServerLoad092.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/TestSerialization.java
> HServerLoad$RegionLoad breaks 0.92<->0.94 compatibility
> ---
>
> Key: HBASE-5795
> URL: https://issues.apache.org/jira/browse/HBASE-5795
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: Zhihong Yu
> Fix For: 0.94.0, 0.96.0
>
> Attachments: 5795-v2.txt, 5795-v3.txt, 5795.unittest.txt
>
>
> This commit broke our 0.92/0.94 compatibility:
> {code}
>
> r1136686 | stack | 2011-06-16 14:18:08 -0700 (Thu, 16 Jun 2011) | 1 line
> HBASE-3927 display total uncompressed byte size of a region in web UI
> {code}
> I just tried the new RC for 0.94. I brought up a 0.94 master on a 0.92
> cluster and rather than just digest version 1 of the HServerLoad, I get this:
> {code}
> 2012-04-14 22:47:59,752 WARN org.apache.hadoop.ipc.HBaseServer: Unable to
> read call parameters for client 10.4.14.38
> java.io.IOException: Error in readFields
> at
> org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:684)
> at
> org.apache.hadoop.hbase.ipc.Invocation.readFields(Invocation.java:125)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.processData(HBaseServer.java:1269)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.readAndProcess(HBaseServer.java:1184)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:722)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:513)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:488)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: A record version mismatch occured. Expecting v2, found v1
> at
> org.apache.hadoop.io.VersionedWritable.readFields(VersionedWritable.java:46)
> at
> org.apache.hadoop.hbase.HServerLoad$RegionLoad.readFields(HServerLoad.java:379)
> at
> org.apache.hadoop.hbase.HServerLoad.readFields(HServerLoad.java:686)
> at
> org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:681)
> ... 9 more
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5788) Move Dynamic Metrics storage off of HRegion.
[ https://issues.apache.org/jira/browse/HBASE-5788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-5788: - Attachment: HBASE-5788-2.patch Added header to new file. > Move Dynamic Metrics storage off of HRegion. > > > Key: HBASE-5788 > URL: https://issues.apache.org/jira/browse/HBASE-5788 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: Elliott Clark >Assignee: Elliott Clark >Priority: Minor > Attachments: HBASE-5788-0.patch, HBASE-5788-1.patch, > HBASE-5788-2.patch > > > HRegion right now has the responsibility of storing static counts and latency > numbers for use by the metrics package. Since these maps are incremented and > set from lots of places it makes adding functionality hard. > > So move the metrics functionality into SchemaMetrics making it more than just > a class for naming. The next step will be to simplify the api exposed so > that using it will be easier. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5792) HLog Performance Evaluation Tool
[ https://issues.apache.org/jira/browse/HBASE-5792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255091#comment-13255091 ] Hudson commented on HBASE-5792: --- Integrated in HBase-TRUNK #2769 (See [https://builds.apache.org/job/HBase-TRUNK/2769/]) HBASE-5792 HLog Performance Evaluation Tool (Revision 1326793) HBASE-5792 HLog Performance Evaluation Tool (Revision 1326792) Result = SUCCESS stack : Files : * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/wal/HLogPerformanceEvaluation.java stack : Files : * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java > HLog Performance Evaluation Tool > > > Key: HBASE-5792 > URL: https://issues.apache.org/jira/browse/HBASE-5792 > Project: HBase > Issue Type: Test > Components: wal >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi >Priority: Minor > Labels: performance, wal > Fix For: 0.96.0 > > Attachments: HBASE-5792-v0.patch, HBASE-5792-v1.patch, > HBASE-5792-v2.patch, verify.txt, verify.txt > > > Related to HDFS-3280 and the HBase WAL slowdown on 0.23+ > It would be nice to have a simple tool like HFilePerformanceEvaluation, ... > to be able to check easily the HLog performance. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5782) Not all the regions are getting assigned after the log splitting.
[ https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255094#comment-13255094 ] Lars Hofhansl commented on HBASE-5782: -- The short term choices we have are: # revert HBASE-4528, HBASE-4487, and HBASE-5541 (are there others?) # Partially revert HBASE-2467 (or devise other ways to have strictly one thread flushing an HLog). Maybe Todd as the author of HBASE-2467 could chime in... Todd? > Not all the regions are getting assigned after the log splitting. > - > > Key: HBASE-5782 > URL: https://issues.apache.org/jira/browse/HBASE-5782 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.94.0 >Reporter: Gopinathan A >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 0.94.0 > > Attachments: HBASE-5782.patch > > > Create a table with 1000 splits, after the region assignemnt, kill the > regionserver wich contains META table. > Here few regions are missing after the log splitting and region assigment. > HBCK report shows multiple region holes are got created. > Same scenario was verified mulitple times in 0.92.1, no issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5782) Not all the regions are getting assigned after the log splitting.
[ https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-5782: - Attachment: 5782.txt Simple patch to ensure only one thread flushes the log. Don't hate me, just throwing this out there. > Not all the regions are getting assigned after the log splitting. > - > > Key: HBASE-5782 > URL: https://issues.apache.org/jira/browse/HBASE-5782 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.94.0 >Reporter: Gopinathan A >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 0.94.0 > > Attachments: 5782.txt, HBASE-5782.patch > > > Create a table with 1000 splits, after the region assignemnt, kill the > regionserver wich contains META table. > Here few regions are missing after the log splitting and region assigment. > HBCK report shows multiple region holes are got created. > Same scenario was verified mulitple times in 0.92.1, no issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5795) HServerLoad$RegionLoad breaks 0.92<->0.94 compatibility
[
https://issues.apache.org/jira/browse/HBASE-5795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255099#comment-13255099
]
Hudson commented on HBASE-5795:
---
Integrated in HBase-0.94 #120 (See
[https://builds.apache.org/job/HBase-0.94/120/])
HBASE-5795 HServerLoad$RegionLoad breaks 0.92<->0.94 compatibility
(Revision 1326791)
Result = SUCCESS
tedyu :
Files :
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/HServerLoad.java
* /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/HServerLoad092.java
*
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/TestSerialization.java
> HServerLoad$RegionLoad breaks 0.92<->0.94 compatibility
> ---
>
> Key: HBASE-5795
> URL: https://issues.apache.org/jira/browse/HBASE-5795
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: Zhihong Yu
> Fix For: 0.94.0, 0.96.0
>
> Attachments: 5795-v2.txt, 5795-v3.txt, 5795.unittest.txt
>
>
> This commit broke our 0.92/0.94 compatibility:
> {code}
>
> r1136686 | stack | 2011-06-16 14:18:08 -0700 (Thu, 16 Jun 2011) | 1 line
> HBASE-3927 display total uncompressed byte size of a region in web UI
> {code}
> I just tried the new RC for 0.94. I brought up a 0.94 master on a 0.92
> cluster and rather than just digest version 1 of the HServerLoad, I get this:
> {code}
> 2012-04-14 22:47:59,752 WARN org.apache.hadoop.ipc.HBaseServer: Unable to
> read call parameters for client 10.4.14.38
> java.io.IOException: Error in readFields
> at
> org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:684)
> at
> org.apache.hadoop.hbase.ipc.Invocation.readFields(Invocation.java:125)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.processData(HBaseServer.java:1269)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.readAndProcess(HBaseServer.java:1184)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:722)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:513)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:488)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: A record version mismatch occured. Expecting v2, found v1
> at
> org.apache.hadoop.io.VersionedWritable.readFields(VersionedWritable.java:46)
> at
> org.apache.hadoop.hbase.HServerLoad$RegionLoad.readFields(HServerLoad.java:379)
> at
> org.apache.hadoop.hbase.HServerLoad.readFields(HServerLoad.java:686)
> at
> org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:681)
> ... 9 more
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
