[jira] [Updated] (HBASE-8557) fix coverage org.apache.hadoop.hbase.rest.metrics
[ https://issues.apache.org/jira/browse/HBASE-8557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Gorshkov updated HBASE-8557: Attachment: HBASE-8557-0.94.patch > fix coverage org.apache.hadoop.hbase.rest.metrics > - > > Key: HBASE-8557 > URL: https://issues.apache.org/jira/browse/HBASE-8557 > Project: HBase > Issue Type: Test >Affects Versions: 0.94.9 >Reporter: Aleksey Gorshkov > Attachments: HBASE-8557-0.94.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8557) fix coverage org.apache.hadoop.hbase.rest.metrics
[ https://issues.apache.org/jira/browse/HBASE-8557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Gorshkov updated HBASE-8557: Attachment: (was: HBASE-8557-0.94.patch) > fix coverage org.apache.hadoop.hbase.rest.metrics > - > > Key: HBASE-8557 > URL: https://issues.apache.org/jira/browse/HBASE-8557 > Project: HBase > Issue Type: Test >Affects Versions: 0.94.9 >Reporter: Aleksey Gorshkov > Attachments: HBASE-8557-0.94.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8557) fix coverage org.apache.hadoop.hbase.rest.metrics
[ https://issues.apache.org/jira/browse/HBASE-8557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Gorshkov updated HBASE-8557: Status: Patch Available (was: Open) > fix coverage org.apache.hadoop.hbase.rest.metrics > - > > Key: HBASE-8557 > URL: https://issues.apache.org/jira/browse/HBASE-8557 > Project: HBase > Issue Type: Test >Reporter: Aleksey Gorshkov > Attachments: HBASE-8557-0.94.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8557) fix coverage org.apache.hadoop.hbase.rest.metrics
[ https://issues.apache.org/jira/browse/HBASE-8557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Gorshkov updated HBASE-8557: Attachment: HBASE-8557-0.94.patch > fix coverage org.apache.hadoop.hbase.rest.metrics > - > > Key: HBASE-8557 > URL: https://issues.apache.org/jira/browse/HBASE-8557 > Project: HBase > Issue Type: Test >Reporter: Aleksey Gorshkov > Attachments: HBASE-8557-0.94.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8557) fix coverage org.apache.hadoop.hbase.rest.metrics
[ https://issues.apache.org/jira/browse/HBASE-8557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Gorshkov updated HBASE-8557: Affects Version/s: 0.94.9 > fix coverage org.apache.hadoop.hbase.rest.metrics > - > > Key: HBASE-8557 > URL: https://issues.apache.org/jira/browse/HBASE-8557 > Project: HBase > Issue Type: Test >Affects Versions: 0.94.9 >Reporter: Aleksey Gorshkov > Attachments: HBASE-8557-0.94.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8557) fix coverage org.apache.hadoop.hbase.rest.metrics
[ https://issues.apache.org/jira/browse/HBASE-8557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Gorshkov updated HBASE-8557: Attachment: (was: HBASE-8557-0.94.patch) > fix coverage org.apache.hadoop.hbase.rest.metrics > - > > Key: HBASE-8557 > URL: https://issues.apache.org/jira/browse/HBASE-8557 > Project: HBase > Issue Type: Test >Reporter: Aleksey Gorshkov > Attachments: HBASE-8557-0.94.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8557) fix coverage org.apache.hadoop.hbase.rest.metrics
[ https://issues.apache.org/jira/browse/HBASE-8557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Gorshkov updated HBASE-8557: Attachment: HBASE-8557-0.94.patch > fix coverage org.apache.hadoop.hbase.rest.metrics > - > > Key: HBASE-8557 > URL: https://issues.apache.org/jira/browse/HBASE-8557 > Project: HBase > Issue Type: Test >Reporter: Aleksey Gorshkov > Attachments: HBASE-8557-0.94.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8557) fix coverage org.apache.hadoop.hbase.rest.metrics
[ https://issues.apache.org/jira/browse/HBASE-8557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13660385#comment-13660385 ] Aleksey Gorshkov commented on HBASE-8557: - this patch for branch 0.94 only. > fix coverage org.apache.hadoop.hbase.rest.metrics > - > > Key: HBASE-8557 > URL: https://issues.apache.org/jira/browse/HBASE-8557 > Project: HBase > Issue Type: Test >Reporter: Aleksey Gorshkov > Attachments: HBASE-8557-0.94.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8462) Custom timestamps should not be allowed to be negative
[ https://issues.apache.org/jira/browse/HBASE-8462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13660378#comment-13660378 ] Lars Hofhansl commented on HBASE-8462: -- I see. Also, rather that checking on the client, should be check on the server and throw back a DoNotRetryIOException? > Custom timestamps should not be allowed to be negative > -- > > Key: HBASE-8462 > URL: https://issues.apache.org/jira/browse/HBASE-8462 > Project: HBase > Issue Type: Bug > Components: Client >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: 0.98.0, 0.94.8, 0.95.1 > > Attachments: hbase-8462_v1.patch, hbase-8462_v2.patch > > > Client supplied timestamps should not be allowed to be negative, otherwise > unpredictable results will follow. Especially, since we are encoding the ts > using Bytes.Bytes(long), negative timestamps are sorted after positive ones. > Plus, the new PB messages define ts' as uint64. > Credit goes to Huned Lokhandwala for reporting this. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8561) [replication] Don't instantiate a ReplicationSource if the passed implementation isn't found
[
https://issues.apache.org/jira/browse/HBASE-8561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-8561:
-
Fix Version/s: (was: 0.94.8)
0.94.9
Seems safe to do this in 0.94.9.
> [replication] Don't instantiate a ReplicationSource if the passed
> implementation isn't found
>
>
> Key: HBASE-8561
> URL: https://issues.apache.org/jira/browse/HBASE-8561
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 0.94.6.1
>Reporter: Jean-Daniel Cryans
> Fix For: 0.98.0, 0.95.2, 0.94.9
>
>
> I was debugging a case where the region servers were dying with:
> {noformat}
> ABORTING region server someserver.com,60020,1368123702806: Writing
> replication status
> org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode =
> NoNode for
> /hbase/replication/rs/someserver.com,60020,1368123702806/etcetcetc/somserver.com%2C60020%2C1368123702740.1368123705091
>
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> at org.apache.zookeeper.ZooKeeper.setData(ZooKeeper.java:1266)
> at
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.setData(RecoverableZooKeeper.java:354)
>
> at org.apache.hadoop.hbase.zookeeper.ZKUtil.setData(ZKUtil.java:846)
> at org.apache.hadoop.hbase.zookeeper.ZKUtil.setData(ZKUtil.java:898)
> at org.apache.hadoop.hbase.zookeeper.ZKUtil.setData(ZKUtil.java:892)
> at
> org.apache.hadoop.hbase.replication.ReplicationZookeeper.writeReplicationStatus(ReplicationZookeeper.java:558)
>
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.logPositionAndCleanOldLogs(ReplicationSourceManager.java:154)
>
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.shipEdits(ReplicationSource.java:638)
>
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.run(ReplicationSource.java:387)
> {noformat}
> Turns out the problem really was:
> {noformat}
> 2013-05-09 11:21:45,625 WARN
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager:
> Passed replication source implementation throws errors, defaulting to
> ReplicationSource
> java.lang.ClassNotFoundException: Some.Other.ReplicationSource.Implementation
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:186)
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.getReplicationSource(ReplicationSourceManager.java:324)
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.addSource(ReplicationSourceManager.java:202)
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.init(ReplicationSourceManager.java:174)
> at
> org.apache.hadoop.hbase.replication.regionserver.Replication.startReplicationService(Replication.java:171)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.startServiceThreads(HRegionServer.java:1583)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.handleReportForDutyResponse(HRegionServer.java:1042)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:698)
> at java.lang.Thread.run(Thread.java:722)
> {noformat}
> So I think instantiating a ReplicationSource here is wrong and makes it
> harder to debug.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8454) TestImportTsv failing due to configuration issues
[
https://issues.apache.org/jira/browse/HBASE-8454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-8454:
-
Fix Version/s: (was: 0.94.8)
0.94.9
Moving to 0.94.9
> TestImportTsv failing due to configuration issues
> -
>
> Key: HBASE-8454
> URL: https://issues.apache.org/jira/browse/HBASE-8454
> Project: HBase
> Issue Type: Sub-task
> Components: mapreduce, test
>Affects Versions: 0.98.0, 0.94.8, 0.95.1
>Reporter: Andrew Purtell
> Fix For: 0.98.0, 0.95.1, 0.94.9
>
> Attachments: 8454-branch-0.94.patch, 8454.patch
>
>
> It looks like TestImportTsv can be fixed once HBASE-8453 is applied with this
> additional delta:
> {noformat}
> diff --git
> a/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportTsv.java
> b/src/test/java/org/
> index 77d044b..0da6f0a 100644
> --- a/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportTsv.java
> +++ b/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportTsv.java
> @@ -26,6 +26,7 @@ import java.util.ArrayList;
> import org.apache.commons.logging.Log;
> import org.apache.commons.logging.LogFactory;
> import org.apache.hadoop.hbase.*;
> +import org.apache.hadoop.mapred.JobConf;
> import org.apache.hadoop.mapreduce.Job;
> import org.apache.hadoop.fs.FSDataOutputStream;
> import org.apache.hadoop.fs.Path;
> @@ -289,7 +290,10 @@ public class TestImportTsv {
> LOG.info("set the hbaseAdmin");
> ImportTsv.createHbaseAdmin(conf);
>}
> - Job job = ImportTsv.createSubmittableJob(conf, args);
> +
> + JobConf jobConf = htu1.getMRCluster().createJobConf();
> + HBaseConfiguration.merge(jobConf, conf);
> + Job job = ImportTsv.createSubmittableJob(jobConf, args);
>job.waitForCompletion(false);
>assertTrue(job.isSuccessful());
> {noformat}
> Tested with Hadoop 2.0.4 + HBase 0.94.7.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8453) TestImportExport failing again due to configuration issues
[
https://issues.apache.org/jira/browse/HBASE-8453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-8453:
-
Fix Version/s: (was: 0.94.8)
0.94.9
Moving to 0.94.9.
> TestImportExport failing again due to configuration issues
> --
>
> Key: HBASE-8453
> URL: https://issues.apache.org/jira/browse/HBASE-8453
> Project: HBase
> Issue Type: Sub-task
> Components: mapreduce, test
>Affects Versions: 0.98.0, 0.94.8, 0.95.1
>Reporter: Andrew Purtell
> Fix For: 0.98.0, 0.95.1, 0.94.9
>
> Attachments: 8453.patch, 8453-v1-0.94.patch
>
>
> TestImportExport fails for me with a connection refused exception:
> {noformat}
> java.lang.reflect.UndeclaredThrowableException
> at
> org.apache.hadoop.yarn.exceptions.impl.pb.YarnRemoteExceptionPBImpl.unwrapAndThrowException(YarnRemoteExceptionPBImpl.java:135)
> at
> org.apache.hadoop.yarn.api.impl.pb.client.ClientRMProtocolPBClientImpl.getNewApplication(ClientRMProtocolPBClientImpl.java:162)
> at
> org.apache.hadoop.yarn.client.YarnClientImpl.getNewApplication(YarnClientImpl.java:121)
> at
> org.apache.hadoop.mapred.ResourceMgrDelegate.getNewJobID(ResourceMgrDelegate.java:107)
> at org.apache.hadoop.mapred.YARNRunner.getNewJobID(YARNRunner.java:231)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:352)
> [...]
> Caused by: com.google.protobuf.ServiceException: java.net.ConnectException:
> Call From ip-10-174-75-236/10.174.75.236 to 0.0.0.0:8032 failed on connection
> exception: java.net.ConnectException: Connection refused; For more details
> see: http://wiki.apache.org/hadoop/ConnectionRefused
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:212)
> at com.sun.proxy.$Proxy89.getNewApplication(Unknown Source)
> at
> org.apache.hadoop.yarn.api.impl.pb.client.ClientRMProtocolPBClientImpl.getNewApplication(ClientRMProtocolPBClientImpl.java:159)
> ... 42 more
> {noformat}
> Settings in the MiniMRCluster configuration are not properly propagated in
> this test.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8353) -ROOT-/.META. regions are hanging if master restarted while closing -ROOT-/.META. regions on dead RS
[
https://issues.apache.org/jira/browse/HBASE-8353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-8353:
-
Fix Version/s: (was: 0.94.8)
0.94.9
Kicking it down to the road to 0.94.9
> -ROOT-/.META. regions are hanging if master restarted while closing
> -ROOT-/.META. regions on dead RS
>
>
> Key: HBASE-8353
> URL: https://issues.apache.org/jira/browse/HBASE-8353
> Project: HBase
> Issue Type: Bug
> Components: Region Assignment
>Affects Versions: 0.94.6
>Reporter: rajeshbabu
>Assignee: rajeshbabu
> Fix For: 0.94.9
>
> Attachments: HBASE-8353_94_2.patch, HBASE-8353_94_3.patch,
> HBASE-8353_94_4.patch, HBASE-8353_94.patch
>
>
> ROOT/META are not getting assigned if master restarted while closing
> ROOT/META.
> Lets suppose catalog table regions in M_ZK_REGION_CLOSING state during master
> initialization and then just we are adding the them to RIT and waiting for
> TM. {code}
> if (isOnDeadServer(regionInfo, deadServers) &&
> (data.getOrigin() == null ||
> !serverManager.isServerOnline(data.getOrigin( {
> // If was on dead server, its closed now. Force to OFFLINE and this
> // will get it reassigned if appropriate
> forceOffline(regionInfo, data);
> } else {
> // Just insert region into RIT.
> // If this never updates the timeout will trigger new assignment
> regionsInTransition.put(encodedRegionName, new RegionState(
> regionInfo, RegionState.State.CLOSING,
> data.getStamp(), data.getOrigin()));
> }
> {code}
> isOnDeadServer always return false to ROOT/META because deadServers is null.
> Even TM cannot close them properly because its not available in online
> regions since its not yet assigned.
> {code}
> synchronized (this.regions) {
> // Check if this region is currently assigned
> if (!regions.containsKey(region)) {
> LOG.debug("Attempted to unassign region " +
> region.getRegionNameAsString() + " but it is not " +
> "currently assigned anywhere");
> return;
> }
> }
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7896) make rename_table working in 92/94
[ https://issues.apache.org/jira/browse/HBASE-7896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-7896: - Fix Version/s: (was: 0.94.8) 0.94.9 Moving to 0.94.9. Can we just add Matteo's workaround to the book? > make rename_table working in 92/94 > -- > > Key: HBASE-7896 > URL: https://issues.apache.org/jira/browse/HBASE-7896 > Project: HBase > Issue Type: Bug > Components: scripts >Affects Versions: 0.92.2, 0.94.5 >Reporter: Tianying Chang >Assignee: Tianying Chang > Fix For: 0.92.3, 0.94.9 > > Attachments: rename_table.rb > > > The rename_table function is very useful for our customers. However, > rename_table.rb does not work for 92/94. It has several bugs. It will be > useful to fix them so that users can solve their problems. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7983) FuzzyRowFilter is not added to REST model
[ https://issues.apache.org/jira/browse/HBASE-7983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-7983: - Fix Version/s: (was: 0.94.8) Still no patch. Removing from 0.94. Feel free to re-add. > FuzzyRowFilter is not added to REST model > - > > Key: HBASE-7983 > URL: https://issues.apache.org/jira/browse/HBASE-7983 > Project: HBase > Issue Type: Bug > Components: Filters, REST >Affects Versions: 0.94.5, 0.95.0 >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 0.98.0 > > > I can not see this in ScannerModel -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7709) Infinite loop possible in Master/Master replication
[ https://issues.apache.org/jira/browse/HBASE-7709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-7709: - Fix Version/s: (was: 0.94.8) 0.94.9 Would #6 still allow rolling upgrades from a prio version of HBase. It looks like it would not since we have to increase the HLogKey version. Moving to 0.94.9. > Infinite loop possible in Master/Master replication > --- > > Key: HBASE-7709 > URL: https://issues.apache.org/jira/browse/HBASE-7709 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 0.94.6, 0.95.1 >Reporter: Lars Hofhansl > Fix For: 0.98.0, 0.95.1, 0.94.9 > > > We just discovered the following scenario: > # Cluster A and B are setup in master/master replication > # By accident we had Cluster C replicate to Cluster A. > Now all edit originating from C will be bouncing between A and B. Forever! > The reason is that when the edit come in from C the cluster ID is already set > and won't be reset. > We have a couple of options here: > # Optionally only support master/master (not cycles of more than two > clusters). In that case we can always reset the cluster ID in the > ReplicationSource. That means that now cycles > 2 will have the data cycle > forever. This is the only option that requires no changes in the HLog format. > # Instead of a single cluster id per edit maintain a (unordered) set of > cluster id that have seen this edit. Then in ReplicationSource we drop any > edit that the sink has seen already. The is the cleanest approach, but it > might need a lot of data stored per edit if there are many clusters involved. > # Maintain a configurable counter of the maximum cycle side we want to > support. Could default to 10 (even maybe even just). Store a hop-count in the > WAL and the ReplicationSource increases that hop-count on each hop. If we're > over the max, just drop the edit. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-7210) Backport HBASE-6059 to 0.94
[ https://issues.apache.org/jira/browse/HBASE-7210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl resolved HBASE-7210. -- Resolution: Later Fix Version/s: (was: 0.94.8) Closing as "Later". Please reopen if you feel so inclined. > Backport HBASE-6059 to 0.94 > --- > > Key: HBASE-7210 > URL: https://issues.apache.org/jira/browse/HBASE-7210 > Project: HBase > Issue Type: Bug >Affects Versions: 0.94.2 >Reporter: ramkrishna.s.vasudevan > Attachments: 6059-94.patch, 7120.txt > > > HBASE-6059 seems to be an important issue. Chunhui has already given a patch > for 94. Need to rebase if it does not apply cleanly. > Raising a new one as the old issue is already closed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8400) Load coprocessor jar from various protocols (HTTP, HTTPS, FTP, etc.)
[ https://issues.apache.org/jira/browse/HBASE-8400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-8400: - Fix Version/s: (was: 0.94.8) 0.94.9 Moving out to 0.94.9. > Load coprocessor jar from various protocols (HTTP, HTTPS, FTP, etc.) > > > Key: HBASE-8400 > URL: https://issues.apache.org/jira/browse/HBASE-8400 > Project: HBase > Issue Type: Improvement > Components: Coprocessors >Affects Versions: 0.94.3, 0.98.0 >Reporter: Julian Zhou >Assignee: Julian Zhou >Priority: Minor > Fix For: 0.98.0, 0.95.1, 0.94.9 > > > In some application testing and production environment, after developed > coprocessors and generated jars, currently we need to transfer them to hdfs > first, then specify the URI in table descriptor to point to HDFS compatible > addresses of jars. Common used HTTP/HTTPS/FTP or other protocols were not > supported yet? To save time and make the life easier without transferring > from http/ftp or other servers, just modified the CoprocessorHost to use > http/ftp url connection (http and ftp servers are the most cases need to be > support) to stream jars to coprocessor jar spaces automatically. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-8538) HBaseAdmin#isTableEnabled() should check table existence before checking zk state.
[ https://issues.apache.org/jira/browse/HBASE-8538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl resolved HBASE-8538. -- Resolution: Fixed Hadoop Flags: Reviewed Committed to 0.94 (including comment suggested by Matteo) > HBaseAdmin#isTableEnabled() should check table existence before checking zk > state. > -- > > Key: HBASE-8538 > URL: https://issues.apache.org/jira/browse/HBASE-8538 > Project: HBase > Issue Type: Bug > Components: Admin >Reporter: rajeshbabu >Assignee: rajeshbabu > Fix For: 0.94.8 > > Attachments: HBASE-8538_94.patch > > > To avoid compatibility issues with older versions HBaseAdmin#isTableEnabled > returning true even if the table state is null. Its also returning true even > a table is not present. We should confirm table existence from .META. before > checking in zk. If table not present or deleted, then It will throw > TableNotFoundException. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8564) TestMetricsRegionServer depends on test order
[ https://issues.apache.org/jira/browse/HBASE-8564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13660353#comment-13660353 ] Hudson commented on HBASE-8564: --- Integrated in HBase-TRUNK #4126 (See [https://builds.apache.org/job/HBase-TRUNK/4126/]) HBASE-8564 TestMetricsRegionServer depends on test order (Revision 1483617) Result = SUCCESS eclark : Files : * /hbase/trunk/hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/CompatibilitySingletonFactory.java * /hbase/trunk/hbase-hadoop-compat/src/test/java/org/apache/hadoop/hbase/RandomStringGenerator.java * /hbase/trunk/hbase-hadoop-compat/src/test/java/org/apache/hadoop/hbase/RandomStringGeneratorImpl.java * /hbase/trunk/hbase-hadoop-compat/src/test/java/org/apache/hadoop/hbase/TestCompatibilitySingletonFactory.java * /hbase/trunk/hbase-hadoop-compat/src/test/resources * /hbase/trunk/hbase-hadoop-compat/src/test/resources/META-INF * /hbase/trunk/hbase-hadoop-compat/src/test/resources/META-INF/services * /hbase/trunk/hbase-hadoop-compat/src/test/resources/META-INF/services/org.apache.hadoop.hbase.RandomStringGenerator * /hbase/trunk/hbase-hadoop1-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerSourceFactoryImpl.java * /hbase/trunk/hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionAggregateSourceImpl.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegion.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServer.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMetricsRegionServer.java > TestMetricsRegionServer depends on test order > - > > Key: HBASE-8564 > URL: https://issues.apache.org/jira/browse/HBASE-8564 > Project: HBase > Issue Type: Bug >Affects Versions: 0.98.0, 0.95.0 >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 0.98.0, 0.95.1 > > Attachments: HBASE-8564-0.patch, HBASE-8564-1.patch > > > TestMetricsRegionServer has failed a few times on trunk. It's passing > depends upon test ordering so will fail sometimes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8564) TestMetricsRegionServer depends on test order
[ https://issues.apache.org/jira/browse/HBASE-8564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13660352#comment-13660352 ] Hudson commented on HBASE-8564: --- Integrated in hbase-0.95 #200 (See [https://builds.apache.org/job/hbase-0.95/200/]) HBASE-8564 TestMetricsRegionServer depends on test order (Revision 1483619) Result = SUCCESS eclark : Files : * /hbase/branches/0.95/hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/CompatibilitySingletonFactory.java * /hbase/branches/0.95/hbase-hadoop-compat/src/test/java/org/apache/hadoop/hbase/RandomStringGenerator.java * /hbase/branches/0.95/hbase-hadoop-compat/src/test/java/org/apache/hadoop/hbase/RandomStringGeneratorImpl.java * /hbase/branches/0.95/hbase-hadoop-compat/src/test/java/org/apache/hadoop/hbase/TestCompatibilitySingletonFactory.java * /hbase/branches/0.95/hbase-hadoop-compat/src/test/resources * /hbase/branches/0.95/hbase-hadoop-compat/src/test/resources/META-INF * /hbase/branches/0.95/hbase-hadoop-compat/src/test/resources/META-INF/services * /hbase/branches/0.95/hbase-hadoop-compat/src/test/resources/META-INF/services/org.apache.hadoop.hbase.RandomStringGenerator * /hbase/branches/0.95/hbase-hadoop1-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerSourceFactoryImpl.java * /hbase/branches/0.95/hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionAggregateSourceImpl.java * /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegion.java * /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServer.java * /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMetricsRegionServer.java > TestMetricsRegionServer depends on test order > - > > Key: HBASE-8564 > URL: https://issues.apache.org/jira/browse/HBASE-8564 > Project: HBase > Issue Type: Bug >Affects Versions: 0.98.0, 0.95.0 >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 0.98.0, 0.95.1 > > Attachments: HBASE-8564-0.patch, HBASE-8564-1.patch > > > TestMetricsRegionServer has failed a few times on trunk. It's passing > depends upon test ordering so will fail sometimes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8563) Double count of read requests for Gets
[ https://issues.apache.org/jira/browse/HBASE-8563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13660350#comment-13660350 ] Lars Hofhansl commented on HBASE-8563: -- +1 > Double count of read requests for Gets > --- > > Key: HBASE-8563 > URL: https://issues.apache.org/jira/browse/HBASE-8563 > Project: HBase > Issue Type: Bug >Reporter: Francis Liu >Assignee: Francis Liu > Fix For: 0.94.8 > > Attachments: HBASE-8563_94.patch > > > Whenever a RegionScanner is created via HRegion.getScanner(), the read > request count is incremented. Since get is implemented as a scan internally. > Each Get request is counted twice. Scans will have an extra count as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8539) Double(or tripple ...) ZooKeeper listeners of the same type when Master recovers from ZK SessionExpiredException
[ https://issues.apache.org/jira/browse/HBASE-8539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13660349#comment-13660349 ] Lars Hofhansl commented on HBASE-8539: -- In the interest of keeping HBase in a releasable state let's either commit the addendum or revert the original patch until we have a full solution no later than tomorrow. > Double(or tripple ...) ZooKeeper listeners of the same type when Master > recovers from ZK SessionExpiredException > > > Key: HBASE-8539 > URL: https://issues.apache.org/jira/browse/HBASE-8539 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 0.98.0, 0.94.7, 0.95.0 >Reporter: Jeffrey Zhong >Assignee: Jeffrey Zhong > Fix For: 0.98.0, 0.94.8, 0.95.1 > > Attachments: 8539-addendum-v2.patch, double-registered listeners.png, > hbase-8539-0.94-addendum.patch, hbase-8539-0.94.patch, > hbase-8539-addendum.patch, hbase-8539.patch, hbase-8539.patch > > > When Master tries to recover from zookeeper session expired exceptions, we > don't clean old registered listener instances. Therefore, it may end up we > have two(or more) listeners to double handling same events. Attached a screen > shot from debugger to show the issue. > I considered to limit one listener per class while I think that would limit > the listener usage so I choose to clear exiting listeners during recovery for > the fix. > (This issue is unrelated to the issue HBASE-8365 because I verified there is > no dup-listeners when HBASE-8365 happened) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8420) Port HBASE-6874 Implement prefetching for scanners from 0.89-fb
[ https://issues.apache.org/jira/browse/HBASE-8420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13660341#comment-13660341 ] Lars Hofhansl commented on HBASE-8420: -- Should we prefetch in multiples of the HDFS block size? > Port HBASE-6874 Implement prefetching for scanners from 0.89-fb > - > > Key: HBASE-8420 > URL: https://issues.apache.org/jira/browse/HBASE-8420 > Project: HBase > Issue Type: Improvement >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang > Attachments: 0.94-8420_v1.patch, 0.94-8420_v2.patch, > trunk-8420_v1.patch, trunk-8420_v2.patch, trunk-8420_v3.patch, > trunk-8420_v4.patch > > > This should help scanner performance. We should have it in trunk. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8545) Meta stuck in transition when it is assigned to a just restarted dead region sever
[
https://issues.apache.org/jira/browse/HBASE-8545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13660327#comment-13660327
]
ramkrishna.s.vasudevan commented on HBASE-8545:
---
@Jimmy
Going thro the logs
{code}
[main] master.ServerManager(736): New admin connection to
RSVASUDE-MOBL.gar.corp.intel.com,52745,1368765576099
2013-05-17 10:09:54,939 INFO [PRI IPC Server handler 0 on 52745]
regionserver.HRegionServer(3456): Received request to open region:
testAssignRegionOnRestartedServer,A,1368765580368.4c096a61387cfc7d63df0be881875f03.
on RSVASUDE-MOBL.gar.corp.intel.com,52745,1368765576199
{code}
The server ending with 099 is the dead server and the server ending with 199 is
the actual new alive server. So should we validate this on the RS side and
make it FAILED_OPEN so that master can carry out the assignment with a new
plan. This is a case where an old server goes down and a new server
immediately comes up in the same node.
> Meta stuck in transition when it is assigned to a just restarted dead region
> sever
> ---
>
> Key: HBASE-8545
> URL: https://issues.apache.org/jira/browse/HBASE-8545
> Project: HBase
> Issue Type: Bug
> Components: Region Assignment
>Reporter: Jimmy Xiang
>Assignee: Jimmy Xiang
> Attachments: trunk-8545.patch, trunk-8545_v2.patch
>
>
> Support the meta region server is down, and the SSH tries to re-assign it.
> This could happen:
> 1. AM plans to assign meta to a region server (R_old);
> 2. Now R_old is dead, the new region server (R_new) starts up on the same
> host, port, but gets a different start code;
> 3. AM sends the open region request to R_new and the Meta is opened on it;
> 4. AM gets ZK event, but it is from a different region server instance
> (R_new), not the expected one (R_old), so it sends a close region request to
> R_new;
> 5. Now, the meta is stuck in transition and won't be assigned.
> This won't happen to a user region since the SSH for R_old will find out the
> user region stuck in transition and re-assign it. For meta, it is a little
> different. AM checks if a dead region server carries the meta based on the
> ZK info, which is changed to the new region server R_new at step 3 by the
> open region handler.
> The fix I was thinking about is:
> 1. In checking if a region server carries a region, uses the region
> transition information if it exists (which is the source of truth, to
> master), if not, checks the ZK data as before;
> 2. In open region handler, when transition assign zk node from offline to
> opening, make sure the current region server is the expected one
> (ZK#transitionNode, existing code doesn't check the target server name).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8539) Double(or tripple ...) ZooKeeper listeners of the same type when Master recovers from ZK SessionExpiredException
[
https://issues.apache.org/jira/browse/HBASE-8539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13660320#comment-13660320
]
Hadoop QA commented on HBASE-8539:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12583606/8539-addendum-v2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.TestZooKeeper
org.apache.hadoop.hbase.zookeeper.lock.TestZKInterProcessReadWriteLock
{color:red}-1 core zombie tests{color}. There are 2 zombie test(s):
at
org.apache.hadoop.hbase.master.TestActiveMasterManager.testRestartMaster(TestActiveMasterManager.java:98)
at
org.apache.hadoop.hbase.zookeeper.TestZooKeeperNodeTracker.testNodeTracker(TestZooKeeperNodeTracker.java:145)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/5734//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5734//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5734//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5734//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5734//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5734//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5734//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5734//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5734//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/5734//console
This message is automatically generated.
> Double(or tripple ...) ZooKeeper listeners of the same type when Master
> recovers from ZK SessionExpiredException
>
>
> Key: HBASE-8539
> URL: https://issues.apache.org/jira/browse/HBASE-8539
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 0.98.0, 0.94.7, 0.95.0
>Reporter: Jeffrey Zhong
>Assignee: Jeffrey Zhong
> Fix For: 0.98.0, 0.94.8, 0.95.1
>
> Attachments: 8539-addendum-v2.patch, double-registered listeners.png,
> hbase-8539-0.94-addendum.patch, hbase-8539-0.94.patch,
> hbase-8539-addendum.patch, hbase-8539.patch, hbase-8539.patch
>
>
> When Master tries to recover from zookeeper session expired exceptions, we
> don't clean old registered listener instances. Therefore, it may end up we
> have two(or more) listeners to double handling same events. Attached a screen
> shot from debugger to show the issue.
> I considered to limit one listener per class while I think that would limit
> the listener usage so I choose to clear exiting listeners during recovery for
> the fix.
> (This issue is unrelated to the issue HBASE-8365 because I verified there is
> no dup-listeners when HBASE-8365 happened)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8567) TestDistributedLogSplitting#testLogReplayForDisablingTable fails on hadoop 2.0
[
https://issues.apache.org/jira/browse/HBASE-8567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13660317#comment-13660317
]
Hadoop QA commented on HBASE-8567:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12583604/hbase-8567.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.master.TestTableLockManager
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/5733//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5733//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5733//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5733//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5733//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5733//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5733//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5733//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5733//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/5733//console
This message is automatically generated.
> TestDistributedLogSplitting#testLogReplayForDisablingTable fails on hadoop 2.0
> --
>
> Key: HBASE-8567
> URL: https://issues.apache.org/jira/browse/HBASE-8567
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Jeffrey Zhong
> Fix For: 0.98.0, 0.95.1
>
> Attachments: hbase-8567.patch
>
>
> The test failed with:
> {code}
> testLogReplayForDisablingTable(org.apache.hadoop.hbase.master.TestDistributedLogSplitting)
> Time elapsed: 0.144 sec <<< ERROR!
> java.io.FileNotFoundException: File
> hdfs://localhost:56794/user/hortonzy/hbase/disableTable/6676df27ed855d9c61ad20ed650700e7/recovered.edits
> does not exist.
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:405)
> at
> org.apache.hadoop.hbase.master.TestDistributedLogSplitting.testLogReplayForDisablingTable(TestDistributedLogSplitting.java:774)
> {code}
> This was due to FileNotFoundException being thrown out of listStatus() for
> non-existent directory in hadoop 2.0.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8547) Fix java.lang.RuntimeException: Cached an already cached block
[
https://issues.apache.org/jira/browse/HBASE-8547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13660288#comment-13660288
]
Hudson commented on HBASE-8547:
---
Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #533 (See
[https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/533/])
HBASE-8547. Fix java.lang.RuntimeException: Cached an already cached block
(Addendum2 to check cached buffers for equality) (Revision 1483476)
Result = FAILURE
enis :
Files :
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java
> Fix java.lang.RuntimeException: Cached an already cached block
> --
>
> Key: HBASE-8547
> URL: https://issues.apache.org/jira/browse/HBASE-8547
> Project: HBase
> Issue Type: Bug
> Components: io, regionserver
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 0.98.0, 0.94.8, 0.95.1
>
> Attachments: hbase-8547_v1-0.94.patch, hbase-8547_v1-0.94.patch,
> hbase-8547_v1.patch, hbase-8547_v2-0.94-reduced.patch,
> hbase-8547_v2-addendum2.patch, hbase-8547_v2-addendum2.patch,
> hbase-8547_v2-trunk.patch
>
>
> In one test, one of the region servers received the following on 0.94.
> Note HalfStoreFileReader in the stack trace. I think the root cause is that
> after the region is split, the mid point can be in the middle of the block
> (for store files that the mid point is not chosen from). Each half store file
> tries to load the half block and put it in the block cache. Since IdLock is
> instantiated per store file reader, they do not share the same IdLock
> instance, thus does not lock against each other effectively.
> {code}
> 2013-05-12 01:30:37,733 ERROR
> org.apache.hadoop.hbase.regionserver.HRegionServer:ยท
> java.lang.RuntimeException: Cached an already cached block
> at
> org.apache.hadoop.hbase.io.hfile.LruBlockCache.cacheBlock(LruBlockCache.java:279)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:353)
> at
> org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.loadDataBlockWithScanInfo(HFileBlockIndex.java:254)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.seekTo(HFileReaderV2.java:480)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.seekTo(HFileReaderV2.java:501)
> at
> org.apache.hadoop.hbase.io.HalfStoreFileReader$1.seekTo(HalfStoreFileReader.java:237)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.seekAtOrAfter(StoreFileScanner.java:226)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.seek(StoreFileScanner.java:145)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.enforceSeek(StoreFileScanner.java:351)
> at
> org.apache.hadoop.hbase.regionserver.KeyValueHeap.pollRealKV(KeyValueHeap.java:354)
> at
> org.apache.hadoop.hbase.regionserver.KeyValueHeap.generalizedSeek(KeyValueHeap.java:312)
> at
> org.apache.hadoop.hbase.regionserver.KeyValueHeap.requestSeek(KeyValueHeap.java:277)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.reseek(StoreScanner.java:543)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:411)
> at
> org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:143)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.populateResult(HRegion.java:3829)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:3896)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:3778)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:3770)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:2643)
> at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.hadoop.hbase.ipc.SecureRpcEngine$Server.call(SecureRpcEngine.java:308)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1426)
> {code}
> I can see two possible fixes:
> # Allow this kind of rare cases in LruBlockCache by not throwing an
> exception.
> # Move the lock instances to upper layer (possibly in CacheConfig), and let
> half hfile readers share the same IdLock implementation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8461) Provide the ability to delete multiple snapshots through single command
[ https://issues.apache.org/jira/browse/HBASE-8461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13660289#comment-13660289 ] Hudson commented on HBASE-8461: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #533 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/533/]) HBASE-8461 Provide the ability to delete multiple snapshots through single command (Ted Yu) (Revision 1483574) Result = FAILURE tedyu : Files : * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestSnapshotFromClient.java > Provide the ability to delete multiple snapshots through single command > --- > > Key: HBASE-8461 > URL: https://issues.apache.org/jira/browse/HBASE-8461 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu >Assignee: Ted Yu > Fix For: 0.98.0, 0.95.1 > > Attachments: 8461-v1.txt, 8461-v2.txt, 8461-v3.txt, 8461-v4.txt > > > Currently HBaseAdmin#deleteSnapshot() accepts name of single snapshot. > It is desirable to allow user to delete multiple snapshots by issuing one > command. > e.g. user may use regular expression to specify the names of snapshots to > delete. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8505) References to split daughters should not be deleted separately from parent META entry
[
https://issues.apache.org/jira/browse/HBASE-8505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13660286#comment-13660286
]
Hudson commented on HBASE-8505:
---
Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #533 (See
[https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/533/])
HBASE-8505 References to split daughters should not be deleted separately
from parent META entry (Revision 1483481)
Result = FAILURE
enis :
Files :
*
/hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/MetaScanner.java
*
/hbase/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/util/Bytes.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/catalog/MetaEditor.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMetaScanner.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java
> References to split daughters should not be deleted separately from parent
> META entry
> -
>
> Key: HBASE-8505
> URL: https://issues.apache.org/jira/browse/HBASE-8505
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 0.98.0, 0.94.8, 0.95.1
>
> Attachments: hbase-8505_v1-0.94.patch, hbase-8505_v2-0.94.patch,
> hbase-8505_v2-0.94-reduced.patch, hbase-8505_v2.patch, hbase-8505_v2.patch
>
>
> In CatalogJanitor, we clean up the parent regions whose daughters does not
> have any more references to their parent regions. In doing so, we do two
> Delete's one for removing the split daughter columns, and the other for
> removing the row.
> The first one seems unnecessary, and causes NPE from concurrent MetaScanner.
> Stack trace:
> {code}
> 2013-05-07 04:49:40,828|machine|INFO|Exception in thread "main"
> java.lang.NullPointerException
> 2013-05-07 04:49:40,828|machine|INFO|at
> org.apache.hadoop.hbase.util.Writables.getWritable(Writables.java:103)
> 2013-05-07 04:49:40,828|machine|INFO|at
> org.apache.hadoop.hbase.util.Writables.getHRegionInfo(Writables.java:147)
> 2013-05-07 04:49:40,829|machine|INFO|at
> org.apache.hadoop.hbase.client.MetaScanner$BlockingMetaScannerVisitor.processRow(MetaScanner.java:406)
> 2013-05-07 04:49:40,829|machine|INFO|at
> org.apache.hadoop.hbase.client.MetaScanner$TableMetaScannerVisitor.processRow(MetaScanner.java:487)
> 2013-05-07 04:49:40,830|machine|INFO|at
> org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:224)
> 2013-05-07 04:49:40,830|machine|INFO|at
> org.apache.hadoop.hbase.client.MetaScanner.access$000(MetaScanner.java:54)
> 2013-05-07 04:49:40,830|machine|INFO|at
> org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:133)
> 2013-05-07 04:49:40,831|machine|INFO|at
> org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:130)
> 2013-05-07 04:49:40,831|machine|INFO|at
> org.apache.hadoop.hbase.client.HConnectionManager.execute(HConnectionManager.java:384)
> 2013-05-07 04:49:40,831|machine|INFO|at
> org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:130)
> 2013-05-07 04:49:40,832|machine|INFO|at
> org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:105)
> 2013-05-07 04:49:40,832|machine|INFO|at
> org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:83)
> 2013-05-07 04:49:40,832|machine|INFO|at
> org.apache.hadoop.hbase.client.MetaScanner.allTableRegions(MetaScanner.java:323)
> 2013-05-07 04:49:40,833|machine|INFO|at
> org.apache.hadoop.hbase.client.HTable.getRegionLocations(HTable.java:485)
> 2013-05-07 04:49:40,833|machine|INFO|at
> org.apache.hadoop.hbase.client.HTable.getStartEndKeys(HTable.java:438)
> {code}
> Master is doing the CatalogJanitor concurrently:
> {code}
> 2013-05-07 04:49:40,636 INFO org.apache.hadoop.hbase.catalog.MetaEditor:
> Deleted daughters references, qualifier=splitA and qualifier=splitB, from
> parent
> IntegrationTestBigLinkedList,\x07\xFB\x98\xB7a\x89\xF5\xE6,1367898577620.4ef1329ff0e8911db998ac8ccd32108d.
> 2013-05-07 04:49:40,666 INFO org.apache.hadoop.hbase.catalog.MetaEditor:
> Deleted region
> IntegrationTestBigLinkedList,\x07\xFB\x98\xB7a\x89\xF5\xE6,1367898577620.4ef1329ff0e8911db998ac8ccd32108d.
> from META
> 2013-05-07 04:49:40,690 INFO org.apache.hadoop.hbase.catalog.MetaEditor:
> Deleted daughters references, qualifier=splitA and qualifier=splitB, from
> parent
> IntegrationTestBigLinkedList,\x0B\xF8n\xEA\xD3\xAA\xA9\x92,1367898577620.b502376df2623cb0be3f0c1664d799a6.
> 2013-05-07 04:49:40,716 INFO org.apache.hadoop.hbase.catalog.MetaEditor:
> Deleted region
> IntegrationTestB
[jira] [Commented] (HBASE-8560) TestMasterShutdown failing in trunk 0.95/trunk -- "Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error)"
[ https://issues.apache.org/jira/browse/HBASE-8560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13660287#comment-13660287 ] Hudson commented on HBASE-8560: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #533 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/533/]) HBASE-8560 TestMasterShutdown failing in trunk 0.95/trunk -- "Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error)" (Revision 1483547) Result = FAILURE stack : Files : * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java > TestMasterShutdown failing in trunk 0.95/trunk -- "Unable to get data of > znode /hbase/meta-region-server because node does not exist (not an error)" > > > Key: HBASE-8560 > URL: https://issues.apache.org/jira/browse/HBASE-8560 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: Jeffrey Zhong > Fix For: 0.98.0, 0.95.1 > > Attachments: hbase-8560.patch, hbase-8560-v2.patch > > > I'm looking at this too... [~jeffreyz] you are too? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8564) TestMetricsRegionServer depends on test order
[ https://issues.apache.org/jira/browse/HBASE-8564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13660285#comment-13660285 ] Hudson commented on HBASE-8564: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #533 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/533/]) HBASE-8564 TestMetricsRegionServer depends on test order (Revision 1483617) Result = FAILURE eclark : Files : * /hbase/trunk/hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/CompatibilitySingletonFactory.java * /hbase/trunk/hbase-hadoop-compat/src/test/java/org/apache/hadoop/hbase/RandomStringGenerator.java * /hbase/trunk/hbase-hadoop-compat/src/test/java/org/apache/hadoop/hbase/RandomStringGeneratorImpl.java * /hbase/trunk/hbase-hadoop-compat/src/test/java/org/apache/hadoop/hbase/TestCompatibilitySingletonFactory.java * /hbase/trunk/hbase-hadoop-compat/src/test/resources * /hbase/trunk/hbase-hadoop-compat/src/test/resources/META-INF * /hbase/trunk/hbase-hadoop-compat/src/test/resources/META-INF/services * /hbase/trunk/hbase-hadoop-compat/src/test/resources/META-INF/services/org.apache.hadoop.hbase.RandomStringGenerator * /hbase/trunk/hbase-hadoop1-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerSourceFactoryImpl.java * /hbase/trunk/hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionAggregateSourceImpl.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegion.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServer.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMetricsRegionServer.java > TestMetricsRegionServer depends on test order > - > > Key: HBASE-8564 > URL: https://issues.apache.org/jira/browse/HBASE-8564 > Project: HBase > Issue Type: Bug >Affects Versions: 0.98.0, 0.95.0 >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 0.98.0, 0.95.1 > > Attachments: HBASE-8564-0.patch, HBASE-8564-1.patch > > > TestMetricsRegionServer has failed a few times on trunk. It's passing > depends upon test ordering so will fail sometimes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8539) Double(or tripple ...) ZooKeeper listeners of the same type when Master recovers from ZK SessionExpiredException
[ https://issues.apache.org/jira/browse/HBASE-8539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13660284#comment-13660284 ] Sergey Shelukhin commented on HBASE-8539: - This is exactly why I commented above about dangerous defaults :) Can you please remove the one-parameter method, so that every caller has to decide explicitly, and add javadoc? I will run the tests meanwhile... > Double(or tripple ...) ZooKeeper listeners of the same type when Master > recovers from ZK SessionExpiredException > > > Key: HBASE-8539 > URL: https://issues.apache.org/jira/browse/HBASE-8539 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 0.98.0, 0.94.7, 0.95.0 >Reporter: Jeffrey Zhong >Assignee: Jeffrey Zhong > Fix For: 0.98.0, 0.94.8, 0.95.1 > > Attachments: 8539-addendum-v2.patch, double-registered listeners.png, > hbase-8539-0.94-addendum.patch, hbase-8539-0.94.patch, > hbase-8539-addendum.patch, hbase-8539.patch, hbase-8539.patch > > > When Master tries to recover from zookeeper session expired exceptions, we > don't clean old registered listener instances. Therefore, it may end up we > have two(or more) listeners to double handling same events. Attached a screen > shot from debugger to show the issue. > I considered to limit one listener per class while I think that would limit > the listener usage so I choose to clear exiting listeners during recovery for > the fix. > (This issue is unrelated to the issue HBASE-8365 because I verified there is > no dup-listeners when HBASE-8365 happened) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8567) TestDistributedLogSplitting#testLogReplayForDisablingTable fails on hadoop 2.0
[
https://issues.apache.org/jira/browse/HBASE-8567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-8567:
--
Fix Version/s: 0.95.1
0.98.0
Status: Patch Available (was: Open)
> TestDistributedLogSplitting#testLogReplayForDisablingTable fails on hadoop 2.0
> --
>
> Key: HBASE-8567
> URL: https://issues.apache.org/jira/browse/HBASE-8567
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Jeffrey Zhong
> Fix For: 0.98.0, 0.95.1
>
> Attachments: hbase-8567.patch
>
>
> The test failed with:
> {code}
> testLogReplayForDisablingTable(org.apache.hadoop.hbase.master.TestDistributedLogSplitting)
> Time elapsed: 0.144 sec <<< ERROR!
> java.io.FileNotFoundException: File
> hdfs://localhost:56794/user/hortonzy/hbase/disableTable/6676df27ed855d9c61ad20ed650700e7/recovered.edits
> does not exist.
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:405)
> at
> org.apache.hadoop.hbase.master.TestDistributedLogSplitting.testLogReplayForDisablingTable(TestDistributedLogSplitting.java:774)
> {code}
> This was due to FileNotFoundException being thrown out of listStatus() for
> non-existent directory in hadoop 2.0.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8539) Double(or tripple ...) ZooKeeper listeners of the same type when Master recovers from ZK SessionExpiredException
[ https://issues.apache.org/jira/browse/HBASE-8539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-8539: -- Attachment: 8539-addendum-v2.patch Addendum v2 fixes TestZKProcedureControllers > Double(or tripple ...) ZooKeeper listeners of the same type when Master > recovers from ZK SessionExpiredException > > > Key: HBASE-8539 > URL: https://issues.apache.org/jira/browse/HBASE-8539 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 0.98.0, 0.94.7, 0.95.0 >Reporter: Jeffrey Zhong >Assignee: Jeffrey Zhong > Fix For: 0.98.0, 0.94.8, 0.95.1 > > Attachments: 8539-addendum-v2.patch, double-registered listeners.png, > hbase-8539-0.94-addendum.patch, hbase-8539-0.94.patch, > hbase-8539-addendum.patch, hbase-8539.patch, hbase-8539.patch > > > When Master tries to recover from zookeeper session expired exceptions, we > don't clean old registered listener instances. Therefore, it may end up we > have two(or more) listeners to double handling same events. Attached a screen > shot from debugger to show the issue. > I considered to limit one listener per class while I think that would limit > the listener usage so I choose to clear exiting listeners during recovery for > the fix. > (This issue is unrelated to the issue HBASE-8365 because I verified there is > no dup-listeners when HBASE-8365 happened) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8567) TestDistributedLogSplitting#testLogReplayForDisablingTable fails on hadoop 2.0
[
https://issues.apache.org/jira/browse/HBASE-8567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeffrey Zhong updated HBASE-8567:
-
Attachment: hbase-8567.patch
The failure is due to the reason that hadoop 2.0.4 throws FileNotFoundException
when a folder doesn't exists when using listStatus
> TestDistributedLogSplitting#testLogReplayForDisablingTable fails on hadoop 2.0
> --
>
> Key: HBASE-8567
> URL: https://issues.apache.org/jira/browse/HBASE-8567
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
> Attachments: hbase-8567.patch
>
>
> The test failed with:
> {code}
> testLogReplayForDisablingTable(org.apache.hadoop.hbase.master.TestDistributedLogSplitting)
> Time elapsed: 0.144 sec <<< ERROR!
> java.io.FileNotFoundException: File
> hdfs://localhost:56794/user/hortonzy/hbase/disableTable/6676df27ed855d9c61ad20ed650700e7/recovered.edits
> does not exist.
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:405)
> at
> org.apache.hadoop.hbase.master.TestDistributedLogSplitting.testLogReplayForDisablingTable(TestDistributedLogSplitting.java:774)
> {code}
> This was due to FileNotFoundException being thrown out of listStatus() for
> non-existent directory in hadoop 2.0.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-8567) TestDistributedLogSplitting#testLogReplayForDisablingTable fails on hadoop 2.0
[
https://issues.apache.org/jira/browse/HBASE-8567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeffrey Zhong reassigned HBASE-8567:
Assignee: Jeffrey Zhong
> TestDistributedLogSplitting#testLogReplayForDisablingTable fails on hadoop 2.0
> --
>
> Key: HBASE-8567
> URL: https://issues.apache.org/jira/browse/HBASE-8567
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Jeffrey Zhong
> Attachments: hbase-8567.patch
>
>
> The test failed with:
> {code}
> testLogReplayForDisablingTable(org.apache.hadoop.hbase.master.TestDistributedLogSplitting)
> Time elapsed: 0.144 sec <<< ERROR!
> java.io.FileNotFoundException: File
> hdfs://localhost:56794/user/hortonzy/hbase/disableTable/6676df27ed855d9c61ad20ed650700e7/recovered.edits
> does not exist.
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:405)
> at
> org.apache.hadoop.hbase.master.TestDistributedLogSplitting.testLogReplayForDisablingTable(TestDistributedLogSplitting.java:774)
> {code}
> This was due to FileNotFoundException being thrown out of listStatus() for
> non-existent directory in hadoop 2.0.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8563) Double count of read requests for Gets
[ https://issues.apache.org/jira/browse/HBASE-8563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13660277#comment-13660277 ] Ted Yu commented on HBASE-8563: --- TestMultiSlaveReplication#testMultiSlaveReplication fails locally with or without patch. I couldn't reproduce the other test failures. Will integrate tomorrow if there is no further review comment. > Double count of read requests for Gets > --- > > Key: HBASE-8563 > URL: https://issues.apache.org/jira/browse/HBASE-8563 > Project: HBase > Issue Type: Bug >Reporter: Francis Liu >Assignee: Francis Liu > Fix For: 0.94.7 > > Attachments: HBASE-8563_94.patch > > > Whenever a RegionScanner is created via HRegion.getScanner(), the read > request count is incremented. Since get is implemented as a scan internally. > Each Get request is counted twice. Scans will have an extra count as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8563) Double count of read requests for Gets
[ https://issues.apache.org/jira/browse/HBASE-8563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-8563: -- Fix Version/s: (was: 0.94.7) 0.94.8 Hadoop Flags: Reviewed > Double count of read requests for Gets > --- > > Key: HBASE-8563 > URL: https://issues.apache.org/jira/browse/HBASE-8563 > Project: HBase > Issue Type: Bug >Reporter: Francis Liu >Assignee: Francis Liu > Fix For: 0.94.8 > > Attachments: HBASE-8563_94.patch > > > Whenever a RegionScanner is created via HRegion.getScanner(), the read > request count is incremented. Since get is implemented as a scan internally. > Each Get request is counted twice. Scans will have an extra count as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8420) Port HBASE-6874 Implement prefetching for scanners from 0.89-fb
[ https://issues.apache.org/jira/browse/HBASE-8420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13660276#comment-13660276 ] Jimmy Xiang commented on HBASE-8420: Patch v4 changed the default prefetched result size cap from 512M to 256M; fixed a the prefetched result size counter leakage in closeScanner(); added some tests to make sure the cap is followed, i.e. prefetching only if the cap is not met; and added some stuff to make the new test possible. > Port HBASE-6874 Implement prefetching for scanners from 0.89-fb > - > > Key: HBASE-8420 > URL: https://issues.apache.org/jira/browse/HBASE-8420 > Project: HBase > Issue Type: Improvement >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang > Attachments: 0.94-8420_v1.patch, 0.94-8420_v2.patch, > trunk-8420_v1.patch, trunk-8420_v2.patch, trunk-8420_v3.patch, > trunk-8420_v4.patch > > > This should help scanner performance. We should have it in trunk. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7679) implement store file management for stripe compactions
[
https://issues.apache.org/jira/browse/HBASE-7679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13660265#comment-13660265
]
Hadoop QA commented on HBASE-7679:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12583576/HBASE-7679-v13.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 9 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.TestFullLogReconstruction
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/5732//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5732//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5732//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5732//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5732//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5732//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5732//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5732//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5732//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/5732//console
This message is automatically generated.
> implement store file management for stripe compactions
> --
>
> Key: HBASE-7679
> URL: https://issues.apache.org/jira/browse/HBASE-7679
> Project: HBase
> Issue Type: Sub-task
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: HBASE-7667-and-7603-v0-incomplete.patch,
> HBASE-7667-and-7603-v0-incomplete.patch, HBASE-7667-and-7603-v1.patch,
> HBASE-7667-and-7603-v1.patch, HBASE-7667-v1.patch, HBASE-7667-v1.patch,
> HBASE-7667-v2.patch, HBASE-7667-v2.patch, HBASE-7667-v3.patch,
> HBASE-7679-v10.patch, HBASE-7679-v11.patch, HBASE-7679-v12.patch,
> HBASE-7679-v12.patch, HBASE-7679-v13.patch, HBASE-7679-v13.patch,
> HBASE-7679-v4.patch, HBASE-7679-v5.patch, HBASE-7679-v6.patch,
> HBASE-7679-v7-.patch, HBASE-7679-v7.patch, HBASE-7679-v8.patch,
> HBASE-7679-v9.patch
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8547) Fix java.lang.RuntimeException: Cached an already cached block
[
https://issues.apache.org/jira/browse/HBASE-8547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13660262#comment-13660262
]
Hudson commented on HBASE-8547:
---
Integrated in hbase-0.95-on-hadoop2 #102 (See
[https://builds.apache.org/job/hbase-0.95-on-hadoop2/102/])
HBASE-8547. Fix java.lang.RuntimeException: Cached an already cached block
(Addendum2 to check cached buffers for equality) (Revision 1483478)
Result = FAILURE
enis :
Files :
*
/hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java
> Fix java.lang.RuntimeException: Cached an already cached block
> --
>
> Key: HBASE-8547
> URL: https://issues.apache.org/jira/browse/HBASE-8547
> Project: HBase
> Issue Type: Bug
> Components: io, regionserver
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 0.98.0, 0.94.8, 0.95.1
>
> Attachments: hbase-8547_v1-0.94.patch, hbase-8547_v1-0.94.patch,
> hbase-8547_v1.patch, hbase-8547_v2-0.94-reduced.patch,
> hbase-8547_v2-addendum2.patch, hbase-8547_v2-addendum2.patch,
> hbase-8547_v2-trunk.patch
>
>
> In one test, one of the region servers received the following on 0.94.
> Note HalfStoreFileReader in the stack trace. I think the root cause is that
> after the region is split, the mid point can be in the middle of the block
> (for store files that the mid point is not chosen from). Each half store file
> tries to load the half block and put it in the block cache. Since IdLock is
> instantiated per store file reader, they do not share the same IdLock
> instance, thus does not lock against each other effectively.
> {code}
> 2013-05-12 01:30:37,733 ERROR
> org.apache.hadoop.hbase.regionserver.HRegionServer:ยท
> java.lang.RuntimeException: Cached an already cached block
> at
> org.apache.hadoop.hbase.io.hfile.LruBlockCache.cacheBlock(LruBlockCache.java:279)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:353)
> at
> org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.loadDataBlockWithScanInfo(HFileBlockIndex.java:254)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.seekTo(HFileReaderV2.java:480)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.seekTo(HFileReaderV2.java:501)
> at
> org.apache.hadoop.hbase.io.HalfStoreFileReader$1.seekTo(HalfStoreFileReader.java:237)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.seekAtOrAfter(StoreFileScanner.java:226)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.seek(StoreFileScanner.java:145)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.enforceSeek(StoreFileScanner.java:351)
> at
> org.apache.hadoop.hbase.regionserver.KeyValueHeap.pollRealKV(KeyValueHeap.java:354)
> at
> org.apache.hadoop.hbase.regionserver.KeyValueHeap.generalizedSeek(KeyValueHeap.java:312)
> at
> org.apache.hadoop.hbase.regionserver.KeyValueHeap.requestSeek(KeyValueHeap.java:277)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.reseek(StoreScanner.java:543)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:411)
> at
> org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:143)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.populateResult(HRegion.java:3829)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:3896)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:3778)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:3770)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:2643)
> at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.hadoop.hbase.ipc.SecureRpcEngine$Server.call(SecureRpcEngine.java:308)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1426)
> {code}
> I can see two possible fixes:
> # Allow this kind of rare cases in LruBlockCache by not throwing an
> exception.
> # Move the lock instances to upper layer (possibly in CacheConfig), and let
> half hfile readers share the same IdLock implementation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8461) Provide the ability to delete multiple snapshots through single command
[ https://issues.apache.org/jira/browse/HBASE-8461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13660263#comment-13660263 ] Hudson commented on HBASE-8461: --- Integrated in hbase-0.95-on-hadoop2 #102 (See [https://builds.apache.org/job/hbase-0.95-on-hadoop2/102/]) HBASE-8461 Provide the ability to delete multiple snapshots through single command (Ted Yu) (Revision 1483576) Result = FAILURE tedyu : Files : * /hbase/branches/0.95/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java * /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestSnapshotFromClient.java > Provide the ability to delete multiple snapshots through single command > --- > > Key: HBASE-8461 > URL: https://issues.apache.org/jira/browse/HBASE-8461 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu >Assignee: Ted Yu > Fix For: 0.98.0, 0.95.1 > > Attachments: 8461-v1.txt, 8461-v2.txt, 8461-v3.txt, 8461-v4.txt > > > Currently HBaseAdmin#deleteSnapshot() accepts name of single snapshot. > It is desirable to allow user to delete multiple snapshots by issuing one > command. > e.g. user may use regular expression to specify the names of snapshots to > delete. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8560) TestMasterShutdown failing in trunk 0.95/trunk -- "Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error)"
[ https://issues.apache.org/jira/browse/HBASE-8560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13660261#comment-13660261 ] Hudson commented on HBASE-8560: --- Integrated in hbase-0.95-on-hadoop2 #102 (See [https://builds.apache.org/job/hbase-0.95-on-hadoop2/102/]) HBASE-8560 TestMasterShutdown failing in trunk 0.95/trunk -- "Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error)" (Revision 1483548) Result = FAILURE stack : Files : * /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java > TestMasterShutdown failing in trunk 0.95/trunk -- "Unable to get data of > znode /hbase/meta-region-server because node does not exist (not an error)" > > > Key: HBASE-8560 > URL: https://issues.apache.org/jira/browse/HBASE-8560 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: Jeffrey Zhong > Fix For: 0.98.0, 0.95.1 > > Attachments: hbase-8560.patch, hbase-8560-v2.patch > > > I'm looking at this too... [~jeffreyz] you are too? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8505) References to split daughters should not be deleted separately from parent META entry
[
https://issues.apache.org/jira/browse/HBASE-8505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13660260#comment-13660260
]
Hudson commented on HBASE-8505:
---
Integrated in hbase-0.95-on-hadoop2 #102 (See
[https://builds.apache.org/job/hbase-0.95-on-hadoop2/102/])
HBASE-8505 References to split daughters should not be deleted separately
from parent META entry (Revision 1483483)
Result = FAILURE
enis :
Files :
*
/hbase/branches/0.95/hbase-client/src/main/java/org/apache/hadoop/hbase/client/MetaScanner.java
*
/hbase/branches/0.95/hbase-common/src/main/java/org/apache/hadoop/hbase/util/Bytes.java
*
/hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/catalog/MetaEditor.java
*
/hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java
*
/hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMetaScanner.java
*
/hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java
> References to split daughters should not be deleted separately from parent
> META entry
> -
>
> Key: HBASE-8505
> URL: https://issues.apache.org/jira/browse/HBASE-8505
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 0.98.0, 0.94.8, 0.95.1
>
> Attachments: hbase-8505_v1-0.94.patch, hbase-8505_v2-0.94.patch,
> hbase-8505_v2-0.94-reduced.patch, hbase-8505_v2.patch, hbase-8505_v2.patch
>
>
> In CatalogJanitor, we clean up the parent regions whose daughters does not
> have any more references to their parent regions. In doing so, we do two
> Delete's one for removing the split daughter columns, and the other for
> removing the row.
> The first one seems unnecessary, and causes NPE from concurrent MetaScanner.
> Stack trace:
> {code}
> 2013-05-07 04:49:40,828|machine|INFO|Exception in thread "main"
> java.lang.NullPointerException
> 2013-05-07 04:49:40,828|machine|INFO|at
> org.apache.hadoop.hbase.util.Writables.getWritable(Writables.java:103)
> 2013-05-07 04:49:40,828|machine|INFO|at
> org.apache.hadoop.hbase.util.Writables.getHRegionInfo(Writables.java:147)
> 2013-05-07 04:49:40,829|machine|INFO|at
> org.apache.hadoop.hbase.client.MetaScanner$BlockingMetaScannerVisitor.processRow(MetaScanner.java:406)
> 2013-05-07 04:49:40,829|machine|INFO|at
> org.apache.hadoop.hbase.client.MetaScanner$TableMetaScannerVisitor.processRow(MetaScanner.java:487)
> 2013-05-07 04:49:40,830|machine|INFO|at
> org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:224)
> 2013-05-07 04:49:40,830|machine|INFO|at
> org.apache.hadoop.hbase.client.MetaScanner.access$000(MetaScanner.java:54)
> 2013-05-07 04:49:40,830|machine|INFO|at
> org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:133)
> 2013-05-07 04:49:40,831|machine|INFO|at
> org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:130)
> 2013-05-07 04:49:40,831|machine|INFO|at
> org.apache.hadoop.hbase.client.HConnectionManager.execute(HConnectionManager.java:384)
> 2013-05-07 04:49:40,831|machine|INFO|at
> org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:130)
> 2013-05-07 04:49:40,832|machine|INFO|at
> org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:105)
> 2013-05-07 04:49:40,832|machine|INFO|at
> org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:83)
> 2013-05-07 04:49:40,832|machine|INFO|at
> org.apache.hadoop.hbase.client.MetaScanner.allTableRegions(MetaScanner.java:323)
> 2013-05-07 04:49:40,833|machine|INFO|at
> org.apache.hadoop.hbase.client.HTable.getRegionLocations(HTable.java:485)
> 2013-05-07 04:49:40,833|machine|INFO|at
> org.apache.hadoop.hbase.client.HTable.getStartEndKeys(HTable.java:438)
> {code}
> Master is doing the CatalogJanitor concurrently:
> {code}
> 2013-05-07 04:49:40,636 INFO org.apache.hadoop.hbase.catalog.MetaEditor:
> Deleted daughters references, qualifier=splitA and qualifier=splitB, from
> parent
> IntegrationTestBigLinkedList,\x07\xFB\x98\xB7a\x89\xF5\xE6,1367898577620.4ef1329ff0e8911db998ac8ccd32108d.
> 2013-05-07 04:49:40,666 INFO org.apache.hadoop.hbase.catalog.MetaEditor:
> Deleted region
> IntegrationTestBigLinkedList,\x07\xFB\x98\xB7a\x89\xF5\xE6,1367898577620.4ef1329ff0e8911db998ac8ccd32108d.
> from META
> 2013-05-07 04:49:40,690 INFO org.apache.hadoop.hbase.catalog.MetaEditor:
> Deleted daughters references, qualifier=splitA and qualifier=splitB, from
> parent
> IntegrationTestBigLinkedList,\x0B\xF8n\xEA\xD3\xAA\xA9\x92,1367898577620.b502376df2623cb0be3f0c1664d799a6.
> 2013-05-07 04:49:40,716 INFO org.apache.hadoop.hbase.catalog.MetaEditor:
[jira] [Commented] (HBASE-8564) TestMetricsRegionServer depends on test order
[ https://issues.apache.org/jira/browse/HBASE-8564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13660259#comment-13660259 ] Hudson commented on HBASE-8564: --- Integrated in hbase-0.95-on-hadoop2 #102 (See [https://builds.apache.org/job/hbase-0.95-on-hadoop2/102/]) HBASE-8564 TestMetricsRegionServer depends on test order (Revision 1483619) Result = FAILURE eclark : Files : * /hbase/branches/0.95/hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/CompatibilitySingletonFactory.java * /hbase/branches/0.95/hbase-hadoop-compat/src/test/java/org/apache/hadoop/hbase/RandomStringGenerator.java * /hbase/branches/0.95/hbase-hadoop-compat/src/test/java/org/apache/hadoop/hbase/RandomStringGeneratorImpl.java * /hbase/branches/0.95/hbase-hadoop-compat/src/test/java/org/apache/hadoop/hbase/TestCompatibilitySingletonFactory.java * /hbase/branches/0.95/hbase-hadoop-compat/src/test/resources * /hbase/branches/0.95/hbase-hadoop-compat/src/test/resources/META-INF * /hbase/branches/0.95/hbase-hadoop-compat/src/test/resources/META-INF/services * /hbase/branches/0.95/hbase-hadoop-compat/src/test/resources/META-INF/services/org.apache.hadoop.hbase.RandomStringGenerator * /hbase/branches/0.95/hbase-hadoop1-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerSourceFactoryImpl.java * /hbase/branches/0.95/hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionAggregateSourceImpl.java * /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegion.java * /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServer.java * /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMetricsRegionServer.java > TestMetricsRegionServer depends on test order > - > > Key: HBASE-8564 > URL: https://issues.apache.org/jira/browse/HBASE-8564 > Project: HBase > Issue Type: Bug >Affects Versions: 0.98.0, 0.95.0 >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 0.98.0, 0.95.1 > > Attachments: HBASE-8564-0.patch, HBASE-8564-1.patch > > > TestMetricsRegionServer has failed a few times on trunk. It's passing > depends upon test ordering so will fail sometimes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8539) Double(or tripple ...) ZooKeeper listeners of the same type when Master recovers from ZK SessionExpiredException
[ https://issues.apache.org/jira/browse/HBASE-8539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13660243#comment-13660243 ] Sergey Shelukhin commented on HBASE-8539: - Scratch that... org.apache.hadoop.hbase.procedure.TestZKProcedureControllers#testZKCoordinatorControllerMultipleCohort fails for me on 94 with this patch, and passes without on the same clone. Can you check? > Double(or tripple ...) ZooKeeper listeners of the same type when Master > recovers from ZK SessionExpiredException > > > Key: HBASE-8539 > URL: https://issues.apache.org/jira/browse/HBASE-8539 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 0.98.0, 0.94.7, 0.95.0 >Reporter: Jeffrey Zhong >Assignee: Jeffrey Zhong > Fix For: 0.98.0, 0.94.8, 0.95.1 > > Attachments: double-registered listeners.png, > hbase-8539-0.94-addendum.patch, hbase-8539-0.94.patch, > hbase-8539-addendum.patch, hbase-8539.patch, hbase-8539.patch > > > When Master tries to recover from zookeeper session expired exceptions, we > don't clean old registered listener instances. Therefore, it may end up we > have two(or more) listeners to double handling same events. Attached a screen > shot from debugger to show the issue. > I considered to limit one listener per class while I think that would limit > the listener usage so I choose to clear exiting listeners during recovery for > the fix. > (This issue is unrelated to the issue HBASE-8365 because I verified there is > no dup-listeners when HBASE-8365 happened) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-8567) TestDistributedLogSplitting#testLogReplayForDisablingTable fails on hadoop 2.0
Ted Yu created HBASE-8567:
-
Summary:
TestDistributedLogSplitting#testLogReplayForDisablingTable fails on hadoop 2.0
Key: HBASE-8567
URL: https://issues.apache.org/jira/browse/HBASE-8567
Project: HBase
Issue Type: Bug
Reporter: Ted Yu
The test failed with:
{code}
testLogReplayForDisablingTable(org.apache.hadoop.hbase.master.TestDistributedLogSplitting)
Time elapsed: 0.144 sec <<< ERROR!
java.io.FileNotFoundException: File
hdfs://localhost:56794/user/hortonzy/hbase/disableTable/6676df27ed855d9c61ad20ed650700e7/recovered.edits
does not exist.
at
org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:405)
at
org.apache.hadoop.hbase.master.TestDistributedLogSplitting.testLogReplayForDisablingTable(TestDistributedLogSplitting.java:774)
{code}
This was due to FileNotFoundException being thrown out of listStatus() for
non-existent directory in hadoop 2.0.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8461) Provide the ability to delete multiple snapshots through single command
[ https://issues.apache.org/jira/browse/HBASE-8461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13660236#comment-13660236 ] Hudson commented on HBASE-8461: --- Integrated in HBase-TRUNK #4125 (See [https://builds.apache.org/job/HBase-TRUNK/4125/]) HBASE-8461 Provide the ability to delete multiple snapshots through single command (Ted Yu) (Revision 1483574) Result = SUCCESS tedyu : Files : * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestSnapshotFromClient.java > Provide the ability to delete multiple snapshots through single command > --- > > Key: HBASE-8461 > URL: https://issues.apache.org/jira/browse/HBASE-8461 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu >Assignee: Ted Yu > Fix For: 0.98.0, 0.95.1 > > Attachments: 8461-v1.txt, 8461-v2.txt, 8461-v3.txt, 8461-v4.txt > > > Currently HBaseAdmin#deleteSnapshot() accepts name of single snapshot. > It is desirable to allow user to delete multiple snapshots by issuing one > command. > e.g. user may use regular expression to specify the names of snapshots to > delete. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7680) implement compaction policy for stripe compactions
[
https://issues.apache.org/jira/browse/HBASE-7680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13660231#comment-13660231
]
Hadoop QA commented on HBASE-7680:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12583583/HBASE-7680-v9-with-7679.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 22 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.coprocessor.TestRegionObserverScannerOpenHook
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/5731//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5731//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5731//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5731//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5731//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5731//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5731//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5731//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5731//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/5731//console
This message is automatically generated.
> implement compaction policy for stripe compactions
> --
>
> Key: HBASE-7680
> URL: https://issues.apache.org/jira/browse/HBASE-7680
> Project: HBase
> Issue Type: Sub-task
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: HBASE-7680-v0.patch, HBASE-7680-v1.patch,
> HBASE-7680-v1-with-7679.patch, HBASE-7680-v2.patch,
> HBASE-7680-v2-with-7679-and-8034.patch, HBASE-7680-v3.patch,
> HBASE-7680-v3-with-7679.patch, HBASE-7680-v4.patch,
> HBASE-7680-v4-with-7679.patch, HBASE-7680-v5.patch,
> HBASE-7680-v5-with-7679.patch, HBASE-7680-v6.patch,
> HBASE-7680-v6-with-7679.patch, HBASE-7680-v7.patch, HBASE-7680-v8.patch,
> HBASE-7680-v8-with-7679.patch, HBASE-7680-v9.patch,
> HBASE-7680-v9-with-7679.patch
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8563) Double count of read requests for Gets
[ https://issues.apache.org/jira/browse/HBASE-8563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13660228#comment-13660228 ] Francis Liu commented on HBASE-8563: Ran all tests, there were some failures. Tho none of them seem related to the patch. Results : Tests run: 683, Failures: 0, Errors: 0, Skipped: 0 Results : Failed tests: testMultiSlaveReplication(org.apache.hadoop.hbase.replication.TestMultiSlaveReplication): Waited too much time for put replication testCyclicReplication(org.apache.hadoop.hbase.replication.TestMasterReplication): Waited too much time for put replication testSimplePutDelete(org.apache.hadoop.hbase.replication.TestMasterReplication): Waited too much time for put replication testLeaderSelection(org.apache.hadoop.hbase.zookeeper.TestZKLeaderManager): New leader should exist Tests in error: org.apache.hadoop.hbase.replication.TestReplicationQueueFailover: Waiting for startup of standalone server org.apache.hadoop.hbase.replication.TestReplicationQueueFailover testVerifyRepJob(org.apache.hadoop.hbase.replication.TestReplicationSmallTests): test timed out after 30 milliseconds testRunThriftServer[17](org.apache.hadoop.hbase.thrift.TestThriftServerCmdLine): test timed out after 12 milliseconds testRunThriftServer[18](org.apache.hadoop.hbase.thrift.TestThriftServerCmdLine) testRunThriftServer[19](org.apache.hadoop.hbase.thrift.TestThriftServerCmdLine) loadTest[1](org.apache.hadoop.hbase.util.TestMiniClusterLoadSequential): test timed out after 18 milliseconds Tests run: 1319, Failures: 4, Errors: 7, Skipped: 13 > Double count of read requests for Gets > --- > > Key: HBASE-8563 > URL: https://issues.apache.org/jira/browse/HBASE-8563 > Project: HBase > Issue Type: Bug >Reporter: Francis Liu >Assignee: Francis Liu > Fix For: 0.94.7 > > Attachments: HBASE-8563_94.patch > > > Whenever a RegionScanner is created via HRegion.getScanner(), the read > request count is incremented. Since get is implemented as a scan internally. > Each Get request is counted twice. Scans will have an extra count as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8420) Port HBASE-6874 Implement prefetching for scanners from 0.89-fb
[
https://issues.apache.org/jira/browse/HBASE-8420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13660221#comment-13660221
]
Hadoop QA commented on HBASE-8420:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12583575/trunk-8420_v4.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 12 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces lines longer than
100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/5730//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5730//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5730//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5730//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5730//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5730//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5730//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5730//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5730//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/5730//console
This message is automatically generated.
> Port HBASE-6874 Implement prefetching for scanners from 0.89-fb
> -
>
> Key: HBASE-8420
> URL: https://issues.apache.org/jira/browse/HBASE-8420
> Project: HBase
> Issue Type: Improvement
>Reporter: Jimmy Xiang
>Assignee: Jimmy Xiang
> Attachments: 0.94-8420_v1.patch, 0.94-8420_v2.patch,
> trunk-8420_v1.patch, trunk-8420_v2.patch, trunk-8420_v3.patch,
> trunk-8420_v4.patch
>
>
> This should help scanner performance. We should have it in trunk.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8563) Double count of read requests for Gets
[ https://issues.apache.org/jira/browse/HBASE-8563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13660218#comment-13660218 ] Jieshan Bean commented on HBASE-8563: - Makes sense to me. +1. > Double count of read requests for Gets > --- > > Key: HBASE-8563 > URL: https://issues.apache.org/jira/browse/HBASE-8563 > Project: HBase > Issue Type: Bug >Reporter: Francis Liu >Assignee: Francis Liu > Fix For: 0.94.7 > > Attachments: HBASE-8563_94.patch > > > Whenever a RegionScanner is created via HRegion.getScanner(), the read > request count is incremented. Since get is implemented as a scan internally. > Each Get request is counted twice. Scans will have an extra count as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8566) Master-Slave replication: truncate action isn't sent over to slave cluster and cause data inconsistency
[
https://issues.apache.org/jira/browse/HBASE-8566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13660216#comment-13660216
]
Enis Soztutar commented on HBASE-8566:
--
This is currently by design, that DDL operations are not propagated. I think we
should allow that option for users in general, but it is for another jira.
> Master-Slave replication: truncate action isn't sent over to slave cluster
> and cause data inconsistency
> ---
>
> Key: HBASE-8566
> URL: https://issues.apache.org/jira/browse/HBASE-8566
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Affects Versions: 0.94.3
> Environment: two 2-nodes clusters setup as Master and Slave for
> replication of table 'usertable'
>Reporter: Demai Ni
>Priority: Minor
>
> after successfully setup the replcation. put some rows into 'usertable' ,
> At Master cluster
> $truncate 'usertable'
> The truncate(or mass delete from user perspective) request isn't sent over to
> slave cluster.
> From internal, the truncate is 'disable', 'drop' and 'create'. Such
> operations are not designed for replication. However, from external/user
> perspective, this is a 'delete everything' operation, which should be part of
> the replication.
> This JIRA is to add this support
> ---
> additional information. I did a few loads using YCSB into 'usertable', with
> different # of rows(from 1000 to 10). And did truncate a couple times in
> between. Then the slave cluster began to throw errors:
> {code:title=count failed on slave cluster|borderStyle=solid}
> ERROR: org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after
> attempts=7, exceptions:
> Thu May 16 15:00:13 PDT 2013,
> org.apache.hadoop.hbase.client.ScannerCallable@4c474c47,
> java.net.ConnectException: Connection refused
> Thu May 16 15:00:32 PDT 2013,
> org.apache.hadoop.hbase.client.ScannerCallable@4c474c47,
> org.apache.hadoop.hbase.client.NoServerForRegionException: Unable to find
> region for usertable,,99 after 7 tries.
> Thu May 16 15:00:51 PDT 2013,
> org.apache.hadoop.hbase.client.ScannerCallable@4c474c47,
> org.apache.hadoop.hbase.client.NoServerForRegionException: Unable to find
> region for usertable,,99 after 7 tries.
> Thu May 16 15:01:11 PDT 2013,
> org.apache.hadoop.hbase.client.ScannerCallable@4c474c47,
> org.apache.hadoop.hbase.client.NoServerForRegionException: Unable to find
> region for usertable,,99 after 7 tries.
> {code}
> The regionserver log of slave cluster throws :
> {code:title=regionserver log of slave cluster|borderStyle=solid}
> 2013-05-16 14:59:59,655 ERROR
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSink: Unable to
> accept edit because:
> java.io.IOException: java.lang.InterruptedException
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSink.batch(ReplicationSink.java:220)
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSink.replicateEntries(ReplicationSink.java:154)
> at
> org.apache.hadoop.hbase.replication.regionserver.Replication.replicateLogEntries(Replication.java:140)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.replicateLogEntries(HRegionServer.java:3797)
> at sun.reflect.GeneratedMethodAccessor29.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
> at java.lang.reflect.Method.invoke(Method.java:611)
> at
> org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1426)
> Caused by: java.lang.InterruptedException
> at java.lang.Thread.sleep(Native Method)
> at java.lang.Thread.sleep(Thread.java:853)
> at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1507)
> at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1400)
> at org.apache.hadoop.hbase.client.HTable.batch(HTable.java:699)
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSink.batch(ReplicationSink.java:217)
> ... 8 more
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8564) TestMetricsRegionServer depends on test order
[ https://issues.apache.org/jira/browse/HBASE-8564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-8564: - Resolution: Fixed Fix Version/s: 0.95.1 0.98.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) > TestMetricsRegionServer depends on test order > - > > Key: HBASE-8564 > URL: https://issues.apache.org/jira/browse/HBASE-8564 > Project: HBase > Issue Type: Bug >Affects Versions: 0.98.0, 0.95.0 >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 0.98.0, 0.95.1 > > Attachments: HBASE-8564-0.patch, HBASE-8564-1.patch > > > TestMetricsRegionServer has failed a few times on trunk. It's passing > depends upon test ordering so will fail sometimes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8564) TestMetricsRegionServer depends on test order
[ https://issues.apache.org/jira/browse/HBASE-8564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13660207#comment-13660207 ] Elliott Clark commented on HBASE-8564: -- In lets see if this fixes the test issues. > TestMetricsRegionServer depends on test order > - > > Key: HBASE-8564 > URL: https://issues.apache.org/jira/browse/HBASE-8564 > Project: HBase > Issue Type: Bug >Affects Versions: 0.98.0, 0.95.0 >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 0.98.0, 0.95.1 > > Attachments: HBASE-8564-0.patch, HBASE-8564-1.patch > > > TestMetricsRegionServer has failed a few times on trunk. It's passing > depends upon test ordering so will fail sometimes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8518) Get rid of hbase.hstore.compaction.complete setting
[ https://issues.apache.org/jira/browse/HBASE-8518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13660192#comment-13660192 ] Enis Soztutar commented on HBASE-8518: -- Thanks for the patch. You should not remove the test but fix it instead. > Get rid of hbase.hstore.compaction.complete setting > --- > > Key: HBASE-8518 > URL: https://issues.apache.org/jira/browse/HBASE-8518 > Project: HBase > Issue Type: Improvement >Reporter: Sergey Shelukhin >Priority: Minor > Labels: noob > Attachments: HBASE-8518-1.patch > > > hbase.hstore.compaction.complete is a strange setting that causes the > finished compaction to not complete (files are just left in tmp) in HStore. > It's used by one test. > The setting with the same name is also used by CompactionTool, but that usage > is semi-unrelated and could probably be removed easily. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8566) Master-Slave replication: truncate action isn't sent over to slave cluster and cause data inconsistency
[
https://issues.apache.org/jira/browse/HBASE-8566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13660177#comment-13660177
]
Matteo Bertozzi commented on HBASE-8566:
This is a more generic problem, none of the Master operations goes through
replication.
e.g. add/remove column, snapshot restore and co
> Master-Slave replication: truncate action isn't sent over to slave cluster
> and cause data inconsistency
> ---
>
> Key: HBASE-8566
> URL: https://issues.apache.org/jira/browse/HBASE-8566
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Affects Versions: 0.94.3
> Environment: two 2-nodes clusters setup as Master and Slave for
> replication of table 'usertable'
>Reporter: Demai Ni
>Priority: Minor
>
> after successfully setup the replcation. put some rows into 'usertable' ,
> At Master cluster
> $truncate 'usertable'
> The truncate(or mass delete from user perspective) request isn't sent over to
> slave cluster.
> From internal, the truncate is 'disable', 'drop' and 'create'. Such
> operations are not designed for replication. However, from external/user
> perspective, this is a 'delete everything' operation, which should be part of
> the replication.
> This JIRA is to add this support
> ---
> additional information. I did a few loads using YCSB into 'usertable', with
> different # of rows(from 1000 to 10). And did truncate a couple times in
> between. Then the slave cluster began to throw errors:
> {code:title=count failed on slave cluster|borderStyle=solid}
> ERROR: org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after
> attempts=7, exceptions:
> Thu May 16 15:00:13 PDT 2013,
> org.apache.hadoop.hbase.client.ScannerCallable@4c474c47,
> java.net.ConnectException: Connection refused
> Thu May 16 15:00:32 PDT 2013,
> org.apache.hadoop.hbase.client.ScannerCallable@4c474c47,
> org.apache.hadoop.hbase.client.NoServerForRegionException: Unable to find
> region for usertable,,99 after 7 tries.
> Thu May 16 15:00:51 PDT 2013,
> org.apache.hadoop.hbase.client.ScannerCallable@4c474c47,
> org.apache.hadoop.hbase.client.NoServerForRegionException: Unable to find
> region for usertable,,99 after 7 tries.
> Thu May 16 15:01:11 PDT 2013,
> org.apache.hadoop.hbase.client.ScannerCallable@4c474c47,
> org.apache.hadoop.hbase.client.NoServerForRegionException: Unable to find
> region for usertable,,99 after 7 tries.
> {code}
> The regionserver log of slave cluster throws :
> {code:title=regionserver log of slave cluster|borderStyle=solid}
> 2013-05-16 14:59:59,655 ERROR
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSink: Unable to
> accept edit because:
> java.io.IOException: java.lang.InterruptedException
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSink.batch(ReplicationSink.java:220)
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSink.replicateEntries(ReplicationSink.java:154)
> at
> org.apache.hadoop.hbase.replication.regionserver.Replication.replicateLogEntries(Replication.java:140)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.replicateLogEntries(HRegionServer.java:3797)
> at sun.reflect.GeneratedMethodAccessor29.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
> at java.lang.reflect.Method.invoke(Method.java:611)
> at
> org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1426)
> Caused by: java.lang.InterruptedException
> at java.lang.Thread.sleep(Native Method)
> at java.lang.Thread.sleep(Thread.java:853)
> at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1507)
> at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1400)
> at org.apache.hadoop.hbase.client.HTable.batch(HTable.java:699)
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSink.batch(ReplicationSink.java:217)
> ... 8 more
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8375) Streamline Table durability settings
[ https://issues.apache.org/jira/browse/HBASE-8375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13660161#comment-13660161 ] Sergey Shelukhin commented on HBASE-8375: - few small comments on rb > Streamline Table durability settings > > > Key: HBASE-8375 > URL: https://issues.apache.org/jira/browse/HBASE-8375 > Project: HBase > Issue Type: Sub-task >Reporter: Lars Hofhansl >Assignee: Enis Soztutar > Fix For: 0.98.0, 0.95.1 > > Attachments: hbase-8375_v1.patch, hbase-8375_v2.patch > > > HBASE-7801 introduces the notion of per mutation fine grained durability > settings. > This issue is to consider and the discuss the same for the per table settings > (i.e. what would be used if the mutation indicates USE_DEFAULT). I propose > the following setting per table: > * SKIP_WAL (i.e. an unlogged table) > * ASYNC_WAL (the current deferred log flush) > * SYNC_WAL (the current default) > * FSYNC_WAL (for future uses of HDFS' hsync()) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7567) [replication] Create an interface for replication peers
[
https://issues.apache.org/jira/browse/HBASE-7567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13660159#comment-13660159
]
Hadoop QA commented on HBASE-7567:
--
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12583562/HBASE-7567-trunk-95-v5.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 12 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/5729//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5729//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5729//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5729//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5729//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5729//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5729//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5729//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5729//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/5729//console
This message is automatically generated.
> [replication] Create an interface for replication peers
> ---
>
> Key: HBASE-7567
> URL: https://issues.apache.org/jira/browse/HBASE-7567
> Project: HBase
> Issue Type: Improvement
> Components: Replication
>Reporter: Chris Trezzo
>Assignee: Chris Trezzo
> Fix For: 0.95.1
>
> Attachments: HBASE-7567-trunk-95-v1.patch,
> HBASE-7567-trunk-95-v2.patch, HBASE-7567-trunk-95-v3.patch,
> HBASE-7567-trunk-95-v4.patch, HBASE-7567-trunk-95-v5.patch
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8564) TestMetricsRegionServer depends on test order
[ https://issues.apache.org/jira/browse/HBASE-8564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13660156#comment-13660156 ] stack commented on HBASE-8564: -- Or, rather +1 on the patch. Elliott do you want commit? > TestMetricsRegionServer depends on test order > - > > Key: HBASE-8564 > URL: https://issues.apache.org/jira/browse/HBASE-8564 > Project: HBase > Issue Type: Bug >Affects Versions: 0.98.0, 0.95.0 >Reporter: Elliott Clark >Assignee: Elliott Clark > Attachments: HBASE-8564-0.patch, HBASE-8564-1.patch > > > TestMetricsRegionServer has failed a few times on trunk. It's passing > depends upon test ordering so will fail sometimes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8517) Stochastic Loadbalancer isn't finding steady state on real clusters
[
https://issues.apache.org/jira/browse/HBASE-8517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13660157#comment-13660157
]
Sergey Shelukhin commented on HBASE-8517:
-
can you please put it on rb? maybe even v1 and v2 in sequence to be able to
diff the diffs :)
> Stochastic Loadbalancer isn't finding steady state on real clusters
> ---
>
> Key: HBASE-8517
> URL: https://issues.apache.org/jira/browse/HBASE-8517
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.95.0
>Reporter: Elliott Clark
>Assignee: Elliott Clark
> Attachments: HBASE-8517-0.patch, HBASE-8517-1.patch,
> HBASE-8517-2.patch
>
>
> I have a cluster that runs IT tests. Last night after all tests were done I
> noticed that the balancer was thrashing regions around.
> The number of regions on each machine is not correct.
> The balancer seems to value the cost of moving a region way too little.
> {code}
> 2013-05-09 16:34:58,920 DEBUG [IPC Server handler 4 on 6]
> org.apache.hadoop.hbase.master.balancer.StochasticLoadBalancer: Finished
> computing new load balance plan. Computation took 5367ms to try 8910
> different iterations. Found a solution that moves 37 regions; Going from a
> computed cost of 56.50254222730425 to a new cost of 11.214035466575254
> 2013-05-09 16:37:48,715 DEBUG [IPC Server handler 7 on 6]
> org.apache.hadoop.hbase.master.balancer.StochasticLoadBalancer: Finished
> computing new load balance plan. Computation took 4735ms to try 8910
> different iterations. Found a solution that moves 38 regions; Going from a
> computed cost of 56.612624531830996 to a new cost of 11.275763861636982
> 2013-05-09 16:38:11,398 DEBUG [IPC Server handler 6 on 6]
> org.apache.hadoop.hbase.master.balancer.StochasticLoadBalancer: Finished
> computing new load balance plan. Computation took 4502ms to try 8910
> different iterations. Found a solution that moves 39 regions; Going from a
> computed cost of 56.50048461413552 to a new cost of 11.225352339003237
> {code}
> Each of those balancer runs were triggered when there was no load on the
> cluster.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7680) implement compaction policy for stripe compactions
[ https://issues.apache.org/jira/browse/HBASE-7680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HBASE-7680: Attachment: HBASE-7680-v9-with-7679.patch HBASE-7680-v9.patch feedback from r > implement compaction policy for stripe compactions > -- > > Key: HBASE-7680 > URL: https://issues.apache.org/jira/browse/HBASE-7680 > Project: HBase > Issue Type: Sub-task >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HBASE-7680-v0.patch, HBASE-7680-v1.patch, > HBASE-7680-v1-with-7679.patch, HBASE-7680-v2.patch, > HBASE-7680-v2-with-7679-and-8034.patch, HBASE-7680-v3.patch, > HBASE-7680-v3-with-7679.patch, HBASE-7680-v4.patch, > HBASE-7680-v4-with-7679.patch, HBASE-7680-v5.patch, > HBASE-7680-v5-with-7679.patch, HBASE-7680-v6.patch, > HBASE-7680-v6-with-7679.patch, HBASE-7680-v7.patch, HBASE-7680-v8.patch, > HBASE-7680-v8-with-7679.patch, HBASE-7680-v9.patch, > HBASE-7680-v9-with-7679.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8564) TestMetricsRegionServer depends on test order
[ https://issues.apache.org/jira/browse/HBASE-8564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13660151#comment-13660151 ] stack commented on HBASE-8564: -- I don't think a class named RandomStringGenerator needs javadoc. Let me apply this since it is responsible for a bunch of our build breakage of late. > TestMetricsRegionServer depends on test order > - > > Key: HBASE-8564 > URL: https://issues.apache.org/jira/browse/HBASE-8564 > Project: HBase > Issue Type: Bug >Affects Versions: 0.98.0, 0.95.0 >Reporter: Elliott Clark >Assignee: Elliott Clark > Attachments: HBASE-8564-0.patch, HBASE-8564-1.patch > > > TestMetricsRegionServer has failed a few times on trunk. It's passing > depends upon test ordering so will fail sometimes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8461) Provide the ability to delete multiple snapshots through single command
[ https://issues.apache.org/jira/browse/HBASE-8461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13660153#comment-13660153 ] Hudson commented on HBASE-8461: --- Integrated in hbase-0.95 #199 (See [https://builds.apache.org/job/hbase-0.95/199/]) HBASE-8461 Provide the ability to delete multiple snapshots through single command (Ted Yu) (Revision 1483576) Result = FAILURE tedyu : Files : * /hbase/branches/0.95/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java * /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestSnapshotFromClient.java > Provide the ability to delete multiple snapshots through single command > --- > > Key: HBASE-8461 > URL: https://issues.apache.org/jira/browse/HBASE-8461 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu >Assignee: Ted Yu > Fix For: 0.98.0, 0.95.1 > > Attachments: 8461-v1.txt, 8461-v2.txt, 8461-v3.txt, 8461-v4.txt > > > Currently HBaseAdmin#deleteSnapshot() accepts name of single snapshot. > It is desirable to allow user to delete multiple snapshots by issuing one > command. > e.g. user may use regular expression to specify the names of snapshots to > delete. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8564) TestMetricsRegionServer depends on test order
[
https://issues.apache.org/jira/browse/HBASE-8564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13660137#comment-13660137
]
Hadoop QA commented on HBASE-8564:
--
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12583551/HBASE-8564-1.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 17 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/5728//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5728//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5728//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5728//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5728//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5728//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5728//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5728//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5728//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/5728//console
This message is automatically generated.
> TestMetricsRegionServer depends on test order
> -
>
> Key: HBASE-8564
> URL: https://issues.apache.org/jira/browse/HBASE-8564
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.98.0, 0.95.0
>Reporter: Elliott Clark
>Assignee: Elliott Clark
> Attachments: HBASE-8564-0.patch, HBASE-8564-1.patch
>
>
> TestMetricsRegionServer has failed a few times on trunk. It's passing
> depends upon test ordering so will fail sometimes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7679) implement store file management for stripe compactions
[ https://issues.apache.org/jira/browse/HBASE-7679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HBASE-7679: Attachment: HBASE-7679-v13.patch > implement store file management for stripe compactions > -- > > Key: HBASE-7679 > URL: https://issues.apache.org/jira/browse/HBASE-7679 > Project: HBase > Issue Type: Sub-task >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HBASE-7667-and-7603-v0-incomplete.patch, > HBASE-7667-and-7603-v0-incomplete.patch, HBASE-7667-and-7603-v1.patch, > HBASE-7667-and-7603-v1.patch, HBASE-7667-v1.patch, HBASE-7667-v1.patch, > HBASE-7667-v2.patch, HBASE-7667-v2.patch, HBASE-7667-v3.patch, > HBASE-7679-v10.patch, HBASE-7679-v11.patch, HBASE-7679-v12.patch, > HBASE-7679-v12.patch, HBASE-7679-v13.patch, HBASE-7679-v13.patch, > HBASE-7679-v4.patch, HBASE-7679-v5.patch, HBASE-7679-v6.patch, > HBASE-7679-v7-.patch, HBASE-7679-v7.patch, HBASE-7679-v8.patch, > HBASE-7679-v9.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8420) Port HBASE-6874 Implement prefetching for scanners from 0.89-fb
[ https://issues.apache.org/jira/browse/HBASE-8420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-8420: --- Status: Patch Available (was: Open) > Port HBASE-6874 Implement prefetching for scanners from 0.89-fb > - > > Key: HBASE-8420 > URL: https://issues.apache.org/jira/browse/HBASE-8420 > Project: HBase > Issue Type: Improvement >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang > Attachments: 0.94-8420_v1.patch, 0.94-8420_v2.patch, > trunk-8420_v1.patch, trunk-8420_v2.patch, trunk-8420_v3.patch, > trunk-8420_v4.patch > > > This should help scanner performance. We should have it in trunk. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8420) Port HBASE-6874 Implement prefetching for scanners from 0.89-fb
[ https://issues.apache.org/jira/browse/HBASE-8420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-8420: --- Attachment: trunk-8420_v4.patch > Port HBASE-6874 Implement prefetching for scanners from 0.89-fb > - > > Key: HBASE-8420 > URL: https://issues.apache.org/jira/browse/HBASE-8420 > Project: HBase > Issue Type: Improvement >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang > Attachments: 0.94-8420_v1.patch, 0.94-8420_v2.patch, > trunk-8420_v1.patch, trunk-8420_v2.patch, trunk-8420_v3.patch, > trunk-8420_v4.patch > > > This should help scanner performance. We should have it in trunk. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7679) implement store file management for stripe compactions
[ https://issues.apache.org/jira/browse/HBASE-7679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HBASE-7679: Attachment: HBASE-7679-v13.patch r feedback, also cleaned up one test that was hard to understand > implement store file management for stripe compactions > -- > > Key: HBASE-7679 > URL: https://issues.apache.org/jira/browse/HBASE-7679 > Project: HBase > Issue Type: Sub-task >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HBASE-7667-and-7603-v0-incomplete.patch, > HBASE-7667-and-7603-v0-incomplete.patch, HBASE-7667-and-7603-v1.patch, > HBASE-7667-and-7603-v1.patch, HBASE-7667-v1.patch, HBASE-7667-v1.patch, > HBASE-7667-v2.patch, HBASE-7667-v2.patch, HBASE-7667-v3.patch, > HBASE-7679-v10.patch, HBASE-7679-v11.patch, HBASE-7679-v12.patch, > HBASE-7679-v12.patch, HBASE-7679-v13.patch, HBASE-7679-v4.patch, > HBASE-7679-v5.patch, HBASE-7679-v6.patch, HBASE-7679-v7-.patch, > HBASE-7679-v7.patch, HBASE-7679-v8.patch, HBASE-7679-v9.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8555) FilterList correctness was dominated by sub-filter(list) ordering randomly
[
https://issues.apache.org/jira/browse/HBASE-8555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13660122#comment-13660122
]
Hadoop QA commented on HBASE-8555:
--
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12583545/8555-trunk-v1.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 6 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/5727//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5727//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5727//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5727//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5727//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5727//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5727//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5727//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5727//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/5727//console
This message is automatically generated.
> FilterList correctness was dominated by sub-filter(list) ordering randomly
> --
>
> Key: HBASE-8555
> URL: https://issues.apache.org/jira/browse/HBASE-8555
> Project: HBase
> Issue Type: Bug
> Components: Filters
>Affects Versions: 0.94.3
>Reporter: Liang Xie
>Assignee: Liang Xie
>Priority: Critical
> Attachments: 8555-trunk-v1.txt, HBASE-8555-0.94.txt
>
>
> say, ther're 10 rows, column value is i%2:
> row0 0
> row1 1
> row2 0
> row3 1
> row4 0
> row5 1
> row6 0
> row7 1
> row8 0
> row9 1
> 1: filter : row filter > row4 ===> row5 row6 row7 row8 row9
> 2: subFilterList: row filter <= row4 && column==0===> row0 row2 row4
> 3.1 filterlist[expected] filter || subFilterList ===> row0 row2 row4 row5
> row6 row7 row8 row9
> 3.2 filterlist[BUGON!] subFilterList || filter ===> row0 row1 row2 row3 row4
> row5 row6 row7 row8 row9
> (Please refer to the new testNestedFilterListWithSCVF case)
> It was found when i managed to transform the following SQL into HBase scan
> statement:
> select xxx from xxx where (pk <= xxx and column1 = xxx) or pk > xxx
> My finding is that we had an assumption for filter methods call sequence:
> e.g. filterRowKey() should be called before filterKeyValue().
> and the orignial filterList.filterRowKey impl broke it due to fast
> short-circuit returning.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8566) Master-Slave replication: truncate action isn't sent over to slave cluster and cause data inconsistency
[
https://issues.apache.org/jira/browse/HBASE-8566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Demai Ni updated HBASE-8566:
Summary: Master-Slave replication: truncate action isn't sent over to slave
cluster and cause data inconsistency (was: Master-Slave replication: truncate
action isn't sent over to slave cluster and cause data inconsisten)
> Master-Slave replication: truncate action isn't sent over to slave cluster
> and cause data inconsistency
> ---
>
> Key: HBASE-8566
> URL: https://issues.apache.org/jira/browse/HBASE-8566
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Affects Versions: 0.94.3
> Environment: two 2-nodes clusters setup as Master and Slave for
> replication of table 'usertable'
>Reporter: Demai Ni
>Priority: Minor
>
> after successfully setup the replcation. put some rows into 'usertable' ,
> At Master cluster
> $truncate 'usertable'
> The truncate(or mass delete from user perspective) request isn't sent over to
> slave cluster.
> From internal, the truncate is 'disable', 'drop' and 'create'. Such
> operations are not designed for replication. However, from external/user
> perspective, this is a 'delete everything' operation, which should be part of
> the replication.
> This JIRA is to add this support
> ---
> additional information. I did a few loads using YCSB into 'usertable', with
> different # of rows(from 1000 to 10). And did truncate a couple times in
> between. Then the slave cluster began to throw errors:
> {code:title=count failed on slave cluster|borderStyle=solid}
> ERROR: org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after
> attempts=7, exceptions:
> Thu May 16 15:00:13 PDT 2013,
> org.apache.hadoop.hbase.client.ScannerCallable@4c474c47,
> java.net.ConnectException: Connection refused
> Thu May 16 15:00:32 PDT 2013,
> org.apache.hadoop.hbase.client.ScannerCallable@4c474c47,
> org.apache.hadoop.hbase.client.NoServerForRegionException: Unable to find
> region for usertable,,99 after 7 tries.
> Thu May 16 15:00:51 PDT 2013,
> org.apache.hadoop.hbase.client.ScannerCallable@4c474c47,
> org.apache.hadoop.hbase.client.NoServerForRegionException: Unable to find
> region for usertable,,99 after 7 tries.
> Thu May 16 15:01:11 PDT 2013,
> org.apache.hadoop.hbase.client.ScannerCallable@4c474c47,
> org.apache.hadoop.hbase.client.NoServerForRegionException: Unable to find
> region for usertable,,99 after 7 tries.
> {code}
> The regionserver log of slave cluster throws :
> {code:title=regionserver log of slave cluster|borderStyle=solid}
> 2013-05-16 14:59:59,655 ERROR
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSink: Unable to
> accept edit because:
> java.io.IOException: java.lang.InterruptedException
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSink.batch(ReplicationSink.java:220)
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSink.replicateEntries(ReplicationSink.java:154)
> at
> org.apache.hadoop.hbase.replication.regionserver.Replication.replicateLogEntries(Replication.java:140)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.replicateLogEntries(HRegionServer.java:3797)
> at sun.reflect.GeneratedMethodAccessor29.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
> at java.lang.reflect.Method.invoke(Method.java:611)
> at
> org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1426)
> Caused by: java.lang.InterruptedException
> at java.lang.Thread.sleep(Native Method)
> at java.lang.Thread.sleep(Thread.java:853)
> at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1507)
> at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1400)
> at org.apache.hadoop.hbase.client.HTable.batch(HTable.java:699)
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSink.batch(ReplicationSink.java:217)
> ... 8 more
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8548) postOpen hook called twice
[
https://issues.apache.org/jira/browse/HBASE-8548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13660097#comment-13660097
]
Nicolas Liochon commented on HBASE-8548:
I tried with a wider patch
{noformat}
Index:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
===
---
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
(revision 1482641)
+++
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
(working copy)
@@ -682,38 +682,7 @@
// Recover any edits if available.
maxSeqId = Math.max(maxSeqId, replayRecoveredEditsIfAny(
this.fs.getRegionDir(), maxSeqIdInStores, reporter, status));
-
-status.setStatus("Cleaning up detritus from prior splits");
-// Get rid of any splits or merges that were lost in-progress. Clean out
-// these directories here on open. We may be opening a region that was
-// being split but we crashed in the middle of it all.
-fs.cleanupAnySplitDetritus();
-fs.cleanupMergesDir();
-this.writestate.setReadOnly(this.htableDescriptor.isReadOnly());
-
-this.writestate.flushRequested = false;
-this.writestate.compacting = 0;
-
-// Initialize split policy
-this.splitPolicy = RegionSplitPolicy.create(this, conf);
-
-this.lastFlushTime = EnvironmentEdgeManager.currentTimeMillis();
-// Use maximum of log sequenceid or that which was found in stores
-// (particularly if no recovered edits, seqid will be -1).
-long nextSeqid = maxSeqId + 1;
-LOG.info("Onlined " + this.toString() + "; next sequenceid=" + nextSeqid);
-
-// A region can be reopened if failed a split; reset flags
-this.closing.set(false);
-this.closed.set(false);
-
-if (coprocessorHost != null) {
- status.setStatus("Running coprocessor post-open hooks");
- coprocessorHost.postOpen();
-}
-
-status.markComplete("Region opened successfully");
-return nextSeqid;
+return maxSeqId;
}
/*
{noformat}
but this doesn't pass the tests. I will have a deeper look tomorrow.
> postOpen hook called twice
> --
>
> Key: HBASE-8548
> URL: https://issues.apache.org/jira/browse/HBASE-8548
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 0.95.0
> Environment: CentOS
>Reporter: Roger Ruiz-Carrillo
> Attachments: HRegion_HBASE-8548-0.95.patch
>
>
> postOpen hook is called twice when a region is initializing:
> Once at the end of the body of the initializeRegionInternals() method of the
> HRegion class.
> Once at the end initializeRegionStores() method of the HRegion class;
> initializeRegionStores() is called inside initializeRegionInternals() and as
> such causes the postOpen hook to be called twice.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-8566) Master-Slave replication: truncate action isn't sent over to slave cluster and cause data inconsisten
Demai Ni created HBASE-8566:
---
Summary: Master-Slave replication: truncate action isn't sent over
to slave cluster and cause data inconsisten
Key: HBASE-8566
URL: https://issues.apache.org/jira/browse/HBASE-8566
Project: HBase
Issue Type: Bug
Components: Replication
Affects Versions: 0.94.3
Environment: two 2-nodes clusters setup as Master and Slave for
replication of table 'usertable'
Reporter: Demai Ni
Priority: Minor
after successfully setup the replcation. put some rows into 'usertable' ,
At Master cluster
$truncate 'usertable'
The truncate(or mass delete from user perspective) request isn't sent over to
slave cluster.
>From internal, the truncate is 'disable', 'drop' and 'create'. Such operations
>are not designed for replication. However, from external/user perspective,
>this is a 'delete everything' operation, which should be part of the
>replication.
This JIRA is to add this support
---
additional information. I did a few loads using YCSB into 'usertable', with
different # of rows(from 1000 to 10). And did truncate a couple times in
between. Then the slave cluster began to throw errors:
{code:title=count failed on slave cluster|borderStyle=solid}
ERROR: org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after
attempts=7, exceptions:
Thu May 16 15:00:13 PDT 2013,
org.apache.hadoop.hbase.client.ScannerCallable@4c474c47,
java.net.ConnectException: Connection refused
Thu May 16 15:00:32 PDT 2013,
org.apache.hadoop.hbase.client.ScannerCallable@4c474c47,
org.apache.hadoop.hbase.client.NoServerForRegionException: Unable to find
region for usertable,,99 after 7 tries.
Thu May 16 15:00:51 PDT 2013,
org.apache.hadoop.hbase.client.ScannerCallable@4c474c47,
org.apache.hadoop.hbase.client.NoServerForRegionException: Unable to find
region for usertable,,99 after 7 tries.
Thu May 16 15:01:11 PDT 2013,
org.apache.hadoop.hbase.client.ScannerCallable@4c474c47,
org.apache.hadoop.hbase.client.NoServerForRegionException: Unable to find
region for usertable,,99 after 7 tries.
{code}
The regionserver log of slave cluster throws :
{code:title=regionserver log of slave cluster|borderStyle=solid}
2013-05-16 14:59:59,655 ERROR
org.apache.hadoop.hbase.replication.regionserver.ReplicationSink: Unable to
accept edit because:
java.io.IOException: java.lang.InterruptedException
at
org.apache.hadoop.hbase.replication.regionserver.ReplicationSink.batch(ReplicationSink.java:220)
at
org.apache.hadoop.hbase.replication.regionserver.ReplicationSink.replicateEntries(ReplicationSink.java:154)
at
org.apache.hadoop.hbase.replication.regionserver.Replication.replicateLogEntries(Replication.java:140)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.replicateLogEntries(HRegionServer.java:3797)
at sun.reflect.GeneratedMethodAccessor29.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
at java.lang.reflect.Method.invoke(Method.java:611)
at
org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1426)
Caused by: java.lang.InterruptedException
at java.lang.Thread.sleep(Native Method)
at java.lang.Thread.sleep(Thread.java:853)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1507)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1400)
at org.apache.hadoop.hbase.client.HTable.batch(HTable.java:699)
at
org.apache.hadoop.hbase.replication.regionserver.ReplicationSink.batch(ReplicationSink.java:217)
... 8 more
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8547) Fix java.lang.RuntimeException: Cached an already cached block
[
https://issues.apache.org/jira/browse/HBASE-8547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13660092#comment-13660092
]
Hudson commented on HBASE-8547:
---
Integrated in HBase-TRUNK #4124 (See
[https://builds.apache.org/job/HBase-TRUNK/4124/])
HBASE-8547. Fix java.lang.RuntimeException: Cached an already cached block
(Addendum2 to check cached buffers for equality) (Revision 1483476)
Result = SUCCESS
enis :
Files :
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java
> Fix java.lang.RuntimeException: Cached an already cached block
> --
>
> Key: HBASE-8547
> URL: https://issues.apache.org/jira/browse/HBASE-8547
> Project: HBase
> Issue Type: Bug
> Components: io, regionserver
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 0.98.0, 0.94.8, 0.95.1
>
> Attachments: hbase-8547_v1-0.94.patch, hbase-8547_v1-0.94.patch,
> hbase-8547_v1.patch, hbase-8547_v2-0.94-reduced.patch,
> hbase-8547_v2-addendum2.patch, hbase-8547_v2-addendum2.patch,
> hbase-8547_v2-trunk.patch
>
>
> In one test, one of the region servers received the following on 0.94.
> Note HalfStoreFileReader in the stack trace. I think the root cause is that
> after the region is split, the mid point can be in the middle of the block
> (for store files that the mid point is not chosen from). Each half store file
> tries to load the half block and put it in the block cache. Since IdLock is
> instantiated per store file reader, they do not share the same IdLock
> instance, thus does not lock against each other effectively.
> {code}
> 2013-05-12 01:30:37,733 ERROR
> org.apache.hadoop.hbase.regionserver.HRegionServer:ยท
> java.lang.RuntimeException: Cached an already cached block
> at
> org.apache.hadoop.hbase.io.hfile.LruBlockCache.cacheBlock(LruBlockCache.java:279)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:353)
> at
> org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.loadDataBlockWithScanInfo(HFileBlockIndex.java:254)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.seekTo(HFileReaderV2.java:480)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.seekTo(HFileReaderV2.java:501)
> at
> org.apache.hadoop.hbase.io.HalfStoreFileReader$1.seekTo(HalfStoreFileReader.java:237)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.seekAtOrAfter(StoreFileScanner.java:226)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.seek(StoreFileScanner.java:145)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.enforceSeek(StoreFileScanner.java:351)
> at
> org.apache.hadoop.hbase.regionserver.KeyValueHeap.pollRealKV(KeyValueHeap.java:354)
> at
> org.apache.hadoop.hbase.regionserver.KeyValueHeap.generalizedSeek(KeyValueHeap.java:312)
> at
> org.apache.hadoop.hbase.regionserver.KeyValueHeap.requestSeek(KeyValueHeap.java:277)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.reseek(StoreScanner.java:543)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:411)
> at
> org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:143)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.populateResult(HRegion.java:3829)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:3896)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:3778)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:3770)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:2643)
> at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.hadoop.hbase.ipc.SecureRpcEngine$Server.call(SecureRpcEngine.java:308)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1426)
> {code}
> I can see two possible fixes:
> # Allow this kind of rare cases in LruBlockCache by not throwing an
> exception.
> # Move the lock instances to upper layer (possibly in CacheConfig), and let
> half hfile readers share the same IdLock implementation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8505) References to split daughters should not be deleted separately from parent META entry
[
https://issues.apache.org/jira/browse/HBASE-8505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13660090#comment-13660090
]
Hudson commented on HBASE-8505:
---
Integrated in HBase-TRUNK #4124 (See
[https://builds.apache.org/job/HBase-TRUNK/4124/])
HBASE-8505 References to split daughters should not be deleted separately
from parent META entry (Revision 1483481)
Result = SUCCESS
enis :
Files :
*
/hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/MetaScanner.java
*
/hbase/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/util/Bytes.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/catalog/MetaEditor.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMetaScanner.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java
> References to split daughters should not be deleted separately from parent
> META entry
> -
>
> Key: HBASE-8505
> URL: https://issues.apache.org/jira/browse/HBASE-8505
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 0.98.0, 0.94.8, 0.95.1
>
> Attachments: hbase-8505_v1-0.94.patch, hbase-8505_v2-0.94.patch,
> hbase-8505_v2-0.94-reduced.patch, hbase-8505_v2.patch, hbase-8505_v2.patch
>
>
> In CatalogJanitor, we clean up the parent regions whose daughters does not
> have any more references to their parent regions. In doing so, we do two
> Delete's one for removing the split daughter columns, and the other for
> removing the row.
> The first one seems unnecessary, and causes NPE from concurrent MetaScanner.
> Stack trace:
> {code}
> 2013-05-07 04:49:40,828|machine|INFO|Exception in thread "main"
> java.lang.NullPointerException
> 2013-05-07 04:49:40,828|machine|INFO|at
> org.apache.hadoop.hbase.util.Writables.getWritable(Writables.java:103)
> 2013-05-07 04:49:40,828|machine|INFO|at
> org.apache.hadoop.hbase.util.Writables.getHRegionInfo(Writables.java:147)
> 2013-05-07 04:49:40,829|machine|INFO|at
> org.apache.hadoop.hbase.client.MetaScanner$BlockingMetaScannerVisitor.processRow(MetaScanner.java:406)
> 2013-05-07 04:49:40,829|machine|INFO|at
> org.apache.hadoop.hbase.client.MetaScanner$TableMetaScannerVisitor.processRow(MetaScanner.java:487)
> 2013-05-07 04:49:40,830|machine|INFO|at
> org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:224)
> 2013-05-07 04:49:40,830|machine|INFO|at
> org.apache.hadoop.hbase.client.MetaScanner.access$000(MetaScanner.java:54)
> 2013-05-07 04:49:40,830|machine|INFO|at
> org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:133)
> 2013-05-07 04:49:40,831|machine|INFO|at
> org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:130)
> 2013-05-07 04:49:40,831|machine|INFO|at
> org.apache.hadoop.hbase.client.HConnectionManager.execute(HConnectionManager.java:384)
> 2013-05-07 04:49:40,831|machine|INFO|at
> org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:130)
> 2013-05-07 04:49:40,832|machine|INFO|at
> org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:105)
> 2013-05-07 04:49:40,832|machine|INFO|at
> org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:83)
> 2013-05-07 04:49:40,832|machine|INFO|at
> org.apache.hadoop.hbase.client.MetaScanner.allTableRegions(MetaScanner.java:323)
> 2013-05-07 04:49:40,833|machine|INFO|at
> org.apache.hadoop.hbase.client.HTable.getRegionLocations(HTable.java:485)
> 2013-05-07 04:49:40,833|machine|INFO|at
> org.apache.hadoop.hbase.client.HTable.getStartEndKeys(HTable.java:438)
> {code}
> Master is doing the CatalogJanitor concurrently:
> {code}
> 2013-05-07 04:49:40,636 INFO org.apache.hadoop.hbase.catalog.MetaEditor:
> Deleted daughters references, qualifier=splitA and qualifier=splitB, from
> parent
> IntegrationTestBigLinkedList,\x07\xFB\x98\xB7a\x89\xF5\xE6,1367898577620.4ef1329ff0e8911db998ac8ccd32108d.
> 2013-05-07 04:49:40,666 INFO org.apache.hadoop.hbase.catalog.MetaEditor:
> Deleted region
> IntegrationTestBigLinkedList,\x07\xFB\x98\xB7a\x89\xF5\xE6,1367898577620.4ef1329ff0e8911db998ac8ccd32108d.
> from META
> 2013-05-07 04:49:40,690 INFO org.apache.hadoop.hbase.catalog.MetaEditor:
> Deleted daughters references, qualifier=splitA and qualifier=splitB, from
> parent
> IntegrationTestBigLinkedList,\x0B\xF8n\xEA\xD3\xAA\xA9\x92,1367898577620.b502376df2623cb0be3f0c1664d799a6.
> 2013-05-07 04:49:40,716 INFO org.apache.hadoop.hbase.catalog.MetaEditor:
> Deleted region
> IntegrationTestBigLinkedList,\x0B\xF8n\xEA\xD3
[jira] [Commented] (HBASE-8560) TestMasterShutdown failing in trunk 0.95/trunk -- "Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error)"
[ https://issues.apache.org/jira/browse/HBASE-8560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13660091#comment-13660091 ] Hudson commented on HBASE-8560: --- Integrated in HBase-TRUNK #4124 (See [https://builds.apache.org/job/HBase-TRUNK/4124/]) HBASE-8560 TestMasterShutdown failing in trunk 0.95/trunk -- "Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error)" (Revision 1483547) Result = SUCCESS stack : Files : * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java > TestMasterShutdown failing in trunk 0.95/trunk -- "Unable to get data of > znode /hbase/meta-region-server because node does not exist (not an error)" > > > Key: HBASE-8560 > URL: https://issues.apache.org/jira/browse/HBASE-8560 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: Jeffrey Zhong > Fix For: 0.98.0, 0.95.1 > > Attachments: hbase-8560.patch, hbase-8560-v2.patch > > > I'm looking at this too... [~jeffreyz] you are too? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7567) [replication] Create an interface for replication peers
[ https://issues.apache.org/jira/browse/HBASE-7567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Trezzo updated HBASE-7567: Attachment: HBASE-7567-trunk-95-v5.patch Rebase of the patch. All replication tests pass locally. Let's see if it goes through QA. > [replication] Create an interface for replication peers > --- > > Key: HBASE-7567 > URL: https://issues.apache.org/jira/browse/HBASE-7567 > Project: HBase > Issue Type: Improvement > Components: Replication >Reporter: Chris Trezzo >Assignee: Chris Trezzo > Fix For: 0.95.1 > > Attachments: HBASE-7567-trunk-95-v1.patch, > HBASE-7567-trunk-95-v2.patch, HBASE-7567-trunk-95-v3.patch, > HBASE-7567-trunk-95-v4.patch, HBASE-7567-trunk-95-v5.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8553) improve unit-test coverage of package org.apache.hadoop.hbase.mapreduce.hadoopbackport
[
https://issues.apache.org/jira/browse/HBASE-8553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13660071#comment-13660071
]
Enis Soztutar commented on HBASE-8553:
--
Does this fix a real issue or not:
{code}
-TotalOrderPartitioner.setPartitionFile(getConf(), outf);
-for (String s : otherArgs) {
+TotalOrderPartitioner.setPartitionFile(job.getConfiguration(), outf);
{code}
Should we pick the tests from hadoop trunk instead of 0.23?
> improve unit-test coverage of package
> org.apache.hadoop.hbase.mapreduce.hadoopbackport
> --
>
> Key: HBASE-8553
> URL: https://issues.apache.org/jira/browse/HBASE-8553
> Project: HBase
> Issue Type: Test
>Affects Versions: 0.94.9
>Reporter: Ivan A. Veselovsky
> Attachments: HBASE-8553-0.94--N2.patch
>
>
> The patch is for branch 0.94 only.
> The class InputSampler is modified because need to fix bug there: in method
> "run(String[] args)" should be
> "TotalOrderPartitioner.setPartitionFile(job.getConfiguration(), outf);"
> instead of "TotalOrderPartitioner.setPartitionFile(getConf(), outf);".
> Otherwise it is impossible to set output file correctly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8565) stop-hbase.sh clean up: backup master
[
https://issues.apache.org/jira/browse/HBASE-8565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jerry He updated HBASE-8565:
Component/s: scripts
master
Affects Version/s: 0.94.7
0.95.0
Fix Version/s: 0.94.9
0.95.2
0.98.0
> stop-hbase.sh clean up: backup master
> -
>
> Key: HBASE-8565
> URL: https://issues.apache.org/jira/browse/HBASE-8565
> Project: HBase
> Issue Type: Bug
> Components: master, scripts
>Affects Versions: 0.94.7, 0.95.0
>Reporter: Jerry He
>Priority: Minor
> Fix For: 0.98.0, 0.95.2, 0.94.9
>
>
> In stop-hbase.sh:
> {code}
> # TODO: store backup masters in ZooKeeper and have the primary send them a
> shutdown message
> # stop any backup masters
> "$bin"/hbase-daemons.sh --config "${HBASE_CONF_DIR}" \
> --hosts "${HBASE_BACKUP_MASTERS}" stop master-backup
> {code}
> After HBASE-5213, stop-hbase.sh -> hbase master stop will bring up the backup
> master too via the cluster status znode.
> We should not need the above code anymore.
> Another issue happens when the current master died and the backup master
> became the active master.
> {code}
> nohup nice -n ${HBASE_NICENESS:-0} "$HBASE_HOME"/bin/hbase \
>--config "${HBASE_CONF_DIR}" \
>master stop "$@" > "$logout" 2>&1 < /dev/null &
> waitForProcessEnd `cat $pid` 'stop-master-command'
> {code}
> We can still issue 'hbase-stop.sh' from the old master.
> stop-hbase.sh -> hbase master stop -> look for active master -> request
> shutdown
> This process still works.
> But the waitForProcessEnd statement will not work since the local master pid
> is not relevant anymore.
> What is the best way in the this case?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-8565) stop-hbase.sh clean up: backup master
Jerry He created HBASE-8565:
---
Summary: stop-hbase.sh clean up: backup master
Key: HBASE-8565
URL: https://issues.apache.org/jira/browse/HBASE-8565
Project: HBase
Issue Type: Bug
Reporter: Jerry He
Priority: Minor
In stop-hbase.sh:
{code}
# TODO: store backup masters in ZooKeeper and have the primary send them a
shutdown message
# stop any backup masters
"$bin"/hbase-daemons.sh --config "${HBASE_CONF_DIR}" \
--hosts "${HBASE_BACKUP_MASTERS}" stop master-backup
{code}
After HBASE-5213, stop-hbase.sh -> hbase master stop will bring up the backup
master too via the cluster status znode.
We should not need the above code anymore.
Another issue happens when the current master died and the backup master became
the active master.
{code}
nohup nice -n ${HBASE_NICENESS:-0} "$HBASE_HOME"/bin/hbase \
--config "${HBASE_CONF_DIR}" \
master stop "$@" > "$logout" 2>&1 < /dev/null &
waitForProcessEnd `cat $pid` 'stop-master-command'
{code}
We can still issue 'hbase-stop.sh' from the old master.
stop-hbase.sh -> hbase master stop -> look for active master -> request shutdown
This process still works.
But the waitForProcessEnd statement will not work since the local master pid is
not relevant anymore.
What is the best way in the this case?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8564) TestMetricsRegionServer depends on test order
[ https://issues.apache.org/jira/browse/HBASE-8564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13660059#comment-13660059 ] Ted Yu commented on HBASE-8564: --- For RandomStringGeneratorImpl and RandomStringGenerator, please add class javadoc. > TestMetricsRegionServer depends on test order > - > > Key: HBASE-8564 > URL: https://issues.apache.org/jira/browse/HBASE-8564 > Project: HBase > Issue Type: Bug >Affects Versions: 0.98.0, 0.95.0 >Reporter: Elliott Clark >Assignee: Elliott Clark > Attachments: HBASE-8564-0.patch, HBASE-8564-1.patch > > > TestMetricsRegionServer has failed a few times on trunk. It's passing > depends upon test ordering so will fail sometimes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8545) Meta stuck in transition when it is assigned to a just restarted dead region sever
[
https://issues.apache.org/jira/browse/HBASE-8545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13660056#comment-13660056
]
Hadoop QA commented on HBASE-8545:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12583540/trunk-8545_v2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 6 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.master.TestTableLockManager
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/5726//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5726//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5726//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5726//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5726//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5726//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5726//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5726//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5726//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/5726//console
This message is automatically generated.
> Meta stuck in transition when it is assigned to a just restarted dead region
> sever
> ---
>
> Key: HBASE-8545
> URL: https://issues.apache.org/jira/browse/HBASE-8545
> Project: HBase
> Issue Type: Bug
> Components: Region Assignment
>Reporter: Jimmy Xiang
>Assignee: Jimmy Xiang
> Attachments: trunk-8545.patch, trunk-8545_v2.patch
>
>
> Support the meta region server is down, and the SSH tries to re-assign it.
> This could happen:
> 1. AM plans to assign meta to a region server (R_old);
> 2. Now R_old is dead, the new region server (R_new) starts up on the same
> host, port, but gets a different start code;
> 3. AM sends the open region request to R_new and the Meta is opened on it;
> 4. AM gets ZK event, but it is from a different region server instance
> (R_new), not the expected one (R_old), so it sends a close region request to
> R_new;
> 5. Now, the meta is stuck in transition and won't be assigned.
> This won't happen to a user region since the SSH for R_old will find out the
> user region stuck in transition and re-assign it. For meta, it is a little
> different. AM checks if a dead region server carries the meta based on the
> ZK info, which is changed to the new region server R_new at step 3 by the
> open region handler.
> The fix I was thinking about is:
> 1. In checking if a region server carries a region, uses the region
> transition information if it exists (which is the source of truth, to
> master), if not, checks the ZK data as before;
> 2. In open region handler, when transition assign zk node from offline to
> opening, make sure the current region server is the expected one
> (ZK#transitionNode, existing code doesn't check the target server name).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please
[jira] [Commented] (HBASE-8505) References to split daughters should not be deleted separately from parent META entry
[
https://issues.apache.org/jira/browse/HBASE-8505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13660054#comment-13660054
]
Hudson commented on HBASE-8505:
---
Integrated in HBase-0.94 #985 (See
[https://builds.apache.org/job/HBase-0.94/985/])
HBASE-8505 References to split daughters should not be deleted separately
from parent META entry (Revision 1483485)
Result = SUCCESS
enis :
Files :
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/catalog/MetaEditor.java
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/client/MetaScanner.java
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/Bytes.java
*
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/client/TestMetaScanner.java
*
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java
> References to split daughters should not be deleted separately from parent
> META entry
> -
>
> Key: HBASE-8505
> URL: https://issues.apache.org/jira/browse/HBASE-8505
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 0.98.0, 0.94.8, 0.95.1
>
> Attachments: hbase-8505_v1-0.94.patch, hbase-8505_v2-0.94.patch,
> hbase-8505_v2-0.94-reduced.patch, hbase-8505_v2.patch, hbase-8505_v2.patch
>
>
> In CatalogJanitor, we clean up the parent regions whose daughters does not
> have any more references to their parent regions. In doing so, we do two
> Delete's one for removing the split daughter columns, and the other for
> removing the row.
> The first one seems unnecessary, and causes NPE from concurrent MetaScanner.
> Stack trace:
> {code}
> 2013-05-07 04:49:40,828|machine|INFO|Exception in thread "main"
> java.lang.NullPointerException
> 2013-05-07 04:49:40,828|machine|INFO|at
> org.apache.hadoop.hbase.util.Writables.getWritable(Writables.java:103)
> 2013-05-07 04:49:40,828|machine|INFO|at
> org.apache.hadoop.hbase.util.Writables.getHRegionInfo(Writables.java:147)
> 2013-05-07 04:49:40,829|machine|INFO|at
> org.apache.hadoop.hbase.client.MetaScanner$BlockingMetaScannerVisitor.processRow(MetaScanner.java:406)
> 2013-05-07 04:49:40,829|machine|INFO|at
> org.apache.hadoop.hbase.client.MetaScanner$TableMetaScannerVisitor.processRow(MetaScanner.java:487)
> 2013-05-07 04:49:40,830|machine|INFO|at
> org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:224)
> 2013-05-07 04:49:40,830|machine|INFO|at
> org.apache.hadoop.hbase.client.MetaScanner.access$000(MetaScanner.java:54)
> 2013-05-07 04:49:40,830|machine|INFO|at
> org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:133)
> 2013-05-07 04:49:40,831|machine|INFO|at
> org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:130)
> 2013-05-07 04:49:40,831|machine|INFO|at
> org.apache.hadoop.hbase.client.HConnectionManager.execute(HConnectionManager.java:384)
> 2013-05-07 04:49:40,831|machine|INFO|at
> org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:130)
> 2013-05-07 04:49:40,832|machine|INFO|at
> org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:105)
> 2013-05-07 04:49:40,832|machine|INFO|at
> org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:83)
> 2013-05-07 04:49:40,832|machine|INFO|at
> org.apache.hadoop.hbase.client.MetaScanner.allTableRegions(MetaScanner.java:323)
> 2013-05-07 04:49:40,833|machine|INFO|at
> org.apache.hadoop.hbase.client.HTable.getRegionLocations(HTable.java:485)
> 2013-05-07 04:49:40,833|machine|INFO|at
> org.apache.hadoop.hbase.client.HTable.getStartEndKeys(HTable.java:438)
> {code}
> Master is doing the CatalogJanitor concurrently:
> {code}
> 2013-05-07 04:49:40,636 INFO org.apache.hadoop.hbase.catalog.MetaEditor:
> Deleted daughters references, qualifier=splitA and qualifier=splitB, from
> parent
> IntegrationTestBigLinkedList,\x07\xFB\x98\xB7a\x89\xF5\xE6,1367898577620.4ef1329ff0e8911db998ac8ccd32108d.
> 2013-05-07 04:49:40,666 INFO org.apache.hadoop.hbase.catalog.MetaEditor:
> Deleted region
> IntegrationTestBigLinkedList,\x07\xFB\x98\xB7a\x89\xF5\xE6,1367898577620.4ef1329ff0e8911db998ac8ccd32108d.
> from META
> 2013-05-07 04:49:40,690 INFO org.apache.hadoop.hbase.catalog.MetaEditor:
> Deleted daughters references, qualifier=splitA and qualifier=splitB, from
> parent
> IntegrationTestBigLinkedList,\x0B\xF8n\xEA\xD3\xAA\xA9\x92,1367898577620.b502376df2623cb0be3f0c1664d799a6.
> 2013-05-07 04:49:40,716 INFO org.apache.hadoop.hbase.catalog.MetaEditor:
> Deleted region
> IntegrationTestBigLinkedList,\x0B\xF8n\xEA\xD3\xAA\xA9\x92,1367898577620.b502376d
[jira] [Updated] (HBASE-8564) TestMetricsRegionServer depends on test order
[ https://issues.apache.org/jira/browse/HBASE-8564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-8564: - Attachment: HBASE-8564-1.patch Make MetricsRegion use a singleton factory. > TestMetricsRegionServer depends on test order > - > > Key: HBASE-8564 > URL: https://issues.apache.org/jira/browse/HBASE-8564 > Project: HBase > Issue Type: Bug >Affects Versions: 0.98.0, 0.95.0 >Reporter: Elliott Clark >Assignee: Elliott Clark > Attachments: HBASE-8564-0.patch, HBASE-8564-1.patch > > > TestMetricsRegionServer has failed a few times on trunk. It's passing > depends upon test ordering so will fail sometimes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8505) References to split daughters should not be deleted separately from parent META entry
[
https://issues.apache.org/jira/browse/HBASE-8505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13660044#comment-13660044
]
Hudson commented on HBASE-8505:
---
Integrated in HBase-0.94-security #142 (See
[https://builds.apache.org/job/HBase-0.94-security/142/])
HBASE-8505 References to split daughters should not be deleted separately
from parent META entry (Revision 1483485)
Result = FAILURE
enis :
Files :
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/catalog/MetaEditor.java
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/client/MetaScanner.java
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/Bytes.java
*
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/client/TestMetaScanner.java
*
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java
> References to split daughters should not be deleted separately from parent
> META entry
> -
>
> Key: HBASE-8505
> URL: https://issues.apache.org/jira/browse/HBASE-8505
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 0.98.0, 0.94.8, 0.95.1
>
> Attachments: hbase-8505_v1-0.94.patch, hbase-8505_v2-0.94.patch,
> hbase-8505_v2-0.94-reduced.patch, hbase-8505_v2.patch, hbase-8505_v2.patch
>
>
> In CatalogJanitor, we clean up the parent regions whose daughters does not
> have any more references to their parent regions. In doing so, we do two
> Delete's one for removing the split daughter columns, and the other for
> removing the row.
> The first one seems unnecessary, and causes NPE from concurrent MetaScanner.
> Stack trace:
> {code}
> 2013-05-07 04:49:40,828|machine|INFO|Exception in thread "main"
> java.lang.NullPointerException
> 2013-05-07 04:49:40,828|machine|INFO|at
> org.apache.hadoop.hbase.util.Writables.getWritable(Writables.java:103)
> 2013-05-07 04:49:40,828|machine|INFO|at
> org.apache.hadoop.hbase.util.Writables.getHRegionInfo(Writables.java:147)
> 2013-05-07 04:49:40,829|machine|INFO|at
> org.apache.hadoop.hbase.client.MetaScanner$BlockingMetaScannerVisitor.processRow(MetaScanner.java:406)
> 2013-05-07 04:49:40,829|machine|INFO|at
> org.apache.hadoop.hbase.client.MetaScanner$TableMetaScannerVisitor.processRow(MetaScanner.java:487)
> 2013-05-07 04:49:40,830|machine|INFO|at
> org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:224)
> 2013-05-07 04:49:40,830|machine|INFO|at
> org.apache.hadoop.hbase.client.MetaScanner.access$000(MetaScanner.java:54)
> 2013-05-07 04:49:40,830|machine|INFO|at
> org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:133)
> 2013-05-07 04:49:40,831|machine|INFO|at
> org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:130)
> 2013-05-07 04:49:40,831|machine|INFO|at
> org.apache.hadoop.hbase.client.HConnectionManager.execute(HConnectionManager.java:384)
> 2013-05-07 04:49:40,831|machine|INFO|at
> org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:130)
> 2013-05-07 04:49:40,832|machine|INFO|at
> org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:105)
> 2013-05-07 04:49:40,832|machine|INFO|at
> org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:83)
> 2013-05-07 04:49:40,832|machine|INFO|at
> org.apache.hadoop.hbase.client.MetaScanner.allTableRegions(MetaScanner.java:323)
> 2013-05-07 04:49:40,833|machine|INFO|at
> org.apache.hadoop.hbase.client.HTable.getRegionLocations(HTable.java:485)
> 2013-05-07 04:49:40,833|machine|INFO|at
> org.apache.hadoop.hbase.client.HTable.getStartEndKeys(HTable.java:438)
> {code}
> Master is doing the CatalogJanitor concurrently:
> {code}
> 2013-05-07 04:49:40,636 INFO org.apache.hadoop.hbase.catalog.MetaEditor:
> Deleted daughters references, qualifier=splitA and qualifier=splitB, from
> parent
> IntegrationTestBigLinkedList,\x07\xFB\x98\xB7a\x89\xF5\xE6,1367898577620.4ef1329ff0e8911db998ac8ccd32108d.
> 2013-05-07 04:49:40,666 INFO org.apache.hadoop.hbase.catalog.MetaEditor:
> Deleted region
> IntegrationTestBigLinkedList,\x07\xFB\x98\xB7a\x89\xF5\xE6,1367898577620.4ef1329ff0e8911db998ac8ccd32108d.
> from META
> 2013-05-07 04:49:40,690 INFO org.apache.hadoop.hbase.catalog.MetaEditor:
> Deleted daughters references, qualifier=splitA and qualifier=splitB, from
> parent
> IntegrationTestBigLinkedList,\x0B\xF8n\xEA\xD3\xAA\xA9\x92,1367898577620.b502376df2623cb0be3f0c1664d799a6.
> 2013-05-07 04:49:40,716 INFO org.apache.hadoop.hbase.catalog.MetaEditor:
> Deleted region
> IntegrationTestBigLinkedList,\x0B\xF8n\xEA\xD3\xAA\xA9\x92,1367
[jira] [Updated] (HBASE-8564) TestMetricsRegionServer depends on test order
[ https://issues.apache.org/jira/browse/HBASE-8564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-8564: - Affects Version/s: 0.98.0 0.95.0 Status: Patch Available (was: Open) > TestMetricsRegionServer depends on test order > - > > Key: HBASE-8564 > URL: https://issues.apache.org/jira/browse/HBASE-8564 > Project: HBase > Issue Type: Bug >Affects Versions: 0.95.0, 0.98.0 >Reporter: Elliott Clark >Assignee: Elliott Clark > Attachments: HBASE-8564-0.patch > > > TestMetricsRegionServer has failed a few times on trunk. It's passing > depends upon test ordering so will fail sometimes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8564) TestMetricsRegionServer depends on test order
[ https://issues.apache.org/jira/browse/HBASE-8564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-8564: - Attachment: HBASE-8564-0.patch This patch: * Adds a test case for the compatibility singleton class factory * Cleans up the factory * Makes MetricsRegionServer more easily testable * Cleans up some un-used loggers. > TestMetricsRegionServer depends on test order > - > > Key: HBASE-8564 > URL: https://issues.apache.org/jira/browse/HBASE-8564 > Project: HBase > Issue Type: Bug >Reporter: Elliott Clark >Assignee: Elliott Clark > Attachments: HBASE-8564-0.patch > > > TestMetricsRegionServer has failed a few times on trunk. It's passing > depends upon test ordering so will fail sometimes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8461) Provide the ability to delete multiple snapshots through single command
[ https://issues.apache.org/jira/browse/HBASE-8461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13660027#comment-13660027 ] Ted Yu commented on HBASE-8461: --- Integrated to 0.95 and trunk. Thanks for the review, Matteo. > Provide the ability to delete multiple snapshots through single command > --- > > Key: HBASE-8461 > URL: https://issues.apache.org/jira/browse/HBASE-8461 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu >Assignee: Ted Yu > Fix For: 0.98.0, 0.95.1 > > Attachments: 8461-v1.txt, 8461-v2.txt, 8461-v3.txt, 8461-v4.txt > > > Currently HBaseAdmin#deleteSnapshot() accepts name of single snapshot. > It is desirable to allow user to delete multiple snapshots by issuing one > command. > e.g. user may use regular expression to specify the names of snapshots to > delete. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8539) Double(or tripple ...) ZooKeeper listeners of the same type when Master recovers from ZK SessionExpiredException
[ https://issues.apache.org/jira/browse/HBASE-8539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13660025#comment-13660025 ] Sergey Shelukhin commented on HBASE-8539: - I will commit later today if there are no objections, to all 3 branches > Double(or tripple ...) ZooKeeper listeners of the same type when Master > recovers from ZK SessionExpiredException > > > Key: HBASE-8539 > URL: https://issues.apache.org/jira/browse/HBASE-8539 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 0.98.0, 0.94.7, 0.95.0 >Reporter: Jeffrey Zhong >Assignee: Jeffrey Zhong > Fix For: 0.98.0, 0.94.8, 0.95.1 > > Attachments: double-registered listeners.png, > hbase-8539-0.94-addendum.patch, hbase-8539-0.94.patch, > hbase-8539-addendum.patch, hbase-8539.patch, hbase-8539.patch > > > When Master tries to recover from zookeeper session expired exceptions, we > don't clean old registered listener instances. Therefore, it may end up we > have two(or more) listeners to double handling same events. Attached a screen > shot from debugger to show the issue. > I considered to limit one listener per class while I think that would limit > the listener usage so I choose to clear exiting listeners during recovery for > the fix. > (This issue is unrelated to the issue HBASE-8365 because I verified there is > no dup-listeners when HBASE-8365 happened) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8539) Double(or tripple ...) ZooKeeper listeners of the same type when Master recovers from ZK SessionExpiredException
[ https://issues.apache.org/jira/browse/HBASE-8539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13660023#comment-13660023 ] Sergey Shelukhin commented on HBASE-8539: - +1 > Double(or tripple ...) ZooKeeper listeners of the same type when Master > recovers from ZK SessionExpiredException > > > Key: HBASE-8539 > URL: https://issues.apache.org/jira/browse/HBASE-8539 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 0.98.0, 0.94.7, 0.95.0 >Reporter: Jeffrey Zhong >Assignee: Jeffrey Zhong > Fix For: 0.98.0, 0.94.8, 0.95.1 > > Attachments: double-registered listeners.png, > hbase-8539-0.94-addendum.patch, hbase-8539-0.94.patch, > hbase-8539-addendum.patch, hbase-8539.patch, hbase-8539.patch > > > When Master tries to recover from zookeeper session expired exceptions, we > don't clean old registered listener instances. Therefore, it may end up we > have two(or more) listeners to double handling same events. Attached a screen > shot from debugger to show the issue. > I considered to limit one listener per class while I think that would limit > the listener usage so I choose to clear exiting listeners during recovery for > the fix. > (This issue is unrelated to the issue HBASE-8365 because I verified there is > no dup-listeners when HBASE-8365 happened) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8461) Provide the ability to delete multiple snapshots through single command
[ https://issues.apache.org/jira/browse/HBASE-8461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-8461: -- Fix Version/s: 0.95.1 0.98.0 Hadoop Flags: Reviewed > Provide the ability to delete multiple snapshots through single command > --- > > Key: HBASE-8461 > URL: https://issues.apache.org/jira/browse/HBASE-8461 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu >Assignee: Ted Yu > Fix For: 0.98.0, 0.95.1 > > Attachments: 8461-v1.txt, 8461-v2.txt, 8461-v3.txt, 8461-v4.txt > > > Currently HBaseAdmin#deleteSnapshot() accepts name of single snapshot. > It is desirable to allow user to delete multiple snapshots by issuing one > command. > e.g. user may use regular expression to specify the names of snapshots to > delete. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-8564) TestMetricsRegionServer depends on test order
Elliott Clark created HBASE-8564: Summary: TestMetricsRegionServer depends on test order Key: HBASE-8564 URL: https://issues.apache.org/jira/browse/HBASE-8564 Project: HBase Issue Type: Bug Reporter: Elliott Clark Assignee: Elliott Clark TestMetricsRegionServer has failed a few times on trunk. It's passing depends upon test ordering so will fail sometimes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8539) Double(or tripple ...) ZooKeeper listeners of the same type when Master recovers from ZK SessionExpiredException
[ https://issues.apache.org/jira/browse/HBASE-8539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13660020#comment-13660020 ] Ted Yu commented on HBASE-8539: --- +1 addendum. > Double(or tripple ...) ZooKeeper listeners of the same type when Master > recovers from ZK SessionExpiredException > > > Key: HBASE-8539 > URL: https://issues.apache.org/jira/browse/HBASE-8539 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 0.98.0, 0.94.7, 0.95.0 >Reporter: Jeffrey Zhong >Assignee: Jeffrey Zhong > Fix For: 0.98.0, 0.94.8, 0.95.1 > > Attachments: double-registered listeners.png, > hbase-8539-0.94-addendum.patch, hbase-8539-0.94.patch, > hbase-8539-addendum.patch, hbase-8539.patch, hbase-8539.patch > > > When Master tries to recover from zookeeper session expired exceptions, we > don't clean old registered listener instances. Therefore, it may end up we > have two(or more) listeners to double handling same events. Attached a screen > shot from debugger to show the issue. > I considered to limit one listener per class while I think that would limit > the listener usage so I choose to clear exiting listeners during recovery for > the fix. > (This issue is unrelated to the issue HBASE-8365 because I verified there is > no dup-listeners when HBASE-8365 happened) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8555) FilterList correctness was dominated by sub-filter(list) ordering randomly
[
https://issues.apache.org/jira/browse/HBASE-8555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13660016#comment-13660016
]
Ted Yu commented on HBASE-8555:
---
{code}
+Boolean flag = (this.operator == Operator.MUST_PASS_ONE) ? true : false;
{code}
There is no need to use Boolean type - boolean should suffice.
{code}
-assertTrue(Filter.ReturnCode.INCLUDE == filterMPONE.filterKeyValue(kv));
+assertFalse(Filter.ReturnCode.INCLUDE == filterMPONE.filterKeyValue(kv));
+assertTrue(filterMPONE.filterRow());
{code}
I couldn't get the last assertion pass - neither 0.94 nor trunk.
> FilterList correctness was dominated by sub-filter(list) ordering randomly
> --
>
> Key: HBASE-8555
> URL: https://issues.apache.org/jira/browse/HBASE-8555
> Project: HBase
> Issue Type: Bug
> Components: Filters
>Affects Versions: 0.94.3
>Reporter: Liang Xie
>Assignee: Liang Xie
>Priority: Critical
> Attachments: 8555-trunk-v1.txt, HBASE-8555-0.94.txt
>
>
> say, ther're 10 rows, column value is i%2:
> row0 0
> row1 1
> row2 0
> row3 1
> row4 0
> row5 1
> row6 0
> row7 1
> row8 0
> row9 1
> 1: filter : row filter > row4 ===> row5 row6 row7 row8 row9
> 2: subFilterList: row filter <= row4 && column==0===> row0 row2 row4
> 3.1 filterlist[expected] filter || subFilterList ===> row0 row2 row4 row5
> row6 row7 row8 row9
> 3.2 filterlist[BUGON!] subFilterList || filter ===> row0 row1 row2 row3 row4
> row5 row6 row7 row8 row9
> (Please refer to the new testNestedFilterListWithSCVF case)
> It was found when i managed to transform the following SQL into HBase scan
> statement:
> select xxx from xxx where (pk <= xxx and column1 = xxx) or pk > xxx
> My finding is that we had an assumption for filter methods call sequence:
> e.g. filterRowKey() should be called before filterKeyValue().
> and the orignial filterList.filterRowKey impl broke it due to fast
> short-circuit returning.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8555) FilterList correctness was dominated by sub-filter(list) ordering randomly
[ https://issues.apache.org/jira/browse/HBASE-8555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-8555: -- Attachment: 8555-trunk-v1.txt Patch for trunk. > FilterList correctness was dominated by sub-filter(list) ordering randomly > -- > > Key: HBASE-8555 > URL: https://issues.apache.org/jira/browse/HBASE-8555 > Project: HBase > Issue Type: Bug > Components: Filters >Affects Versions: 0.94.3 >Reporter: Liang Xie >Assignee: Liang Xie >Priority: Critical > Attachments: 8555-trunk-v1.txt, HBASE-8555-0.94.txt > > > say, ther're 10 rows, column value is i%2: > row0 0 > row1 1 > row2 0 > row3 1 > row4 0 > row5 1 > row6 0 > row7 1 > row8 0 > row9 1 > 1: filter : row filter > row4 ===> row5 row6 row7 row8 row9 > 2: subFilterList: row filter <= row4 && column==0===> row0 row2 row4 > 3.1 filterlist[expected] filter || subFilterList ===> row0 row2 row4 row5 > row6 row7 row8 row9 > 3.2 filterlist[BUGON!] subFilterList || filter ===> row0 row1 row2 row3 row4 > row5 row6 row7 row8 row9 > (Please refer to the new testNestedFilterListWithSCVF case) > It was found when i managed to transform the following SQL into HBase scan > statement: > select xxx from xxx where (pk <= xxx and column1 = xxx) or pk > xxx > My finding is that we had an assumption for filter methods call sequence: > e.g. filterRowKey() should be called before filterKeyValue(). > and the orignial filterList.filterRowKey impl broke it due to fast > short-circuit returning. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8553) improve unit-test coverage of package org.apache.hadoop.hbase.mapreduce.hadoopbackport
[ https://issues.apache.org/jira/browse/HBASE-8553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13660005#comment-13660005 ] Jonathan Hsieh commented on HBASE-8553: --- Is the patch relevant for trunk as well or is this because the backport didn't include the tests? > improve unit-test coverage of package > org.apache.hadoop.hbase.mapreduce.hadoopbackport > -- > > Key: HBASE-8553 > URL: https://issues.apache.org/jira/browse/HBASE-8553 > Project: HBase > Issue Type: Test >Affects Versions: 0.94.9 >Reporter: Ivan A. Veselovsky > Attachments: HBASE-8553-0.94--N2.patch > > > The patch is for branch 0.94 only. > The class InputSampler is modified because need to fix bug there: in method > "run(String[] args)" should be > "TotalOrderPartitioner.setPartitionFile(job.getConfiguration(), outf);" > instead of "TotalOrderPartitioner.setPartitionFile(getConf(), outf);". > Otherwise it is impossible to set output file correctly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8505) References to split daughters should not be deleted separately from parent META entry
[
https://issues.apache.org/jira/browse/HBASE-8505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13660002#comment-13660002
]
Hudson commented on HBASE-8505:
---
Integrated in hbase-0.95 #198 (See
[https://builds.apache.org/job/hbase-0.95/198/])
HBASE-8505 References to split daughters should not be deleted separately
from parent META entry (Revision 1483483)
Result = FAILURE
enis :
Files :
*
/hbase/branches/0.95/hbase-client/src/main/java/org/apache/hadoop/hbase/client/MetaScanner.java
*
/hbase/branches/0.95/hbase-common/src/main/java/org/apache/hadoop/hbase/util/Bytes.java
*
/hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/catalog/MetaEditor.java
*
/hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java
*
/hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMetaScanner.java
*
/hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java
> References to split daughters should not be deleted separately from parent
> META entry
> -
>
> Key: HBASE-8505
> URL: https://issues.apache.org/jira/browse/HBASE-8505
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 0.98.0, 0.94.8, 0.95.1
>
> Attachments: hbase-8505_v1-0.94.patch, hbase-8505_v2-0.94.patch,
> hbase-8505_v2-0.94-reduced.patch, hbase-8505_v2.patch, hbase-8505_v2.patch
>
>
> In CatalogJanitor, we clean up the parent regions whose daughters does not
> have any more references to their parent regions. In doing so, we do two
> Delete's one for removing the split daughter columns, and the other for
> removing the row.
> The first one seems unnecessary, and causes NPE from concurrent MetaScanner.
> Stack trace:
> {code}
> 2013-05-07 04:49:40,828|machine|INFO|Exception in thread "main"
> java.lang.NullPointerException
> 2013-05-07 04:49:40,828|machine|INFO|at
> org.apache.hadoop.hbase.util.Writables.getWritable(Writables.java:103)
> 2013-05-07 04:49:40,828|machine|INFO|at
> org.apache.hadoop.hbase.util.Writables.getHRegionInfo(Writables.java:147)
> 2013-05-07 04:49:40,829|machine|INFO|at
> org.apache.hadoop.hbase.client.MetaScanner$BlockingMetaScannerVisitor.processRow(MetaScanner.java:406)
> 2013-05-07 04:49:40,829|machine|INFO|at
> org.apache.hadoop.hbase.client.MetaScanner$TableMetaScannerVisitor.processRow(MetaScanner.java:487)
> 2013-05-07 04:49:40,830|machine|INFO|at
> org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:224)
> 2013-05-07 04:49:40,830|machine|INFO|at
> org.apache.hadoop.hbase.client.MetaScanner.access$000(MetaScanner.java:54)
> 2013-05-07 04:49:40,830|machine|INFO|at
> org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:133)
> 2013-05-07 04:49:40,831|machine|INFO|at
> org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:130)
> 2013-05-07 04:49:40,831|machine|INFO|at
> org.apache.hadoop.hbase.client.HConnectionManager.execute(HConnectionManager.java:384)
> 2013-05-07 04:49:40,831|machine|INFO|at
> org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:130)
> 2013-05-07 04:49:40,832|machine|INFO|at
> org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:105)
> 2013-05-07 04:49:40,832|machine|INFO|at
> org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:83)
> 2013-05-07 04:49:40,832|machine|INFO|at
> org.apache.hadoop.hbase.client.MetaScanner.allTableRegions(MetaScanner.java:323)
> 2013-05-07 04:49:40,833|machine|INFO|at
> org.apache.hadoop.hbase.client.HTable.getRegionLocations(HTable.java:485)
> 2013-05-07 04:49:40,833|machine|INFO|at
> org.apache.hadoop.hbase.client.HTable.getStartEndKeys(HTable.java:438)
> {code}
> Master is doing the CatalogJanitor concurrently:
> {code}
> 2013-05-07 04:49:40,636 INFO org.apache.hadoop.hbase.catalog.MetaEditor:
> Deleted daughters references, qualifier=splitA and qualifier=splitB, from
> parent
> IntegrationTestBigLinkedList,\x07\xFB\x98\xB7a\x89\xF5\xE6,1367898577620.4ef1329ff0e8911db998ac8ccd32108d.
> 2013-05-07 04:49:40,666 INFO org.apache.hadoop.hbase.catalog.MetaEditor:
> Deleted region
> IntegrationTestBigLinkedList,\x07\xFB\x98\xB7a\x89\xF5\xE6,1367898577620.4ef1329ff0e8911db998ac8ccd32108d.
> from META
> 2013-05-07 04:49:40,690 INFO org.apache.hadoop.hbase.catalog.MetaEditor:
> Deleted daughters references, qualifier=splitA and qualifier=splitB, from
> parent
> IntegrationTestBigLinkedList,\x0B\xF8n\xEA\xD3\xAA\xA9\x92,1367898577620.b502376df2623cb0be3f0c1664d799a6.
> 2013-05-07 04:49:40,716 INFO org.apache.hadoop.hbase.catalog.MetaEditor:
> Deleted region
> In
[jira] [Commented] (HBASE-8547) Fix java.lang.RuntimeException: Cached an already cached block
[
https://issues.apache.org/jira/browse/HBASE-8547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13660004#comment-13660004
]
Hudson commented on HBASE-8547:
---
Integrated in hbase-0.95 #198 (See
[https://builds.apache.org/job/hbase-0.95/198/])
HBASE-8547. Fix java.lang.RuntimeException: Cached an already cached block
(Addendum2 to check cached buffers for equality) (Revision 1483478)
Result = FAILURE
enis :
Files :
*
/hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java
> Fix java.lang.RuntimeException: Cached an already cached block
> --
>
> Key: HBASE-8547
> URL: https://issues.apache.org/jira/browse/HBASE-8547
> Project: HBase
> Issue Type: Bug
> Components: io, regionserver
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 0.98.0, 0.94.8, 0.95.1
>
> Attachments: hbase-8547_v1-0.94.patch, hbase-8547_v1-0.94.patch,
> hbase-8547_v1.patch, hbase-8547_v2-0.94-reduced.patch,
> hbase-8547_v2-addendum2.patch, hbase-8547_v2-addendum2.patch,
> hbase-8547_v2-trunk.patch
>
>
> In one test, one of the region servers received the following on 0.94.
> Note HalfStoreFileReader in the stack trace. I think the root cause is that
> after the region is split, the mid point can be in the middle of the block
> (for store files that the mid point is not chosen from). Each half store file
> tries to load the half block and put it in the block cache. Since IdLock is
> instantiated per store file reader, they do not share the same IdLock
> instance, thus does not lock against each other effectively.
> {code}
> 2013-05-12 01:30:37,733 ERROR
> org.apache.hadoop.hbase.regionserver.HRegionServer:ยท
> java.lang.RuntimeException: Cached an already cached block
> at
> org.apache.hadoop.hbase.io.hfile.LruBlockCache.cacheBlock(LruBlockCache.java:279)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:353)
> at
> org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.loadDataBlockWithScanInfo(HFileBlockIndex.java:254)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.seekTo(HFileReaderV2.java:480)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.seekTo(HFileReaderV2.java:501)
> at
> org.apache.hadoop.hbase.io.HalfStoreFileReader$1.seekTo(HalfStoreFileReader.java:237)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.seekAtOrAfter(StoreFileScanner.java:226)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.seek(StoreFileScanner.java:145)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.enforceSeek(StoreFileScanner.java:351)
> at
> org.apache.hadoop.hbase.regionserver.KeyValueHeap.pollRealKV(KeyValueHeap.java:354)
> at
> org.apache.hadoop.hbase.regionserver.KeyValueHeap.generalizedSeek(KeyValueHeap.java:312)
> at
> org.apache.hadoop.hbase.regionserver.KeyValueHeap.requestSeek(KeyValueHeap.java:277)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.reseek(StoreScanner.java:543)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:411)
> at
> org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:143)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.populateResult(HRegion.java:3829)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:3896)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:3778)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:3770)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:2643)
> at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.hadoop.hbase.ipc.SecureRpcEngine$Server.call(SecureRpcEngine.java:308)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1426)
> {code}
> I can see two possible fixes:
> # Allow this kind of rare cases in LruBlockCache by not throwing an
> exception.
> # Move the lock instances to upper layer (possibly in CacheConfig), and let
> half hfile readers share the same IdLock implementation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8560) TestMasterShutdown failing in trunk 0.95/trunk -- "Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error)"
[ https://issues.apache.org/jira/browse/HBASE-8560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13660003#comment-13660003 ] Hudson commented on HBASE-8560: --- Integrated in hbase-0.95 #198 (See [https://builds.apache.org/job/hbase-0.95/198/]) HBASE-8560 TestMasterShutdown failing in trunk 0.95/trunk -- "Unable to get data of znode /hbase/meta-region-server because node does not exist (not an error)" (Revision 1483548) Result = FAILURE stack : Files : * /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java > TestMasterShutdown failing in trunk 0.95/trunk -- "Unable to get data of > znode /hbase/meta-region-server because node does not exist (not an error)" > > > Key: HBASE-8560 > URL: https://issues.apache.org/jira/browse/HBASE-8560 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: Jeffrey Zhong > Fix For: 0.98.0, 0.95.1 > > Attachments: hbase-8560.patch, hbase-8560-v2.patch > > > I'm looking at this too... [~jeffreyz] you are too? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
