[jira] [Commented] (HBASE-7658) grant with an empty string as permission should throw an exception
[
https://issues.apache.org/jira/browse/HBASE-7658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628722#comment-13628722
]
Hudson commented on HBASE-7658:
---
Integrated in HBase-0.94 #955 (See
[https://builds.apache.org/job/HBase-0.94/955/])
HBASE-7658 grant with an empty string as permission should throw an
exception (Revision 1466723)
Result = SUCCESS
mbertozzi :
Files :
*
/hbase/branches/0.94/security/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java
* /hbase/branches/0.94/src/main/ruby/hbase/security.rb
* /hbase/branches/0.94/src/main/ruby/shell/commands/grant.rb
* /hbase/branches/0.94/src/main/ruby/shell/commands/revoke.rb
> grant with an empty string as permission should throw an exception
> --
>
> Key: HBASE-7658
> URL: https://issues.apache.org/jira/browse/HBASE-7658
> Project: HBase
> Issue Type: Bug
> Components: security
>Affects Versions: 0.95.2
>Reporter: Matteo Bertozzi
>Assignee: Matteo Bertozzi
>Priority: Trivial
> Fix For: 0.94.7, 0.95.1
>
> Attachments: HBASE-7658-0.94.patch, HBASE-7658-v0.patch,
> HBASE-7658-v1.patch
>
>
> If someone specify an empty permission
> {code}grant 'user', ''{code}
> AccessControlLists.addUserPermission() output a log message and doesn't
> change the permission, but the user doesn't know about it.
> {code}
> if ((actions == null) || (actions.length == 0)) {
> LOG.warn("No actions associated with user
> '"+Bytes.toString(userPerm.getUser())+"'");
> return;
> }
> {code}
> I think we should throw an exception instead of just logging.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7824) Improve master start up time when there is log splitting work
[ https://issues.apache.org/jira/browse/HBASE-7824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628721#comment-13628721 ] Hudson commented on HBASE-7824: --- Integrated in HBase-0.94 #955 (See [https://builds.apache.org/job/HBase-0.94/955/]) HBASE-7824 Improve master start up time when there is log splitting work (Jeffrey Zhong) (Revision 1466725) Result = SUCCESS tedyu : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/catalog/CatalogTracker.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/HMaster.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/handler/MetaServerShutdownHandler.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/handler/ServerShutdownHandler.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperNodeTracker.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/MiniHBaseCluster.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/master/TestMasterFailover.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/TestRSKilledWhenMasterInitializing.java > Improve master start up time when there is log splitting work > - > > Key: HBASE-7824 > URL: https://issues.apache.org/jira/browse/HBASE-7824 > Project: HBase > Issue Type: Bug > Components: master >Reporter: Jeffrey Zhong >Assignee: Jeffrey Zhong > Fix For: 0.94.7 > > Attachments: hbase-7824.patch, hbase-7824-v10.patch, > hbase-7824_v2.patch, hbase-7824_v3.patch, hbase-7824-v7.patch, > hbase-7824-v8.patch, hbase-7824-v9.patch > > > When there is log split work going on, master start up waits till all log > split work completes even though the log split has nothing to do with meta > region servers. > It's a bad behavior considering a master node can run when log split is > happening while its start up is blocking by log split work. > Since master is kind of single point of failure, we should start it ASAP. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8119) Optimize StochasticLoadBalancer
[
https://issues.apache.org/jira/browse/HBASE-8119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628717#comment-13628717
]
Hadoop QA commented on HBASE-8119:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12578136/hbase-8119_v2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 6 new
or modified tests.
{color:red}-1 hadoop2.0{color}. The patch failed to compile against the
hadoop 2.0 profile.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/5257//console
This message is automatically generated.
> Optimize StochasticLoadBalancer
> ---
>
> Key: HBASE-8119
> URL: https://issues.apache.org/jira/browse/HBASE-8119
> Project: HBase
> Issue Type: Bug
> Components: Region Assignment
>Affects Versions: 0.95.0
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
>Priority: Critical
> Fix For: 0.95.1
>
> Attachments: hbase-8119_v2.patch
>
>
> On a 5 node trunk cluster, I ran into a weird problem with
> StochasticLoadBalancer:
> server1 Thu Mar 14 03:42:50 UTC 20130.0 33
> server2 Thu Mar 14 03:47:53 UTC 20130.0 34
> server3 Thu Mar 14 03:46:53 UTC 2013465.0 42
> server4 Thu Mar 14 03:47:53 UTC 201311455.0 282
> server5 Thu Mar 14 03:47:53 UTC 20130.0 34
> Total:5 11920 425
> Notice that server4 has 282 regions, while the others have much less. Plus
> for one table with 260 regions has been super imbalanced:
> {code}
> Regions by Region Server
> Region Server Region Count
> http://server3:60030/ 10
> http://server4:60030/ 250
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7437) Improve CompactSelection
[
https://issues.apache.org/jira/browse/HBASE-7437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hiroshi Ikeda updated HBASE-7437:
-
Attachment: HBASE-7437-V4.patch
Patch v4 from review board.
> Improve CompactSelection
>
>
> Key: HBASE-7437
> URL: https://issues.apache.org/jira/browse/HBASE-7437
> Project: HBase
> Issue Type: Improvement
> Components: Compaction
>Reporter: Hiroshi Ikeda
>Assignee: Hiroshi Ikeda
>Priority: Minor
> Attachments: HBASE-7437.patch, HBASE-7437-V2.patch,
> HBASE-7437-V3.patch, HBASE-7437-V4.patch
>
>
> 1. Using AtomicLong makes CompactSelection simple and improve its performance.
> 2. There are unused fields and methods.
> 3. The fields should be private.
> 4. Assertion in the method finishRequest seems wrong:
> {code}
> public void finishRequest() {
> if (isOffPeakCompaction) {
> long newValueToLog = -1;
> synchronized(compactionCountLock) {
> assert !isOffPeakCompaction : "Double-counting off-peak count for
> compaction";
> {code}
> The above assertion seems almost always false.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8285) HBaseClient never recovers for single HTable.get() calls with no retries when regions move
[ https://issues.apache.org/jira/browse/HBASE-8285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628701#comment-13628701 ] Varun Sharma commented on HBASE-8285: - We could but we know the region location already. So we can simply remove the region location from the cache. On the other hand, deleteCachedLocation will first go from "row"->"region_location" and then remove the location. We already know the "region_location", in this case. > HBaseClient never recovers for single HTable.get() calls with no retries when > regions move > -- > > Key: HBASE-8285 > URL: https://issues.apache.org/jira/browse/HBASE-8285 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 0.94.6.1 >Reporter: Varun Sharma >Assignee: Varun Sharma >Priority: Critical > Fix For: 0.98.0, 0.94.7, 0.95.1 > > Attachments: 8285-0.94.txt, 8285-0.94-v2.txt, 8285-0.94-v3.txt, > 8285-trunk.txt, 8285-trunk-v2.txt > > > Steps to reproduce this bug: > 1) Gracefull restart a region server causing regions to get redistributed. > 2) Client call to this region keeps failing since Meta Cache is never purged > on the client for the region that moved. > Reason behind the bug: > 1) Client continues to hit the old region server. > 2) The old region server throws NotServingRegionException which is not > handled correctly and the META cache entries are never purged for that server > causing the client to keep hitting the old server. > The reason lies in ServerCallable code since we only purge META cache entries > when there is a RetriesExhaustedException, SocketTimeoutException or > ConnectException. However, there is no case check for > NotServingRegionException(s). > Why is this not a problem for Scan(s) and Put(s) ? > a) If a region server is not hosting a region/scanner, then an > UnknownScannerException is thrown which causes a relocateRegion() call > causing a refresh of the META cache for that particular region. > b) For put(s), the processBatchCallback() interface in HConnectionManager is > used which clears out META cache entries for all kinds of exceptions except > DoNotRetryException. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8316) JoinedHeap for non essential column families should reseek instead of seek
[ https://issues.apache.org/jira/browse/HBASE-8316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628690#comment-13628690 ] Hudson commented on HBASE-8316: --- Integrated in hbase-0.95-on-hadoop2 #65 (See [https://builds.apache.org/job/hbase-0.95-on-hadoop2/65/]) HBASE-8316 JoinedHeap for non essential column families should reseek instead of seek (Revision 1466712) Result = FAILURE larsh : Files : * /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java > JoinedHeap for non essential column families should reseek instead of seek > -- > > Key: HBASE-8316 > URL: https://issues.apache.org/jira/browse/HBASE-8316 > Project: HBase > Issue Type: Sub-task > Components: Filters, Performance, regionserver >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl > Fix For: 0.98.0, 0.94.7, 0.95.1 > > Attachments: 8316-0.94.txt, 8316-trunk.txt, 8316-trunk.txt, > FDencode.png, noencode.png > > > This was raised by the Phoenix team. During a profiling session we noticed > that catching the joinedHeap up to the current rows via seek causes a > performance regression, which makes the joinedHeap only efficient when either > a high or low percentage is matched by the filter. > (High is fine, because the joinedHeap will not get behind as often and does > not need to be caught up, low is fine, because the seek isn't happening > frequently). > In our tests we found that the solution is quite simple: Replace seek with > reseek. Patch coming soon. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8266) Master cannot start if TableNotFoundException is thrown while partial table recovery
[
https://issues.apache.org/jira/browse/HBASE-8266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628689#comment-13628689
]
Hudson commented on HBASE-8266:
---
Integrated in hbase-0.95-on-hadoop2 #65 (See
[https://builds.apache.org/job/hbase-0.95-on-hadoop2/65/])
HBASE-8266-Master cannot start if TableNotFoundException is thrown while
partial table recovery(Ram) (Revision 1466566)
Result = FAILURE
ramkrishna :
Files :
*
/hbase/branches/0.95/hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKTable.java
*
/hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/CreateTableHandler.java
*
/hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/EnableTableHandler.java
*
/hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/master/handler/TestCreateTableHandler.java
> Master cannot start if TableNotFoundException is thrown while partial table
> recovery
>
>
> Key: HBASE-8266
> URL: https://issues.apache.org/jira/browse/HBASE-8266
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.94.6, 0.95.0
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Critical
> Fix For: 0.98.0, 0.94.7, 0.95.1
>
> Attachments: HBASE-8266_0.94.patch, HBASE-8266_1.patch,
> HBASE-8266.patch
>
>
> I was trying to create a table. The table creation failed
> {code}
> java.io.IOException: java.util.concurrent.ExecutionException:
> java.lang.IllegalStateException: Could not instantiate a region instance.
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegions(ModifyRegionUtils.java:133)
> at
> org.apache.hadoop.hbase.master.handler.CreateTableHandler.handleCreateHdfsRegions(CreateTableHandler.java:256)
> at
> org.apache.hadoop.hbase.master.handler.CreateTableHandler.handleCreateTable(CreateTableHandler.java:204)
> at
> org.apache.hadoop.hbase.master.handler.CreateTableHandler.process(CreateTableHandler.java:153)
> at
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:130)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: java.util.concurrent.ExecutionException:
> java.lang.IllegalStateException: Could not instantiate a region instance.
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
> at java.util.concurrent.FutureTask.get(FutureTask.java:83)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegions(ModifyRegionUtils.java:126)
> ... 7 more
> Caused by: java.lang.IllegalStateException: Could not instantiate a region
> instance.
> at
> org.apache.hadoop.hbase.regionserver.HRegion.newHRegion(HRegion.java:3765)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.createHRegion(HRegion.java:3870)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils$1.call(ModifyRegionUtils.java:106)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils$1.call(ModifyRegionUtils.java:103)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> ... 3 more
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.newHRegion(HRegion.java:3762)
> ... 11 more
> Caused by: java.lang.NoClassDefFoundError:
> org/apache/hadoop/hbase/CompoundConfiguration$1
> at
> org.apache.hadoop.hbase.CompoundConfiguration.add(CompoundConfiguration.java:82)
> at org.apache.hadoop.hbase.regionserver.HRegion.(HRegion.java:438)
> at org.apache.hadoop.hbase.regionserver.HRegion.(HRegion.java:401)
> ... 16 more
> {code}
> Am not sure of the above failure. The same setup is able to create new
> tables.
> Now the table is already in ENABLING state. The master was restarted.
> Now as the table was found in ENABLING state but not a
[jira] [Commented] (HBASE-8165) Update our protobuf to 2.5 from 2.4.1
[ https://issues.apache.org/jira/browse/HBASE-8165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628688#comment-13628688 ] Hudson commented on HBASE-8165: --- Integrated in hbase-0.95-on-hadoop2 #65 (See [https://builds.apache.org/job/hbase-0.95-on-hadoop2/65/]) HBASE-8165 Update our protobuf to 2.5 from 2.4.1; REVERT (Revision 1466762) HBASE-8165 Update our protobuf to 2.5 from 2.4.1; REVERT (Revision 1466761) HBASE-8165 Update our protobuf to 2.5 from 2.4.1 (Revision 1466557) Result = FAILURE stack : Files : * /hbase/branches/0.95/hbase-server/src/test/protobuf/README.txt stack : Files : * /hbase/branches/0.95/hbase-client/src/main/java/org/apache/hadoop/hbase/ServerName.java * /hbase/branches/0.95/hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java * /hbase/branches/0.95/hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java * /hbase/branches/0.95/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/AccessControlProtos.java * /hbase/branches/0.95/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java * /hbase/branches/0.95/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/AggregateProtos.java * /hbase/branches/0.95/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/AuthenticationProtos.java * /hbase/branches/0.95/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java * /hbase/branches/0.95/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClusterIdProtos.java * /hbase/branches/0.95/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClusterStatusProtos.java * /hbase/branches/0.95/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ComparatorProtos.java * /hbase/branches/0.95/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ErrorHandlingProtos.java * /hbase/branches/0.95/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/FSProtos.java * /hbase/branches/0.95/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/FilterProtos.java * /hbase/branches/0.95/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java * /hbase/branches/0.95/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/HFileProtos.java * /hbase/branches/0.95/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/LoadBalancerProtos.java * /hbase/branches/0.95/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/MapReduceProtos.java * /hbase/branches/0.95/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/MasterAdminProtos.java * /hbase/branches/0.95/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/MasterMonitorProtos.java * /hbase/branches/0.95/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/MasterProtos.java * /hbase/branches/0.95/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/MultiRowMutation.java * /hbase/branches/0.95/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/MultiRowMutationProcessorProtos.java * /hbase/branches/0.95/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/RPCProtos.java * /hbase/branches/0.95/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionServerStatusProtos.java * /hbase/branches/0.95/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/RowProcessorProtos.java * /hbase/branches/0.95/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/SecureBulkLoadProtos.java * /hbase/branches/0.95/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/Tracing.java * /hbase/branches/0.95/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ZooKeeperProtos.java * /hbase/branches/0.95/hbase-protocol/src/main/protobuf/MasterAdmin.proto * /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/HBaseServer.java * /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/rest/protobuf/generated/CellMessage.java * /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/rest/protobuf/generated/CellSetMessage.java * /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/rest/protobuf/generated/ColumnSchemaMessage.java * /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/rest/protobuf/generated/ScannerMessage.java * /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/rest/protobuf/generated/StorageClusterStatusMessage.java * /hbase/branches/0.95/hbase-server/src/main/java/org/apache/ha
[jira] [Commented] (HBASE-8300) TestSplitTransaction fails to delete files due to open handles left when region is split
[
https://issues.apache.org/jira/browse/HBASE-8300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628683#comment-13628683
]
Hadoop QA commented on HBASE-8300:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12578134/hbase-8300_v1-0.95.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/5256//console
This message is automatically generated.
> TestSplitTransaction fails to delete files due to open handles left when
> region is split
>
>
> Key: HBASE-8300
> URL: https://issues.apache.org/jira/browse/HBASE-8300
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 0.95.0
> Environment: Windows
>Reporter: Malie Yin
> Labels: patch
> Fix For: 0.98.0, 0.95.1
>
> Attachments: hbase-8300_v1-0.95.patch, hbase-8300_v1-0.95.patch
>
> Original Estimate: 2h
> Remaining Estimate: 2h
>
> This issue is related to HBASE-6823. logs below.
> TestSplitTransaction
> org.apache.hadoop.hbase.regionserver.TestSplitTransaction
> testWholesomeSplit(org.apache.hadoop.hbase.regionserver.TestSplitTransaction)
> java.io.IOException: Failed delete of
> C:/springSpace/org.apache.hbase.hbase-0.95.0-SNAPSHOT/hbase-server/target/test-data/e5089331-c2bf-43d0-816d-25c6bed71f26/org.apache.hadoop.hbase.regionserver.TestSplitTransaction/table/4851a041b5e9befef50c135b5659243b
> at
> org.apache.hadoop.hbase.regionserver.TestSplitTransaction.teardown(TestSplitTransaction.java:100)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> at
> org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
> at
> org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
> testRollback(org.apache.hadoop.hbase.regionserver.TestSplitTransaction)
> java.io.IOException: Failed delete of
> C:/springSpace/org.apache.hbase.hbase-0.95.0-SNAPSHOT/hbase-server/target/test-data/9140a440-3925-4eaf-8d5d-62744609d775/org.apache.hadoop.hbase.regionserver.TestSplitTransaction/table/6f0ef0cbe59b3fb02c081ad1ffc78a9d
> at
> org.apache.hadoop.hbase.regionserver.TestSplitTransaction.teardown(TestSplitTransaction.java:100)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invok
[jira] [Updated] (HBASE-8324) TestHFileOutputFormat.testMRIncremental* fails against hadoop2 profile
[
https://issues.apache.org/jira/browse/HBASE-8324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hsieh updated HBASE-8324:
--
Affects Version/s: 0.95.0
> TestHFileOutputFormat.testMRIncremental* fails against hadoop2 profile
> --
>
> Key: HBASE-8324
> URL: https://issues.apache.org/jira/browse/HBASE-8324
> Project: HBase
> Issue Type: Sub-task
>Affects Versions: 0.95.0
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
> Fix For: 0.95.1
>
>
> Two tests cases are failing:
> testMRIncrementalLoad, testMRIncrementalloadWithSplit
> {code}
> classname="org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat"
> name="testMRIncrementalLoad">
> java.lang.AssertionError
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertTrue(Assert.java:52)
> at
> org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.runIncrementalPELoad(TestHFileOutputFormat.java:468)
> at
> org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.doIncrementalLoadTest(TestHFileOutputFormat.java:378)
> at
> org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.testMRIncrementalLoad(TestHFileOutputFormat.java:348)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ...
>classname="org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat"
> name="testMRIncrementalLoadWithSplit">
> java.lang.AssertionError
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertTrue(Assert.java:52)
> at
> org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.runIncrementalPELoad(TestHFileOutputFormat.java:468)
> at
> org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.doIncrementalLoadTest(TestHFileOutputFormat.java:378)
> at
> org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.testMRIncrementalLoadWithSplit(TestHFileOutputFormat.java:354)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ...
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8324) TestHFileOutputFormat.testMRIncremental* fails against hadoop2 profile
[
https://issues.apache.org/jira/browse/HBASE-8324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hsieh updated HBASE-8324:
--
Component/s: test
hadoop2
> TestHFileOutputFormat.testMRIncremental* fails against hadoop2 profile
> --
>
> Key: HBASE-8324
> URL: https://issues.apache.org/jira/browse/HBASE-8324
> Project: HBase
> Issue Type: Sub-task
> Components: hadoop2, test
>Affects Versions: 0.95.0
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
> Fix For: 0.95.1
>
>
> Two tests cases are failing:
> testMRIncrementalLoad, testMRIncrementalloadWithSplit
> {code}
> classname="org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat"
> name="testMRIncrementalLoad">
> java.lang.AssertionError
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertTrue(Assert.java:52)
> at
> org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.runIncrementalPELoad(TestHFileOutputFormat.java:468)
> at
> org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.doIncrementalLoadTest(TestHFileOutputFormat.java:378)
> at
> org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.testMRIncrementalLoad(TestHFileOutputFormat.java:348)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ...
>classname="org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat"
> name="testMRIncrementalLoadWithSplit">
> java.lang.AssertionError
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertTrue(Assert.java:52)
> at
> org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.runIncrementalPELoad(TestHFileOutputFormat.java:468)
> at
> org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.doIncrementalLoadTest(TestHFileOutputFormat.java:378)
> at
> org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.testMRIncrementalLoadWithSplit(TestHFileOutputFormat.java:354)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ...
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-8324) TestHFileOutputFormat.testMRIncremental* fails against hadoop2 profile
Jonathan Hsieh created HBASE-8324:
-
Summary: TestHFileOutputFormat.testMRIncremental* fails against
hadoop2 profile
Key: HBASE-8324
URL: https://issues.apache.org/jira/browse/HBASE-8324
Project: HBase
Issue Type: Sub-task
Reporter: Jonathan Hsieh
Two tests cases are failing:
testMRIncrementalLoad, testMRIncrementalloadWithSplit
{code}
java.lang.AssertionError
at org.junit.Assert.fail(Assert.java:86)
at org.junit.Assert.assertTrue(Assert.java:41)
at org.junit.Assert.assertTrue(Assert.java:52)
at
org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.runIncrementalPELoad(TestHFileOutputFormat.java:468)
at
org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.doIncrementalLoadTest(TestHFileOutputFormat.java:378)
at
org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.testMRIncrementalLoad(TestHFileOutputFormat.java:348)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
...
java.lang.AssertionError
at org.junit.Assert.fail(Assert.java:86)
at org.junit.Assert.assertTrue(Assert.java:41)
at org.junit.Assert.assertTrue(Assert.java:52)
at
org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.runIncrementalPELoad(TestHFileOutputFormat.java:468)
at
org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.doIncrementalLoadTest(TestHFileOutputFormat.java:378)
at
org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.testMRIncrementalLoadWithSplit(TestHFileOutputFormat.java:354)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
...
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-8324) TestHFileOutputFormat.testMRIncremental* fails against hadoop2 profile
[
https://issues.apache.org/jira/browse/HBASE-8324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hsieh reassigned HBASE-8324:
-
Assignee: Jonathan Hsieh
> TestHFileOutputFormat.testMRIncremental* fails against hadoop2 profile
> --
>
> Key: HBASE-8324
> URL: https://issues.apache.org/jira/browse/HBASE-8324
> Project: HBase
> Issue Type: Sub-task
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
> Fix For: 0.95.1
>
>
> Two tests cases are failing:
> testMRIncrementalLoad, testMRIncrementalloadWithSplit
> {code}
> classname="org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat"
> name="testMRIncrementalLoad">
> java.lang.AssertionError
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertTrue(Assert.java:52)
> at
> org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.runIncrementalPELoad(TestHFileOutputFormat.java:468)
> at
> org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.doIncrementalLoadTest(TestHFileOutputFormat.java:378)
> at
> org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.testMRIncrementalLoad(TestHFileOutputFormat.java:348)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ...
>classname="org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat"
> name="testMRIncrementalLoadWithSplit">
> java.lang.AssertionError
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertTrue(Assert.java:52)
> at
> org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.runIncrementalPELoad(TestHFileOutputFormat.java:468)
> at
> org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.doIncrementalLoadTest(TestHFileOutputFormat.java:378)
> at
> org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.testMRIncrementalLoadWithSplit(TestHFileOutputFormat.java:354)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ...
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8165) Update our protobuf to 2.5 from 2.4.1
[ https://issues.apache.org/jira/browse/HBASE-8165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628680#comment-13628680 ] Enis Soztutar commented on HBASE-8165: -- Looks interesting except for the name. > Update our protobuf to 2.5 from 2.4.1 > - > > Key: HBASE-8165 > URL: https://issues.apache.org/jira/browse/HBASE-8165 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack > Fix For: 0.95.1 > > Attachments: 8165.txt, 8165v2.txt, 8165v3.txt, 8165v4.txt, 8165v5.txt > > > Update to new 2.5 pb. Some speedups and a new PARSER idiom that bypasses > making a builder. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7704) migration tool that checks presence of HFile V1 files
[ https://issues.apache.org/jira/browse/HBASE-7704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Himanshu Vashishtha updated HBASE-7704: --- Attachment: HBase-7704-v4.patch Incorporated Ted's comments. Please review. > migration tool that checks presence of HFile V1 files > - > > Key: HBASE-7704 > URL: https://issues.apache.org/jira/browse/HBASE-7704 > Project: HBase > Issue Type: Task >Reporter: Ted Yu >Assignee: Himanshu Vashishtha >Priority: Blocker > Fix For: 0.95.1 > > Attachments: HBase-7704-v1.patch, HBase-7704-v2.patch, > HBase-7704-v3.patch, HBase-7704-v4.patch > > > Below was Stack's comment from HBASE-7660: > Regards the migration 'tool', or 'tool' to check for presence of v1 files, I > imagine it as an addition to the hfile tool > http://hbase.apache.org/book.html#hfile_tool2 The hfile tool already takes a > bunch of args including printing out meta. We could add an option to print > out version only – or return 1 if version 1 or some such – and then do a bit > of code to just list all hfiles and run this script against each. Could MR it > if too many files. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7122) Proper warning message when opening a log file with no entries (idle cluster)
[
https://issues.apache.org/jira/browse/HBASE-7122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Himanshu Vashishtha updated HBASE-7122:
---
Attachment: HBase-7122-95-v3.patch
done the formatting.
> Proper warning message when opening a log file with no entries (idle cluster)
> -
>
> Key: HBASE-7122
> URL: https://issues.apache.org/jira/browse/HBASE-7122
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Affects Versions: 0.94.2
>Reporter: Himanshu Vashishtha
>Assignee: Himanshu Vashishtha
> Fix For: 0.95.1
>
> Attachments: HBase-7122-94.patch, HBase-7122-95.patch,
> HBase-7122-95-v2.patch, HBase-7122-95-v3.patch, HBase-7122.patch,
> HBASE-7122.v2.patch
>
>
> In case the cluster is idle and the log has rolled (offset to 0),
> replicationSource tries to open the log and gets an EOF exception. This gets
> printed after every 10 sec until an entry is inserted in it.
> {code}
> 2012-11-07 15:47:40,924 DEBUG regionserver.ReplicationSource
> (ReplicationSource.java:openReader(487)) - Opening log for replication
> c0315.hal.cloudera.com%2C40020%2C1352324202860.1352327804874 at 0
> 2012-11-07 15:47:40,926 WARN regionserver.ReplicationSource
> (ReplicationSource.java:openReader(543)) - 1 Got:
> java.io.EOFException
> at java.io.DataInputStream.readFully(DataInputStream.java:180)
> at java.io.DataInputStream.readFully(DataInputStream.java:152)
> at org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1508)
> at
> org.apache.hadoop.io.SequenceFile$Reader.(SequenceFile.java:1486)
> at
> org.apache.hadoop.io.SequenceFile$Reader.(SequenceFile.java:1475)
> at
> org.apache.hadoop.io.SequenceFile$Reader.(SequenceFile.java:1470)
> at
> org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader$WALReader.(SequenceFileLogReader.java:55)
> at
> org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader.init(SequenceFileLogReader.java:175)
> at
> org.apache.hadoop.hbase.regionserver.wal.HLog.getReader(HLog.java:716)
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.openReader(ReplicationSource.java:491)
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.run(ReplicationSource.java:290)
> 2012-11-07 15:47:40,927 WARN regionserver.ReplicationSource
> (ReplicationSource.java:openReader(547)) - Waited too long for this file,
> considering dumping
> 2012-11-07 15:47:40,927 DEBUG regionserver.ReplicationSource
> (ReplicationSource.java:sleepForRetries(562)) - Unable to open a reader,
> sleeping 1000 times 10
> {code}
> We should reduce the log spewing in this case (or some informative message,
> based on the offset).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8165) Update our protobuf to 2.5 from 2.4.1
[ https://issues.apache.org/jira/browse/HBASE-8165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628667#comment-13628667 ] stack commented on HBASE-8165: -- This is where we are going next (via Andrew): http://kentonv.github.io/capnproto/ > Update our protobuf to 2.5 from 2.4.1 > - > > Key: HBASE-8165 > URL: https://issues.apache.org/jira/browse/HBASE-8165 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack > Fix For: 0.95.1 > > Attachments: 8165.txt, 8165v2.txt, 8165v3.txt, 8165v4.txt, 8165v5.txt > > > Update to new 2.5 pb. Some speedups and a new PARSER idiom that bypasses > making a builder. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7704) migration tool that checks presence of HFile V1 files
[ https://issues.apache.org/jira/browse/HBASE-7704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628666#comment-13628666 ] Himanshu Vashishtha commented on HBASE-7704: bq. For ExecutionException, the print may be eclipsed by successive logs. I think we should store such error and print warning at the end. The exception is caught in the processRegion method, and the file is added to the corrupt files data structure, which is printed at the end of the method call. I think that should suffice. Will correct the grammatical errors in the next patch. > migration tool that checks presence of HFile V1 files > - > > Key: HBASE-7704 > URL: https://issues.apache.org/jira/browse/HBASE-7704 > Project: HBase > Issue Type: Task >Reporter: Ted Yu >Assignee: Himanshu Vashishtha >Priority: Blocker > Fix For: 0.95.1 > > Attachments: HBase-7704-v1.patch, HBase-7704-v2.patch, > HBase-7704-v3.patch > > > Below was Stack's comment from HBASE-7660: > Regards the migration 'tool', or 'tool' to check for presence of v1 files, I > imagine it as an addition to the hfile tool > http://hbase.apache.org/book.html#hfile_tool2 The hfile tool already takes a > bunch of args including printing out meta. We could add an option to print > out version only – or return 1 if version 1 or some such – and then do a bit > of code to just list all hfiles and run this script against each. Could MR it > if too many files. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8165) Update our protobuf to 2.5 from 2.4.1
[ https://issues.apache.org/jira/browse/HBASE-8165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628659#comment-13628659 ] Lars Hofhansl commented on HBASE-8165: -- Did I mention that I do not like PB? :) > Update our protobuf to 2.5 from 2.4.1 > - > > Key: HBASE-8165 > URL: https://issues.apache.org/jira/browse/HBASE-8165 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack > Fix For: 0.95.1 > > Attachments: 8165.txt, 8165v2.txt, 8165v3.txt, 8165v4.txt, 8165v5.txt > > > Update to new 2.5 pb. Some speedups and a new PARSER idiom that bypasses > making a builder. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7704) migration tool that checks presence of HFile V1 files
[
https://issues.apache.org/jira/browse/HBASE-7704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628655#comment-13628655
]
Ted Yu commented on HBASE-7704:
---
In processTable():
{code}
+for (Future f : regionLevelResults) {
+ try {
+if (f.get() != null) {
+ regionsWithHFileV1.add(f.get());
+}
+ } catch (InterruptedException e) {
+System.err.println(e);
+ } catch (ExecutionException e) {
+System.err.println(e); // might be a bad hfile. We print it at the end.
+ }
+}
{code}
For ExecutionException, the print may be eclipsed by successive logs. I think
we should store such error and print warning at the end.
Please also fix grammatical errors mentioned above.
> migration tool that checks presence of HFile V1 files
> -
>
> Key: HBASE-7704
> URL: https://issues.apache.org/jira/browse/HBASE-7704
> Project: HBase
> Issue Type: Task
>Reporter: Ted Yu
>Assignee: Himanshu Vashishtha
>Priority: Blocker
> Fix For: 0.95.1
>
> Attachments: HBase-7704-v1.patch, HBase-7704-v2.patch,
> HBase-7704-v3.patch
>
>
> Below was Stack's comment from HBASE-7660:
> Regards the migration 'tool', or 'tool' to check for presence of v1 files, I
> imagine it as an addition to the hfile tool
> http://hbase.apache.org/book.html#hfile_tool2 The hfile tool already takes a
> bunch of args including printing out meta. We could add an option to print
> out version only – or return 1 if version 1 or some such – and then do a bit
> of code to just list all hfiles and run this script against each. Could MR it
> if too many files.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8165) Update our protobuf to 2.5 from 2.4.1
[ https://issues.apache.org/jira/browse/HBASE-8165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628648#comment-13628648 ] stack commented on HBASE-8165: -- So, yeah. Passed fine on hadoop1 because no pb in hadoop1 but then fails in hadoop2 since dfsclient blah blah is pb'ing. > Update our protobuf to 2.5 from 2.4.1 > - > > Key: HBASE-8165 > URL: https://issues.apache.org/jira/browse/HBASE-8165 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack > Fix For: 0.95.1 > > Attachments: 8165.txt, 8165v2.txt, 8165v3.txt, 8165v4.txt, 8165v5.txt > > > Update to new 2.5 pb. Some speedups and a new PARSER idiom that bypasses > making a builder. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8165) Update our protobuf to 2.5 from 2.4.1
[ https://issues.apache.org/jira/browse/HBASE-8165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628646#comment-13628646 ] stack commented on HBASE-8165: -- Reverted. We can't update pbs while hadoop generated pb code that is in our classpath was made using older version of protoc. Sorry fo inconvenience caused. (What a PITA). > Update our protobuf to 2.5 from 2.4.1 > - > > Key: HBASE-8165 > URL: https://issues.apache.org/jira/browse/HBASE-8165 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack > Fix For: 0.95.1 > > Attachments: 8165.txt, 8165v2.txt, 8165v3.txt, 8165v4.txt, 8165v5.txt > > > Update to new 2.5 pb. Some speedups and a new PARSER idiom that bypasses > making a builder. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8165) Update our protobuf to 2.5 from 2.4.1
[ https://issues.apache.org/jira/browse/HBASE-8165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628641#comment-13628641 ] stack commented on HBASE-8165: -- Reverting. One sec. > Update our protobuf to 2.5 from 2.4.1 > - > > Key: HBASE-8165 > URL: https://issues.apache.org/jira/browse/HBASE-8165 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack > Fix For: 0.95.1 > > Attachments: 8165.txt, 8165v2.txt, 8165v3.txt, 8165v4.txt, 8165v5.txt > > > Update to new 2.5 pb. Some speedups and a new PARSER idiom that bypasses > making a builder. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8165) Update our protobuf to 2.5 from 2.4.1
[ https://issues.apache.org/jira/browse/HBASE-8165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628634#comment-13628634 ] Lars Hofhansl commented on HBASE-8165: -- Same as Sergey. I am currently stuck on HBASE-7801 because of this. (I get PB 2.4.1 nicely through my distro, 2.5 I'd have to install/maitain myself. :) ) I'll hold off updating PB then. > Update our protobuf to 2.5 from 2.4.1 > - > > Key: HBASE-8165 > URL: https://issues.apache.org/jira/browse/HBASE-8165 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack > Fix For: 0.95.1 > > Attachments: 8165.txt, 8165v2.txt, 8165v3.txt, 8165v4.txt, 8165v5.txt > > > Update to new 2.5 pb. Some speedups and a new PARSER idiom that bypasses > making a builder. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7801) Allow a deferred sync option per Mutation.
[ https://issues.apache.org/jira/browse/HBASE-7801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628633#comment-13628633 ] Lars Hofhansl commented on HBASE-7801: -- I think I ran into the PB 2.5 vs 2.4.1 issue. > Allow a deferred sync option per Mutation. > -- > > Key: HBASE-7801 > URL: https://issues.apache.org/jira/browse/HBASE-7801 > Project: HBase > Issue Type: Sub-task >Affects Versions: 0.94.6, 0.95.0 >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl > Fix For: 0.98.0, 0.94.7, 0.95.1 > > Attachments: 7801-0.94-v1.txt, 7801-0.94-v2.txt, 7801-0.94-v3.txt, > 7801-0.96-full-v2.txt, 7801-0.96-full-v3.txt, 7801-0.96-full-v4.txt, > 7801-0.96-full-v5.txt, 7801-0.96-v1.txt, 7801-0.96-v6.txt, 7801-0.96-v7.txt, > 7801-0.96-v8.txt, 7801-0.96-v9.txt > > > Won't have time for parent. But a deferred sync option on a per operation > basis comes up quite frequently. > In 0.96 this can be handled cleanly via protobufs and 0.94 we can have a > special mutation attribute. > For batch operation we'd take the safest sync option of any of the mutations. > I.e. if there is at least one that wants to be flushed we'd sync the batch, > if there's none of those but at least one that wants deferred flush we defer > flush the batch, etc. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7801) Allow a deferred sync option per Mutation.
[ https://issues.apache.org/jira/browse/HBASE-7801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628631#comment-13628631 ] Lars Hofhansl commented on HBASE-7801: -- Need to rebase the patch again. Grrr. > Allow a deferred sync option per Mutation. > -- > > Key: HBASE-7801 > URL: https://issues.apache.org/jira/browse/HBASE-7801 > Project: HBase > Issue Type: Sub-task >Affects Versions: 0.94.6, 0.95.0 >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl > Fix For: 0.98.0, 0.94.7, 0.95.1 > > Attachments: 7801-0.94-v1.txt, 7801-0.94-v2.txt, 7801-0.94-v3.txt, > 7801-0.96-full-v2.txt, 7801-0.96-full-v3.txt, 7801-0.96-full-v4.txt, > 7801-0.96-full-v5.txt, 7801-0.96-v1.txt, 7801-0.96-v6.txt, 7801-0.96-v7.txt, > 7801-0.96-v8.txt, 7801-0.96-v9.txt > > > Won't have time for parent. But a deferred sync option on a per operation > basis comes up quite frequently. > In 0.96 this can be handled cleanly via protobufs and 0.94 we can have a > special mutation attribute. > For batch operation we'd take the safest sync option of any of the mutations. > I.e. if there is at least one that wants to be flushed we'd sync the batch, > if there's none of those but at least one that wants deferred flush we defer > flush the batch, etc. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8256) Add category Flaky for tests which are flaky
[
https://issues.apache.org/jira/browse/HBASE-8256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628618#comment-13628618
]
Hadoop QA commented on HBASE-8256:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12578124/trunk-8256_v1.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 18 new
or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.TestZooKeeper
org.junit.runner.manipulation.Filter
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/5255//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5255//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5255//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5255//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5255//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5255//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5255//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5255//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5255//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/5255//console
This message is automatically generated.
> Add category Flaky for tests which are flaky
>
>
> Key: HBASE-8256
> URL: https://issues.apache.org/jira/browse/HBASE-8256
> Project: HBase
> Issue Type: Bug
> Components: build, test
>Affects Versions: 0.98.0, 0.95.1
>Reporter: Jimmy Xiang
>Assignee: Jimmy Xiang
> Attachments: trunk-8256_v0.patch, trunk-8256_v1.patch
>
>
> To make the Jenkin build more useful, it is good to keep it blue/green. We
> can mark those flaky tests flaky, and don't run them by default. However,
> people can still run them. We can also set up a Jekin build just for those
> flaky tests.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8165) Update our protobuf to 2.5 from 2.4.1
[ https://issues.apache.org/jira/browse/HBASE-8165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628607#comment-13628607 ] Ted Yu commented on HBASE-8165: --- @Sergey: Even if you install protobuf 2.5.0 locally, tests based on hadoop 2.0 would still fail because hadoop and HBase use incompatible protobuf versions, embedded in their RPC engine. > Update our protobuf to 2.5 from 2.4.1 > - > > Key: HBASE-8165 > URL: https://issues.apache.org/jira/browse/HBASE-8165 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack > Fix For: 0.95.1 > > Attachments: 8165.txt, 8165v2.txt, 8165v3.txt, 8165v4.txt, 8165v5.txt > > > Update to new 2.5 pb. Some speedups and a new PARSER idiom that bypasses > making a builder. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (HBASE-8165) Update our protobuf to 2.5 from 2.4.1
[ https://issues.apache.org/jira/browse/HBASE-8165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628600#comment-13628600 ] Jonathan Hsieh edited comment on HBASE-8165 at 4/11/13 2:22 AM: \+1 to revert -- working off of a branch before this commit, I've got the trunk against hadoop2 down to 2-3 fails cases in one test class. With the patch we have 200\+ unittest failures. was (Author: jmhsieh): +1 to revert -- working off of a branch before this commit, I've got the trunk against hadoop2 down to 2-3 fails cases in one test class. With the patch we have 200+ unittest failures. > Update our protobuf to 2.5 from 2.4.1 > - > > Key: HBASE-8165 > URL: https://issues.apache.org/jira/browse/HBASE-8165 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack > Fix For: 0.95.1 > > Attachments: 8165.txt, 8165v2.txt, 8165v3.txt, 8165v4.txt, 8165v5.txt > > > Update to new 2.5 pb. Some speedups and a new PARSER idiom that bypasses > making a builder. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8165) Update our protobuf to 2.5 from 2.4.1
[ https://issues.apache.org/jira/browse/HBASE-8165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628600#comment-13628600 ] Jonathan Hsieh commented on HBASE-8165: --- +1 to revert -- working off of a branch before this commit, I've got the trunk against hadoop2 down to 2-3 fails cases in one test class. With the patch we have 200+ unittest failures. > Update our protobuf to 2.5 from 2.4.1 > - > > Key: HBASE-8165 > URL: https://issues.apache.org/jira/browse/HBASE-8165 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack > Fix For: 0.95.1 > > Attachments: 8165.txt, 8165v2.txt, 8165v3.txt, 8165v4.txt, 8165v5.txt > > > Update to new 2.5 pb. Some speedups and a new PARSER idiom that bypasses > making a builder. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8165) Update our protobuf to 2.5 from 2.4.1
[ https://issues.apache.org/jira/browse/HBASE-8165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628596#comment-13628596 ] Sergey Shelukhin commented on HBASE-8165: - Blocking? It requires one to upgrade protobufs but I don't think it's blocking, am I missing something > Update our protobuf to 2.5 from 2.4.1 > - > > Key: HBASE-8165 > URL: https://issues.apache.org/jira/browse/HBASE-8165 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack > Fix For: 0.95.1 > > Attachments: 8165.txt, 8165v2.txt, 8165v3.txt, 8165v4.txt, 8165v5.txt > > > Update to new 2.5 pb. Some speedups and a new PARSER idiom that bypasses > making a builder. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7636) TestDistributedLogSplitting#testThreeRSAbort fails against hadoop 2.0
[
https://issues.apache.org/jira/browse/HBASE-7636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hsieh updated HBASE-7636:
--
Component/s: test
hadoop2
> TestDistributedLogSplitting#testThreeRSAbort fails against hadoop 2.0
> -
>
> Key: HBASE-7636
> URL: https://issues.apache.org/jira/browse/HBASE-7636
> Project: HBase
> Issue Type: Sub-task
> Components: hadoop2, test
>Affects Versions: 0.95.0
>Reporter: Ted Yu
>Assignee: Jonathan Hsieh
> Fix For: 0.98.0, 0.95.1
>
> Attachments: hbase-7636.patch
>
>
> From
> https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/364/testReport/org.apache.hadoop.hbase.master/TestDistributedLogSplitting/testThreeRSAbort/
> :
> {code}
> 2013-01-21 11:49:34,276 DEBUG
> [MASTER_SERVER_OPERATIONS-juno.apache.org,57966,1358768818594-0]
> client.HConnectionManager$HConnectionImplementation(956): Looked up root
> region location, connection=hconnection 0x12f19fe;
> serverName=juno.apache.org,55531,1358768819479
> 2013-01-21 11:49:34,278 INFO
> [MASTER_SERVER_OPERATIONS-juno.apache.org,57966,1358768818594-0]
> catalog.CatalogTracker(576): Failed verification of .META.,,1 at
> address=juno.apache.org,57582,1358768819456;
> org.apache.hadoop.hbase.ipc.HBaseClient$FailedServerException: This server is
> in the failed servers list: juno.apache.org/67.195.138.61:57582
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7636) TestDistributedLogSplitting#testThreeRSAbort fails against hadoop 2.0
[
https://issues.apache.org/jira/browse/HBASE-7636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hsieh updated HBASE-7636:
--
Fix Version/s: 0.95.1
0.98.0
Affects Version/s: 0.95.0
Status: Patch Available (was: Reopened)
> TestDistributedLogSplitting#testThreeRSAbort fails against hadoop 2.0
> -
>
> Key: HBASE-7636
> URL: https://issues.apache.org/jira/browse/HBASE-7636
> Project: HBase
> Issue Type: Sub-task
>Affects Versions: 0.95.0
>Reporter: Ted Yu
>Assignee: Jonathan Hsieh
> Fix For: 0.98.0, 0.95.1
>
> Attachments: hbase-7636.patch
>
>
> From
> https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/364/testReport/org.apache.hadoop.hbase.master/TestDistributedLogSplitting/testThreeRSAbort/
> :
> {code}
> 2013-01-21 11:49:34,276 DEBUG
> [MASTER_SERVER_OPERATIONS-juno.apache.org,57966,1358768818594-0]
> client.HConnectionManager$HConnectionImplementation(956): Looked up root
> region location, connection=hconnection 0x12f19fe;
> serverName=juno.apache.org,55531,1358768819479
> 2013-01-21 11:49:34,278 INFO
> [MASTER_SERVER_OPERATIONS-juno.apache.org,57966,1358768818594-0]
> catalog.CatalogTracker(576): Failed verification of .META.,,1 at
> address=juno.apache.org,57582,1358768819456;
> org.apache.hadoop.hbase.ipc.HBaseClient$FailedServerException: This server is
> in the failed servers list: juno.apache.org/67.195.138.61:57582
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7636) TestDistributedLogSplitting#testThreeRSAbort fails against hadoop 2.0
[
https://issues.apache.org/jira/browse/HBASE-7636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hsieh updated HBASE-7636:
--
Attachment: hbase-7636.patch
I don't like the hardcoding of the value workaround because I don't quite
understand why hfds short-circuit-read causes this test to fail.
However, it does work.
Will be pinging some HDFS folks about what the causes could be.
> TestDistributedLogSplitting#testThreeRSAbort fails against hadoop 2.0
> -
>
> Key: HBASE-7636
> URL: https://issues.apache.org/jira/browse/HBASE-7636
> Project: HBase
> Issue Type: Sub-task
>Reporter: Ted Yu
>Assignee: Jonathan Hsieh
> Attachments: hbase-7636.patch
>
>
> From
> https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/364/testReport/org.apache.hadoop.hbase.master/TestDistributedLogSplitting/testThreeRSAbort/
> :
> {code}
> 2013-01-21 11:49:34,276 DEBUG
> [MASTER_SERVER_OPERATIONS-juno.apache.org,57966,1358768818594-0]
> client.HConnectionManager$HConnectionImplementation(956): Looked up root
> region location, connection=hconnection 0x12f19fe;
> serverName=juno.apache.org,55531,1358768819479
> 2013-01-21 11:49:34,278 INFO
> [MASTER_SERVER_OPERATIONS-juno.apache.org,57966,1358768818594-0]
> catalog.CatalogTracker(576): Failed verification of .META.,,1 at
> address=juno.apache.org,57582,1358768819456;
> org.apache.hadoop.hbase.ipc.HBaseClient$FailedServerException: This server is
> in the failed servers list: juno.apache.org/67.195.138.61:57582
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8119) Optimize StochasticLoadBalancer
[
https://issues.apache.org/jira/browse/HBASE-8119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Enis Soztutar updated HBASE-8119:
-
Attachment: hbase-8119_v2.patch
Had to rebase the patch. Also will run hadoopqa before commit.
> Optimize StochasticLoadBalancer
> ---
>
> Key: HBASE-8119
> URL: https://issues.apache.org/jira/browse/HBASE-8119
> Project: HBase
> Issue Type: Bug
> Components: Region Assignment
>Affects Versions: 0.95.0
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
>Priority: Critical
> Fix For: 0.95.1
>
> Attachments: hbase-8119_v2.patch
>
>
> On a 5 node trunk cluster, I ran into a weird problem with
> StochasticLoadBalancer:
> server1 Thu Mar 14 03:42:50 UTC 20130.0 33
> server2 Thu Mar 14 03:47:53 UTC 20130.0 34
> server3 Thu Mar 14 03:46:53 UTC 2013465.0 42
> server4 Thu Mar 14 03:47:53 UTC 201311455.0 282
> server5 Thu Mar 14 03:47:53 UTC 20130.0 34
> Total:5 11920 425
> Notice that server4 has 282 regions, while the others have much less. Plus
> for one table with 260 regions has been super imbalanced:
> {code}
> Regions by Region Server
> Region Server Region Count
> http://server3:60030/ 10
> http://server4:60030/ 250
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8119) Optimize StochasticLoadBalancer
[
https://issues.apache.org/jira/browse/HBASE-8119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Enis Soztutar updated HBASE-8119:
-
Status: Patch Available (was: Open)
> Optimize StochasticLoadBalancer
> ---
>
> Key: HBASE-8119
> URL: https://issues.apache.org/jira/browse/HBASE-8119
> Project: HBase
> Issue Type: Bug
> Components: Region Assignment
>Affects Versions: 0.95.0
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
>Priority: Critical
> Fix For: 0.95.1
>
> Attachments: hbase-8119_v2.patch
>
>
> On a 5 node trunk cluster, I ran into a weird problem with
> StochasticLoadBalancer:
> server1 Thu Mar 14 03:42:50 UTC 20130.0 33
> server2 Thu Mar 14 03:47:53 UTC 20130.0 34
> server3 Thu Mar 14 03:46:53 UTC 2013465.0 42
> server4 Thu Mar 14 03:47:53 UTC 201311455.0 282
> server5 Thu Mar 14 03:47:53 UTC 20130.0 34
> Total:5 11920 425
> Notice that server4 has 282 regions, while the others have much less. Plus
> for one table with 260 regions has been super imbalanced:
> {code}
> Regions by Region Server
> Region Server Region Count
> http://server3:60030/ 10
> http://server4:60030/ 250
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6870) HTable#coprocessorExec always scan the whole table
[ https://issues.apache.org/jira/browse/HBASE-6870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628586#comment-13628586 ] chunhui shen commented on HBASE-6870: - About the performance: Total 256 regions in the table Concurrent 170 thread Each thread do AggregationClient#rowCount for one region Before the patch: Total 5s+ After the patch: Total 200ms+ > HTable#coprocessorExec always scan the whole table > --- > > Key: HBASE-6870 > URL: https://issues.apache.org/jira/browse/HBASE-6870 > Project: HBase > Issue Type: Improvement > Components: Coprocessors >Affects Versions: 0.94.1, 0.95.0, 0.95.2 >Reporter: chunhui shen >Assignee: chunhui shen >Priority: Critical > Fix For: 0.98.0, 0.95.1 > > Attachments: 6870-v4.txt, HBASE-6870.patch, > HBASE-6870-testPerformance.patch, HBASE-6870v2.patch, HBASE-6870v3.patch, > hbase-6870v5.patch > > > In current logic, HTable#coprocessorExec always scan the whole table, its > efficiency is low and will affect the Regionserver carrying .META. under > large coprocessorExec requests -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8300) TestSplitTransaction fails to delete files due to open handles left when region is split
[ https://issues.apache.org/jira/browse/HBASE-8300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-8300: - Attachment: hbase-8300_v1-0.95.patch Reattching the patch to trigger hadoopqa. Somehow it is not picking this up. > TestSplitTransaction fails to delete files due to open handles left when > region is split > > > Key: HBASE-8300 > URL: https://issues.apache.org/jira/browse/HBASE-8300 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 0.95.0 > Environment: Windows >Reporter: Malie Yin > Labels: patch > Fix For: 0.98.0, 0.95.1 > > Attachments: hbase-8300_v1-0.95.patch, hbase-8300_v1-0.95.patch > > Original Estimate: 2h > Remaining Estimate: 2h > > This issue is related to HBASE-6823. logs below. > TestSplitTransaction > org.apache.hadoop.hbase.regionserver.TestSplitTransaction > testWholesomeSplit(org.apache.hadoop.hbase.regionserver.TestSplitTransaction) > java.io.IOException: Failed delete of > C:/springSpace/org.apache.hbase.hbase-0.95.0-SNAPSHOT/hbase-server/target/test-data/e5089331-c2bf-43d0-816d-25c6bed71f26/org.apache.hadoop.hbase.regionserver.TestSplitTransaction/table/4851a041b5e9befef50c135b5659243b > at > org.apache.hadoop.hbase.regionserver.TestSplitTransaction.teardown(TestSplitTransaction.java:100) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) > at > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) > at > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) > at > org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50) > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238) > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63) > at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236) > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53) > at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229) > at org.junit.runners.ParentRunner.run(ParentRunner.java:309) > at > org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50) > at > org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38) > at > org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467) > at > org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683) > at > org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390) > at > org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197) > testRollback(org.apache.hadoop.hbase.regionserver.TestSplitTransaction) > java.io.IOException: Failed delete of > C:/springSpace/org.apache.hbase.hbase-0.95.0-SNAPSHOT/hbase-server/target/test-data/9140a440-3925-4eaf-8d5d-62744609d775/org.apache.hadoop.hbase.regionserver.TestSplitTransaction/table/6f0ef0cbe59b3fb02c081ad1ffc78a9d > at > org.apache.hadoop.hbase.regionserver.TestSplitTransaction.teardown(TestSplitTransaction.java:100) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) > at > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) > at > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) > at > org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70) > at > org.junit.runners.BlockJUnit4ClassRunne
[jira] [Updated] (HBASE-8300) TestSplitTransaction fails to delete files due to open handles left when region is split
[ https://issues.apache.org/jira/browse/HBASE-8300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-8300: - Status: Open (was: Patch Available) > TestSplitTransaction fails to delete files due to open handles left when > region is split > > > Key: HBASE-8300 > URL: https://issues.apache.org/jira/browse/HBASE-8300 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 0.95.0 > Environment: Windows >Reporter: Malie Yin > Labels: patch > Fix For: 0.98.0, 0.95.1 > > Attachments: hbase-8300_v1-0.95.patch, hbase-8300_v1-0.95.patch > > Original Estimate: 2h > Remaining Estimate: 2h > > This issue is related to HBASE-6823. logs below. > TestSplitTransaction > org.apache.hadoop.hbase.regionserver.TestSplitTransaction > testWholesomeSplit(org.apache.hadoop.hbase.regionserver.TestSplitTransaction) > java.io.IOException: Failed delete of > C:/springSpace/org.apache.hbase.hbase-0.95.0-SNAPSHOT/hbase-server/target/test-data/e5089331-c2bf-43d0-816d-25c6bed71f26/org.apache.hadoop.hbase.regionserver.TestSplitTransaction/table/4851a041b5e9befef50c135b5659243b > at > org.apache.hadoop.hbase.regionserver.TestSplitTransaction.teardown(TestSplitTransaction.java:100) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) > at > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) > at > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) > at > org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50) > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238) > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63) > at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236) > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53) > at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229) > at org.junit.runners.ParentRunner.run(ParentRunner.java:309) > at > org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50) > at > org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38) > at > org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467) > at > org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683) > at > org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390) > at > org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197) > testRollback(org.apache.hadoop.hbase.regionserver.TestSplitTransaction) > java.io.IOException: Failed delete of > C:/springSpace/org.apache.hbase.hbase-0.95.0-SNAPSHOT/hbase-server/target/test-data/9140a440-3925-4eaf-8d5d-62744609d775/org.apache.hadoop.hbase.regionserver.TestSplitTransaction/table/6f0ef0cbe59b3fb02c081ad1ffc78a9d > at > org.apache.hadoop.hbase.regionserver.TestSplitTransaction.teardown(TestSplitTransaction.java:100) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) > at > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) > at > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) > at > org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50) > at org.junit.runners.ParentRunner$3.run
[jira] [Updated] (HBASE-8300) TestSplitTransaction fails to delete files due to open handles left when region is split
[ https://issues.apache.org/jira/browse/HBASE-8300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-8300: - Status: Patch Available (was: Open) > TestSplitTransaction fails to delete files due to open handles left when > region is split > > > Key: HBASE-8300 > URL: https://issues.apache.org/jira/browse/HBASE-8300 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 0.95.0 > Environment: Windows >Reporter: Malie Yin > Labels: patch > Fix For: 0.98.0, 0.95.1 > > Attachments: hbase-8300_v1-0.95.patch, hbase-8300_v1-0.95.patch > > Original Estimate: 2h > Remaining Estimate: 2h > > This issue is related to HBASE-6823. logs below. > TestSplitTransaction > org.apache.hadoop.hbase.regionserver.TestSplitTransaction > testWholesomeSplit(org.apache.hadoop.hbase.regionserver.TestSplitTransaction) > java.io.IOException: Failed delete of > C:/springSpace/org.apache.hbase.hbase-0.95.0-SNAPSHOT/hbase-server/target/test-data/e5089331-c2bf-43d0-816d-25c6bed71f26/org.apache.hadoop.hbase.regionserver.TestSplitTransaction/table/4851a041b5e9befef50c135b5659243b > at > org.apache.hadoop.hbase.regionserver.TestSplitTransaction.teardown(TestSplitTransaction.java:100) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) > at > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) > at > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) > at > org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50) > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238) > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63) > at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236) > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53) > at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229) > at org.junit.runners.ParentRunner.run(ParentRunner.java:309) > at > org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50) > at > org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38) > at > org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467) > at > org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683) > at > org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390) > at > org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197) > testRollback(org.apache.hadoop.hbase.regionserver.TestSplitTransaction) > java.io.IOException: Failed delete of > C:/springSpace/org.apache.hbase.hbase-0.95.0-SNAPSHOT/hbase-server/target/test-data/9140a440-3925-4eaf-8d5d-62744609d775/org.apache.hadoop.hbase.regionserver.TestSplitTransaction/table/6f0ef0cbe59b3fb02c081ad1ffc78a9d > at > org.apache.hadoop.hbase.regionserver.TestSplitTransaction.teardown(TestSplitTransaction.java:100) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) > at > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) > at > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) > at > org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50) > at org.junit.runners.ParentRunner$3.run
[jira] [Commented] (HBASE-8165) Update our protobuf to 2.5 from 2.4.1
[ https://issues.apache.org/jira/browse/HBASE-8165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628576#comment-13628576 ] Enis Soztutar commented on HBASE-8165: -- +1 on revert until we clear out the test situation. This is also blocking commits that change proto files. > Update our protobuf to 2.5 from 2.4.1 > - > > Key: HBASE-8165 > URL: https://issues.apache.org/jira/browse/HBASE-8165 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack > Fix For: 0.95.1 > > Attachments: 8165.txt, 8165v2.txt, 8165v3.txt, 8165v4.txt, 8165v5.txt > > > Update to new 2.5 pb. Some speedups and a new PARSER idiom that bypasses > making a builder. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8317) Seek returns wrong result with PREFIX_TREE Encoding
[
https://issues.apache.org/jira/browse/HBASE-8317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628574#comment-13628574
]
chunhui shen commented on HBASE-8317:
-
bq.batch0_row2* it is behaving bit different for others bit different.
[~ram_krish]
The generated test data is not completely consecutive (in order to reproduce
the bug),
{code}
+ private static ByteBuffer generateFixedTestData(
+for (int i = 0; i < NUM_ROWS_PER_BATCH; ++i) {
+ if (i / 10 % 2 == 1) continue;
{code}
Hope it's your doubt point
> Seek returns wrong result with PREFIX_TREE Encoding
> ---
>
> Key: HBASE-8317
> URL: https://issues.apache.org/jira/browse/HBASE-8317
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.95.0
>Reporter: chunhui shen
>Assignee: chunhui shen
> Attachments: hbase-trunk-8317.patch
>
>
> TestPrefixTreeEncoding#testSeekWithFixedData from the patch could reproduce
> the bug.
> An example of the bug case:
> Suppose the following rows:
> 1.row3/c1:q1/
> 2.row3/c1:q2/
> 3.row3/c1:q3/
> 4.row4/c1:q1/
> 5.row4/c1:q2/
> After seeking the row 'row30', the expected peek KV is row4/c1:q1/, but
> actual is row3/c1:q1/.
> I just fix this bug case in the patch,
> Maybe we can do more for other potential problems if anyone is familiar with
> the code of PREFIX_TREE
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8316) JoinedHeap for non essential column families should reseek instead of seek
[ https://issues.apache.org/jira/browse/HBASE-8316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628571#comment-13628571 ] Hudson commented on HBASE-8316: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #492 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/492/]) HBASE-8316 JoinedHeap for non essential column families should reseek instead of seek (Revision 1466711) Result = FAILURE larsh : Files : * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java > JoinedHeap for non essential column families should reseek instead of seek > -- > > Key: HBASE-8316 > URL: https://issues.apache.org/jira/browse/HBASE-8316 > Project: HBase > Issue Type: Sub-task > Components: Filters, Performance, regionserver >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl > Fix For: 0.98.0, 0.94.7, 0.95.1 > > Attachments: 8316-0.94.txt, 8316-trunk.txt, 8316-trunk.txt, > FDencode.png, noencode.png > > > This was raised by the Phoenix team. During a profiling session we noticed > that catching the joinedHeap up to the current rows via seek causes a > performance regression, which makes the joinedHeap only efficient when either > a high or low percentage is matched by the filter. > (High is fine, because the joinedHeap will not get behind as often and does > not need to be caught up, low is fine, because the seek isn't happening > frequently). > In our tests we found that the solution is quite simple: Replace seek with > reseek. Patch coming soon. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7658) grant with an empty string as permission should throw an exception
[
https://issues.apache.org/jira/browse/HBASE-7658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628572#comment-13628572
]
Hudson commented on HBASE-7658:
---
Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #492 (See
[https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/492/])
HBASE-7658 grant with an empty string as permission should throw an
exception (Revision 1466600)
Result = FAILURE
mbertozzi :
Files :
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java
* /hbase/trunk/hbase-server/src/main/ruby/hbase/security.rb
* /hbase/trunk/hbase-server/src/main/ruby/shell/commands/grant.rb
* /hbase/trunk/hbase-server/src/main/ruby/shell/commands/revoke.rb
> grant with an empty string as permission should throw an exception
> --
>
> Key: HBASE-7658
> URL: https://issues.apache.org/jira/browse/HBASE-7658
> Project: HBase
> Issue Type: Bug
> Components: security
>Affects Versions: 0.95.2
>Reporter: Matteo Bertozzi
>Assignee: Matteo Bertozzi
>Priority: Trivial
> Fix For: 0.94.7, 0.95.1
>
> Attachments: HBASE-7658-0.94.patch, HBASE-7658-v0.patch,
> HBASE-7658-v1.patch
>
>
> If someone specify an empty permission
> {code}grant 'user', ''{code}
> AccessControlLists.addUserPermission() output a log message and doesn't
> change the permission, but the user doesn't know about it.
> {code}
> if ((actions == null) || (actions.length == 0)) {
> LOG.warn("No actions associated with user
> '"+Bytes.toString(userPerm.getUser())+"'");
> return;
> }
> {code}
> I think we should throw an exception instead of just logging.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8266) Master cannot start if TableNotFoundException is thrown while partial table recovery
[
https://issues.apache.org/jira/browse/HBASE-8266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628570#comment-13628570
]
Hudson commented on HBASE-8266:
---
Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #492 (See
[https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/492/])
HBASE-8266-Master cannot start if TableNotFoundException is thrown while
partial table recovery(Ram) (Revision 1466565)
Result = FAILURE
ramkrishna :
Files :
*
/hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKTable.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/CreateTableHandler.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/EnableTableHandler.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/master/handler/TestCreateTableHandler.java
> Master cannot start if TableNotFoundException is thrown while partial table
> recovery
>
>
> Key: HBASE-8266
> URL: https://issues.apache.org/jira/browse/HBASE-8266
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.94.6, 0.95.0
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Critical
> Fix For: 0.98.0, 0.94.7, 0.95.1
>
> Attachments: HBASE-8266_0.94.patch, HBASE-8266_1.patch,
> HBASE-8266.patch
>
>
> I was trying to create a table. The table creation failed
> {code}
> java.io.IOException: java.util.concurrent.ExecutionException:
> java.lang.IllegalStateException: Could not instantiate a region instance.
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegions(ModifyRegionUtils.java:133)
> at
> org.apache.hadoop.hbase.master.handler.CreateTableHandler.handleCreateHdfsRegions(CreateTableHandler.java:256)
> at
> org.apache.hadoop.hbase.master.handler.CreateTableHandler.handleCreateTable(CreateTableHandler.java:204)
> at
> org.apache.hadoop.hbase.master.handler.CreateTableHandler.process(CreateTableHandler.java:153)
> at
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:130)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: java.util.concurrent.ExecutionException:
> java.lang.IllegalStateException: Could not instantiate a region instance.
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
> at java.util.concurrent.FutureTask.get(FutureTask.java:83)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegions(ModifyRegionUtils.java:126)
> ... 7 more
> Caused by: java.lang.IllegalStateException: Could not instantiate a region
> instance.
> at
> org.apache.hadoop.hbase.regionserver.HRegion.newHRegion(HRegion.java:3765)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.createHRegion(HRegion.java:3870)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils$1.call(ModifyRegionUtils.java:106)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils$1.call(ModifyRegionUtils.java:103)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> ... 3 more
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.newHRegion(HRegion.java:3762)
> ... 11 more
> Caused by: java.lang.NoClassDefFoundError:
> org/apache/hadoop/hbase/CompoundConfiguration$1
> at
> org.apache.hadoop.hbase.CompoundConfiguration.add(CompoundConfiguration.java:82)
> at org.apache.hadoop.hbase.regionserver.HRegion.(HRegion.java:438)
> at org.apache.hadoop.hbase.regionserver.HRegion.(HRegion.java:401)
> ... 16 more
> {code}
> Am not sure of the above failure. The same setup is able to create new
> tables.
> Now the table is already in ENABLING state. The master was restarted.
> Now as the table was found in ENABLING state but not added to META the
[jira] [Commented] (HBASE-8320) test-patch.sh should post back compilation error against hadoop 2.0
[
https://issues.apache.org/jira/browse/HBASE-8320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628569#comment-13628569
]
Hudson commented on HBASE-8320:
---
Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #492 (See
[https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/492/])
HBASE-8320 test-patch.sh should post back compilation error against hadoop
2.0 (Revision 1466684)
Result = FAILURE
tedyu :
Files :
* /hbase/trunk/dev-support/test-patch.sh
> test-patch.sh should post back compilation error against hadoop 2.0
> ---
>
> Key: HBASE-8320
> URL: https://issues.apache.org/jira/browse/HBASE-8320
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 0.98.0
>
> Attachments: 8320.txt
>
>
> One example was:
> https://builds.apache.org/job/PreCommit-HBASE-Build/5242/console
> {code}
> if [[ $? != 0 ]] ; then
> JIRA_COMMENT="$JIRA_COMMENT
> {color:red}-1 hadoop2.0{color}. The patch failed to compile against the
> hadoop 2.0 profile."
> cleanupAndExit 1
> fi
> {code}
> cleanupAndExit doesn't post the comment.
> I think the correct call should be:
> {code}
> submitJiraComment 1
> cleanupAndExit 1
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8089) Add type support
[ https://issues.apache.org/jira/browse/HBASE-8089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628566#comment-13628566 ] Owen O'Malley commented on HBASE-8089: -- Nick, The documentation BinarySortableSerde is here: http://hive.apache.org/docs/r0.10.0/api/org/apache/hadoop/hive/serde2/binarysortable/BinarySortableSerDe.html In Hive, it is only used in MapReduce to cut down the cost of the sort during the shuffle. > Add type support > > > Key: HBASE-8089 > URL: https://issues.apache.org/jira/browse/HBASE-8089 > Project: HBase > Issue Type: New Feature > Components: Client >Reporter: Nick Dimiduk > Attachments: HBASE-8089-types.txt, HBASE-8089-types.txt, > HBASE-8089-types.txt, HBASE-8089-types.txt > > > This proposal outlines an improvement to HBase that provides for a set of > types, above and beyond the existing "byte-bucket" strategy. This is intended > to reduce user-level duplication of effort, provide better support for > 3rd-party integration, and provide an overall improved experience for > developers using HBase. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8165) Update our protobuf to 2.5 from 2.4.1
[ https://issues.apache.org/jira/browse/HBASE-8165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628568#comment-13628568 ] Hudson commented on HBASE-8165: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #492 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/492/]) HBASE-8165 Update our protobuf to 2.5 from 2.4.1 (Revision 1466556) Result = FAILURE stack : Files : * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/ServerName.java * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/AccessControlProtos.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/AggregateProtos.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/AuthenticationProtos.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClusterIdProtos.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClusterStatusProtos.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ComparatorProtos.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ErrorHandlingProtos.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/FSProtos.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/FilterProtos.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/HFileProtos.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/LoadBalancerProtos.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/MapReduceProtos.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/MasterAdminProtos.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/MasterMonitorProtos.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/MasterProtos.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/MultiRowMutation.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/MultiRowMutationProcessorProtos.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/RPCProtos.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionServerStatusProtos.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/RowProcessorProtos.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/SecureBulkLoadProtos.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/Tracing.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ZooKeeperProtos.java * /hbase/trunk/hbase-protocol/src/main/protobuf/MasterAdmin.proto * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/HBaseServer.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/rest/protobuf/generated/CellMessage.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/rest/protobuf/generated/CellSetMessage.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/rest/protobuf/generated/ColumnSchemaMessage.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/rest/protobuf/generated/ScannerMessage.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/rest/protobuf/generated/StorageClusterStatusMessage.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/rest/protobuf/generated/TableInfoMessage.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/rest/protobuf/generated/TableListMessage.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/rest/protobuf/generated/TableSchemaMessage.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/rest/protobuf/generated/VersionMessage.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/protobuf/generated/ColumnAggregationProtos.java *
[jira] [Commented] (HBASE-8316) JoinedHeap for non essential column families should reseek instead of seek
[ https://issues.apache.org/jira/browse/HBASE-8316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628561#comment-13628561 ] Hudson commented on HBASE-8316: --- Integrated in HBase-TRUNK #4051 (See [https://builds.apache.org/job/HBase-TRUNK/4051/]) HBASE-8316 JoinedHeap for non essential column families should reseek instead of seek (Revision 1466711) Result = SUCCESS larsh : Files : * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java > JoinedHeap for non essential column families should reseek instead of seek > -- > > Key: HBASE-8316 > URL: https://issues.apache.org/jira/browse/HBASE-8316 > Project: HBase > Issue Type: Sub-task > Components: Filters, Performance, regionserver >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl > Fix For: 0.98.0, 0.94.7, 0.95.1 > > Attachments: 8316-0.94.txt, 8316-trunk.txt, 8316-trunk.txt, > FDencode.png, noencode.png > > > This was raised by the Phoenix team. During a profiling session we noticed > that catching the joinedHeap up to the current rows via seek causes a > performance regression, which makes the joinedHeap only efficient when either > a high or low percentage is matched by the filter. > (High is fine, because the joinedHeap will not get behind as often and does > not need to be caught up, low is fine, because the seek isn't happening > frequently). > In our tests we found that the solution is quite simple: Replace seek with > reseek. Patch coming soon. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7658) grant with an empty string as permission should throw an exception
[
https://issues.apache.org/jira/browse/HBASE-7658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628562#comment-13628562
]
Hudson commented on HBASE-7658:
---
Integrated in HBase-TRUNK #4051 (See
[https://builds.apache.org/job/HBase-TRUNK/4051/])
HBASE-7658 grant with an empty string as permission should throw an
exception (Revision 1466600)
Result = SUCCESS
mbertozzi :
Files :
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java
* /hbase/trunk/hbase-server/src/main/ruby/hbase/security.rb
* /hbase/trunk/hbase-server/src/main/ruby/shell/commands/grant.rb
* /hbase/trunk/hbase-server/src/main/ruby/shell/commands/revoke.rb
> grant with an empty string as permission should throw an exception
> --
>
> Key: HBASE-7658
> URL: https://issues.apache.org/jira/browse/HBASE-7658
> Project: HBase
> Issue Type: Bug
> Components: security
>Affects Versions: 0.95.2
>Reporter: Matteo Bertozzi
>Assignee: Matteo Bertozzi
>Priority: Trivial
> Fix For: 0.94.7, 0.95.1
>
> Attachments: HBASE-7658-0.94.patch, HBASE-7658-v0.patch,
> HBASE-7658-v1.patch
>
>
> If someone specify an empty permission
> {code}grant 'user', ''{code}
> AccessControlLists.addUserPermission() output a log message and doesn't
> change the permission, but the user doesn't know about it.
> {code}
> if ((actions == null) || (actions.length == 0)) {
> LOG.warn("No actions associated with user
> '"+Bytes.toString(userPerm.getUser())+"'");
> return;
> }
> {code}
> I think we should throw an exception instead of just logging.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8266) Master cannot start if TableNotFoundException is thrown while partial table recovery
[
https://issues.apache.org/jira/browse/HBASE-8266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628560#comment-13628560
]
Hudson commented on HBASE-8266:
---
Integrated in HBase-TRUNK #4051 (See
[https://builds.apache.org/job/HBase-TRUNK/4051/])
HBASE-8266-Master cannot start if TableNotFoundException is thrown while
partial table recovery(Ram) (Revision 1466565)
Result = SUCCESS
ramkrishna :
Files :
*
/hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKTable.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/CreateTableHandler.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/EnableTableHandler.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/master/handler/TestCreateTableHandler.java
> Master cannot start if TableNotFoundException is thrown while partial table
> recovery
>
>
> Key: HBASE-8266
> URL: https://issues.apache.org/jira/browse/HBASE-8266
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.94.6, 0.95.0
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Critical
> Fix For: 0.98.0, 0.94.7, 0.95.1
>
> Attachments: HBASE-8266_0.94.patch, HBASE-8266_1.patch,
> HBASE-8266.patch
>
>
> I was trying to create a table. The table creation failed
> {code}
> java.io.IOException: java.util.concurrent.ExecutionException:
> java.lang.IllegalStateException: Could not instantiate a region instance.
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegions(ModifyRegionUtils.java:133)
> at
> org.apache.hadoop.hbase.master.handler.CreateTableHandler.handleCreateHdfsRegions(CreateTableHandler.java:256)
> at
> org.apache.hadoop.hbase.master.handler.CreateTableHandler.handleCreateTable(CreateTableHandler.java:204)
> at
> org.apache.hadoop.hbase.master.handler.CreateTableHandler.process(CreateTableHandler.java:153)
> at
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:130)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: java.util.concurrent.ExecutionException:
> java.lang.IllegalStateException: Could not instantiate a region instance.
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
> at java.util.concurrent.FutureTask.get(FutureTask.java:83)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegions(ModifyRegionUtils.java:126)
> ... 7 more
> Caused by: java.lang.IllegalStateException: Could not instantiate a region
> instance.
> at
> org.apache.hadoop.hbase.regionserver.HRegion.newHRegion(HRegion.java:3765)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.createHRegion(HRegion.java:3870)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils$1.call(ModifyRegionUtils.java:106)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils$1.call(ModifyRegionUtils.java:103)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> ... 3 more
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.newHRegion(HRegion.java:3762)
> ... 11 more
> Caused by: java.lang.NoClassDefFoundError:
> org/apache/hadoop/hbase/CompoundConfiguration$1
> at
> org.apache.hadoop.hbase.CompoundConfiguration.add(CompoundConfiguration.java:82)
> at org.apache.hadoop.hbase.regionserver.HRegion.(HRegion.java:438)
> at org.apache.hadoop.hbase.regionserver.HRegion.(HRegion.java:401)
> ... 16 more
> {code}
> Am not sure of the above failure. The same setup is able to create new
> tables.
> Now the table is already in ENABLING state. The master was restarted.
> Now as the table was found in ENABLING state but not added to META the
> EnableTableHandler
> {code}
[jira] [Commented] (HBASE-8320) test-patch.sh should post back compilation error against hadoop 2.0
[
https://issues.apache.org/jira/browse/HBASE-8320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628559#comment-13628559
]
Hudson commented on HBASE-8320:
---
Integrated in HBase-TRUNK #4051 (See
[https://builds.apache.org/job/HBase-TRUNK/4051/])
HBASE-8320 test-patch.sh should post back compilation error against hadoop
2.0 (Revision 1466684)
Result = SUCCESS
tedyu :
Files :
* /hbase/trunk/dev-support/test-patch.sh
> test-patch.sh should post back compilation error against hadoop 2.0
> ---
>
> Key: HBASE-8320
> URL: https://issues.apache.org/jira/browse/HBASE-8320
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 0.98.0
>
> Attachments: 8320.txt
>
>
> One example was:
> https://builds.apache.org/job/PreCommit-HBASE-Build/5242/console
> {code}
> if [[ $? != 0 ]] ; then
> JIRA_COMMENT="$JIRA_COMMENT
> {color:red}-1 hadoop2.0{color}. The patch failed to compile against the
> hadoop 2.0 profile."
> cleanupAndExit 1
> fi
> {code}
> cleanupAndExit doesn't post the comment.
> I think the correct call should be:
> {code}
> submitJiraComment 1
> cleanupAndExit 1
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (HBASE-7636) TestDistributedLogSplitting#testThreeRSAbort fails against hadoop 2.0
[
https://issues.apache.org/jira/browse/HBASE-7636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628556#comment-13628556
]
Jonathan Hsieh edited comment on HBASE-7636 at 4/11/13 1:45 AM:
This test passes under hadoop2 when hdfs shortcircuit read is off. This was a
feature enabled by default for tests in HBASE-6783.
{code}
mvn clean test -Dtest=TestDistributedLogSplitting#testThreeRSAbort
-Dhadoop.profile=2.0 -Dhbase.tests.use.shortcircuit.reads=false
-Dmaven.test.redirectTestOutputToFile=true
...
---
T E S T S
---
Running org.apache.hadoop.hbase.master.TestDistributedLogSplitting
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 113.299 sec
{code}
was (Author: jmhsieh):
This test passes under hadoop2 when hdfs shortcircuit read is off. This
was a feature enabled by default for tests in HBASE-6783.
The PriviledgedActionException is [~tlipcon]
{code}
mvn clean test -Dtest=TestDistributedLogSplitting#testThreeRSAbort
-Dhadoop.profile=2.0 -Dhbase.tests.use.shortcircuit.reads=false
-Dmaven.test.redirectTestOutputToFile=true
...
---
T E S T S
---
Running org.apache.hadoop.hbase.master.TestDistributedLogSplitting
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 113.299 sec
{code}
> TestDistributedLogSplitting#testThreeRSAbort fails against hadoop 2.0
> -
>
> Key: HBASE-7636
> URL: https://issues.apache.org/jira/browse/HBASE-7636
> Project: HBase
> Issue Type: Sub-task
>Reporter: Ted Yu
>Assignee: Jonathan Hsieh
>
> From
> https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/364/testReport/org.apache.hadoop.hbase.master/TestDistributedLogSplitting/testThreeRSAbort/
> :
> {code}
> 2013-01-21 11:49:34,276 DEBUG
> [MASTER_SERVER_OPERATIONS-juno.apache.org,57966,1358768818594-0]
> client.HConnectionManager$HConnectionImplementation(956): Looked up root
> region location, connection=hconnection 0x12f19fe;
> serverName=juno.apache.org,55531,1358768819479
> 2013-01-21 11:49:34,278 INFO
> [MASTER_SERVER_OPERATIONS-juno.apache.org,57966,1358768818594-0]
> catalog.CatalogTracker(576): Failed verification of .META.,,1 at
> address=juno.apache.org,57582,1358768819456;
> org.apache.hadoop.hbase.ipc.HBaseClient$FailedServerException: This server is
> in the failed servers list: juno.apache.org/67.195.138.61:57582
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7636) TestDistributedLogSplitting#testThreeRSAbort fails against hadoop 2.0
[
https://issues.apache.org/jira/browse/HBASE-7636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628556#comment-13628556
]
Jonathan Hsieh commented on HBASE-7636:
---
This test passes under hadoop2 when hdfs shortcircuit read is off. This was a
feature enabled by default for tests in HBASE-6783.
The PriviledgedActionException is [~tlipcon]
{code}
mvn clean test -Dtest=TestDistributedLogSplitting#testThreeRSAbort
-Dhadoop.profile=2.0 -Dhbase.tests.use.shortcircuit.reads=false
-Dmaven.test.redirectTestOutputToFile=true
...
---
T E S T S
---
Running org.apache.hadoop.hbase.master.TestDistributedLogSplitting
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 113.299 sec
{code}
> TestDistributedLogSplitting#testThreeRSAbort fails against hadoop 2.0
> -
>
> Key: HBASE-7636
> URL: https://issues.apache.org/jira/browse/HBASE-7636
> Project: HBase
> Issue Type: Sub-task
>Reporter: Ted Yu
>Assignee: Jonathan Hsieh
>
> From
> https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/364/testReport/org.apache.hadoop.hbase.master/TestDistributedLogSplitting/testThreeRSAbort/
> :
> {code}
> 2013-01-21 11:49:34,276 DEBUG
> [MASTER_SERVER_OPERATIONS-juno.apache.org,57966,1358768818594-0]
> client.HConnectionManager$HConnectionImplementation(956): Looked up root
> region location, connection=hconnection 0x12f19fe;
> serverName=juno.apache.org,55531,1358768819479
> 2013-01-21 11:49:34,278 INFO
> [MASTER_SERVER_OPERATIONS-juno.apache.org,57966,1358768818594-0]
> catalog.CatalogTracker(576): Failed verification of .META.,,1 at
> address=juno.apache.org,57582,1358768819456;
> org.apache.hadoop.hbase.ipc.HBaseClient$FailedServerException: This server is
> in the failed servers list: juno.apache.org/67.195.138.61:57582
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8322) Re-enable hbase checksums by default
[
https://issues.apache.org/jira/browse/HBASE-8322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628541#comment-13628541
]
Hadoop QA commented on HBASE-8322:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12578122/hbase-8322_v1.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces lines longer than
100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
at
org.apache.hadoop.hbase.regionserver.TestResettingCounters.testResettingCounters(TestResettingCounters.java:97)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/5254//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5254//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5254//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5254//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5254//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5254//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5254//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5254//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5254//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/5254//console
This message is automatically generated.
> Re-enable hbase checksums by default
>
>
> Key: HBASE-8322
> URL: https://issues.apache.org/jira/browse/HBASE-8322
> Project: HBase
> Issue Type: Improvement
> Components: Filesystem Integration
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 0.98.0, 0.95.1
>
> Attachments: hbase-8322_v1.patch
>
>
> Double checksumming issues in HBase level checksums(HBASE-5074) has been
> fixed in HBASE-6868. However, that patch also disables hbase checksums by
> default. I think we should re-enable it by default, and document the
> interaction with shortcircuit reads.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8165) Update our protobuf to 2.5 from 2.4.1
[ https://issues.apache.org/jira/browse/HBASE-8165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628535#comment-13628535 ] Sergey Shelukhin commented on HBASE-8165: - is this going to be reverted or kept? I am wondering whether I should install protobufs 2.5 to generate some things, or not yet :) > Update our protobuf to 2.5 from 2.4.1 > - > > Key: HBASE-8165 > URL: https://issues.apache.org/jira/browse/HBASE-8165 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack > Fix For: 0.95.1 > > Attachments: 8165.txt, 8165v2.txt, 8165v3.txt, 8165v4.txt, 8165v5.txt > > > Update to new 2.5 pb. Some speedups and a new PARSER idiom that bypasses > making a builder. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8322) Re-enable hbase checksums by default
[ https://issues.apache.org/jira/browse/HBASE-8322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628526#comment-13628526 ] Enis Soztutar commented on HBASE-8322: -- One thing that we need to think about is that HDFS-3429 is only committed to Hadoop2. Under Hadoop-1, non-local reads will still cause seeks for checksum, and hbase will redo the checksum. One option is to live with it, since we expect majority of the hfile reads to be local. The other option might be to only enable this by default under Hadoop2+. > Re-enable hbase checksums by default > > > Key: HBASE-8322 > URL: https://issues.apache.org/jira/browse/HBASE-8322 > Project: HBase > Issue Type: Improvement > Components: Filesystem Integration >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: 0.98.0, 0.95.1 > > Attachments: hbase-8322_v1.patch > > > Double checksumming issues in HBase level checksums(HBASE-5074) has been > fixed in HBASE-6868. However, that patch also disables hbase checksums by > default. I think we should re-enable it by default, and document the > interaction with shortcircuit reads. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8256) Add category Flaky for tests which are flaky
[ https://issues.apache.org/jira/browse/HBASE-8256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628525#comment-13628525 ] Jimmy Xiang commented on HBASE-8256: Posted a new patch which is not as radical. The issue with this patch is that if any method is marked as flaky, the whole test class is ignored in runAllTests, although runFlakyTests works fine. This could be related to SUREFIRE-862. Does our surefire have this fix? > Add category Flaky for tests which are flaky > > > Key: HBASE-8256 > URL: https://issues.apache.org/jira/browse/HBASE-8256 > Project: HBase > Issue Type: Bug > Components: build, test >Affects Versions: 0.98.0, 0.95.1 >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang > Attachments: trunk-8256_v0.patch, trunk-8256_v1.patch > > > To make the Jenkin build more useful, it is good to keep it blue/green. We > can mark those flaky tests flaky, and don't run them by default. However, > people can still run them. We can also set up a Jekin build just for those > flaky tests. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8089) Add type support
[
https://issues.apache.org/jira/browse/HBASE-8089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk updated HBASE-8089:
Attachment: HBASE-8089-types.txt
Updates the definition of {{BOOLEAN}} type to support NULL. Updates serialized
definition for {{\{VAR,\}CHAR}} to also invert the termination byte, thus
preserving sort order. Add a working definition for {{STRUCT}}.
> Add type support
>
>
> Key: HBASE-8089
> URL: https://issues.apache.org/jira/browse/HBASE-8089
> Project: HBase
> Issue Type: New Feature
> Components: Client
>Reporter: Nick Dimiduk
> Attachments: HBASE-8089-types.txt, HBASE-8089-types.txt,
> HBASE-8089-types.txt, HBASE-8089-types.txt
>
>
> This proposal outlines an improvement to HBase that provides for a set of
> types, above and beyond the existing "byte-bucket" strategy. This is intended
> to reduce user-level duplication of effort, provide better support for
> 3rd-party integration, and provide an overall improved experience for
> developers using HBase.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8256) Add category Flaky for tests which are flaky
[ https://issues.apache.org/jira/browse/HBASE-8256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-8256: --- Attachment: trunk-8256_v1.patch > Add category Flaky for tests which are flaky > > > Key: HBASE-8256 > URL: https://issues.apache.org/jira/browse/HBASE-8256 > Project: HBase > Issue Type: Bug > Components: build, test >Affects Versions: 0.98.0, 0.95.1 >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang > Attachments: trunk-8256_v0.patch, trunk-8256_v1.patch > > > To make the Jenkin build more useful, it is good to keep it blue/green. We > can mark those flaky tests flaky, and don't run them by default. However, > people can still run them. We can also set up a Jekin build just for those > flaky tests. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-8323) Low hanging checksum improvements
Enis Soztutar created HBASE-8323: Summary: Low hanging checksum improvements Key: HBASE-8323 URL: https://issues.apache.org/jira/browse/HBASE-8323 Project: HBase Issue Type: Improvement Reporter: Enis Soztutar Over at Hadoop land, [~tlipcon] had done some improvements for checksums, a native implementation for CRC32C (HADOOP-7445) and bulk verify of checksums (HADOOP-7444). In HBase, we can do - Also develop a bulk verify API. Regardless of hbase.hstore.bytes.per.checksum we always want to verify of the whole checksum for the hfile block. - Enable NativeCrc32 to be used as a checksum algo. It is not clear how much gain we can expect over pure java CRC32. Though, longer term we should focus on convincing hdfs guys for inline checksums (HDFS-2699) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8089) Add type support
[ https://issues.apache.org/jira/browse/HBASE-8089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628518#comment-13628518 ] Nick Dimiduk commented on HBASE-8089: - bq. Hive includes a standard serialization library that produces serializations that memcmp into the natural sort order, which it uses for MapReduce key serialization. I didn't know about this feature in Hive, I'll check it out. Thanks for the reference, [~owen.omalley]. The memcmp feature is critical for our needs; this is why most existing tools (ie, protobuf) don't work in this context. Do these Hive formats support NULLs -- I'm curious how the trade-off for fixed-width types was handled. How does it handle compound keys? It looks like I have more homework to do :) > Add type support > > > Key: HBASE-8089 > URL: https://issues.apache.org/jira/browse/HBASE-8089 > Project: HBase > Issue Type: New Feature > Components: Client >Reporter: Nick Dimiduk > Attachments: HBASE-8089-types.txt, HBASE-8089-types.txt, > HBASE-8089-types.txt > > > This proposal outlines an improvement to HBase that provides for a set of > types, above and beyond the existing "byte-bucket" strategy. This is intended > to reduce user-level duplication of effort, provide better support for > 3rd-party integration, and provide an overall improved experience for > developers using HBase. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7704) migration tool that checks presence of HFile V1 files
[ https://issues.apache.org/jira/browse/HBASE-7704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628515#comment-13628515 ] Sergey Shelukhin commented on HBASE-7704: - +1 > migration tool that checks presence of HFile V1 files > - > > Key: HBASE-7704 > URL: https://issues.apache.org/jira/browse/HBASE-7704 > Project: HBase > Issue Type: Task >Reporter: Ted Yu >Assignee: Himanshu Vashishtha >Priority: Blocker > Fix For: 0.95.1 > > Attachments: HBase-7704-v1.patch, HBase-7704-v2.patch, > HBase-7704-v3.patch > > > Below was Stack's comment from HBASE-7660: > Regards the migration 'tool', or 'tool' to check for presence of v1 files, I > imagine it as an addition to the hfile tool > http://hbase.apache.org/book.html#hfile_tool2 The hfile tool already takes a > bunch of args including printing out meta. We could add an option to print > out version only – or return 1 if version 1 or some such – and then do a bit > of code to just list all hfiles and run this script against each. Could MR it > if too many files. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8322) Re-enable hbase checksums by default
[ https://issues.apache.org/jira/browse/HBASE-8322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-8322: - Status: Patch Available (was: Open) > Re-enable hbase checksums by default > > > Key: HBASE-8322 > URL: https://issues.apache.org/jira/browse/HBASE-8322 > Project: HBase > Issue Type: Improvement > Components: Filesystem Integration >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: 0.98.0, 0.95.1 > > Attachments: hbase-8322_v1.patch > > > Double checksumming issues in HBase level checksums(HBASE-5074) has been > fixed in HBASE-6868. However, that patch also disables hbase checksums by > default. I think we should re-enable it by default, and document the > interaction with shortcircuit reads. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8322) Re-enable hbase checksums by default
[ https://issues.apache.org/jira/browse/HBASE-8322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-8322: - Attachment: hbase-8322_v1.patch How about something like this: - Adds documentation - Adds default values in hbase-default.xml - Changes default checksum algorithm to CRC32C. - Enables hbase level checksums - Warn the user if dfs.client.read.shortcircuit.skip.checksum is set to true. > Re-enable hbase checksums by default > > > Key: HBASE-8322 > URL: https://issues.apache.org/jira/browse/HBASE-8322 > Project: HBase > Issue Type: Improvement > Components: Filesystem Integration >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: 0.98.0, 0.95.1 > > Attachments: hbase-8322_v1.patch > > > Double checksumming issues in HBase level checksums(HBASE-5074) has been > fixed in HBASE-6868. However, that patch also disables hbase checksums by > default. I think we should re-enable it by default, and document the > interaction with shortcircuit reads. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7801) Allow a deferred sync option per Mutation.
[ https://issues.apache.org/jira/browse/HBASE-7801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628508#comment-13628508 ] Enis Soztutar commented on HBASE-7801: -- bq. I'm thinking about how I could test this. Would need to something like holding the async flush, and doing that on request in a test. If you plug in a mock hlog implementation, and do no-op in sync(), or do a latch there you might be able to do that. However, I suspect it can still get messy. > Allow a deferred sync option per Mutation. > -- > > Key: HBASE-7801 > URL: https://issues.apache.org/jira/browse/HBASE-7801 > Project: HBase > Issue Type: Sub-task >Affects Versions: 0.94.6, 0.95.0 >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl > Fix For: 0.98.0, 0.94.7, 0.95.1 > > Attachments: 7801-0.94-v1.txt, 7801-0.94-v2.txt, 7801-0.94-v3.txt, > 7801-0.96-full-v2.txt, 7801-0.96-full-v3.txt, 7801-0.96-full-v4.txt, > 7801-0.96-full-v5.txt, 7801-0.96-v1.txt, 7801-0.96-v6.txt, 7801-0.96-v7.txt, > 7801-0.96-v8.txt, 7801-0.96-v9.txt > > > Won't have time for parent. But a deferred sync option on a per operation > basis comes up quite frequently. > In 0.96 this can be handled cleanly via protobufs and 0.94 we can have a > special mutation attribute. > For batch operation we'd take the safest sync option of any of the mutations. > I.e. if there is at least one that wants to be flushed we'd sync the batch, > if there's none of those but at least one that wants deferred flush we defer > flush the batch, etc. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7704) migration tool that checks presence of HFile V1 files
[
https://issues.apache.org/jira/browse/HBASE-7704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628506#comment-13628506
]
Hadoop QA commented on HBASE-7704:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12578086/HBase-7704-v3.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/5252//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5252//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5252//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5252//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5252//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5252//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5252//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5252//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5252//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/5252//console
This message is automatically generated.
> migration tool that checks presence of HFile V1 files
> -
>
> Key: HBASE-7704
> URL: https://issues.apache.org/jira/browse/HBASE-7704
> Project: HBase
> Issue Type: Task
>Reporter: Ted Yu
>Assignee: Himanshu Vashishtha
>Priority: Blocker
> Fix For: 0.95.1
>
> Attachments: HBase-7704-v1.patch, HBase-7704-v2.patch,
> HBase-7704-v3.patch
>
>
> Below was Stack's comment from HBASE-7660:
> Regards the migration 'tool', or 'tool' to check for presence of v1 files, I
> imagine it as an addition to the hfile tool
> http://hbase.apache.org/book.html#hfile_tool2 The hfile tool already takes a
> bunch of args including printing out meta. We could add an option to print
> out version only – or return 1 if version 1 or some such – and then do a bit
> of code to just list all hfiles and run this script against each. Could MR it
> if too many files.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8318) TableOutputFormat.TableRecordWriter should accept Increments
[
https://issues.apache.org/jira/browse/HBASE-8318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628501#comment-13628501
]
Nick Dimiduk commented on HBASE-8318:
-
{noformat}
else if (value instanceof Delete) this.table.delete(new
Delete((Delete)value));
+ else if (value instanceof Increment) this.table.increment(new
Increment((Increment)value));
else throw new IOException("Pass a Delete or a Put");
{noformat}
please update the exception text as well.
I'd ask you to add support to mapred.TableOutputFormat as well, but it appears
to have suffered significant drift from the other namespace.
+1.
> TableOutputFormat.TableRecordWriter should accept Increments
>
>
> Key: HBASE-8318
> URL: https://issues.apache.org/jira/browse/HBASE-8318
> Project: HBase
> Issue Type: Bug
>Reporter: Jean-Marc Spaggiari
>Assignee: Jean-Marc Spaggiari
> Attachments: HBASE-8318-v0-trunk.patch
>
>
> TableOutputFormat.TableRecordWriter can take Puts and Deletes but it should
> also accept Increments.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8303) Increse the test timeout to 60s when they are less than 20s
[
https://issues.apache.org/jira/browse/HBASE-8303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628502#comment-13628502
]
Andrew Purtell commented on HBASE-8303:
---
Local tests results with the 0.94 patch:
{noformat}
Results :
Tests run: 866, Failures: 0, Errors: 0, Skipped: 2
{noformat}
> Increse the test timeout to 60s when they are less than 20s
> ---
>
> Key: HBASE-8303
> URL: https://issues.apache.org/jira/browse/HBASE-8303
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.94.7, 0.95.0
>Reporter: Nicolas Liochon
>Assignee: Nicolas Liochon
> Fix For: 0.98.0, 0.95.1
>
> Attachments: 8303-0.94.patch, 8303.v1.patch, 8303.v1.patch
>
>
> Short test timeouts are dangerous because:
> - if the test is executed in the same jvm as another, GC, thread priority
> can play a role
> - we don't know the machine used to execute the tests, nor what's running on
> it;.
> For this reason, a test timeout of 60s allows us to be on the safe side.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8316) JoinedHeap for non essential column families should reseek instead of seek
[ https://issues.apache.org/jira/browse/HBASE-8316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628497#comment-13628497 ] Hudson commented on HBASE-8316: --- Integrated in hbase-0.95 #139 (See [https://builds.apache.org/job/hbase-0.95/139/]) HBASE-8316 JoinedHeap for non essential column families should reseek instead of seek (Revision 1466712) Result = SUCCESS larsh : Files : * /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java > JoinedHeap for non essential column families should reseek instead of seek > -- > > Key: HBASE-8316 > URL: https://issues.apache.org/jira/browse/HBASE-8316 > Project: HBase > Issue Type: Sub-task > Components: Filters, Performance, regionserver >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl > Fix For: 0.98.0, 0.94.7, 0.95.1 > > Attachments: 8316-0.94.txt, 8316-trunk.txt, 8316-trunk.txt, > FDencode.png, noencode.png > > > This was raised by the Phoenix team. During a profiling session we noticed > that catching the joinedHeap up to the current rows via seek causes a > performance regression, which makes the joinedHeap only efficient when either > a high or low percentage is matched by the filter. > (High is fine, because the joinedHeap will not get behind as often and does > not need to be caught up, low is fine, because the seek isn't happening > frequently). > In our tests we found that the solution is quite simple: Replace seek with > reseek. Patch coming soon. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8266) Master cannot start if TableNotFoundException is thrown while partial table recovery
[
https://issues.apache.org/jira/browse/HBASE-8266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628496#comment-13628496
]
Hudson commented on HBASE-8266:
---
Integrated in hbase-0.95 #139 (See
[https://builds.apache.org/job/hbase-0.95/139/])
HBASE-8266-Master cannot start if TableNotFoundException is thrown while
partial table recovery(Ram) (Revision 1466566)
Result = SUCCESS
ramkrishna :
Files :
*
/hbase/branches/0.95/hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKTable.java
*
/hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/CreateTableHandler.java
*
/hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/EnableTableHandler.java
*
/hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/master/handler/TestCreateTableHandler.java
> Master cannot start if TableNotFoundException is thrown while partial table
> recovery
>
>
> Key: HBASE-8266
> URL: https://issues.apache.org/jira/browse/HBASE-8266
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.94.6, 0.95.0
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Critical
> Fix For: 0.98.0, 0.94.7, 0.95.1
>
> Attachments: HBASE-8266_0.94.patch, HBASE-8266_1.patch,
> HBASE-8266.patch
>
>
> I was trying to create a table. The table creation failed
> {code}
> java.io.IOException: java.util.concurrent.ExecutionException:
> java.lang.IllegalStateException: Could not instantiate a region instance.
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegions(ModifyRegionUtils.java:133)
> at
> org.apache.hadoop.hbase.master.handler.CreateTableHandler.handleCreateHdfsRegions(CreateTableHandler.java:256)
> at
> org.apache.hadoop.hbase.master.handler.CreateTableHandler.handleCreateTable(CreateTableHandler.java:204)
> at
> org.apache.hadoop.hbase.master.handler.CreateTableHandler.process(CreateTableHandler.java:153)
> at
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:130)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: java.util.concurrent.ExecutionException:
> java.lang.IllegalStateException: Could not instantiate a region instance.
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
> at java.util.concurrent.FutureTask.get(FutureTask.java:83)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegions(ModifyRegionUtils.java:126)
> ... 7 more
> Caused by: java.lang.IllegalStateException: Could not instantiate a region
> instance.
> at
> org.apache.hadoop.hbase.regionserver.HRegion.newHRegion(HRegion.java:3765)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.createHRegion(HRegion.java:3870)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils$1.call(ModifyRegionUtils.java:106)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils$1.call(ModifyRegionUtils.java:103)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> ... 3 more
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.newHRegion(HRegion.java:3762)
> ... 11 more
> Caused by: java.lang.NoClassDefFoundError:
> org/apache/hadoop/hbase/CompoundConfiguration$1
> at
> org.apache.hadoop.hbase.CompoundConfiguration.add(CompoundConfiguration.java:82)
> at org.apache.hadoop.hbase.regionserver.HRegion.(HRegion.java:438)
> at org.apache.hadoop.hbase.regionserver.HRegion.(HRegion.java:401)
> ... 16 more
> {code}
> Am not sure of the above failure. The same setup is able to create new
> tables.
> Now the table is already in ENABLING state. The master was restarted.
> Now as the table was found in ENABLING state but not added to META the
>
[jira] [Comment Edited] (HBASE-8285) HBaseClient never recovers for single HTable.get() calls with no retries when regions move
[ https://issues.apache.org/jira/browse/HBASE-8285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628482#comment-13628482 ] Lars Hofhansl edited comment on HBASE-8285 at 4/11/13 12:31 AM: Why can't we just add deleteCachedLocation to HConnection (and make it public in HConnectionImplementation) and use it unchanged? We know the row already. was (Author: lhofhansl): Why can't we just add deleteCachedLocation HConnection (and make it public in HConnectionImplementation) and use it unchanged? We know the row already. > HBaseClient never recovers for single HTable.get() calls with no retries when > regions move > -- > > Key: HBASE-8285 > URL: https://issues.apache.org/jira/browse/HBASE-8285 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 0.94.6.1 >Reporter: Varun Sharma >Assignee: Varun Sharma >Priority: Critical > Fix For: 0.98.0, 0.94.7, 0.95.1 > > Attachments: 8285-0.94.txt, 8285-0.94-v2.txt, 8285-0.94-v3.txt, > 8285-trunk.txt, 8285-trunk-v2.txt > > > Steps to reproduce this bug: > 1) Gracefull restart a region server causing regions to get redistributed. > 2) Client call to this region keeps failing since Meta Cache is never purged > on the client for the region that moved. > Reason behind the bug: > 1) Client continues to hit the old region server. > 2) The old region server throws NotServingRegionException which is not > handled correctly and the META cache entries are never purged for that server > causing the client to keep hitting the old server. > The reason lies in ServerCallable code since we only purge META cache entries > when there is a RetriesExhaustedException, SocketTimeoutException or > ConnectException. However, there is no case check for > NotServingRegionException(s). > Why is this not a problem for Scan(s) and Put(s) ? > a) If a region server is not hosting a region/scanner, then an > UnknownScannerException is thrown which causes a relocateRegion() call > causing a refresh of the META cache for that particular region. > b) For put(s), the processBatchCallback() interface in HConnectionManager is > used which clears out META cache entries for all kinds of exceptions except > DoNotRetryException. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8165) Update our protobuf to 2.5 from 2.4.1
[ https://issues.apache.org/jira/browse/HBASE-8165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628495#comment-13628495 ] Hudson commented on HBASE-8165: --- Integrated in hbase-0.95 #139 (See [https://builds.apache.org/job/hbase-0.95/139/]) HBASE-8165 Update our protobuf to 2.5 from 2.4.1 (Revision 1466557) Result = SUCCESS stack : Files : * /hbase/branches/0.95/hbase-client/src/main/java/org/apache/hadoop/hbase/ServerName.java * /hbase/branches/0.95/hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java * /hbase/branches/0.95/hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java * /hbase/branches/0.95/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/AccessControlProtos.java * /hbase/branches/0.95/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java * /hbase/branches/0.95/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/AggregateProtos.java * /hbase/branches/0.95/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/AuthenticationProtos.java * /hbase/branches/0.95/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java * /hbase/branches/0.95/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClusterIdProtos.java * /hbase/branches/0.95/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClusterStatusProtos.java * /hbase/branches/0.95/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ComparatorProtos.java * /hbase/branches/0.95/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ErrorHandlingProtos.java * /hbase/branches/0.95/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/FSProtos.java * /hbase/branches/0.95/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/FilterProtos.java * /hbase/branches/0.95/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java * /hbase/branches/0.95/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/HFileProtos.java * /hbase/branches/0.95/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/LoadBalancerProtos.java * /hbase/branches/0.95/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/MapReduceProtos.java * /hbase/branches/0.95/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/MasterAdminProtos.java * /hbase/branches/0.95/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/MasterMonitorProtos.java * /hbase/branches/0.95/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/MasterProtos.java * /hbase/branches/0.95/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/MultiRowMutation.java * /hbase/branches/0.95/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/MultiRowMutationProcessorProtos.java * /hbase/branches/0.95/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/RPCProtos.java * /hbase/branches/0.95/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionServerStatusProtos.java * /hbase/branches/0.95/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/RowProcessorProtos.java * /hbase/branches/0.95/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/SecureBulkLoadProtos.java * /hbase/branches/0.95/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/Tracing.java * /hbase/branches/0.95/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ZooKeeperProtos.java * /hbase/branches/0.95/hbase-protocol/src/main/protobuf/MasterAdmin.proto * /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/HBaseServer.java * /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/rest/protobuf/generated/CellMessage.java * /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/rest/protobuf/generated/CellSetMessage.java * /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/rest/protobuf/generated/ColumnSchemaMessage.java * /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/rest/protobuf/generated/ScannerMessage.java * /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/rest/protobuf/generated/StorageClusterStatusMessage.java * /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/rest/protobuf/generated/TableInfoMessage.java * /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/rest/protobuf/generated/TableListMessage.java * /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/res
[jira] [Commented] (HBASE-7801) Allow a deferred sync option per Mutation.
[ https://issues.apache.org/jira/browse/HBASE-7801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628494#comment-13628494 ] Lars Hofhansl commented on HBASE-7801: -- Expect a (hopefully) final patch soon. > Allow a deferred sync option per Mutation. > -- > > Key: HBASE-7801 > URL: https://issues.apache.org/jira/browse/HBASE-7801 > Project: HBase > Issue Type: Sub-task >Affects Versions: 0.94.6, 0.95.0 >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl > Fix For: 0.98.0, 0.94.7, 0.95.1 > > Attachments: 7801-0.94-v1.txt, 7801-0.94-v2.txt, 7801-0.94-v3.txt, > 7801-0.96-full-v2.txt, 7801-0.96-full-v3.txt, 7801-0.96-full-v4.txt, > 7801-0.96-full-v5.txt, 7801-0.96-v1.txt, 7801-0.96-v6.txt, 7801-0.96-v7.txt, > 7801-0.96-v8.txt, 7801-0.96-v9.txt > > > Won't have time for parent. But a deferred sync option on a per operation > basis comes up quite frequently. > In 0.96 this can be handled cleanly via protobufs and 0.94 we can have a > special mutation attribute. > For batch operation we'd take the safest sync option of any of the mutations. > I.e. if there is at least one that wants to be flushed we'd sync the batch, > if there's none of those but at least one that wants deferred flush we defer > flush the batch, etc. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8285) HBaseClient never recovers for single HTable.get() calls with no retries when regions move
[ https://issues.apache.org/jira/browse/HBASE-8285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628482#comment-13628482 ] Lars Hofhansl commented on HBASE-8285: -- Why can't we just add deleteCachedLocation HConnection (and make it public in HConnectionImplementation) and use it unchanged? We know the row already. > HBaseClient never recovers for single HTable.get() calls with no retries when > regions move > -- > > Key: HBASE-8285 > URL: https://issues.apache.org/jira/browse/HBASE-8285 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 0.94.6.1 >Reporter: Varun Sharma >Assignee: Varun Sharma >Priority: Critical > Fix For: 0.98.0, 0.94.7, 0.95.1 > > Attachments: 8285-0.94.txt, 8285-0.94-v2.txt, 8285-0.94-v3.txt, > 8285-trunk.txt, 8285-trunk-v2.txt > > > Steps to reproduce this bug: > 1) Gracefull restart a region server causing regions to get redistributed. > 2) Client call to this region keeps failing since Meta Cache is never purged > on the client for the region that moved. > Reason behind the bug: > 1) Client continues to hit the old region server. > 2) The old region server throws NotServingRegionException which is not > handled correctly and the META cache entries are never purged for that server > causing the client to keep hitting the old server. > The reason lies in ServerCallable code since we only purge META cache entries > when there is a RetriesExhaustedException, SocketTimeoutException or > ConnectException. However, there is no case check for > NotServingRegionException(s). > Why is this not a problem for Scan(s) and Put(s) ? > a) If a region server is not hosting a region/scanner, then an > UnknownScannerException is thrown which causes a relocateRegion() call > causing a refresh of the META cache for that particular region. > b) For put(s), the processBatchCallback() interface in HConnectionManager is > used which clears out META cache entries for all kinds of exceptions except > DoNotRetryException. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7636) TestDistributedLogSplitting#testThreeRSAbort fails against hadoop 2.0
[
https://issues.apache.org/jira/browse/HBASE-7636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628481#comment-13628481
]
Jonathan Hsieh commented on HBASE-7636:
---
I bumped the some timeouts higher and improved failure messages in an
experiment.
The test basically creates a table on 6 rs's (thread), kills 3 rs's, and then
makes sure that all regions get back up.
When running this, we get to a state where a region cannot be opened after a RS
is aborted.
{code}
java.lang.AssertionError:
Took too long to get all the regions back online. Have 36 but want at least 41
at org.junit.Assert.fail(Assert.java:88)
at
org.apache.hadoop.hbase.master.TestDistributedLogSplitting.testThreeRSAbort(TestDistributedLogSplitting.java:276)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
{code}
The root is some sort of hadoop access control problem (with different combos
fo IPC Server handler # and user jon.hfs.*)
{code}
2013-04-10 14:46:17,426 ERROR [IPC Server handler 7 on 46469]
security.UserGroupInformation(1370): PriviledgedActionException as:jon.hfs.1
(auth:SIMPLE) cause:org.apache.hadoop.security.AccessControlException:
Can't continue with getBlockLocalPathInfo() authorization. The user
jon.hfs.1 is not allowed to call getBlockLocalPathInfo
{code}
when trying to open the region (different region server codes, but all with the
same region failing):
{code}
2013-04-10 14:44:16,384 ERROR [RS_OPEN_REGION-localhost,50866,1365630214743-0]
handler.OpenRegionHandler(464): Failed open of region=table,,136563021
7156.7c1b2def33bf6627d581ecf6fbb389f3., starting to roll back the global
memstore size.
{code}
> TestDistributedLogSplitting#testThreeRSAbort fails against hadoop 2.0
> -
>
> Key: HBASE-7636
> URL: https://issues.apache.org/jira/browse/HBASE-7636
> Project: HBase
> Issue Type: Sub-task
>Reporter: Ted Yu
>Assignee: Jonathan Hsieh
>
> From
> https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/364/testReport/org.apache.hadoop.hbase.master/TestDistributedLogSplitting/testThreeRSAbort/
> :
> {code}
> 2013-01-21 11:49:34,276 DEBUG
> [MASTER_SERVER_OPERATIONS-juno.apache.org,57966,1358768818594-0]
> client.HConnectionManager$HConnectionImplementation(956): Looked up root
> region location, connection=hconnection 0x12f19fe;
> serverName=juno.apache.org,55531,1358768819479
> 2013-01-21 11:49:34,278 INFO
> [MASTER_SERVER_OPERATIONS-juno.apache.org,57966,1358768818594-0]
> catalog.CatalogTracker(576): Failed verification of .META.,,1 at
> address=juno.apache.org,57582,1358768819456;
> org.apache.hadoop.hbase.ipc.HBaseClient$FailedServerException: This server is
> in the failed servers list: juno.apache.org/67.195.138.61:57582
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8089) Add type support
[ https://issues.apache.org/jira/browse/HBASE-8089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628470#comment-13628470 ] Owen O'Malley commented on HBASE-8089: -- You should also look at the other types from Hive: * Byte * Timestamp * List * Map * Union Hive includes a standard serialization library that produces serializations that memcmp into the natural sort order, which it uses for MapReduce key serialization. > Add type support > > > Key: HBASE-8089 > URL: https://issues.apache.org/jira/browse/HBASE-8089 > Project: HBase > Issue Type: New Feature > Components: Client >Reporter: Nick Dimiduk > Attachments: HBASE-8089-types.txt, HBASE-8089-types.txt, > HBASE-8089-types.txt > > > This proposal outlines an improvement to HBase that provides for a set of > types, above and beyond the existing "byte-bucket" strategy. This is intended > to reduce user-level duplication of effort, provide better support for > 3rd-party integration, and provide an overall improved experience for > developers using HBase. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8089) Add type support
[ https://issues.apache.org/jira/browse/HBASE-8089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628465#comment-13628465 ] Owen O'Malley commented on HBASE-8089: -- Nick, ORC gets a lot of mileage by doing type-specific compression. In particular, the integer columns use a vint representation (protobuf vint encoding) and run length encoding. The string columns use an adaptive dictionary (the writer switches between dictionary or direct encoding based on the 100k initial values) approach. That allows both tighter representation before turning on the relatively expensive zlib or even tighter encodings when combined with zlib. > Add type support > > > Key: HBASE-8089 > URL: https://issues.apache.org/jira/browse/HBASE-8089 > Project: HBase > Issue Type: New Feature > Components: Client >Reporter: Nick Dimiduk > Attachments: HBASE-8089-types.txt, HBASE-8089-types.txt, > HBASE-8089-types.txt > > > This proposal outlines an improvement to HBase that provides for a set of > types, above and beyond the existing "byte-bucket" strategy. This is intended > to reduce user-level duplication of effort, provide better support for > 3rd-party integration, and provide an overall improved experience for > developers using HBase. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8285) HBaseClient never recovers for single HTable.get() calls with no retries when regions move
[
https://issues.apache.org/jira/browse/HBASE-8285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628462#comment-13628462
]
Hadoop QA commented on HBASE-8285:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12578112/8285-0.94-v3.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/5253//console
This message is automatically generated.
> HBaseClient never recovers for single HTable.get() calls with no retries when
> regions move
> --
>
> Key: HBASE-8285
> URL: https://issues.apache.org/jira/browse/HBASE-8285
> Project: HBase
> Issue Type: Bug
> Components: Client
>Affects Versions: 0.94.6.1
>Reporter: Varun Sharma
>Assignee: Varun Sharma
>Priority: Critical
> Fix For: 0.98.0, 0.94.7, 0.95.1
>
> Attachments: 8285-0.94.txt, 8285-0.94-v2.txt, 8285-0.94-v3.txt,
> 8285-trunk.txt, 8285-trunk-v2.txt
>
>
> Steps to reproduce this bug:
> 1) Gracefull restart a region server causing regions to get redistributed.
> 2) Client call to this region keeps failing since Meta Cache is never purged
> on the client for the region that moved.
> Reason behind the bug:
> 1) Client continues to hit the old region server.
> 2) The old region server throws NotServingRegionException which is not
> handled correctly and the META cache entries are never purged for that server
> causing the client to keep hitting the old server.
> The reason lies in ServerCallable code since we only purge META cache entries
> when there is a RetriesExhaustedException, SocketTimeoutException or
> ConnectException. However, there is no case check for
> NotServingRegionException(s).
> Why is this not a problem for Scan(s) and Put(s) ?
> a) If a region server is not hosting a region/scanner, then an
> UnknownScannerException is thrown which causes a relocateRegion() call
> causing a refresh of the META cache for that particular region.
> b) For put(s), the processBatchCallback() interface in HConnectionManager is
> used which clears out META cache entries for all kinds of exceptions except
> DoNotRetryException.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8285) HBaseClient never recovers for single HTable.get() calls with no retries when regions move
[
https://issues.apache.org/jira/browse/HBASE-8285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628461#comment-13628461
]
Ted Yu commented on HBASE-8285:
---
{code}
+ // Reload this specific region into META cache since we don't call
+ // connect(true) and hence do not refresh the META cache.
+ getConnection().deleteCachedRegionLocation(tableName, location);
{code}
Looks like comment doesn't match updated code.
Thinking about backward compatibility, since ServerCallable.java and
HConnectionManager.java are packaged in the same jar, patch v3 should be Okay -
new client would manage its cache better.
Of course, binary compatibility is not maintained.
> HBaseClient never recovers for single HTable.get() calls with no retries when
> regions move
> --
>
> Key: HBASE-8285
> URL: https://issues.apache.org/jira/browse/HBASE-8285
> Project: HBase
> Issue Type: Bug
> Components: Client
>Affects Versions: 0.94.6.1
>Reporter: Varun Sharma
>Assignee: Varun Sharma
>Priority: Critical
> Fix For: 0.98.0, 0.94.7, 0.95.1
>
> Attachments: 8285-0.94.txt, 8285-0.94-v2.txt, 8285-0.94-v3.txt,
> 8285-trunk.txt, 8285-trunk-v2.txt
>
>
> Steps to reproduce this bug:
> 1) Gracefull restart a region server causing regions to get redistributed.
> 2) Client call to this region keeps failing since Meta Cache is never purged
> on the client for the region that moved.
> Reason behind the bug:
> 1) Client continues to hit the old region server.
> 2) The old region server throws NotServingRegionException which is not
> handled correctly and the META cache entries are never purged for that server
> causing the client to keep hitting the old server.
> The reason lies in ServerCallable code since we only purge META cache entries
> when there is a RetriesExhaustedException, SocketTimeoutException or
> ConnectException. However, there is no case check for
> NotServingRegionException(s).
> Why is this not a problem for Scan(s) and Put(s) ?
> a) If a region server is not hosting a region/scanner, then an
> UnknownScannerException is thrown which causes a relocateRegion() call
> causing a refresh of the META cache for that particular region.
> b) For put(s), the processBatchCallback() interface in HConnectionManager is
> used which clears out META cache entries for all kinds of exceptions except
> DoNotRetryException.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8285) HBaseClient never recovers for single HTable.get() calls with no retries when regions move
[ https://issues.apache.org/jira/browse/HBASE-8285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628450#comment-13628450 ] Varun Sharma commented on HBASE-8285: - Attached patched with new API for 0.94, if it looks good, will work on a similar patch for trunk > HBaseClient never recovers for single HTable.get() calls with no retries when > regions move > -- > > Key: HBASE-8285 > URL: https://issues.apache.org/jira/browse/HBASE-8285 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 0.94.6.1 >Reporter: Varun Sharma >Assignee: Varun Sharma >Priority: Critical > Fix For: 0.98.0, 0.94.7, 0.95.1 > > Attachments: 8285-0.94.txt, 8285-0.94-v2.txt, 8285-0.94-v3.txt, > 8285-trunk.txt, 8285-trunk-v2.txt > > > Steps to reproduce this bug: > 1) Gracefull restart a region server causing regions to get redistributed. > 2) Client call to this region keeps failing since Meta Cache is never purged > on the client for the region that moved. > Reason behind the bug: > 1) Client continues to hit the old region server. > 2) The old region server throws NotServingRegionException which is not > handled correctly and the META cache entries are never purged for that server > causing the client to keep hitting the old server. > The reason lies in ServerCallable code since we only purge META cache entries > when there is a RetriesExhaustedException, SocketTimeoutException or > ConnectException. However, there is no case check for > NotServingRegionException(s). > Why is this not a problem for Scan(s) and Put(s) ? > a) If a region server is not hosting a region/scanner, then an > UnknownScannerException is thrown which causes a relocateRegion() call > causing a refresh of the META cache for that particular region. > b) For put(s), the processBatchCallback() interface in HConnectionManager is > used which clears out META cache entries for all kinds of exceptions except > DoNotRetryException. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-8322) Re-enabled hbase checksums by default
Enis Soztutar created HBASE-8322: Summary: Re-enabled hbase checksums by default Key: HBASE-8322 URL: https://issues.apache.org/jira/browse/HBASE-8322 Project: HBase Issue Type: Improvement Components: Filesystem Integration Reporter: Enis Soztutar Assignee: Enis Soztutar Fix For: 0.98.0, 0.95.1 Double checksumming issues in HBase level checksums(HBASE-5074) has been fixed in HBASE-6868. However, that patch also disables hbase checksums by default. I think we should re-enable it by default, and document the interaction with shortcircuit reads. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8322) Re-enable hbase checksums by default
[ https://issues.apache.org/jira/browse/HBASE-8322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-8322: - Summary: Re-enable hbase checksums by default (was: Re-enabled hbase checksums by default) > Re-enable hbase checksums by default > > > Key: HBASE-8322 > URL: https://issues.apache.org/jira/browse/HBASE-8322 > Project: HBase > Issue Type: Improvement > Components: Filesystem Integration >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: 0.98.0, 0.95.1 > > > Double checksumming issues in HBase level checksums(HBASE-5074) has been > fixed in HBASE-6868. However, that patch also disables hbase checksums by > default. I think we should re-enable it by default, and document the > interaction with shortcircuit reads. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8320) test-patch.sh should post back compilation error against hadoop 2.0
[
https://issues.apache.org/jira/browse/HBASE-8320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-8320:
--
Resolution: Fixed
Status: Resolved (was: Patch Available)
> test-patch.sh should post back compilation error against hadoop 2.0
> ---
>
> Key: HBASE-8320
> URL: https://issues.apache.org/jira/browse/HBASE-8320
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 0.98.0
>
> Attachments: 8320.txt
>
>
> One example was:
> https://builds.apache.org/job/PreCommit-HBASE-Build/5242/console
> {code}
> if [[ $? != 0 ]] ; then
> JIRA_COMMENT="$JIRA_COMMENT
> {color:red}-1 hadoop2.0{color}. The patch failed to compile against the
> hadoop 2.0 profile."
> cleanupAndExit 1
> fi
> {code}
> cleanupAndExit doesn't post the comment.
> I think the correct call should be:
> {code}
> submitJiraComment 1
> cleanupAndExit 1
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8285) HBaseClient never recovers for single HTable.get() calls with no retries when regions move
[ https://issues.apache.org/jira/browse/HBASE-8285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Sharma updated HBASE-8285: Attachment: 8285-0.94-v3.txt > HBaseClient never recovers for single HTable.get() calls with no retries when > regions move > -- > > Key: HBASE-8285 > URL: https://issues.apache.org/jira/browse/HBASE-8285 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 0.94.6.1 >Reporter: Varun Sharma >Assignee: Varun Sharma >Priority: Critical > Fix For: 0.98.0, 0.94.7, 0.95.1 > > Attachments: 8285-0.94.txt, 8285-0.94-v2.txt, 8285-0.94-v3.txt, > 8285-trunk.txt, 8285-trunk-v2.txt > > > Steps to reproduce this bug: > 1) Gracefull restart a region server causing regions to get redistributed. > 2) Client call to this region keeps failing since Meta Cache is never purged > on the client for the region that moved. > Reason behind the bug: > 1) Client continues to hit the old region server. > 2) The old region server throws NotServingRegionException which is not > handled correctly and the META cache entries are never purged for that server > causing the client to keep hitting the old server. > The reason lies in ServerCallable code since we only purge META cache entries > when there is a RetriesExhaustedException, SocketTimeoutException or > ConnectException. However, there is no case check for > NotServingRegionException(s). > Why is this not a problem for Scan(s) and Put(s) ? > a) If a region server is not hosting a region/scanner, then an > UnknownScannerException is thrown which causes a relocateRegion() call > causing a refresh of the META cache for that particular region. > b) For put(s), the processBatchCallback() interface in HConnectionManager is > used which clears out META cache entries for all kinds of exceptions except > DoNotRetryException. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6782) HBase shell's 'status 'detailed'' should escape the printed keys
[ https://issues.apache.org/jira/browse/HBASE-6782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628443#comment-13628443 ] Lars Hofhansl commented on HBASE-6782: -- That is a good point. What do other folks think? Would it be confusing to change that now? If so, then let's not do that. > HBase shell's 'status 'detailed'' should escape the printed keys > > > Key: HBASE-6782 > URL: https://issues.apache.org/jira/browse/HBASE-6782 > Project: HBase > Issue Type: Bug > Components: shell >Affects Versions: 0.90.1 >Reporter: Viji >Assignee: Viji >Priority: Minor > Fix For: 0.95.0 > > Attachments: HBASE-6782.patch > > > Currently the HBase shell's status command prints unescaped keys on the > terminal causing the terminal to print garbage characters. We should escape > the printed keys. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7704) migration tool that checks presence of HFile V1 files
[ https://issues.apache.org/jira/browse/HBASE-7704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628442#comment-13628442 ] Ted Yu commented on HBASE-7704: --- Compaction would be initiated per region. Discard my first comment above. > migration tool that checks presence of HFile V1 files > - > > Key: HBASE-7704 > URL: https://issues.apache.org/jira/browse/HBASE-7704 > Project: HBase > Issue Type: Task >Reporter: Ted Yu >Assignee: Himanshu Vashishtha >Priority: Blocker > Fix For: 0.95.1 > > Attachments: HBase-7704-v1.patch, HBase-7704-v2.patch, > HBase-7704-v3.patch > > > Below was Stack's comment from HBASE-7660: > Regards the migration 'tool', or 'tool' to check for presence of v1 files, I > imagine it as an addition to the hfile tool > http://hbase.apache.org/book.html#hfile_tool2 The hfile tool already takes a > bunch of args including printing out meta. We could add an option to print > out version only – or return 1 if version 1 or some such – and then do a bit > of code to just list all hfiles and run this script against each. Could MR it > if too many files. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8316) JoinedHeap for non essential column families should reseek instead of seek
[ https://issues.apache.org/jira/browse/HBASE-8316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628441#comment-13628441 ] Lars Hofhansl commented on HBASE-8316: -- In TestJoinedScanner I tried with 1k and 128k large CFs. For the Phoenix test: * The small CF has one column of 8 bytes * the large CF has three columns (200 bytes, 200 bytes, 8 bytes - 408 bytes in total) > JoinedHeap for non essential column families should reseek instead of seek > -- > > Key: HBASE-8316 > URL: https://issues.apache.org/jira/browse/HBASE-8316 > Project: HBase > Issue Type: Sub-task > Components: Filters, Performance, regionserver >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl > Fix For: 0.98.0, 0.94.7, 0.95.1 > > Attachments: 8316-0.94.txt, 8316-trunk.txt, 8316-trunk.txt, > FDencode.png, noencode.png > > > This was raised by the Phoenix team. During a profiling session we noticed > that catching the joinedHeap up to the current rows via seek causes a > performance regression, which makes the joinedHeap only efficient when either > a high or low percentage is matched by the filter. > (High is fine, because the joinedHeap will not get behind as often and does > not need to be caught up, low is fine, because the seek isn't happening > frequently). > In our tests we found that the solution is quite simple: Replace seek with > reseek. Patch coming soon. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8310) HBase snapshot timeout default values and TableLockManger timeout
[ https://issues.apache.org/jira/browse/HBASE-8310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628436#comment-13628436 ] Jerry He commented on HBASE-8310: - Any objection to drop the snapshot wait time for table lock to 1 min? It makes sense since snapshot operation is more like 'I try' instead of 'I have to'. > HBase snapshot timeout default values and TableLockManger timeout > - > > Key: HBASE-8310 > URL: https://issues.apache.org/jira/browse/HBASE-8310 > Project: HBase > Issue Type: Bug > Components: snapshots >Affects Versions: 0.95.0 >Reporter: Jerry He >Assignee: Jerry He >Priority: Minor > Fix For: 0.98.0, 0.94.8, 0.95.2 > > > There are a few timeout values and defaults being used by HBase snapshot. > DEFAULT_MAX_WAIT_TIME (6 milli sec, 1 min) for client response > TIMEOUT_MILLIS_DEFAULT (6 milli sec, 1 min) for Procedure timeout > SNAPSHOT_TIMEOUT_MILLIS_DEFAULT (6 milli sec, 1 min) for region server > subprocedure > There is also other timeout involved, for example, > DEFAULT_TABLE_WRITE_LOCK_TIMEOUT_MS (10 mins) for > TakeSnapshotHandler#prepare() > We could have this case: > The user issues a sync snapshot request, waits for 1 min, and gets an > exception. > In the meantime the snapshot handler is blocked on the table lock, and the > snapshot may continue to finish after 10 mins. > But the user will probably re-issue the snapshot request during the 10 mins. > This is a little confusing and messy when this happens. > To be more reasonable, we should either increase the DEFAULT_MAX_WAIT_TIME or > decrease the table lock waiting time. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7704) migration tool that checks presence of HFile V1 files
[
https://issues.apache.org/jira/browse/HBASE-7704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628437#comment-13628437
]
Ted Yu commented on HBASE-7704:
---
In processRegion():
{code}
+hFileV1Set.add(storeFilePath);
+// return this region path, as it needs to be compacted.
+return path;
{code}
There may be more than one V1 HFile in the region, right ? Why do we return
early ?
{code}
+ * Tool to detect presence of any HFileV1 in the given directory. It prints
all such regions which
+ * has such files.
{code}
'has such files' -> 'have such files'
{code}
+ * It also support -h, --help, -help options.
{code}
'support' -> 'supports'
> migration tool that checks presence of HFile V1 files
> -
>
> Key: HBASE-7704
> URL: https://issues.apache.org/jira/browse/HBASE-7704
> Project: HBase
> Issue Type: Task
>Reporter: Ted Yu
>Assignee: Himanshu Vashishtha
>Priority: Blocker
> Fix For: 0.95.1
>
> Attachments: HBase-7704-v1.patch, HBase-7704-v2.patch,
> HBase-7704-v3.patch
>
>
> Below was Stack's comment from HBASE-7660:
> Regards the migration 'tool', or 'tool' to check for presence of v1 files, I
> imagine it as an addition to the hfile tool
> http://hbase.apache.org/book.html#hfile_tool2 The hfile tool already takes a
> bunch of args including printing out meta. We could add an option to print
> out version only – or return 1 if version 1 or some such – and then do a bit
> of code to just list all hfiles and run this script against each. Could MR it
> if too many files.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6782) HBase shell's 'status 'detailed'' should escape the printed keys
[ https://issues.apache.org/jira/browse/HBASE-6782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628435#comment-13628435 ] Jimmy Xiang commented on HBASE-6782: I was wondering if users would be confused to see different results. We can get it in 0.94 if it is not a concern at all. > HBase shell's 'status 'detailed'' should escape the printed keys > > > Key: HBASE-6782 > URL: https://issues.apache.org/jira/browse/HBASE-6782 > Project: HBase > Issue Type: Bug > Components: shell >Affects Versions: 0.90.1 >Reporter: Viji >Assignee: Viji >Priority: Minor > Fix For: 0.95.0 > > Attachments: HBASE-6782.patch > > > Currently the HBase shell's status command prints unescaped keys on the > terminal causing the terminal to print garbage characters. We should escape > the printed keys. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8316) JoinedHeap for non essential column families should reseek instead of seek
[ https://issues.apache.org/jira/browse/HBASE-8316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628429#comment-13628429 ] Hudson commented on HBASE-8316: --- Integrated in HBase-0.94 #954 (See [https://builds.apache.org/job/HBase-0.94/954/]) HBASE-8316 JoinedHeap for non essential column families should reseek instead of seek (Revision 1466708) Result = SUCCESS larsh : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java > JoinedHeap for non essential column families should reseek instead of seek > -- > > Key: HBASE-8316 > URL: https://issues.apache.org/jira/browse/HBASE-8316 > Project: HBase > Issue Type: Sub-task > Components: Filters, Performance, regionserver >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl > Fix For: 0.98.0, 0.94.7, 0.95.1 > > Attachments: 8316-0.94.txt, 8316-trunk.txt, 8316-trunk.txt, > FDencode.png, noencode.png > > > This was raised by the Phoenix team. During a profiling session we noticed > that catching the joinedHeap up to the current rows via seek causes a > performance regression, which makes the joinedHeap only efficient when either > a high or low percentage is matched by the filter. > (High is fine, because the joinedHeap will not get behind as often and does > not need to be caught up, low is fine, because the seek isn't happening > frequently). > In our tests we found that the solution is quite simple: Replace seek with > reseek. Patch coming soon. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8266) Master cannot start if TableNotFoundException is thrown while partial table recovery
[
https://issues.apache.org/jira/browse/HBASE-8266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628428#comment-13628428
]
Hudson commented on HBASE-8266:
---
Integrated in HBase-0.94 #954 (See
[https://builds.apache.org/job/HBase-0.94/954/])
HBASE-8266-Master cannot start if TableNotFoundException is thrown while
partial table recovery (Ram) (Revision 1466567)
Result = SUCCESS
ramkrishna :
Files :
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/handler/CreateTableHandler.java
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/handler/EnableTableHandler.java
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKTable.java
*
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/master/handler/TestCreateTableHandler.java
> Master cannot start if TableNotFoundException is thrown while partial table
> recovery
>
>
> Key: HBASE-8266
> URL: https://issues.apache.org/jira/browse/HBASE-8266
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.94.6, 0.95.0
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Critical
> Fix For: 0.98.0, 0.94.7, 0.95.1
>
> Attachments: HBASE-8266_0.94.patch, HBASE-8266_1.patch,
> HBASE-8266.patch
>
>
> I was trying to create a table. The table creation failed
> {code}
> java.io.IOException: java.util.concurrent.ExecutionException:
> java.lang.IllegalStateException: Could not instantiate a region instance.
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegions(ModifyRegionUtils.java:133)
> at
> org.apache.hadoop.hbase.master.handler.CreateTableHandler.handleCreateHdfsRegions(CreateTableHandler.java:256)
> at
> org.apache.hadoop.hbase.master.handler.CreateTableHandler.handleCreateTable(CreateTableHandler.java:204)
> at
> org.apache.hadoop.hbase.master.handler.CreateTableHandler.process(CreateTableHandler.java:153)
> at
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:130)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: java.util.concurrent.ExecutionException:
> java.lang.IllegalStateException: Could not instantiate a region instance.
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
> at java.util.concurrent.FutureTask.get(FutureTask.java:83)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegions(ModifyRegionUtils.java:126)
> ... 7 more
> Caused by: java.lang.IllegalStateException: Could not instantiate a region
> instance.
> at
> org.apache.hadoop.hbase.regionserver.HRegion.newHRegion(HRegion.java:3765)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.createHRegion(HRegion.java:3870)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils$1.call(ModifyRegionUtils.java:106)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils$1.call(ModifyRegionUtils.java:103)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> ... 3 more
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.newHRegion(HRegion.java:3762)
> ... 11 more
> Caused by: java.lang.NoClassDefFoundError:
> org/apache/hadoop/hbase/CompoundConfiguration$1
> at
> org.apache.hadoop.hbase.CompoundConfiguration.add(CompoundConfiguration.java:82)
> at org.apache.hadoop.hbase.regionserver.HRegion.(HRegion.java:438)
> at org.apache.hadoop.hbase.regionserver.HRegion.(HRegion.java:401)
> ... 16 more
> {code}
> Am not sure of the above failure. The same setup is able to create new
> tables.
> Now the table is already in ENABLING state. The master was restarted.
> Now as the table was found in ENABLING state but not added to META the
> EnableTableHandler
> {code}
> 2013-04-03 18:33:03,
[jira] [Updated] (HBASE-7704) migration tool that checks presence of HFile V1 files
[ https://issues.apache.org/jira/browse/HBASE-7704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-7704: -- Status: Patch Available (was: Open) > migration tool that checks presence of HFile V1 files > - > > Key: HBASE-7704 > URL: https://issues.apache.org/jira/browse/HBASE-7704 > Project: HBase > Issue Type: Task >Reporter: Ted Yu >Assignee: Himanshu Vashishtha >Priority: Blocker > Fix For: 0.95.1 > > Attachments: HBase-7704-v1.patch, HBase-7704-v2.patch, > HBase-7704-v3.patch > > > Below was Stack's comment from HBASE-7660: > Regards the migration 'tool', or 'tool' to check for presence of v1 files, I > imagine it as an addition to the hfile tool > http://hbase.apache.org/book.html#hfile_tool2 The hfile tool already takes a > bunch of args including printing out meta. We could add an option to print > out version only – or return 1 if version 1 or some such – and then do a bit > of code to just list all hfiles and run this script against each. Could MR it > if too many files. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8316) JoinedHeap for non essential column families should reseek instead of seek
[ https://issues.apache.org/jira/browse/HBASE-8316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628414#comment-13628414 ] Ted Yu commented on HBASE-8316: --- @Lars: Can you illustrate the ratio between the sizes of the narrow and wide column families ? Thanks > JoinedHeap for non essential column families should reseek instead of seek > -- > > Key: HBASE-8316 > URL: https://issues.apache.org/jira/browse/HBASE-8316 > Project: HBase > Issue Type: Sub-task > Components: Filters, Performance, regionserver >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl > Fix For: 0.98.0, 0.94.7, 0.95.1 > > Attachments: 8316-0.94.txt, 8316-trunk.txt, 8316-trunk.txt, > FDencode.png, noencode.png > > > This was raised by the Phoenix team. During a profiling session we noticed > that catching the joinedHeap up to the current rows via seek causes a > performance regression, which makes the joinedHeap only efficient when either > a high or low percentage is matched by the filter. > (High is fine, because the joinedHeap will not get behind as often and does > not need to be caught up, low is fine, because the seek isn't happening > frequently). > In our tests we found that the solution is quite simple: Replace seek with > reseek. Patch coming soon. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-7824) Improve master start up time when there is log splitting work
[ https://issues.apache.org/jira/browse/HBASE-7824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl resolved HBASE-7824. -- Resolution: Fixed > Improve master start up time when there is log splitting work > - > > Key: HBASE-7824 > URL: https://issues.apache.org/jira/browse/HBASE-7824 > Project: HBase > Issue Type: Bug > Components: master >Reporter: Jeffrey Zhong >Assignee: Jeffrey Zhong > Fix For: 0.94.7 > > Attachments: hbase-7824.patch, hbase-7824-v10.patch, > hbase-7824_v2.patch, hbase-7824_v3.patch, hbase-7824-v7.patch, > hbase-7824-v8.patch, hbase-7824-v9.patch > > > When there is log split work going on, master start up waits till all log > split work completes even though the log split has nothing to do with meta > region servers. > It's a bad behavior considering a master node can run when log split is > happening while its start up is blocking by log split work. > Since master is kind of single point of failure, we should start it ASAP. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8303) Increse the test timeout to 60s when they are less than 20s
[ https://issues.apache.org/jira/browse/HBASE-8303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-8303: -- Attachment: 8303-0.94.patch Testing 0.94 patch locally now. > Increse the test timeout to 60s when they are less than 20s > --- > > Key: HBASE-8303 > URL: https://issues.apache.org/jira/browse/HBASE-8303 > Project: HBase > Issue Type: Bug >Affects Versions: 0.94.7, 0.95.0 >Reporter: Nicolas Liochon >Assignee: Nicolas Liochon > Fix For: 0.98.0, 0.95.1 > > Attachments: 8303-0.94.patch, 8303.v1.patch, 8303.v1.patch > > > Short test timeouts are dangerous because: > - if the test is executed in the same jvm as another, GC, thread priority > can play a role > - we don't know the machine used to execute the tests, nor what's running on > it;. > For this reason, a test timeout of 60s allows us to be on the safe side. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6782) HBase shell's 'status 'detailed'' should escape the printed keys
[ https://issues.apache.org/jira/browse/HBASE-6782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628405#comment-13628405 ] Lars Hofhansl commented on HBASE-6782: -- Seems like we should have this 0.94. > HBase shell's 'status 'detailed'' should escape the printed keys > > > Key: HBASE-6782 > URL: https://issues.apache.org/jira/browse/HBASE-6782 > Project: HBase > Issue Type: Bug > Components: shell >Affects Versions: 0.90.1 >Reporter: Viji >Assignee: Viji >Priority: Minor > Fix For: 0.95.0 > > Attachments: HBASE-6782.patch > > > Currently the HBase shell's status command prints unescaped keys on the > terminal causing the terminal to print garbage characters. We should escape > the printed keys. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7824) Improve master start up time when there is log splitting work
[ https://issues.apache.org/jira/browse/HBASE-7824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628401#comment-13628401 ] Ted Yu commented on HBASE-7824: --- Integrated to 0.94 Thanks for the continued effort, Jeff. Thanks for the reviews, Ram, Chunhui and Rajesh. > Improve master start up time when there is log splitting work > - > > Key: HBASE-7824 > URL: https://issues.apache.org/jira/browse/HBASE-7824 > Project: HBase > Issue Type: Bug > Components: master >Reporter: Jeffrey Zhong >Assignee: Jeffrey Zhong > Fix For: 0.94.7 > > Attachments: hbase-7824.patch, hbase-7824-v10.patch, > hbase-7824_v2.patch, hbase-7824_v3.patch, hbase-7824-v7.patch, > hbase-7824-v8.patch, hbase-7824-v9.patch > > > When there is log split work going on, master start up waits till all log > split work completes even though the log split has nothing to do with meta > region servers. > It's a bad behavior considering a master node can run when log split is > happening while its start up is blocking by log split work. > Since master is kind of single point of failure, we should start it ASAP. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8096) [replication] NPE while replicating a log that is acquiring a new block from HDFS
[
https://issues.apache.org/jira/browse/HBASE-8096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628398#comment-13628398
]
Chris Trezzo commented on HBASE-8096:
-
Looks good to me. ReplicationHLogReaderManager.openReader is the only place
where replication tries to reset the HLogReader. I think it makes sense to
sleep/retry when the reset is not successful, and I like relying on
ReplicationSource.run for the retry.
+1 on the patch for 0.94 and trunk/0.95.
> [replication] NPE while replicating a log that is acquiring a new block from
> HDFS
> --
>
> Key: HBASE-8096
> URL: https://issues.apache.org/jira/browse/HBASE-8096
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.94.5
>Reporter: Ian Friedman
>Assignee: Dave Latham
> Attachments: HBASE-8096.0.94.patch, HBASE-8096.patch
>
>
> We're getting an NPE during replication, which causes replication for that
> RegionServer to stop until we restart it.
> {noformat}
> 2013-03-10 12:49:12,679 ERROR
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
> Unexpected exception in ReplicationSource,
> currentPath=hdfs://hmaster1:9000/hbase/.logs/hslave1177,60020,1362549511446/hslave1177%2C60020%2C1362549511446.1362944946489
> java.lang.NullPointerException
> at
> org.apache.hadoop.hdfs.DFSClient$DFSInputStream.updateBlockInfo(DFSClient.java:1882)
> at
> org.apache.hadoop.hdfs.DFSClient$DFSInputStream.openInfo(DFSClient.java:1855)
> at
> org.apache.hadoop.hdfs.DFSClient$DFSInputStream.(DFSClient.java:1831)
> at org.apache.hadoop.hdfs.DFSClient.open(DFSClient.java:578)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.open(DistributedFileSystem.java:154)
> at
> org.apache.hadoop.fs.FilterFileSystem.open(FilterFileSystem.java:108)
> at
> org.apache.hadoop.io.SequenceFile$Reader.openFile(SequenceFile.java:1495)
> at
> org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader$WALReader.openFile(SequenceFileLogReader.java:62)
> at
> org.apache.hadoop.io.SequenceFile$Reader.(SequenceFile.java:1482)
> at
> org.apache.hadoop.io.SequenceFile$Reader.(SequenceFile.java:1475)
> at
> org.apache.hadoop.io.SequenceFile$Reader.(SequenceFile.java:1470)
> at
> org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader$WALReader.(SequenceFileLogReader.java:55)
> at
> org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader.reset(SequenceFileLogReader.java:308)
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationHLogReaderManager.openReader(ReplicationHLogReaderManager.java:69)
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.openReader(ReplicationSource.java:505)
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.run(ReplicationSource.java:313)
> {noformat}
> Some extra digging into the DataNode and NameNode logs makes this seem
> related to HBASE-7530 and HDFS-4380
> Here's the relevant snipped portions of the RS, DN, and NN logs:
> {noformat}
> RS 2013-03-10 12:49:12,618 INFO
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager:
> Going to report log #hslave1177%2C60020%2C1362549511446.1362944946489 for
> position 59670826 in
> hdfs://hmaster1:9000/hbase/.logs/hslave1177,60020,1362549511446/hslave1177%2C60020%2C1362549511446.1362944946489
> RS 2013-03-10 12:49:12,621 DEBUG
> org.apache.hadoop.hbase.regionserver.LogRoller: HLog roll requested
> RS 2013-03-10 12:49:12,623 DEBUG
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
> Replicated in total: 31500300
> RS 2013-03-10 12:49:12,623 DEBUG
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Opening
> log for replication hslave1177%2C60020%2C1362549511446.1362944946489 at
> 59670826
> NN 2013-03-10 12:49:12,627 INFO org.apache.hadoop.hdfs.StateChange: BLOCK*
> NameSystem.allocateBlock:
> /hbase/.logs/hslave1177,60020,1362549511446/hslave1177%2C60020%2C1362549511446.1362944946489.
> blk_6905758215335505153_656717631
> RS 2013-03-10 12:49:12,679 ERROR
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
> Unexpected exception in ReplicationSource,
> currentPath=hdfs://hmaster1:9000/hbase/.logs/hslave1177,60020,1362549511446/hslave1177%2C60020%2C1362549511446.1362944946489
> DN 2013-03-10 12:49:12,680 INFO
> org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving block
> blk_6905758215335505153_656717631 src: /192.168.44.1:43503 dest:
> /192.168.44.1:50010
> NN 2013-03-10 12:49:12,804 INFO org.apache.hadoop.hdfs.StateChange: BLOCK*
> NameSystem.fsync: file
> /hbase/.logs/hslave1177,60020,1362549511446/hslave1177%2C6002
