[jira] [Updated] (HBASE-19769) IllegalAccessError on package-private Hadoop metrics2 classes in MapReduce jobs
[
https://issues.apache.org/jira/browse/HBASE-19769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Elser updated HBASE-19769:
---
Status: Patch Available (was: Open)
> IllegalAccessError on package-private Hadoop metrics2 classes in MapReduce
> jobs
> ---
>
> Key: HBASE-19769
> URL: https://issues.apache.org/jira/browse/HBASE-19769
> Project: HBase
> Issue Type: Bug
> Components: mapreduce, metrics
>Affects Versions: 2.0.0-beta-1
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Critical
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19769.001.branch-2.patch
>
>
> issues for context: HBASE-17170, HBASE-17448, TEZ-3299, HADOOP-10893
> Since Hadoop 2.6.0, the {{yarn jar}} entry point to submit a YARN job has
> been using a custom classloader to separate Hadoop dependencies from the
> user's JAR being run. A separate classloader is created for the user-provided
> jar, and then this classloader is set as the contextClassLoader before the
> Tool is executed by Hadoop's RunJar class. This has been (mostly?) fine for
> us to date because we don't try to access any Hadoop internal classes
> client-side.
> However, with the ZK metrics, clients are pushing ZK metrics to metrics2. The
> problem is that Hadoop metrics2 implementations which we reference from the
> same package are loaded by a different classloader than our HBase code is
> loaded from. This makes the expected package-private access of these Metrics2
> classes (e.g. MetricsInfoImpl) fail with an IllegalAccessError.
> {noformat}
> java.lang.RuntimeException: Could not create interface
> org.apache.hadoop.hbase.zookeeper.MetricsZooKeeperSource Is the hadoop
> compatibility jar on the classpath?
> at
> org.apache.hadoop.hbase.CompatibilitySingletonFactory.getInstance(CompatibilitySingletonFactory.java:75)
> at
> org.apache.hadoop.hbase.zookeeper.ZKMetrics.(ZKMetrics.java:36)
> at
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.(RecoverableZooKeeper.java:115)
> at org.apache.hadoop.hbase.zookeeper.ZKUtil.connect(ZKUtil.java:139)
> at
> org.apache.hadoop.hbase.zookeeper.ZKWatcher.(ZKWatcher.java:128)
> at
> org.apache.hadoop.hbase.zookeeper.ZKWatcher.(ZKWatcher.java:102)
> at
> org.apache.hadoop.hbase.security.token.TokenUtil.getAuthToken(TokenUtil.java:293)
> at
> org.apache.hadoop.hbase.security.token.TokenUtil.addTokenForJob(TokenUtil.java:259)
> at
> org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.initCredentials(TableMapReduceUtil.java:535)
> at
> org.apache.phoenix.mapreduce.MultiHfileOutputFormat.configureIncrementalLoad(MultiHfileOutputFormat.java:712)
> at
> org.apache.phoenix.mapreduce.AbstractBulkLoadTool.submitJob(AbstractBulkLoadTool.java:300)
> at
> org.apache.phoenix.mapreduce.AbstractBulkLoadTool.loadData(AbstractBulkLoadTool.java:267)
> at
> org.apache.phoenix.mapreduce.AbstractBulkLoadTool.run(AbstractBulkLoadTool.java:180)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
> at
> org.apache.phoenix.mapreduce.CsvBulkLoadTool.main(CsvBulkLoadTool.java:109)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:239)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:153)
> Caused by: java.util.ServiceConfigurationError:
> org.apache.hadoop.hbase.zookeeper.MetricsZooKeeperSource: Provider
> org.apache.hadoop.hbase.zookeeper.MetricsZooKeeperSourceImpl could not be
> instantiated
> at java.util.ServiceLoader.fail(ServiceLoader.java:232)
> at java.util.ServiceLoader.access$100(ServiceLoader.java:185)
> at
> java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:384)
> at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
> at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
> at
> org.apache.hadoop.hbase.CompatibilitySingletonFactory.getInstance(CompatibilitySingletonFactory.java:59)
> ... 21 more
> Caused by: java.lang.IllegalAccessError: tried to access class
> org.apache.hadoop.metrics2.lib.MetricsInfoImpl from class
> org.apache.hadoop.metrics2.lib.DynamicMetricsRegistry
> at
> org.apache.hadoop.metrics2.lib.DynamicMetricsRegistry.newGauge(DynamicMetricsRegistry.java:139)
> at
> org.apa
[jira] [Created] (HBASE-19777) RemoteProcedureException#unwrapRemoteIOException doesn't handle the case where cause is null
Ted Yu created HBASE-19777:
--
Summary: RemoteProcedureException#unwrapRemoteIOException doesn't
handle the case where cause is null
Key: HBASE-19777
URL: https://issues.apache.org/jira/browse/HBASE-19777
Project: HBase
Issue Type: Bug
Reporter: Romil Choksi
Assignee: Ted Yu
Romil noted the following when running IntegrationTestDDLMasterFailover:
{code}
2018-01-11 20:51:50,348|INFO|MainThread|machine.py:164 -
run()||GUID=48bb0b38-83c9-4b57-be4d-2c0ce4d765f1|2018-01-11 20:51:50,347 ERROR
[main] hbase.IntegrationTestDDLMasterFailover: Found exception in thread:
Thread-9
2018-01-11 20:51:50,348|INFO|MainThread|machine.py:164 -
run()||GUID=48bb0b38-83c9-4b57-be4d-2c0ce4d765f1|java.io.IOException:
java.lang.NullPointerException
2018-01-11 20:51:50,348|INFO|MainThread|machine.py:164 -
run()||GUID=48bb0b38-83c9-4b57-be4d-2c0ce4d765f1|at
org.apache.hadoop.hbase.procedure2.RemoteProcedureException.unwrapRemoteIOException(RemoteProcedureException.java:82)
2018-01-11 20:51:50,348|INFO|MainThread|machine.py:164 -
run()||GUID=48bb0b38-83c9-4b57-be4d-2c0ce4d765f1|at
org.apache.hadoop.hbase.master.MasterRpcServices.getProcedureResult(MasterRpcServices.java:1165)
2018-01-11 20:51:50,348|INFO|MainThread|machine.py:164 -
run()||GUID=48bb0b38-83c9-4b57-be4d-2c0ce4d765f1|at
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
2018-01-11 20:51:50,349|INFO|MainThread|machine.py:164 -
run()||GUID=48bb0b38-83c9-4b57-be4d-2c0ce4d765f1|at
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:404)
2018-01-11 20:51:50,349|INFO|MainThread|machine.py:164 -
run()||GUID=48bb0b38-83c9-4b57-be4d-2c0ce4d765f1|at
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
2018-01-11 20:51:50,349|INFO|MainThread|machine.py:164 -
run()||GUID=48bb0b38-83c9-4b57-be4d-2c0ce4d765f1|at
org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
2018-01-11 20:51:50,349|INFO|MainThread|machine.py:164 -
run()||GUID=48bb0b38-83c9-4b57-be4d-2c0ce4d765f1|at
org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
{code}
During this time frame, master was disabling ittable-1095976240 .
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-19746) Add default impl to Cell#getType
[
https://issues.apache.org/jira/browse/HBASE-19746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-19746:
--
Resolution: Fixed
Status: Resolved (was: Patch Available)
+1 Pushed to get it into beta-1. Thanks [~chia7712]
> Add default impl to Cell#getType
>
>
> Key: HBASE-19746
> URL: https://issues.apache.org/jira/browse/HBASE-19746
> Project: HBase
> Issue Type: Sub-task
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-19746.v0.patch, HBASE-19746.v1.patch,
> HBASE-19746.v1.qa.patch
>
>
> Noticed this issue when migrating the app to branch-2.
> {{Cell}} is IA.Public so it should obey our compatibility rules. Not sure
> whether any related discussion had be in HBASE-19112. It worthwhile, however,
> to raise this issue again.
> FYI [~anoopsamjohn] [~ram_krish] [~stack]
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19776) shell gives unfriendly errors for nonexistant table
[
https://issues.apache.org/jira/browse/HBASE-19776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16323167#comment-16323167
]
Sergey Soldatov commented on HBASE-19776:
-
Just in case, actually that scary error more related to HBASE-19774, but we see
it because HBASE-19775.
> shell gives unfriendly errors for nonexistant table
> ---
>
> Key: HBASE-19776
> URL: https://issues.apache.org/jira/browse/HBASE-19776
> Project: HBase
> Issue Type: Bug
>Reporter: Mike Drob
>
> When attempting operations against a non-existent table, some of the errors
> that we get are very unintuitive.
> {noformat}
> $ bin/hbase shell -n <<< "count 'no_such_table'"
> 2018-01-11 16:22:54,499 WARN [main] util.NativeCodeLoader: Unable to load
> native-hadoop library for your platform... using builtin-java classes where
> applicable
> Took 0.4709 seconds
>
>
> java exception
> ERROR Java::OrgApacheHadoopHbase::TableNotFoundException: Region of
> 'hbase:namespace,,1515709347596.af9aaa132e8baa3dcf2874ce6d93.' is
> expected in the table of 'no_such_table', but hbase:meta says it is in the
> table of 'hbase:namespace'. hbase:meta might be damaged.
> {noformat}
> Similar errors from scan, and from interactive shell instead on
> noninteractive mode.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-19752) RSGroupBasedLoadBalancer#getMisplacedRegions() should handle the case where rs group cannot be determined
[
https://issues.apache.org/jira/browse/HBASE-19752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-19752:
---
Attachment: 19752.v6.txt
> RSGroupBasedLoadBalancer#getMisplacedRegions() should handle the case where
> rs group cannot be determined
> -
>
> Key: HBASE-19752
> URL: https://issues.apache.org/jira/browse/HBASE-19752
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 1.4.1, 1.5.0
>
> Attachments: 19752.v1.txt, 19752.v2.txt, 19752.v3.txt, 19752.v4.txt,
> 19752.v5.txt, 19752.v6.txt
>
>
> Observed the following in rs group test output:
> {code}
> 2018-01-10 14:17:23,006 DEBUG [AssignmentThread]
> rsgroup.RSGroupBasedLoadBalancer(316): Found misplaced region:
> hbase:acl,,1515593841277.ecf47ecb7522d7fab40db0a237f973fd. on server:
> localhost,1,1 found in group: null outside of group: UNKNOWN
> {code}
> Here is corresponding code:
> {code}
> if (assignedServer != null &&
> (info == null ||
> !info.containsServer(assignedServer.getAddress( {
> RSGroupInfo otherInfo = null;
> otherInfo =
> rsGroupInfoManager.getRSGroupOfServer(assignedServer.getAddress());
> LOG.debug("Found misplaced region: " +
> regionInfo.getRegionNameAsString() +
> {code}
> As you can see, both info and otherInfo were null.
> In this case, the region should not be placed in misplacedRegions.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-19767) Master web UI shows negative values for Remaining KVs
[ https://issues.apache.org/jira/browse/HBASE-19767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-19767: -- Fix Version/s: 2.0.0-beta-2 > Master web UI shows negative values for Remaining KVs > - > > Key: HBASE-19767 > URL: https://issues.apache.org/jira/browse/HBASE-19767 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0-alpha-4 >Reporter: Jean-Marc Spaggiari > Fix For: 2.0.0-beta-2 > > > In the Master Web UI, under the compaction tab, the Remaining KVs sometimes > shows negative values. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19768) RegionServer startup failing when DN is dead
[
https://issues.apache.org/jira/browse/HBASE-19768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-19768:
--
Fix Version/s: 2.0.0-beta-2
> RegionServer startup failing when DN is dead
>
>
> Key: HBASE-19768
> URL: https://issues.apache.org/jira/browse/HBASE-19768
> Project: HBase
> Issue Type: Bug
>Reporter: Jean-Marc Spaggiari
>Priority: Critical
> Fix For: 2.0.0-beta-2
>
>
> When starting HBase, if the datanode hosted on the same host is dead but not
> yet detected by the namenode, HBase will fail to start
> {code}
> 515691223393/node8.distparser.com%2C16020%2C1515691223393.1515691238778
> failed, retry = 7
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException:
> syscall:getsockopt(..) failed: Connexion refusée: /192.168.23.2:50010
> at
> org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(..)(Unknown
> Source)
> Caused by:
> org.apache.hbase.thirdparty.io.netty.channel.unix.Errors$NativeConnectException:
> syscall:getsockopt(..) failed: Connexion refusée
> ... 1 more
> {code}
> and will also get stuck to stop:
> {code}
> hbase@node2:~/hbase-2.0.0-beta-1$ bin/stop-hbase.sh
> stopping
> hbase^C
> hbase@node2:~/hbase-2.0.0-beta-1$ bin/stop-hbase.sh
> stopping
> hbase..
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in
> [jar:file:/home/hbase/hbase-2.0.0-beta-1/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/home/hbase/hbase-2.0.0-beta-1/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
> {code}
> The most interesting is that it seems to fail the same way even if the DN is
> declared dead on HDFS side:
> {code}
> 515692041367/node8.distparser.com%2C16020%2C1515692041367.1515692057716
> failed, retry = 4
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException:
> syscall:getsockopt(..) failed: Connexion refusée: /192.168.23.2:50010
> at
> org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(..)(Unknown
> Source)
> Caused by:
> org.apache.hbase.thirdparty.io.netty.channel.unix.Errors$NativeConnectException:
> syscall:getsockopt(..) failed: Connexion refusée
> ... 1 more
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19163) "Maximum lock count exceeded" from region server's batch processing
[
https://issues.apache.org/jira/browse/HBASE-19163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16323157#comment-16323157
]
stack commented on HBASE-19163:
---
Is it in branch-2 [~huaxiang] +1 for branch-2 sir.
> "Maximum lock count exceeded" from region server's batch processing
> ---
>
> Key: HBASE-19163
> URL: https://issues.apache.org/jira/browse/HBASE-19163
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 3.0.0, 1.2.7, 2.0.0-alpha-3
>Reporter: huaxiang sun
>Assignee: huaxiang sun
> Attachments: HBASE-19163-branch-1-v001.patch,
> HBASE-19163-branch-1-v001.patch, HBASE-19163-master-v001.patch,
> HBASE-19163.master.001.patch, HBASE-19163.master.002.patch,
> HBASE-19163.master.004.patch, HBASE-19163.master.005.patch,
> HBASE-19163.master.006.patch, HBASE-19163.master.007.patch,
> HBASE-19163.master.008.patch, HBASE-19163.master.009.patch,
> HBASE-19163.master.009.patch, HBASE-19163.master.010.patch, unittest-case.diff
>
>
> In one of use cases, we found the following exception and replication is
> stuck.
> {code}
> 2017-10-25 19:41:17,199 WARN [hconnection-0x28db294f-shared--pool4-t936]
> client.AsyncProcess: #3, table=foo, attempt=5/5 failed=262836ops, last
> exception: java.io.IOException: java.io.IOException: Maximum lock count
> exceeded
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2215)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:109)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:185)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:165)
> Caused by: java.lang.Error: Maximum lock count exceeded
> at
> java.util.concurrent.locks.ReentrantReadWriteLock$Sync.fullTryAcquireShared(ReentrantReadWriteLock.java:528)
> at
> java.util.concurrent.locks.ReentrantReadWriteLock$Sync.tryAcquireShared(ReentrantReadWriteLock.java:488)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1327)
> at
> java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.tryLock(ReentrantReadWriteLock.java:871)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.getRowLock(HRegion.java:5163)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutation(HRegion.java:3018)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2877)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2819)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:753)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:715)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2148)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:33656)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2170)
> ... 3 more
> {code}
> While we are still examining the data pattern, it is sure that there are too
> many mutations in the batch against the same row, this exceeds the maximum
> 64k shared lock count and it throws an error and failed the whole batch.
> There are two approaches to solve this issue.
> 1). Let's say there are mutations against the same row in the batch, we just
> need to acquire the lock once for the same row vs to acquire the lock for
> each mutation.
> 2). We catch the error and start to process whatever it gets and loop back.
> With HBASE-17924, approach 1 seems easy to implement now.
> Create the jira and will post update/patch when investigation moving forward.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-19768) RegionServer startup failing when DN is dead
[
https://issues.apache.org/jira/browse/HBASE-19768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-19768:
--
Priority: Critical (was: Major)
> RegionServer startup failing when DN is dead
>
>
> Key: HBASE-19768
> URL: https://issues.apache.org/jira/browse/HBASE-19768
> Project: HBase
> Issue Type: Bug
>Reporter: Jean-Marc Spaggiari
>Priority: Critical
> Fix For: 2.0.0-beta-2
>
>
> When starting HBase, if the datanode hosted on the same host is dead but not
> yet detected by the namenode, HBase will fail to start
> {code}
> 515691223393/node8.distparser.com%2C16020%2C1515691223393.1515691238778
> failed, retry = 7
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException:
> syscall:getsockopt(..) failed: Connexion refusée: /192.168.23.2:50010
> at
> org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(..)(Unknown
> Source)
> Caused by:
> org.apache.hbase.thirdparty.io.netty.channel.unix.Errors$NativeConnectException:
> syscall:getsockopt(..) failed: Connexion refusée
> ... 1 more
> {code}
> and will also get stuck to stop:
> {code}
> hbase@node2:~/hbase-2.0.0-beta-1$ bin/stop-hbase.sh
> stopping
> hbase^C
> hbase@node2:~/hbase-2.0.0-beta-1$ bin/stop-hbase.sh
> stopping
> hbase..
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in
> [jar:file:/home/hbase/hbase-2.0.0-beta-1/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/home/hbase/hbase-2.0.0-beta-1/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
> {code}
> The most interesting is that it seems to fail the same way even if the DN is
> declared dead on HDFS side:
> {code}
> 515692041367/node8.distparser.com%2C16020%2C1515692041367.1515692057716
> failed, retry = 4
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException:
> syscall:getsockopt(..) failed: Connexion refusée: /192.168.23.2:50010
> at
> org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(..)(Unknown
> Source)
> Caused by:
> org.apache.hbase.thirdparty.io.netty.channel.unix.Errors$NativeConnectException:
> syscall:getsockopt(..) failed: Connexion refusée
> ... 1 more
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19752) RSGroupBasedLoadBalancer#getMisplacedRegions() should handle the case where rs group cannot be determined
[
https://issues.apache.org/jira/browse/HBASE-19752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16323158#comment-16323158
]
Mike Drob commented on HBASE-19752:
---
Add VisibleForTesting annotation please. I don't really understand the rest of
the issue here, so I will leave the rest of it to [~apurtell]
> RSGroupBasedLoadBalancer#getMisplacedRegions() should handle the case where
> rs group cannot be determined
> -
>
> Key: HBASE-19752
> URL: https://issues.apache.org/jira/browse/HBASE-19752
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 1.4.1, 1.5.0
>
> Attachments: 19752.v1.txt, 19752.v2.txt, 19752.v3.txt, 19752.v4.txt,
> 19752.v5.txt
>
>
> Observed the following in rs group test output:
> {code}
> 2018-01-10 14:17:23,006 DEBUG [AssignmentThread]
> rsgroup.RSGroupBasedLoadBalancer(316): Found misplaced region:
> hbase:acl,,1515593841277.ecf47ecb7522d7fab40db0a237f973fd. on server:
> localhost,1,1 found in group: null outside of group: UNKNOWN
> {code}
> Here is corresponding code:
> {code}
> if (assignedServer != null &&
> (info == null ||
> !info.containsServer(assignedServer.getAddress( {
> RSGroupInfo otherInfo = null;
> otherInfo =
> rsGroupInfoManager.getRSGroupOfServer(assignedServer.getAddress());
> LOG.debug("Found misplaced region: " +
> regionInfo.getRegionNameAsString() +
> {code}
> As you can see, both info and otherInfo were null.
> In this case, the region should not be placed in misplacedRegions.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19769) IllegalAccessError on package-private Hadoop metrics2 classes in MapReduce jobs
[
https://issues.apache.org/jira/browse/HBASE-19769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16323152#comment-16323152
]
Josh Elser commented on HBASE-19769:
bq. Gotcha! I was still stuck on leaving them server-side. Removing them
outright is fine.
Let me make a patch for this. I had essentially done this already to verify
that this was the problem. After nulling out the ZKMetrics class, I was able to
run through Phoenix's CSVBulkLoad tool.
> IllegalAccessError on package-private Hadoop metrics2 classes in MapReduce
> jobs
> ---
>
> Key: HBASE-19769
> URL: https://issues.apache.org/jira/browse/HBASE-19769
> Project: HBase
> Issue Type: Bug
> Components: mapreduce, metrics
>Affects Versions: 2.0.0-beta-1
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Critical
> Fix For: 2.0.0-beta-2
>
>
> issues for context: HBASE-17170, HBASE-17448, TEZ-3299, HADOOP-10893
> Since Hadoop 2.6.0, the {{yarn jar}} entry point to submit a YARN job has
> been using a custom classloader to separate Hadoop dependencies from the
> user's JAR being run. A separate classloader is created for the user-provided
> jar, and then this classloader is set as the contextClassLoader before the
> Tool is executed by Hadoop's RunJar class. This has been (mostly?) fine for
> us to date because we don't try to access any Hadoop internal classes
> client-side.
> However, with the ZK metrics, clients are pushing ZK metrics to metrics2. The
> problem is that Hadoop metrics2 implementations which we reference from the
> same package are loaded by a different classloader than our HBase code is
> loaded from. This makes the expected package-private access of these Metrics2
> classes (e.g. MetricsInfoImpl) fail with an IllegalAccessError.
> {noformat}
> java.lang.RuntimeException: Could not create interface
> org.apache.hadoop.hbase.zookeeper.MetricsZooKeeperSource Is the hadoop
> compatibility jar on the classpath?
> at
> org.apache.hadoop.hbase.CompatibilitySingletonFactory.getInstance(CompatibilitySingletonFactory.java:75)
> at
> org.apache.hadoop.hbase.zookeeper.ZKMetrics.(ZKMetrics.java:36)
> at
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.(RecoverableZooKeeper.java:115)
> at org.apache.hadoop.hbase.zookeeper.ZKUtil.connect(ZKUtil.java:139)
> at
> org.apache.hadoop.hbase.zookeeper.ZKWatcher.(ZKWatcher.java:128)
> at
> org.apache.hadoop.hbase.zookeeper.ZKWatcher.(ZKWatcher.java:102)
> at
> org.apache.hadoop.hbase.security.token.TokenUtil.getAuthToken(TokenUtil.java:293)
> at
> org.apache.hadoop.hbase.security.token.TokenUtil.addTokenForJob(TokenUtil.java:259)
> at
> org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.initCredentials(TableMapReduceUtil.java:535)
> at
> org.apache.phoenix.mapreduce.MultiHfileOutputFormat.configureIncrementalLoad(MultiHfileOutputFormat.java:712)
> at
> org.apache.phoenix.mapreduce.AbstractBulkLoadTool.submitJob(AbstractBulkLoadTool.java:300)
> at
> org.apache.phoenix.mapreduce.AbstractBulkLoadTool.loadData(AbstractBulkLoadTool.java:267)
> at
> org.apache.phoenix.mapreduce.AbstractBulkLoadTool.run(AbstractBulkLoadTool.java:180)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
> at
> org.apache.phoenix.mapreduce.CsvBulkLoadTool.main(CsvBulkLoadTool.java:109)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:239)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:153)
> Caused by: java.util.ServiceConfigurationError:
> org.apache.hadoop.hbase.zookeeper.MetricsZooKeeperSource: Provider
> org.apache.hadoop.hbase.zookeeper.MetricsZooKeeperSourceImpl could not be
> instantiated
> at java.util.ServiceLoader.fail(ServiceLoader.java:232)
> at java.util.ServiceLoader.access$100(ServiceLoader.java:185)
> at
> java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:384)
> at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
> at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
> at
> org.apache.hadoop.hbase.CompatibilitySingletonFactory.getInstance(CompatibilitySingletonFactory.java:59)
> ... 21 more
> Caused by: java.lang.IllegalAccessError: tried to access class
[jira] [Commented] (HBASE-15321) Ability to open a HRegion from hdfs snapshot.
[ https://issues.apache.org/jira/browse/HBASE-15321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16323147#comment-16323147 ] stack commented on HBASE-15321: --- +1 for Master and branch-2. Nice tool. Needs RM so folks can find it. > Ability to open a HRegion from hdfs snapshot. > - > > Key: HBASE-15321 > URL: https://issues.apache.org/jira/browse/HBASE-15321 > Project: HBase > Issue Type: New Feature >Affects Versions: 2.0.0, 3.0.0, 1.5.0 >Reporter: churro morales >Assignee: churro morales > Attachments: HBASE-15321.patch, HBASE-15321.v01.patch, > HBASE-15321.v02.patch > > > Now that hdfs snapshots are here, we started to run our mapreduce jobs over > hdfs snapshots. The thing is, hdfs snapshots are read-only point-in-time > copies of the file system. Thus we had to modify the section of code that > initialized the region internals in HRegion. We have to skip cleanup of > certain directories if the HRegion is backed by a hdfs snapshot. I have a > patch for trunk with some basic tests if folks are interested. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19775) hbase shell doesn't handle the exceptions that are wrapped in java.io.UncheckedIOException
[
https://issues.apache.org/jira/browse/HBASE-19775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16323146#comment-16323146
]
Mike Drob commented on HBASE-19775:
---
I was able to manually verify the fix, so the logic is good.
> hbase shell doesn't handle the exceptions that are wrapped in
> java.io.UncheckedIOException
> --
>
> Key: HBASE-19775
> URL: https://issues.apache.org/jira/browse/HBASE-19775
> Project: HBase
> Issue Type: Bug
> Components: shell
>Affects Versions: 2.0.0-beta-1
>Reporter: Sergey Soldatov
>Assignee: Sergey Soldatov
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19775-branch-2.patch
>
>
> HBase shell doesn't have a notion of UncheckedIOException, so it may not
> handle it correctly. For an example, if we scan not existing table the error
> look weird:
> {noformat}
> hbase(main):001:0> scan 'a'
> ROW
> COLUMN+CELL
> ERROR: a
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-19775) hbase shell doesn't handle the exceptions that are wrapped in java.io.UncheckedIOException
[
https://issues.apache.org/jira/browse/HBASE-19775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Drob updated HBASE-19775:
--
Fix Version/s: (was: 2.0.0-beta-1)
2.0.0-beta-2
> hbase shell doesn't handle the exceptions that are wrapped in
> java.io.UncheckedIOException
> --
>
> Key: HBASE-19775
> URL: https://issues.apache.org/jira/browse/HBASE-19775
> Project: HBase
> Issue Type: Bug
> Components: shell
>Affects Versions: 2.0.0-beta-1
>Reporter: Sergey Soldatov
>Assignee: Sergey Soldatov
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19775-branch-2.patch
>
>
> HBase shell doesn't have a notion of UncheckedIOException, so it may not
> handle it correctly. For an example, if we scan not existing table the error
> look weird:
> {noformat}
> hbase(main):001:0> scan 'a'
> ROW
> COLUMN+CELL
> ERROR: a
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-11409) Add more flexibility for input directory structure to LoadIncrementalHFiles
[ https://issues.apache.org/jira/browse/HBASE-11409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16323141#comment-16323141 ] stack commented on HBASE-11409: --- Does applying v5 work to master branch work for you [~churromorales]? git am fails. When I try git apply --reject it does this: kalashnikov:hbase.git.clean stack$ git apply --reject .git/rebase-apply/patch Checking patch src/main/java/org/apache/hadoop/hbase/tool/LoadIncrementalHFiles.java... error: src/main/java/org/apache/hadoop/hbase/tool/LoadIncrementalHFiles.java: No such file or directory Checking patch src/test/java/org/apache/hadoop/hbase/tool/TestLoadIncrementalHFiles.java... error: src/test/java/org/apache/hadoop/hbase/tool/TestLoadIncrementalHFiles.java: No such file or directory Maybe I'm doing something wrong? > Add more flexibility for input directory structure to LoadIncrementalHFiles > --- > > Key: HBASE-11409 > URL: https://issues.apache.org/jira/browse/HBASE-11409 > Project: HBase > Issue Type: Bug >Affects Versions: 3.0.0 >Reporter: churro morales >Assignee: churro morales > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-11409.v1.patch, HBASE-11409.v2.patch, > HBASE-11409.v3.patch, HBASE-11409.v4.patch, HBASE-11409.v5.patch, > HBASE-11409.v6.branch-1.patch > > > Use case: > We were trying to combine two very large tables into a single table. Thus we > ran jobs in one datacenter that populated certain column families and another > datacenter which populated other column families. Took a snapshot and > exported them to their respective datacenters. Wanted to simply take the > hdfs restored snapshot and use LoadIncremental to merge the data. > It would be nice to add support where we could run LoadIncremental on a > directory where the depth of store files is something other than two (current > behavior). > With snapshots it would be nice if you could pass a restored hdfs snapshot's > directory and have the tool run. > I am attaching a patch where I parameterize the bulkLoad timeout as well as > the default store file depth. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19775) hbase shell doesn't handle the exceptions that are wrapped in java.io.UncheckedIOException
[
https://issues.apache.org/jira/browse/HBASE-19775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16323142#comment-16323142
]
Mike Drob commented on HBASE-19775:
---
yep, @josh is right about the oneliner style - it's much more ruby-esque.
{{cause = cause.getCause if cause.is_a? java.io.UncheckedIOException}}
can we add a test in {{hbase-shell/src/test/ruby/hbase/table_test.rb}}?
> hbase shell doesn't handle the exceptions that are wrapped in
> java.io.UncheckedIOException
> --
>
> Key: HBASE-19775
> URL: https://issues.apache.org/jira/browse/HBASE-19775
> Project: HBase
> Issue Type: Bug
> Components: shell
>Affects Versions: 2.0.0-beta-1
>Reporter: Sergey Soldatov
>Assignee: Sergey Soldatov
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-19775-branch-2.patch
>
>
> HBase shell doesn't have a notion of UncheckedIOException, so it may not
> handle it correctly. For an example, if we scan not existing table the error
> look weird:
> {noformat}
> hbase(main):001:0> scan 'a'
> ROW
> COLUMN+CELL
> ERROR: a
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19769) IllegalAccessError on package-private Hadoop metrics2 classes in MapReduce jobs
[
https://issues.apache.org/jira/browse/HBASE-19769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16323137#comment-16323137
]
Josh Elser commented on HBASE-19769:
bq. Not sure I follow. To be clear, strip all zk metrics. The thing is too
verbose anyways and can look at ensemble and hbase logs to figure whats going
on?
Gotcha! I was still stuck on leaving them server-side. Removing them outright
is fine.
I think for client-metrics, we should use the work Ron has put up in
HBASE-18409 as the path forward. That will help avoid this hitting us yet again
(I think).
> IllegalAccessError on package-private Hadoop metrics2 classes in MapReduce
> jobs
> ---
>
> Key: HBASE-19769
> URL: https://issues.apache.org/jira/browse/HBASE-19769
> Project: HBase
> Issue Type: Bug
> Components: mapreduce, metrics
>Affects Versions: 2.0.0-beta-1
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Critical
> Fix For: 2.0.0-beta-2
>
>
> issues for context: HBASE-17170, HBASE-17448, TEZ-3299, HADOOP-10893
> Since Hadoop 2.6.0, the {{yarn jar}} entry point to submit a YARN job has
> been using a custom classloader to separate Hadoop dependencies from the
> user's JAR being run. A separate classloader is created for the user-provided
> jar, and then this classloader is set as the contextClassLoader before the
> Tool is executed by Hadoop's RunJar class. This has been (mostly?) fine for
> us to date because we don't try to access any Hadoop internal classes
> client-side.
> However, with the ZK metrics, clients are pushing ZK metrics to metrics2. The
> problem is that Hadoop metrics2 implementations which we reference from the
> same package are loaded by a different classloader than our HBase code is
> loaded from. This makes the expected package-private access of these Metrics2
> classes (e.g. MetricsInfoImpl) fail with an IllegalAccessError.
> {noformat}
> java.lang.RuntimeException: Could not create interface
> org.apache.hadoop.hbase.zookeeper.MetricsZooKeeperSource Is the hadoop
> compatibility jar on the classpath?
> at
> org.apache.hadoop.hbase.CompatibilitySingletonFactory.getInstance(CompatibilitySingletonFactory.java:75)
> at
> org.apache.hadoop.hbase.zookeeper.ZKMetrics.(ZKMetrics.java:36)
> at
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.(RecoverableZooKeeper.java:115)
> at org.apache.hadoop.hbase.zookeeper.ZKUtil.connect(ZKUtil.java:139)
> at
> org.apache.hadoop.hbase.zookeeper.ZKWatcher.(ZKWatcher.java:128)
> at
> org.apache.hadoop.hbase.zookeeper.ZKWatcher.(ZKWatcher.java:102)
> at
> org.apache.hadoop.hbase.security.token.TokenUtil.getAuthToken(TokenUtil.java:293)
> at
> org.apache.hadoop.hbase.security.token.TokenUtil.addTokenForJob(TokenUtil.java:259)
> at
> org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.initCredentials(TableMapReduceUtil.java:535)
> at
> org.apache.phoenix.mapreduce.MultiHfileOutputFormat.configureIncrementalLoad(MultiHfileOutputFormat.java:712)
> at
> org.apache.phoenix.mapreduce.AbstractBulkLoadTool.submitJob(AbstractBulkLoadTool.java:300)
> at
> org.apache.phoenix.mapreduce.AbstractBulkLoadTool.loadData(AbstractBulkLoadTool.java:267)
> at
> org.apache.phoenix.mapreduce.AbstractBulkLoadTool.run(AbstractBulkLoadTool.java:180)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
> at
> org.apache.phoenix.mapreduce.CsvBulkLoadTool.main(CsvBulkLoadTool.java:109)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:239)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:153)
> Caused by: java.util.ServiceConfigurationError:
> org.apache.hadoop.hbase.zookeeper.MetricsZooKeeperSource: Provider
> org.apache.hadoop.hbase.zookeeper.MetricsZooKeeperSourceImpl could not be
> instantiated
> at java.util.ServiceLoader.fail(ServiceLoader.java:232)
> at java.util.ServiceLoader.access$100(ServiceLoader.java:185)
> at
> java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:384)
> at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
> at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
> at
> org.apache.hadoop.hbase.CompatibilitySingletonFactory.getInstance(Compati
[jira] [Commented] (HBASE-19658) Fix and reenable TestCompactingToCellFlatMapMemStore#testFlatteningToJumboCellChunkMap
[ https://issues.apache.org/jira/browse/HBASE-19658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16323136#comment-16323136 ] stack commented on HBASE-19658: --- Say more [~jmspaggi] Paste in exception. It seems fine up on jenkins: https://builds.apache.org/view/H-L/view/HBase/job/HBase%20Nightly/job/branch-2/180/testReport/org.apache.hadoop.hbase.regionserver/TestCompactingToCellFlatMapMemStore/ > Fix and reenable > TestCompactingToCellFlatMapMemStore#testFlatteningToJumboCellChunkMap > -- > > Key: HBASE-19658 > URL: https://issues.apache.org/jira/browse/HBASE-19658 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 2.0.0-beta-1 >Reporter: stack >Assignee: Anastasia Braginsky > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19658-V01.patch, HBASE-19658-V02.patch, > HBASE-19658-V03.patch > > > testFlatteningToJumboCellChunkMap was disabled so could commit HBASE-19282 on > branch-2. This test is failing reliably. Assigned to [~anastas]. This issue > is about fixing the failing test and reenabling it in time for beta-2. Thanks > A. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19752) RSGroupBasedLoadBalancer#getMisplacedRegions() should handle the case where rs group cannot be determined
[
https://issues.apache.org/jira/browse/HBASE-19752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16323130#comment-16323130
]
Ted Yu commented on HBASE-19752:
[~apurtell]:
Do you have other comment ?
> RSGroupBasedLoadBalancer#getMisplacedRegions() should handle the case where
> rs group cannot be determined
> -
>
> Key: HBASE-19752
> URL: https://issues.apache.org/jira/browse/HBASE-19752
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 1.4.1, 1.5.0
>
> Attachments: 19752.v1.txt, 19752.v2.txt, 19752.v3.txt, 19752.v4.txt,
> 19752.v5.txt
>
>
> Observed the following in rs group test output:
> {code}
> 2018-01-10 14:17:23,006 DEBUG [AssignmentThread]
> rsgroup.RSGroupBasedLoadBalancer(316): Found misplaced region:
> hbase:acl,,1515593841277.ecf47ecb7522d7fab40db0a237f973fd. on server:
> localhost,1,1 found in group: null outside of group: UNKNOWN
> {code}
> Here is corresponding code:
> {code}
> if (assignedServer != null &&
> (info == null ||
> !info.containsServer(assignedServer.getAddress( {
> RSGroupInfo otherInfo = null;
> otherInfo =
> rsGroupInfoManager.getRSGroupOfServer(assignedServer.getAddress());
> LOG.debug("Found misplaced region: " +
> regionInfo.getRegionNameAsString() +
> {code}
> As you can see, both info and otherInfo were null.
> In this case, the region should not be placed in misplacedRegions.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (HBASE-19776) shell gives unfriendly errors for nonexistant table
[
https://issues.apache.org/jira/browse/HBASE-19776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16323126#comment-16323126
]
Mike Drob edited comment on HBASE-19776 at 1/11/18 10:40 PM:
-
Beat to the punch by [~sergey.soldatov] on HBASE-19775
was (Author: mdrob):
Beat to the punch by [~romil.choksi] on HBASE-19775
> shell gives unfriendly errors for nonexistant table
> ---
>
> Key: HBASE-19776
> URL: https://issues.apache.org/jira/browse/HBASE-19776
> Project: HBase
> Issue Type: Bug
>Reporter: Mike Drob
>
> When attempting operations against a non-existent table, some of the errors
> that we get are very unintuitive.
> {noformat}
> $ bin/hbase shell -n <<< "count 'no_such_table'"
> 2018-01-11 16:22:54,499 WARN [main] util.NativeCodeLoader: Unable to load
> native-hadoop library for your platform... using builtin-java classes where
> applicable
> Took 0.4709 seconds
>
>
> java exception
> ERROR Java::OrgApacheHadoopHbase::TableNotFoundException: Region of
> 'hbase:namespace,,1515709347596.af9aaa132e8baa3dcf2874ce6d93.' is
> expected in the table of 'no_such_table', but hbase:meta says it is in the
> table of 'hbase:namespace'. hbase:meta might be damaged.
> {noformat}
> Similar errors from scan, and from interactive shell instead on
> noninteractive mode.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19769) IllegalAccessError on package-private Hadoop metrics2 classes in MapReduce jobs
[
https://issues.apache.org/jira/browse/HBASE-19769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16323128#comment-16323128
]
stack commented on HBASE-19769:
---
bq. The problem with #1 is that I'm not sure how we determine when we're in the
context of a user and when we're in the context of a service.
Not sure I follow. To be clear, strip all zk metrics. The thing is too verbose
anyways and can look at ensemble and hbase logs to figure whats going on?
> IllegalAccessError on package-private Hadoop metrics2 classes in MapReduce
> jobs
> ---
>
> Key: HBASE-19769
> URL: https://issues.apache.org/jira/browse/HBASE-19769
> Project: HBase
> Issue Type: Bug
> Components: mapreduce, metrics
>Affects Versions: 2.0.0-beta-1
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Critical
> Fix For: 2.0.0-beta-2
>
>
> issues for context: HBASE-17170, HBASE-17448, TEZ-3299, HADOOP-10893
> Since Hadoop 2.6.0, the {{yarn jar}} entry point to submit a YARN job has
> been using a custom classloader to separate Hadoop dependencies from the
> user's JAR being run. A separate classloader is created for the user-provided
> jar, and then this classloader is set as the contextClassLoader before the
> Tool is executed by Hadoop's RunJar class. This has been (mostly?) fine for
> us to date because we don't try to access any Hadoop internal classes
> client-side.
> However, with the ZK metrics, clients are pushing ZK metrics to metrics2. The
> problem is that Hadoop metrics2 implementations which we reference from the
> same package are loaded by a different classloader than our HBase code is
> loaded from. This makes the expected package-private access of these Metrics2
> classes (e.g. MetricsInfoImpl) fail with an IllegalAccessError.
> {noformat}
> java.lang.RuntimeException: Could not create interface
> org.apache.hadoop.hbase.zookeeper.MetricsZooKeeperSource Is the hadoop
> compatibility jar on the classpath?
> at
> org.apache.hadoop.hbase.CompatibilitySingletonFactory.getInstance(CompatibilitySingletonFactory.java:75)
> at
> org.apache.hadoop.hbase.zookeeper.ZKMetrics.(ZKMetrics.java:36)
> at
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.(RecoverableZooKeeper.java:115)
> at org.apache.hadoop.hbase.zookeeper.ZKUtil.connect(ZKUtil.java:139)
> at
> org.apache.hadoop.hbase.zookeeper.ZKWatcher.(ZKWatcher.java:128)
> at
> org.apache.hadoop.hbase.zookeeper.ZKWatcher.(ZKWatcher.java:102)
> at
> org.apache.hadoop.hbase.security.token.TokenUtil.getAuthToken(TokenUtil.java:293)
> at
> org.apache.hadoop.hbase.security.token.TokenUtil.addTokenForJob(TokenUtil.java:259)
> at
> org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.initCredentials(TableMapReduceUtil.java:535)
> at
> org.apache.phoenix.mapreduce.MultiHfileOutputFormat.configureIncrementalLoad(MultiHfileOutputFormat.java:712)
> at
> org.apache.phoenix.mapreduce.AbstractBulkLoadTool.submitJob(AbstractBulkLoadTool.java:300)
> at
> org.apache.phoenix.mapreduce.AbstractBulkLoadTool.loadData(AbstractBulkLoadTool.java:267)
> at
> org.apache.phoenix.mapreduce.AbstractBulkLoadTool.run(AbstractBulkLoadTool.java:180)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
> at
> org.apache.phoenix.mapreduce.CsvBulkLoadTool.main(CsvBulkLoadTool.java:109)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:239)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:153)
> Caused by: java.util.ServiceConfigurationError:
> org.apache.hadoop.hbase.zookeeper.MetricsZooKeeperSource: Provider
> org.apache.hadoop.hbase.zookeeper.MetricsZooKeeperSourceImpl could not be
> instantiated
> at java.util.ServiceLoader.fail(ServiceLoader.java:232)
> at java.util.ServiceLoader.access$100(ServiceLoader.java:185)
> at
> java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:384)
> at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
> at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
> at
> org.apache.hadoop.hbase.CompatibilitySingletonFactory.getInstance(CompatibilitySingletonFactory.java:59)
> ... 21 more
> Caused by: java.lang.IllegalAccessError: tried to access class
> o
[jira] [Resolved] (HBASE-19776) shell gives unfriendly errors for nonexistant table
[
https://issues.apache.org/jira/browse/HBASE-19776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Drob resolved HBASE-19776.
---
Resolution: Duplicate
Fix Version/s: (was: 2.0.0-beta-2)
Beat to the punch by [~romil.choksi] on HBASE-19775
> shell gives unfriendly errors for nonexistant table
> ---
>
> Key: HBASE-19776
> URL: https://issues.apache.org/jira/browse/HBASE-19776
> Project: HBase
> Issue Type: Bug
>Reporter: Mike Drob
>
> When attempting operations against a non-existent table, some of the errors
> that we get are very unintuitive.
> {noformat}
> $ bin/hbase shell -n <<< "count 'no_such_table'"
> 2018-01-11 16:22:54,499 WARN [main] util.NativeCodeLoader: Unable to load
> native-hadoop library for your platform... using builtin-java classes where
> applicable
> Took 0.4709 seconds
>
>
> java exception
> ERROR Java::OrgApacheHadoopHbase::TableNotFoundException: Region of
> 'hbase:namespace,,1515709347596.af9aaa132e8baa3dcf2874ce6d93.' is
> expected in the table of 'no_such_table', but hbase:meta says it is in the
> table of 'hbase:namespace'. hbase:meta might be damaged.
> {noformat}
> Similar errors from scan, and from interactive shell instead on
> noninteractive mode.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19769) IllegalAccessError on package-private Hadoop metrics2 classes in MapReduce jobs
[
https://issues.apache.org/jira/browse/HBASE-19769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16323120#comment-16323120
]
Josh Elser commented on HBASE-19769:
bq. #1 sounds easy-to-do? Nice to have but an aspect of general client metrics
and user is not completely lost in that they can look at zk ensemble metrics.
#1 has never shipped in any hbase?
Sadly, it shipped in 1.4.0 -- makes me surprised how we didn't surface this in
1.4.0 testing. I guess we launch our Tools directly, and don't exec out to
yarn-jar or anything which would explain it. I haven't tested there yet.
The problem with #1 is that I'm not sure how we determine when we're in the
context of a user and when we're in the context of a service.
> IllegalAccessError on package-private Hadoop metrics2 classes in MapReduce
> jobs
> ---
>
> Key: HBASE-19769
> URL: https://issues.apache.org/jira/browse/HBASE-19769
> Project: HBase
> Issue Type: Bug
> Components: mapreduce, metrics
>Affects Versions: 2.0.0-beta-1
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Critical
> Fix For: 2.0.0-beta-2
>
>
> issues for context: HBASE-17170, HBASE-17448, TEZ-3299, HADOOP-10893
> Since Hadoop 2.6.0, the {{yarn jar}} entry point to submit a YARN job has
> been using a custom classloader to separate Hadoop dependencies from the
> user's JAR being run. A separate classloader is created for the user-provided
> jar, and then this classloader is set as the contextClassLoader before the
> Tool is executed by Hadoop's RunJar class. This has been (mostly?) fine for
> us to date because we don't try to access any Hadoop internal classes
> client-side.
> However, with the ZK metrics, clients are pushing ZK metrics to metrics2. The
> problem is that Hadoop metrics2 implementations which we reference from the
> same package are loaded by a different classloader than our HBase code is
> loaded from. This makes the expected package-private access of these Metrics2
> classes (e.g. MetricsInfoImpl) fail with an IllegalAccessError.
> {noformat}
> java.lang.RuntimeException: Could not create interface
> org.apache.hadoop.hbase.zookeeper.MetricsZooKeeperSource Is the hadoop
> compatibility jar on the classpath?
> at
> org.apache.hadoop.hbase.CompatibilitySingletonFactory.getInstance(CompatibilitySingletonFactory.java:75)
> at
> org.apache.hadoop.hbase.zookeeper.ZKMetrics.(ZKMetrics.java:36)
> at
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.(RecoverableZooKeeper.java:115)
> at org.apache.hadoop.hbase.zookeeper.ZKUtil.connect(ZKUtil.java:139)
> at
> org.apache.hadoop.hbase.zookeeper.ZKWatcher.(ZKWatcher.java:128)
> at
> org.apache.hadoop.hbase.zookeeper.ZKWatcher.(ZKWatcher.java:102)
> at
> org.apache.hadoop.hbase.security.token.TokenUtil.getAuthToken(TokenUtil.java:293)
> at
> org.apache.hadoop.hbase.security.token.TokenUtil.addTokenForJob(TokenUtil.java:259)
> at
> org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.initCredentials(TableMapReduceUtil.java:535)
> at
> org.apache.phoenix.mapreduce.MultiHfileOutputFormat.configureIncrementalLoad(MultiHfileOutputFormat.java:712)
> at
> org.apache.phoenix.mapreduce.AbstractBulkLoadTool.submitJob(AbstractBulkLoadTool.java:300)
> at
> org.apache.phoenix.mapreduce.AbstractBulkLoadTool.loadData(AbstractBulkLoadTool.java:267)
> at
> org.apache.phoenix.mapreduce.AbstractBulkLoadTool.run(AbstractBulkLoadTool.java:180)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
> at
> org.apache.phoenix.mapreduce.CsvBulkLoadTool.main(CsvBulkLoadTool.java:109)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:239)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:153)
> Caused by: java.util.ServiceConfigurationError:
> org.apache.hadoop.hbase.zookeeper.MetricsZooKeeperSource: Provider
> org.apache.hadoop.hbase.zookeeper.MetricsZooKeeperSourceImpl could not be
> instantiated
> at java.util.ServiceLoader.fail(ServiceLoader.java:232)
> at java.util.ServiceLoader.access$100(ServiceLoader.java:185)
> at
> java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:384)
> at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
>
[jira] [Commented] (HBASE-19770) `list` shell command does not return an array of table names
[
https://issues.apache.org/jira/browse/HBASE-19770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16323114#comment-16323114
]
Hadoop QA commented on HBASE-19770:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
9s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} branch-2 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
10s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
32s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
34s{color} | {color:green} branch-2 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
10s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
32s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} rubocop {color} | {color:red} 0m
7s{color} | {color:red} The patch generated 3 new + 48 unchanged - 2 fixed = 51
total (was 50) {color} |
| {color:green}+1{color} | {color:green} ruby-lint {color} | {color:green} 0m
2s{color} | {color:green} There were no new ruby-lint issues. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
32s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}138m 53s{color}
| {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
38s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}152m 38s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.client.TestShell |
| | hadoop.hbase.client.TestReplicationShell |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:9f2f2db |
| JIRA Issue | HBASE-19770 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12905736/HBASE-19770.001.branch-2.patch
|
| Optional Tests | asflicense rubocop ruby_lint javac javadoc unit |
| uname | Linux 89f4c3c43d8b 3.13.0-129-generic #178-Ubuntu SMP Fri Aug 11
12:48:20 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | branch-2 / 0dacdde0f9 |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| rubocop | v0.52.1 |
| rubocop |
https://builds.apache.org/job/PreCommit-HBASE-Build/11027/artifact/patchprocess/diff-patch-rubocop.txt
|
| ruby-lint | v2.3.1 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/11027/artifact/patchprocess/patch-unit-root.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/11027/testReport/ |
| modules | C: hbase-shell . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/11027/console |
| Powered by | Apache Yetus 0.6.0 http://yetus.apache.org |
This message was automatically generated.
> `list` shell command does not return an array of table names
>
>
> Key: HBASE-19770
> URL: https://issues.apache.org/jira/browse/HBASE-19770
> Project: HBase
> Issue Type: Bug
> Components: shell
>Reporter: Romil Choksi
>Assignee: Josh Elser
>Priority: Critical
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19770.001.branch-2.patch
>
>
> Another good find by our Romil.
> {code}
> hbase(main):001:0> list
> TABLE
> a
> 1 row(s)
> Took 0.8385 seconds
> hb
[jira] [Updated] (HBASE-19694) The initialization order for a fresh cluster is incorrect
[
https://issues.apache.org/jira/browse/HBASE-19694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-19694:
--
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Pushed to master and branch-2
> The initialization order for a fresh cluster is incorrect
> -
>
> Key: HBASE-19694
> URL: https://issues.apache.org/jira/browse/HBASE-19694
> Project: HBase
> Issue Type: Bug
>Reporter: Duo Zhang
>Assignee: stack
>Priority: Critical
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-19694.branch-2.001.patch,
> HBASE-19694.branch-2.002.patch, HBASE-19694.branch-2.003.patch,
> HBASE-19694.branch-2.004.patch, HBASE-19694.branch-2.005.patch,
> HBASE-19694.branch-2.006.patch, HBASE-19694.branch-2.007.patch,
> HBASE-19694.branch-2.008.patch, HBASE-19694.branch-2.009.patch,
> HBASE-19694.branch-2.010.patch, HBASE-19694.branch-2.011.patch,
> HBASE-19694.branch-2.012.patch
>
>
> The cluster id will set once we become the active master in
> finishActiveMasterInitialization, but the blockUntilBecomingActiveMaster and
> finishActiveMasterInitialization are both called in a thread to make the
> constructor of HMaster return without blocking. And since HMaster itself is
> also a HRegionServer, it will create a Connection and then start calling
> reportForDuty. And when creating the ConnectionImplementation, we will read
> the cluster id from zk, but the cluster id may have not been set yet since it
> is set in another thread, we will get an exception and use the default
> cluster id instead.
> I always get this when running UTs which will start a mini cluster
> {noformat}
> 2018-01-03 15:16:37,916 WARN [M:0;zhangduo-ubuntu:32848]
> client.ConnectionImplementation(528): Retrieve cluster id failed
> java.util.concurrent.ExecutionException:
> org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode =
> NoNode for /hbase/hbaseid
> at
> java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
> at
> java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1895)
> at
> org.apache.hadoop.hbase.client.ConnectionImplementation.retrieveClusterId(ConnectionImplementation.java:526)
> at
> org.apache.hadoop.hbase.client.ConnectionImplementation.(ConnectionImplementation.java:286)
> at
> org.apache.hadoop.hbase.client.ConnectionUtils$ShortCircuitingClusterConnection.(ConnectionUtils.java:141)
> at
> org.apache.hadoop.hbase.client.ConnectionUtils$ShortCircuitingClusterConnection.(ConnectionUtils.java:137)
> at
> org.apache.hadoop.hbase.client.ConnectionUtils.createShortCircuitConnection(ConnectionUtils.java:185)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.createClusterConnection(HRegionServer.java:781)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.setupClusterConnection(HRegionServer.java:812)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.preRegistrationInitialization(HRegionServer.java:827)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:938)
> at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:550)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.zookeeper.KeeperException$NoNodeException:
> KeeperErrorCode = NoNode for /hbase/hbaseid
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> at
> org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$ZKTask$1.exec(ReadOnlyZKClient.java:163)
> at
> org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:311)
> ... 1 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19775) hbase shell doesn't handle the exceptions that are wrapped in java.io.UncheckedIOException
[
https://issues.apache.org/jira/browse/HBASE-19775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16323103#comment-16323103
]
Hadoop QA commented on HBASE-19775:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 3m
58s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} branch-2 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
19s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
17s{color} | {color:green} branch-2 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
5s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} rubocop {color} | {color:red} 0m
3s{color} | {color:red} The patch generated 6 new + 24 unchanged - 4 fixed = 30
total (was 28) {color} |
| {color:red}-1{color} | {color:red} ruby-lint {color} | {color:red} 0m
2s{color} | {color:red} The patch generated 1 new + 13 unchanged - 0 fixed = 14
total (was 13) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
10s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 12m
32s{color} | {color:green} hbase-shell in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 28m 52s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:9f2f2db |
| JIRA Issue | HBASE-19775 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12905749/HBASE-19775-branch-2.patch
|
| Optional Tests | asflicense javac javadoc unit rubocop ruby_lint |
| uname | Linux b6f3c7287163 3.13.0-137-generic #186-Ubuntu SMP Mon Dec 4
19:09:19 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | branch-2 / 25e4bf8f37 |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| rubocop | v0.52.1 |
| rubocop |
https://builds.apache.org/job/PreCommit-HBASE-Build/11029/artifact/patchprocess/diff-patch-rubocop.txt
|
| ruby-lint | v2.3.1 |
| ruby-lint |
https://builds.apache.org/job/PreCommit-HBASE-Build/11029/artifact/patchprocess/diff-patch-ruby-lint.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/11029/testReport/ |
| modules | C: hbase-shell U: hbase-shell |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/11029/console |
| Powered by | Apache Yetus 0.6.0 http://yetus.apache.org |
This message was automatically generated.
> hbase shell doesn't handle the exceptions that are wrapped in
> java.io.UncheckedIOException
> --
>
> Key: HBASE-19775
> URL: https://issues.apache.org/jira/browse/HBASE-19775
> Project: HBase
> Issue Type: Bug
> Components: shell
>Affects Versions: 2.0.0-beta-1
>Reporter: Sergey Soldatov
>Assignee: Sergey Soldatov
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-19775-branch-2.patch
>
>
> HBase shell doesn't have a notion of UncheckedIOException, so it may not
> handle it correctly. For an example, if we scan not existing table the error
> look weird:
> {noformat}
> hbase(main):001:0> scan 'a'
> ROW
> COLUMN+CELL
> ERROR: a
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (HBASE-19776) shell gives unfriendly errors for nonexistant table
Mike Drob created HBASE-19776:
-
Summary: shell gives unfriendly errors for nonexistant table
Key: HBASE-19776
URL: https://issues.apache.org/jira/browse/HBASE-19776
Project: HBase
Issue Type: Bug
Reporter: Mike Drob
Fix For: 2.0.0-beta-2
When attempting operations against a non-existent table, some of the errors
that we get are very unintuitive.
{noformat}
$ bin/hbase shell -n <<< "count 'no_such_table'"
2018-01-11 16:22:54,499 WARN [main] util.NativeCodeLoader: Unable to load
native-hadoop library for your platform... using builtin-java classes where
applicable
Took 0.4709 seconds
java exception
ERROR Java::OrgApacheHadoopHbase::TableNotFoundException: Region of
'hbase:namespace,,1515709347596.af9aaa132e8baa3dcf2874ce6d93.' is expected
in the table of 'no_such_table', but hbase:meta says it is in the table of
'hbase:namespace'. hbase:meta might be damaged.
{noformat}
Similar errors from scan, and from interactive shell instead on noninteractive
mode.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19694) The initialization order for a fresh cluster is incorrect
[
https://issues.apache.org/jira/browse/HBASE-19694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16323099#comment-16323099
]
stack commented on HBASE-19694:
---
New set of timeouts TestMultiColumnScanner, or TestCompoundBloomFilter and
TestMultiColumnScanner which don't even start a cluster and are just medium
sized test.
Going to push this. Work on flakies separately.
> The initialization order for a fresh cluster is incorrect
> -
>
> Key: HBASE-19694
> URL: https://issues.apache.org/jira/browse/HBASE-19694
> Project: HBase
> Issue Type: Bug
>Reporter: Duo Zhang
>Assignee: stack
>Priority: Critical
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-19694.branch-2.001.patch,
> HBASE-19694.branch-2.002.patch, HBASE-19694.branch-2.003.patch,
> HBASE-19694.branch-2.004.patch, HBASE-19694.branch-2.005.patch,
> HBASE-19694.branch-2.006.patch, HBASE-19694.branch-2.007.patch,
> HBASE-19694.branch-2.008.patch, HBASE-19694.branch-2.009.patch,
> HBASE-19694.branch-2.010.patch, HBASE-19694.branch-2.011.patch,
> HBASE-19694.branch-2.012.patch
>
>
> The cluster id will set once we become the active master in
> finishActiveMasterInitialization, but the blockUntilBecomingActiveMaster and
> finishActiveMasterInitialization are both called in a thread to make the
> constructor of HMaster return without blocking. And since HMaster itself is
> also a HRegionServer, it will create a Connection and then start calling
> reportForDuty. And when creating the ConnectionImplementation, we will read
> the cluster id from zk, but the cluster id may have not been set yet since it
> is set in another thread, we will get an exception and use the default
> cluster id instead.
> I always get this when running UTs which will start a mini cluster
> {noformat}
> 2018-01-03 15:16:37,916 WARN [M:0;zhangduo-ubuntu:32848]
> client.ConnectionImplementation(528): Retrieve cluster id failed
> java.util.concurrent.ExecutionException:
> org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode =
> NoNode for /hbase/hbaseid
> at
> java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
> at
> java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1895)
> at
> org.apache.hadoop.hbase.client.ConnectionImplementation.retrieveClusterId(ConnectionImplementation.java:526)
> at
> org.apache.hadoop.hbase.client.ConnectionImplementation.(ConnectionImplementation.java:286)
> at
> org.apache.hadoop.hbase.client.ConnectionUtils$ShortCircuitingClusterConnection.(ConnectionUtils.java:141)
> at
> org.apache.hadoop.hbase.client.ConnectionUtils$ShortCircuitingClusterConnection.(ConnectionUtils.java:137)
> at
> org.apache.hadoop.hbase.client.ConnectionUtils.createShortCircuitConnection(ConnectionUtils.java:185)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.createClusterConnection(HRegionServer.java:781)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.setupClusterConnection(HRegionServer.java:812)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.preRegistrationInitialization(HRegionServer.java:827)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:938)
> at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:550)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.zookeeper.KeeperException$NoNodeException:
> KeeperErrorCode = NoNode for /hbase/hbaseid
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> at
> org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$ZKTask$1.exec(ReadOnlyZKClient.java:163)
> at
> org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:311)
> ... 1 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-11409) Add more flexibility for input directory structure to LoadIncrementalHFiles
[ https://issues.apache.org/jira/browse/HBASE-11409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16323097#comment-16323097 ] churro morales commented on HBASE-11409: Those tests failures seem unrelated and when i ran them locally in isolation they all passed. [~apurtell] the branch-1 patch is a bit different from the master, 2x patch. I didn't want to change the API's so I had to do things a bit differently. If someone doesn't mind reviewing, I can get this into branch-1 and branch-1.4 Thanks > Add more flexibility for input directory structure to LoadIncrementalHFiles > --- > > Key: HBASE-11409 > URL: https://issues.apache.org/jira/browse/HBASE-11409 > Project: HBase > Issue Type: Bug >Affects Versions: 3.0.0 >Reporter: churro morales >Assignee: churro morales > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-11409.v1.patch, HBASE-11409.v2.patch, > HBASE-11409.v3.patch, HBASE-11409.v4.patch, HBASE-11409.v5.patch, > HBASE-11409.v6.branch-1.patch > > > Use case: > We were trying to combine two very large tables into a single table. Thus we > ran jobs in one datacenter that populated certain column families and another > datacenter which populated other column families. Took a snapshot and > exported them to their respective datacenters. Wanted to simply take the > hdfs restored snapshot and use LoadIncremental to merge the data. > It would be nice to add support where we could run LoadIncremental on a > directory where the depth of store files is something other than two (current > behavior). > With snapshots it would be nice if you could pass a restored hdfs snapshot's > directory and have the tool run. > I am attaching a patch where I parameterize the bulkLoad timeout as well as > the default store file depth. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19775) hbase shell doesn't handle the exceptions that are wrapped in java.io.UncheckedIOException
[
https://issues.apache.org/jira/browse/HBASE-19775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16323072#comment-16323072
]
Josh Elser commented on HBASE-19775:
If it were me, I'd turn it into a oneline thing, but it's semantically
equivalent and maybe just my take :)
{code}
cause = cause.getCause() if cause.is_a? java.io.UncheckedIOException
{code}
We do have some Ruby tests in hbase-shell/src/test/ruby (invoked via some JUnit
tests e.g. hbase-shell/src/test/java/**/TestShell.java). What was the context
in which you saw this UncheckedIOException -- something we could reproduce
easily?
[~mdrob] has been in here recently too. Maybe he can drop a review :)
> hbase shell doesn't handle the exceptions that are wrapped in
> java.io.UncheckedIOException
> --
>
> Key: HBASE-19775
> URL: https://issues.apache.org/jira/browse/HBASE-19775
> Project: HBase
> Issue Type: Bug
> Components: shell
>Affects Versions: 2.0.0-beta-1
>Reporter: Sergey Soldatov
>Assignee: Sergey Soldatov
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-19775-branch-2.patch
>
>
> HBase shell doesn't have a notion of UncheckedIOException, so it may not
> handle it correctly. For an example, if we scan not existing table the error
> look weird:
> {noformat}
> hbase(main):001:0> scan 'a'
> ROW
> COLUMN+CELL
> ERROR: a
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19694) The initialization order for a fresh cluster is incorrect
[
https://issues.apache.org/jira/browse/HBASE-19694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16323065#comment-16323065
]
Hadoop QA commented on HBASE-19694:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
10s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} branch-2 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
45s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
45s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
8s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
13s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
30s{color} | {color:green} branch-2 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
58s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
15m 45s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}107m 0s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}139m 38s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:9f2f2db |
| JIRA Issue | HBASE-19694 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12905735/HBASE-19694.branch-2.012.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux cc1911df2985 3.13.0-133-generic #182-Ubuntu SMP Tue Sep 19
15:49:21 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | branch-2 / 0dacdde0f9 |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/11026/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/11026/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/11026/console |
| Powered by | Apache Yetus 0.6.0 http://yetus.apache.org |
This message was automatically generated.
> The initialization order for a fresh cluster is incorrect
> --
[jira] [Commented] (HBASE-19769) IllegalAccessError on package-private Hadoop metrics2 classes in MapReduce jobs
[
https://issues.apache.org/jira/browse/HBASE-19769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16323051#comment-16323051
]
stack commented on HBASE-19769:
---
We don't want to use hadoops metrics long-term if thats of any use to. Its only
been a pain. Modules for hadoop1 vs hadoop2. We have hacks in place to kill it
and restart it because no other way of onlining new metrics -- e.g. region
added/removed. We spend most of our CPU counting so we should probably own the
metrics system rather than defer to another.
#1 sounds easy-to-do? Nice to have but an aspect of general client metrics and
user is not completely lost in that they can look at zk ensemble metrics. #1
has never shipped in any hbase?
> IllegalAccessError on package-private Hadoop metrics2 classes in MapReduce
> jobs
> ---
>
> Key: HBASE-19769
> URL: https://issues.apache.org/jira/browse/HBASE-19769
> Project: HBase
> Issue Type: Bug
> Components: mapreduce, metrics
>Affects Versions: 2.0.0-beta-1
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Critical
> Fix For: 2.0.0-beta-2
>
>
> issues for context: HBASE-17170, HBASE-17448, TEZ-3299, HADOOP-10893
> Since Hadoop 2.6.0, the {{yarn jar}} entry point to submit a YARN job has
> been using a custom classloader to separate Hadoop dependencies from the
> user's JAR being run. A separate classloader is created for the user-provided
> jar, and then this classloader is set as the contextClassLoader before the
> Tool is executed by Hadoop's RunJar class. This has been (mostly?) fine for
> us to date because we don't try to access any Hadoop internal classes
> client-side.
> However, with the ZK metrics, clients are pushing ZK metrics to metrics2. The
> problem is that Hadoop metrics2 implementations which we reference from the
> same package are loaded by a different classloader than our HBase code is
> loaded from. This makes the expected package-private access of these Metrics2
> classes (e.g. MetricsInfoImpl) fail with an IllegalAccessError.
> {noformat}
> java.lang.RuntimeException: Could not create interface
> org.apache.hadoop.hbase.zookeeper.MetricsZooKeeperSource Is the hadoop
> compatibility jar on the classpath?
> at
> org.apache.hadoop.hbase.CompatibilitySingletonFactory.getInstance(CompatibilitySingletonFactory.java:75)
> at
> org.apache.hadoop.hbase.zookeeper.ZKMetrics.(ZKMetrics.java:36)
> at
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.(RecoverableZooKeeper.java:115)
> at org.apache.hadoop.hbase.zookeeper.ZKUtil.connect(ZKUtil.java:139)
> at
> org.apache.hadoop.hbase.zookeeper.ZKWatcher.(ZKWatcher.java:128)
> at
> org.apache.hadoop.hbase.zookeeper.ZKWatcher.(ZKWatcher.java:102)
> at
> org.apache.hadoop.hbase.security.token.TokenUtil.getAuthToken(TokenUtil.java:293)
> at
> org.apache.hadoop.hbase.security.token.TokenUtil.addTokenForJob(TokenUtil.java:259)
> at
> org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.initCredentials(TableMapReduceUtil.java:535)
> at
> org.apache.phoenix.mapreduce.MultiHfileOutputFormat.configureIncrementalLoad(MultiHfileOutputFormat.java:712)
> at
> org.apache.phoenix.mapreduce.AbstractBulkLoadTool.submitJob(AbstractBulkLoadTool.java:300)
> at
> org.apache.phoenix.mapreduce.AbstractBulkLoadTool.loadData(AbstractBulkLoadTool.java:267)
> at
> org.apache.phoenix.mapreduce.AbstractBulkLoadTool.run(AbstractBulkLoadTool.java:180)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
> at
> org.apache.phoenix.mapreduce.CsvBulkLoadTool.main(CsvBulkLoadTool.java:109)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:239)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:153)
> Caused by: java.util.ServiceConfigurationError:
> org.apache.hadoop.hbase.zookeeper.MetricsZooKeeperSource: Provider
> org.apache.hadoop.hbase.zookeeper.MetricsZooKeeperSourceImpl could not be
> instantiated
> at java.util.ServiceLoader.fail(ServiceLoader.java:232)
> at java.util.ServiceLoader.access$100(ServiceLoader.java:185)
> at
> java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:384)
> at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
> at java.util.Se
[jira] [Commented] (HBASE-11409) Add more flexibility for input directory structure to LoadIncrementalHFiles
[
https://issues.apache.org/jira/browse/HBASE-11409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16323035#comment-16323035
]
Hadoop QA commented on HBASE-11409:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 13m
29s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} branch-1 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
4s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
32s{color} | {color:green} branch-1 passed with JDK v1.8.0_152 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
37s{color} | {color:green} branch-1 passed with JDK v1.7.0_161 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
17s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
50s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
15s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
28s{color} | {color:green} branch-1 passed with JDK v1.8.0_152 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
36s{color} | {color:green} branch-1 passed with JDK v1.7.0_161 {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
39s{color} | {color:green} the patch passed with JDK v1.8.0_152 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
38s{color} | {color:green} the patch passed with JDK v1.7.0_161 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
16s{color} | {color:red} hbase-server: The patch generated 2 new + 20 unchanged
- 5 fixed = 22 total (was 25) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
34s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
10m 14s{color} | {color:green} Patch does not cause any errors with Hadoop
2.4.1 2.5.2 2.6.5 2.7.4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
27s{color} | {color:green} the patch passed with JDK v1.8.0_152 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
34s{color} | {color:green} the patch passed with JDK v1.7.0_161 {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 99m 59s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}142m 55s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.client.TestAdmin1 |
| | hadoop.hbase.regionserver.TestRegionServerAbort |
| | hadoop.hbase.util.TestHBaseFsck |
| | hadoop.hbase.procedure.TestZKProcedure |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:36a7029 |
| JIRA Issue | H
[jira] [Updated] (HBASE-19774) incorrect behavior of locateRegionInMeta
[
https://issues.apache.org/jira/browse/HBASE-19774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Soldatov updated HBASE-19774:
Status: Patch Available (was: Open)
> incorrect behavior of locateRegionInMeta
>
>
> Key: HBASE-19774
> URL: https://issues.apache.org/jira/browse/HBASE-19774
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0-beta-1
>Reporter: Romil Choksi
>Assignee: Sergey Soldatov
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-19774-wip-branch-2.patch
>
>
> When we try to operate with not existing table, in some circumstances we get
> an incorrect report about the not existing table:
> {noformat}
> ERROR: Region of
> 'hbase:namespace,,1510363071508.0d8ddea7654f95130959218e9bc9c89c.' is
> expected in the table of 'nonExistentUsertable', but hbase:meta says it is in
> the table of 'hbase:namespace'. hbase:meta might be damaged.
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-19774) incorrect behavior of locateRegionInMeta
[
https://issues.apache.org/jira/browse/HBASE-19774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Soldatov updated HBASE-19774:
Attachment: HBASE-19774-wip-branch-2.patch
PoC for (1) to check whether it breaks something.
> incorrect behavior of locateRegionInMeta
>
>
> Key: HBASE-19774
> URL: https://issues.apache.org/jira/browse/HBASE-19774
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0-beta-1
>Reporter: Romil Choksi
>Assignee: Sergey Soldatov
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-19774-wip-branch-2.patch
>
>
> When we try to operate with not existing table, in some circumstances we get
> an incorrect report about the not existing table:
> {noformat}
> ERROR: Region of
> 'hbase:namespace,,1510363071508.0d8ddea7654f95130959218e9bc9c89c.' is
> expected in the table of 'nonExistentUsertable', but hbase:meta says it is in
> the table of 'hbase:namespace'. hbase:meta might be damaged.
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19775) hbase shell doesn't handle the exceptions that are wrapped in java.io.UncheckedIOException
[
https://issues.apache.org/jira/browse/HBASE-19775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16322982#comment-16322982
]
Sergey Soldatov commented on HBASE-19775:
-
FYI [~elserj], [~romil.choksi]
> hbase shell doesn't handle the exceptions that are wrapped in
> java.io.UncheckedIOException
> --
>
> Key: HBASE-19775
> URL: https://issues.apache.org/jira/browse/HBASE-19775
> Project: HBase
> Issue Type: Bug
> Components: shell
>Affects Versions: 2.0.0-beta-1
>Reporter: Sergey Soldatov
>Assignee: Sergey Soldatov
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-19775-branch-2.patch
>
>
> HBase shell doesn't have a notion of UncheckedIOException, so it may not
> handle it correctly. For an example, if we scan not existing table the error
> look weird:
> {noformat}
> hbase(main):001:0> scan 'a'
> ROW
> COLUMN+CELL
> ERROR: a
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-19775) hbase shell doesn't handle the exceptions that are wrapped in java.io.UncheckedIOException
[
https://issues.apache.org/jira/browse/HBASE-19775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Soldatov updated HBASE-19775:
Attachment: HBASE-19775-branch-2.patch
A simple fix. I'm not a ruby guy, so, possible there is a more fancy way to
handle it exists.
> hbase shell doesn't handle the exceptions that are wrapped in
> java.io.UncheckedIOException
> --
>
> Key: HBASE-19775
> URL: https://issues.apache.org/jira/browse/HBASE-19775
> Project: HBase
> Issue Type: Bug
> Components: shell
>Affects Versions: 2.0.0-beta-1
>Reporter: Sergey Soldatov
>Assignee: Sergey Soldatov
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-19775-branch-2.patch
>
>
> HBase shell doesn't have a notion of UncheckedIOException, so it may not
> handle it correctly. For an example, if we scan not existing table the error
> look weird:
> {noformat}
> hbase(main):001:0> scan 'a'
> ROW
> COLUMN+CELL
> ERROR: a
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-19775) hbase shell doesn't handle the exceptions that are wrapped in java.io.UncheckedIOException
[
https://issues.apache.org/jira/browse/HBASE-19775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Soldatov updated HBASE-19775:
Status: Patch Available (was: Open)
> hbase shell doesn't handle the exceptions that are wrapped in
> java.io.UncheckedIOException
> --
>
> Key: HBASE-19775
> URL: https://issues.apache.org/jira/browse/HBASE-19775
> Project: HBase
> Issue Type: Bug
> Components: shell
>Affects Versions: 2.0.0-beta-1
>Reporter: Sergey Soldatov
>Assignee: Sergey Soldatov
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-19775-branch-2.patch
>
>
> HBase shell doesn't have a notion of UncheckedIOException, so it may not
> handle it correctly. For an example, if we scan not existing table the error
> look weird:
> {noformat}
> hbase(main):001:0> scan 'a'
> ROW
> COLUMN+CELL
> ERROR: a
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (HBASE-19775) hbase shell doesn't handle the exceptions that are wrapped in java.io.UncheckedIOException
Sergey Soldatov created HBASE-19775:
---
Summary: hbase shell doesn't handle the exceptions that are
wrapped in java.io.UncheckedIOException
Key: HBASE-19775
URL: https://issues.apache.org/jira/browse/HBASE-19775
Project: HBase
Issue Type: Bug
Components: shell
Affects Versions: 2.0.0-beta-1
Reporter: Sergey Soldatov
Assignee: Sergey Soldatov
Fix For: 2.0.0-beta-1
HBase shell doesn't have a notion of UncheckedIOException, so it may not handle
it correctly. For an example, if we scan not existing table the error look
weird:
{noformat}
hbase(main):001:0> scan 'a'
ROW
COLUMN+CELL
ERROR: a
{noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19774) incorrect behavior of locateRegionInMeta
[
https://issues.apache.org/jira/browse/HBASE-19774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16322965#comment-16322965
]
Josh Elser commented on HBASE-19774:
bq. 1. Add stop row to the corresponding scan. Not sure whether we have any
corner cases that may be affected.
This option makes more sense to me. Doing the validation in #2 makes me think
that we don't trust HBase to give us the data we asked for :P
> incorrect behavior of locateRegionInMeta
>
>
> Key: HBASE-19774
> URL: https://issues.apache.org/jira/browse/HBASE-19774
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0-beta-1
>Reporter: Romil Choksi
>Assignee: Sergey Soldatov
> Fix For: 2.0.0-beta-1
>
>
> When we try to operate with not existing table, in some circumstances we get
> an incorrect report about the not existing table:
> {noformat}
> ERROR: Region of
> 'hbase:namespace,,1510363071508.0d8ddea7654f95130959218e9bc9c89c.' is
> expected in the table of 'nonExistentUsertable', but hbase:meta says it is in
> the table of 'hbase:namespace'. hbase:meta might be damaged.
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Resolved] (HBASE-19773) Adding javadoc around getting instance of RawCellBuilder
[ https://issues.apache.org/jira/browse/HBASE-19773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy resolved HBASE-19773. -- Resolution: Fixed > Adding javadoc around getting instance of RawCellBuilder > > > Key: HBASE-19773 > URL: https://issues.apache.org/jira/browse/HBASE-19773 > Project: HBase > Issue Type: Bug >Reporter: Appy >Assignee: Appy >Priority: Minor > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19773.master.001.patch, > HBASE-19773.master.002.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19773) Adding javadoc around getting instance of RawCellBuilder
[ https://issues.apache.org/jira/browse/HBASE-19773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16322962#comment-16322962 ] Appy commented on HBASE-19773: -- Yeah, it seemed odd (since i couldn't use @link in javadoc) but seemed important. Thanks for quick review. Pushed to branch-2 and master. > Adding javadoc around getting instance of RawCellBuilder > > > Key: HBASE-19773 > URL: https://issues.apache.org/jira/browse/HBASE-19773 > Project: HBase > Issue Type: Bug >Reporter: Appy >Assignee: Appy >Priority: Minor > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19773.master.001.patch, > HBASE-19773.master.002.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19773) Adding javadoc around getting instance of RawCellBuilder
[ https://issues.apache.org/jira/browse/HBASE-19773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-19773: - Fix Version/s: 2.0.0-beta-2 > Adding javadoc around getting instance of RawCellBuilder > > > Key: HBASE-19773 > URL: https://issues.apache.org/jira/browse/HBASE-19773 > Project: HBase > Issue Type: Bug >Reporter: Appy >Assignee: Appy >Priority: Minor > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19773.master.001.patch, > HBASE-19773.master.002.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19774) incorrect behavior of locateRegionInMeta
[
https://issues.apache.org/jira/browse/HBASE-19774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16322957#comment-16322957
]
Sergey Soldatov commented on HBASE-19774:
-
Well, that problem persists for quite period of time. But for 1.x releases this
message was not that scary (that was something like "Table X not found, got
Y"). The problem is that in locateRegionInMeta we are using ReverseScan without
setting stopRow. So if the given table doesn't exist we may return a record any
other table. In branch-2 the exception message was changed and now it actually
looks scary and may lead the user to think that HBase has problems with meta
table.
I see two ways how to fix it :
1. Add stop row to the corresponding scan. Not sure whether we have any corner
cases that may be affected.
2. Consider that the returned record as an expected result for the missing
table and just report in a regular way without that scary message.
Personally, I would prefer the first way. Any suggestions, [~stack], [~elserj],
[~te...@apache.org] ?
> incorrect behavior of locateRegionInMeta
>
>
> Key: HBASE-19774
> URL: https://issues.apache.org/jira/browse/HBASE-19774
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0-beta-1
>Reporter: Romil Choksi
>Assignee: Sergey Soldatov
> Fix For: 2.0.0-beta-1
>
>
> When we try to operate with not existing table, in some circumstances we get
> an incorrect report about the not existing table:
> {noformat}
> ERROR: Region of
> 'hbase:namespace,,1510363071508.0d8ddea7654f95130959218e9bc9c89c.' is
> expected in the table of 'nonExistentUsertable', but hbase:meta says it is in
> the table of 'hbase:namespace'. hbase:meta might be damaged.
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19773) Adding javadoc around getting instance of RawCellBuilder
[ https://issues.apache.org/jira/browse/HBASE-19773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16322954#comment-16322954 ] stack commented on HBASE-19773: --- I suppose doc is good. Its odd talking about cps out in hbase-common... but will help someone I think... +1 > Adding javadoc around getting instance of RawCellBuilder > > > Key: HBASE-19773 > URL: https://issues.apache.org/jira/browse/HBASE-19773 > Project: HBase > Issue Type: Bug >Reporter: Appy >Assignee: Appy >Priority: Minor > Attachments: HBASE-19773.master.001.patch, > HBASE-19773.master.002.patch > > > Need to make RawCellBuilderFactory LP.Coproc so that CPs can make an instance > of RawCellBuilder type. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19773) Adding javadoc around getting instance of RawCellBuilder
[ https://issues.apache.org/jira/browse/HBASE-19773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-19773: - Description: (was: Need to make RawCellBuilderFactory LP.Coproc so that CPs can make an instance of RawCellBuilder type. ) > Adding javadoc around getting instance of RawCellBuilder > > > Key: HBASE-19773 > URL: https://issues.apache.org/jira/browse/HBASE-19773 > Project: HBase > Issue Type: Bug >Reporter: Appy >Assignee: Appy >Priority: Minor > Attachments: HBASE-19773.master.001.patch, > HBASE-19773.master.002.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19773) Adding javadoc around getting instance of RawCellBuilder
[ https://issues.apache.org/jira/browse/HBASE-19773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-19773: - Summary: Adding javadoc around getting instance of RawCellBuilder (was: RawCellBuilder is LP.Coproc, it's factory should have same InterfaceAudience) > Adding javadoc around getting instance of RawCellBuilder > > > Key: HBASE-19773 > URL: https://issues.apache.org/jira/browse/HBASE-19773 > Project: HBase > Issue Type: Bug >Reporter: Appy >Assignee: Appy > Attachments: HBASE-19773.master.001.patch, > HBASE-19773.master.002.patch > > > Need to make RawCellBuilderFactory LP.Coproc so that CPs can make an instance > of RawCellBuilder type. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19773) Adding javadoc around getting instance of RawCellBuilder
[ https://issues.apache.org/jira/browse/HBASE-19773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-19773: - Priority: Minor (was: Major) > Adding javadoc around getting instance of RawCellBuilder > > > Key: HBASE-19773 > URL: https://issues.apache.org/jira/browse/HBASE-19773 > Project: HBase > Issue Type: Bug >Reporter: Appy >Assignee: Appy >Priority: Minor > Attachments: HBASE-19773.master.001.patch, > HBASE-19773.master.002.patch > > > Need to make RawCellBuilderFactory LP.Coproc so that CPs can make an instance > of RawCellBuilder type. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19773) RawCellBuilder is LP.Coproc, it's factory should have same InterfaceAudience
[ https://issues.apache.org/jira/browse/HBASE-19773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16322952#comment-16322952 ] Appy commented on HBASE-19773: -- Oh i see. Makes sense. It was good to keep it in RegionCpEnv! v2 adds documentation to make it easy to figure out. > RawCellBuilder is LP.Coproc, it's factory should have same InterfaceAudience > > > Key: HBASE-19773 > URL: https://issues.apache.org/jira/browse/HBASE-19773 > Project: HBase > Issue Type: Bug >Reporter: Appy >Assignee: Appy > Attachments: HBASE-19773.master.001.patch, > HBASE-19773.master.002.patch > > > Need to make RawCellBuilderFactory LP.Coproc so that CPs can make an instance > of RawCellBuilder type. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19773) RawCellBuilder is LP.Coproc, it's factory should have same InterfaceAudience
[ https://issues.apache.org/jira/browse/HBASE-19773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-19773: - Attachment: HBASE-19773.master.002.patch > RawCellBuilder is LP.Coproc, it's factory should have same InterfaceAudience > > > Key: HBASE-19773 > URL: https://issues.apache.org/jira/browse/HBASE-19773 > Project: HBase > Issue Type: Bug >Reporter: Appy >Assignee: Appy > Attachments: HBASE-19773.master.001.patch, > HBASE-19773.master.002.patch > > > Need to make RawCellBuilderFactory LP.Coproc so that CPs can make an instance > of RawCellBuilder type. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19773) RawCellBuilder is LP.Coproc, it's factory should have same InterfaceAudience
[
https://issues.apache.org/jira/browse/HBASE-19773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16322945#comment-16322945
]
stack commented on HBASE-19773:
---
[~appy] Idea is you get it from the CPEnv. Does that work for you Not
obvious.. How we fix that?
hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionCoprocessorEnvironment.java:
RawCellBuilder getCellBuilder();
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionCoprocessorHost.java:
public RawCellBuilder getCellBuilder() {
> RawCellBuilder is LP.Coproc, it's factory should have same InterfaceAudience
>
>
> Key: HBASE-19773
> URL: https://issues.apache.org/jira/browse/HBASE-19773
> Project: HBase
> Issue Type: Bug
>Reporter: Appy
>Assignee: Appy
> Attachments: HBASE-19773.master.001.patch
>
>
> Need to make RawCellBuilderFactory LP.Coproc so that CPs can make an instance
> of RawCellBuilder type.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19773) RawCellBuilder is LP.Coproc, it's factory should have same InterfaceAudience
[ https://issues.apache.org/jira/browse/HBASE-19773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16322944#comment-16322944 ] stack commented on HBASE-19773: --- [~chia7712] [~anoop.hbase] FYI > RawCellBuilder is LP.Coproc, it's factory should have same InterfaceAudience > > > Key: HBASE-19773 > URL: https://issues.apache.org/jira/browse/HBASE-19773 > Project: HBase > Issue Type: Bug >Reporter: Appy >Assignee: Appy > Attachments: HBASE-19773.master.001.patch > > > Need to make RawCellBuilderFactory LP.Coproc so that CPs can make an instance > of RawCellBuilder type. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19774) incorrect behavior of locateRegionInMeta
[
https://issues.apache.org/jira/browse/HBASE-19774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Elser updated HBASE-19774:
---
Reporter: Romil Choksi (was: Sergey Soldatov)
> incorrect behavior of locateRegionInMeta
>
>
> Key: HBASE-19774
> URL: https://issues.apache.org/jira/browse/HBASE-19774
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0-beta-1
>Reporter: Romil Choksi
>Assignee: Sergey Soldatov
> Fix For: 2.0.0-beta-1
>
>
> When we try to operate with not existing table, in some circumstances we get
> an incorrect report about the not existing table:
> {noformat}
> ERROR: Region of
> 'hbase:namespace,,1510363071508.0d8ddea7654f95130959218e9bc9c89c.' is
> expected in the table of 'nonExistentUsertable', but hbase:meta says it is in
> the table of 'hbase:namespace'. hbase:meta might be damaged.
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (HBASE-19774) incorrect behavior of locateRegionInMeta
Sergey Soldatov created HBASE-19774:
---
Summary: incorrect behavior of locateRegionInMeta
Key: HBASE-19774
URL: https://issues.apache.org/jira/browse/HBASE-19774
Project: HBase
Issue Type: Bug
Affects Versions: 2.0.0-beta-1
Reporter: Sergey Soldatov
Assignee: Sergey Soldatov
Fix For: 2.0.0-beta-1
When we try to operate with not existing table, in some circumstances we get an
incorrect report about the not existing table:
{noformat}
ERROR: Region of
'hbase:namespace,,1510363071508.0d8ddea7654f95130959218e9bc9c89c.' is expected
in the table of 'nonExistentUsertable', but hbase:meta says it is in the table
of 'hbase:namespace'. hbase:meta might be damaged.
{noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-19773) RawCellBuilder is LP.Coproc, it's factory should have same InterfaceAudience
[ https://issues.apache.org/jira/browse/HBASE-19773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-19773: - Description: Need to make RawCellBuilderFactory LP.Coproc so that CPs can make an instance of RawCellBuilder type. > RawCellBuilder is LP.Coproc, it's factory should have same InterfaceAudience > > > Key: HBASE-19773 > URL: https://issues.apache.org/jira/browse/HBASE-19773 > Project: HBase > Issue Type: Bug >Reporter: Appy >Assignee: Appy > Attachments: HBASE-19773.master.001.patch > > > Need to make RawCellBuilderFactory LP.Coproc so that CPs can make an instance > of RawCellBuilder type. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19773) RawCellBuilder is LP.Coproc, it's factory should have same InterfaceAudience
[ https://issues.apache.org/jira/browse/HBASE-19773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-19773: - Summary: RawCellBuilder is LP.Coproc, it's factory should have same InterfaceAudience (was: RawCellBuilder is LP.Coproc, it's factory should have save InterfaceAudience) > RawCellBuilder is LP.Coproc, it's factory should have same InterfaceAudience > > > Key: HBASE-19773 > URL: https://issues.apache.org/jira/browse/HBASE-19773 > Project: HBase > Issue Type: Bug >Reporter: Appy >Assignee: Appy > Attachments: HBASE-19773.master.001.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19773) RawCellBuilder is LP.Coproc, it's factory should have save InterfaceAudience
[ https://issues.apache.org/jira/browse/HBASE-19773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-19773: - Attachment: HBASE-19773.master.001.patch > RawCellBuilder is LP.Coproc, it's factory should have save InterfaceAudience > > > Key: HBASE-19773 > URL: https://issues.apache.org/jira/browse/HBASE-19773 > Project: HBase > Issue Type: Bug >Reporter: Appy >Assignee: Appy > Attachments: HBASE-19773.master.001.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19773) RawCellBuilder is LP.Coproc, it's factory should have save InterfaceAudience
Appy created HBASE-19773: Summary: RawCellBuilder is LP.Coproc, it's factory should have save InterfaceAudience Key: HBASE-19773 URL: https://issues.apache.org/jira/browse/HBASE-19773 Project: HBase Issue Type: Bug Reporter: Appy Assignee: Appy -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19658) Fix and reenable TestCompactingToCellFlatMapMemStore#testFlatteningToJumboCellChunkMap
[ https://issues.apache.org/jira/browse/HBASE-19658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16322895#comment-16322895 ] Jean-Marc Spaggiari commented on HBASE-19658: - This keeps failing for me with 2.0.0... > Fix and reenable > TestCompactingToCellFlatMapMemStore#testFlatteningToJumboCellChunkMap > -- > > Key: HBASE-19658 > URL: https://issues.apache.org/jira/browse/HBASE-19658 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 2.0.0-beta-1 >Reporter: stack >Assignee: Anastasia Braginsky > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19658-V01.patch, HBASE-19658-V02.patch, > HBASE-19658-V03.patch > > > testFlatteningToJumboCellChunkMap was disabled so could commit HBASE-19282 on > branch-2. This test is failing reliably. Assigned to [~anastas]. This issue > is about fixing the failing test and reenabling it in time for beta-2. Thanks > A. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19770) `list` shell command does not return an array of table names
[
https://issues.apache.org/jira/browse/HBASE-19770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Elser updated HBASE-19770:
---
Attachment: HBASE-19770.001.branch-2.patch
.001 Here's what I was thinking instead:
Creates a new option {{-r | --return-values}} which controls whether or not the
caller will get return values from shell commands.
This gives interactive shells the option to get return_values (my goal), but am
wondering if there is ever a reason that non-interactive shells would _not_
want return_values...
> `list` shell command does not return an array of table names
>
>
> Key: HBASE-19770
> URL: https://issues.apache.org/jira/browse/HBASE-19770
> Project: HBase
> Issue Type: Bug
> Components: shell
>Reporter: Romil Choksi
>Assignee: Josh Elser
>Priority: Critical
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19770.001.branch-2.patch
>
>
> Another good find by our Romil.
> {code}
> hbase(main):001:0> list
> TABLE
> a
> 1 row(s)
> Took 0.8385 seconds
> hbase(main):002:0> tables=list
> TABLE
> a
> 1 row(s)
> Took 0.0267 seconds
> hbase(main):003:0> puts tables
> hbase(main):004:0> p tables
> nil
> {code}
> The {{list}} command should be returning {{\['a'\]}} but is not.
> The command class itself appears to be doing the right thing -- maybe the
> retval is getting lost somewhere else?
> FYI [~stack].
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-19770) `list` shell command does not return an array of table names
[
https://issues.apache.org/jira/browse/HBASE-19770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Elser updated HBASE-19770:
---
Status: Patch Available (was: Open)
> `list` shell command does not return an array of table names
>
>
> Key: HBASE-19770
> URL: https://issues.apache.org/jira/browse/HBASE-19770
> Project: HBase
> Issue Type: Bug
> Components: shell
>Reporter: Romil Choksi
>Assignee: Josh Elser
>Priority: Critical
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19770.001.branch-2.patch
>
>
> Another good find by our Romil.
> {code}
> hbase(main):001:0> list
> TABLE
> a
> 1 row(s)
> Took 0.8385 seconds
> hbase(main):002:0> tables=list
> TABLE
> a
> 1 row(s)
> Took 0.0267 seconds
> hbase(main):003:0> puts tables
> hbase(main):004:0> p tables
> nil
> {code}
> The {{list}} command should be returning {{\['a'\]}} but is not.
> The command class itself appears to be doing the right thing -- maybe the
> retval is getting lost somewhere else?
> FYI [~stack].
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19694) The initialization order for a fresh cluster is incorrect
[
https://issues.apache.org/jira/browse/HBASE-19694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16322848#comment-16322848
]
stack commented on HBASE-19694:
---
.012 is .005 on top of the zk changes, HBASE-19753.
> The initialization order for a fresh cluster is incorrect
> -
>
> Key: HBASE-19694
> URL: https://issues.apache.org/jira/browse/HBASE-19694
> Project: HBase
> Issue Type: Bug
>Reporter: Duo Zhang
>Assignee: stack
>Priority: Critical
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-19694.branch-2.001.patch,
> HBASE-19694.branch-2.002.patch, HBASE-19694.branch-2.003.patch,
> HBASE-19694.branch-2.004.patch, HBASE-19694.branch-2.005.patch,
> HBASE-19694.branch-2.006.patch, HBASE-19694.branch-2.007.patch,
> HBASE-19694.branch-2.008.patch, HBASE-19694.branch-2.009.patch,
> HBASE-19694.branch-2.010.patch, HBASE-19694.branch-2.011.patch,
> HBASE-19694.branch-2.012.patch
>
>
> The cluster id will set once we become the active master in
> finishActiveMasterInitialization, but the blockUntilBecomingActiveMaster and
> finishActiveMasterInitialization are both called in a thread to make the
> constructor of HMaster return without blocking. And since HMaster itself is
> also a HRegionServer, it will create a Connection and then start calling
> reportForDuty. And when creating the ConnectionImplementation, we will read
> the cluster id from zk, but the cluster id may have not been set yet since it
> is set in another thread, we will get an exception and use the default
> cluster id instead.
> I always get this when running UTs which will start a mini cluster
> {noformat}
> 2018-01-03 15:16:37,916 WARN [M:0;zhangduo-ubuntu:32848]
> client.ConnectionImplementation(528): Retrieve cluster id failed
> java.util.concurrent.ExecutionException:
> org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode =
> NoNode for /hbase/hbaseid
> at
> java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
> at
> java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1895)
> at
> org.apache.hadoop.hbase.client.ConnectionImplementation.retrieveClusterId(ConnectionImplementation.java:526)
> at
> org.apache.hadoop.hbase.client.ConnectionImplementation.(ConnectionImplementation.java:286)
> at
> org.apache.hadoop.hbase.client.ConnectionUtils$ShortCircuitingClusterConnection.(ConnectionUtils.java:141)
> at
> org.apache.hadoop.hbase.client.ConnectionUtils$ShortCircuitingClusterConnection.(ConnectionUtils.java:137)
> at
> org.apache.hadoop.hbase.client.ConnectionUtils.createShortCircuitConnection(ConnectionUtils.java:185)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.createClusterConnection(HRegionServer.java:781)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.setupClusterConnection(HRegionServer.java:812)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.preRegistrationInitialization(HRegionServer.java:827)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:938)
> at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:550)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.zookeeper.KeeperException$NoNodeException:
> KeeperErrorCode = NoNode for /hbase/hbaseid
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> at
> org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$ZKTask$1.exec(ReadOnlyZKClient.java:163)
> at
> org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:311)
> ... 1 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-19694) The initialization order for a fresh cluster is incorrect
[
https://issues.apache.org/jira/browse/HBASE-19694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-19694:
--
Attachment: HBASE-19694.branch-2.012.patch
> The initialization order for a fresh cluster is incorrect
> -
>
> Key: HBASE-19694
> URL: https://issues.apache.org/jira/browse/HBASE-19694
> Project: HBase
> Issue Type: Bug
>Reporter: Duo Zhang
>Assignee: stack
>Priority: Critical
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-19694.branch-2.001.patch,
> HBASE-19694.branch-2.002.patch, HBASE-19694.branch-2.003.patch,
> HBASE-19694.branch-2.004.patch, HBASE-19694.branch-2.005.patch,
> HBASE-19694.branch-2.006.patch, HBASE-19694.branch-2.007.patch,
> HBASE-19694.branch-2.008.patch, HBASE-19694.branch-2.009.patch,
> HBASE-19694.branch-2.010.patch, HBASE-19694.branch-2.011.patch,
> HBASE-19694.branch-2.012.patch
>
>
> The cluster id will set once we become the active master in
> finishActiveMasterInitialization, but the blockUntilBecomingActiveMaster and
> finishActiveMasterInitialization are both called in a thread to make the
> constructor of HMaster return without blocking. And since HMaster itself is
> also a HRegionServer, it will create a Connection and then start calling
> reportForDuty. And when creating the ConnectionImplementation, we will read
> the cluster id from zk, but the cluster id may have not been set yet since it
> is set in another thread, we will get an exception and use the default
> cluster id instead.
> I always get this when running UTs which will start a mini cluster
> {noformat}
> 2018-01-03 15:16:37,916 WARN [M:0;zhangduo-ubuntu:32848]
> client.ConnectionImplementation(528): Retrieve cluster id failed
> java.util.concurrent.ExecutionException:
> org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode =
> NoNode for /hbase/hbaseid
> at
> java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
> at
> java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1895)
> at
> org.apache.hadoop.hbase.client.ConnectionImplementation.retrieveClusterId(ConnectionImplementation.java:526)
> at
> org.apache.hadoop.hbase.client.ConnectionImplementation.(ConnectionImplementation.java:286)
> at
> org.apache.hadoop.hbase.client.ConnectionUtils$ShortCircuitingClusterConnection.(ConnectionUtils.java:141)
> at
> org.apache.hadoop.hbase.client.ConnectionUtils$ShortCircuitingClusterConnection.(ConnectionUtils.java:137)
> at
> org.apache.hadoop.hbase.client.ConnectionUtils.createShortCircuitConnection(ConnectionUtils.java:185)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.createClusterConnection(HRegionServer.java:781)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.setupClusterConnection(HRegionServer.java:812)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.preRegistrationInitialization(HRegionServer.java:827)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:938)
> at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:550)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.zookeeper.KeeperException$NoNodeException:
> KeeperErrorCode = NoNode for /hbase/hbaseid
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> at
> org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$ZKTask$1.exec(ReadOnlyZKClient.java:163)
> at
> org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:311)
> ... 1 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-19694) The initialization order for a fresh cluster is incorrect
[
https://issues.apache.org/jira/browse/HBASE-19694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-19694:
--
Attachment: HBASE-19694.branch-2.011.patch
> The initialization order for a fresh cluster is incorrect
> -
>
> Key: HBASE-19694
> URL: https://issues.apache.org/jira/browse/HBASE-19694
> Project: HBase
> Issue Type: Bug
>Reporter: Duo Zhang
>Assignee: stack
>Priority: Critical
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-19694.branch-2.001.patch,
> HBASE-19694.branch-2.002.patch, HBASE-19694.branch-2.003.patch,
> HBASE-19694.branch-2.004.patch, HBASE-19694.branch-2.005.patch,
> HBASE-19694.branch-2.006.patch, HBASE-19694.branch-2.007.patch,
> HBASE-19694.branch-2.008.patch, HBASE-19694.branch-2.009.patch,
> HBASE-19694.branch-2.010.patch, HBASE-19694.branch-2.011.patch,
> HBASE-19694.branch-2.012.patch
>
>
> The cluster id will set once we become the active master in
> finishActiveMasterInitialization, but the blockUntilBecomingActiveMaster and
> finishActiveMasterInitialization are both called in a thread to make the
> constructor of HMaster return without blocking. And since HMaster itself is
> also a HRegionServer, it will create a Connection and then start calling
> reportForDuty. And when creating the ConnectionImplementation, we will read
> the cluster id from zk, but the cluster id may have not been set yet since it
> is set in another thread, we will get an exception and use the default
> cluster id instead.
> I always get this when running UTs which will start a mini cluster
> {noformat}
> 2018-01-03 15:16:37,916 WARN [M:0;zhangduo-ubuntu:32848]
> client.ConnectionImplementation(528): Retrieve cluster id failed
> java.util.concurrent.ExecutionException:
> org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode =
> NoNode for /hbase/hbaseid
> at
> java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
> at
> java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1895)
> at
> org.apache.hadoop.hbase.client.ConnectionImplementation.retrieveClusterId(ConnectionImplementation.java:526)
> at
> org.apache.hadoop.hbase.client.ConnectionImplementation.(ConnectionImplementation.java:286)
> at
> org.apache.hadoop.hbase.client.ConnectionUtils$ShortCircuitingClusterConnection.(ConnectionUtils.java:141)
> at
> org.apache.hadoop.hbase.client.ConnectionUtils$ShortCircuitingClusterConnection.(ConnectionUtils.java:137)
> at
> org.apache.hadoop.hbase.client.ConnectionUtils.createShortCircuitConnection(ConnectionUtils.java:185)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.createClusterConnection(HRegionServer.java:781)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.setupClusterConnection(HRegionServer.java:812)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.preRegistrationInitialization(HRegionServer.java:827)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:938)
> at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:550)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.zookeeper.KeeperException$NoNodeException:
> KeeperErrorCode = NoNode for /hbase/hbaseid
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> at
> org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$ZKTask$1.exec(ReadOnlyZKClient.java:163)
> at
> org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:311)
> ... 1 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-19758) Split TestHCM to several smaller tests
[ https://issues.apache.org/jira/browse/HBASE-19758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-19758: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.0.0-beta-1 Status: Resolved (was: Patch Available) Pushed this nice refactoring to branch-2 and master. > Split TestHCM to several smaller tests > -- > > Key: HBASE-19758 > URL: https://issues.apache.org/jira/browse/HBASE-19758 > Project: HBase > Issue Type: Improvement > Components: test >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19758.patch, HBASE-19758.patch > > > It takes too much time to finish, especially the timeout related tests. We > should move these tests out. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19772) ReadOnlyZKClient improvements
[
https://issues.apache.org/jira/browse/HBASE-19772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16322834#comment-16322834
]
Hadoop QA commented on HBASE-19772:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 7s{color}
| {color:red} HBASE-19772 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.6.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HBASE-19772 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12905733/HBASE-19772.master.001.patch
|
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/11025/console |
| Powered by | Apache Yetus 0.6.0 http://yetus.apache.org |
This message was automatically generated.
> ReadOnlyZKClient improvements
> -
>
> Key: HBASE-19772
> URL: https://issues.apache.org/jira/browse/HBASE-19772
> Project: HBase
> Issue Type: Sub-task
> Components: Zookeeper
>Reporter: stack
>Assignee: Duo Zhang
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-19772.master.001.patch
>
>
> Here is [~Apache9] 's patch from the parent so it applies on top of what was
> committed in the parent.
> Patch makes it so we we don't close out zk if available Tasks to run and
> nicer logging.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19772) ReadOnlyZKClient improvements
[
https://issues.apache.org/jira/browse/HBASE-19772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16322833#comment-16322833
]
Jean-Marc Spaggiari commented on HBASE-19772:
-
Are you talking about this?
{code}
18/01/11 14:39:45 INFO zookeeper.ReadOnlyZKClient: 0x4685fae6 no activities for
6 ms, close active connection. Will reconnect next time when there are new
requests.
{code}
Will be happy to see them going away ;) Maybe debug?
> ReadOnlyZKClient improvements
> -
>
> Key: HBASE-19772
> URL: https://issues.apache.org/jira/browse/HBASE-19772
> Project: HBase
> Issue Type: Sub-task
> Components: Zookeeper
>Reporter: stack
>Assignee: Duo Zhang
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-19772.master.001.patch
>
>
> Here is [~Apache9] 's patch from the parent so it applies on top of what was
> committed in the parent.
> Patch makes it so we we don't close out zk if available Tasks to run and
> nicer logging.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-19753) Miscellany of fixes for hbase-zookeeper tests to make them more robust
[ https://issues.apache.org/jira/browse/HBASE-19753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-19753: -- Release Note: When we create a ZooKeeper instance, wait on connection before using it. > Miscellany of fixes for hbase-zookeeper tests to make them more robust > -- > > Key: HBASE-19753 > URL: https://issues.apache.org/jira/browse/HBASE-19753 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack > Attachments: > 0001-HBASE-19753-Miscellany-of-fixes-for-hbase-zookeeper-.patch, > HBASE-19753.branch-2.001.patch, HBASE-19753.branch-2.002.patch, > HBASE-19753.branch-2.003.patch, HBASE-19753.branch-2.004.patch, > HBASE-19753.branch-2.005.patch, HBASE-19753.branch-2.006.patch, > HBASE-19753.branch-2.007.patch, HBASE-19753.branch-2.008.patch, keepalive.diff > > > On my cluster which slows zk, tests hbase-zookeeper rarely all pass. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19772) ReadOnlyZKClient improvements
[ https://issues.apache.org/jira/browse/HBASE-19772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-19772: -- Status: Patch Available (was: Open) > ReadOnlyZKClient improvements > - > > Key: HBASE-19772 > URL: https://issues.apache.org/jira/browse/HBASE-19772 > Project: HBase > Issue Type: Sub-task > Components: Zookeeper >Reporter: stack >Assignee: Duo Zhang > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19772.master.001.patch > > > Here is [~Apache9] 's patch from the parent so it applies on top of what was > committed in the parent. > Patch makes it so we we don't close out zk if available Tasks to run and > nicer logging. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19772) ReadOnlyZKClient improvements
[ https://issues.apache.org/jira/browse/HBASE-19772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-19772: -- Attachment: HBASE-19772.master.001.patch > ReadOnlyZKClient improvements > - > > Key: HBASE-19772 > URL: https://issues.apache.org/jira/browse/HBASE-19772 > Project: HBase > Issue Type: Sub-task > Components: Zookeeper >Reporter: stack >Assignee: Duo Zhang > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19772.master.001.patch > > > Here is [~Apache9] 's patch from the parent so it applies on top of what was > committed in the parent. > Patch makes it so we we don't close out zk if available Tasks to run and > nicer logging. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19772) ReadOnlyZKClient improvements
[ https://issues.apache.org/jira/browse/HBASE-19772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-19772: -- Issue Type: Sub-task (was: Bug) Parent: HBASE-19753 > ReadOnlyZKClient improvements > - > > Key: HBASE-19772 > URL: https://issues.apache.org/jira/browse/HBASE-19772 > Project: HBase > Issue Type: Sub-task > Components: Zookeeper >Reporter: stack >Assignee: Duo Zhang > Fix For: 2.0.0-beta-1 > > > Here is [~Apache9] 's patch from the parent so it applies on top of what was > committed in the parent. > Patch makes it so we we don't close out zk if available Tasks to run and > nicer logging. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19772) ReadOnlyZKClient improvements
[ https://issues.apache.org/jira/browse/HBASE-19772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-19772: -- Issue Type: Bug (was: Sub-task) Parent: (was: HBASE-19694) > ReadOnlyZKClient improvements > - > > Key: HBASE-19772 > URL: https://issues.apache.org/jira/browse/HBASE-19772 > Project: HBase > Issue Type: Bug > Components: Zookeeper >Reporter: stack >Assignee: Duo Zhang > Fix For: 2.0.0-beta-1 > > > Here is [~Apache9] 's patch from the parent so it applies on top of what was > committed in the parent. > Patch makes it so we we don't close out zk if available Tasks to run and > nicer logging. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19772) ReadOnlyZKClient improvements
stack created HBASE-19772: - Summary: ReadOnlyZKClient improvements Key: HBASE-19772 URL: https://issues.apache.org/jira/browse/HBASE-19772 Project: HBase Issue Type: Sub-task Components: Zookeeper Reporter: stack Assignee: Duo Zhang Here is [~Apache9] 's patch from the parent so it applies on top of what was committed in the parent. Patch makes it so we we don't close out zk if available Tasks to run and nicer logging. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19753) Miscellany of fixes for hbase-zookeeper tests to make them more robust
[
https://issues.apache.org/jira/browse/HBASE-19753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16322816#comment-16322816
]
Hadoop QA commented on HBASE-19753:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 5s{color}
| {color:red} HBASE-19753 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.6.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HBASE-19753 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12905728/0001-HBASE-19753-Miscellany-of-fixes-for-hbase-zookeeper-.patch
|
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/11024/console |
| Powered by | Apache Yetus 0.6.0 http://yetus.apache.org |
This message was automatically generated.
> Miscellany of fixes for hbase-zookeeper tests to make them more robust
> --
>
> Key: HBASE-19753
> URL: https://issues.apache.org/jira/browse/HBASE-19753
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: stack
> Attachments:
> 0001-HBASE-19753-Miscellany-of-fixes-for-hbase-zookeeper-.patch,
> HBASE-19753.branch-2.001.patch, HBASE-19753.branch-2.002.patch,
> HBASE-19753.branch-2.003.patch, HBASE-19753.branch-2.004.patch,
> HBASE-19753.branch-2.005.patch, HBASE-19753.branch-2.006.patch,
> HBASE-19753.branch-2.007.patch, HBASE-19753.branch-2.008.patch, keepalive.diff
>
>
> On my cluster which slows zk, tests hbase-zookeeper rarely all pass.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19770) `list` shell command does not return an array of table names
[
https://issues.apache.org/jira/browse/HBASE-19770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16322814#comment-16322814
]
Josh Elser commented on HBASE-19770:
bq. I'm not sure I'd agree with the assertion that "all interactive shells do
not want return values".
Thinking some more on this: this change forces users into having their script
fail on the first error. There's no option for me to say "Please give me
command return values, but don't kill my shell session on command error."
> `list` shell command does not return an array of table names
>
>
> Key: HBASE-19770
> URL: https://issues.apache.org/jira/browse/HBASE-19770
> Project: HBase
> Issue Type: Bug
> Components: shell
>Reporter: Romil Choksi
>Assignee: Josh Elser
>Priority: Critical
> Fix For: 2.0.0-beta-2
>
>
> Another good find by our Romil.
> {code}
> hbase(main):001:0> list
> TABLE
> a
> 1 row(s)
> Took 0.8385 seconds
> hbase(main):002:0> tables=list
> TABLE
> a
> 1 row(s)
> Took 0.0267 seconds
> hbase(main):003:0> puts tables
> hbase(main):004:0> p tables
> nil
> {code}
> The {{list}} command should be returning {{\['a'\]}} but is not.
> The command class itself appears to be doing the right thing -- maybe the
> retval is getting lost somewhere else?
> FYI [~stack].
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19770) `list` shell command does not return an array of table names
[
https://issues.apache.org/jira/browse/HBASE-19770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16322800#comment-16322800
]
Josh Elser commented on HBASE-19770:
Looks like this one is caused by HBASE-15965. FYI [~appy].
I can understand wanting to prevent IRB from printing the object from the
command (when the formatter printed it nicely), but are we sure we want to do
this by default? Caught me by surprise. I'm not sure I'd agree with the
assertion that "all interactive shells do not want return values".
> `list` shell command does not return an array of table names
>
>
> Key: HBASE-19770
> URL: https://issues.apache.org/jira/browse/HBASE-19770
> Project: HBase
> Issue Type: Bug
> Components: shell
>Reporter: Romil Choksi
>Assignee: Josh Elser
>Priority: Critical
> Fix For: 2.0.0-beta-2
>
>
> Another good find by our Romil.
> {code}
> hbase(main):001:0> list
> TABLE
> a
> 1 row(s)
> Took 0.8385 seconds
> hbase(main):002:0> tables=list
> TABLE
> a
> 1 row(s)
> Took 0.0267 seconds
> hbase(main):003:0> puts tables
> hbase(main):004:0> p tables
> nil
> {code}
> The {{list}} command should be returning {{\['a'\]}} but is not.
> The command class itself appears to be doing the right thing -- maybe the
> retval is getting lost somewhere else?
> FYI [~stack].
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-19753) Miscellany of fixes for hbase-zookeeper tests to make them more robust
[ https://issues.apache.org/jira/browse/HBASE-19753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-19753: -- Attachment: 0001-HBASE-19753-Miscellany-of-fixes-for-hbase-zookeeper-.patch What I pushed which is .007 with checkstyle fixes. Let me open issue to do keepalive. > Miscellany of fixes for hbase-zookeeper tests to make them more robust > -- > > Key: HBASE-19753 > URL: https://issues.apache.org/jira/browse/HBASE-19753 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack > Attachments: > 0001-HBASE-19753-Miscellany-of-fixes-for-hbase-zookeeper-.patch, > HBASE-19753.branch-2.001.patch, HBASE-19753.branch-2.002.patch, > HBASE-19753.branch-2.003.patch, HBASE-19753.branch-2.004.patch, > HBASE-19753.branch-2.005.patch, HBASE-19753.branch-2.006.patch, > HBASE-19753.branch-2.007.patch, HBASE-19753.branch-2.008.patch, keepalive.diff > > > On my cluster which slows zk, tests hbase-zookeeper rarely all pass. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-11409) Add more flexibility for input directory structure to LoadIncrementalHFiles
[ https://issues.apache.org/jira/browse/HBASE-11409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] churro morales updated HBASE-11409: --- Status: Patch Available (was: Reopened) > Add more flexibility for input directory structure to LoadIncrementalHFiles > --- > > Key: HBASE-11409 > URL: https://issues.apache.org/jira/browse/HBASE-11409 > Project: HBase > Issue Type: Bug >Affects Versions: 3.0.0 >Reporter: churro morales >Assignee: churro morales > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-11409.v1.patch, HBASE-11409.v2.patch, > HBASE-11409.v3.patch, HBASE-11409.v4.patch, HBASE-11409.v5.patch, > HBASE-11409.v6.branch-1.patch > > > Use case: > We were trying to combine two very large tables into a single table. Thus we > ran jobs in one datacenter that populated certain column families and another > datacenter which populated other column families. Took a snapshot and > exported them to their respective datacenters. Wanted to simply take the > hdfs restored snapshot and use LoadIncremental to merge the data. > It would be nice to add support where we could run LoadIncremental on a > directory where the depth of store files is something other than two (current > behavior). > With snapshots it would be nice if you could pass a restored hdfs snapshot's > directory and have the tool run. > I am attaching a patch where I parameterize the bulkLoad timeout as well as > the default store file depth. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-11409) Add more flexibility for input directory structure to LoadIncrementalHFiles
[ https://issues.apache.org/jira/browse/HBASE-11409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] churro morales updated HBASE-11409: --- Attachment: HBASE-11409.v6.branch-1.patch For the branch-1 backport: Because the IA was Public, I couldn't change API and LoadIncrementalHFiles (branch-1) is a quite different from branch-2 and master. Let me know if this works for you guys. > Add more flexibility for input directory structure to LoadIncrementalHFiles > --- > > Key: HBASE-11409 > URL: https://issues.apache.org/jira/browse/HBASE-11409 > Project: HBase > Issue Type: Bug >Affects Versions: 3.0.0 >Reporter: churro morales >Assignee: churro morales > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-11409.v1.patch, HBASE-11409.v2.patch, > HBASE-11409.v3.patch, HBASE-11409.v4.patch, HBASE-11409.v5.patch, > HBASE-11409.v6.branch-1.patch > > > Use case: > We were trying to combine two very large tables into a single table. Thus we > ran jobs in one datacenter that populated certain column families and another > datacenter which populated other column families. Took a snapshot and > exported them to their respective datacenters. Wanted to simply take the > hdfs restored snapshot and use LoadIncremental to merge the data. > It would be nice to add support where we could run LoadIncremental on a > directory where the depth of store files is something other than two (current > behavior). > With snapshots it would be nice if you could pass a restored hdfs snapshot's > directory and have the tool run. > I am attaching a patch where I parameterize the bulkLoad timeout as well as > the default store file depth. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Reopened] (HBASE-11409) Add more flexibility for input directory structure to LoadIncrementalHFiles
[ https://issues.apache.org/jira/browse/HBASE-11409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] churro morales reopened HBASE-11409: > Add more flexibility for input directory structure to LoadIncrementalHFiles > --- > > Key: HBASE-11409 > URL: https://issues.apache.org/jira/browse/HBASE-11409 > Project: HBase > Issue Type: Bug >Affects Versions: 3.0.0 >Reporter: churro morales >Assignee: churro morales > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-11409.v1.patch, HBASE-11409.v2.patch, > HBASE-11409.v3.patch, HBASE-11409.v4.patch, HBASE-11409.v5.patch, > HBASE-11409.v6.branch-1.patch > > > Use case: > We were trying to combine two very large tables into a single table. Thus we > ran jobs in one datacenter that populated certain column families and another > datacenter which populated other column families. Took a snapshot and > exported them to their respective datacenters. Wanted to simply take the > hdfs restored snapshot and use LoadIncremental to merge the data. > It would be nice to add support where we could run LoadIncremental on a > directory where the depth of store files is something other than two (current > behavior). > With snapshots it would be nice if you could pass a restored hdfs snapshot's > directory and have the tool run. > I am attaching a patch where I parameterize the bulkLoad timeout as well as > the default store file depth. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (HBASE-19754) Backport HBASE-11409 to branch-1 and branch-1.4
[ https://issues.apache.org/jira/browse/HBASE-19754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] churro morales resolved HBASE-19754. Resolution: Fixed Moving this ticket back to the original HBASE-11409 for the branch-1 backport > Backport HBASE-11409 to branch-1 and branch-1.4 > --- > > Key: HBASE-19754 > URL: https://issues.apache.org/jira/browse/HBASE-19754 > Project: HBase > Issue Type: Bug >Affects Versions: 1.4.0, 1.5.0 >Reporter: churro morales >Assignee: churro morales >Priority: Minor > Attachments: HBASE-19754.branch-1.patch > > > backport HBASE-11409 to branch-1, branch-1.4 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19754) Backport HBASE-11409 to branch-1 and branch-1.4
[ https://issues.apache.org/jira/browse/HBASE-19754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] churro morales updated HBASE-19754: --- Status: Open (was: Patch Available) > Backport HBASE-11409 to branch-1 and branch-1.4 > --- > > Key: HBASE-19754 > URL: https://issues.apache.org/jira/browse/HBASE-19754 > Project: HBase > Issue Type: Bug >Affects Versions: 1.4.0, 1.5.0 >Reporter: churro morales >Assignee: churro morales >Priority: Minor > Attachments: HBASE-19754.branch-1.patch > > > backport HBASE-11409 to branch-1, branch-1.4 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-15321) Ability to open a HRegion from hdfs snapshot.
[ https://issues.apache.org/jira/browse/HBASE-15321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16322772#comment-16322772 ] churro morales commented on HBASE-15321: [~stack] [~enis] [~anoopamz] you guys good with the latest patch? If you guys are good with it I can go ahead and commit to master. > Ability to open a HRegion from hdfs snapshot. > - > > Key: HBASE-15321 > URL: https://issues.apache.org/jira/browse/HBASE-15321 > Project: HBase > Issue Type: New Feature >Affects Versions: 2.0.0, 3.0.0, 1.5.0 >Reporter: churro morales >Assignee: churro morales > Attachments: HBASE-15321.patch, HBASE-15321.v01.patch, > HBASE-15321.v02.patch > > > Now that hdfs snapshots are here, we started to run our mapreduce jobs over > hdfs snapshots. The thing is, hdfs snapshots are read-only point-in-time > copies of the file system. Thus we had to modify the section of code that > initialized the region internals in HRegion. We have to skip cleanup of > certain directories if the HRegion is backed by a hdfs snapshot. I have a > patch for trunk with some basic tests if folks are interested. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-11409) Add more flexibility for input directory structure to LoadIncrementalHFiles
[ https://issues.apache.org/jira/browse/HBASE-11409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16322769#comment-16322769 ] churro morales commented on HBASE-11409: @TedYu sorry about that, reverted and re-committed with the proper commit message for master and branch-2 > Add more flexibility for input directory structure to LoadIncrementalHFiles > --- > > Key: HBASE-11409 > URL: https://issues.apache.org/jira/browse/HBASE-11409 > Project: HBase > Issue Type: Bug >Affects Versions: 3.0.0 >Reporter: churro morales >Assignee: churro morales > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-11409.v1.patch, HBASE-11409.v2.patch, > HBASE-11409.v3.patch, HBASE-11409.v4.patch, HBASE-11409.v5.patch > > > Use case: > We were trying to combine two very large tables into a single table. Thus we > ran jobs in one datacenter that populated certain column families and another > datacenter which populated other column families. Took a snapshot and > exported them to their respective datacenters. Wanted to simply take the > hdfs restored snapshot and use LoadIncremental to merge the data. > It would be nice to add support where we could run LoadIncremental on a > directory where the depth of store files is something other than two (current > behavior). > With snapshots it would be nice if you could pass a restored hdfs snapshot's > directory and have the tool run. > I am attaching a patch where I parameterize the bulkLoad timeout as well as > the default store file depth. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19694) The initialization order for a fresh cluster is incorrect
[
https://issues.apache.org/jira/browse/HBASE-19694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16322757#comment-16322757
]
Hadoop QA commented on HBASE-19694:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m
52s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 6 new or modified test
files. {color} |
|| || || || {color:brown} branch-2 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
10s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
33s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
13s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
45s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
42s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
2s{color} | {color:green} branch-2 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
11s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
18s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
27s{color} | {color:red} hbase-client: The patch generated 4 new + 0 unchanged
- 0 fixed = 4 total (was 0) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
2s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
15m 57s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
50s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
35s{color} | {color:green} hbase-zookeeper in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}101m 25s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
48s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}143m 44s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:9f2f2db |
| JIRA Issue | HBASE-19694 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12905705/HBASE-19694.branch-2.010.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile xml |
| uname | Linux a351b09cbf90 3.13.0-129-generic #178-Ubuntu SMP Fri Aug 11
12:48:20 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins
[jira] [Resolved] (HBASE-19483) Add proper privilege check for rsgroup commands
[ https://issues.apache.org/jira/browse/HBASE-19483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu resolved HBASE-19483. Resolution: Fixed Addendum integrated to branch-1.4 and branch-1 Thanks Guangxu. > Add proper privilege check for rsgroup commands > --- > > Key: HBASE-19483 > URL: https://issues.apache.org/jira/browse/HBASE-19483 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Guangxu Cheng > Fix For: 1.4.1, 1.5.0, 2.0.0-beta-1 > > Attachments: 19483.master.011.patch, 19483.v11.patch, > 19483.v11.patch, HBASE-19483.addendum-1.patch, HBASE-19483.addendum.patch, > HBASE-19483.branch-1.001.patch, HBASE-19483.branch-1.addendum.patch, > HBASE-19483.branch-2.001.patch, HBASE-19483.branch-2.002.patch, > HBASE-19483.branch-2.003.patch, HBASE-19483.master.001.patch, > HBASE-19483.master.002.patch, HBASE-19483.master.003.patch, > HBASE-19483.master.004.patch, HBASE-19483.master.005.patch, > HBASE-19483.master.006.patch, HBASE-19483.master.007.patch, > HBASE-19483.master.008.patch, HBASE-19483.master.009.patch, > HBASE-19483.master.010.patch, HBASE-19483.master.011.patch, > HBASE-19483.master.011.patch, HBASE-19483.master.012.patch, > HBASE-19483.master.013.patch, HBASE-19483.master.014.patch > > > Currently list_rsgroups command can be executed by any user. > This is inconsistent with other list commands such as list_peers and > list_peer_configs. > We should add proper privilege check for list_rsgroups command. > privilege check should be added for get_table_rsgroup / get_server_rsgroup / > get_rsgroup commands. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19685) Fix TestFSErrorsExposed#testFullSystemBubblesFSErrors
[
https://issues.apache.org/jira/browse/HBASE-19685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16322750#comment-16322750
]
Hudson commented on HBASE-19685:
SUCCESS: Integrated in Jenkins build HBase-1.3-IT #334 (See
[https://builds.apache.org/job/HBase-1.3-IT/334/])
HBASE-19685 Fix TestFSErrorsExposed#testFullSystemBubblesFSErrors by (appy: rev
bccc706d63905a65f90a2f71f8e2ea6be6f0efa0)
* (edit)
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestFSErrorsExposed.java
> Fix TestFSErrorsExposed#testFullSystemBubblesFSErrors
> -
>
> Key: HBASE-19685
> URL: https://issues.apache.org/jira/browse/HBASE-19685
> Project: HBase
> Issue Type: Bug
> Components: test
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
> Fix For: 1.3.2, 1.4.1, 1.5.0, 1.2.7, 2.0.0-beta-2
>
> Attachments: HBASE-19685.v0.patch
>
>
> {code}
> java.lang.AssertionError
> at
> org.apache.hadoop.hbase.regionserver.TestFSErrorsExposed.testFullSystemBubblesFSErrors(TestFSErrorsExposed.java:221)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19685) Fix TestFSErrorsExposed#testFullSystemBubblesFSErrors
[
https://issues.apache.org/jira/browse/HBASE-19685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16322744#comment-16322744
]
Hudson commented on HBASE-19685:
SUCCESS: Integrated in Jenkins build HBase-1.2-IT #1058 (See
[https://builds.apache.org/job/HBase-1.2-IT/1058/])
HBASE-19685 Fix TestFSErrorsExposed#testFullSystemBubblesFSErrors by (appy: rev
af41979bb4bbfa223cf06ea614bf926855891ad5)
* (edit)
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestFSErrorsExposed.java
> Fix TestFSErrorsExposed#testFullSystemBubblesFSErrors
> -
>
> Key: HBASE-19685
> URL: https://issues.apache.org/jira/browse/HBASE-19685
> Project: HBase
> Issue Type: Bug
> Components: test
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
> Fix For: 1.3.2, 1.4.1, 1.5.0, 1.2.7, 2.0.0-beta-2
>
> Attachments: HBASE-19685.v0.patch
>
>
> {code}
> java.lang.AssertionError
> at
> org.apache.hadoop.hbase.regionserver.TestFSErrorsExposed.testFullSystemBubblesFSErrors(TestFSErrorsExposed.java:221)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-19771) restore_snapshot shell command gives wrong namespace if the namespace doesn't exist
[
https://issues.apache.org/jira/browse/HBASE-19771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-19771:
---
Affects Version/s: 2.0.0-beta-1
> restore_snapshot shell command gives wrong namespace if the namespace doesn't
> exist
> ---
>
> Key: HBASE-19771
> URL: https://issues.apache.org/jira/browse/HBASE-19771
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0-beta-1
>Reporter: Ted Yu
>Priority: Minor
>
> For the following snapshot:
> {code}
> snapshot_table_tav6mrq397
> NS592088:table_tav6mrq397 (2018-01-10 16:20:39 UTC)
> {code}
> namespace NS592088 is dropped.
> restore_snapshot on the snapshot gave the following :
> {code}
> hbase(main):003:0> restore_snapshot 'snapshot_table_tav6mrq397'
> ERROR: Unknown namespace snapshot_table_tav6mrq397!
> {code}
> The namespace in the message was incorrect.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (HBASE-19771) restore_snapshot shell command gives wrong namespace if the namespace doesn't exist
Ted Yu created HBASE-19771:
--
Summary: restore_snapshot shell command gives wrong namespace if
the namespace doesn't exist
Key: HBASE-19771
URL: https://issues.apache.org/jira/browse/HBASE-19771
Project: HBase
Issue Type: Bug
Reporter: Ted Yu
Priority: Minor
For the following snapshot:
{code}
snapshot_table_tav6mrq397
NS592088:table_tav6mrq397 (2018-01-10 16:20:39 UTC)
{code}
namespace NS592088 is dropped.
restore_snapshot on the snapshot gave the following :
{code}
hbase(main):003:0> restore_snapshot 'snapshot_table_tav6mrq397'
ERROR: Unknown namespace snapshot_table_tav6mrq397!
{code}
The namespace in the message was incorrect.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-19685) Fix TestFSErrorsExposed#testFullSystemBubblesFSErrors
[
https://issues.apache.org/jira/browse/HBASE-19685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Appy updated HBASE-19685:
-
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Fix TestFSErrorsExposed#testFullSystemBubblesFSErrors
> -
>
> Key: HBASE-19685
> URL: https://issues.apache.org/jira/browse/HBASE-19685
> Project: HBase
> Issue Type: Bug
> Components: test
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
> Fix For: 1.3.2, 1.4.1, 1.5.0, 1.2.7, 2.0.0-beta-2
>
> Attachments: HBASE-19685.v0.patch
>
>
> {code}
> java.lang.AssertionError
> at
> org.apache.hadoop.hbase.regionserver.TestFSErrorsExposed.testFullSystemBubblesFSErrors(TestFSErrorsExposed.java:221)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-19685) Fix TestFSErrorsExposed#testFullSystemBubblesFSErrors
[
https://issues.apache.org/jira/browse/HBASE-19685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Appy updated HBASE-19685:
-
Fix Version/s: 1.2.7
1.5.0
1.4.1
1.3.2
> Fix TestFSErrorsExposed#testFullSystemBubblesFSErrors
> -
>
> Key: HBASE-19685
> URL: https://issues.apache.org/jira/browse/HBASE-19685
> Project: HBase
> Issue Type: Bug
> Components: test
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
> Fix For: 1.3.2, 1.4.1, 1.5.0, 1.2.7, 2.0.0-beta-2
>
> Attachments: HBASE-19685.v0.patch
>
>
> {code}
> java.lang.AssertionError
> at
> org.apache.hadoop.hbase.regionserver.TestFSErrorsExposed.testFullSystemBubblesFSErrors(TestFSErrorsExposed.java:221)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19163) "Maximum lock count exceeded" from region server's batch processing
[
https://issues.apache.org/jira/browse/HBASE-19163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16322700#comment-16322700
]
huaxiang sun commented on HBASE-19163:
--
Thanks [~uagashe]! Given [~saint@gmail.com] already gave +1 on the patch
for the master, I am going to push this to branch-1.
> "Maximum lock count exceeded" from region server's batch processing
> ---
>
> Key: HBASE-19163
> URL: https://issues.apache.org/jira/browse/HBASE-19163
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 3.0.0, 1.2.7, 2.0.0-alpha-3
>Reporter: huaxiang sun
>Assignee: huaxiang sun
> Attachments: HBASE-19163-branch-1-v001.patch,
> HBASE-19163-branch-1-v001.patch, HBASE-19163-master-v001.patch,
> HBASE-19163.master.001.patch, HBASE-19163.master.002.patch,
> HBASE-19163.master.004.patch, HBASE-19163.master.005.patch,
> HBASE-19163.master.006.patch, HBASE-19163.master.007.patch,
> HBASE-19163.master.008.patch, HBASE-19163.master.009.patch,
> HBASE-19163.master.009.patch, HBASE-19163.master.010.patch, unittest-case.diff
>
>
> In one of use cases, we found the following exception and replication is
> stuck.
> {code}
> 2017-10-25 19:41:17,199 WARN [hconnection-0x28db294f-shared--pool4-t936]
> client.AsyncProcess: #3, table=foo, attempt=5/5 failed=262836ops, last
> exception: java.io.IOException: java.io.IOException: Maximum lock count
> exceeded
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2215)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:109)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:185)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:165)
> Caused by: java.lang.Error: Maximum lock count exceeded
> at
> java.util.concurrent.locks.ReentrantReadWriteLock$Sync.fullTryAcquireShared(ReentrantReadWriteLock.java:528)
> at
> java.util.concurrent.locks.ReentrantReadWriteLock$Sync.tryAcquireShared(ReentrantReadWriteLock.java:488)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1327)
> at
> java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.tryLock(ReentrantReadWriteLock.java:871)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.getRowLock(HRegion.java:5163)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutation(HRegion.java:3018)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2877)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2819)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:753)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:715)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2148)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:33656)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2170)
> ... 3 more
> {code}
> While we are still examining the data pattern, it is sure that there are too
> many mutations in the batch against the same row, this exceeds the maximum
> 64k shared lock count and it throws an error and failed the whole batch.
> There are two approaches to solve this issue.
> 1). Let's say there are mutations against the same row in the batch, we just
> need to acquire the lock once for the same row vs to acquire the lock for
> each mutation.
> 2). We catch the error and start to process whatever it gets and loop back.
> With HBASE-17924, approach 1 seems easy to implement now.
> Create the jira and will post update/patch when investigation moving forward.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19753) Miscellany of fixes for hbase-zookeeper tests to make them more robust
[ https://issues.apache.org/jira/browse/HBASE-19753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16322699#comment-16322699 ] stack commented on HBASE-19753: --- Ok. Compound patch is hanging on my dodgy hardware... its a hard hang so I'm going to punt on bundling the keepalive for now. > Miscellany of fixes for hbase-zookeeper tests to make them more robust > -- > > Key: HBASE-19753 > URL: https://issues.apache.org/jira/browse/HBASE-19753 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack > Attachments: HBASE-19753.branch-2.001.patch, > HBASE-19753.branch-2.002.patch, HBASE-19753.branch-2.003.patch, > HBASE-19753.branch-2.004.patch, HBASE-19753.branch-2.005.patch, > HBASE-19753.branch-2.006.patch, HBASE-19753.branch-2.007.patch, > HBASE-19753.branch-2.008.patch, keepalive.diff > > > On my cluster which slows zk, tests hbase-zookeeper rarely all pass. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19685) Fix TestFSErrorsExposed#testFullSystemBubblesFSErrors
[
https://issues.apache.org/jira/browse/HBASE-19685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16322696#comment-16322696
]
Appy commented on HBASE-19685:
--
Oh damn! i just spent 2 hrs debugging this one and arrived at same conclusion -
"60s too less to finish the operation with expected exception, increasing
timeout should be first thing to try".
+1 on your patch. Pushing to master and branch-2.
> Fix TestFSErrorsExposed#testFullSystemBubblesFSErrors
> -
>
> Key: HBASE-19685
> URL: https://issues.apache.org/jira/browse/HBASE-19685
> Project: HBase
> Issue Type: Bug
> Components: test
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19685.v0.patch
>
>
> {code}
> java.lang.AssertionError
> at
> org.apache.hadoop.hbase.regionserver.TestFSErrorsExposed.testFullSystemBubblesFSErrors(TestFSErrorsExposed.java:221)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (HBASE-19770) `list` shell command does not return an array of table names
Josh Elser created HBASE-19770:
--
Summary: `list` shell command does not return an array of table
names
Key: HBASE-19770
URL: https://issues.apache.org/jira/browse/HBASE-19770
Project: HBase
Issue Type: Bug
Components: shell
Reporter: Romil Choksi
Assignee: Josh Elser
Priority: Critical
Fix For: 2.0.0-beta-2
Another good find by our Romil.
{code}
hbase(main):001:0> list
TABLE
a
1 row(s)
Took 0.8385 seconds
hbase(main):002:0> tables=list
TABLE
a
1 row(s)
Took 0.0267 seconds
hbase(main):003:0> puts tables
hbase(main):004:0> p tables
nil
{code}
The {{list}} command should be returning {{\['a'\]}} but is not.
The command class itself appears to be doing the right thing -- maybe the
retval is getting lost somewhere else?
FYI [~stack].
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (HBASE-19769) IllegalAccessError on package-private Hadoop metrics2 classes in MapReduce jobs
Josh Elser created HBASE-19769:
--
Summary: IllegalAccessError on package-private Hadoop metrics2
classes in MapReduce jobs
Key: HBASE-19769
URL: https://issues.apache.org/jira/browse/HBASE-19769
Project: HBase
Issue Type: Bug
Components: mapreduce, metrics
Affects Versions: 2.0.0-beta-1
Reporter: Josh Elser
Assignee: Josh Elser
Priority: Critical
Fix For: 2.0.0-beta-2
issues for context: HBASE-17170, HBASE-17448, TEZ-3299, HADOOP-10893
Since Hadoop 2.6.0, the {{yarn jar}} entry point to submit a YARN job has been
using a custom classloader to separate Hadoop dependencies from the user's JAR
being run. A separate classloader is created for the user-provided jar, and
then this classloader is set as the contextClassLoader before the Tool is
executed by Hadoop's RunJar class. This has been (mostly?) fine for us to date
because we don't try to access any Hadoop internal classes client-side.
However, with the ZK metrics, clients are pushing ZK metrics to metrics2. The
problem is that Hadoop metrics2 implementations which we reference from the
same package are loaded by a different classloader than our HBase code is
loaded from. This makes the expected package-private access of these Metrics2
classes (e.g. MetricsInfoImpl) fail with an IllegalAccessError.
{noformat}
java.lang.RuntimeException: Could not create interface
org.apache.hadoop.hbase.zookeeper.MetricsZooKeeperSource Is the hadoop
compatibility jar on the classpath?
at
org.apache.hadoop.hbase.CompatibilitySingletonFactory.getInstance(CompatibilitySingletonFactory.java:75)
at org.apache.hadoop.hbase.zookeeper.ZKMetrics.(ZKMetrics.java:36)
at
org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.(RecoverableZooKeeper.java:115)
at org.apache.hadoop.hbase.zookeeper.ZKUtil.connect(ZKUtil.java:139)
at
org.apache.hadoop.hbase.zookeeper.ZKWatcher.(ZKWatcher.java:128)
at
org.apache.hadoop.hbase.zookeeper.ZKWatcher.(ZKWatcher.java:102)
at
org.apache.hadoop.hbase.security.token.TokenUtil.getAuthToken(TokenUtil.java:293)
at
org.apache.hadoop.hbase.security.token.TokenUtil.addTokenForJob(TokenUtil.java:259)
at
org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.initCredentials(TableMapReduceUtil.java:535)
at
org.apache.phoenix.mapreduce.MultiHfileOutputFormat.configureIncrementalLoad(MultiHfileOutputFormat.java:712)
at
org.apache.phoenix.mapreduce.AbstractBulkLoadTool.submitJob(AbstractBulkLoadTool.java:300)
at
org.apache.phoenix.mapreduce.AbstractBulkLoadTool.loadData(AbstractBulkLoadTool.java:267)
at
org.apache.phoenix.mapreduce.AbstractBulkLoadTool.run(AbstractBulkLoadTool.java:180)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
at
org.apache.phoenix.mapreduce.CsvBulkLoadTool.main(CsvBulkLoadTool.java:109)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:239)
at org.apache.hadoop.util.RunJar.main(RunJar.java:153)
Caused by: java.util.ServiceConfigurationError:
org.apache.hadoop.hbase.zookeeper.MetricsZooKeeperSource: Provider
org.apache.hadoop.hbase.zookeeper.MetricsZooKeeperSourceImpl could not be
instantiated
at java.util.ServiceLoader.fail(ServiceLoader.java:232)
at java.util.ServiceLoader.access$100(ServiceLoader.java:185)
at
java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:384)
at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
at
org.apache.hadoop.hbase.CompatibilitySingletonFactory.getInstance(CompatibilitySingletonFactory.java:59)
... 21 more
Caused by: java.lang.IllegalAccessError: tried to access class
org.apache.hadoop.metrics2.lib.MetricsInfoImpl from class
org.apache.hadoop.metrics2.lib.DynamicMetricsRegistry
at
org.apache.hadoop.metrics2.lib.DynamicMetricsRegistry.newGauge(DynamicMetricsRegistry.java:139)
at
org.apache.hadoop.hbase.zookeeper.MetricsZooKeeperSourceImpl.(MetricsZooKeeperSourceImpl.java:59)
at
org.apache.hadoop.hbase.zookeeper.MetricsZooKeeperSourceImpl.(MetricsZooKeeperSourceImpl.java:51)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
sun.reflect.DelegatingConstructorAccessorImpl.new
[jira] [Commented] (HBASE-19754) Backport HBASE-11409 to branch-1 and branch-1.4
[ https://issues.apache.org/jira/browse/HBASE-19754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16322631#comment-16322631 ] churro morales commented on HBASE-19754: fair enough, ill move the patch back to the original ticket and close this one off. Sorry about that, this patch is pretty different and I assumed it might go under a different ticket. > Backport HBASE-11409 to branch-1 and branch-1.4 > --- > > Key: HBASE-19754 > URL: https://issues.apache.org/jira/browse/HBASE-19754 > Project: HBase > Issue Type: Bug >Affects Versions: 1.4.0, 1.5.0 >Reporter: churro morales >Assignee: churro morales >Priority: Minor > Attachments: HBASE-19754.branch-1.patch > > > backport HBASE-11409 to branch-1, branch-1.4 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19768) RegionServer startup failing when DN is dead
Jean-Marc Spaggiari created HBASE-19768:
---
Summary: RegionServer startup failing when DN is dead
Key: HBASE-19768
URL: https://issues.apache.org/jira/browse/HBASE-19768
Project: HBase
Issue Type: Bug
Reporter: Jean-Marc Spaggiari
When starting HBase, if the datanode hosted on the same host is dead but not
yet detected by the namenode, HBase will fail to start
{code}
515691223393/node8.distparser.com%2C16020%2C1515691223393.1515691238778 failed,
retry = 7
org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException:
syscall:getsockopt(..) failed: Connexion refusée: /192.168.23.2:50010
at
org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(..)(Unknown
Source)
Caused by:
org.apache.hbase.thirdparty.io.netty.channel.unix.Errors$NativeConnectException:
syscall:getsockopt(..) failed: Connexion refusée
... 1 more
{code}
and will also get stuck to stop:
{code}
hbase@node2:~/hbase-2.0.0-beta-1$ bin/stop-hbase.sh
stopping
hbase^C
hbase@node2:~/hbase-2.0.0-beta-1$ bin/stop-hbase.sh
stopping
hbase..
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/home/hbase/hbase-2.0.0-beta-1/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/home/hbase/hbase-2.0.0-beta-1/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
{code}
The most interesting is that it seems to fail the same way even if the DN is
declared dead on HDFS side:
{code}
515692041367/node8.distparser.com%2C16020%2C1515692041367.1515692057716 failed,
retry = 4
org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException:
syscall:getsockopt(..) failed: Connexion refusée: /192.168.23.2:50010
at
org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(..)(Unknown
Source)
Caused by:
org.apache.hbase.thirdparty.io.netty.channel.unix.Errors$NativeConnectException:
syscall:getsockopt(..) failed: Connexion refusée
... 1 more
{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19752) RSGroupBasedLoadBalancer#getMisplacedRegions() should handle the case where rs group cannot be determined
[
https://issues.apache.org/jira/browse/HBASE-19752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16322599#comment-16322599
]
Hadoop QA commented on HBASE-19752:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
8s{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m
2s{color} | {color:blue} The patch file was not named according to hbase's
naming conventions. Please see
https://yetus.apache.org/documentation/0.6.0/precommit-patchnames for
instructions. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
49s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
52s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
20s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
43s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
20m 56s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
2s{color} | {color:green} hbase-rsgroup in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
8s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 40m 45s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19752 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12905707/19752.v5.txt |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 36a62299bf7a 3.13.0-129-generic #178-Ubuntu SMP Fri Aug 11
12:48:20 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | master / 71a1192d67 |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/11022/testReport/ |
| modules | C: hbase-rsgroup U: hbase-rsgroup |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/11022/console |
| Powered by | Apache Yetus 0.6.0 http://yetus.apa
