Yes, if I change the version on hbase_client_service_1_1_2 to
nifi-hbase_1_1_2-client-service-nar-1.6.0-mapr.nar (nifi 1.6 code compiled with
mapr profile) it just connects fine.
If I change client service to
nifi-hbase_1_1_2-client-service-nar-1.7.1-mapr.nar (nifi 1.7.1 compiled with
mapr profile), this error is coming.
See if below strack trace is sent with proper format:
Sun Nov 25 22:36:44 UTC 2018, RpcRetryingCaller{globalStartTime=1543185404532,
pause=100, retries=1}, org.apache.hadoop.hbase.MasterNotRunningException:
org.apache.hadoop.hbase.MasterNotRunningException: Can't get connection to
ZooKeeper: KeeperErrorCode = AuthFailed for /hbase
: {}
org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after
attempts=1, exceptions:
Sun Nov 25 22:36:44 UTC 2018, RpcRetryingCaller{globalStartTime=1543185404532,
pause=100, retries=1}, org.apache.hadoop.hbase.MasterNotRunningException:
org.apache.hadoop.hbase.MasterNotRunningException: Can't get connection to
ZooKeeper: KeeperErrorCode = AuthFailed for /hbase
at
org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:147)
at
org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:3917)
at
org.apache.hadoop.hbase.client.HBaseAdmin.listTableNames(HBaseAdmin.java:413)
at
org.apache.hadoop.hbase.client.HBaseAdmin.listTableNames(HBaseAdmin.java:397)
at
org.apache.nifi.hbase.HBase_1_1_2_ClientService.onEnabled(HBase_1_1_2_ClientService.java:264)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:142)
at
org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:130)
at
org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:75)
at
org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotation(ReflectionUtils.java:52)
at
org.apache.nifi.controller.service.StandardControllerServiceNode$2.run(StandardControllerServiceNode.java:433)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.hadoop.hbase.MasterNotRunningException:
org.apache.hadoop.hbase.MasterNotRunningException: Can't get connection to
ZooKeeper: KeeperErrorCode = AuthFailed for /hbase
at
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStub(ConnectionManager.java:1533)
at
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$MasterServiceStubMaker.makeStub(ConnectionManager.java:1553)
at
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getKeepAliveMasterService(ConnectionManager.java:1704)
at
org.apache.hadoop.hbase.client.MasterCallable.prepare(MasterCallable.java:38)
at
org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:124)
... 20 common frames omitted
Caused by: org.apache.hadoop.hbase.MasterNotRunningException: Can't get
connection to ZooKeeper: KeeperErrorCode = AuthFailed for /hbase
at
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.checkIfBaseNodeAvailable(ConnectionManager.java:906)
at
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.access$400(ConnectionManager.java:545)
at
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStubNoRetries(ConnectionManager.java:1483)
at
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStub(ConnectionManager.java:1524)
... 24 common frames omitted
Caused by: org.apache.zookeeper.KeeperException$AuthFailedException:
KeeperErrorCode = AuthFailed for /hbase
at org.apache.zookeeper.KeeperException.create(KeeperException.java:123)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1045)
at
org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:221)
at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:541)
at
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.checkIfBaseNodeAvailable(ConnectionManager.java:895)
... 27 common frames omitted
2018-11-25 22:36:44,535 ERROR [Timer-Driven Process Thread-11]
o.a.n.c.s.StandardControllerServiceNode Failed to invoke @OnEnabled method of
HBase_1_1_2_ClientService[id=549bd5c3-0166-1000-0000-00000546f21d] due to
org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after
attempts=1, exceptions:
Thanks,
Ravi Papisetti
On 25/11/18, 4:47 PM, "Joe Witt" <joe.w...@gmail.com> wrote:
Ravi
it is very tough to read that stack trace as provided but the core
piece of it seems to be that authorization is failing for "/hbase".
Have you confirmed that the proper authorization configuration is
present?
Thanks
On Sun, Nov 25, 2018 at 4:50 PM Ravi Papisetti (rpapiset)
<rpapi...@cisco.com> wrote:
>
> Hi,
>
>
>
> Until recently we were using nifi-1.6 nifi-hbase_1_1_2-client-service-nar
compiled with mapr profile. It has been working fine connecting to mapr
database without any issues.
>
>
>
> Recently we are upgrading to nifi-1.7, hbase client service is failing to
connect with same mapr distribution.
>
>
>
> Appreciate any insights to understand what changed in nifi 1.7 that is
failing to connect with maprdb.
>
>
>
> Below is the error message, appreciate any help:
>
> Wed Nov 21 06:09:13 UTC 2018,
RpcRetryingCaller{globalStartTime=1542780553492, pause=100, retries=3},
org.apache.hadoop.hbase.MasterNotRunningException:
org.apache.hadoop.hbase.MasterNotRunningException: Can't get connection to
ZooKeeper: KeeperErrorCode = AuthFailed for /hbase Wed Nov 21 06:09:13 UTC
2018, RpcRetryingCaller{globalStartTime=1542780553492, pause=100, retries=3},
org.apache.hadoop.hbase.MasterNotRunningException:
org.apache.hadoop.hbase.MasterNotRunningException: Can't get connection to
ZooKeeper: KeeperErrorCode = AuthFailed for /hbase at
org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:147)
at
org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:3917)
at
org.apache.hadoop.hbase.client.HBaseAdmin.listTableNames(HBaseAdmin.java:413)
at
org.apache.hadoop.hbase.client.HBaseAdmin.listTableNames(HBaseAdmin.java:397)
at
org.apache.nifi.hbase.HBase_1_1_2_ClientService.onEnabled(HBase_1_1_2_ClientService.java:264)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498) at
org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:142)
at
org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:130)
at
org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:75)
at
org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotation(ReflectionUtils.java:52)
at
org.apache.nifi.controller.service.StandardControllerServiceNode$2.run(StandardControllerServiceNode.java:433)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at
java.util.concurrent.FutureTask.run(FutureTask.java:266) at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748) Caused by:
org.apache.hadoop.hbase.MasterNotRunningException:
org.apache.hadoop.hbase.MasterNotRunningException: Can't get connection to
ZooKeeper: KeeperErrorCode = AuthFailed for /hbase at
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStub(ConnectionManager.java:1533)
at
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$MasterServiceStubMaker.makeStub(ConnectionManager.java:1553)
at
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getKeepAliveMasterService(ConnectionManager.java:1704)
at
org.apache.hadoop.hbase.client.MasterCallable.prepare(MasterCallable.java:38)
at
org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:124)
... 20 common frames omitted Caused by:
org.apache.hadoop.hbase.MasterNotRunningException: Can't get connection to
ZooKeeper: KeeperErrorCode = AuthFailed for /hbase at
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.checkIfBaseNodeAvailable(ConnectionManager.java:906)
at
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.access$400(ConnectionManager.java:545)
at
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStubNoRetries(ConnectionManager.java:1483)
at
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStub(ConnectionManager.java:1524)
... 24 common frames omitted Caused by:
org.apache.zookeeper.KeeperException$AuthFailedException: KeeperErrorCode =
AuthFailed for /hbase at
org.apache.zookeeper.KeeperException.create(KeeperException.java:123) at
org.apache.zookeeper.KeeperException.create(KeeperException.java:51) at
org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1045) at
org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:221)
at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:541) at
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.checkIfBaseNodeAvailable(ConnectionManager.java:895)
... 27 common frames omitted
>
>
>
>
>
> Thanks,
>
> Ravi Papisetti
>
>
>
>
