Re: org.apache.phoenix.shaded.org.apache.thrift.TException: Unable to discover transaction service. -> TException: Unable to discover transaction service.
ation.parse(Configuration.java:2787) at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2838) at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2812) at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2689) : running regionserver, logging to /opt/hbase/logs/hbase--regionserver-m401b01-phoenix.m401b01.out : SLF4J: Class path contains multiple SLF4J bindings. : SLF4J: Found binding in [jar:file:/opt/hbase/lib/phoenix-5.0.0-HBase-2.0-client.jar!/org/slf4j/impl/StaticLoggerBinder.class] : SLF4J: Found binding in [jar:file:/opt/hbase/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class] : SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. : Exception in thread "main" java.lang.NoSuchMethodError: com.ctc.wstx.stax.WstxInputFactory.createSR(Lcom/ctc/wstx/api/ReaderConfig;Lcom/ctc/wstx/io/SystemId;Lcom/ctc/wstx/io/InputBootstrapper;ZZ)Lorg/codehaus/stax2/XMLStreamReader2; : at org.apache.hadoop.conf.Configuration.parse(Configuration.java:2803) : at org.apache.hadoop.conf.Configuration.parse(Configuration.java:2787) : at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2838) : at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2812) : at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2689) I built HBase with 2.8.5 and was able to resolve the hasKerberosKeyTab methodNotFound error. The only problem is that it took 4 hours to run an automated build on Docker cloud to build HBase and the build failed eventually. I think I am going to download the Hadoop jars from maven, rather than build HBase. On 27/09/2018 12:56 AM, Josh Elser wrote: If you're using HBase with Hadoop3, HBase should have Hadoop3 jars. Re-build HBase using the -Dhadoop.profile=3.0 (I think it is) CLI option. On 9/26/18 7:21 AM, Francis Chuang wrote: Upon further investigation, it appears that this is because org.apache.hadoop.security.authentication.util.KerberosUtil.hasKerberosKeyTab is only available in Hadoop 2.8+. HBase ships with Hadoop 2.7.4 jars. I noticed that Hadoop was bumped from 2.7.4 to 3.0.0 a few months ago to fix PQS/Avatica issues: https://github.com/apache/phoenix/blame/master/pom.xml#L70 I think this causes Phoenix to expect some things that are available in Hadoop 3.0.0, but are not present in HBase's Hadoop 2.7.4 jars. I think I can try and replace the hadoop-*.jar files in hbase/lib with the equivalent 2.8.5 versions, however I am not familiar with Java and the hadoop project, so I am not sure if this is going to introduce issues. On 26/09/2018 4:44 PM, Francis Chuang wrote: I wonder if this is because: - HBase's binary distribution ships with Hadoop 2.7.4 jars. - Phoenix 5.0.0 has Hadoop 3.0.0 declared in its pom.xml: https://github.com/apache/phoenix/blob/8a819c6c3b4befce190c6ac759f744df511de61d/pom.xml#L70 - Tephra has Hadoop 2.2.0 declared in its pom.xml: https://github.com/apache/incubator-tephra/blob/master/pom.xml#L211 On 26/09/2018 4:03 PM, Francis Chuang wrote: Hi all, I am using Phoenix 5.0.0 with HBase 2.0.0. I am seeing errors while trying to create transactional tables using Phoenix. I am using my Phoenix + HBase all in one docker image available here: https://github.com/Boostport/hbase-phoenix-all-in-one This is the error: org.apache.phoenix.shaded.org.apache.thrift.TException: Unable to discover transaction service. -> TException: Unable to discover transaction service. I checked the tephra logs and got the following: Exception in thread "HDFSTransactionStateStorage STARTING" Exception in thread "ThriftRPCServer" com.google.common.util.concurrent.ExecutionError: java.lang.NoSuchMethodError: org.apache.hadoop.security.authentication.util.KerberosUtil.hasKerberosKeyTab(Ljavax/security/auth/Subject;)Z at com.google.common.util.concurrent.Futures.wrapAndThrowUnchecked(Futures.java:1008) at com.google.common.util.concurrent.Futures.getUnchecked(Futures.java:1001) at com.google.common.util.concurrent.AbstractService.startAndWait(AbstractService.java:220) at com.google.common.util.concurrent.AbstractIdleService.startAndWait(AbstractIdleService.java:106) at org.apache.tephra.TransactionManager.doStart(TransactionManager.java:245) at com.google.common.util.concurrent.AbstractService.start(AbstractService.java:170) at com.google.common.util.concurrent.AbstractService.startAndWait(AbstractService.java:220) at org.apache.tephra.distributed.TransactionServiceThriftHandler.init(TransactionServiceThriftHandler.java:249) at org.apache.tephra.rpc.ThriftRPCServer.startUp(ThriftRPCServer.java:177) at com.google.common.util.concurrent.AbstractExecutionThreadService$1$1.run(AbstractExecutionThreadService.java:47) at java.lang.Thr
Re: org.apache.phoenix.shaded.org.apache.thrift.TException: Unable to discover transaction service. -> TException: Unable to discover transaction service.
2.0-client.jar!/org/slf4j/impl/StaticLoggerBinder.class] : SLF4J: Found binding in [jar:file:/opt/hbase/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class] : SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. : Exception in thread "main" java.lang.NoSuchMethodError: com.ctc.wstx.stax.WstxInputFactory.createSR(Lcom/ctc/wstx/api/ReaderConfig;Lcom/ctc/wstx/io/SystemId;Lcom/ctc/wstx/io/InputBootstrapper;ZZ)Lorg/codehaus/stax2/XMLStreamReader2; : at org.apache.hadoop.conf.Configuration.parse(Configuration.java:2803) : at org.apache.hadoop.conf.Configuration.parse(Configuration.java:2787) : at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2838) : at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2812) : at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2689) I built HBase with 2.8.5 and was able to resolve the hasKerberosKeyTab methodNotFound error. The only problem is that it took 4 hours to run an automated build on Docker cloud to build HBase and the build failed eventually. I think I am going to download the Hadoop jars from maven, rather than build HBase. On 27/09/2018 12:56 AM, Josh Elser wrote: If you're using HBase with Hadoop3, HBase should have Hadoop3 jars. Re-build HBase using the -Dhadoop.profile=3.0 (I think it is) CLI option. On 9/26/18 7:21 AM, Francis Chuang wrote: Upon further investigation, it appears that this is because org.apache.hadoop.security.authentication.util.KerberosUtil.hasKerberosKeyTab is only available in Hadoop 2.8+. HBase ships with Hadoop 2.7.4 jars. I noticed that Hadoop was bumped from 2.7.4 to 3.0.0 a few months ago to fix PQS/Avatica issues: https://github.com/apache/phoenix/blame/master/pom.xml#L70 I think this causes Phoenix to expect some things that are available in Hadoop 3.0.0, but are not present in HBase's Hadoop 2.7.4 jars. I think I can try and replace the hadoop-*.jar files in hbase/lib with the equivalent 2.8.5 versions, however I am not familiar with Java and the hadoop project, so I am not sure if this is going to introduce issues. On 26/09/2018 4:44 PM, Francis Chuang wrote: I wonder if this is because: - HBase's binary distribution ships with Hadoop 2.7.4 jars. - Phoenix 5.0.0 has Hadoop 3.0.0 declared in its pom.xml: https://github.com/apache/phoenix/blob/8a819c6c3b4befce190c6ac759f744df511de61d/pom.xml#L70 - Tephra has Hadoop 2.2.0 declared in its pom.xml: https://github.com/apache/incubator-tephra/blob/master/pom.xml#L211 On 26/09/2018 4:03 PM, Francis Chuang wrote: Hi all, I am using Phoenix 5.0.0 with HBase 2.0.0. I am seeing errors while trying to create transactional tables using Phoenix. I am using my Phoenix + HBase all in one docker image available here: https://github.com/Boostport/hbase-phoenix-all-in-one This is the error: org.apache.phoenix.shaded.org.apache.thrift.TException: Unable to discover transaction service. -> TException: Unable to discover transaction service. I checked the tephra logs and got the following: Exception in thread "HDFSTransactionStateStorage STARTING" Exception in thread "ThriftRPCServer" com.google.common.util.concurrent.ExecutionError: java.lang.NoSuchMethodError: org.apache.hadoop.security.authentication.util.KerberosUtil.hasKerberosKeyTab(Ljavax/security/auth/Subject;)Z at com.google.common.util.concurrent.Futures.wrapAndThrowUnchecked(Futures.java:1008) at com.google.common.util.concurrent.Futures.getUnchecked(Futures.java:1001) at com.google.common.util.concurrent.AbstractService.startAndWait(AbstractService.java:220) at com.google.common.util.concurrent.AbstractIdleService.startAndWait(AbstractIdleService.java:106) at org.apache.tephra.TransactionManager.doStart(TransactionManager.java:245) at com.google.common.util.concurrent.AbstractService.start(AbstractService.java:170) at com.google.common.util.concurrent.AbstractService.startAndWait(AbstractService.java:220) at org.apache.tephra.distributed.TransactionServiceThriftHandler.init(TransactionServiceThriftHandler.java:249) at org.apache.tephra.rpc.ThriftRPCServer.startUp(ThriftRPCServer.java:177) at com.google.common.util.concurrent.AbstractExecutionThreadService$1$1.run(AbstractExecutionThreadService.java:47) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.NoSuchMethodError: org.apache.hadoop.security.authentication.util.KerberosUtil.hasKerberosKeyTab(Ljavax/security/auth/Subject;)Z at org.apache.hadoop.security.UserGroupInformation.(UserGroupInformation.java:715) at org.apache.hadoop.security.UserGroupInformation.loginUserFromSubject(UserGroupInformation.java:925) at org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:873) at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroup
Re: org.apache.phoenix.shaded.org.apache.thrift.TException: Unable to discover transaction service. -> TException: Unable to discover transaction service.
If you're using HBase with Hadoop3, HBase should have Hadoop3 jars. Re-build HBase using the -Dhadoop.profile=3.0 (I think it is) CLI option. On 9/26/18 7:21 AM, Francis Chuang wrote: Upon further investigation, it appears that this is because org.apache.hadoop.security.authentication.util.KerberosUtil.hasKerberosKeyTab is only available in Hadoop 2.8+. HBase ships with Hadoop 2.7.4 jars. I noticed that Hadoop was bumped from 2.7.4 to 3.0.0 a few months ago to fix PQS/Avatica issues: https://github.com/apache/phoenix/blame/master/pom.xml#L70 I think this causes Phoenix to expect some things that are available in Hadoop 3.0.0, but are not present in HBase's Hadoop 2.7.4 jars. I think I can try and replace the hadoop-*.jar files in hbase/lib with the equivalent 2.8.5 versions, however I am not familiar with Java and the hadoop project, so I am not sure if this is going to introduce issues. On 26/09/2018 4:44 PM, Francis Chuang wrote: I wonder if this is because: - HBase's binary distribution ships with Hadoop 2.7.4 jars. - Phoenix 5.0.0 has Hadoop 3.0.0 declared in its pom.xml: https://github.com/apache/phoenix/blob/8a819c6c3b4befce190c6ac759f744df511de61d/pom.xml#L70 - Tephra has Hadoop 2.2.0 declared in its pom.xml: https://github.com/apache/incubator-tephra/blob/master/pom.xml#L211 On 26/09/2018 4:03 PM, Francis Chuang wrote: Hi all, I am using Phoenix 5.0.0 with HBase 2.0.0. I am seeing errors while trying to create transactional tables using Phoenix. I am using my Phoenix + HBase all in one docker image available here: https://github.com/Boostport/hbase-phoenix-all-in-one This is the error: org.apache.phoenix.shaded.org.apache.thrift.TException: Unable to discover transaction service. -> TException: Unable to discover transaction service. I checked the tephra logs and got the following: Exception in thread "HDFSTransactionStateStorage STARTING" Exception in thread "ThriftRPCServer" com.google.common.util.concurrent.ExecutionError: java.lang.NoSuchMethodError: org.apache.hadoop.security.authentication.util.KerberosUtil.hasKerberosKeyTab(Ljavax/security/auth/Subject;)Z at com.google.common.util.concurrent.Futures.wrapAndThrowUnchecked(Futures.java:1008) at com.google.common.util.concurrent.Futures.getUnchecked(Futures.java:1001) at com.google.common.util.concurrent.AbstractService.startAndWait(AbstractService.java:220) at com.google.common.util.concurrent.AbstractIdleService.startAndWait(AbstractIdleService.java:106) at org.apache.tephra.TransactionManager.doStart(TransactionManager.java:245) at com.google.common.util.concurrent.AbstractService.start(AbstractService.java:170) at com.google.common.util.concurrent.AbstractService.startAndWait(AbstractService.java:220) at org.apache.tephra.distributed.TransactionServiceThriftHandler.init(TransactionServiceThriftHandler.java:249) at org.apache.tephra.rpc.ThriftRPCServer.startUp(ThriftRPCServer.java:177) at com.google.common.util.concurrent.AbstractExecutionThreadService$1$1.run(AbstractExecutionThreadService.java:47) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.NoSuchMethodError: org.apache.hadoop.security.authentication.util.KerberosUtil.hasKerberosKeyTab(Ljavax/security/auth/Subject;)Z at org.apache.hadoop.security.UserGroupInformation.(UserGroupInformation.java:715) at org.apache.hadoop.security.UserGroupInformation.loginUserFromSubject(UserGroupInformation.java:925) at org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:873) at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:740) at org.apache.hadoop.fs.FileSystem$Cache$Key.(FileSystem.java:3472) at org.apache.hadoop.fs.FileSystem$Cache.getUnique(FileSystem.java:3310) at org.apache.hadoop.fs.FileSystem.newInstance(FileSystem.java:529) at org.apache.tephra.persist.HDFSTransactionStateStorage.startUp(HDFSTransactionStateStorage.java:104) at com.google.common.util.concurrent.AbstractIdleService$1$1.run(AbstractIdleService.java:43) ... 1 more 2018-09-26 04:31:11,290 INFO [leader-election-tx.service-leader] distributed.TransactionService (TransactionService.java:leader(115)) - Transaction Thrift Service didn't start on /0.0.0.0:15165 java.lang.NoSuchMethodError: org.apache.hadoop.security.authentication.util.KerberosUtil.hasKerberosKeyTab(Ljavax/security/auth/Subject;)Z at org.apache.hadoop.security.UserGroupInformation.(UserGroupInformation.java:715) at org.apache.hadoop.security.UserGroupInformation.loginUserFromSubject(UserGroupInformation.java:925) at org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:873) at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:740) at org.apache.hadoop.fs.FileSystem$Cac
Re: org.apache.phoenix.shaded.org.apache.thrift.TException: Unable to discover transaction service. -> TException: Unable to discover transaction service.
Upon further investigation, it appears that this is because org.apache.hadoop.security.authentication.util.KerberosUtil.hasKerberosKeyTab is only available in Hadoop 2.8+. HBase ships with Hadoop 2.7.4 jars. I noticed that Hadoop was bumped from 2.7.4 to 3.0.0 a few months ago to fix PQS/Avatica issues: https://github.com/apache/phoenix/blame/master/pom.xml#L70 I think this causes Phoenix to expect some things that are available in Hadoop 3.0.0, but are not present in HBase's Hadoop 2.7.4 jars. I think I can try and replace the hadoop-*.jar files in hbase/lib with the equivalent 2.8.5 versions, however I am not familiar with Java and the hadoop project, so I am not sure if this is going to introduce issues. On 26/09/2018 4:44 PM, Francis Chuang wrote: I wonder if this is because: - HBase's binary distribution ships with Hadoop 2.7.4 jars. - Phoenix 5.0.0 has Hadoop 3.0.0 declared in its pom.xml: https://github.com/apache/phoenix/blob/8a819c6c3b4befce190c6ac759f744df511de61d/pom.xml#L70 - Tephra has Hadoop 2.2.0 declared in its pom.xml: https://github.com/apache/incubator-tephra/blob/master/pom.xml#L211 On 26/09/2018 4:03 PM, Francis Chuang wrote: Hi all, I am using Phoenix 5.0.0 with HBase 2.0.0. I am seeing errors while trying to create transactional tables using Phoenix. I am using my Phoenix + HBase all in one docker image available here: https://github.com/Boostport/hbase-phoenix-all-in-one This is the error: org.apache.phoenix.shaded.org.apache.thrift.TException: Unable to discover transaction service. -> TException: Unable to discover transaction service. I checked the tephra logs and got the following: Exception in thread "HDFSTransactionStateStorage STARTING" Exception in thread "ThriftRPCServer" com.google.common.util.concurrent.ExecutionError: java.lang.NoSuchMethodError: org.apache.hadoop.security.authentication.util.KerberosUtil.hasKerberosKeyTab(Ljavax/security/auth/Subject;)Z at com.google.common.util.concurrent.Futures.wrapAndThrowUnchecked(Futures.java:1008) at com.google.common.util.concurrent.Futures.getUnchecked(Futures.java:1001) at com.google.common.util.concurrent.AbstractService.startAndWait(AbstractService.java:220) at com.google.common.util.concurrent.AbstractIdleService.startAndWait(AbstractIdleService.java:106) at org.apache.tephra.TransactionManager.doStart(TransactionManager.java:245) at com.google.common.util.concurrent.AbstractService.start(AbstractService.java:170) at com.google.common.util.concurrent.AbstractService.startAndWait(AbstractService.java:220) at org.apache.tephra.distributed.TransactionServiceThriftHandler.init(TransactionServiceThriftHandler.java:249) at org.apache.tephra.rpc.ThriftRPCServer.startUp(ThriftRPCServer.java:177) at com.google.common.util.concurrent.AbstractExecutionThreadService$1$1.run(AbstractExecutionThreadService.java:47) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.NoSuchMethodError: org.apache.hadoop.security.authentication.util.KerberosUtil.hasKerberosKeyTab(Ljavax/security/auth/Subject;)Z at org.apache.hadoop.security.UserGroupInformation.(UserGroupInformation.java:715) at org.apache.hadoop.security.UserGroupInformation.loginUserFromSubject(UserGroupInformation.java:925) at org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:873) at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:740) at org.apache.hadoop.fs.FileSystem$Cache$Key.(FileSystem.java:3472) at org.apache.hadoop.fs.FileSystem$Cache.getUnique(FileSystem.java:3310) at org.apache.hadoop.fs.FileSystem.newInstance(FileSystem.java:529) at org.apache.tephra.persist.HDFSTransactionStateStorage.startUp(HDFSTransactionStateStorage.java:104) at com.google.common.util.concurrent.AbstractIdleService$1$1.run(AbstractIdleService.java:43) ... 1 more 2018-09-26 04:31:11,290 INFO [leader-election-tx.service-leader] distributed.TransactionService (TransactionService.java:leader(115)) - Transaction Thrift Service didn't start on /0.0.0.0:15165 java.lang.NoSuchMethodError: org.apache.hadoop.security.authentication.util.KerberosUtil.hasKerberosKeyTab(Ljavax/security/auth/Subject;)Z at org.apache.hadoop.security.UserGroupInformation.(UserGroupInformation.java:715) at org.apache.hadoop.security.UserGroupInformation.loginUserFromSubject(UserGroupInformation.java:925) at org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:873) at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:740) at org.apache.hadoop.fs.FileSystem$Cache$Key.(FileSystem.java:3472) at org.apache.hadoop.fs.FileSystem$Cache.getUnique(FileSystem.java:3310) at org.apache.hadoop.fs.FileSystem.newInstance(FileSystem.java:529) at org.apache.tephra.persist.HDFSTransact
Re: org.apache.phoenix.shaded.org.apache.thrift.TException: Unable to discover transaction service. -> TException: Unable to discover transaction service.
I wonder if this is because: - HBase's binary distribution ships with Hadoop 2.7.4 jars. - Phoenix 5.0.0 has Hadoop 3.0.0 declared in its pom.xml: https://github.com/apache/phoenix/blob/8a819c6c3b4befce190c6ac759f744df511de61d/pom.xml#L70 - Tephra has Hadoop 2.2.0 declared in its pom.xml: https://github.com/apache/incubator-tephra/blob/master/pom.xml#L211 On 26/09/2018 4:03 PM, Francis Chuang wrote: Hi all, I am using Phoenix 5.0.0 with HBase 2.0.0. I am seeing errors while trying to create transactional tables using Phoenix. I am using my Phoenix + HBase all in one docker image available here: https://github.com/Boostport/hbase-phoenix-all-in-one This is the error: org.apache.phoenix.shaded.org.apache.thrift.TException: Unable to discover transaction service. -> TException: Unable to discover transaction service. I checked the tephra logs and got the following: Exception in thread "HDFSTransactionStateStorage STARTING" Exception in thread "ThriftRPCServer" com.google.common.util.concurrent.ExecutionError: java.lang.NoSuchMethodError: org.apache.hadoop.security.authentication.util.KerberosUtil.hasKerberosKeyTab(Ljavax/security/auth/Subject;)Z at com.google.common.util.concurrent.Futures.wrapAndThrowUnchecked(Futures.java:1008) at com.google.common.util.concurrent.Futures.getUnchecked(Futures.java:1001) at com.google.common.util.concurrent.AbstractService.startAndWait(AbstractService.java:220) at com.google.common.util.concurrent.AbstractIdleService.startAndWait(AbstractIdleService.java:106) at org.apache.tephra.TransactionManager.doStart(TransactionManager.java:245) at com.google.common.util.concurrent.AbstractService.start(AbstractService.java:170) at com.google.common.util.concurrent.AbstractService.startAndWait(AbstractService.java:220) at org.apache.tephra.distributed.TransactionServiceThriftHandler.init(TransactionServiceThriftHandler.java:249) at org.apache.tephra.rpc.ThriftRPCServer.startUp(ThriftRPCServer.java:177) at com.google.common.util.concurrent.AbstractExecutionThreadService$1$1.run(AbstractExecutionThreadService.java:47) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.NoSuchMethodError: org.apache.hadoop.security.authentication.util.KerberosUtil.hasKerberosKeyTab(Ljavax/security/auth/Subject;)Z at org.apache.hadoop.security.UserGroupInformation.(UserGroupInformation.java:715) at org.apache.hadoop.security.UserGroupInformation.loginUserFromSubject(UserGroupInformation.java:925) at org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:873) at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:740) at org.apache.hadoop.fs.FileSystem$Cache$Key.(FileSystem.java:3472) at org.apache.hadoop.fs.FileSystem$Cache.getUnique(FileSystem.java:3310) at org.apache.hadoop.fs.FileSystem.newInstance(FileSystem.java:529) at org.apache.tephra.persist.HDFSTransactionStateStorage.startUp(HDFSTransactionStateStorage.java:104) at com.google.common.util.concurrent.AbstractIdleService$1$1.run(AbstractIdleService.java:43) ... 1 more 2018-09-26 04:31:11,290 INFO [leader-election-tx.service-leader] distributed.TransactionService (TransactionService.java:leader(115)) - Transaction Thrift Service didn't start on /0.0.0.0:15165 java.lang.NoSuchMethodError: org.apache.hadoop.security.authentication.util.KerberosUtil.hasKerberosKeyTab(Ljavax/security/auth/Subject;)Z at org.apache.hadoop.security.UserGroupInformation.(UserGroupInformation.java:715) at org.apache.hadoop.security.UserGroupInformation.loginUserFromSubject(UserGroupInformation.java:925) at org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:873) at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:740) at org.apache.hadoop.fs.FileSystem$Cache$Key.(FileSystem.java:3472) at org.apache.hadoop.fs.FileSystem$Cache.getUnique(FileSystem.java:3310) at org.apache.hadoop.fs.FileSystem.newInstance(FileSystem.java:529) at org.apache.tephra.persist.HDFSTransactionStateStorage.startUp(HDFSTransactionStateStorage.java:104) at com.google.common.util.concurrent.AbstractIdleService$1$1.run(AbstractIdleService.java:43) at java.lang.Thread.run(Thread.java:748) I know that HBase ships with the Hadoop 2.7.4 jars and I was not able to find "hasKerberosKeyTab" grepping through the source code for hadoop 2.7.4. However, I checked the Hadoop 2.7.4 source files from the stack trace above and the line numbers do not match up. Interestingly, I only see this issue on my older machine (Core i7 920 with 12GB of RAM) and Gitlab's CI environment (a Google Cloud n1-standard-1 instance with 1vCPU and 3.75GB of RAM). I know Michael also encountered this problem while running the Phoenix tests for calcite-avatica-go
org.apache.phoenix.shaded.org.apache.thrift.TException: Unable to discover transaction service. -> TException: Unable to discover transaction service.
Hi all, I am using Phoenix 5.0.0 with HBase 2.0.0. I am seeing errors while trying to create transactional tables using Phoenix. I am using my Phoenix + HBase all in one docker image available here: https://github.com/Boostport/hbase-phoenix-all-in-one This is the error: org.apache.phoenix.shaded.org.apache.thrift.TException: Unable to discover transaction service. -> TException: Unable to discover transaction service. I checked the tephra logs and got the following: Exception in thread "HDFSTransactionStateStorage STARTING" Exception in thread "ThriftRPCServer" com.google.common.util.concurrent.ExecutionError: java.lang.NoSuchMethodError: org.apache.hadoop.security.authentication.util.KerberosUtil.hasKerberosKeyTab(Ljavax/security/auth/Subject;)Z at com.google.common.util.concurrent.Futures.wrapAndThrowUnchecked(Futures.java:1008) at com.google.common.util.concurrent.Futures.getUnchecked(Futures.java:1001) at com.google.common.util.concurrent.AbstractService.startAndWait(AbstractService.java:220) at com.google.common.util.concurrent.AbstractIdleService.startAndWait(AbstractIdleService.java:106) at org.apache.tephra.TransactionManager.doStart(TransactionManager.java:245) at com.google.common.util.concurrent.AbstractService.start(AbstractService.java:170) at com.google.common.util.concurrent.AbstractService.startAndWait(AbstractService.java:220) at org.apache.tephra.distributed.TransactionServiceThriftHandler.init(TransactionServiceThriftHandler.java:249) at org.apache.tephra.rpc.ThriftRPCServer.startUp(ThriftRPCServer.java:177) at com.google.common.util.concurrent.AbstractExecutionThreadService$1$1.run(AbstractExecutionThreadService.java:47) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.NoSuchMethodError: org.apache.hadoop.security.authentication.util.KerberosUtil.hasKerberosKeyTab(Ljavax/security/auth/Subject;)Z at org.apache.hadoop.security.UserGroupInformation.(UserGroupInformation.java:715) at org.apache.hadoop.security.UserGroupInformation.loginUserFromSubject(UserGroupInformation.java:925) at org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:873) at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:740) at org.apache.hadoop.fs.FileSystem$Cache$Key.(FileSystem.java:3472) at org.apache.hadoop.fs.FileSystem$Cache.getUnique(FileSystem.java:3310) at org.apache.hadoop.fs.FileSystem.newInstance(FileSystem.java:529) at org.apache.tephra.persist.HDFSTransactionStateStorage.startUp(HDFSTransactionStateStorage.java:104) at com.google.common.util.concurrent.AbstractIdleService$1$1.run(AbstractIdleService.java:43) ... 1 more 2018-09-26 04:31:11,290 INFO [leader-election-tx.service-leader] distributed.TransactionService (TransactionService.java:leader(115)) - Transaction Thrift Service didn't start on /0.0.0.0:15165 java.lang.NoSuchMethodError: org.apache.hadoop.security.authentication.util.KerberosUtil.hasKerberosKeyTab(Ljavax/security/auth/Subject;)Z at org.apache.hadoop.security.UserGroupInformation.(UserGroupInformation.java:715) at org.apache.hadoop.security.UserGroupInformation.loginUserFromSubject(UserGroupInformation.java:925) at org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:873) at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:740) at org.apache.hadoop.fs.FileSystem$Cache$Key.(FileSystem.java:3472) at org.apache.hadoop.fs.FileSystem$Cache.getUnique(FileSystem.java:3310) at org.apache.hadoop.fs.FileSystem.newInstance(FileSystem.java:529) at org.apache.tephra.persist.HDFSTransactionStateStorage.startUp(HDFSTransactionStateStorage.java:104) at com.google.common.util.concurrent.AbstractIdleService$1$1.run(AbstractIdleService.java:43) at java.lang.Thread.run(Thread.java:748) I know that HBase ships with the Hadoop 2.7.4 jars and I was not able to find "hasKerberosKeyTab" grepping through the source code for hadoop 2.7.4. However, I checked the Hadoop 2.7.4 source files from the stack trace above and the line numbers do not match up. Interestingly, I only see this issue on my older machine (Core i7 920 with 12GB of RAM) and Gitlab's CI environment (a Google Cloud n1-standard-1 instance with 1vCPU and 3.75GB of RAM). I know Michael also encountered this problem while running the Phoenix tests for calcite-avatica-go on an older i5 machine from 2011. It does seem to be pretty weird that we are only seeing this on machines where the CPU is not very powerful. I also printed the classpath for tephra by doing: $ # export HBASE_CONF_DIR=/opt/hbase/conf $ # export HBASE_CP=/opt/hbase/lib $ # export HBASE_HOME=/opt/hbase $ # /opt/hbase/bin/tephra classpath /opt/hbase/bin/../lib/*:/opt/hbase/bin/../conf/:/opt/hbase/phoeni
