回复: flink1.11 sql 发布到yarn session时找不到hbase相关的类

2020-07-29 文章 wind.fly....@outlook.com 
Hi，Xintong：
是把相关的jar配到hadoop_classpath然后再export吗？

Best,
Junbao Zhang

发件人: Xintong Song 
发送时间: 2020年7月30日 10:28
收件人: user-zh 
主题: Re: flink1.11 sql 发布到yarn session时找不到hbase相关的类

export HADOOP_CLASSPATH 就可以了


Thank you~

Xintong Song



On Wed, Jul 29, 2020 at 6:43 PM wind.fly@outlook.com <
wind.fly@outlook.com> wrote:

> Hi,all：
> 最近在升级flink1.11，sql中用到hbase connctor，发布到yarn-session时，报如下异常：
> 2020-07-29 11:49:55
> org.apache.hadoop.hbase.DoNotRetryIOException:
> java.lang.NoClassDefFoundError: Could not initialize class
> org.apache.hadoop.hbase.util.ByteStringer
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.translateException(RpcRetryingCaller.java:248)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:221)
> at
> org.apache.hadoop.hbase.client.ScannerCallableWithReplicas$RetryingRPC.call(ScannerCallableWithReplicas.java:388)
> at
> org.apache.hadoop.hbase.client.ScannerCallableWithReplicas$RetryingRPC.call(ScannerCallableWithReplicas.java:362)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:142)
> at
> org.apache.hadoop.hbase.client.ResultBoundedCompletionService$QueueingFuture.run(ResultBoundedCompletionService.java:80)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.lang.NoClassDefFoundError: Could not initialize class
> org.apache.hadoop.hbase.util.ByteStringer
> at
> org.apache.hadoop.hbase.protobuf.RequestConverter.buildRegionSpecifier(RequestConverter.java:1053)
> at
> org.apache.hadoop.hbase.protobuf.RequestConverter.buildScanRequest(RequestConverter.java:496)
> at
> org.apache.hadoop.hbase.client.ScannerCallable.openScanner(ScannerCallable.java:402)
> at
> org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:274)
> at
> org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:62)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:219)
> ... 7 more
>
> 注意到官方文档有Note:
> Note: To use HBase connector in SQL Client or Flink cluster, it’s highly
> recommended to add HBase dependency jars to Hadoop classpath. Flink will
> load all jars under Hadoop classpath automatically, please refer to HBase,
> MapReduce, and the CLASSPATH<
> https://hbase.apache.org/book.html#hbase.mapreduce.classpath> about how
> to add HBase dependency jars to Hadoop classpath.
>   但是在yarn session下怎么设置classpath呢
>

Re: flink1.11 sql 发布到yarn session时找不到hbase相关的类

2020-07-29 文章 Xintong Song 
export HADOOP_CLASSPATH 就可以了


Thank you~

Xintong Song



On Wed, Jul 29, 2020 at 6:43 PM wind.fly@outlook.com <
wind.fly@outlook.com> wrote:

> Hi,all：
> 最近在升级flink1.11，sql中用到hbase connctor，发布到yarn-session时，报如下异常：
> 2020-07-29 11:49:55
> org.apache.hadoop.hbase.DoNotRetryIOException:
> java.lang.NoClassDefFoundError: Could not initialize class
> org.apache.hadoop.hbase.util.ByteStringer
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.translateException(RpcRetryingCaller.java:248)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:221)
> at
> org.apache.hadoop.hbase.client.ScannerCallableWithReplicas$RetryingRPC.call(ScannerCallableWithReplicas.java:388)
> at
> org.apache.hadoop.hbase.client.ScannerCallableWithReplicas$RetryingRPC.call(ScannerCallableWithReplicas.java:362)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:142)
> at
> org.apache.hadoop.hbase.client.ResultBoundedCompletionService$QueueingFuture.run(ResultBoundedCompletionService.java:80)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.lang.NoClassDefFoundError: Could not initialize class
> org.apache.hadoop.hbase.util.ByteStringer
> at
> org.apache.hadoop.hbase.protobuf.RequestConverter.buildRegionSpecifier(RequestConverter.java:1053)
> at
> org.apache.hadoop.hbase.protobuf.RequestConverter.buildScanRequest(RequestConverter.java:496)
> at
> org.apache.hadoop.hbase.client.ScannerCallable.openScanner(ScannerCallable.java:402)
> at
> org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:274)
> at
> org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:62)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:219)
> ... 7 more
>
> 注意到官方文档有Note:
> Note: To use HBase connector in SQL Client or Flink cluster, it’s highly
> recommended to add HBase dependency jars to Hadoop classpath. Flink will
> load all jars under Hadoop classpath automatically, please refer to HBase,
> MapReduce, and the CLASSPATH<
> https://hbase.apache.org/book.html#hbase.mapreduce.classpath> about how
> to add HBase dependency jars to Hadoop classpath.
>   但是在yarn session下怎么设置classpath呢
>

flink1.11 sql 发布到yarn session时找不到hbase相关的类

2020-07-29 文章 wind.fly....@outlook.com 
Hi,all：
最近在升级flink1.11，sql中用到hbase connctor，发布到yarn-session时，报如下异常：
2020-07-29 11:49:55
org.apache.hadoop.hbase.DoNotRetryIOException: java.lang.NoClassDefFoundError: 
Could not initialize class org.apache.hadoop.hbase.util.ByteStringer
at 
org.apache.hadoop.hbase.client.RpcRetryingCaller.translateException(RpcRetryingCaller.java:248)
at 
org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:221)
at 
org.apache.hadoop.hbase.client.ScannerCallableWithReplicas$RetryingRPC.call(ScannerCallableWithReplicas.java:388)
at 
org.apache.hadoop.hbase.client.ScannerCallableWithReplicas$RetryingRPC.call(ScannerCallableWithReplicas.java:362)
at 
org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:142)
at 
org.apache.hadoop.hbase.client.ResultBoundedCompletionService$QueueingFuture.run(ResultBoundedCompletionService.java:80)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NoClassDefFoundError: Could not initialize class 
org.apache.hadoop.hbase.util.ByteStringer
at 
org.apache.hadoop.hbase.protobuf.RequestConverter.buildRegionSpecifier(RequestConverter.java:1053)
at 
org.apache.hadoop.hbase.protobuf.RequestConverter.buildScanRequest(RequestConverter.java:496)
at 
org.apache.hadoop.hbase.client.ScannerCallable.openScanner(ScannerCallable.java:402)
at org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:274)
at org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:62)
at 
org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:219)
... 7 more

注意到官方文档有Note:
Note: To use HBase connector in SQL Client or Flink cluster, it’s highly 
recommended to add HBase dependency jars to Hadoop classpath. Flink will load 
all jars under Hadoop classpath automatically, please refer to HBase, 
MapReduce, and the 
CLASSPATH about 
how to add HBase dependency jars to Hadoop classpath.
  但是在yarn session下怎么设置classpath呢