Re: flink时态表:两个Hbase左关联有报错情况
是不是加了 'lookup.async' = 'true',当 rowkey 为 null 的时候会出现这个问题 https://issues.apache.org/jira/browse/FLINK-24528 Michael Ran 于2021年7月23日周五 上午10:44写道: > java.util.concurrent.ExecutionException: java.lang.NoClassDefFoundError: > org/apache/flink/hbase/shaded/org/apache/commons/io/IOUtils缺jar > 在 2021-07-14 09:39:53,"xie_guo...@163.com" 写道: > >您好,有关flinkSQL时态表左关联时遇到了问题。 > >具体场景: > > > 两个Hbase表做时态表关联,左表定义了处理时间,右表原本不变(rowkey,cf),FLINK版本1.13.0,报错情况如下,麻烦问一下大家又遇到类似问题嘛,该怎么处理! > > > >2021-07-14 09:22:20.592 WARN org.apache.flink.runtime.taskmanager.Task > --- 2021-07-14 09:22:20.596 WARN > org.apache.flink.runtime.taskmanager.Task --- > LookupJoin(table=[default_catalog.default_database.hbase_source_pollution_dwb_enterprise_wodhz], > joinType=[LeftOuterJoin], async=[true], lookup=[code=$f4], select=[code, > data1, data2, p, $f4, code0, data]) -> Calc(select=[code, > ROW(,,data.activ) -> NotNullEnforcer(fields=[EXPR$1, EXPR$2, p, > EXPR$4]) (3/3)#3 (4ada188e117c67ccd9bd6488ae95216a) switched from RUNNING > to FAILED with failure cause: java.util.concurrent.ExecutionException: > java.lang.NoClassDefFoundError: > org/apache/flink/hbase/shaded/org/apache/commons/io/IOUtils > >at > java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357) > >at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1928 > >at > org.apache.flink.streaming.runtime.tasks.StreamOperatorWrapper.quiesceTimeServiceAndCloseOperator(StreamOperatorWrapper.java:168) > >at > org.apache.flink.streaming.runtime.tasks.StreamOperatorWrapper.close(StreamOperatorWrapper.java:131) > >at > org.apache.flink.streaming.runtime.tasks.OperatorChain.closeOperators(OperatorChain.java:448) > >at > org.apache.flink.streaming.runtime.tasks.StreamTask.afterInvoke(StreamTask.java:671) > >at > org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:629) > >at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:776) > >at org.apache.flink.runtime.taskmanager.Task.run(Task.java:563) > >at java.lang.Thread.run(Thread.java:748) > >Caused by: java.lang.NoClassDefFoundError: > org/apache/flink/hbase/shaded/org/apache/commons/io/IOUtils > >at > org.apache.flink.hbase.shaded.org.apache.hadoop.hbase.client.AsyncConnectionImpl.close(AsyncConnectionImpl.java:193) > >at > org.apache.flink.connector.hbase2.source.HBaseRowDataAsyncLookupFunction.close(HBaseRowDataAsyncLookupFunction.java:251) > >at LookupFunction$3.close(Unknown Source > > > > >ps:同样的代码,左表换成Kafka,能够正常运行。网上搜索了一下相关报错,好像时缺包,目前pom文件是有hbase-client、hbase-commen、flink-sql-connector-hbase。 > > > > > > > >Sincerely, > >xie_guo...@163.com >
Re: flink时态表:两个Hbase左关联有报错情况
Hi! flink classpath 下是否有 hbase connector 相关的 jar 包呢?如果用户代码里没有用到 hbase 相关的类和方法,其实用户代码的 pom 文件里是不需要这些依赖的。 xie_guo...@163.com 于2021年7月14日周三 上午9:43写道: > 您好,有关flinkSQL时态表左关联时遇到了问题。 > 具体场景: > > 两个Hbase表做时态表关联,左表定义了处理时间,右表原本不变(rowkey,cf),FLINK版本1.13.0,报错情况如下,麻烦问一下大家又遇到类似问题嘛,该怎么处理! > > 2021-07-14 09:22:20.592 WARN org.apache.flink.runtime.taskmanager.Task > --- 2021-07-14 09:22:20.596 WARN > org.apache.flink.runtime.taskmanager.Task --- > LookupJoin(table=[default_catalog.default_database.hbase_source_pollution_dwb_enterprise_wodhz], > joinType=[LeftOuterJoin], async=[true], lookup=[code=$f4], select=[code, > data1, data2, p, $f4, code0, data]) -> Calc(select=[code, > ROW(,,data.activ) -> NotNullEnforcer(fields=[EXPR$1, EXPR$2, p, > EXPR$4]) (3/3)#3 (4ada188e117c67ccd9bd6488ae95216a) switched from RUNNING > to FAILED with failure cause: java.util.concurrent.ExecutionException: > java.lang.NoClassDefFoundError: > org/apache/flink/hbase/shaded/org/apache/commons/io/IOUtils > at > java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357) > at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1928 > at > org.apache.flink.streaming.runtime.tasks.StreamOperatorWrapper.quiesceTimeServiceAndCloseOperator(StreamOperatorWrapper.java:168) > at > org.apache.flink.streaming.runtime.tasks.StreamOperatorWrapper.close(StreamOperatorWrapper.java:131) > at > org.apache.flink.streaming.runtime.tasks.OperatorChain.closeOperators(OperatorChain.java:448) > at > org.apache.flink.streaming.runtime.tasks.StreamTask.afterInvoke(StreamTask.java:671) > at > org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:629) > at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:776) > at org.apache.flink.runtime.taskmanager.Task.run(Task.java:563) > at java.lang.Thread.run(Thread.java:748) > Caused by: java.lang.NoClassDefFoundError: > org/apache/flink/hbase/shaded/org/apache/commons/io/IOUtils > at > org.apache.flink.hbase.shaded.org.apache.hadoop.hbase.client.AsyncConnectionImpl.close(AsyncConnectionImpl.java:193) > at > org.apache.flink.connector.hbase2.source.HBaseRowDataAsyncLookupFunction.close(HBaseRowDataAsyncLookupFunction.java:251) > at LookupFunction$3.close(Unknown Source > > > ps:同样的代码,左表换成Kafka,能够正常运行。网上搜索了一下相关报错,好像时缺包,目前pom文件是有hbase-client、hbase-commen、flink-sql-connector-hbase。 > > > > Sincerely, > xie_guo...@163.com >
flink时态表:两个Hbase左关联有报错情况
您好,有关flinkSQL时态表左关联时遇到了问题。 具体场景: 两个Hbase表做时态表关联,左表定义了处理时间,右表原本不变(rowkey,cf),FLINK版本1.13.0,报错情况如下,麻烦问一下大家又遇到类似问题嘛,该怎么处理! 2021-07-14 09:22:20.592 WARN org.apache.flink.runtime.taskmanager.Task --- 2021-07-14 09:22:20.596 WARN org.apache.flink.runtime.taskmanager.Task --- LookupJoin(table=[default_catalog.default_database.hbase_source_pollution_dwb_enterprise_wodhz], joinType=[LeftOuterJoin], async=[true], lookup=[code=$f4], select=[code, data1, data2, p, $f4, code0, data]) -> Calc(select=[code, ROW(,,data.activ) -> NotNullEnforcer(fields=[EXPR$1, EXPR$2, p, EXPR$4]) (3/3)#3 (4ada188e117c67ccd9bd6488ae95216a) switched from RUNNING to FAILED with failure cause: java.util.concurrent.ExecutionException: java.lang.NoClassDefFoundError: org/apache/flink/hbase/shaded/org/apache/commons/io/IOUtils at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357) at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1928 at org.apache.flink.streaming.runtime.tasks.StreamOperatorWrapper.quiesceTimeServiceAndCloseOperator(StreamOperatorWrapper.java:168) at org.apache.flink.streaming.runtime.tasks.StreamOperatorWrapper.close(StreamOperatorWrapper.java:131) at org.apache.flink.streaming.runtime.tasks.OperatorChain.closeOperators(OperatorChain.java:448) at org.apache.flink.streaming.runtime.tasks.StreamTask.afterInvoke(StreamTask.java:671) at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:629) at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:776) at org.apache.flink.runtime.taskmanager.Task.run(Task.java:563) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.NoClassDefFoundError: org/apache/flink/hbase/shaded/org/apache/commons/io/IOUtils at org.apache.flink.hbase.shaded.org.apache.hadoop.hbase.client.AsyncConnectionImpl.close(AsyncConnectionImpl.java:193) at org.apache.flink.connector.hbase2.source.HBaseRowDataAsyncLookupFunction.close(HBaseRowDataAsyncLookupFunction.java:251) at LookupFunction$3.close(Unknown Source ps:同样的代码,左表换成Kafka,能够正常运行。网上搜索了一下相关报错,好像时缺包,目前pom文件是有hbase-client、hbase-commen、flink-sql-connector-hbase。 Sincerely, xie_guo...@163.com