自定义Connector 报错

Wed, 30 Dec 2020 02:25:32 -0800 

HI，大家好：
我有一个应用场景，利用Flinksql读取Kafka数据写入Kudu，由于官方没有Kudu Connector，自定义了一个Kudu Sink 
Connector，在本地IDEA下测试可以 正常跑通；
可是将代码打包，并将kudu-client.jar的依赖包放置Flink的lib目录下之后，提交任务到集群报错如下：


java.lang.NoClassDefFoundError: com/stumbleupon/async/Callback
        at 
org.apache.kudu.client.AsyncKuduClient$AsyncKuduClientBuilder.build(AsyncKuduClient.java:2525)
 ~[kudu-client-1.9.0.jar:1.9.0]
        at 
org.apache.kudu.client.KuduClient$KuduClientBuilder.build(KuduClient.java:533) 
~[kudu-client-1.9.0.jar:1.9.0]
        at 
com.java.flink.sql.kudu.KuduSinkFunction.open(KuduSinkFunction.java:38) 
~[FlinkProject.jar:?]
        at 
org.apache.flink.api.common.functions.util.FunctionUtils.openFunction(FunctionUtils.java:36)
 ~[flink-dist_2.11-1.12.0.jar:1.12.0]
        at 
org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.open(AbstractUdfStreamOperator.java:102)
 ~[flink-dist_2.11-1.12.0.jar:1.12.0]
        at 
org.apache.flink.table.runtime.operators.sink.SinkOperator.open(SinkOperator.java:63)
 ~[flink-table-blink_2.11-1.12.0.jar:1.12.0]
        at 
org.apache.flink.streaming.runtime.tasks.OperatorChain.initializeStateAndOpenOperators(OperatorChain.java:401)
 ~[flink-dist_2.11-1.12.0.jar:1.12.0]
        at 
org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$beforeInvoke$2(StreamTask.java:507)
 ~[flink-dist_2.11-1.12.0.jar:1.12.0]
        at 
org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$SynchronizedStreamTaskActionExecutor.runThrowing(StreamTaskActionExecutor.java:92)
 ~[flink-dist_2.11-1.12.0.jar:1.12.0]
        at 
org.apache.flink.streaming.runtime.tasks.StreamTask.beforeInvoke(StreamTask.java:501)
 ~[flink-dist_2.11-1.12.0.jar:1.12.0]
        at 
org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:531) 
~[flink-dist_2.11-1.12.0.jar:1.12.0]
        at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:722) 
~[flink-dist_2.11-1.12.0.jar:1.12.0]
        at org.apache.flink.runtime.taskmanager.Task.run(Task.java:547) 
~[flink-dist_2.11-1.12.0.jar:1.12.0]
        at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_162]
        Suppressed: java.lang.NullPointerException
                at 
com.java.flink.sql.kudu.KuduSinkFunction.close(KuduSinkFunction.java:86) 
~[FlinkProject.jar:?]
                at 
org.apache.flink.api.common.functions.util.FunctionUtils.closeFunction(FunctionUtils.java:43)
 ~[flink-dist_2.11-1.12.0.jar:1.12.0]
                at 
org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.dispose(AbstractUdfStreamOperator.java:117)
 ~[flink-dist_2.11-1.12.0.jar:1.12.0]
                at 
org.apache.flink.streaming.runtime.tasks.StreamTask.disposeAllOperators(StreamTask.java:740)
 ~[flink-dist_2.11-1.12.0.jar:1.12.0]
                at 
org.apache.flink.streaming.runtime.tasks.StreamTask.runAndSuppressThrowable(StreamTask.java:720)
 ~[flink-dist_2.11-1.12.0.jar:1.12.0]
                at 
org.apache.flink.streaming.runtime.tasks.StreamTask.cleanUpInvoke(StreamTask.java:643)
 ~[flink-dist_2.11-1.12.0.jar:1.12.0]
                at 
org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:552) 
~[flink-dist_2.11-1.12.0.jar:1.12.0]
                at 
org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:722) 
~[flink-dist_2.11-1.12.0.jar:1.12.0]
                at org.apache.flink.runtime.taskmanager.Task.run(Task.java:547) 
~[flink-dist_2.11-1.12.0.jar:1.12.0]
                at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_162]
Caused by: java.lang.ClassNotFoundException: com.stumbleupon.async.Callback
        at java.net.URLClassLoader.findClass(URLClassLoader.java:381) 
~[?:1.8.0_162]
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[?:1.8.0_162]
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:338) 
~[?:1.8.0_162]
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:1.8.0_162]
        ... 14 more


不是很清楚这个 com.stumbleupon.async.Callback 类的依赖来自哪里，该怎么解决？
| |
superainbower
|
|
superainbo...@163.com
|
签名由网易邮箱大师定制

回复