> 在 2018年8月3日，19:03，Joshua Fan <joshuafat...@gmail.com> 写道：
>
> Hi,
>
> I'd like to submit a job with dependency jars by flink run, but it failed.
>
> Here is the script,
>
> /usr/bin/hadoop/software/flink-1.4.2/bin/flink run \
> -m yarn-cluster -yn 1 -ys 8 -yjm 2148 -ytm 4096 -ynm jarsTest \
> -c StreamExample \
> -C file:/home/work/xxx/lib/commons-math3-3.5.jar \
> -C file:/home/work/xxx/lib/flink-connector-kafka-0.8_2.11-1.4.2.jar \
> ...
> xxx-1.0.jar
>
> As described in
> https://ci.apache.org/projects/flink/flink-docs-release-1.5/ops/cli.html#usage
> , "-C" means to provide the dependency jar.
>
> After I execute the command, the job succeed to submit, but can not run in
> flink cluster on yarn. Exceptions is like below:
>
> org.apache.flink.streaming.runtime.tasks.StreamTaskException: Cannot load
> user class: org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer08
> ClassLoader info: URL ClassLoader:
> file: '/home/work/xxx/lib/commons-math3-3.5.jar' (missing)
> file: '/home/work/xxx/lib/flink-connector-kafka-0.8_2.11-1.4.2.jar'
> (missing)
> .......
> Class not resolvable through given classloader.
> at
> org.apache.flink.streaming.api.graph.StreamConfig.getStreamOperator(StreamConfig.java:232)
> at
> org.apache.flink.streaming.runtime.tasks.OperatorChain.<init>(OperatorChain.java:95)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:231)
> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:718)
> at java.lang.Thread.run(Thread.java:745)
>
> It appears that the two dependency jar cannot be found in TaskManager, so I
> dig into the source code, from CliFrontend to PackagedProgram to
> ClusterClient to JobGraph. It seems like the dependency jars is put in
> classpath and userCodeClassLoader in PackagedProgram, but never upload to the
> BlobServer in JobGraph where the xxx-1.0.jar is uploaded.
>
> Am I missing something? In Flink 1.4.2, dependency jar is not supported?
>
> Hope someone can give me some hint.
>
> Appreciate it very mush.
>
>
> Yours Sincerely
>
> Joshua
>
>
>
Hi Joshua,
I think what you’re looking for is `-yt` option, which is used for distributing
a specified directory via YARN to the TaskManager nodes. And you can find its
description in the Flink client by executing `bin/flink`.
> -yt,--yarnship <arg> Ship files in the specified directory (t
> for transfer)
Best Regards,
Paul Lam
