Kai Chen created FLINK-23449:
--------------------------------
Summary: YarnTaskExecutorRunner does not contains MapReduce
classes
Key: FLINK-23449
URL: https://issues.apache.org/jira/browse/FLINK-23449
Project: Flink
Issue Type: Bug
Components: Deployment / YARN
Affects Versions: 1.11.3
Environment: flink-1.11
flink on yarn cluster
jdk1.8
hive1.2.1
hadoop2.7
hadoop classes is provided with {{export HADOOP_CLASSPATH=`hadoop classpath`
when submitting test APP. （described in
[https://ci.apache.org/projects/flink/flink-docs-release-1.11/ops/deployment/hadoop.html]
）}}
{{}}
Reporter: Kai Chen
I followed instructions described in
[https://ci.apache.org/projects/flink/flink-docs-release-1.11/dev/table/hive]
and tested hive streaming sink, met this exception
Caused by: java.lang.ClassNotFoundException:
org.apache.hadoop.mapred.JobConf
[http://apache-flink.147419.n8.nabble.com/Flink-td7866.html] met the same
problem.
I checked TM jvm envs and the code and found that flink only set up
YARN_APPLICATION_CLASSPATH, but without MAPREDUCE_APPLICATION_CLASSPATH.
See:
[https://github.com/apache/flink/blob/ed39fb2efc790af038c1babd4a48847b7b39f91e/flink-yarn/src/main/java/org/apache/flink/yarn/Utils.java#L119]
I think we should add MAPREDUCE_APPLICATION_CLASSPATH as well, as the same as
spark does.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
