[ https://issues.apache.org/jira/browse/FLINK-14626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16969267#comment-16969267 ]
Chesnay Schepler commented on FLINK-14626: ------------------------------------------ This would change existing behavior and could result in subtle bugs. I'm also a bit concerned about making the existing YARN code even more complicated (it is a minefield for bugs as is), in particular without a use-case that can't be summed up as "user error". > User jar packaged with hadoop dependencies may cause class conflit with > hadoop jars on yarn > ------------------------------------------------------------------------------------------- > > Key: FLINK-14626 > URL: https://issues.apache.org/jira/browse/FLINK-14626 > Project: Flink > Issue Type: Bug > Components: Deployment / YARN > Affects Versions: 1.9.1 > Reporter: Victor Wong > Priority: Major > > Currently, the yarn application classpath is placed behind Flink classpath > (including user jars), which will cause conflict if the user jar accidentally > included hadoop dependencies. > {code:java} > // org.apache.flink.yarn.Utils#setupYarnClassPath > public static void setupYarnClassPath(Configuration conf, Map<String, String> > appMasterEnv) { > addToEnvironment( > appMasterEnv, > Environment.CLASSPATH.name(), > appMasterEnv.get(ENV_FLINK_CLASSPATH)); > String[] applicationClassPathEntries = conf.getStrings( > YarnConfiguration.YARN_APPLICATION_CLASSPATH, > YarnConfiguration.DEFAULT_YARN_APPLICATION_CLASSPATH); > for (String c : applicationClassPathEntries) { > addToEnvironment(appMasterEnv, Environment.CLASSPATH.name(), c.trim()); > } > } > {code} > Maybe we should place the user jars behind yarn application classpath when > `org.apache.flink.yarn.configuration.YarnConfigOptions.UserJarInclusion` is > set to LAST, like this "flink-xxx.jar:hadoop-xxx.jar:user-xxx.jar". > -- This message was sent by Atlassian Jira (v8.3.4#803005)