李伟高 created FLINK-24346: --------------------------- Summary: Flink on yarn application mode,LinkageError Key: FLINK-24346 URL: https://issues.apache.org/jira/browse/FLINK-24346 Project: Flink Issue Type: Bug Components: API / DataStream Affects Versions: 1.13.1 Environment: hadoop version 2.6.x Reporter: 李伟高
Hello, I'm changing from per job mode to application mode to submit tasks to yarn.All jars that my task depends on are typed into my task jar.I submit the task as perjob and work normally, but change to application mode and report an error. {code:java} [0;39mjava.util.concurrent.CompletionException: org.apache.flink.client.deployment.application.ApplicationExecutionException: Could not execute application. at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:292) ~[na:1.8.0_271] at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:308) ~[na:1.8.0_271] at java.util.concurrent.CompletableFuture.uniCompose(CompletableFuture.java:957) ~[na:1.8.0_271] at java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:940) ~[na:1.8.0_271] at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) ~[na:1.8.0_271] at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1990) ~[na:1.8.0_271] at org.apache.flink.client.deployment.application.ApplicationDispatcherBootstrap.runApplicationEntryPoint(ApplicationDispatcherBootstrap.java:257) ~[hb-bigdata-wash-location-1.0-SNAPSHOT.jar:na] at org.apache.flink.client.deployment.application.ApplicationDispatcherBootstrap.lambda$runApplicationAsync$1(ApplicationDispatcherBootstrap.java:212) ~[hb-bigdata-wash-location-1.0-SNAPSHOT.jar:na] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[na:1.8.0_271] at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[na:1.8.0_271] at org.apache.flink.runtime.concurrent.akka.ActorSystemScheduledExecutorAdapter$ScheduledFutureTask.run(ActorSystemScheduledExecutorAdapter.java:159) ~[hb-bigdata-wash-location-1.0-SNAPSHOT.jar:na] at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:40) ~[hb-bigdata-wash-location-1.0-SNAPSHOT.jar:na] at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:44) ~[hb-bigdata-wash-location-1.0-SNAPSHOT.jar:na] at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) ~[hb-bigdata-wash-location-1.0-SNAPSHOT.jar:na] at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) ~[hb-bigdata-wash-location-1.0-SNAPSHOT.jar:na] at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) ~[hb-bigdata-wash-location-1.0-SNAPSHOT.jar:na] at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) ~[hb-bigdata-wash-location-1.0-SNAPSHOT.jar:na] Caused by: org.apache.flink.client.deployment.application.ApplicationExecutionException: Could not execute application. ... 11 common frames omitted Caused by: java.lang.LinkageError: loader constraint violation: loader (instance of org/apache/flink/util/ChildFirstClassLoader) previously initiated loading for a different type with name "org/elasticsearch/client/RestClientBuilder" at java.lang.ClassLoader.defineClass1(Native Method) ~[na:1.8.0_271] at java.lang.ClassLoader.defineClass(ClassLoader.java:756) ~[na:1.8.0_271] at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) ~[na:1.8.0_271] at java.net.URLClassLoader.defineClass(URLClassLoader.java:468) ~[na:1.8.0_271] at java.net.URLClassLoader.access$100(URLClassLoader.java:74) ~[na:1.8.0_271] at java.net.URLClassLoader$1.run(URLClassLoader.java:369) ~[na:1.8.0_271] at java.net.URLClassLoader$1.run(URLClassLoader.java:363) ~[na:1.8.0_271] at java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_271] at java.net.URLClassLoader.findClass(URLClassLoader.java:362) ~[na:1.8.0_271] at org.apache.flink.util.ChildFirstClassLoader.loadClassWithoutExceptionHandling(ChildFirstClassLoader.java:71) ~[hb-bigdata-wash-location-1.0-SNAPSHOT.jar:na] at org.apache.flink.util.FlinkUserCodeClassLoader.loadClass(FlinkUserCodeClassLoader.java:48) ~[hb-bigdata-wash-location-1.0-SNAPSHOT.jar:na] at java.lang.ClassLoader.loadClass(ClassLoader.java:351) ~[na:1.8.0_271] at java.lang.Class.getDeclaredMethods0(Native Method) ~[na:1.8.0_271] at java.lang.Class.privateGetDeclaredMethods(Class.java:2701) ~[na:1.8.0_271] at java.lang.Class.getDeclaredMethod(Class.java:2128) ~[na:1.8.0_271] at java.io.ObjectStreamClass.getPrivateMethod(ObjectStreamClass.java:1644) ~[na:1.8.0_271] at java.io.ObjectStreamClass.access$1700(ObjectStreamClass.java:79) ~[na:1.8.0_271] at java.io.ObjectStreamClass$3.run(ObjectStreamClass.java:520) ~[na:1.8.0_271] at java.io.ObjectStreamClass$3.run(ObjectStreamClass.java:494) ~[na:1.8.0_271] at java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_271] {code} I checked a lot of relevant information and told me to modify the {code:java} classloader.resolve-order {code} parameter. I tried back and forth from `parent-first` to `child-first`.The error is still reported. I think this question should be asked on stackoverflow or other websites. I have tried many times,I don't know what's wrong. I test the example you gave. It works well. Shouldn't the jar that my project depends on be typed into my task jar? If so, why is per job or yen session OK, We look forward to your reply , Thanks! -- This message was sent by Atlassian Jira (v8.3.4#803005)