Hi folks, In my spark application, executor task depends on snakeyaml-1.10.jar I build it with Maven and it works fine: spark-submit --master local --jars d:\snakeyaml-1.10.jar ...........
But when I try to run it in yarn, I have issue, it seems spark executor cannot find the jar file: spark-submit --master yarn-cluster --jars hdfs://....../snakeyaml-1.10.jar .......... java.lang.NoSuchMethodError: org.yaml.snakeyaml.Yaml.<init>(Lorg/yaml/snakeyaml/constructor/BaseConstructor;)V I check one executor container folder, snakeyaml-1.10.jar has been successfully downloaded, and in spark driver page, in environment tab, spark.yarn.secondary.jars also contains snakeyaml-1.10.jar I have no idea why it doesn't work. Could some one help to take a look? Thanks
