KevinSmile edited a comment on pull request #29653: URL: https://github.com/apache/spark/pull/29653#issuecomment-688031240
I updated my patch code and maybe the new version patch can explain my point better. The following snippet shows that **The first unrecognized arg is treated as the primaryResource**(use `Utils.resolveURI(opt).toString` to get app-jar in correct format ): https://github.com/apache/spark/blob/f5360e761ef161f7e04526b59a4baf53f1cf8cd5/core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala#L450-L470 Yes, when you do `run-example`, you just specify the class name(e.g. SparkPi), and no need to specify the app-jar. But in backend code, `appResource` should be auto-find-and-set as the examples' main app jar (e.g. `./examples/jars/spark-examples_2.12-3.0.0.jar`), and then be added as an arg ( this arg is the so-called `first unrecognized arg` and will later be used as the `primaryResource`). **So actually, app-jar arg is always needed in backend.** The bug is, the original backend code forgot to add app-jar, and so the first appArg(e.g. you use SparkPi example and set `100` as its first arg) will be treated as app-jar, check the` :: _jarUrl ::` part at the following snippet: https://github.com/apache/spark/blob/f55694638d45f34ab91f6f6ec2066cbf7631f4af/core/src/main/scala/org/apache/spark/deploy/ClientArguments.scala#L74-L89 In the original code, `spark-internal` is added, so `spark-internal` is is treated as primaryResource(aka app-jar).  `spark-internal` is useless in this case, but useful in many other cases (covered in some unit tests), so I prefer not to omit it here, or omit it only when you do `run-example`. P.S. Standalone-cluster is a working mode. https://github.com/apache/spark/blob/de44e9cfa07e32d293d68355916ac0dbd31d5c54/core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala#L297 https://github.com/apache/spark/blob/de44e9cfa07e32d293d68355916ac0dbd31d5c54/core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala#L685  ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
