[ https://issues.apache.org/jira/browse/SPARK-25995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16682762#comment-16682762 ]
Felix Cheung commented on SPARK-25995: -------------------------------------- sparkR is just taking the whole string as-is [https://github.com/apache/spark/blob/141953f4c44dbad1c2a7059e92bec5fe770af932/R/pkg/R/client.R#L59] you see sparkSubmitOpts is before args (args is the file with port number) I think we should avoid duplicating the submit arg parsing in R, which we would need to break before {code:java} fooarg {code} ? Is it easier/better to always set the temp file with port as the last arg instead? > sparkR should ensure user args are after the argument used for the port > ----------------------------------------------------------------------- > > Key: SPARK-25995 > URL: https://issues.apache.org/jira/browse/SPARK-25995 > Project: Spark > Issue Type: Bug > Components: SparkR > Affects Versions: 2.3.2 > Reporter: Thomas Graves > Priority: Minor > > Currently if you run sparkR and accidentally specify an argument, it fails > with a useless error message. For example: > $SPARK_HOME/bin/sparkR --master yarn --deploy-mode client fooarg > This gets turned into: > Launching java with spark-submit command spark-submit "--master" "yarn" > "--deploy-mode" "client" "sparkr-shell" "fooarg" > /tmp/Rtmp6XBGz2/backend_port162806ea36bca > Notice that "fooarg" got put before /tmp file which is how R and jvm know > which port to connect to. SparkR eventually fails with timeout exception > after 10 seconds. > > SparkR should either not allow args or make sure the order is correct so the > backend_port is always first. see > https://github.com/apache/spark/blob/master/R/pkg/R/sparkR.R#L129 -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org