[ https://issues.apache.org/jira/browse/SPARK-28525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Marcelo Vanzin resolved SPARK-28525. ------------------------------------ Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 25265 [https://github.com/apache/spark/pull/25265] > Allow Launcher to be applied Java options > ----------------------------------------- > > Key: SPARK-28525 > URL: https://issues.apache.org/jira/browse/SPARK-28525 > Project: Spark > Issue Type: Improvement > Components: Deploy > Affects Versions: 3.0.0 > Reporter: Kousuke Saruta > Assignee: Kousuke Saruta > Priority: Minor > Fix For: 3.0.0 > > > Launcher is implemented as a Java application and sometimes I'd like to apply > Java options. > One situation I have met is the time I try to attach debugger to Launcher. > Launcher is launched from bin/spark-class but there is no room to apply Java > options. > {code} > build_command() { > "$RUNNER" -Xmx128m -cp "$LAUNCH_CLASSPATH" org.apache.spark.launcher.Main > "$@" > printf "%d\0" $? > } > {code} > Considering that it's not so many times to apply Java options to Launcher, > one compromise would just modify spark-class by user like as follows. > {code} > build_command() { > "$RUNNER" -Xmx128m $SPARK_LAUNCHER_OPTS -cp "$LAUNCH_CLASSPATH" > org.apache.spark.launcher.Main "$@" > printf "%d\0" $? > } > {code} > But it doesn't work when any text related to Java options is output to > standard output because whole output is used as command-string for > spark-shell and spark-submit in current implementation. > One example is jdwp. When apply agentlib option to use jdwp for debug, we > will get output like as follows. > {code} > Listening for transport dt_socket at address: 9876 > {code} > The output shown above is not a command-string so spark-submit and > spark-shell will fail. > To enable Java options for Launcher, we need treat command-string and others. -- This message was sent by Atlassian JIRA (v7.6.14#76016) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org