[ https://issues.apache.org/jira/browse/SPARK-2722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15557158#comment-15557158 ]
holdenk commented on SPARK-2722: -------------------------------- I think at this point trying to change the escaping of the different configuration escaping mechanisms is likely to lead to more broken issues than we avoid - how do people feel about closing this? > Mechanism for escaping spark configs is not consistent > ------------------------------------------------------ > > Key: SPARK-2722 > URL: https://issues.apache.org/jira/browse/SPARK-2722 > Project: Spark > Issue Type: Bug > Components: Spark Core > Affects Versions: 1.0.1 > Reporter: Andrew Or > > Currently, you can specify a spark config in spark-defaults.conf as follows: > {code} > spark.magic "Mr. Johnson" > {code} > and this will preserve the double quotes as part of the string. Naturally, if > you want to do the equivalent in spark.*.extraJavaOptions, you would use the > following: > {code} > spark.executor.extraJavaOptions "-Dmagic=\"Mr. Johnson\"" > {code} > However, this fails because the backslashes go away and it tries to interpret > "Johnson" as the main class argument. Instead, you have to do the following: > {code} > spark.executor.extraJavaOptions "-Dmagic=\\\"Mr. Johnson\\\"" > {code} > which is not super intuitive. > Note that this only applies to standalone mode. In YARN it's not even > possible to use quoted strings in config values (SPARK-2718). -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org