[ https://issues.apache.org/jira/browse/YARN-6830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16407924#comment-16407924 ]
Jim Brennan commented on YARN-6830: ----------------------------------- I think this patch is ready for review. As I said above, this patch changes the regex to allow commas within a singe or double quoted string. It preserves the surrounding quotes though, so when you go to interpret a variable that has a list, you need to remove the surrounding quotes if they are there. I did this for the docker mount list cases. I think it's important to preserve the quotes to prevent breaking existing uses. I have not put up a patch yet for the alternative approach detailed in [YARN-8029]. cc: [~shaneku...@gmail.com], [~jlowe], [~templedf], [~aw] > Support quoted strings for environment variables > ------------------------------------------------ > > Key: YARN-6830 > URL: https://issues.apache.org/jira/browse/YARN-6830 > Project: Hadoop YARN > Issue Type: Bug > Reporter: Shane Kumpf > Assignee: Jim Brennan > Priority: Major > Attachments: YARN-6830.001.patch, YARN-6830.002.patch, > YARN-6830.003.patch, YARN-6830.004.patch > > > There are cases where it is necessary to allow for quoted string literals > within environment variables values when passed via the yarn command line > interface. > For example, consider the follow environment variables for a MR map task. > {{MODE=bar}} > {{IMAGE_NAME=foo}} > {{MOUNTS=/tmp/foo,/tmp/bar}} > When running the MR job, these environment variables are supplied as a comma > delimited string. > {{-Dmapreduce.map.env="MODE=bar,IMAGE_NAME=foo,MOUNTS=/tmp/foo,/tmp/bar"}} > In this case, {{MOUNTS}} will be parsed and added to the task environment as > {{MOUNTS=/tmp/foo}}. Any attempts to quote the embedded comma separated value > results in quote characters becoming part of the value, and parsing still > breaks down at the comma. > This issue is to allow for quoting the comma separated value (escaped double > or single quote). This was mentioned on YARN-4595 and will impact YARN-5534 > as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org