[ https://issues.apache.org/jira/browse/YARN-8071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jim Brennan updated YARN-8071: ------------------------------ Description: YARN-6830 describes a problem where environment variables that contain commas cannot be specified via {{-Dmapreduce.map.env}}. For example: {{-Dmapreduce.map.env="MODE=bar,IMAGE_NAME=foo,MOUNTS=/tmp/foo,/tmp/bar"}} will set {{MOUNTS}} to {{/tmp/foo}} In that Jira, [~aw] suggested that we change the API to provide a way to specify environment variables individually, the same way that Spark does. {quote}Rather than fight with a regex why not redefine the API instead? -Dmapreduce.map.env.MODE=bar -Dmapreduce.map.env.IMAGE_NAME=foo -Dmapreduce.map.env.MOUNTS=/tmp/foo,/tmp/bar ... e.g, mapreduce.map.env.[foo]=bar gets turned into foo=bar This greatly simplifies the input validation needed and makes it clear what is actually being defined. {quote} The mapreduce properties were dealt with in [MAPREDUCE-7069]. This Jira will address the YARN properties. was: YARN-6830 describes a problem where environment variables that contain commas cannot be specified via {{-Dmapreduce.map.env}}. For example: {{-Dmapreduce.map.env="MODE=bar,IMAGE_NAME=foo,MOUNTS=/tmp/foo,/tmp/bar"}} will set {{MOUNTS}} to {{/tmp/foo}} In that Jira, [~aw] suggested that we change the API to provide a way to specify environment variables individually, the same way that Spark does. {quote}Rather than fight with a regex why not redefine the API instead? -Dmapreduce.map.env.MODE=bar -Dmapreduce.map.env.IMAGE_NAME=foo -Dmapreduce.map.env.MOUNTS=/tmp/foo,/tmp/bar ... e.g, mapreduce.map.env.[foo]=bar gets turned into foo=bar This greatly simplifies the input validation needed and makes it clear what is actually being defined. {quote} > Add ability to specify nodemanager environment variables individually > --------------------------------------------------------------------- > > Key: YARN-8071 > URL: https://issues.apache.org/jira/browse/YARN-8071 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn > Affects Versions: 3.0.0 > Reporter: Jim Brennan > Assignee: Jim Brennan > Priority: Major > Attachments: YARN-8071.001.patch > > > YARN-6830 describes a problem where environment variables that contain commas > cannot be specified via {{-Dmapreduce.map.env}}. > For example: > {{-Dmapreduce.map.env="MODE=bar,IMAGE_NAME=foo,MOUNTS=/tmp/foo,/tmp/bar"}} > will set {{MOUNTS}} to {{/tmp/foo}} > In that Jira, [~aw] suggested that we change the API to provide a way to > specify environment variables individually, the same way that Spark does. > {quote}Rather than fight with a regex why not redefine the API instead? > > -Dmapreduce.map.env.MODE=bar > -Dmapreduce.map.env.IMAGE_NAME=foo > -Dmapreduce.map.env.MOUNTS=/tmp/foo,/tmp/bar > ... > e.g, mapreduce.map.env.[foo]=bar gets turned into foo=bar > This greatly simplifies the input validation needed and makes it clear what > is actually being defined. > {quote} > The mapreduce properties were dealt with in [MAPREDUCE-7069]. This Jira will > address the YARN properties. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org