[
https://issues.apache.org/jira/browse/YARN-8071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jim Brennan updated YARN-8071:
------------------------------
Description:
YARN-6830 describes a problem where environment variables that contain commas
cannot be specified via {{-Dmapreduce.map.env}}.
For example:
{{-Dmapreduce.map.env="MODE=bar,IMAGE_NAME=foo,MOUNTS=/tmp/foo,/tmp/bar"}}
will set {{MOUNTS}} to {{/tmp/foo}}
In that Jira, [~aw] suggested that we change the API to provide a way to
specify environment variables individually, the same way that Spark does.
{quote}Rather than fight with a regex why not redefine the API instead?
-Dmapreduce.map.env.MODE=bar
-Dmapreduce.map.env.IMAGE_NAME=foo
-Dmapreduce.map.env.MOUNTS=/tmp/foo,/tmp/bar
...
e.g, mapreduce.map.env.[foo]=bar gets turned into foo=bar
This greatly simplifies the input validation needed and makes it clear what is
actually being defined.
{quote}
The mapreduce properties were dealt with in [MAPREDUCE-7069]. This Jira will
address the YARN properties.
was:
YARN-6830 describes a problem where environment variables that contain commas
cannot be specified via {{-Dmapreduce.map.env}}.
For example:
{{-Dmapreduce.map.env="MODE=bar,IMAGE_NAME=foo,MOUNTS=/tmp/foo,/tmp/bar"}}
will set {{MOUNTS}} to {{/tmp/foo}}
In that Jira, [~aw] suggested that we change the API to provide a way to
specify environment variables individually, the same way that Spark does.
{quote}Rather than fight with a regex why not redefine the API instead?
-Dmapreduce.map.env.MODE=bar
-Dmapreduce.map.env.IMAGE_NAME=foo
-Dmapreduce.map.env.MOUNTS=/tmp/foo,/tmp/bar
...
e.g, mapreduce.map.env.[foo]=bar gets turned into foo=bar
This greatly simplifies the input validation needed and makes it clear what is
actually being defined.
{quote}
> Add ability to specify nodemanager environment variables individually
> ---------------------------------------------------------------------
>
> Key: YARN-8071
> URL: https://issues.apache.org/jira/browse/YARN-8071
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn
> Affects Versions: 3.0.0
> Reporter: Jim Brennan
> Assignee: Jim Brennan
> Priority: Major
> Attachments: YARN-8071.001.patch
>
>
> YARN-6830 describes a problem where environment variables that contain commas
> cannot be specified via {{-Dmapreduce.map.env}}.
> For example:
> {{-Dmapreduce.map.env="MODE=bar,IMAGE_NAME=foo,MOUNTS=/tmp/foo,/tmp/bar"}}
> will set {{MOUNTS}} to {{/tmp/foo}}
> In that Jira, [~aw] suggested that we change the API to provide a way to
> specify environment variables individually, the same way that Spark does.
> {quote}Rather than fight with a regex why not redefine the API instead?
>
> -Dmapreduce.map.env.MODE=bar
> -Dmapreduce.map.env.IMAGE_NAME=foo
> -Dmapreduce.map.env.MOUNTS=/tmp/foo,/tmp/bar
> ...
> e.g, mapreduce.map.env.[foo]=bar gets turned into foo=bar
> This greatly simplifies the input validation needed and makes it clear what
> is actually being defined.
> {quote}
> The mapreduce properties were dealt with in [MAPREDUCE-7069]. This Jira will
> address the YARN properties.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
