[jira] [Updated] (YARN-5219) When an export var command fails in launch_container.sh, the full container launch should fail
[
https://issues.apache.org/jira/browse/YARN-5219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sunil G updated YARN-5219:
--
Attachment: YARN-5219.007.patch
Thanks [~suma.shivaprasad]
Updating new patch as per the comments
> When an export var command fails in launch_container.sh, the full container
> launch should fail
> --
>
> Key: YARN-5219
> URL: https://issues.apache.org/jira/browse/YARN-5219
> Project: Hadoop YARN
> Issue Type: Sub-task
>Affects Versions: 3.0.0-alpha1
>Reporter: Hitesh Shah
>Assignee: Sunil G
> Attachments: YARN-5219.001.patch, YARN-5219.003.patch,
> YARN-5219.004.patch, YARN-5219.005.patch, YARN-5219.006.patch,
> YARN-5219.007.patch, YARN-5219-branch-2.001.patch
>
>
> Today, a container fails if certain files fail to localize. However, if
> certain env vars fail to get setup properly either due to bugs in the yarn
> application or misconfiguration, the actual process launch still gets
> triggered. This results in either confusing error messages if the process
> fails to launch or worse yet the process launches but then starts behaving
> wrongly if the env var is used to control some behavioral aspects.
> In this scenario, the issue was reproduced by trying to do export
> abc="$\{foo.bar}" which is invalid as var names cannot contain "." in bash.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5219) When an export var command fails in launch_container.sh, the full container launch should fail
[
https://issues.apache.org/jira/browse/YARN-5219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vinod Kumar Vavilapalli updated YARN-5219:
--
Issue Type: Sub-task (was: Bug)
Parent: YARN-522
> When an export var command fails in launch_container.sh, the full container
> launch should fail
> --
>
> Key: YARN-5219
> URL: https://issues.apache.org/jira/browse/YARN-5219
> Project: Hadoop YARN
> Issue Type: Sub-task
>Affects Versions: 3.0.0-alpha1
>Reporter: Hitesh Shah
>Assignee: Sunil G
> Attachments: YARN-5219.001.patch, YARN-5219.003.patch,
> YARN-5219.004.patch, YARN-5219.005.patch, YARN-5219.006.patch,
> YARN-5219-branch-2.001.patch
>
>
> Today, a container fails if certain files fail to localize. However, if
> certain env vars fail to get setup properly either due to bugs in the yarn
> application or misconfiguration, the actual process launch still gets
> triggered. This results in either confusing error messages if the process
> fails to launch or worse yet the process launches but then starts behaving
> wrongly if the env var is used to control some behavioral aspects.
> In this scenario, the issue was reproduced by trying to do export
> abc="$\{foo.bar}" which is invalid as var names cannot contain "." in bash.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5219) When an export var command fails in launch_container.sh, the full container launch should fail
[
https://issues.apache.org/jira/browse/YARN-5219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sunil G updated YARN-5219:
--
Attachment: YARN-5219.006.patch
rebased patch.
[~vvasudev], could you please help to check
> When an export var command fails in launch_container.sh, the full container
> launch should fail
> --
>
> Key: YARN-5219
> URL: https://issues.apache.org/jira/browse/YARN-5219
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 3.0.0-alpha1
>Reporter: Hitesh Shah
>Assignee: Sunil G
> Attachments: YARN-5219.001.patch, YARN-5219.003.patch,
> YARN-5219.004.patch, YARN-5219.005.patch, YARN-5219.006.patch,
> YARN-5219-branch-2.001.patch
>
>
> Today, a container fails if certain files fail to localize. However, if
> certain env vars fail to get setup properly either due to bugs in the yarn
> application or misconfiguration, the actual process launch still gets
> triggered. This results in either confusing error messages if the process
> fails to launch or worse yet the process launches but then starts behaving
> wrongly if the env var is used to control some behavioral aspects.
> In this scenario, the issue was reproduced by trying to do export
> abc="$\{foo.bar}" which is invalid as var names cannot contain "." in bash.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5219) When an export var command fails in launch_container.sh, the full container launch should fail
[
https://issues.apache.org/jira/browse/YARN-5219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sunil G updated YARN-5219:
--
Attachment: YARN-5219.005.patch
Ok. "set -e" could be set for all file so that any script error will be thrown
out as an exception.
> When an export var command fails in launch_container.sh, the full container
> launch should fail
> --
>
> Key: YARN-5219
> URL: https://issues.apache.org/jira/browse/YARN-5219
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 3.0.0-alpha1
>Reporter: Hitesh Shah
>Assignee: Sunil G
> Attachments: YARN-5219.001.patch, YARN-5219.003.patch,
> YARN-5219.004.patch, YARN-5219.005.patch, YARN-5219-branch-2.001.patch
>
>
> Today, a container fails if certain files fail to localize. However, if
> certain env vars fail to get setup properly either due to bugs in the yarn
> application or misconfiguration, the actual process launch still gets
> triggered. This results in either confusing error messages if the process
> fails to launch or worse yet the process launches but then starts behaving
> wrongly if the env var is used to control some behavioral aspects.
> In this scenario, the issue was reproduced by trying to do export
> abc="$\{foo.bar}" which is invalid as var names cannot contain "." in bash.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5219) When an export var command fails in launch_container.sh, the full container launch should fail
[
https://issues.apache.org/jira/browse/YARN-5219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sunil G updated YARN-5219:
--
Affects Version/s: 3.0.0-alpha1
> When an export var command fails in launch_container.sh, the full container
> launch should fail
> --
>
> Key: YARN-5219
> URL: https://issues.apache.org/jira/browse/YARN-5219
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 3.0.0-alpha1
>Reporter: Hitesh Shah
>Assignee: Sunil G
> Attachments: YARN-5219.001.patch, YARN-5219.003.patch,
> YARN-5219.004.patch, YARN-5219-branch-2.001.patch
>
>
> Today, a container fails if certain files fail to localize. However, if
> certain env vars fail to get setup properly either due to bugs in the yarn
> application or misconfiguration, the actual process launch still gets
> triggered. This results in either confusing error messages if the process
> fails to launch or worse yet the process launches but then starts behaving
> wrongly if the env var is used to control some behavioral aspects.
> In this scenario, the issue was reproduced by trying to do export
> abc="$\{foo.bar}" which is invalid as var names cannot contain "." in bash.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5219) When an export var command fails in launch_container.sh, the full container launch should fail
[
https://issues.apache.org/jira/browse/YARN-5219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sunil G updated YARN-5219:
--
Attachment: YARN-5219.004.patch
Sorry for a lil long delay here. It fell off my radar.
Yes, its correct. We can use "set -e" for validating only env variables. After
the export section in launch_container.sh, i think its fine to add back "set
+e".
Updating a patch for same. Kindly review.
> When an export var command fails in launch_container.sh, the full container
> launch should fail
> --
>
> Key: YARN-5219
> URL: https://issues.apache.org/jira/browse/YARN-5219
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Hitesh Shah
>Assignee: Sunil G
> Attachments: YARN-5219-branch-2.001.patch, YARN-5219.001.patch,
> YARN-5219.003.patch, YARN-5219.004.patch
>
>
> Today, a container fails if certain files fail to localize. However, if
> certain env vars fail to get setup properly either due to bugs in the yarn
> application or misconfiguration, the actual process launch still gets
> triggered. This results in either confusing error messages if the process
> fails to launch or worse yet the process launches but then starts behaving
> wrongly if the env var is used to control some behavioral aspects.
> In this scenario, the issue was reproduced by trying to do export
> abc="$\{foo.bar}" which is invalid as var names cannot contain "." in bash.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5219) When an export var command fails in launch_container.sh, the full container launch should fail
[
https://issues.apache.org/jira/browse/YARN-5219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sunil G updated YARN-5219:
--
Attachment: YARN-5219-branch-2.001.patch
Attaching branch-2 patch. This will only print debug logs in the event of any
empty shell variable substitutions.
> When an export var command fails in launch_container.sh, the full container
> launch should fail
> --
>
> Key: YARN-5219
> URL: https://issues.apache.org/jira/browse/YARN-5219
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Hitesh Shah
>Assignee: Sunil G
> Attachments: YARN-5219-branch-2.001.patch, YARN-5219.001.patch,
> YARN-5219.003.patch
>
>
> Today, a container fails if certain files fail to localize. However, if
> certain env vars fail to get setup properly either due to bugs in the yarn
> application or misconfiguration, the actual process launch still gets
> triggered. This results in either confusing error messages if the process
> fails to launch or worse yet the process launches but then starts behaving
> wrongly if the env var is used to control some behavioral aspects.
> In this scenario, the issue was reproduced by trying to do export
> abc="$\{foo.bar}" which is invalid as var names cannot contain "." in bash.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5219) When an export var command fails in launch_container.sh, the full container launch should fail
[
https://issues.apache.org/jira/browse/YARN-5219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sunil G updated YARN-5219:
--
Attachment: YARN-5219.003.patch
Had offline discussion with [~vvasudev] and thought of doing below way.
Any shell variable substitution with *empty* value can be considered as error
and *launch_container.sh* can be failed. This can help to avoid any type of
unwanted and invalid variables which are exposed as env variables. Its better
to do this in trunk.
For branch-2, we can report such errors and log it for now. So script will have
some validators and will write errors to console. It can atleast help to debug
whether all env variables are cleanly exported or not.
Attaching script based on trunk solution. I am also attaching a trunk patch for
now. Will add branch-2 patch in a short while. pls share your thoughts.
{noformat}
#!/bin/bash
err_report() {
echo "Error on line ${1} while substituting for ${2}"
}
verify_shell_variable() {
echo "Shell variable ${1} is defined as ${2:? `err_report ${LINENO} ${1}`}"
}
export foo="2.4.6"
export testVar="version${foo}"
toValidate=${foo}
verify_shell_variable testVar ${toValidate}
exec /bin/bash -c ""
hadoop_shell_errorcode=$?
if [ $hadoop_shell_errorcode -ne 0 ]
then
exit $hadoop_shell_errorcode
fi
{noformat}
> When an export var command fails in launch_container.sh, the full container
> launch should fail
> --
>
> Key: YARN-5219
> URL: https://issues.apache.org/jira/browse/YARN-5219
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Hitesh Shah
>Assignee: Sunil G
> Attachments: YARN-5219.001.patch, YARN-5219.003.patch
>
>
> Today, a container fails if certain files fail to localize. However, if
> certain env vars fail to get setup properly either due to bugs in the yarn
> application or misconfiguration, the actual process launch still gets
> triggered. This results in either confusing error messages if the process
> fails to launch or worse yet the process launches but then starts behaving
> wrongly if the env var is used to control some behavioral aspects.
> In this scenario, the issue was reproduced by trying to do export
> abc="$\{foo.bar}" which is invalid as var names cannot contain "." in bash.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5219) When an export var command fails in launch_container.sh, the full container launch should fail
[
https://issues.apache.org/jira/browse/YARN-5219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sunil G updated YARN-5219:
--
Attachment: (was: YARN-5219.002.patch)
> When an export var command fails in launch_container.sh, the full container
> launch should fail
> --
>
> Key: YARN-5219
> URL: https://issues.apache.org/jira/browse/YARN-5219
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Hitesh Shah
>Assignee: Sunil G
> Attachments: YARN-5219.001.patch
>
>
> Today, a container fails if certain files fail to localize. However, if
> certain env vars fail to get setup properly either due to bugs in the yarn
> application or misconfiguration, the actual process launch still gets
> triggered. This results in either confusing error messages if the process
> fails to launch or worse yet the process launches but then starts behaving
> wrongly if the env var is used to control some behavioral aspects.
> In this scenario, the issue was reproduced by trying to do export
> abc="$\{foo.bar}" which is invalid as var names cannot contain "." in bash.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5219) When an export var command fails in launch_container.sh, the full container launch should fail
[
https://issues.apache.org/jira/browse/YARN-5219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sunil G updated YARN-5219:
--
Attachment: YARN-5219.002.patch
Attaching new patch with test case
> When an export var command fails in launch_container.sh, the full container
> launch should fail
> --
>
> Key: YARN-5219
> URL: https://issues.apache.org/jira/browse/YARN-5219
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Hitesh Shah
>Assignee: Sunil G
> Attachments: YARN-5219.001.patch, YARN-5219.002.patch
>
>
> Today, a container fails if certain files fail to localize. However, if
> certain env vars fail to get setup properly either due to bugs in the yarn
> application or misconfiguration, the actual process launch still gets
> triggered. This results in either confusing error messages if the process
> fails to launch or worse yet the process launches but then starts behaving
> wrongly if the env var is used to control some behavioral aspects.
> In this scenario, the issue was reproduced by trying to do export
> abc="$\{foo.bar}" which is invalid as var names cannot contain "." in bash.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5219) When an export var command fails in launch_container.sh, the full container launch should fail
[
https://issues.apache.org/jira/browse/YARN-5219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sunil G updated YARN-5219:
--
Attachment: YARN-5219.001.patch
Hi
Attaching an initial patch for review. I have tried to validate those shell
variables which has *$* character in it. Because this means there is a
substitution happening for that shell variable.
{{verify_shell_variable}} does the validation.
Attaching {{launc_container.sh}}
{noformat}
#!/bin/bash
verify_shell_variable() {
echo "Variable ${1} to be defined as ${2:?}"
}
export
HADOOP_CONF_DIR=${HADOOP_CONF_DIR:-"/opt/hadoop/trunk/hadoop-3.0.0-alpha1-SNAPSHOT/etc/hadoop"}
export JAVA_HOME=${JAVA_HOME:-"/usr/lib/jvm/java-8-oracle"}
export APP_SUBMIT_TIME_ENV="1465983732983"
export NM_HOST="localhost"
export LD_LIBRARY_PATH="$PWD:$HADOOP_COMMON_HOME/lib/native"
verify_shell_variable LD_LIBRARY_PATH ${LD_LIBRARY_PATH}
export
HADOOP_HDFS_HOME=${HADOOP_HDFS_HOME:-"/opt/hadoop/trunk/hadoop-3.0.0-alpha1-SNAPSHOT"}
export LOGNAME="root"
export JVM_PID="$$"
verify_shell_variable JVM_PID ${JVM_PID}
export HADOOP_MAPRED_HOME="/opt/hadoop/trunk/hadoop-3.0.0-alpha1-SNAPSHOT"
export
PWD="/Users/sunil/local/project_home/builds/tmp/nm-local-dir/usercache/root/appcache/application_1465983701333_0001/container_1465983701333_0001_01_01"
export
HADOOP_COMMON_HOME=${HADOOP_COMMON_HOME:-"/opt/hadoop/trunk/hadoop-3.0.0-alpha1-SNAPSHOT"}
export
LOCAL_DIRS="/Users/sunil/local/project_home/builds/tmp/nm-local-dir/usercache/root/appcache/application_1465983701333_0001"
export APPLICATION_WEB_PROXY_BASE="/proxy/application_1465983701333_0001"
export SHELL="/bin/bash"
export NM_HTTP_PORT="25008"
export
LOG_DIRS="/Users/sunil/local/project_home/builds/tmp/userlogs/application_1465983701333_0001/container_1465983701333_0001_01_01"
export
NM_AUX_SERVICE_mapreduce_shuffle="AAA0+gA=^M
"
export NM_PORT="25006"
export USER="root"
export
HADOOP_YARN_HOME=${HADOOP_YARN_HOME:-"/opt/hadoop/trunk/hadoop-3.0.0-alpha1-SNAPSHOT"}
export
CLASSPATH="$PWD:$HADOOP_CONF_DIR:$HADOOP_COMMON_HOME/share/hadoop/common/*:$HADOOP_COMMON_HOME/share/hadoop/common/lib/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*:$HADOOP_YARN_HOME/share/hadoop/yarn/*:$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*:$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*:$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*:job.jar/job.jar:job.jar/classes/:job.jar/lib/*:$PWD/*"
verify_shell_variable CLASSPATH ${CLASSPATH}
export
HADOOP_TOKEN_FILE_LOCATION="/Users/sunil/local/project_home/builds/tmp/nm-local-dir/usercache/root/appcache/application_1465983701333_0001/container_1465983701333_0001_01_01/container_tokens"
export YARN_RESOURCEMANAGER_APPLICATION_QUEUE="default"
export HOME="/home/"
export CONTAINER_ID="container_1465983701333_0001_01_01"
export MALLOC_ARENA_MAX="4"
ln -sf
"/Users/sunil/local/project_home/builds/tmp/nm-local-dir/usercache/root/appcache/application_1465983701333_0001/filecache/11/job.jar"
"job.jar"
hadoop_shell_errorcode=$?
if [ $hadoop_shell_errorcode -ne 0 ]
then
exit $hadoop_shell_errorcode
fi
mkdir -p jobSubmitDir
hadoop_shell_errorcode=$?
if [ $hadoop_shell_errorcode -ne 0 ]
then
exit $hadoop_shell_errorcode
fi
ln -sf
"/Users/sunil/local/project_home/builds/tmp/nm-local-dir/usercache/root/appcache/application_1465983701333_0001/filecache/12/job.split"
"jobSubmitDir/job.split"
hadoop_shell_errorcode=$?
if [ $hadoop_shell_errorcode -ne 0 ]
then
exit $hadoop_shell_errorcode
fi
ln -sf
"/Users/sunil/local/project_home/builds/tmp/nm-local-dir/usercache/root/appcache/application_1465983701333_0001/filecache/13/job.xml"
"job.xml"
hadoop_shell_errorcode=$?
if [ $hadoop_shell_errorcode -ne 0 ]
then
exit $hadoop_shell_errorcode
fi
mkdir -p jobSubmitDir
hadoop_shell_errorcode=$?
if [ $hadoop_shell_errorcode -ne 0 ]
then
exit $hadoop_shell_errorcode
fi
ln -sf
"/Users/sunil/local/project_home/builds/tmp/nm-local-dir/usercache/root/appcache/application_1465983701333_0001/filecache/10/job.splitmetainfo"
"jobSubmitDir/job.splitmetainfo"
hadoop_shell_errorcode=$?
if [ $hadoop_shell_errorcode -ne 0 ]
then
exit $hadoop_shell_errorcode
fi
# Creating copy of launch script
cp "launch_container.sh"
"/Users/sunil/local/project_home/builds/tmp/userlogs/application_1465983701333_0001/container_1465983701333_0001_01_01/launch_container.sh"
chmod 640
"/Users/sunil/local/project_home/builds/tmp/userlogs/application_1465983701333_0001/container_1465983701333_0001_01_01/launch_container.sh"
# Determining directory contents
echo "ls -l:"
1>"/Users/sunil/local/project_home/builds/tmp/userlogs/application_1465983701333_0001/container_1465983701333_0001_01_01/directory.info"
ls -l
1>>"/Users/sunil/local/project_home/builds/tmp/userlogs/application_1465983701333_0001/container_1465983701333_0001_01_01/directory.info"
echo "find -L . -maxdepth 5 -ls:"
1
[jira] [Updated] (YARN-5219) When an export var command fails in launch_container.sh, the full container launch should fail
[
https://issues.apache.org/jira/browse/YARN-5219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hitesh Shah updated YARN-5219:
--
Description:
Today, a container fails if certain files fail to localize. However, if certain
env vars fail to get setup properly either due to bugs in the yarn application
or misconfiguration, the actual process launch still gets triggered. This
results in either confusing error messages if the process fails to launch or
worse yet the process launches but then starts behaving wrongly if the env var
is used to control some behavioral aspects.
In this scenario, the issue was reproduced by trying to do export
abc="$\{foo.bar}" which is invalid as var names cannot contain "." in bash.
was:
Today, a container fails if certain files fail to localize. However, if certain
env vars fail to get setup properly either due to bugs in the yarn application
or misconfiguration, the actual process launch still gets triggered. This
results in either confusing error messages if the process fails to launch or
worse yet the process launches but then starts behaving wrongly if the env var
is used to control some behavioral aspects.
In this scenario, the issue was reproduced by trying to do export
abc="$\X{foo.bar}" which is invalid as var names cannot contain "." in bash.
> When an export var command fails in launch_container.sh, the full container
> launch should fail
> --
>
> Key: YARN-5219
> URL: https://issues.apache.org/jira/browse/YARN-5219
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Hitesh Shah
>
> Today, a container fails if certain files fail to localize. However, if
> certain env vars fail to get setup properly either due to bugs in the yarn
> application or misconfiguration, the actual process launch still gets
> triggered. This results in either confusing error messages if the process
> fails to launch or worse yet the process launches but then starts behaving
> wrongly if the env var is used to control some behavioral aspects.
> In this scenario, the issue was reproduced by trying to do export
> abc="$\{foo.bar}" which is invalid as var names cannot contain "." in bash.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5219) When an export var command fails in launch_container.sh, the full container launch should fail
[
https://issues.apache.org/jira/browse/YARN-5219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hitesh Shah updated YARN-5219:
--
Description:
Today, a container fails if certain files fail to localize. However, if certain
env vars fail to get setup properly either due to bugs in the yarn application
or misconfiguration, the actual process launch still gets triggered. This
results in either confusing error messages if the process fails to launch or
worse yet the process launches but then starts behaving wrongly if the env var
is used to control some behavioral aspects.
In this scenario, the issue was reproduced by trying to do export
abc="$\X{foo.bar}" which is invalid as var names cannot contain "." in bash.
was:
Today, a container fails if certain files fail to localize. However, if certain
env vars fail to get setup properly either due to bugs in the yarn application
or misconfiguration, the actual process launch still gets triggered. This
results in either confusing error messages if the process fails to launch or
worse yet the process launches but then starts behaving wrongly if the env var
is used to control some behavioral aspects.
In this scenario, the issue was reproduced by trying to do export
abc="${foo.bar}" which is invalid as var names cannot contain "." in bash.
> When an export var command fails in launch_container.sh, the full container
> launch should fail
> --
>
> Key: YARN-5219
> URL: https://issues.apache.org/jira/browse/YARN-5219
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Hitesh Shah
>
> Today, a container fails if certain files fail to localize. However, if
> certain env vars fail to get setup properly either due to bugs in the yarn
> application or misconfiguration, the actual process launch still gets
> triggered. This results in either confusing error messages if the process
> fails to launch or worse yet the process launches but then starts behaving
> wrongly if the env var is used to control some behavioral aspects.
> In this scenario, the issue was reproduced by trying to do export
> abc="$\X{foo.bar}" which is invalid as var names cannot contain "." in bash.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
