[
https://issues.apache.org/jira/browse/HADOOP-18327?focusedWorklogId=796302&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-796302
]
ASF GitHub Bot logged work on HADOOP-18327:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 29/Jul/22 08:57
Start Date: 29/Jul/22 08:57
Worklog Time Spent: 10m
Work Description: ashutoshcipher commented on PR #4536:
URL: https://github.com/apache/hadoop/pull/4536#issuecomment-1199039873
> Hmm, hadoop_add_param does not work if the value contains `"`.
(`HADOOP_OPTS='-XX:OnOutOfMemoryError="kill -9 %p"'` instead of previous
`HADOOP_OPTS="-XX:OnOutOfMemoryError='kill -9 %p'"`) @ashutoshcipher
>
> ```
> $ export HADOOP_OPTS='-XX:OnOutOfMemoryError="kill -9 %p"'
> $ echo $HADOOP_OPTS
> -XX:OnOutOfMemoryError="kill -9 %p"
>
> $ bash -x -c 'export SHELLOPTS && bin/hadoop version' 2>&1 | less
> ...
> + hadoop_add_param HADOOP_OPTS yarn.log.dir
-Dyarn.log.dir=/home/rocky/dist/hadoop-3.4.0-SNAPSHOT/logs
> + [[ ! -XX:OnOutOfMemoryError="kill -9 %p" =~ yarn.log.dir ]]
> + eval 'HADOOP_OPTS="-XX:OnOutOfMemoryError="kill -9 %p"
-Dyarn.log.dir=/home/rocky/dist/hadoop-3.4.0-SNAPSHOT/logs"'
> ++ HADOOP_OPTS=-XX:OnOutOfMemoryError=kill
> ++ -9 '%p -Dyarn.log.dir=/home/rocky/dist/hadoop-3.4.0-SNAPSHOT/logs'
> /home/rocky/dist/hadoop-3.4.0-SNAPSHOT/bin/../libexec/hadoop-functions.sh:
line 1143: -9: command not found
> ...
> ```
@iwasakims
Prefix exec by eval in Hadoop bin scripts Prior to this change, if
HADOOP_OPTS contains any arguments that include a space, the command is not
parsed correctly. For example, if HADOOP_OPTS="... -XX:OnOutOfMemoryError="kill
-9 %p" ...", the bin/hadoop script will fail with the error "Unrecognized
option: -9". No amount of clever escaping of the quotes or spaces in the "kill
-9 %p" command will fix this. The only alternative appears to be to use 'eval'.
Switching to use 'eval' instead of 'exec' also works, but it results in an
intermediate bash process being left alive throughout the entire lifetime of
the Java proces being started. Using 'exec' prefixed by 'eval' as has been done
in this commit gets the best of both worlds, in that options with spaces are
parsed correctly, and you don't end up with an intermediate bash process as the
parent of the Java process.
Issue Time Tracking
-------------------
Worklog Id: (was: 796302)
Time Spent: 1h 50m (was: 1h 40m)
> Fix eval expression in hadoop-functions.sh
> ------------------------------------------
>
> Key: HADOOP-18327
> URL: https://issues.apache.org/jira/browse/HADOOP-18327
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 3.3.3
> Reporter: groot
> Assignee: groot
> Priority: Minor
> Labels: pull-request-available
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> Need to fix the eval expression.
> 1. Prefix exec by eval in Hadoop bin scripts Prior to this change, if
> HADOOP_OPTS contains any arguments that include a space, the command is not
> parsed correctly. For example, if HADOOP_OPTS="...
> -XX:OnOutOfMemoryError=\"kill -9 %p\" ...", the bin/hadoop script will fail
> with the error "Unrecognized option: -9". No amount of clever escaping of the
> quotes or spaces in the "kill -9 %p" command will fix this. The only
> alternative appears to be to use 'eval'. Switching to use 'eval' *instead of*
> 'exec' also works, but it results in an intermediate bash process being left
> alive throughout the entire lifetime of the Java proces being started. Using
> 'exec' prefixed by 'eval' as has been done in this commit gets the best of
> both worlds, in that options with spaces are parsed correctly, and you don't
> end up with an intermediate bash process as the parent of the Java process.
> 2. We can replace single quote with escape-char and a double quote.
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]