[ https://issues.apache.org/jira/browse/MAPREDUCE-4368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13401832#comment-13401832 ]
John Gordon commented on MAPREDUCE-4368: ---------------------------------------- On quotes with no spaces, here are two example input strings with an option that has quotes but no embedded spaces: [EX 1 -- problem] -Djava.library.path=/usr/lib/java/;/usr/lib64/java;"/opt/sun/sunstudio/somejars";/lib/java -esa This splits to: -Djava.library.path=/usr/lib/java/;/usr/lib64/java;"/opt/sun/sunstudio/somejars";/lib/java -esa then goes to the command line as: java "-Djava.library.path=/usr/lib/java/;/usr/lib64/java;"/opt/sun/sunstudio/somejars";/lib/java" "-esa" job.jar This still has the semicolon termination problem due to the " being present, it is interpreted by the shell parser (on win/linux) as: java "-Djava.library.path=/usr/lib/java/;/usr/lib64/java;"/opt/sun/sunstudio/somejars" <semicolon line break> /lib/java" "-esa" job.jar [EX 2 -- no problem] -Djava.library.path=/usr/lib/java/;/usr/lib64/java;"/opt/sun/sunstudio/somejars" -esa This splits to: -Djava.library.path=/usr/lib/java/;/usr/lib64/java;"/opt/sun/sunstudio/somejars" -esa then goes to the command line as: java "-Djava.library.path=/usr/lib/java/;/usr/lib64/java;"/opt/sun/sunstudio/somejars"" "-esa" job.jar In this case, the quotes are interpreted by the command-line parser wrong, but in a benign way and it has no effect. The two problematic cases for this code around quotes are: 1.) "string with spaces" 2.) "anything" [ ; | > && || ] -- any quoted string followed by a shell character that is implicitly or explicitly end-of-command > TaskRunner fails to start jars when the java.library.path contains a quoted > path with embedded spaces > ----------------------------------------------------------------------------------------------------- > > Key: MAPREDUCE-4368 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-4368 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: tasktracker > Affects Versions: 1-win > Environment: on Windows: > set PATH=%PATH%;"C:\this memorable place". > Reporter: John Gordon > Labels: newbie, patch > Attachments: TaskRunner.patch > > Original Estimate: 24h > Remaining Estimate: 24h > > TaskRunner splits arguments by space before it adds them back to the vargs > list, so it loses all context of quote escaped strings with embedded spaces. > This gets fixed up later by wrapping all arguments with " -- so you get > something like java "-D<opt>=<value>". This is problematic for paths with > embedded spaces, where we end up creating "-D<opt>=<first part" "last part>". > To java, the jar being run is last part. So with the environment above, you > will see "ClassNoDefFoundError: memorable" and the jar will fail to start. > In this particular case, we know that java.libarary.path contains paths and > the tests often use %PATH% to seed this, so the fix is to remove embedded > quotes in listed path elements because we know the aggregate will be quoted > when the JVM is started. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira