[
https://issues.apache.org/jira/browse/YARN-1340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Per Bergland updated YARN-1340:
-------------------------------
Description:
We found that our tests based on the ClusterMapReduceTestCase class failed when
the jenkins job contained spaces and were able to reproduce the error by just
renaming the project directory to contain a space character.
The failure happens when validatePaths method in LocalDirsHandlerService tries
to interpret the paths as URLs "new URL(dir)" and this fails.
https://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.6-alpha/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/LocalDirsHandlerService.java
The code in the MiniYARNCluster.prepareDirs method needs to be modified to
create properly escaped file://-based URLs instead of raw file paths OR the
receiving end in LocalDirsHandlerService needs to stop interpreting the
directories as urls. Since MiniYARNCluster is a test class I suspect that the
former needs to be done.
was:
We found that our tests based on the ClusterMapReduceTestCase class failed when
the jenkins job contained spaces and were able to reproduce the error by just
renaming the project directory to create a space character.
The failure happens when validatePaths method in LocalDirsHandlerService tries
to interpret the paths as URLs "new URL(dir)" and this fails.
https://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.6-alpha/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/LocalDirsHandlerService.java
The code in the MiniYARNCluster.prepareDirs method needs to be modified to
create properly escaped file://-based URLs instead of raw file paths OR the
receiving end in LocalDirsHandlerService needs to stop interpreting the
directories as urls. Since MiniYARNCluster is a test class I suspect that the
former needs to be done.
> MiniYARNCluster generates wrong style directories in
> YarnConfiguration.NM_LOCAL_DIR, causes tests to fail if path contains space
> --------------------------------------------------------------------------------------------------------------------------------
>
> Key: YARN-1340
> URL: https://issues.apache.org/jira/browse/YARN-1340
> Project: Hadoop YARN
> Issue Type: Bug
> Affects Versions: 2.0.6-alpha
> Environment: Mac OS X 10.8.5, CentOS 6.3
> Reporter: Per Bergland
>
> We found that our tests based on the ClusterMapReduceTestCase class failed
> when the jenkins job contained spaces and were able to reproduce the error by
> just renaming the project directory to contain a space character.
> The failure happens when validatePaths method in LocalDirsHandlerService
> tries to interpret the paths as URLs "new URL(dir)" and this fails.
> https://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.6-alpha/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/LocalDirsHandlerService.java
> The code in the MiniYARNCluster.prepareDirs method needs to be modified to
> create properly escaped file://-based URLs instead of raw file paths OR the
> receiving end in LocalDirsHandlerService needs to stop interpreting the
> directories as urls. Since MiniYARNCluster is a test class I suspect that the
> former needs to be done.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
