[
https://issues.apache.org/jira/browse/YARN-2929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14256390#comment-14256390
]
Chris Nauroth commented on YARN-2929:
-------------------------------------
[~ozawa], thank you for providing the additional details.
Right now, the typical workflow is that the application submission context
controls the Java classpath by setting the {{CLASSPATH}} environment variable.
If we take the example of MapReduce, the relevant code is in {{YARNRunner}} and
{{MRApps}}. There is support in there for handling environment variables in
cross-platform application submission. However, even putting that aside, I
believe there is no problem with using '/' for a Windows job submission if you
use this technique. The NodeManager ultimately translates the classpath
through {{Path}} and bundles the whole classpath into a jar file manifest to be
referenced by the running container.
I believe the only problem shown in the example is that the application
submission is trying to set classpath by command-line argument. Is it possible
to switch to using the {{CLASSPATH}} environment variable technique, similar to
how the MapReduce code does it? If yes, then there is no need for the proposed
patch.
There are also some other potential issues with trying to pass classpath on the
command line in Windows. It's very easy to hit the Windows maximum command
line length limitation of 8191 characters. NodeManager already has the logic
to work around this by bundling the classpath into a jar file manifest, and
you'd get that for free by using the environment variable technique.
> Adding separator ApplicationConstants.FILE_PATH_SEPARATOR for better Windows
> support
> ------------------------------------------------------------------------------------
>
> Key: YARN-2929
> URL: https://issues.apache.org/jira/browse/YARN-2929
> Project: Hadoop YARN
> Issue Type: Improvement
> Reporter: Tsuyoshi OZAWA
> Assignee: Tsuyoshi OZAWA
> Attachments: YARN-2929.001.patch
>
>
> Some frameworks like Spark is tackling to run jobs on Windows(SPARK-1825).
> For better multiple platform support, we should introduce
> ApplicationConstants.FILE_PATH_SEPARATOR for making filepath
> platform-independent.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
