[
https://issues.apache.org/jira/browse/YARN-9008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16712143#comment-16712143
]
Haibo Chen commented on YARN-9008:
----------------------------------
Thanks [~pbacsko] for the patch! A few minor comments
1) We are missing one unit test for upload a non-existent file and one for a
directory.
2) The new commandline option 'appname' should probably be renamed to
'app_name' for the sake of consistency with other options
3) All IOExceptions are wrapped in a RunTimeException. But I am not sure why
benefits it provides than just directly throwing IOException.
4) I notice 2.9.1 is included in the affect version. Do you intend to backport
this into branch-2? If so, we shall not use stream api that is only supported
in Java 8.
5) The relative path of a file is composed of the app_name, appId and the file
name. We have two copies of the same code in both ApplicationMaster and Client.
If only one copy is changed in the future, the feature would fail. Can we
centralize them in one place?
6) 'localized_files' sounds very much into the implementation details.
MapReduce jobs client can add lib files at submission time, which are under the
hood uploaded to HDFS and localized for access. We have almost the same idea
here. What do you think of renaming it to 'lib'?
> Extend YARN distributed shell with file localization feature
> ------------------------------------------------------------
>
> Key: YARN-9008
> URL: https://issues.apache.org/jira/browse/YARN-9008
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: yarn
> Affects Versions: 2.9.1, 3.1.1
> Reporter: Peter Bacsko
> Assignee: Peter Bacsko
> Priority: Major
> Attachments: YARN-9008-001.patch, YARN-9008-002.patch,
> YARN-9008-003.patch, YARN-9008-004.patch
>
>
> YARN distributed shell is a very handy tool to test various features of YARN.
> However, it lacks support for file localization - that is, you define files
> in the command line that you wish to be localized remotely. This can be
> extremely useful in certain scenarios.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
