[ 
https://issues.apache.org/jira/browse/YARN-2185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16307944#comment-16307944
 ] 

Gergo Repas commented on YARN-2185:
-----------------------------------

Thanks [~miklos.szeg...@cloudera.com] for the patch, I like this improvement. I 
have a couple of comments and questions:
In {{FileUtil.runCommandOnStream()}}:
# the closing of {{process.getOutputStream()}} will not happen if there is an 
exception in the first {{org.apache.commons.io.IOUtils.copy(inputStream, 
process.getOutputStream());}} call.
# The process's outputstream may be closed before IOUtils.toString() has a 
chance to read from it on the executor thread.
# The std error stream is not closed.
# {{org.apache.commons.io.IOUtils.copy(inputStream, 
process.getOutputStream());}} appears twice: once before the process.waitFor() 
call, and once after - what's the reason for the second call?

In {{RunJar.unJarAndSave()}} there is no need to use multiple try blocks, a 
single try-with can handle multiple Closeable-s.

> Use pipes when localizing archives
> ----------------------------------
>
>                 Key: YARN-2185
>                 URL: https://issues.apache.org/jira/browse/YARN-2185
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: nodemanager
>    Affects Versions: 2.4.0
>            Reporter: Jason Lowe
>            Assignee: Miklos Szegedi
>         Attachments: YARN-2185.000.patch, YARN-2185.001.patch
>
>
> Currently the nodemanager downloads an archive to a local file, unpacks it, 
> and then removes it.  It would be more efficient to stream the data as it's 
> being unpacked to avoid both the extra disk space requirements and the 
> additional disk activity from storing the archive.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

Reply via email to