[ https://issues.apache.org/jira/browse/YARN-2185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16307944#comment-16307944 ]
Gergo Repas commented on YARN-2185: ----------------------------------- Thanks [~miklos.szeg...@cloudera.com] for the patch, I like this improvement. I have a couple of comments and questions: In {{FileUtil.runCommandOnStream()}}: # the closing of {{process.getOutputStream()}} will not happen if there is an exception in the first {{org.apache.commons.io.IOUtils.copy(inputStream, process.getOutputStream());}} call. # The process's outputstream may be closed before IOUtils.toString() has a chance to read from it on the executor thread. # The std error stream is not closed. # {{org.apache.commons.io.IOUtils.copy(inputStream, process.getOutputStream());}} appears twice: once before the process.waitFor() call, and once after - what's the reason for the second call? In {{RunJar.unJarAndSave()}} there is no need to use multiple try blocks, a single try-with can handle multiple Closeable-s. > Use pipes when localizing archives > ---------------------------------- > > Key: YARN-2185 > URL: https://issues.apache.org/jira/browse/YARN-2185 > Project: Hadoop YARN > Issue Type: Improvement > Components: nodemanager > Affects Versions: 2.4.0 > Reporter: Jason Lowe > Assignee: Miklos Szegedi > Attachments: YARN-2185.000.patch, YARN-2185.001.patch > > > Currently the nodemanager downloads an archive to a local file, unpacks it, > and then removes it. It would be more efficient to stream the data as it's > being unpacked to avoid both the extra disk space requirements and the > additional disk activity from storing the archive. -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org