[ https://issues.apache.org/jira/browse/YARN-2185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16338279#comment-16338279 ]
Miklos Szegedi commented on YARN-2185: -------------------------------------- Thank you, [~jlowe]. I updated the patch. {quote}makeShellPath is a pre-existing, public function,... {quote} If we want to be conservative with this jira I would be more conservative with security rather than compatibility. I used both Windows and Linux but I have never seen a legitimate path with ' or ". That said, I would rather throw an exception in these cases rather than trying to be backward compatible for existing malware. :) I changed the patch to have a new makeSecureShellPath function, and I ignore Windows, since it is dead code right now, and it is not so trivial to avoid code injection. {quote}Attempting to get the futures from the executor could result in an ExecutionException... {quote} I caught the only IOException thrown there but anyone can change the code later, so I refactored the it a little bit to protect this scenario. I also log now on the fly, to avoid OOM errors. Let me know, what you think. > Use pipes when localizing archives > ---------------------------------- > > Key: YARN-2185 > URL: https://issues.apache.org/jira/browse/YARN-2185 > Project: Hadoop YARN > Issue Type: Improvement > Components: nodemanager > Affects Versions: 2.4.0 > Reporter: Jason Lowe > Assignee: Miklos Szegedi > Priority: Major > Attachments: YARN-2185.000.patch, YARN-2185.001.patch, > YARN-2185.002.patch, YARN-2185.003.patch, YARN-2185.004.patch, > YARN-2185.005.patch, YARN-2185.006.patch, YARN-2185.007.patch, > YARN-2185.008.patch, YARN-2185.009.patch, YARN-2185.010.patch > > > Currently the nodemanager downloads an archive to a local file, unpacks it, > and then removes it. It would be more efficient to stream the data as it's > being unpacked to avoid both the extra disk space requirements and the > additional disk activity from storing the archive. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org