[jira] [Commented] (YARN-109) .tmp file is not deleted for localized archives
[ https://issues.apache.org/jira/browse/YARN-109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17019411#comment-17019411 ] kailiu_dev commented on YARN-109: - [~hudson]I find a bug，NM has many dirs such as " nm-local-dir/filecache/1889677_tmp" not clear and leak on disk, do you know why? > .tmp file is not deleted for localized archives > --- > > Key: YARN-109 > URL: https://issues.apache.org/jira/browse/YARN-109 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 0.23.3, 2.0.0-alpha >Reporter: Jason Darrell Lowe >Assignee: Mayank Bansal >Priority: Major > Fix For: 0.23.7, 2.1.0-beta > > Attachments: YARN-109-trunk-1.patch, YARN-109-trunk-2.patch, > YARN-109-trunk-3.patch, YARN-109-trunk-4.patch, YARN-109-trunk-5.patch, > YARN-109-trunk.patch > > > When archives are localized they are initially created as a .tmp file and > unpacked from that file. However the .tmp file is not deleted afterwards. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-109) .tmp file is not deleted for localized archives
[ https://issues.apache.org/jira/browse/YARN-109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13614015#comment-13614015 ] Hudson commented on YARN-109: - Integrated in Hadoop-Mapreduce-trunk #1384 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1384/]) YARN-109. .tmp file is not deleted for localized archives (Mayank Bansal via bobby) (Revision 1460723) Result = FAILURE bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1460723 Files : * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/FSDownload.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestFSDownload.java > .tmp file is not deleted for localized archives > --- > > Key: YARN-109 > URL: https://issues.apache.org/jira/browse/YARN-109 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 0.23.3, 2.0.0-alpha >Reporter: Jason Lowe >Assignee: Mayank Bansal > Fix For: 3.0.0, 0.23.7, 2.0.5-beta > > Attachments: YARN-109-trunk-1.patch, YARN-109-trunk-2.patch, > YARN-109-trunk-3.patch, YARN-109-trunk-4.patch, YARN-109-trunk-5.patch, > YARN-109-trunk.patch > > > When archives are localized they are initially created as a .tmp file and > unpacked from that file. However the .tmp file is not deleted afterwards. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-109) .tmp file is not deleted for localized archives
[ https://issues.apache.org/jira/browse/YARN-109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13613749#comment-13613749 ] Hudson commented on YARN-109: - Integrated in Hadoop-Hdfs-trunk #1356 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1356/]) YARN-109. .tmp file is not deleted for localized archives (Mayank Bansal via bobby) (Revision 1460723) Result = FAILURE bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1460723 Files : * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/FSDownload.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestFSDownload.java > .tmp file is not deleted for localized archives > --- > > Key: YARN-109 > URL: https://issues.apache.org/jira/browse/YARN-109 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 0.23.3, 2.0.0-alpha >Reporter: Jason Lowe >Assignee: Mayank Bansal > Fix For: 3.0.0, 0.23.7, 2.0.5-beta > > Attachments: YARN-109-trunk-1.patch, YARN-109-trunk-2.patch, > YARN-109-trunk-3.patch, YARN-109-trunk-4.patch, YARN-109-trunk-5.patch, > YARN-109-trunk.patch > > > When archives are localized they are initially created as a .tmp file and > unpacked from that file. However the .tmp file is not deleted afterwards. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-109) .tmp file is not deleted for localized archives
[ https://issues.apache.org/jira/browse/YARN-109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13613732#comment-13613732 ] Hudson commented on YARN-109: - Integrated in Hadoop-Hdfs-0.23-Build #565 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/565/]) svn merge -c 1460723 FIXES: YARN-109. .tmp file is not deleted for localized archives (Mayank Bansal via bobby) (Revision 1460734) Result = UNSTABLE bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1460734 Files : * /hadoop/common/branches/branch-0.23/hadoop-yarn-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/FSDownload.java * /hadoop/common/branches/branch-0.23/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestFSDownload.java > .tmp file is not deleted for localized archives > --- > > Key: YARN-109 > URL: https://issues.apache.org/jira/browse/YARN-109 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 0.23.3, 2.0.0-alpha >Reporter: Jason Lowe >Assignee: Mayank Bansal > Fix For: 3.0.0, 0.23.7, 2.0.5-beta > > Attachments: YARN-109-trunk-1.patch, YARN-109-trunk-2.patch, > YARN-109-trunk-3.patch, YARN-109-trunk-4.patch, YARN-109-trunk-5.patch, > YARN-109-trunk.patch > > > When archives are localized they are initially created as a .tmp file and > unpacked from that file. However the .tmp file is not deleted afterwards. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-109) .tmp file is not deleted for localized archives
[ https://issues.apache.org/jira/browse/YARN-109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13613676#comment-13613676 ] Hudson commented on YARN-109: - Integrated in Hadoop-Yarn-trunk #167 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/167/]) YARN-109. .tmp file is not deleted for localized archives (Mayank Bansal via bobby) (Revision 1460723) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1460723 Files : * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/FSDownload.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestFSDownload.java > .tmp file is not deleted for localized archives > --- > > Key: YARN-109 > URL: https://issues.apache.org/jira/browse/YARN-109 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 0.23.3, 2.0.0-alpha >Reporter: Jason Lowe >Assignee: Mayank Bansal > Fix For: 3.0.0, 0.23.7, 2.0.5-beta > > Attachments: YARN-109-trunk-1.patch, YARN-109-trunk-2.patch, > YARN-109-trunk-3.patch, YARN-109-trunk-4.patch, YARN-109-trunk-5.patch, > YARN-109-trunk.patch > > > When archives are localized they are initially created as a .tmp file and > unpacked from that file. However the .tmp file is not deleted afterwards. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-109) .tmp file is not deleted for localized archives
[ https://issues.apache.org/jira/browse/YARN-109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13612757#comment-13612757 ] Hudson commented on YARN-109: - Integrated in Hadoop-trunk-Commit #3521 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/3521/]) YARN-109. .tmp file is not deleted for localized archives (Mayank Bansal via bobby) (Revision 1460723) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1460723 Files : * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/FSDownload.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestFSDownload.java > .tmp file is not deleted for localized archives > --- > > Key: YARN-109 > URL: https://issues.apache.org/jira/browse/YARN-109 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 0.23.3, 2.0.0-alpha >Reporter: Jason Lowe >Assignee: Mayank Bansal > Fix For: 3.0.0, 0.23.7, 2.0.5-beta > > Attachments: YARN-109-trunk-1.patch, YARN-109-trunk-2.patch, > YARN-109-trunk-3.patch, YARN-109-trunk-4.patch, YARN-109-trunk-5.patch, > YARN-109-trunk.patch > > > When archives are localized they are initially created as a .tmp file and > unpacked from that file. However the .tmp file is not deleted afterwards. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-109) .tmp file is not deleted for localized archives
[ https://issues.apache.org/jira/browse/YARN-109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13612732#comment-13612732 ] Robert Joseph Evans commented on YARN-109: -- The changes look good to me. The test seems a bit brittle, looking to be sure that a .tmp is not there, but I don't know any other way to do it. +1 I'll check it in. > .tmp file is not deleted for localized archives > --- > > Key: YARN-109 > URL: https://issues.apache.org/jira/browse/YARN-109 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 0.23.3, 2.0.0-alpha >Reporter: Jason Lowe >Assignee: Mayank Bansal > Attachments: YARN-109-trunk-1.patch, YARN-109-trunk-2.patch, > YARN-109-trunk-3.patch, YARN-109-trunk-4.patch, YARN-109-trunk-5.patch, > YARN-109-trunk.patch > > > When archives are localized they are initially created as a .tmp file and > unpacked from that file. However the .tmp file is not deleted afterwards. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-109) .tmp file is not deleted for localized archives
[ https://issues.apache.org/jira/browse/YARN-109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13609589#comment-13609589 ] Mayank Bansal commented on YARN-109: [~revans2] Thanks for the review. Do you mind committing it as well? Thanks, Mayank > .tmp file is not deleted for localized archives > --- > > Key: YARN-109 > URL: https://issues.apache.org/jira/browse/YARN-109 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 0.23.3, 2.0.0-alpha >Reporter: Jason Lowe >Assignee: Mayank Bansal > Attachments: YARN-109-trunk-1.patch, YARN-109-trunk-2.patch, > YARN-109-trunk-3.patch, YARN-109-trunk-4.patch, YARN-109-trunk-5.patch, > YARN-109-trunk.patch > > > When archives are localized they are initially created as a .tmp file and > unpacked from that file. However the .tmp file is not deleted afterwards. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-109) .tmp file is not deleted for localized archives
[
https://issues.apache.org/jira/browse/YARN-109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13609581#comment-13609581
]
Hadoop QA commented on YARN-109:
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12574891/YARN-109-trunk-5.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 tests included appear to have a timeout.{color}
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/563//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/563//console
This message is automatically generated.
> .tmp file is not deleted for localized archives
> ---
>
> Key: YARN-109
> URL: https://issues.apache.org/jira/browse/YARN-109
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Affects Versions: 0.23.3, 2.0.0-alpha
>Reporter: Jason Lowe
>Assignee: Mayank Bansal
> Attachments: YARN-109-trunk-1.patch, YARN-109-trunk-2.patch,
> YARN-109-trunk-3.patch, YARN-109-trunk-4.patch, YARN-109-trunk-5.patch,
> YARN-109-trunk.patch
>
>
> When archives are localized they are initially created as a .tmp file and
> unpacked from that file. However the .tmp file is not deleted afterwards.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-109) .tmp file is not deleted for localized archives
[ https://issues.apache.org/jira/browse/YARN-109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13609548#comment-13609548 ] Mayank Bansal commented on YARN-109: [~revans2] Sure I will file a new JIRA for that. Thanks, Mayank > .tmp file is not deleted for localized archives > --- > > Key: YARN-109 > URL: https://issues.apache.org/jira/browse/YARN-109 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 0.23.3, 2.0.0-alpha >Reporter: Jason Lowe >Assignee: Mayank Bansal > Attachments: YARN-109-trunk-1.patch, YARN-109-trunk-2.patch, > YARN-109-trunk-3.patch, YARN-109-trunk-4.patch, YARN-109-trunk.patch > > > When archives are localized they are initially created as a .tmp file and > unpacked from that file. However the .tmp file is not deleted afterwards. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-109) .tmp file is not deleted for localized archives
[
https://issues.apache.org/jira/browse/YARN-109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13609516#comment-13609516
]
Hadoop QA commented on YARN-109:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12574885/YARN-109-trunk-4.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 tests included appear to have a timeout.{color}
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common:
org.apache.hadoop.yarn.util.TestFSDownload
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/559//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/559//console
This message is automatically generated.
> .tmp file is not deleted for localized archives
> ---
>
> Key: YARN-109
> URL: https://issues.apache.org/jira/browse/YARN-109
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Affects Versions: 0.23.3, 2.0.0-alpha
>Reporter: Jason Lowe
>Assignee: Mayank Bansal
> Attachments: YARN-109-trunk-1.patch, YARN-109-trunk-2.patch,
> YARN-109-trunk-3.patch, YARN-109-trunk-4.patch, YARN-109-trunk.patch
>
>
> When archives are localized they are initially created as a .tmp file and
> unpacked from that file. However the .tmp file is not deleted afterwards.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-109) .tmp file is not deleted for localized archives
[ https://issues.apache.org/jira/browse/YARN-109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13609509#comment-13609509 ] Robert Joseph Evans commented on YARN-109: -- That is fine with me. My concern was mostly with Windows support. tar, zip, jar, etc. should be there, but bash may not be. So if you want to file a new JIRA that is fine, if not you can just wait for windows support to be merged in and see if it breaks. > .tmp file is not deleted for localized archives > --- > > Key: YARN-109 > URL: https://issues.apache.org/jira/browse/YARN-109 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 0.23.3, 2.0.0-alpha >Reporter: Jason Lowe >Assignee: Mayank Bansal > Attachments: YARN-109-trunk-1.patch, YARN-109-trunk-2.patch, > YARN-109-trunk-3.patch, YARN-109-trunk-4.patch, YARN-109-trunk.patch > > > When archives are localized they are initially created as a .tmp file and > unpacked from that file. However the .tmp file is not deleted afterwards. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-109) .tmp file is not deleted for localized archives
[ https://issues.apache.org/jira/browse/YARN-109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13609495#comment-13609495 ] Mayank Bansal commented on YARN-109: [~revans2] Thanks for the comments and review. Updating the new patch. For adding the jar/tar/zip files in to repo I am not sure at this moment as i see in other cases it is also generating it on the fly. However I think your point is valid. I will recommend to do that in separate JIRA for everything to maintain consistency. Thoughts? Thanks, Mayank > .tmp file is not deleted for localized archives > --- > > Key: YARN-109 > URL: https://issues.apache.org/jira/browse/YARN-109 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 0.23.3, 2.0.0-alpha >Reporter: Jason Lowe >Assignee: Mayank Bansal > Attachments: YARN-109-trunk-1.patch, YARN-109-trunk-2.patch, > YARN-109-trunk-3.patch, YARN-109-trunk.patch > > > When archives are localized they are initially created as a .tmp file and > unpacked from that file. However the .tmp file is not deleted afterwards. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-109) .tmp file is not deleted for localized archives
[ https://issues.apache.org/jira/browse/YARN-109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13609233#comment-13609233 ] omkar vinit joshi commented on YARN-109: [~mayank_bansal] Thanks for updating it. :) Now it looks good. > .tmp file is not deleted for localized archives > --- > > Key: YARN-109 > URL: https://issues.apache.org/jira/browse/YARN-109 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 0.23.3, 2.0.0-alpha >Reporter: Jason Lowe >Assignee: Mayank Bansal > Attachments: YARN-109-trunk-1.patch, YARN-109-trunk-2.patch, > YARN-109-trunk-3.patch, YARN-109-trunk.patch > > > When archives are localized they are initially created as a .tmp file and > unpacked from that file. However the .tmp file is not deleted afterwards. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-109) .tmp file is not deleted for localized archives
[ https://issues.apache.org/jira/browse/YARN-109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13608919#comment-13608919 ] Robert Joseph Evans commented on YARN-109: -- The findbugs is complaining that you are ignoring the return value of the delete call. It should not be a problem so either use the return value to log a warning when it fails or update the findbugs filter to filter out the error. The -1 for the test timeouts is caused by a bug in the script used to detect these, so you can either ignore it, or add a timeout to any @Test that appears in the patch file, including the ones you didn't add :(. In the test, please uncomment the lines to cleanup after the test. Are they causing a problem for the test to pass? or was it just for debugging? Also I personally would prefer to have a few small jar/tar/zip files checked into the repository instead of generating them on they fly for the test. It will speed up the test and have less dependencies on the system being set up with the exact commands, i.e. bash for windows support. Although if you don't feel like changing it I am fine with that too, most of those commands are used by FSDownload already so it is not that critical. > .tmp file is not deleted for localized archives > --- > > Key: YARN-109 > URL: https://issues.apache.org/jira/browse/YARN-109 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 0.23.3, 2.0.0-alpha >Reporter: Jason Lowe >Assignee: Mayank Bansal > Attachments: YARN-109-trunk-1.patch, YARN-109-trunk-2.patch, > YARN-109-trunk-3.patch, YARN-109-trunk.patch > > > When archives are localized they are initially created as a .tmp file and > unpacked from that file. However the .tmp file is not deleted afterwards. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-109) .tmp file is not deleted for localized archives
[
https://issues.apache.org/jira/browse/YARN-109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13608713#comment-13608713
]
Hadoop QA commented on YARN-109:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12574743/YARN-109-trunk-3.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:red}-1 one of tests included doesn't have a timeout.{color}
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:red}-1 findbugs{color}. The patch appears to introduce 1 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/555//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-YARN-Build/555//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-common.html
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/555//console
This message is automatically generated.
> .tmp file is not deleted for localized archives
> ---
>
> Key: YARN-109
> URL: https://issues.apache.org/jira/browse/YARN-109
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Affects Versions: 0.23.3, 2.0.0-alpha
>Reporter: Jason Lowe
>Assignee: Mayank Bansal
> Attachments: YARN-109-trunk-1.patch, YARN-109-trunk-2.patch,
> YARN-109-trunk-3.patch, YARN-109-trunk.patch
>
>
> When archives are localized they are initially created as a .tmp file and
> unpacked from that file. However the .tmp file is not deleted afterwards.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-109) .tmp file is not deleted for localized archives
[ https://issues.apache.org/jira/browse/YARN-109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13608705#comment-13608705 ] Mayank Bansal commented on YARN-109: [~ojoshi] There are no changes in call its just I formatted that with hadoop formatter. I modified the unpack for all the cases. I added couple of more test methods to test some more cases , Looks to me enough for this change. Not necessary to test all combinations. I add the timeouts. Thanks, Mayank > .tmp file is not deleted for localized archives > --- > > Key: YARN-109 > URL: https://issues.apache.org/jira/browse/YARN-109 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 0.23.3, 2.0.0-alpha >Reporter: Jason Lowe >Assignee: Mayank Bansal > Attachments: YARN-109-trunk-1.patch, YARN-109-trunk-2.patch, > YARN-109-trunk.patch > > > When archives are localized they are initially created as a .tmp file and > unpacked from that file. However the .tmp file is not deleted afterwards. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-109) .tmp file is not deleted for localized archives
[ https://issues.apache.org/jira/browse/YARN-109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13608430#comment-13608430 ] omkar vinit joshi commented on YARN-109: [~mayank_bansal] I looked at the patch. Even though it is not modifying call method logically but still I see changes there. Can we simply keep a flag to track all the places in unpack method where renameTo was called (To indicate not to delete file as it is already deleted). and towards the end before returning we can delete file based on flag value. For file deletion we can even use File.delete(). I guess this will make it cleaner and will keep the tmp file handling logic intact as we need to log an error message in case we are unable to delete the .tmp file. The code is still not handling situation where resource.getType() == ARCHIVE and it is .jar or if it is PATTERN and .zip file. Also I see that entire test code is only testing if the file is a tar file and ARCHIVE type but not for other extensions / TYPE. I think we need to test all the combinations ( .zip , .tar, .jar ) and it is specified as FILE, PATTERN or ARCHIVE. Also for the test I think you will have to add a timeout parameter. @Test(timeout=) to avoid -1. Please let me know your thoughts on this. > .tmp file is not deleted for localized archives > --- > > Key: YARN-109 > URL: https://issues.apache.org/jira/browse/YARN-109 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 0.23.3, 2.0.0-alpha >Reporter: Jason Lowe >Assignee: Mayank Bansal > Attachments: YARN-109-trunk-1.patch, YARN-109-trunk-2.patch, > YARN-109-trunk.patch > > > When archives are localized they are initially created as a .tmp file and > unpacked from that file. However the .tmp file is not deleted afterwards. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-109) .tmp file is not deleted for localized archives
[
https://issues.apache.org/jira/browse/YARN-109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13608390#comment-13608390
]
Hadoop QA commented on YARN-109:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12574666/YARN-109-trunk-2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:red}-1 one of tests included doesn't have a timeout.{color}
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/551//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/551//console
This message is automatically generated.
> .tmp file is not deleted for localized archives
> ---
>
> Key: YARN-109
> URL: https://issues.apache.org/jira/browse/YARN-109
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Affects Versions: 0.23.3, 2.0.0-alpha
>Reporter: Jason Lowe
>Assignee: Mayank Bansal
> Attachments: YARN-109-trunk-1.patch, YARN-109-trunk-2.patch,
> YARN-109-trunk.patch
>
>
> When archives are localized they are initially created as a .tmp file and
> unpacked from that file. However the .tmp file is not deleted afterwards.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-109) .tmp file is not deleted for localized archives
[ https://issues.apache.org/jira/browse/YARN-109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13608378#comment-13608378 ] Mayank Bansal commented on YARN-109: [~ojoshi] Good points. Adding another patch. Thanks, Mayank > .tmp file is not deleted for localized archives > --- > > Key: YARN-109 > URL: https://issues.apache.org/jira/browse/YARN-109 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 0.23.3, 2.0.0-alpha >Reporter: Jason Lowe >Assignee: Mayank Bansal > Attachments: YARN-109-trunk-1.patch, YARN-109-trunk-2.patch, > YARN-109-trunk.patch > > > When archives are localized they are initially created as a .tmp file and > unpacked from that file. However the .tmp file is not deleted afterwards. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-109) .tmp file is not deleted for localized archives
[
https://issues.apache.org/jira/browse/YARN-109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13607050#comment-13607050
]
Hadoop QA commented on YARN-109:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/1257/YARN-109-trunk-1.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:red}-1 one of tests included doesn't have a timeout.{color}
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/545//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/545//console
This message is automatically generated.
> .tmp file is not deleted for localized archives
> ---
>
> Key: YARN-109
> URL: https://issues.apache.org/jira/browse/YARN-109
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Affects Versions: 0.23.3, 2.0.0-alpha
>Reporter: Jason Lowe
>Assignee: Mayank Bansal
> Attachments: YARN-109-trunk-1.patch, YARN-109-trunk.patch
>
>
> When archives are localized they are initially created as a .tmp file and
> unpacked from that file. However the .tmp file is not deleted afterwards.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-109) .tmp file is not deleted for localized archives
[ https://issues.apache.org/jira/browse/YARN-109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13607042#comment-13607042 ] omkar vinit joshi commented on YARN-109: [~mayank_bansal] can we move the file deletion logic to the unpack method? I think that would be the right place to delete the file as it is already happening for resource.getType() == FILE. (during renameTo call). The current patch won't handle the situation where 1) user has specified the file type as ARCHIVE but the extension is neither of these (.jar,.tar,.tar.gz,.tgz) then it will add an incorrect error log message about file not deleted. 2) If the user has specified the file type as PATTERN then this will not remove the file. Let me know what are your thoughts. > .tmp file is not deleted for localized archives > --- > > Key: YARN-109 > URL: https://issues.apache.org/jira/browse/YARN-109 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 0.23.3, 2.0.0-alpha >Reporter: Jason Lowe >Assignee: Mayank Bansal > Attachments: YARN-109-trunk-1.patch, YARN-109-trunk.patch > > > When archives are localized they are initially created as a .tmp file and > unpacked from that file. However the .tmp file is not deleted afterwards. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-109) .tmp file is not deleted for localized archives
[
https://issues.apache.org/jira/browse/YARN-109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13607003#comment-13607003
]
Hadoop QA commented on YARN-109:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12574430/YARN-109-trunk.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:red}-1 one of tests included doesn't have a timeout.{color}
{color:red}-1 javac{color}. The applied patch generated 1362 javac
compiler warnings (more than the trunk's current 1361 warnings).
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:red}-1 eclipse:eclipse{color}. The patch failed to build with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/544//testReport/
Javac warnings:
https://builds.apache.org/job/PreCommit-YARN-Build/544//artifact/trunk/patchprocess/diffJavacWarnings.txt
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/544//console
This message is automatically generated.
> .tmp file is not deleted for localized archives
> ---
>
> Key: YARN-109
> URL: https://issues.apache.org/jira/browse/YARN-109
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Affects Versions: 0.23.3, 2.0.0-alpha
>Reporter: Jason Lowe
>Assignee: Mayank Bansal
> Attachments: YARN-109-trunk.patch
>
>
> When archives are localized they are initially created as a .tmp file and
> unpacked from that file. However the .tmp file is not deleted afterwards.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-109) .tmp file is not deleted for localized archives
[ https://issues.apache.org/jira/browse/YARN-109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13602898#comment-13602898 ] omkar vinit joshi commented on YARN-109: [~mayank_bansal] Are you still looking into this issue? or else I would like to take over. > .tmp file is not deleted for localized archives > --- > > Key: YARN-109 > URL: https://issues.apache.org/jira/browse/YARN-109 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 0.23.3, 2.0.0-alpha >Reporter: Jason Lowe >Assignee: Mayank Bansal > > When archives are localized they are initially created as a .tmp file and > unpacked from that file. However the .tmp file is not deleted afterwards. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
