[jira] [Commented] (HBASE-6949) Automatically delete empty directories in CleanerChore
[ https://issues.apache.org/jira/browse/HBASE-6949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13480543#comment-13480543 ] Hudson commented on HBASE-6949: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #228 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/228/]) HBASE-6949 Automatically delete empty directories in CleanerChore (Revision 1400286) Result = FAILURE stack : Files : * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/cleaner/CleanerChore.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/master/cleaner/TestCleanerChore.java > Automatically delete empty directories in CleanerChore > -- > > Key: HBASE-6949 > URL: https://issues.apache.org/jira/browse/HBASE-6949 > Project: HBase > Issue Type: Bug >Affects Versions: 0.94.3, 0.96.0 >Reporter: Jesse Yates >Assignee: Jesse Yates > Fix For: 0.96.0 > > Attachments: hbase-6949-v0.patch, hbase-6949-v1.patch > > > Currently the CleanerChore asks cleaner delegates if both directories and > files should be deleted. However, this leads to somewhat odd behavior in some > delegates - you don't actually care if the directory hierarchy is preserved, > the files; this means you always will delete directories and then implement > the logic you actually want for preserving files. Instead we can handle this > logic one layer higher in the CleanerChore and let the delegates just worry > about preserving files. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6949) Automatically delete empty directories in CleanerChore
[ https://issues.apache.org/jira/browse/HBASE-6949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13480493#comment-13480493 ] Hudson commented on HBASE-6949: --- Integrated in HBase-TRUNK #3464 (See [https://builds.apache.org/job/HBase-TRUNK/3464/]) HBASE-6949 Automatically delete empty directories in CleanerChore (Revision 1400286) Result = FAILURE stack : Files : * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/cleaner/CleanerChore.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/master/cleaner/TestCleanerChore.java > Automatically delete empty directories in CleanerChore > -- > > Key: HBASE-6949 > URL: https://issues.apache.org/jira/browse/HBASE-6949 > Project: HBase > Issue Type: Bug >Affects Versions: 0.94.3, 0.96.0 >Reporter: Jesse Yates >Assignee: Jesse Yates > Fix For: 0.96.0 > > Attachments: hbase-6949-v0.patch, hbase-6949-v1.patch > > > Currently the CleanerChore asks cleaner delegates if both directories and > files should be deleted. However, this leads to somewhat odd behavior in some > delegates - you don't actually care if the directory hierarchy is preserved, > the files; this means you always will delete directories and then implement > the logic you actually want for preserving files. Instead we can handle this > logic one layer higher in the CleanerChore and let the delegates just worry > about preserving files. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6949) Automatically delete empty directories in CleanerChore
[ https://issues.apache.org/jira/browse/HBASE-6949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13480392#comment-13480392 ] Jesse Yates commented on HBASE-6949: Thanks [~stack]! I was going to roll this into the the backport of HBASE-5547 to 0.94 - I'll bug Lars when that's ready. > Automatically delete empty directories in CleanerChore > -- > > Key: HBASE-6949 > URL: https://issues.apache.org/jira/browse/HBASE-6949 > Project: HBase > Issue Type: Bug >Affects Versions: 0.94.3, 0.96.0 >Reporter: Jesse Yates >Assignee: Jesse Yates > Fix For: 0.96.0 > > Attachments: hbase-6949-v0.patch, hbase-6949-v1.patch > > > Currently the CleanerChore asks cleaner delegates if both directories and > files should be deleted. However, this leads to somewhat odd behavior in some > delegates - you don't actually care if the directory hierarchy is preserved, > the files; this means you always will delete directories and then implement > the logic you actually want for preserving files. Instead we can handle this > logic one layer higher in the CleanerChore and let the delegates just worry > about preserving files. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6949) Automatically delete empty directories in CleanerChore
[ https://issues.apache.org/jira/browse/HBASE-6949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13480288#comment-13480288 ] Jesse Yates commented on HBASE-6949: To follow up on stack's comment, we don't actually have any directory heirarchy for the logs right now - all logs regardless of source (e.g. .logs/[servername]/[hlog]) just get moved to the .oldlogs directory. And since we just added the HFileCleaner there aren't any existing implementations that can be broken by automatically removing empty directories. > Automatically delete empty directories in CleanerChore > -- > > Key: HBASE-6949 > URL: https://issues.apache.org/jira/browse/HBASE-6949 > Project: HBase > Issue Type: Bug >Affects Versions: 0.94.3, 0.96.0 >Reporter: Jesse Yates >Assignee: Jesse Yates > Fix For: 0.94.3, 0.96.0 > > Attachments: hbase-6949-v0.patch, hbase-6949-v1.patch > > > Currently the CleanerChore asks cleaner delegates if both directories and > files should be deleted. However, this leads to somewhat odd behavior in some > delegates - you don't actually care if the directory hierarchy is preserved, > the files; this means you always will delete directories and then implement > the logic you actually want for preserving files. Instead we can handle this > logic one layer higher in the CleanerChore and let the delegates just worry > about preserving files. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6949) Automatically delete empty directories in CleanerChore
[ https://issues.apache.org/jira/browse/HBASE-6949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13477207#comment-13477207 ] Jesse Yates commented on HBASE-6949: [~stack], [~lhofhansl] what do you guys think? Good to go? > Automatically delete empty directories in CleanerChore > -- > > Key: HBASE-6949 > URL: https://issues.apache.org/jira/browse/HBASE-6949 > Project: HBase > Issue Type: Bug >Affects Versions: 0.94.3, 0.96.0 >Reporter: Jesse Yates >Assignee: Jesse Yates > Fix For: 0.94.3, 0.96.0 > > Attachments: hbase-6949-v0.patch, hbase-6949-v1.patch > > > Currently the CleanerChore asks cleaner delegates if both directories and > files should be deleted. However, this leads to somewhat odd behavior in some > delegates - you don't actually care if the directory hierarchy is preserved, > the files; this means you always will delete directories and then implement > the logic you actually want for preserving files. Instead we can handle this > logic one layer higher in the CleanerChore and let the delegates just worry > about preserving files. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6949) Automatically delete empty directories in CleanerChore
[ https://issues.apache.org/jira/browse/HBASE-6949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13475424#comment-13475424 ] Jesse Yates commented on HBASE-6949: bq. I think I understand. Delegates just do files and its the cleaner chore that does the dir remove? Is that it? Yes, that is a much better explanation than the one I gave. Do you want an RB of the patch? > Automatically delete empty directories in CleanerChore > -- > > Key: HBASE-6949 > URL: https://issues.apache.org/jira/browse/HBASE-6949 > Project: HBase > Issue Type: Bug >Affects Versions: 0.94.3, 0.96.0 >Reporter: Jesse Yates >Assignee: Jesse Yates > Fix For: 0.94.3, 0.96.0 > > Attachments: hbase-6949-v0.patch, hbase-6949-v1.patch > > > Currently the CleanerChore asks cleaner delegates if both directories and > files should be deleted. However, this leads to somewhat odd behavior in some > delegates - you don't actually care if the directory hierarchy is preserved, > the files; this means you always will delete directories and then implement > the logic you actually want for preserving files. Instead we can handle this > logic one layer higher in the CleanerChore and let the delegates just worry > about preserving files. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6949) Automatically delete empty directories in CleanerChore
[
https://issues.apache.org/jira/browse/HBASE-6949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13473906#comment-13473906
]
stack commented on HBASE-6949:
--
I like this addition Jesse:
{code}
+// if all the children have been deleted, then we should try to delete
this directory. However,
+// don't do so recursively so we don't delete files that have been added
since we checked.
{code}
The tests look good.
I'm not clear on this "...However, this leads to somewhat odd behavior in some
delegates - you don't actually care if the directory hierarchy is preserved,
the files; this means you always will delete directories and then implement the
logic you actually want for preserving files..." I think I understand.
Delegates just do files and its the cleaner chore that does the dir remove? Is
that it?
> Automatically delete empty directories in CleanerChore
> --
>
> Key: HBASE-6949
> URL: https://issues.apache.org/jira/browse/HBASE-6949
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.94.3, 0.96.0
>Reporter: Jesse Yates
>Assignee: Jesse Yates
> Fix For: 0.94.3, 0.96.0
>
> Attachments: hbase-6949-v0.patch, hbase-6949-v1.patch
>
>
> Currently the CleanerChore asks cleaner delegates if both directories and
> files should be deleted. However, this leads to somewhat odd behavior in some
> delegates - you don't actually care if the directory hierarchy is preserved,
> the files; this means you always will delete directories and then implement
> the logic you actually want for preserving files. Instead we can handle this
> logic one layer higher in the CleanerChore and let the delegates just worry
> about preserving files.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6949) Automatically delete empty directories in CleanerChore
[ https://issues.apache.org/jira/browse/HBASE-6949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13469823#comment-13469823 ] Lars Hofhansl commented on HBASE-6949: -- I like the simplified logic you describe. Need to look at the patch... > Automatically delete empty directories in CleanerChore > -- > > Key: HBASE-6949 > URL: https://issues.apache.org/jira/browse/HBASE-6949 > Project: HBase > Issue Type: Bug >Affects Versions: 0.94.3, 0.96.0 >Reporter: Jesse Yates >Assignee: Jesse Yates > Fix For: 0.94.3, 0.96.0 > > Attachments: hbase-6949-v0.patch, hbase-6949-v1.patch > > > Currently the CleanerChore asks cleaner delegates if both directories and > files should be deleted. However, this leads to somewhat odd behavior in some > delegates - you don't actually care if the directory hierarchy is preserved, > the files; this means you always will delete directories and then implement > the logic you actually want for preserving files. Instead we can handle this > logic one layer higher in the CleanerChore and let the delegates just worry > about preserving files. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6949) Automatically delete empty directories in CleanerChore
[
https://issues.apache.org/jira/browse/HBASE-6949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13469740#comment-13469740
]
Hadoop QA commented on HBASE-6949:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12547816/hbase-6949-v1.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated
81 warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to introduce 5 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.master.TestSplitLogManager
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/3005//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3005//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3005//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3005//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3005//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3005//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/3005//console
This message is automatically generated.
> Automatically delete empty directories in CleanerChore
> --
>
> Key: HBASE-6949
> URL: https://issues.apache.org/jira/browse/HBASE-6949
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.94.3, 0.96.0
>Reporter: Jesse Yates
>Assignee: Jesse Yates
> Fix For: 0.94.3, 0.96.0
>
> Attachments: hbase-6949-v0.patch, hbase-6949-v1.patch
>
>
> Currently the CleanerChore asks cleaner delegates if both directories and
> files should be deleted. However, this leads to somewhat odd behavior in some
> delegates - you don't actually care if the directory hierarchy is preserved,
> the files; this means you always will delete directories and then implement
> the logic you actually want for preserving files. Instead we can handle this
> logic one layer higher in the CleanerChore and let the delegates just worry
> about preserving files.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6949) Automatically delete empty directories in CleanerChore
[
https://issues.apache.org/jira/browse/HBASE-6949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13469718#comment-13469718
]
Hadoop QA commented on HBASE-6949:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12547809/hbase-6949-v0.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated
81 warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to introduce 5 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/3004//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3004//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3004//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3004//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3004//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3004//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/3004//console
This message is automatically generated.
> Automatically delete empty directories in CleanerChore
> --
>
> Key: HBASE-6949
> URL: https://issues.apache.org/jira/browse/HBASE-6949
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.94.3, 0.96.0
>Reporter: Jesse Yates
>Assignee: Jesse Yates
> Fix For: 0.94.3, 0.96.0
>
> Attachments: hbase-6949-v0.patch, hbase-6949-v1.patch
>
>
> Currently the CleanerChore asks cleaner delegates if both directories and
> files should be deleted. However, this leads to somewhat odd behavior in some
> delegates - you don't actually care if the directory hierarchy is preserved,
> the files; this means you always will delete directories and then implement
> the logic you actually want for preserving files. Instead we can handle this
> logic one layer higher in the CleanerChore and let the delegates just worry
> about preserving files.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6949) Automatically delete empty directories in CleanerChore
[ https://issues.apache.org/jira/browse/HBASE-6949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13469637#comment-13469637 ] Jesse Yates commented on HBASE-6949: Attached is the 0.96 version. The 0.94.3 version is part of the backport of HBASE-5547 and will be rolled into that backport, just marking it for some history there. > Automatically delete empty directories in CleanerChore > -- > > Key: HBASE-6949 > URL: https://issues.apache.org/jira/browse/HBASE-6949 > Project: HBase > Issue Type: Bug >Affects Versions: 0.94.3, 0.96.0 >Reporter: Jesse Yates >Assignee: Jesse Yates > Fix For: 0.94.3, 0.96.0 > > Attachments: hbase-6949-v0.patch > > > Currently the CleanerChore asks cleaner delegates if both directories and > files should be deleted. However, this leads to somewhat odd behavior in some > delegates - you don't actually care if the directory hierarchy is preserved, > the files; this means you always will delete directories and then implement > the logic you actually want for preserving files. Instead we can handle this > logic one layer higher in the CleanerChore and let the delegates just worry > about preserving files. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
