[jira] [Commented] (HADOOP-16140) Add emptyTrash option to purge trash immediately
[ https://issues.apache.org/jira/browse/HADOOP-16140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16784471#comment-16784471 ] Steve Loughran commented on HADOOP-16140: - its unrelated..seen it before sometimes. +1, committing to trunk and branch-3.2 > Add emptyTrash option to purge trash immediately > > > Key: HADOOP-16140 > URL: https://issues.apache.org/jira/browse/HADOOP-16140 > Project: Hadoop Common > Issue Type: Improvement > Components: fs >Affects Versions: 3.3.0 >Reporter: Stephen O'Donnell >Assignee: Stephen O'Donnell >Priority: Major > Attachments: HADOOP-14200.002.patch, HADOOP-14200.003.patch, > HADOOP-14200.004.patch, HADOOP-14200.005.patch, HADOOP-14200.006.patch, > HDFS-14200.001.patch > > > I have always felt the HDFS trash is missing a simple way to empty the > current users trash immediately. We have "expunge" but in my experience > supporting clusters, end users find this confusing. When most end users run > expunge, they really want to empty their trash immediately and get confused > when expunge does not do this. > This can result in users performing somewhat dangerous "skipTrash" operations > on the trash to free up space. The alternative, which most users will not > figure out on their own is: > # Run the expunge command once - this will move the current folder to a > checkpoint and remove any old checkpoints older than the retention interval > # Wait over 1 minute and then run expunge again, overriding fs.trash.interval > to 1 minute using the following command hadoop fs -Dfs.trash.interval=1 > -expunge. > With this Jira I am proposing to add a extra command, "hdfs dfs -emptyTrash" > that purges everything in the logged in users Trash directories immediately. > How would the community feel about adding this new option? I will upload a > patch for comments. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16140) Add emptyTrash option to purge trash immediately
[
https://issues.apache.org/jira/browse/HADOOP-16140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16784401#comment-16784401
]
Stephen O'Donnell commented on HADOOP-16140:
I don't think the test failure is related to this patch:
{code}
ERROR] Tests run: 22, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 111.542
s <<< FAILURE! - in org.apache.hadoop.ha.TestZKFailoverController
[ERROR]
testGracefulFailoverMultipleZKfcs(org.apache.hadoop.ha.TestZKFailoverController)
Time elapsed: 70.285 s <<< ERROR!
org.apache.hadoop.ha.ServiceFailedException: Unable to become active. Local
node did not get an opportunity to do so from ZooKeeper, or the local node took
too long to transition to active.
at
org.apache.hadoop.ha.ZKFailoverController.doGracefulFailover(ZKFailoverController.java:692)
at
org.apache.hadoop.ha.ZKFailoverController.access$400(ZKFailoverController.java:60)
at
org.apache.hadoop.ha.ZKFailoverController$3.run(ZKFailoverController.java:609)
at
org.apache.hadoop.ha.ZKFailoverController$3.run(ZKFailoverController.java:606)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1891)
at
org.apache.hadoop.ha.ZKFailoverController.gracefulFailoverToYou(ZKFailoverController.java:606)
at
org.apache.hadoop.ha.ZKFCRpcServer.gracefulFailover(ZKFCRpcServer.java:94)
at
org.apache.hadoop.ha.TestZKFailoverController.testGracefulFailoverMultipleZKfcs(TestZKFailoverController.java:627)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:55)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.rules.TestWatchman$1.evaluate(TestWatchman.java:53)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.lang.Thread.run(Thread.java:748)
{code}
> Add emptyTrash option to purge trash immediately
>
>
> Key: HADOOP-16140
> URL: https://issues.apache.org/jira/browse/HADOOP-16140
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs
>Affects Versions: 3.3.0
>Reporter: Stephen O'Donnell
>Assignee: Stephen O'Donnell
>Priority: Major
> Attachments: HADOOP-14200.002.patch, HADOOP-14200.003.patch,
> HADOOP-14200.004.patch, HADOOP-14200.005.patch, HADOOP-14200.006.patch,
> HDFS-14200.001.patch
>
>
> I have always felt the HDFS trash is missing a simple way to empty the
> current users trash immediately. We have "expunge" but in my experience
> supporting clusters, end users find this confusing. When most end users run
> expunge, they really want to empty their trash immediately and get confused
> when expunge does not do this.
> This can result in users performing somewhat dangerous "skipTrash" operations
> on the trash to free up space. The alternative, which most users will not
> figure out on their own is:
> # Run the expunge command once - this will move the current folder to a
> checkpoint and remove any old checkpoints older than the retention interval
> # Wait over 1 minute and then run expunge again, overriding fs.trash.interval
> to 1 minute using the following command hadoop fs -Dfs.trash.interval=1
> -expunge.
> With this Jira I am proposing to add a extra command, "hdfs dfs -emptyTrash"
> that purges everything in the logged in users Trash directories immediately.
> How would the community feel about adding this new option? I will upload a
> patch for comments.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To
[jira] [Commented] (HADOOP-16140) Add emptyTrash option to purge trash immediately
[
https://issues.apache.org/jira/browse/HADOOP-16140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16784390#comment-16784390
]
Hadoop QA commented on HADOOP-16140:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
17s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 16m
45s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 14m
59s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
54s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
16s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 20s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
38s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
4s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 14m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 14m
2s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 53s{color} | {color:orange} hadoop-common-project/hadoop-common: The patch
generated 1 new + 95 unchanged - 0 fixed = 96 total (was 95) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 44s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 8m 28s{color}
| {color:red} hadoop-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
45s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 89m 46s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.ha.TestZKFailoverController |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f |
| JIRA Issue | HADOOP-16140 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12961158/HADOOP-14200.006.patch
|
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux bf9833b56e30 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2
17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 0aefe28 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_191 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HADOOP-Build/16015/artifact/out/diff-checkstyle-hadoop-common-project_hadoop-common.txt
|
| unit |
https://builds.apache.org/job/PreCommit-HADOOP-Build/16015/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/16015/testReport/ |
| Max. process+thread count | 1698 (vs. ulimit of
[jira] [Commented] (HADOOP-16140) Add emptyTrash option to purge trash immediately
[ https://issues.apache.org/jira/browse/HADOOP-16140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16784335#comment-16784335 ] Stephen O'Donnell commented on HADOOP-16140: [~ste...@apache.org] I have uploaded v6 to address your two points. Please have a check after the build / test run completes. > Add emptyTrash option to purge trash immediately > > > Key: HADOOP-16140 > URL: https://issues.apache.org/jira/browse/HADOOP-16140 > Project: Hadoop Common > Issue Type: Improvement > Components: fs >Affects Versions: 3.3.0 >Reporter: Stephen O'Donnell >Assignee: Stephen O'Donnell >Priority: Major > Attachments: HADOOP-14200.002.patch, HADOOP-14200.003.patch, > HADOOP-14200.004.patch, HADOOP-14200.005.patch, HADOOP-14200.006.patch, > HDFS-14200.001.patch > > > I have always felt the HDFS trash is missing a simple way to empty the > current users trash immediately. We have "expunge" but in my experience > supporting clusters, end users find this confusing. When most end users run > expunge, they really want to empty their trash immediately and get confused > when expunge does not do this. > This can result in users performing somewhat dangerous "skipTrash" operations > on the trash to free up space. The alternative, which most users will not > figure out on their own is: > # Run the expunge command once - this will move the current folder to a > checkpoint and remove any old checkpoints older than the retention interval > # Wait over 1 minute and then run expunge again, overriding fs.trash.interval > to 1 minute using the following command hadoop fs -Dfs.trash.interval=1 > -expunge. > With this Jira I am proposing to add a extra command, "hdfs dfs -emptyTrash" > that purges everything in the logged in users Trash directories immediately. > How would the community feel about adding this new option? I will upload a > patch for comments. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16140) Add emptyTrash option to purge trash immediately
[ https://issues.apache.org/jira/browse/HADOOP-16140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16784291#comment-16784291 ] Steve Loughran commented on HADOOP-16140: - nearly there. # TestTrash: replace System,out use with logging. # in the docs, wrap the use of the -immediate in the text with `` markers so it's converted to code format. Other than that, good to go > Add emptyTrash option to purge trash immediately > > > Key: HADOOP-16140 > URL: https://issues.apache.org/jira/browse/HADOOP-16140 > Project: Hadoop Common > Issue Type: Improvement > Components: fs >Affects Versions: 3.3.0 >Reporter: Stephen O'Donnell >Assignee: Stephen O'Donnell >Priority: Major > Attachments: HADOOP-14200.002.patch, HADOOP-14200.003.patch, > HADOOP-14200.004.patch, HADOOP-14200.005.patch, HDFS-14200.001.patch > > > I have always felt the HDFS trash is missing a simple way to empty the > current users trash immediately. We have "expunge" but in my experience > supporting clusters, end users find this confusing. When most end users run > expunge, they really want to empty their trash immediately and get confused > when expunge does not do this. > This can result in users performing somewhat dangerous "skipTrash" operations > on the trash to free up space. The alternative, which most users will not > figure out on their own is: > # Run the expunge command once - this will move the current folder to a > checkpoint and remove any old checkpoints older than the retention interval > # Wait over 1 minute and then run expunge again, overriding fs.trash.interval > to 1 minute using the following command hadoop fs -Dfs.trash.interval=1 > -expunge. > With this Jira I am proposing to add a extra command, "hdfs dfs -emptyTrash" > that purges everything in the logged in users Trash directories immediately. > How would the community feel about adding this new option? I will upload a > patch for comments. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16140) Add emptyTrash option to purge trash immediately
[
https://issues.apache.org/jira/browse/HADOOP-16140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16783914#comment-16783914
]
Hadoop QA commented on HADOOP-16140:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 17m
49s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m
3s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
51s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
15s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 33s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
48s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
58s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 18m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 18m
50s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 53s{color} | {color:orange} hadoop-common-project/hadoop-common: The patch
generated 1 new + 95 unchanged - 0 fixed = 96 total (was 95) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 5s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m
25s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
40s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 96m 56s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f |
| JIRA Issue | HADOOP-16140 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12961061/HADOOP-14200.005.patch
|
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux d8fa1baf9424 4.4.0-139-generic #165~14.04.1-Ubuntu SMP Wed Oct
31 10:55:11 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / cb0fa0c |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_191 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HADOOP-Build/16013/artifact/out/diff-checkstyle-hadoop-common-project_hadoop-common.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/16013/testReport/ |
| Max. process+thread count | 1382 (vs. ulimit of 1) |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/16013/console |
| Powered by
[jira] [Commented] (HADOOP-16140) Add emptyTrash option to purge trash immediately
[ https://issues.apache.org/jira/browse/HADOOP-16140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16783835#comment-16783835 ] Stephen O'Donnell commented on HADOOP-16140: I have uploaded the v5 patch to add the code block around the new test back in. This will raise a checkstyle warning, but I think its OK as it keeps with the convention already in that section of the code. Assuming the latest patch runs OK, I think this one is ready for wider review and hopefully we can commit it if everyone is happy. > Add emptyTrash option to purge trash immediately > > > Key: HADOOP-16140 > URL: https://issues.apache.org/jira/browse/HADOOP-16140 > Project: Hadoop Common > Issue Type: Improvement > Components: fs >Affects Versions: 3.3.0 >Reporter: Stephen O'Donnell >Assignee: Stephen O'Donnell >Priority: Major > Attachments: HADOOP-14200.002.patch, HADOOP-14200.003.patch, > HADOOP-14200.004.patch, HADOOP-14200.005.patch, HDFS-14200.001.patch > > > I have always felt the HDFS trash is missing a simple way to empty the > current users trash immediately. We have "expunge" but in my experience > supporting clusters, end users find this confusing. When most end users run > expunge, they really want to empty their trash immediately and get confused > when expunge does not do this. > This can result in users performing somewhat dangerous "skipTrash" operations > on the trash to free up space. The alternative, which most users will not > figure out on their own is: > # Run the expunge command once - this will move the current folder to a > checkpoint and remove any old checkpoints older than the retention interval > # Wait over 1 minute and then run expunge again, overriding fs.trash.interval > to 1 minute using the following command hadoop fs -Dfs.trash.interval=1 > -expunge. > With this Jira I am proposing to add a extra command, "hdfs dfs -emptyTrash" > that purges everything in the logged in users Trash directories immediately. > How would the community feel about adding this new option? I will upload a > patch for comments. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16140) Add emptyTrash option to purge trash immediately
[ https://issues.apache.org/jira/browse/HADOOP-16140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16783296#comment-16783296 ] Adam Antal commented on HADOOP-16140: - Hi [~sodonnell]. Agreed - it's a bit old test file anyways and needs some factoring, but the patch is good. Also another thing that came into my mind is that since it is an addition to the CLI that you can add this change to TestCLI through testConf.xml - although I do not find any entry for the expunge command. > Add emptyTrash option to purge trash immediately > > > Key: HADOOP-16140 > URL: https://issues.apache.org/jira/browse/HADOOP-16140 > Project: Hadoop Common > Issue Type: Improvement > Components: fs >Affects Versions: 3.3.0 >Reporter: Stephen O'Donnell >Assignee: Stephen O'Donnell >Priority: Major > Attachments: HADOOP-14200.002.patch, HADOOP-14200.003.patch, > HADOOP-14200.004.patch, HDFS-14200.001.patch > > > I have always felt the HDFS trash is missing a simple way to empty the > current users trash immediately. We have "expunge" but in my experience > supporting clusters, end users find this confusing. When most end users run > expunge, they really want to empty their trash immediately and get confused > when expunge does not do this. > This can result in users performing somewhat dangerous "skipTrash" operations > on the trash to free up space. The alternative, which most users will not > figure out on their own is: > # Run the expunge command once - this will move the current folder to a > checkpoint and remove any old checkpoints older than the retention interval > # Wait over 1 minute and then run expunge again, overriding fs.trash.interval > to 1 minute using the following command hadoop fs -Dfs.trash.interval=1 > -expunge. > With this Jira I am proposing to add a extra command, "hdfs dfs -emptyTrash" > that purges everything in the logged in users Trash directories immediately. > How would the community feel about adding this new option? I will upload a > patch for comments. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16140) Add emptyTrash option to purge trash immediately
[ https://issues.apache.org/jira/browse/HADOOP-16140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16783266#comment-16783266 ] Stephen O'Donnell commented on HADOOP-16140: Hi [~adam.antal] - Thanks for the review. I did originally have the new test in a block scope, but it threw up a checkstyle warning, so I took them out. I feel its probably better to stick with the convention already in the code even if it violates checkstyle and put them back in. I will upload another patch later today and put them back in. Its not simple to move that test into a new method, as that large method contains a series of tests and the method is called in a couple of different places (TestTrash and TestViewFSTrash), which I guess is why it is like that, and somewhat unconventional. > Add emptyTrash option to purge trash immediately > > > Key: HADOOP-16140 > URL: https://issues.apache.org/jira/browse/HADOOP-16140 > Project: Hadoop Common > Issue Type: Improvement > Components: fs >Affects Versions: 3.3.0 >Reporter: Stephen O'Donnell >Assignee: Stephen O'Donnell >Priority: Major > Attachments: HADOOP-14200.002.patch, HADOOP-14200.003.patch, > HADOOP-14200.004.patch, HDFS-14200.001.patch > > > I have always felt the HDFS trash is missing a simple way to empty the > current users trash immediately. We have "expunge" but in my experience > supporting clusters, end users find this confusing. When most end users run > expunge, they really want to empty their trash immediately and get confused > when expunge does not do this. > This can result in users performing somewhat dangerous "skipTrash" operations > on the trash to free up space. The alternative, which most users will not > figure out on their own is: > # Run the expunge command once - this will move the current folder to a > checkpoint and remove any old checkpoints older than the retention interval > # Wait over 1 minute and then run expunge again, overriding fs.trash.interval > to 1 minute using the following command hadoop fs -Dfs.trash.interval=1 > -expunge. > With this Jira I am proposing to add a extra command, "hdfs dfs -emptyTrash" > that purges everything in the logged in users Trash directories immediately. > How would the community feel about adding this new option? I will upload a > patch for comments. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16140) Add emptyTrash option to purge trash immediately
[
https://issues.apache.org/jira/browse/HADOOP-16140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16783260#comment-16783260
]
Adam Antal commented on HADOOP-16140:
-
Thanks for the patch! Looks good overall. I suggest moving the new testcase
into a \{ .. }, as the other testcases do the same (the variables don't leak),
or rather move into a separate testcase.
> Add emptyTrash option to purge trash immediately
>
>
> Key: HADOOP-16140
> URL: https://issues.apache.org/jira/browse/HADOOP-16140
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs
>Affects Versions: 3.3.0
>Reporter: Stephen O'Donnell
>Assignee: Stephen O'Donnell
>Priority: Major
> Attachments: HADOOP-14200.002.patch, HADOOP-14200.003.patch,
> HADOOP-14200.004.patch, HDFS-14200.001.patch
>
>
> I have always felt the HDFS trash is missing a simple way to empty the
> current users trash immediately. We have "expunge" but in my experience
> supporting clusters, end users find this confusing. When most end users run
> expunge, they really want to empty their trash immediately and get confused
> when expunge does not do this.
> This can result in users performing somewhat dangerous "skipTrash" operations
> on the trash to free up space. The alternative, which most users will not
> figure out on their own is:
> # Run the expunge command once - this will move the current folder to a
> checkpoint and remove any old checkpoints older than the retention interval
> # Wait over 1 minute and then run expunge again, overriding fs.trash.interval
> to 1 minute using the following command hadoop fs -Dfs.trash.interval=1
> -expunge.
> With this Jira I am proposing to add a extra command, "hdfs dfs -emptyTrash"
> that purges everything in the logged in users Trash directories immediately.
> How would the community feel about adding this new option? I will upload a
> patch for comments.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16140) Add emptyTrash option to purge trash immediately
[
https://issues.apache.org/jira/browse/HADOOP-16140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16781960#comment-16781960
]
Hadoop QA commented on HADOOP-16140:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
25s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 26m
41s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 24m
35s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
6s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
48s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
15m 49s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
8s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
21s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 21m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 21m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 37s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
18s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 9m
44s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
54s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}125m 19s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f |
| JIRA Issue | HADOOP-16140 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12960802/HADOOP-14200.004.patch
|
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 39145c30e8df 4.4.0-138-generic #164~14.04.1-Ubuntu SMP Fri Oct
5 08:56:16 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / dcaca19 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_191 |
| findbugs | v3.1.0-RC1 |
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/16004/testReport/ |
| Max. process+thread count | 1402 (vs. ulimit of 1) |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/16004/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> Add emptyTrash option to purge trash immediately
>
>
> Key: HADOOP-16140
>
[jira] [Commented] (HADOOP-16140) Add emptyTrash option to purge trash immediately
[ https://issues.apache.org/jira/browse/HADOOP-16140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16781849#comment-16781849 ] Stephen O'Donnell commented on HADOOP-16140: Uploaded one more patch - this contains a note in the docs about the new options and hopefully fixes the remaining checkstyle issues. I think adding the ability to pass a filesystem to the expunge command is quite easy as the Trash itself has the ability to accept a non-default FS when it is created. The tricky part will be adding a test for it I think. I will add a comment to HADOOP-13656 and see if the current assignee wants to work on it. > Add emptyTrash option to purge trash immediately > > > Key: HADOOP-16140 > URL: https://issues.apache.org/jira/browse/HADOOP-16140 > Project: Hadoop Common > Issue Type: Improvement > Components: fs >Affects Versions: 3.3.0 >Reporter: Stephen O'Donnell >Assignee: Stephen O'Donnell >Priority: Major > Attachments: HADOOP-14200.002.patch, HADOOP-14200.003.patch, > HADOOP-14200.004.patch, HDFS-14200.001.patch > > > I have always felt the HDFS trash is missing a simple way to empty the > current users trash immediately. We have "expunge" but in my experience > supporting clusters, end users find this confusing. When most end users run > expunge, they really want to empty their trash immediately and get confused > when expunge does not do this. > This can result in users performing somewhat dangerous "skipTrash" operations > on the trash to free up space. The alternative, which most users will not > figure out on their own is: > # Run the expunge command once - this will move the current folder to a > checkpoint and remove any old checkpoints older than the retention interval > # Wait over 1 minute and then run expunge again, overriding fs.trash.interval > to 1 minute using the following command hadoop fs -Dfs.trash.interval=1 > -expunge. > With this Jira I am proposing to add a extra command, "hdfs dfs -emptyTrash" > that purges everything in the logged in users Trash directories immediately. > How would the community feel about adding this new option? I will upload a > patch for comments. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16140) Add emptyTrash option to purge trash immediately
[
https://issues.apache.org/jira/browse/HADOOP-16140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16779180#comment-16779180
]
Hadoop QA commented on HADOOP-16140:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
17s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 21m
4s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 15m
20s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
49s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
19s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 29s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
32s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
8s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 14m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 14m
26s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 52s{color} | {color:orange} hadoop-common-project/hadoop-common: The patch
generated 2 new + 95 unchanged - 0 fixed = 97 total (was 95) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 27s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m
1s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
41s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 92m 37s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f |
| JIRA Issue | HADOOP-16140 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12960331/HADOOP-14200.003.patch
|
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 4c8e7dee4186 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2
17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 8c30114 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_191 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HADOOP-Build/15986/artifact/out/diff-checkstyle-hadoop-common-project_hadoop-common.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/15986/testReport/ |
| Max. process+thread count | 1389 (vs. ulimit of 1) |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/15986/console |
| Powered by | Apache
[jira] [Commented] (HADOOP-16140) Add emptyTrash option to purge trash immediately
[
https://issues.apache.org/jira/browse/HADOOP-16140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16779147#comment-16779147
]
Stephen O'Donnell commented on HADOOP-16140:
I have changed the exception in the test to be the following to ensure it fails
the test:
{code}
try {
rc = shell.run(new String [] {"-expunge", "-immediate"});
} catch (Exception e) {
fail("Unexpected exception running the trash shell: " +
e.getLocalizedMessage());
}
{code}
The overall method (which performs quite a few tests) currently doesn't throw
any exceptions so if we change it to a throw then other calling methods will
need to expect exceptions, so the change will ripple up. The above will ensure
the new test fails if an exception is thrown.
> Add emptyTrash option to purge trash immediately
>
>
> Key: HADOOP-16140
> URL: https://issues.apache.org/jira/browse/HADOOP-16140
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs
>Affects Versions: 3.3.0
>Reporter: Stephen O'Donnell
>Assignee: Stephen O'Donnell
>Priority: Major
> Attachments: HADOOP-14200.002.patch, HADOOP-14200.003.patch,
> HDFS-14200.001.patch
>
>
> I have always felt the HDFS trash is missing a simple way to empty the
> current users trash immediately. We have "expunge" but in my experience
> supporting clusters, end users find this confusing. When most end users run
> expunge, they really want to empty their trash immediately and get confused
> when expunge does not do this.
> This can result in users performing somewhat dangerous "skipTrash" operations
> on the trash to free up space. The alternative, which most users will not
> figure out on their own is:
> # Run the expunge command once - this will move the current folder to a
> checkpoint and remove any old checkpoints older than the retention interval
> # Wait over 1 minute and then run expunge again, overriding fs.trash.interval
> to 1 minute using the following command hadoop fs -Dfs.trash.interval=1
> -expunge.
> With this Jira I am proposing to add a extra command, "hdfs dfs -emptyTrash"
> that purges everything in the logged in users Trash directories immediately.
> How would the community feel about adding this new option? I will upload a
> patch for comments.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16140) Add emptyTrash option to purge trash immediately
[ https://issues.apache.org/jira/browse/HADOOP-16140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16778725#comment-16778725 ] Steve Loughran commented on HADOOP-16140: - bq. > TestTrash:L509. Dont downgrade an exception to a log, just rethrow bq. In this case I was copying the pattern that already exists in this (rather large) test method where the above is used quite a few times. I wonder if its best to stick with what is there rather than doing something different for this one additional test? well. the other tests are probably broken. If an operation fails, the test should throw an exception so it is reported as a failure. > Add emptyTrash option to purge trash immediately > > > Key: HADOOP-16140 > URL: https://issues.apache.org/jira/browse/HADOOP-16140 > Project: Hadoop Common > Issue Type: Improvement > Components: fs >Affects Versions: 3.3.0 >Reporter: Stephen O'Donnell >Assignee: Stephen O'Donnell >Priority: Major > Attachments: HADOOP-14200.002.patch, HDFS-14200.001.patch > > > I have always felt the HDFS trash is missing a simple way to empty the > current users trash immediately. We have "expunge" but in my experience > supporting clusters, end users find this confusing. When most end users run > expunge, they really want to empty their trash immediately and get confused > when expunge does not do this. > This can result in users performing somewhat dangerous "skipTrash" operations > on the trash to free up space. The alternative, which most users will not > figure out on their own is: > # Run the expunge command once - this will move the current folder to a > checkpoint and remove any old checkpoints older than the retention interval > # Wait over 1 minute and then run expunge again, overriding fs.trash.interval > to 1 minute using the following command hadoop fs -Dfs.trash.interval=1 > -expunge. > With this Jira I am proposing to add a extra command, "hdfs dfs -emptyTrash" > that purges everything in the logged in users Trash directories immediately. > How would the community feel about adding this new option? I will upload a > patch for comments. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16140) Add emptyTrash option to purge trash immediately
[
https://issues.apache.org/jira/browse/HADOOP-16140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16778691#comment-16778691
]
Hadoop QA commented on HADOOP-16140:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
15s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 16m
41s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 15m
21s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
45s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
9s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 17s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
32s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
4s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 13m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 13m
54s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 42s{color} | {color:orange} hadoop-common-project/hadoop-common: The patch
generated 11 new + 95 unchanged - 0 fixed = 106 total (was 95) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch 6 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 7s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m
42s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
37s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 87m 4s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f |
| JIRA Issue | HADOOP-16140 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12960242/HADOOP-14200.002.patch
|
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 01f236053a4a 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2
17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / a106d2d |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_191 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HADOOP-Build/15979/artifact/out/diff-checkstyle-hadoop-common-project_hadoop-common.txt
|
| whitespace |
https://builds.apache.org/job/PreCommit-HADOOP-Build/15979/artifact/out/whitespace-tabs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/15979/testReport/ |
| Max. process+thread count | 1382 (vs. ulimit of 1) |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console
[jira] [Commented] (HADOOP-16140) Add emptyTrash option to purge trash immediately
[ https://issues.apache.org/jira/browse/HADOOP-16140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16778628#comment-16778628 ] Stephen O'Donnell commented on HADOOP-16140: I have uploaded new patch that: # Hopefully fixes the style issues # Fixes the failing test - this passed on Mac OS due to it being a case insensitive FS # Adds the expunge -immediate option and removes the emptyTrash command Regarding the comment from [~ste...@apache.org] > TestTrash:L509. Dont downgrade an exception to a log, just rethrow In this case I was copying the pattern that already exists in this (rather large) test method where the above is used quite a few times. I wonder if its best to stick with what is there rather than doing something different for this one additional test? > Add emptyTrash option to purge trash immediately > > > Key: HADOOP-16140 > URL: https://issues.apache.org/jira/browse/HADOOP-16140 > Project: Hadoop Common > Issue Type: Improvement > Components: fs >Affects Versions: 3.3.0 >Reporter: Stephen O'Donnell >Assignee: Stephen O'Donnell >Priority: Major > Attachments: HADOOP-14200.002.patch, HDFS-14200.001.patch > > > I have always felt the HDFS trash is missing a simple way to empty the > current users trash immediately. We have "expunge" but in my experience > supporting clusters, end users find this confusing. When most end users run > expunge, they really want to empty their trash immediately and get confused > when expunge does not do this. > This can result in users performing somewhat dangerous "skipTrash" operations > on the trash to free up space. The alternative, which most users will not > figure out on their own is: > # Run the expunge command once - this will move the current folder to a > checkpoint and remove any old checkpoints older than the retention interval > # Wait over 1 minute and then run expunge again, overriding fs.trash.interval > to 1 minute using the following command hadoop fs -Dfs.trash.interval=1 > -expunge. > With this Jira I am proposing to add a extra command, "hdfs dfs -emptyTrash" > that purges everything in the logged in users Trash directories immediately. > How would the community feel about adding this new option? I will upload a > patch for comments. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16140) Add emptyTrash option to purge trash immediately
[
https://issues.apache.org/jira/browse/HADOOP-16140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16777820#comment-16777820
]
Adam Antal commented on HADOOP-16140:
-
Thanks for the summary, [~sodonnell].
As I go through the options you suggested I agree that option 2 looks the most
promising.
My main point is that it's not accidental that most OS-es have the same kind of
deleting mechanism: first move the files the to trash, and then delete them
permanently. We can skipTrash-delete files from anywhere, but have to wait the
trash for checkpointing to be able to remove something? It is kinda confusing,
yes! As this jira aims to work on that, I believe the {{dfs -expunge
-immediate}} is the closest what it's missing from the trash system, and would
benefit the most from customer's point of view - so I'd vote on that. What is
your opinion?
I think the other improvements should be delt with in some follow-up jiras, but
they're still important and has to be discussed.
> Add emptyTrash option to purge trash immediately
>
>
> Key: HADOOP-16140
> URL: https://issues.apache.org/jira/browse/HADOOP-16140
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs
>Affects Versions: 3.3.0
>Reporter: Stephen O'Donnell
>Assignee: Stephen O'Donnell
>Priority: Major
> Attachments: HDFS-14200.001.patch
>
>
> I have always felt the HDFS trash is missing a simple way to empty the
> current users trash immediately. We have "expunge" but in my experience
> supporting clusters, end users find this confusing. When most end users run
> expunge, they really want to empty their trash immediately and get confused
> when expunge does not do this.
> This can result in users performing somewhat dangerous "skipTrash" operations
> on the trash to free up space. The alternative, which most users will not
> figure out on their own is:
> # Run the expunge command once - this will move the current folder to a
> checkpoint and remove any old checkpoints older than the retention interval
> # Wait over 1 minute and then run expunge again, overriding fs.trash.interval
> to 1 minute using the following command hadoop fs -Dfs.trash.interval=1
> -expunge.
> With this Jira I am proposing to add a extra command, "hdfs dfs -emptyTrash"
> that purges everything in the logged in users Trash directories immediately.
> How would the community feel about adding this new option? I will upload a
> patch for comments.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16140) Add emptyTrash option to purge trash immediately
[ https://issues.apache.org/jira/browse/HADOOP-16140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16776937#comment-16776937 ] Stephen O'Donnell commented on HADOOP-16140: Thanks all for looking into this. The idea behind this jira is that any time I have seen a support case related to emptying the trash, the users think expunge should empty it immediately. Expunge means "obliterate or remove completely", it does not do that, which is why its so confusing. So we can fix this in a few ways: 1. Make expunge actually empty the trash by default, which is what the command name suggests - I suspect we don't want to do this for compatibility reasons. 2. Add a flag to expunge (-immediate or -immediately), to override the current behaviour and clear the trash now. Having thought about this, I am coming around to this being the best idea. 3. Have a new emptyTrash command, which makes the purpose of expunge even more confusing. Adam has suggested a dry-run option, and earlier in this thread Inigo suggest a confirmation message if you are emptying the trash now. I can see some merits on these, but even with the trash we see a remarkable number of cases where people accidentality delete data with -skipTrash. I fear we will see 'accidental emptying of the trash' no matter what safety checks we add. If the data gets into the trash, and the default expunge action is as before (ie retain trash for 24 hours by default), then if we ask for an "-immediate" flag to be past to delete it now, then we have already offered two lines of defence against accidental deletion. If we go that way, I think a confirmation message is unnecessary. I am not sure about the -dry-run option and how often it would be used over someone just listing the trash they are about to delete. Steve also wants the ability to pass a filesystem as raised in HADOOP-13656 - I wonder if we should solve and commit this jira and then add in the filesystem switch afterwards (I am happy to work on it if we can get this one done). I would also like the ability to pass the trash folder you wish to empty so you can empty your own trash in an EZ or a super user can clear any trash - that could be done here or in a follow up Jira too. Can others chime on the best direction here? Ie: 1. Can we agree the best approach is adding "-immediate" to expunge and forget about the emptyTrash command? 2. Can we keep HADOOP-13656 separate and resolve it after this one? 3. We should allow a specific trash directory to be specified and do that in a separate Jira? 4. Should we add a dry-run option or not when -immediate is past? > Add emptyTrash option to purge trash immediately > > > Key: HADOOP-16140 > URL: https://issues.apache.org/jira/browse/HADOOP-16140 > Project: Hadoop Common > Issue Type: Improvement > Components: fs >Affects Versions: 3.3.0 >Reporter: Stephen O'Donnell >Assignee: Stephen O'Donnell >Priority: Major > Attachments: HDFS-14200.001.patch > > > I have always felt the HDFS trash is missing a simple way to empty the > current users trash immediately. We have "expunge" but in my experience > supporting clusters, end users find this confusing. When most end users run > expunge, they really want to empty their trash immediately and get confused > when expunge does not do this. > This can result in users performing somewhat dangerous "skipTrash" operations > on the trash to free up space. The alternative, which most users will not > figure out on their own is: > # Run the expunge command once - this will move the current folder to a > checkpoint and remove any old checkpoints older than the retention interval > # Wait over 1 minute and then run expunge again, overriding fs.trash.interval > to 1 minute using the following command hadoop fs -Dfs.trash.interval=1 > -expunge. > With this Jira I am proposing to add a extra command, "hdfs dfs -emptyTrash" > that purges everything in the logged in users Trash directories immediately. > How would the community feel about adding this new option? I will upload a > patch for comments. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16140) Add emptyTrash option to purge trash immediately
[ https://issues.apache.org/jira/browse/HADOOP-16140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16776860#comment-16776860 ] Steve Loughran commented on HADOOP-16140: - bq. I wonder whether it would make sense to implement something like git clean --dry-run: that's a really good idea > Add emptyTrash option to purge trash immediately > > > Key: HADOOP-16140 > URL: https://issues.apache.org/jira/browse/HADOOP-16140 > Project: Hadoop Common > Issue Type: Improvement > Components: fs >Affects Versions: 3.3.0 >Reporter: Stephen O'Donnell >Assignee: Stephen O'Donnell >Priority: Major > Attachments: HDFS-14200.001.patch > > > I have always felt the HDFS trash is missing a simple way to empty the > current users trash immediately. We have "expunge" but in my experience > supporting clusters, end users find this confusing. When most end users run > expunge, they really want to empty their trash immediately and get confused > when expunge does not do this. > This can result in users performing somewhat dangerous "skipTrash" operations > on the trash to free up space. The alternative, which most users will not > figure out on their own is: > # Run the expunge command once - this will move the current folder to a > checkpoint and remove any old checkpoints older than the retention interval > # Wait over 1 minute and then run expunge again, overriding fs.trash.interval > to 1 minute using the following command hadoop fs -Dfs.trash.interval=1 > -expunge. > With this Jira I am proposing to add a extra command, "hdfs dfs -emptyTrash" > that purges everything in the logged in users Trash directories immediately. > How would the community feel about adding this new option? I will upload a > patch for comments. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16140) Add emptyTrash option to purge trash immediately
[
https://issues.apache.org/jira/browse/HADOOP-16140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16776663#comment-16776663
]
Adam Antal commented on HADOOP-16140:
-
I wonder whether it would make sense to implement something like {{git clean
--dry-run}}: it does not remove anything, but shows you what the command would
have been done. I think this would be helpful for the user, though technically
do nothing else than just a {{dfs -ls}} on trash root. Maybe {{-emptyTrash}}
command and this extra option could wipe out the confusion about HDFS trash.
What is your opinion?
For your patch uploaded, please add some unit tests and CLI tests for the new
command.
> Add emptyTrash option to purge trash immediately
>
>
> Key: HADOOP-16140
> URL: https://issues.apache.org/jira/browse/HADOOP-16140
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs
>Affects Versions: 3.3.0
>Reporter: Stephen O'Donnell
>Assignee: Stephen O'Donnell
>Priority: Major
> Attachments: HDFS-14200.001.patch
>
>
> I have always felt the HDFS trash is missing a simple way to empty the
> current users trash immediately. We have "expunge" but in my experience
> supporting clusters, end users find this confusing. When most end users run
> expunge, they really want to empty their trash immediately and get confused
> when expunge does not do this.
> This can result in users performing somewhat dangerous "skipTrash" operations
> on the trash to free up space. The alternative, which most users will not
> figure out on their own is:
> # Run the expunge command once - this will move the current folder to a
> checkpoint and remove any old checkpoints older than the retention interval
> # Wait over 1 minute and then run expunge again, overriding fs.trash.interval
> to 1 minute using the following command hadoop fs -Dfs.trash.interval=1
> -expunge.
> With this Jira I am proposing to add a extra command, "hdfs dfs -emptyTrash"
> that purges everything in the logged in users Trash directories immediately.
> How would the community feel about adding this new option? I will upload a
> patch for comments.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16140) Add emptyTrash option to purge trash immediately
[
https://issues.apache.org/jira/browse/HADOOP-16140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16775558#comment-16775558
]
Hadoop QA commented on HADOOP-16140:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
15s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 16m
58s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 15m
6s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
45s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
7s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 49s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
32s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
58s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 14m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 14m
22s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 49s{color} | {color:orange} hadoop-common-project/hadoop-common: The patch
generated 14 new + 95 unchanged - 0 fixed = 109 total (was 95) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch 6 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
9m 49s{color} | {color:green} patch has no errors when building and testing our
client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 8m 42s{color}
| {color:red} hadoop-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
34s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 86m 53s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.fs.viewfs.TestViewFsTrash |
| | hadoop.fs.TestTrash |
| | hadoop.util.TestReadWriteDiskValidator |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f |
| JIRA Issue | HADOOP-16140 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12954628/HDFS-14200.001.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 3279ccffb145 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2
17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / ed13cf8 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_191 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HADOOP-Build/15963/artifact/out/diff-checkstyle-hadoop-common-project_hadoop-common.txt
|
| whitespace |
https://builds.apache.org/job/PreCommit-HADOOP-Build/15963/artifact/out/whitespace-tabs.txt
|
| unit |
[jira] [Commented] (HADOOP-16140) Add emptyTrash option to purge trash immediately
[ https://issues.apache.org/jira/browse/HADOOP-16140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16775488#comment-16775488 ] Stephen O'Donnell commented on HADOOP-16140: I put the version down as 3.3.0 for now and submitted the patch. > Why not just add an option to -expunge. It exists, has tests, documentation, > etc? I am not sure about this. Expunge doesn't really empty the trash as I explained above, it kind of does what the NN does internally. So if we added an option, eg expunge -emptyNow, does that make it more or less confusing? It would be pretty easy to add this to expunge and have it call the new methods I added if we think that makes sense. I am open to doing it either way. Any suggestions on what to call the option? expunge -immediately expunge -emptyTrash expunge -? I will have a look at the other two points shortly. > Add emptyTrash option to purge trash immediately > > > Key: HADOOP-16140 > URL: https://issues.apache.org/jira/browse/HADOOP-16140 > Project: Hadoop Common > Issue Type: Improvement > Components: fs >Affects Versions: 3.3.0 >Reporter: Stephen O'Donnell >Assignee: Stephen O'Donnell >Priority: Major > Attachments: HDFS-14200.001.patch > > > I have always felt the HDFS trash is missing a simple way to empty the > current users trash immediately. We have "expunge" but in my experience > supporting clusters, end users find this confusing. When most end users run > expunge, they really want to empty their trash immediately and get confused > when expunge does not do this. > This can result in users performing somewhat dangerous "skipTrash" operations > on the trash to free up space. The alternative, which most users will not > figure out on their own is: > # Run the expunge command once - this will move the current folder to a > checkpoint and remove any old checkpoints older than the retention interval > # Wait over 1 minute and then run expunge again, overriding fs.trash.interval > to 1 minute using the following command hadoop fs -Dfs.trash.interval=1 > -expunge. > With this Jira I am proposing to add a extra command, "hdfs dfs -emptyTrash" > that purges everything in the logged in users Trash directories immediately. > How would the community feel about adding this new option? I will upload a > patch for comments. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16140) Add emptyTrash option to purge trash immediately
[ https://issues.apache.org/jira/browse/HADOOP-16140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16775473#comment-16775473 ] Steve Loughran commented on HADOOP-16140: - +moved to the right project; can you add the version this affects/you want to get this in to > Add emptyTrash option to purge trash immediately > > > Key: HADOOP-16140 > URL: https://issues.apache.org/jira/browse/HADOOP-16140 > Project: Hadoop Common > Issue Type: Improvement > Components: fs >Reporter: Stephen O'Donnell >Assignee: Stephen O'Donnell >Priority: Major > Attachments: HDFS-14200.001.patch > > > I have always felt the HDFS trash is missing a simple way to empty the > current users trash immediately. We have "expunge" but in my experience > supporting clusters, end users find this confusing. When most end users run > expunge, they really want to empty their trash immediately and get confused > when expunge does not do this. > This can result in users performing somewhat dangerous "skipTrash" operations > on the trash to free up space. The alternative, which most users will not > figure out on their own is: > # Run the expunge command once - this will move the current folder to a > checkpoint and remove any old checkpoints older than the retention interval > # Wait over 1 minute and then run expunge again, overriding fs.trash.interval > to 1 minute using the following command hadoop fs -Dfs.trash.interval=1 > -expunge. > With this Jira I am proposing to add a extra command, "hdfs dfs -emptyTrash" > that purges everything in the logged in users Trash directories immediately. > How would the community feel about adding this new option? I will upload a > patch for comments. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
