[jira] [Commented] (HADOOP-8690) Shell may remove a file without going to trash even if skipTrash is not enabled
[
https://issues.apache.org/jira/browse/HADOOP-8690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17802963#comment-17802963
]
Shilun Fan commented on HADOOP-8690:
Bulk update: moved all 3.4.0 non-blocker issues, please move back if it is a
blocker. Retarget 3.5.0.
> Shell may remove a file without going to trash even if skipTrash is not
> enabled
> ---
>
> Key: HADOOP-8690
> URL: https://issues.apache.org/jira/browse/HADOOP-8690
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 2.0.0-alpha
>Reporter: Eli Collins
>Assignee: Andras Bokor
>Priority: Minor
> Attachments: HADOOP-8690.01.patch
>
>
> Delete.java contains the following comment:
> {noformat}
> // TODO: if the user wants the trash to be used but there is any
> // problem (ie. creating the trash dir, moving the item to be deleted,
> // etc), then the path will just be deleted because moveToTrash returns
> // false and it falls thru to fs.delete. this doesn't seem right
> {noformat}
> If Trash#moveToAppropriateTrash returns false FsShell will delete the path
> even if skipTrash is not enabled. The comment isn't quite right as some of
> these failure scenarios result in exceptions not a false return value, and in
> the case of an exception we don't unconditionally delete the path.
> TrashPolicy#moveToTrash states that it only returns false if the item is
> already in the trash or trash is disabled, and the expected behavior for
> these cases is to just delete the path. However
> TrashPolicyDefault#moveToTrash also returns false if there's a problem
> creating the trash directory, so for this case I think we should throw an
> exception rather than return false (and delete the path bypassing trash).
> I also question the behavior of just deleting when the item is already in the
> trash as it may have changed since previously put in the trash and not been
> checkpointed yet. Seems like in this case we should move it to trash but with
> a file name suffix.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-8690) Shell may remove a file without going to trash even if skipTrash is not enabled
[
https://issues.apache.org/jira/browse/HADOOP-8690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17080936#comment-17080936
]
Hadoop QA commented on HADOOP-8690:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
45s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 26m
28s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m
44s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
45s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
20s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
15m 5s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
13s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 14m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 14m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 14s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 9m
0s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
46s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}108m 20s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.8 Server=19.03.8 Image:yetus/hadoop:e6455cc864d |
| JIRA Issue | HADOOP-8690 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12879359/HADOOP-8690.01.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 86711433b2e7 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6
11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 275c478 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_242 |
| findbugs | v3.1.0-RC1 |
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/16840/testReport/ |
| Max. process+thread count | 3153 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/16840/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> Shell may remove a file without going to trash even if skipTrash is not
> enabled
> ---
[jira] [Commented] (HADOOP-8690) Shell may remove a file without going to trash even if skipTrash is not enabled
[
https://issues.apache.org/jira/browse/HADOOP-8690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16406903#comment-16406903
]
genericqa commented on HADOOP-8690:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
18s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m
31s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 12m
44s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
2s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 23s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
22s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
45s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 11m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
8m 49s{color} | {color:green} patch has no errors when building and testing our
client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 7m 46s{color}
| {color:red} hadoop-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
27s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 75m 5s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.fs.shell.TestCopyFromLocal |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:d4cc50f |
| JIRA Issue | HADOOP-8690 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12879359/HADOOP-8690.01.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux 74373aa15145 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 3ff6977 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_151 |
| findbugs | v3.1.0-RC1 |
| unit |
https://builds.apache.org/job/PreCommit-HADOOP-Build/14334/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/14334/testReport/ |
| Max. process+thread count | 1508 (vs. ulimit of 1) |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/14334/console |
| Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org |
[jira] [Commented] (HADOOP-8690) Shell may remove a file without going to trash even if skipTrash is not enabled
[
https://issues.apache.org/jira/browse/HADOOP-8690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16185035#comment-16185035
]
Hadoop QA commented on HADOOP-8690:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
18s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
7s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 15m
29s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
4s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 33s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
29s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
51s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 12m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 12m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
8m 44s{color} | {color:green} patch has no errors when building and testing our
client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 8m 55s{color}
| {color:red} hadoop-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
29s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 80m 3s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.ha.TestZKFailoverController |
| | hadoop.security.TestKDiag |
| | hadoop.net.TestDNS |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:71bbb86 |
| JIRA Issue | HADOOP-8690 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12879359/HADOOP-8690.01.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux 4eaf41b0880f 3.13.0-117-generic #164-Ubuntu SMP Fri Apr 7
11:05:26 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / ca669f9 |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC1 |
| unit |
https://builds.apache.org/job/PreCommit-HADOOP-Build/13402/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/13402/testReport/ |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/13402/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
>
[jira] [Commented] (HADOOP-8690) Shell may remove a file without going to trash even if skipTrash is not enabled
[
https://issues.apache.org/jira/browse/HADOOP-8690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105167#comment-16105167
]
Andras Bokor commented on HADOOP-8690:
--
JUnit failure is unrelated.
> Shell may remove a file without going to trash even if skipTrash is not
> enabled
> ---
>
> Key: HADOOP-8690
> URL: https://issues.apache.org/jira/browse/HADOOP-8690
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 2.0.0-alpha
>Reporter: Eli Collins
>Assignee: Andras Bokor
>Priority: Minor
> Attachments: HADOOP-8690.01.patch
>
>
> Delete.java contains the following comment:
> {noformat}
> // TODO: if the user wants the trash to be used but there is any
> // problem (ie. creating the trash dir, moving the item to be deleted,
> // etc), then the path will just be deleted because moveToTrash returns
> // false and it falls thru to fs.delete. this doesn't seem right
> {noformat}
> If Trash#moveToAppropriateTrash returns false FsShell will delete the path
> even if skipTrash is not enabled. The comment isn't quite right as some of
> these failure scenarios result in exceptions not a false return value, and in
> the case of an exception we don't unconditionally delete the path.
> TrashPolicy#moveToTrash states that it only returns false if the item is
> already in the trash or trash is disabled, and the expected behavior for
> these cases is to just delete the path. However
> TrashPolicyDefault#moveToTrash also returns false if there's a problem
> creating the trash directory, so for this case I think we should throw an
> exception rather than return false (and delete the path bypassing trash).
> I also question the behavior of just deleting when the item is already in the
> trash as it may have changed since previously put in the trash and not been
> checkpointed yet. Seems like in this case we should move it to trash but with
> a file name suffix.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-8690) Shell may remove a file without going to trash even if skipTrash is not enabled
[
https://issues.apache.org/jira/browse/HADOOP-8690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105148#comment-16105148
]
Hadoop QA commented on HADOOP-8690:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m
31s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 13m
33s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
26s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
26s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
50s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 10m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 10m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 8m 1s{color}
| {color:red} hadoop-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
28s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 57m 11s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.security.TestKDiag |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HADOOP-8690 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12879359/HADOOP-8690.01.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux f9b7df93d8cd 3.13.0-117-generic #164-Ubuntu SMP Fri Apr 7
11:05:26 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 369f731 |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| unit |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12883/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12883/testReport/ |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12883/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Shell may remove a file without going to trash even if skipTrash is not
> enabled
> ---
>
> Key: HADOOP-8690
> URL: https://issues.apache.org/jira/browse/HADOOP-8690
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 2.0.0-alpha
>Reporter: Eli Collins
>Assignee: Andras Bokor
>
[jira] [Commented] (HADOOP-8690) Shell may remove a file without going to trash even if skipTrash is not enabled
[
https://issues.apache.org/jira/browse/HADOOP-8690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105035#comment-16105035
]
Andras Bokor commented on HADOOP-8690:
--
[~eli],
You are right and you are not at the same time.
The first part of your description is correct. If the trash directory cannot be
created the file will be deleted without moving into the trash. Please find my
PoC below.
The second part is not true. The code you linked here is a bit misleading. I
think your understanding is that it examines whether the file to delete (from
anywhere on the filesystem) has already been in the trash or not. But in fact,
it checks that you try to delete a file from trash.
So the condition is false if you delete /user/knoguchi/abcdef recreate then
delete again. But it will be true and the method will return false if you try
to delete /user/knoguchi/.Trash/Current/user/knoguchi/abcdef file. That is a
correct behavior.
My PoC: it has 4 asserts. I assert that the:
* trash is enabled
* rm command success
* file does not exist
* file is not in the trash which means potential dataloss
{code:title=TestTrash.java}
public void testFileNotDeletedWhenTrashCannotBeCreated()
throws Exception {
Configuration conf = new Configuration();
conf.set(FS_TRASH_INTERVAL_KEY, "10");
conf.setClass("fs.file.impl", TestLFS.class, FileSystem.class);
assertTrue(new Trash(conf).isEnabled());
FileSystem fs = FileSystem.getLocal(conf);
Path myFolder = new Path(GenericTestUtils.getRandomizedTempPath());
Path myFile = new Path(myFolder, "myFile");
writeFile(fs, myFile, 10);
FsShell shell = new FsShell(conf);
int val = -1;
FsPermission origPermission =
fs.getFileStatus(TEST_DIR.getParent()).getPermission();
try {
fs.setPermission(TEST_DIR.getParent(), new FsPermission("555"));
assertEquals(shell.run(new String[]{"-rm", myFile.toString()}), 0);
assertFalse(fs.exists(myFile));
checkNotInTrash(fs, shell.getCurrentTrashDir(), myFile.toString());
} catch (Exception e) {
System.err.println("Exception raised from Trash.run " +
e.getLocalizedMessage());
} finally {
fs.setPermission(TEST_DIR.getParent(), origPermission);
fs.delete(myFolder, true);
}
}
{code}
This code passes but it should fail.
I will upload a patch.
> Shell may remove a file without going to trash even if skipTrash is not
> enabled
> ---
>
> Key: HADOOP-8690
> URL: https://issues.apache.org/jira/browse/HADOOP-8690
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 2.0.0-alpha
>Reporter: Eli Collins
>Assignee: Andras Bokor
>Priority: Minor
>
> Delete.java contains the following comment:
> {noformat}
> // TODO: if the user wants the trash to be used but there is any
> // problem (ie. creating the trash dir, moving the item to be deleted,
> // etc), then the path will just be deleted because moveToTrash returns
> // false and it falls thru to fs.delete. this doesn't seem right
> {noformat}
> If Trash#moveToAppropriateTrash returns false FsShell will delete the path
> even if skipTrash is not enabled. The comment isn't quite right as some of
> these failure scenarios result in exceptions not a false return value, and in
> the case of an exception we don't unconditionally delete the path.
> TrashPolicy#moveToTrash states that it only returns false if the item is
> already in the trash or trash is disabled, and the expected behavior for
> these cases is to just delete the path. However
> TrashPolicyDefault#moveToTrash also returns false if there's a problem
> creating the trash directory, so for this case I think we should throw an
> exception rather than return false (and delete the path bypassing trash).
> I also question the behavior of just deleting when the item is already in the
> trash as it may have changed since previously put in the trash and not been
> checkpointed yet. Seems like in this case we should move it to trash but with
> a file name suffix.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-8690) Shell may remove a file without going to trash even if skipTrash is not enabled
[
https://issues.apache.org/jira/browse/HADOOP-8690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13434240#comment-13434240
]
Koji Noguchi commented on HADOOP-8690:
--
bq. Seems like in this case we should move it to trash but with a file name
suffix.
I believe it does that already.
On 0.23,
{noformat}
[knoguchi ~]$ hdfs dfs -touchz abcdef
[knoguchi ~]$ hdfs dfs -rm abcdef
(repeat 3 times)
[knoguchi ~]$ hdfs dfs -ls /user/knoguchi/.Trash/Current/user/knoguchi/abc\*
Found 1 items
-rw-r--r-- 3 knoguchi users 0 2012-08-14 16:30
/user/knoguchi/.Trash/Current/user/knoguchi/abcdef
Found 1 items
-rw-r--r-- 3 knoguchi users 0 2012-08-14 16:31
/user/knoguchi/.Trash/Current/user/knoguchi/abcdef1344961878400
Found 1 items
-rw-r--r-- 3 knoguchi users 0 2012-08-14 16:31
/user/knoguchi/.Trash/Current/user/knoguchi/abcdef1344961912093
{noformat}
Shell may remove a file without going to trash even if skipTrash is not
enabled
---
Key: HADOOP-8690
URL: https://issues.apache.org/jira/browse/HADOOP-8690
Project: Hadoop Common
Issue Type: Bug
Affects Versions: 2.0.0-alpha
Reporter: Eli Collins
Priority: Minor
Delete.java contains the following comment:
{noformat}
// TODO: if the user wants the trash to be used but there is any
// problem (ie. creating the trash dir, moving the item to be deleted,
// etc), then the path will just be deleted because moveToTrash returns
// false and it falls thru to fs.delete. this doesn't seem right
{noformat}
If Trash#moveToAppropriateTrash returns false FsShell will delete the path
even if skipTrash is not enabled. The comment isn't quite right as some of
these failure scenarios result in exceptions not a false return value, and in
the case of an exception we don't unconditionally delete the path.
TrashPolicy#moveToTrash states that it only returns false if the item is
already in the trash or trash is disabled, and the expected behavior for
these cases is to just delete the path. However
TrashPolicyDefault#moveToTrash also returns false if there's a problem
creating the trash directory, so for this case I think we should throw an
exception rather than return false (and delete the path bypassing trash).
I also question the behavior of just deleting when the item is already in the
trash as it may have changed since previously put in the trash and not been
checkpointed yet. Seems like in this case we should move it to trash but with
a file name suffix.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8690) Shell may remove a file without going to trash even if skipTrash is not enabled
[
https://issues.apache.org/jira/browse/HADOOP-8690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13434274#comment-13434274
]
Eli Collins commented on HADOOP-8690:
-
Looks like there's a bug in the following check in
TrashPolicyDefault#moveToTrash and it's falling through to the code below that
appends the current time in milliseconds to the file name.
{code}
if (qpath.startsWith(trash.toString())) {
return false; // already in trash
}
{code}
Shell may remove a file without going to trash even if skipTrash is not
enabled
---
Key: HADOOP-8690
URL: https://issues.apache.org/jira/browse/HADOOP-8690
Project: Hadoop Common
Issue Type: Bug
Affects Versions: 2.0.0-alpha
Reporter: Eli Collins
Priority: Minor
Delete.java contains the following comment:
{noformat}
// TODO: if the user wants the trash to be used but there is any
// problem (ie. creating the trash dir, moving the item to be deleted,
// etc), then the path will just be deleted because moveToTrash returns
// false and it falls thru to fs.delete. this doesn't seem right
{noformat}
If Trash#moveToAppropriateTrash returns false FsShell will delete the path
even if skipTrash is not enabled. The comment isn't quite right as some of
these failure scenarios result in exceptions not a false return value, and in
the case of an exception we don't unconditionally delete the path.
TrashPolicy#moveToTrash states that it only returns false if the item is
already in the trash or trash is disabled, and the expected behavior for
these cases is to just delete the path. However
TrashPolicyDefault#moveToTrash also returns false if there's a problem
creating the trash directory, so for this case I think we should throw an
exception rather than return false (and delete the path bypassing trash).
I also question the behavior of just deleting when the item is already in the
trash as it may have changed since previously put in the trash and not been
checkpointed yet. Seems like in this case we should move it to trash but with
a file name suffix.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8690) Shell may remove a file without going to trash even if skipTrash is not enabled
[
https://issues.apache.org/jira/browse/HADOOP-8690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13433488#comment-13433488
]
Daryn Sharp commented on HADOOP-8690:
-
I added that comment a long time ago. It looks like the code has changed a
lot, but I think I agree with all your current observations.
Shell may remove a file without going to trash even if skipTrash is not
enabled
---
Key: HADOOP-8690
URL: https://issues.apache.org/jira/browse/HADOOP-8690
Project: Hadoop Common
Issue Type: Bug
Affects Versions: 2.0.0-alpha
Reporter: Eli Collins
Priority: Minor
Delete.java contains the following comment:
{noformat}
// TODO: if the user wants the trash to be used but there is any
// problem (ie. creating the trash dir, moving the item to be deleted,
// etc), then the path will just be deleted because moveToTrash returns
// false and it falls thru to fs.delete. this doesn't seem right
{noformat}
If Trash#moveToAppropriateTrash returns false FsShell will delete the path
even if skipTrash is not enabled. The comment isn't quite right as some of
these failure scenarios result in exceptions not a false return value, and in
the case of an exception we don't unconditionally delete the path.
TrashPolicy#moveToTrash states that it only returns false if the item is
already in the trash or trash is disabled, and the expected behavior for
these cases is to just delete the path. However
TrashPolicyDefault#moveToTrash also returns false if there's a problem
creating the trash directory, so for this case I don't think we should throw
an exception rather than return false.
I also question the behavior of just deleting when the item is already in the
trash as it may have changed since previously put in the trash and not been
checkpointed yet. Seems like in this case we should move it to trash but with
a file name suffix.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
