[jira] [Commented] (HDFS-16032) DFSClient#delete supports Trash
[ https://issues.apache.org/jira/browse/HDFS-16032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17494022#comment-17494022 ] Steve Loughran commented on HDFS-16032: --- FYI this would # destroy performance on s3a, where copy is used on rename # make delete faster on abfs where oauth is used to authenticate, as there a delete() call forces a treewalk of all directories to validate caller has the permissions it needs. HADOOP-17691. But when ABFS starts throttling, renames() may fail in ways that delete doesn't. we are discussing adding a plugin point for per-schema trash: HADOOP-18013 it would be possible to add something in FileUtils to use this, but FileSystem.delete() *must not* suddenly move things to trash. > DFSClient#delete supports Trash > > > Key: HDFS-16032 > URL: https://issues.apache.org/jira/browse/HDFS-16032 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hadoop-client, hdfs >Affects Versions: 3.4.0 >Reporter: Xiangyi Zhu >Priority: Major > Labels: pull-request-available > Time Spent: 50m > Remaining Estimate: 0h > > Currently, HDFS can only move deleted data to Trash through Shell commands. > In actual scenarios, most of the data is deleted through DFSClient Api. I > think it should support Trash. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16032) DFSClient#delete supports Trash
[ https://issues.apache.org/jira/browse/HDFS-16032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17351669#comment-17351669 ] Xiangyi Zhu commented on HDFS-16032: [~ayushtkn],[~sodonnell] Thanks a lot for your comments, I use your suggestions to improve it. > DFSClient#delete supports Trash > > > Key: HDFS-16032 > URL: https://issues.apache.org/jira/browse/HDFS-16032 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hadoop-client, hdfs >Affects Versions: 3.4.0 >Reporter: Xiangyi Zhu >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > Currently, HDFS can only move deleted data to Trash through Shell commands. > In actual scenarios, most of the data is deleted through DFSClient Api. I > think it should support Trash. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16032) DFSClient#delete supports Trash
[ https://issues.apache.org/jira/browse/HDFS-16032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17351659#comment-17351659 ] Stephen O'Donnell commented on HDFS-16032: -- I agree with [~ayushtkn]. It does not feel correct to enforce trash at the existing delete API call. One other reason, is that there may be some reason a call to delete does not want to use trash, so you need a "skipTrash" option, which would break compatibility. Exposing a more public "deleteWithTrash" is probably better, and then a user of the API can decide which they want to use. > DFSClient#delete supports Trash > > > Key: HDFS-16032 > URL: https://issues.apache.org/jira/browse/HDFS-16032 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hadoop-client, hdfs >Affects Versions: 3.4.0 >Reporter: Xiangyi Zhu >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > Currently, HDFS can only move deleted data to Trash through Shell commands. > In actual scenarios, most of the data is deleted through DFSClient Api. I > think it should support Trash. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16032) DFSClient#delete supports Trash
[ https://issues.apache.org/jira/browse/HDFS-16032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17351543#comment-17351543 ] Ayush Saxena commented on HDFS-16032: - Trash is a FsShell concept, I would argue if it belongs to DfsClient. Though we have a config to enforce whether it should be enabled or not to prevent compatibility and with this on, but this would be just restricted to HDFS or say DistributedFileSystem, In general Downstream projects using the API's don't make specific assumptions related to FileSystem types changing. So, making a DFS only enhancement doesn't seems to be very right. Maybe just having a util method, like FileUtils.delete(..) or FileUtils.deleteWithTrash(..) can be exposed, if something of that sort isn't already there, which can be generic to all the FileSystems and would keep a strict control in hands of the Applications, and applications can adopt to it if there is a need to that. Secondly, initially the trash was to prevent user level accidents, the API calls are supposed to be coming from some fairly stable applications, knowing what they are doing, if the application wanted to move to trash, or have some precautions, they could have coded that way, there are ways to do that > DFSClient#delete supports Trash > > > Key: HDFS-16032 > URL: https://issues.apache.org/jira/browse/HDFS-16032 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hadoop-client, hdfs >Affects Versions: 3.4.0 >Reporter: Xiangyi Zhu >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > Currently, HDFS can only move deleted data to Trash through Shell commands. > In actual scenarios, most of the data is deleted through DFSClient Api. I > think it should support Trash. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16032) DFSClient#delete supports Trash
[ https://issues.apache.org/jira/browse/HDFS-16032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17351454#comment-17351454 ] Xiangyi Zhu commented on HDFS-16032: [~hexiaoqiao],[~ayushtkn],[Stephen O'Donnell|https://issues.apache.org/jira/secure/ViewProfile.jspa?name=sodonnell] Looking forward your comments. > DFSClient#delete supports Trash > > > Key: HDFS-16032 > URL: https://issues.apache.org/jira/browse/HDFS-16032 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hadoop-client, hdfs >Affects Versions: 3.4.0 >Reporter: Xiangyi Zhu >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > Currently, HDFS can only move deleted data to Trash through Shell commands. > In actual scenarios, most of the data is deleted through DFSClient Api. I > think it should support Trash. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
