[jira] Commented: (HDFS-1403) add -truncate option to fsck
[ https://issues.apache.org/jira/browse/HDFS-1403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12913899#action_12913899 ] dhruba borthakur commented on HDFS-1403: It is possible that we make the fsck -truncate only cleanup files only those files whose last modtime is within the last hour or so. I am a little worried that otherwise it might erroneously cleanup files that do nto need fixing. add -truncate option to fsck Key: HDFS-1403 URL: https://issues.apache.org/jira/browse/HDFS-1403 Project: Hadoop HDFS Issue Type: New Feature Components: hdfs client, name-node Reporter: sam rash When running fsck, it would be useful to be able to tell hdfs to truncate any corrupt file to the last valid position in the latest block. Then, when running hadoop fsck, an admin can cleanup the filesystem. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-1403) add -truncate option to fsck
[ https://issues.apache.org/jira/browse/HDFS-1403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12913908#action_12913908 ] sam rash commented on HDFS-1403: can you elaborate? also, this truncate option will have to work on open files. I think -list-corruptfiles only works on closed ones. we have to handle the missing last block problem (the main reason I filed this) add -truncate option to fsck Key: HDFS-1403 URL: https://issues.apache.org/jira/browse/HDFS-1403 Project: Hadoop HDFS Issue Type: New Feature Components: hdfs client, name-node Reporter: sam rash When running fsck, it would be useful to be able to tell hdfs to truncate any corrupt file to the last valid position in the latest block. Then, when running hadoop fsck, an admin can cleanup the filesystem. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-1403) add -truncate option to fsck
[ https://issues.apache.org/jira/browse/HDFS-1403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12913910#action_12913910 ] dhruba borthakur commented on HDFS-1403: Ok, if this fix works only on currently-open files, then that will be fine. It will automatically disallow fixing of files that are closed... sounds good. add -truncate option to fsck Key: HDFS-1403 URL: https://issues.apache.org/jira/browse/HDFS-1403 Project: Hadoop HDFS Issue Type: New Feature Components: hdfs client, name-node Reporter: sam rash When running fsck, it would be useful to be able to tell hdfs to truncate any corrupt file to the last valid position in the latest block. Then, when running hadoop fsck, an admin can cleanup the filesystem. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-1403) add -truncate option to fsck
[ https://issues.apache.org/jira/browse/HDFS-1403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12910003#action_12910003 ] dhruba borthakur commented on HDFS-1403: This is especially needed when the system supports hflush. A client could issue a hflush, it will persist block locations in the namenode. Then the client could fail even before it could write any bytes to that block. In this case, the last block of the file will be permanently missing. It would be nice to have an option to fsck to delete the last block of a file if it is of size zero and does not have any valid replicas. add -truncate option to fsck Key: HDFS-1403 URL: https://issues.apache.org/jira/browse/HDFS-1403 Project: Hadoop HDFS Issue Type: New Feature Components: hdfs client, name-node Reporter: sam rash When running fsck, it would be useful to be able to tell hdfs to truncate any corrupt file to the last valid position in the latest block. Then, when running hadoop fsck, an admin can cleanup the filesystem. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
