[
https://issues.apache.org/jira/browse/HDFS-3107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13235963#comment-13235963
]
Milind Bhandarkar commented on HDFS-3107:
-----------------------------------------
I really like Sanjay's idea (i.e. 2) for it's simplicity, atomicity, and ease
of implementation. However, I would like the client API to be truncate(file,
length), and namenode RPC to be truncate(file, length, concatFile).
In the client API, i.e. method in DFSClient, length could be anything less than
the file size, whereas in the namenode RPC, we have two options:
1. length must be a multiple of blocksize for file, less than filesize
2. length could be anything less than filesize, but namenode will do length =
(length - (length%blocksize)).
For the client API to support any length, without specifying concatFile, we
need to add o.a.h.fs.FileUtils.createTempFile.
Thoughts ?
Is this futureproof ? What happens where block storage becomes first class
citizen ? Will we have an ability to create unowned block, and append it to an
existing (truncated to block boundary) file ?
> HDFS truncate
> -------------
>
> Key: HDFS-3107
> URL: https://issues.apache.org/jira/browse/HDFS-3107
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: data-node, name-node
> Reporter: Lei Chang
> Attachments: HDFS_truncate_semantics_Mar15.pdf,
> HDFS_truncate_semantics_Mar21.pdf
>
> Original Estimate: 1,344h
> Remaining Estimate: 1,344h
>
> Systems with transaction support often need to undo changes made to the
> underlying storage when a transaction is aborted. Currently HDFS does not
> support truncate (a standard Posix operation) which is a reverse operation of
> append, which makes upper layer applications use ugly workarounds (such as
> keeping track of the discarded byte range per file in a separate metadata
> store, and periodically running a vacuum process to rewrite compacted files)
> to overcome this limitation of HDFS.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
