Copy on write was discussed in HADOOP-334 in the context of periodic
checkpointing of the name-space.
Other than that I remember a discussion about file clone() operation,
which makes a new inode, but uses the
same blocks as the original file, which are copied once they are
modified or appended.
But this functionality would be possible only if we had at least
appends. Since hdfs does not support
modifications the purpose of COW is going to be only to support grouping
of blocks from different
(or is it just one?) files.
I think it is possible, but very non-possix.
And you can always create one-block files and group them in directories
instead.
--Konstantin
Benjamin Reed wrote:
I need to implement COW for HDFS for a project I'm working on. I
vaguely remember it being discussed before, but I can't find any
threads about it. I wanted to at least check for interest/previous
work before proceeding. Hard links would work for me as well, but they
are harder to implement. I was thinking of adding the following to the
client protocol:
public void cow(String src, String clientName, boolean overwrite,
LocatedBlocks blocks) throws IOException;
The call would simply create a new file and populate its contents with
the blocks contained in the LocatedBlocks.
Apart from fast copies, it also allows fast truncations and extensions
of existing files.
(This is not a hard link because it is possible that the set of blocks
may not correspond to any other file.)
Has such a thing been discussed before?
thanx
ben
