[
https://issues.apache.org/jira/browse/HDFS-222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12755575#action_12755575
]
Sanjay Radia commented on HDFS-222:
-----------------------------------
Clearly this is a hack to support parallel copies of large files in distcp. (It
is an embarrassment that hadoop does not support this).
The proper way to do this is to create a first class abstraction for a "file as
a container for blocks". But that is long project.
So the new concat method would be marked as limited-private.
Breaking the FileSystem abstraction issue - I don't get it: All file systems
impls can support a concat of files, though most cannot do this atomically.
Owen are you proposing that we add this to distributedFileSystem and not
FileSystem and that distcp does as "class narrow" to use it if it is available?
I am fine with that.
> Support for concatenating of files into a single file
> -----------------------------------------------------
>
> Key: HDFS-222
> URL: https://issues.apache.org/jira/browse/HDFS-222
> Project: Hadoop HDFS
> Issue Type: New Feature
> Reporter: Venkatesh S
> Assignee: Boris Shkolnik
>
> An API to concatenate files of same size and replication factor on HDFS into
> a single larger file.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
