[
https://issues.apache.org/jira/browse/APEXMALHAR-2484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15984474#comment-15984474
]
ASF GitHub Bot commented on APEXMALHAR-2484:
--------------------------------------------
GitHub user chaithu14 opened a pull request:
https://github.com/apache/apex-malhar/pull/611
APEXMALHAR-2484 Support of PartFileWriter for writing the part files
@shubham-pathak22 @yogidevendra Please review and merge.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/chaithu14/incubator-apex-malhar
APEXMALHAR-2484-partfilewriter
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/apex-malhar/pull/611.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #611
----
commit 6761f7d1efc9ab34076c625cdc0684c8de200151
Author: chaitanya <chai...@apache.org>
Date: 2017-04-26T09:40:52Z
APEXMALHAR-2484 PartFileWriter for writing the part files in specified
directory
----
> BlockWriter for writing the part files into the specified directory
> -------------------------------------------------------------------
>
> Key: APEXMALHAR-2484
> URL: https://issues.apache.org/jira/browse/APEXMALHAR-2484
> Project: Apache Apex Malhar
> Issue Type: Task
> Reporter: Chaitanya
> Assignee: Chaitanya
>
> Use case: Suppose, the size of source file (f1.txt) is 1 GB and the block
> size is 128 MB. I want to copy the file in destination as follows:
> f1.txt.part1
> f2.txt.part2
> ....
> By default, size of each part file is 128 MB except the last part.
> Design: Currently, the BlockWriter is restricted to write the part files into
> the HDFS on which the app is running. To achieve the above use case, operator
> needs the block index and relative path information. BlockMetadata which is
> the input port for the BlockWriter doesn't have these information.
> So, I am creating the new operator(PartFileWriter) which extends from
> BlockWriter with the input port of type FileMetadata.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
