[
https://issues.apache.org/jira/browse/APEXMALHAR-2369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15837265#comment-15837265
]
Yogi Devendra commented on APEXMALHAR-2369:
-------------------------------------------
[~chaithu] Could you please review this?
> S3 output module for tuple based output
> ---------------------------------------
>
> Key: APEXMALHAR-2369
> URL: https://issues.apache.org/jira/browse/APEXMALHAR-2369
> Project: Apache Apex Malhar
> Issue Type: Task
> Reporter: Yogi Devendra
> Assignee: Yogi Devendra
>
> Currently, S3 output is available using S3OutputModule which is restricted
> for copying files from FileSystem to S3. Use-cases where all the
> tuples/records to be written to S3 cannot use this approach. Thus, we need to
> develop alternative module which would take care of writing tuples on S3.
> Design:
> Sending separate requests to S3 for each tuple would be too expensive. This
> module can choose to write tuples to HDFS. And then upload HDFS files to S3.
> This would lead to some end-to-end latency. But, it should OK for the S3
> output case.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
