[
https://issues.apache.org/jira/browse/FLINK-2583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727540#comment-14727540
]
ASF GitHub Bot commented on FLINK-2583:
---------------------------------------
Github user tillrohrmann commented on the pull request:
https://github.com/apache/flink/pull/1084#issuecomment-137143368
Concerning the second short term option: Is the problem there that the
checkpointing is not aligned with the rolling? Thus, you can take a checkpoint
but you still have a temporary file which contains all the records up to the
checkpoint. If the sink now fails before renaming the temporary file, then the
records which have been written before the checkpoint was taken are lost. Did I
understand it correctly? If this is the case, then the records in the temporary
file should actually be part of the state of the sink, right?
> Add Stream Sink For Rolling HDFS Files
> --------------------------------------
>
> Key: FLINK-2583
> URL: https://issues.apache.org/jira/browse/FLINK-2583
> Project: Flink
> Issue Type: New Feature
> Components: Streaming
> Reporter: Aljoscha Krettek
> Assignee: Aljoscha Krettek
> Fix For: 0.10
>
>
> In addition to having configurable file-rolling behavior the Sink should also
> integrate with checkpointing to make it possible to have exactly-once
> semantics throughout the topology.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
