[
https://issues.apache.org/jira/browse/FLINK-3296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15126351#comment-15126351
]
ASF GitHub Bot commented on FLINK-3296:
---------------------------------------
Github user StephanEwen commented on a diff in the pull request:
https://github.com/apache/flink/pull/1563#discussion_r51428065
--- Diff: docs/apis/streaming/index.md ---
@@ -1768,6 +1768,11 @@ greater than 1, the output will also be prepended
with the identifier of the tas
</div>
</div>
+Note that the `write*()` methods on `DataStream` are mainly intended for
debugging purposes.
+They are not participating in Flink's checkpointing (no fault tolerance
guarantees). The
--- End diff --
May be worth adding that this means usually "at-least-once" , but may also
mean data loss in cases where the output formats buffer data and do not
immediately persist it.
> DataStream.write*() methods are not flushing properly
> -----------------------------------------------------
>
> Key: FLINK-3296
> URL: https://issues.apache.org/jira/browse/FLINK-3296
> Project: Flink
> Issue Type: Bug
> Components: Streaming Connectors
> Reporter: Robert Metzger
> Assignee: Robert Metzger
> Priority: Critical
>
> The DataStream.write() methods rely on the {{FileSinkFunctionByMillis}}
> class, which has a logic for flushing records, even though the underlying
> stream is never flushed. This is misleading for users as files are not
> written as they would expect it.
> The code was initial written with FileOutputFormats in mind, but the types
> were not set correctly. This PR opened the write() method to any output
> format: https://github.com/apache/flink/pull/706/files
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
