[ https://issues.apache.org/jira/browse/FLINK-3296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15126351#comment-15126351 ]
ASF GitHub Bot commented on FLINK-3296: --------------------------------------- Github user StephanEwen commented on a diff in the pull request: https://github.com/apache/flink/pull/1563#discussion_r51428065 --- Diff: docs/apis/streaming/index.md --- @@ -1768,6 +1768,11 @@ greater than 1, the output will also be prepended with the identifier of the tas </div> </div> +Note that the `write*()` methods on `DataStream` are mainly intended for debugging purposes. +They are not participating in Flink's checkpointing (no fault tolerance guarantees). The --- End diff -- May be worth adding that this means usually "at-least-once" , but may also mean data loss in cases where the output formats buffer data and do not immediately persist it. > DataStream.write*() methods are not flushing properly > ----------------------------------------------------- > > Key: FLINK-3296 > URL: https://issues.apache.org/jira/browse/FLINK-3296 > Project: Flink > Issue Type: Bug > Components: Streaming Connectors > Reporter: Robert Metzger > Assignee: Robert Metzger > Priority: Critical > > The DataStream.write() methods rely on the {{FileSinkFunctionByMillis}} > class, which has a logic for flushing records, even though the underlying > stream is never flushed. This is misleading for users as files are not > written as they would expect it. > The code was initial written with FileOutputFormats in mind, but the types > were not set correctly. This PR opened the write() method to any output > format: https://github.com/apache/flink/pull/706/files -- This message was sent by Atlassian JIRA (v6.3.4#6332)