hailong wang created FLINK-19425:
------------------------------------
Summary: Correct the usage of BulkWriter#flush and
BulkWriter#finish
Key: FLINK-19425
URL: https://issues.apache.org/jira/browse/FLINK-19425
Project: Flink
Issue Type: Improvement
Components: Connectors / Common
Affects Versions: 1.11.0
Reporter: hailong wang
Fix For: 1.12.0, 1.11.0
>From the comments, BulkWriter#finish method should flush all buffer before
>close.
But some subclasses of it do not flush data. These classes are as follows:
1.AvroBulkWriter#finish
2.HadoopCompressionBulkWriter#finish
3.NoCompressionBulkWriter#finish
4.SequenceFileWriter#finish
We should invoke BulkWriter#flush in this finish methods.
On the other hand, We don't have to invoke BulkWriter#flush in close method.
For BulkWriter#finish will flush all data.
1. HadoopPathBasedPartFileWriter#closeForCommit
2. BulkPartWriter#closeForCommit
3. FileSystemTableSink#OutputFormat#close
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
