hailong wang created FLINK-19425: ------------------------------------ Summary: Correct the usage of BulkWriter#flush and BulkWriter#finish Key: FLINK-19425 URL: https://issues.apache.org/jira/browse/FLINK-19425 Project: Flink Issue Type: Improvement Components: Connectors / Common Affects Versions: 1.11.0 Reporter: hailong wang Fix For: 1.12.0, 1.11.0
>From the comments, BulkWriter#finish method should flush all buffer before >close. But some subclasses of it do not flush data. These classes are as follows: 1.AvroBulkWriter#finish 2.HadoopCompressionBulkWriter#finish 3.NoCompressionBulkWriter#finish 4.SequenceFileWriter#finish We should invoke BulkWriter#flush in this finish methods. On the other hand, We don't have to invoke BulkWriter#flush in close method. For BulkWriter#finish will flush all data. 1. HadoopPathBasedPartFileWriter#closeForCommit 2. BulkPartWriter#closeForCommit 3. FileSystemTableSink#OutputFormat#close -- This message was sent by Atlassian Jira (v8.3.4#803005)