[
https://issues.apache.org/jira/browse/FLINK-24728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jingsong Lee closed FLINK-24728.
--------------------------------
Resolution: Fixed
master:
6af0b9965293cb732a540b9364b6aae76a9b356a
2e9f9ad166f472edd693c8a47857e14e76928dc9
release-1.14:
9a0c5e00839983de23f662e337dfc626d0bdaad9
11d24708be32605243bc404679b17758c4e76e79
release-1.13:
3200e8ef43b3024b0b44f184dfa833d1aa7d7d75
> Batch SQL file sink forgets to close the output stream
> ------------------------------------------------------
>
> Key: FLINK-24728
> URL: https://issues.apache.org/jira/browse/FLINK-24728
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Runtime
> Affects Versions: 1.11.4, 1.14.0, 1.12.5, 1.13.3
> Reporter: Caizhi Weng
> Assignee: Caizhi Weng
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.15.0, 1.13.6, 1.14.3
>
>
> I tried to write a large avro file into HDFS and discover that the displayed
> file size in HDFS is extremely small, but copying that file to local yields
> the correct size. If we create another Flink job and read that avro file from
> HDFS, the job will finish without outputting any record because the file size
> Flink gets from HDFS is the very small file size.
> This is because the output format created in
> {{FileSystemTableSink#createBulkWriterOutputFormat}} only finishes the
> {{BulkWriter}}. According to the java doc of {{BulkWriter#finish}} bulk
> writers should not close the output stream and should leave them to the
> framework.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
