[
https://issues.apache.org/jira/browse/FLINK-29800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17626931#comment-17626931
]
Aitozi commented on FLINK-29800:
--------------------------------
The inProgressPart seems only record current one inprogress file. Between the
checkpoints, maybe serval files will rolled out, these file only should be kept
when they are part of the checkpoints.
And the dispose only close the output stream of the file, the files will still
there.
> Continuous failover will leak the inprogress output file
> --------------------------------------------------------
>
> Key: FLINK-29800
> URL: https://issues.apache.org/jira/browse/FLINK-29800
> Project: Flink
> Issue Type: Bug
> Components: Connectors / FileSystem
> Reporter: Aitozi
> Priority: Major
>
> When running job which sink to the file system, the inprogress files will
> keep growing when job keeps failover, it will do harm to the filesystem. I
> think the clean up to the file which is currently written to should be
> performed when job failing
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
