[ https://issues.apache.org/jira/browse/HUDI-945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17183499#comment-17183499 ]
Balaji Varadarajan commented on HUDI-945: ----------------------------------------- [~baobaoyeye]: Let me know if you are interested in submitting a PR. > Cleanup spillable map files eagerly as part of close > ---------------------------------------------------- > > Key: HUDI-945 > URL: https://issues.apache.org/jira/browse/HUDI-945 > Project: Apache Hudi > Issue Type: Bug > Components: Writer Core > Reporter: Balaji Varadarajan > Priority: Major > Fix For: 0.6.1 > > > Currently, files used by external spillable map or deleted on exits. For > spark-streaming/deltastreamer continuous-mode cases which runs several > iterations, it is better to eagerly delete files on closing the handles using > it. > We need to eagerly delete the files on following cases: > # MergeHandle > # HoodieMergedLogRecordScanner > # SpillableMapBasedFileSystemView -- This message was sent by Atlassian Jira (v8.3.4#803005)