[ https://issues.apache.org/jira/browse/HUDI-945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Balaji Varadarajan updated HUDI-945: ------------------------------------ Status: Open (was: New) > Cleanup spillable map files eagerly as part of close > ---------------------------------------------------- > > Key: HUDI-945 > URL: https://issues.apache.org/jira/browse/HUDI-945 > Project: Apache Hudi > Issue Type: Bug > Components: Writer Core > Reporter: Balaji Varadarajan > Priority: Major > Fix For: 0.6.1 > > > Currently, files used by external spillable map or deleted on exits. For > spark-streaming/deltastreamer continuous-mode cases which runs several > iterations, it is better to eagerly delete files on closing the handles using > it. > We need to eagerly delete the files on following cases: > # MergeHandle > # HoodieMergedLogRecordScanner > # SpillableMapBasedFileSystemView -- This message was sent by Atlassian Jira (v8.3.4#803005)