[ https://issues.apache.org/jira/browse/HUDI-945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinoth Chandar updated HUDI-945: -------------------------------- Priority: Blocker (was: Major) > Cleanup spillable map files eagerly as part of close > ---------------------------------------------------- > > Key: HUDI-945 > URL: https://issues.apache.org/jira/browse/HUDI-945 > Project: Apache Hudi > Issue Type: Bug > Components: Writer Core > Reporter: Balaji Varadarajan > Assignee: Sreeram Ramji > Priority: Blocker > Labels: pull-request-available > Fix For: 0.10.0 > > > Currently, files used by external spillable map are deleted on exits. For > spark-streaming/deltastreamer continuous-mode cases which runs several > iterations, it is better to eagerly delete files on closing the handles using > it. > We need to eagerly delete the files on following cases: > # HoodieMergeHandle > # HoodieMergedLogRecordScanner > # SpillableMapBasedFileSystemView -- This message was sent by Atlassian Jira (v8.3.4#803005)