[ https://issues.apache.org/jira/browse/SPARK-5423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sean Owen updated SPARK-5423: ----------------------------- Component/s: Shuffle > ExternalAppendOnlyMap won't delete temp spilled file if some exception > happens during using it > ----------------------------------------------------------------------------------------------- > > Key: SPARK-5423 > URL: https://issues.apache.org/jira/browse/SPARK-5423 > Project: Spark > Issue Type: Improvement > Components: Shuffle > Reporter: Shixiong Zhu > Priority: Minor > > ExternalAppendOnlyMap won't delete temp spilled file if some exception > happens during using it. > There is already a TODO in the comment: > {code} > // TODO: Ensure this gets called even if the iterator isn't drained. > private def cleanup() { > batchIndex = batchOffsets.length // Prevent reading any other batch > val ds = deserializeStream > deserializeStream = null > fileStream = null > ds.close() > file.delete() > } > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org