Repository: spark Updated Branches: refs/heads/branch-2.4 1303eb5c8 -> c64e7506d
[MINOR][PYSPARK] Always Close the tempFile in _serialize_to_jvm ## What changes were proposed in this pull request? Always close the tempFile after `serializer.dump_stream(data, tempFile)` in _serialize_to_jvm ## How was this patch tested? N/A Closes #22523 from gatorsmile/fixMinor. Authored-by: gatorsmile <gatorsm...@gmail.com> Signed-off-by: hyukjinkwon <gurwls...@apache.org> Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/c64e7506 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/c64e7506 Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/c64e7506 Branch: refs/heads/branch-2.4 Commit: c64e7506dabaccc60f8140aeae589053645f23a6 Parents: 1303eb5 Author: gatorsmile <gatorsm...@gmail.com> Authored: Sun Sep 23 10:16:33 2018 +0800 Committer: hyukjinkwon <gurwls...@apache.org> Committed: Sun Sep 23 10:18:00 2018 +0800 ---------------------------------------------------------------------- python/pyspark/context.py | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/spark/blob/c64e7506/python/pyspark/context.py ---------------------------------------------------------------------- diff --git a/python/pyspark/context.py b/python/pyspark/context.py index 87255c4..0924d3d 100644 --- a/python/pyspark/context.py +++ b/python/pyspark/context.py @@ -537,8 +537,10 @@ class SparkContext(object): # parallelize from there. tempFile = NamedTemporaryFile(delete=False, dir=self._temp_dir) try: - serializer.dump_stream(data, tempFile) - tempFile.close() + try: + serializer.dump_stream(data, tempFile) + finally: + tempFile.close() return reader_func(tempFile.name) finally: # we eagerily reads the file so we can delete right after. --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org