Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21369#discussion_r189892444
--- Diff:
core/src/main/scala/org/apache/spark/util/collection/ExternalAppendOnlyMap.scala
---
@@ -267,7 +273,7 @@ class ExternalAppendOnlyMap[K, V, C](
*/
def destructiveIterator(inMemoryIterator: Iterator[(K, C)]):
Iterator[(K, C)] = {
readingIterator = new SpillableIterator(inMemoryIterator)
- readingIterator
+ readingIterator.toCompletionIterator
--- End diff --
This change the original behavior of `destructiveIterator `. I'd prefer do
like this:
```
CompletionIterator[(K, C), Iterator[(K, C)]](
destructiveIterator(currentMap.iterator), readingIterator.destroy)
```
which keep compatibility with current code, and do not introduce
unnecessary function.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
