Github user 10110346 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22163#discussion_r212168161
--- Diff:
core/src/main/java/org/apache/spark/shuffle/sort/ShuffleExternalSorter.java ---
@@ -206,14 +211,21 @@ private void writeSortedFile(boolean
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22163#discussion_r212167438
--- Diff:
core/src/main/java/org/apache/spark/shuffle/sort/ShuffleExternalSorter.java ---
@@ -206,14 +211,21 @@ private void writeSortedFile(boolean
Github user 10110346 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22163#discussion_r212166322
--- Diff:
core/src/main/java/org/apache/spark/shuffle/sort/ShuffleExternalSorter.java ---
@@ -206,14 +211,21 @@ private void writeSortedFile(boolean
Github user 10110346 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22163#discussion_r212165385
--- Diff:
core/src/main/java/org/apache/spark/shuffle/sort/ShuffleExternalSorter.java ---
@@ -206,14 +211,21 @@ private void writeSortedFile(boolean
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22163#discussion_r212163785
--- Diff:
core/src/main/java/org/apache/spark/shuffle/sort/ShuffleExternalSorter.java ---
@@ -206,14 +211,21 @@ private void writeSortedFile(boolean
Github user 10110346 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22163#discussion_r212162206
--- Diff:
core/src/main/java/org/apache/spark/shuffle/sort/ShuffleExternalSorter.java ---
@@ -206,14 +211,21 @@ private void writeSortedFile(boolean
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22163#discussion_r212160161
--- Diff:
core/src/main/java/org/apache/spark/shuffle/sort/ShuffleExternalSorter.java ---
@@ -206,14 +211,21 @@ private void writeSortedFile(boolean
Github user 10110346 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22163#discussion_r212154100
--- Diff:
core/src/main/java/org/apache/spark/shuffle/sort/ShuffleExternalSorter.java ---
@@ -206,14 +211,21 @@ private void writeSortedFile(boolean
Github user Ngone51 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22163#discussion_r211954019
--- Diff:
core/src/main/java/org/apache/spark/shuffle/sort/ShuffleExternalSorter.java ---
@@ -206,14 +211,21 @@ private void writeSortedFile(boolean
GitHub user 10110346 opened a pull request:
https://github.com/apache/spark/pull/22163
[SPARK-25166][CORE]Reduce the number of write operations for shuffle write.
## What changes were proposed in this pull request?
Currently, only one record is written to a buffer each
10 matches
Mail list logo