Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/20316#discussion_r162514432
--- Diff:
sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/ColumnVectorUtils.java
---
@@ -192,7 +192,7 @@ private static void appendValue(WritableColumnVector
dst, DataType t, Row src, i
*/
public static ColumnarBatch toBatch(
StructType schema, MemoryMode memMode, Iterator<Row> row) {
- int capacity = ColumnarBatch.DEFAULT_BATCH_SIZE;
+ int capacity = 4 * 1024;
--- End diff --
places using `ColumnarBatch` should decide the default size themselves.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
