Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/20316#discussion_r162522356
--- Diff:
sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/VectorizedParquetRecordReader.java
---
@@ -50,6 +50,9 @@
* TODO: make this always return ColumnarBatches.
*/
public class VectorizedParquetRecordReader extends
SpecificParquetRecordReaderBase<Object> {
+ // TODO: make this configurable.
+ private static final int CAPACITY = 4 * 1024;
--- End diff --
but it's not a test, it's an example about how to implement a
`ColumnarBatch` producer. We would remove it when we publish
`WritableColumnVector` and have a stable API for writing `ColumnarBatch`.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
