zhtaoxiang commented on code in PR #9906:
URL: https://github.com/apache/pinot/pull/9906#discussion_r1039997133
##########
pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/blocks/TransferableBlockUtils.java:
##########
@@ -76,9 +76,13 @@ public static boolean isNoOpBlock(TransferableBlock
transferableBlock) {
public static Iterator<TransferableBlock> splitBlock(TransferableBlock
block, DataBlock.Type type, int maxBlockSize) {
List<TransferableBlock> blockChunks = new ArrayList<>();
if (type == DataBlock.Type.ROW) {
- // Use estimated row size, this estimate is not accurate and is used to
estimate numRowsPerChunk only.
- int estimatedRowSizeInBytes =
block.getDataSchema().getColumnNames().length * MEDIAN_COLUMN_SIZE_BYTES;
- int numRowsPerChunk = maxBlockSize / estimatedRowSizeInBytes;
+ // Use 1 row per chunk when block.getDataSchema().size() is 0, we may
want to fine tune it.
+ int numRowsPerChunk = 1;
+ if (block.getDataSchema().size() != 0) {
+ // Use estimated row size, this estimate is not accurate and is used
to estimate numRowsPerChunk only.
+ int estimatedRowSizeInBytes = block.getDataSchema().size() *
MEDIAN_COLUMN_SIZE_BYTES;
+ numRowsPerChunk = maxBlockSize / estimatedRowSizeInBytes;
+ }
Preconditions.checkState(numRowsPerChunk > 0, "row size too large for
query engine to handle, abort!");
Review Comment:
I had an offline discussion with @walterddr . He mentioned that
1/ we should throw exception when dataSchema size is 0
2/ the current built query plan for `select 1 as one from {table} where 1 <
2` is not correct, we should update the query plan building logic.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
