Gopal V created HIVE-15789: ------------------------------ Summary: Vectorization: limit reduce vectorization to 32Mb chunks Key: HIVE-15789 URL: https://issues.apache.org/jira/browse/HIVE-15789 Project: Hive Issue Type: Bug Components: Vectorization Reporter: Gopal V
Reduce vectorization accumulates 1024 rows before forwarding it into the reduce processor. Add a safety limit for 32Mb of writables, so that shorter sequences can be forwarded into the operator trees. {code} rowIdx++; if (rowIdx >= BATCH_SIZE) { VectorizedBatchUtil.setBatchSize(batch, rowIdx); reducer.process(batch, tag); {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346)