Hello Kurt Deschler, Daniel Becker, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/19879
to look at the new patch set (#5).
Change subject: IMPALA-12138: Optimize HS2 result vector allocations
......................................................................
IMPALA-12138: Optimize HS2 result vector allocations
Before this patch the reservation sizes were based on the
number of rows in the RowBatches - as batch_size has lower default
than fetch_size (1024 vs 10240), one fetch is served by multiple row
batches leading to reserving vectors in more than one step.
This patch changes the logic to:
- reserve during the first fetch the old way
- reserve fetch_size in subsequent fetches
This means that queries with small result set should not regress
while in large ones only the first and the last fetches will be
suboptimal.
Also noticed that the current default fetch_size=10240 in impala-shell
is not optimal for RowMaterializationTimer, probably because it is
not a power of 2 and leads to overallocation.
Created IMPALA-12142 for the potential default fetch_size change.
Tested with select * from tpch_parquet.lineitem, and
RowMaterializationTimer was decreased around 10-20%:
fetch_size=10240: 3.6s -> 3.2s
fetch_size=8192: 2.8s -> 2.6s
Change-Id: I7b0e6a0a8fd028e3c0e4f1f4e272a50d2bfb59ba
---
M be/src/service/hs2-util.cc
M be/src/service/hs2-util.h
M be/src/service/impala-hs2-server.cc
M be/src/service/query-result-set.cc
M be/src/service/query-result-set.h
5 files changed, 50 insertions(+), 31 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/79/19879/5
--
To view, visit http://gerrit.cloudera.org:8080/19879
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I7b0e6a0a8fd028e3c0e4f1f4e272a50d2bfb59ba
Gerrit-Change-Number: 19879
Gerrit-PatchSet: 5
Gerrit-Owner: Csaba Ringhofer <[email protected]>
Gerrit-Reviewer: Csaba Ringhofer <[email protected]>
Gerrit-Reviewer: Daniel Becker <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Kurt Deschler <[email protected]>
