Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/19879 )
Change subject: IMPALA-12138: Optimize HS2 result vector allocations ...................................................................... IMPALA-12138: Optimize HS2 result vector allocations Before this patch the reservation sizes were based on the number of rows in the RowBatches - as batch_size has lower default than fetch_size (1024 vs 10240), one fetch is served by multiple row batches leading to reserving vectors in more than one step. This patch changes the logic to: - reserve during the first fetch the old way - reserve fetch_size in subsequent fetches This means that queries with small result set should not regress while in large ones only the first and the last fetches will be suboptimal. Also noticed that the current default fetch_size=10240 in impala-shell is not optimal for RowMaterializationTimer, probably because it is not a power of 2 and leads to overallocation. Created IMPALA-12142 for the potential default fetch_size change. Tested with select * from tpch_parquet.lineitem, and RowMaterializationTimer was decreased around 10-20%: fetch_size=10240: 3.6s -> 3.2s fetch_size=8192: 2.8s -> 2.6s Change-Id: I7b0e6a0a8fd028e3c0e4f1f4e272a50d2bfb59ba Reviewed-on: http://gerrit.cloudera.org:8080/19879 Reviewed-by: Impala Public Jenkins <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M be/src/service/hs2-util.cc M be/src/service/hs2-util.h M be/src/service/impala-hs2-server.cc M be/src/service/query-result-set.cc M be/src/service/query-result-set.h 5 files changed, 50 insertions(+), 31 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/19879 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I7b0e6a0a8fd028e3c0e4f1f4e272a50d2bfb59ba Gerrit-Change-Number: 19879 Gerrit-PatchSet: 7 Gerrit-Owner: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Daniel Becker <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Kurt Deschler <[email protected]>
