Hello Yida Wu, Csaba Ringhofer, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/24154
to look at the new patch set (#15).
Change subject: IMPALA-14852 Codegen tuple TryDeepCopy for Broadcast Exchange
......................................................................
IMPALA-14852 Codegen tuple TryDeepCopy for Broadcast Exchange
Use Tuple::TryDeepCopy in RowBatch/OutboundRowBatch instead of DeepCopy.
For most tuples, this should save an extra size calculation, because the
size of the whole row is not calculated anymore before the copying, only
if a tuple doesn't fit, is its size calculated.
A codegen'd version of Tuple::TryDeepCopy is used for each tuple in the
row, which is generated based on the Tuple's Descriptor.
This patch does not do proper codegen for collection types, instead it
calls the interpreted code for them.
Measurements:
Measured with query:
SELECT l1.* FROM tpch30.lineitem l1
INNER JOIN tpch30.lineitem l2
ON l1.l_shipmode = l2.l_shipmode LIMIT 1;
Where tpch30 is generated by:
bin/load-data.py -s 30 -f --workloads tpch
--table_formats text/none,parquet/snap
SerializeBatchTime changed from 1s797ms to 1s239ms.
This is about 31% improvement.
Codegen:
Before:
NumInstructions: 4.01K (4015)
NumOptimizedInstructions: 3.87K (3874)
ModuleBitcodeSize: 3.68 MB (3860700)
After:
NumInstructions: 5.26K (5259)
NumOptimizedInstructions: 5.15K (5151)
ModuleBitcodeSize: 3.71 MB (3892344)
Change-Id: Iaa43e949c63db047717104dbbe42d47f94ebf2d0
---
M be/src/benchmarks/row-batch-serialize-benchmark.cc
M be/src/codegen/gen_ir_descriptions.py
M be/src/codegen/impala-ir.cc
M be/src/runtime/CMakeLists.txt
A be/src/runtime/descriptors-ir.cc
M be/src/runtime/descriptors.cc
M be/src/runtime/descriptors.h
M be/src/runtime/krpc-data-stream-sender.cc
M be/src/runtime/krpc-data-stream-sender.h
A be/src/runtime/outbound-row-batch-ir.cc
M be/src/runtime/outbound-row-batch.cc
M be/src/runtime/outbound-row-batch.h
M be/src/runtime/outbound-row-batch.inline.h
A be/src/runtime/row-batch-ir.cc
M be/src/runtime/row-batch.cc
M be/src/runtime/row-batch.h
M be/src/runtime/tuple-ir.cc
M be/src/runtime/tuple.cc
M be/src/runtime/tuple.h
M
testdata/workloads/functional-query/queries/QueryTest/datastream-sender-codegen.test
20 files changed, 606 insertions(+), 247 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/54/24154/15
--
To view, visit http://gerrit.cloudera.org:8080/24154
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Iaa43e949c63db047717104dbbe42d47f94ebf2d0
Gerrit-Change-Number: 24154
Gerrit-PatchSet: 15
Gerrit-Owner: Balazs Hevele <[email protected]>
Gerrit-Reviewer: Balazs Hevele <[email protected]>
Gerrit-Reviewer: Csaba Ringhofer <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Yida Wu <[email protected]>
