Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/24190 )
Change subject: IMPALA-14882: part1: Convert arrow record batch to impala tuple batch ...................................................................... Patch Set 35: (4 comments) http://gerrit.cloudera.org:8080/#/c/24190/33/be/src/benchmarks/arrow-converter-benchmark.cc File be/src/benchmarks/arrow-converter-benchmark.cc: http://gerrit.cloudera.org:8080/#/c/24190/33/be/src/benchmarks/arrow-converter-benchmark.cc@48 PS33, Line 48: // Mixed 271 277 281 0.238X 0.237X 0.234X > Added an arrow to arrow copy suite for comparison. Done http://gerrit.cloudera.org:8080/#/c/24190/33/be/src/exec/arrow-converter.cc File be/src/exec/arrow-converter.cc: http://gerrit.cloudera.org:8080/#/c/24190/33/be/src/exec/arrow-converter.cc@54 PS33, Line 54: RETURN_IF_ERROR(WriteSlotBatch(arr.get(), 0, num_rows, slot_desc, > I don't understand what you mean exactly. This essentially iterates row by row (tuple_mem represents a single tuple). Our memory layout doesn't support operating on multiple tuples at once, but I was wondering how performance would be if we could. http://gerrit.cloudera.org:8080/#/c/24190/35/be/src/exec/arrow-converter.cc File be/src/exec/arrow-converter.cc: http://gerrit.cloudera.org:8080/#/c/24190/35/be/src/exec/arrow-converter.cc@231 PS35, Line 231: SlotWriter::init_status_ = Status(ERROR_ARROW_IMPALA_TYPE_MISMATCH); If we hit this branch, we shouldn't try to use timestamp_array_ at line 235. Should probably also add a DCHECK() that init_status_.ok() in methods relying on init success. In other code we usually have a separate Status Init() method to handle initialization that may fail. That would be a more familiar pattern to use. http://gerrit.cloudera.org:8080/#/c/24190/35/be/src/exec/paimon/paimon-jni-scan-node.cc File be/src/exec/paimon/paimon-jni-scan-node.cc: http://gerrit.cloudera.org:8080/#/c/24190/35/be/src/exec/paimon/paimon-jni-scan-node.cc@a186 PS35, Line 186: This check was dropped, which can lead to calling jni_scanner_ when it's uninitialized. -- To view, visit http://gerrit.cloudera.org:8080/24190 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iea544b3c71d9211c893f0fec3527ebe84155ebcd Gerrit-Change-Number: 24190 Gerrit-PatchSet: 35 Gerrit-Owner: Balazs Hevele <[email protected]> Gerrit-Reviewer: Balazs Hevele <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Michael Smith <[email protected]> Gerrit-Reviewer: Xuebin Su <[email protected]> Gerrit-Reviewer: jichen <[email protected]> Gerrit-Comment-Date: Wed, 30 Sep 2026 18:42:31 +0000 Gerrit-HasComments: Yes
