Balazs Hevele has posted comments on this change. ( http://gerrit.cloudera.org:8080/24190 )
Change subject: IMPALA-14882: part1: Convert arrow record batch to impala tuple batch ...................................................................... Patch Set 26: (9 comments) http://gerrit.cloudera.org:8080/#/c/24190/24/be/src/exec/arrow-converter-test.cc File be/src/exec/arrow-converter-test.cc: http://gerrit.cloudera.org:8080/#/c/24190/24/be/src/exec/arrow-converter-test.cc@110 PS24, Line 110: EXPECT_EQ(value->Len(), (int)original_value.siz > nit: comparing int to size_t, probably worth an explicit cast Done http://gerrit.cloudera.org:8080/#/c/24190/24/be/src/exec/arrow-converter.cc File be/src/exec/arrow-converter.cc: http://gerrit.cloudera.org:8080/#/c/24190/24/be/src/exec/arrow-converter.cc@84 PS24, Line 84: slot_desc_(slot_desc) { > A potential optimization could be to optimize for cases with no NULLs to av Done http://gerrit.cloudera.org:8080/#/c/24190/24/be/src/exec/arrow-converter.cc@89 PS24, Line 89: int end_row_idx) { > It may be faster to set all tuples to 0 (meaning non-null) and only set nul Done http://gerrit.cloudera.org:8080/#/c/24190/24/be/src/exec/arrow-converter.cc@138 PS24, Line 138: > Can you add a comment about these types? Or a more descriptive name could h Done http://gerrit.cloudera.org:8080/#/c/24190/24/be/src/exec/arrow-converter.cc@214 PS24, Line 214: > Some of this code could be possibly merged with Parquet scanning code. ParquetTimestampDecoder seems too Parquet specific to be used here. For example, there is arrow::TimeUnit::SECOND unit that I don't see in Parquet, adding that there would affect Parquet scanning. So I found it best to keep this separate. http://gerrit.cloudera.org:8080/#/c/24190/24/be/src/exec/arrow-converter.cc@311 PS24, Line 311: > Missing break; after this! Done http://gerrit.cloudera.org:8080/#/c/24190/24/be/src/exec/arrow-converter.cc@377 PS24, Line 377: > Does arrow store timestamps as UTC? Iceberg uses mainly timestamps without Arrow has a "timezone" type parameter for a Timestamp type, I haven't found much information about it (like what empty value means). I kept the timezone handling as it was in the previous arrow scanning in paimon-jni-row-reader.cc http://gerrit.cloudera.org:8080/#/c/24190/24/be/src/exec/arrow-converter.cc@407 PS24, Line 407: > Unaligned allocation will cause issues with larger types. Done http://gerrit.cloudera.org:8080/#/c/24190/24/be/src/exec/arrow-converter.cc@417 PS24, Line 417: if (UNLIKELY(value_buffer == nul > This will crash in case of complex types. Better to return a status with an Done -- To view, visit http://gerrit.cloudera.org:8080/24190 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iea544b3c71d9211c893f0fec3527ebe84155ebcd Gerrit-Change-Number: 24190 Gerrit-PatchSet: 26 Gerrit-Owner: Balazs Hevele <[email protected]> Gerrit-Reviewer: Balazs Hevele <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Xuebin Su <[email protected]> Gerrit-Reviewer: jichen <[email protected]> Gerrit-Comment-Date: Wed, 22 Jul 2026 10:36:50 +0000 Gerrit-HasComments: Yes
