hubgeter commented on code in PR #66221:
URL: https://github.com/apache/doris/pull/66221#discussion_r3711016503
##########
thirdparty/build-thirdparty.sh:
##########
@@ -1088,9 +1090,7 @@ build_arrow() {
ldflags="-L${TP_LIB_DIR}"
fi
- CPPFLAGS="-I${TP_INCLUDE_DIR}" \
- CXXFLAGS="-I${TP_INCLUDE_DIR}" \
- LDFLAGS="${ldflags}" \
+ LDFLAGS="${ldflags}" \
Review Comment:
Confirmed. The clean macOS third-party run at
https://github.com/apache/doris/actions/runs/30878051788/job/91893553280
succeeded with the relevant build script and Arrow patches identical to the
current head. Its log shows LZO installing include/lzo/lzo1x.h, then [334/387]
compiling src/parquet/CMakeFiles/parquet_objlib.dir/column_reader.cc.o, and
completing the Arrow installation. The include path is propagated through
parquet_objlib -> PARQUET_STATIC_LINK_LIBS -> thrift::thrift ->
INTERFACE_INCLUDE_DIRECTORIES=${TP_INSTALL_DIR}/include. Therefore the header
is already on the compile path and no code change is needed.
##########
thirdparty/patches/apache-arrow-24.0.0-paimon.patch:
##########
@@ -40,26 +36,23 @@ index 285e2a5973..aa6f92f077 100644
this]() mutable -> ::arrow::Result<RecordBatchIterator> {
::arrow::ChunkedArrayVector columns(readers.size());
-- // don't reserve more rows than necessary
+ // don't reserve more rows than necessary
- int64_t batch_size = std::min(properties().batch_size(), num_rows);
- num_rows -= batch_size;
+ int64_t batch_size = 0;
+ if (!num_rows.empty()) {
-+ // don't reserve more rows than necessary
+ batch_size = std::min(properties().batch_size(),
num_rows[row_group_idx]);
+ num_rows[row_group_idx] -= batch_size;
-+ if (num_rows[row_group_idx] == 0 && (num_rows.size() - 1) !=
row_group_idx) {
-+ row_group_idx++;
++ if (num_rows[row_group_idx] == 0 && row_group_idx + 1 <
num_rows.size()) {
Review Comment:
Confirmed. The base commit already contains the same per-row-group num_rows
vector, row_group_idx advance, and NextBatch behavior in the Arrow 17 patch:
https://github.com/apache/doris/blob/dc4fc151b997a6d1909e01ebdc6bcc60eed5b8a2/thirdparty/patches/apache-arrow-17.0.0-paimon.patch#L19-L57.
This PR only ports that existing patch to Arrow 24, so changing zero-row-group
semantics and coordinating Paimon range-index alignment are outside this PR
scope. They should be addressed in a separate PR with a dedicated
empty-then-populated row-group regression.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
