github-actions[bot] commented on code in PR #66221:
URL: https://github.com/apache/doris/pull/66221#discussion_r3704899243
##########
thirdparty/download-thirdparty.sh:
##########
@@ -438,22 +452,31 @@ if [[ " ${TP_ARCHIVES[*]} " =~ " ARROW " ]]; then
fi
cd -
fi
- if [[ "${ARROW_SOURCE}" == "arrow-apache-arrow-17.0.0" ]]; then
- cd "${TP_SOURCE_DIR}/${ARROW_SOURCE}"
- if [[ ! -f "${PATCHED_MARK}" ]]; then
+ if [[ "${ARROW_SOURCE}" == "arrow-apache-arrow-24.0.0" ]]; then
+
arrow_fingerprint_mark="${TP_SOURCE_DIR}/${ARROW_SOURCE}/${ARROW_PAIMON_PATCH_FINGERPRINT_MARK}"
+ if [[ ! -f "${TP_SOURCE_DIR}/${ARROW_SOURCE}/${PATCHED_MARK}" ||
+ ! -f "${arrow_fingerprint_mark}" ||
+ "$(<"${arrow_fingerprint_mark}")" !=
"${ARROW_PAIMON_BUILD_FINGERPRINT}" ]]; then
+ if [[ -f "${TP_SOURCE_DIR}/${ARROW_SOURCE}/${PATCHED_MARK}" ||
Review Comment:
[P2] Recover unmarked partial patch applications
This reset runs only when at least one completion marker exists. If the
first Arrow patch succeeds and a later patch fails (or the process is
cancelled) before lines 475-477 write either marker, the source tree is already
modified but the next documented rerun skips re-extraction. It then reapplies
the first patch and exits under `set -e`; Paimon has the same transition at
lines 777-791. Please re-extract/stage whenever an existing tree lacks the
matching completed fingerprint (including an unmarked tree), or apply the patch
set atomically, and test interruption after each patch boundary.
##########
thirdparty/build-thirdparty.sh:
##########
@@ -2209,6 +2210,7 @@ build_paimon_cpp() {
fi
echo "Paimon-cpp internal dependencies installed successfully"
+ arrow_paimon_build_fingerprint
>"${TP_INSTALL_DIR}/arrow-paimon-build-fingerprint.txt"
Review Comment:
[P2] Do not publish a combined fingerprint from a Paimon-only build
Package arguments are independent, so `./build-thirdparty.sh paimon_cpp` and
`--continue paimon_cpp` reuse whatever Arrow is already installed. This
unconditional write nevertheless advances a fingerprint that also hashes Arrow
patches and build inputs. After an ABI-compatible Arrow-only change (for
example, the LZO patch), Paimon can compile against the old Arrow 24 archives
and stamp them current; the macOS validator then sees the matching hash, the
same version, and all files, and accepts the stale prebuilt. Please publish the
combined marker only after both Arrow and Paimon were rebuilt/validated for
this hash, or maintain and require separate component fingerprints.
##########
be/test/format_v2/parquet/parquet_reader_test.cpp:
##########
@@ -913,16 +918,32 @@ void write_sparse_filter_nested_parquet_file(const
std::string& file_path) {
map_keys.push_back(6000);
auto* map_key_writer =
static_cast<::parquet::Int32Writer*>(row_group->NextColumn());
-
EXPECT_EQ(map_key_writer->WriteBatch(static_cast<int64_t>(map_repetition_levels.size()),
- map_key_definition_levels.data(),
+ constexpr int64_t MAP_PREFIX_LEVELS = 4;
+ constexpr int64_t MAP_KEY_PREFIX_VALUES = 2;
+ constexpr int64_t MAP_SPLIT_LEVELS = MAP_PREFIX_LEVELS +
SPANNING_BATCH_VALUES;
+ constexpr int64_t MAP_KEY_SPLIT_VALUES = MAP_KEY_PREFIX_VALUES +
SPANNING_BATCH_VALUES;
+ EXPECT_EQ(map_key_writer->WriteBatch(MAP_SPLIT_LEVELS,
map_key_definition_levels.data(),
Review Comment:
[P1] Make this fixture actually cross a page boundary
Splitting the record across these `WriteBatch` calls does not force Arrow 24
to flush a physical page. On current head `b19661b`, BE UT build 1013563 fails
this changed test at its continuation-page oracle: `m.key_value.key` has page
starts `{ 0, 0 }`, so `saw_continuation_page` is false. This leaves the
required BE UT red and means the nested reader regression never exercises a
parent row spanning physical pages. Please construct a deterministic V1 fixture
whose later page actually begins with repetition level greater than zero for
each checked path, retain the physical-page assertions, and verify this test
passes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
