Gabriel39 commented on code in PR #70314:
URL: https://github.com/apache/doris/pull/70314#discussion_r4489435948
##########
be/src/format_v2/parquet/parquet_crypto.cpp:
##########
@@ -167,6 +167,19 @@ Status deserialize_crypto_metadata(std::span<const
uint8_t> bytes, uint32_t* con
return deserialize_bounded(bytes, consumed, metadata);
}
+Status deserialize_crypto_metadata(std::span<const uint8_t> bytes, uint32_t*
consumed,
+ tparquet::ColumnIndex* metadata) {
+ return deserialize_bounded(bytes, consumed, metadata);
Review Comment:
Fixed in 7c3c320c7dc0dee5183ef8b32c06a73f248c9086.
ColumnIndex and OffsetIndex now use an index-specific bounded protocol.
Their list counts follow the decoded native-byte budget instead of the footer's
65,536-element cap, and histogram entries are charged as int64_t values. Footer
and page-header limits remain unchanged. Allocation accounting uses the
expected field types rather than trusting the advertised wire element types,
with conservative accounting for unknown or skipped containers.
Added real Arrow 24 encrypted-file regressions for 22,000 pages with 66,000
histogram bins and 30,000 pages with 3.03 million bins. Both fail before the
fix and pass afterward, preserving the complete histograms. The selected ASAN
suites passed: 510 tests passed and 2 existing optional benchmarks were skipped.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
