adamreeve commented on code in PR #8305:
URL: https://github.com/apache/arrow-rs/pull/8305#discussion_r2543948275
##########
parquet/src/file/metadata/thrift/mod.rs:
##########
@@ -1197,13 +1197,30 @@ pub(super) fn serialize_column_meta_data<W: Write>(
if let Some(dictionary_page_offset) = column_chunk.dictionary_page_offset {
last_field_id = dictionary_page_offset.write_thrift_field(w, 11,
last_field_id)?;
}
- // PageStatistics is the same as thrift Statistics, but writable
- let stats = page_stats_to_thrift(column_chunk.statistics());
- if let Some(stats) = stats {
- last_field_id = stats.write_thrift_field(w, 12, last_field_id)?;
+
+ // Only write statistics to plaintext footer if column is not encrypted
+
+ #[cfg(feature = "encryption")]
+ if column_chunk.crypto_metadata().is_none() {
Review Comment:
After refactoring the tests and adding some extra test cases, I realised
this logic isn't correct in the case of uniform encryption with an encrypted
footer. In that scenario, the column chunk has `crypto_metadata` present but no
`encrypted_column_metadata`, and we do need to write the statistics here.
I think the fix is just to check for `encrypted_column_metadata` instead of
`crypto_metadata`, like the code was doing previously. I've edited my comment
above to address that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
