rok commented on code in PR #41821:
URL: https://github.com/apache/arrow/pull/41821#discussion_r1706158737
##########
cpp/src/parquet/file_writer.cc:
##########
@@ -567,6 +567,44 @@ void WriteEncryptedFileMetadata(const FileMetaData&
file_metadata,
}
}
+void WriteEncryptedMetadataFile(
+ const FileMetaData& metadata, std::shared_ptr<::arrow::io::OutputStream>
sink,
+ std::shared_ptr<FileEncryptionProperties> file_encryption_properties) {
+ auto file_encryptor = std::make_unique<InternalFileEncryptor>(
+ file_encryption_properties.get(), ::arrow::default_memory_pool());
+
+ if (file_encryption_properties->encrypted_footer()) {
+ PARQUET_THROW_NOT_OK(sink->Write(kParquetEMagic, 4));
+
+ PARQUET_ASSIGN_OR_THROW(int64_t position, sink->Tell());
+ auto metadata_start = static_cast<uint64_t>(position);
+
+ auto writer_props = parquet::WriterProperties::Builder()
+ .encryption(file_encryption_properties)
+ ->build();
+ auto builder = FileMetaDataBuilder::Make(metadata.schema(), writer_props);
+
+ auto footer_metadata = builder->Finish(metadata.key_value_metadata());
+ auto crypto_metadata = builder->GetCryptoMetaData();
+ WriteFileCryptoMetaData(*crypto_metadata, sink.get());
+
+ auto footer_encryptor = file_encryptor->GetFooterEncryptor();
+ WriteEncryptedFileMetadata(metadata, sink.get(), footer_encryptor, true);
+ PARQUET_ASSIGN_OR_THROW(position, sink->Tell());
+ auto footer_and_crypto_len = static_cast<uint32_t>(position -
metadata_start);
+ PARQUET_THROW_NOT_OK(
+ sink->Write(reinterpret_cast<uint8_t*>(&footer_and_crypto_len), 4));
+ PARQUET_THROW_NOT_OK(sink->Write(kParquetEMagic, 4));
+ } else {
Review Comment:
I'm doing metadata encryption similarly to approach here:
https://github.com/apache/arrow/blob/a4d58e0c8213cd14d0953c75eb047ea1693fddc9/cpp/src/parquet/file_writer.cc#L416-L432
However it seems decryption fails (see below) when using `RowGroup` metadata
(after deserialization and decryption).
https://github.com/apache/arrow/blob/4d5041a3eff486cd82c6790f6dae63fba2321474/cpp/src/arrow/dataset/file_parquet.cc#L1084-L1086
This makes me think I'm either not serializing correctly or there's an issue
with encryption/decryption properties I'm supplying.
@wgtmac @pitrou does anything obviously wrong stands out here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
