autoCAT6 opened a new pull request, #54884: URL: https://github.com/apache/arrow/pull/54884
### Rationale for this change Closes #51669. When writing AES_GCM_CTR_V1 files with a plaintext footer, the writer currently records AES_GCM_V1 in FileMetaData.encryption_algorithm. Readers consequently select GCM for column pages that were written using CTR, and fail while reading encrypted columns. ### What changes are included in this PR? - Serialize the configured file encryption algorithm in the plaintext footer, retaining the existing rule that externally supplied AAD prefixes are omitted from serialized metadata. Footer signing continues to use GCM. - Add a self-contained generated-file regression matrix: GCM/CTR × encrypted/plaintext footer × no/stored/external AAD prefix. Verify the serialized algorithm and AAD fields, all column values through synchronous and asynchronous readers, keyless public-column reads, unavailable column keys, incorrect footer keys, and external-prefix requirements. - The generated tests use local temporary files and do not require new parquet-testing fixtures. The original report is by @dotnwat in #51669; #51693 is an earlier closed fix attempt for the same issue. ### Are these changes tested? - Before the source fix, the new matrix passes 9 cases and fails exactly the 3 CTR/plaintext-footer cases on the algorithm assertion. - `parquet-encryption-test`: 45 passed, including all 12 generated cases. - `parquet-internals-test --gtest_filter='Metadata.*:ApplicationVersion.*:TestReaderProperties.*:TestWriterProperties.*:WriterPropertiesTest/*'`: 50 passed. - `parquet-writer-test`: 296 passed; 2 existing tests are disabled. - clang-format 18.1.8 and `git diff --check` pass. - Local interoperability with released parquet-java 1.18.1: 2 writers × 2 readers × 4 algorithm/footer configurations, with full reads, keyless public projection, unavailable column keys and incorrect footer keys. All 64 read outcomes matched their controls, and 8 raw Java Thrift metadata inspections matched the requested file algorithms. The baseline C++ CTR/plaintext file fails in both readers; the repaired file succeeds in both. This uses the released Java artifact, not a Java master build. The interoperability checks use public synthetic keys and a 32-row schema with one public column and two separately encrypted columns. Keyless public-column projection does not authenticate the footer; CTR data pages do not provide an authentication guarantee. ### Are there any user-facing changes? New CTR/plaintext-footer files declare the correct file algorithm and their encrypted columns can be read. No public API or configuration changes. Previously written malformed files are not rewritten by this fix. **This PR contains a "Critical Fix"**: it corrects invalid encryption algorithm metadata emitted by the writer for the affected configuration. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
