GitHub user adamreeve added a comment to the discussion: [Python] Not able to decrypt the parquet file encrypted using c++ arrow code
Hi @basu45. In your C++ example you are directly setting a footer key using the low-level encryption API. This isn't compatible with PyArrow, which only exposes the higher level "key management tools" API. With this API, the `private_key` you have configured in your `InMemoryKmsClient` isn't the key that decrypts Parquet data, but is instead used to decrypt keys that have been randomly generated by the Arrow library and stored encrypted in the key metadata. There is a design document that describes this API at https://docs.google.com/document/d/1bEu903840yb95k9q2X-BlsYKuXoygE4VnMDl9xz_zhk/edit?usp=sharing. You'll need to also use the key management tools API when writing files in C++ if you need compatibility with PyArrow. I don't think there are any examples or documentation on how to use it, but there are some tests here that could be helpful: https://github.com/apache/arrow/blob/main/cpp/src/parquet/encryption/key_management_test.cc GitHub link: https://github.com/apache/arrow/discussions/45934#discussioncomment-12620372 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
