[ https://issues.apache.org/jira/browse/PARQUET-1178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17098529#comment-17098529 ]
Venkata Satya Pradeep Srikakolapu commented on PARQUET-1178: ------------------------------------------------------------ Thank you [~gershinsky] for quick reply. Could you please point me to an example for implementing this feature in Apache arrow? I am interested to understand key management for encrypting/decrypting columns with Apache Arrow. I am working with a customer from Health Care space. My customer wants to encrypt sensitive columns while persisting data to the disk (Data Lake). I see few options for column encryption & Key Management # Apache Arrow with Python - Do you recommend to use Apache Arrow with Scala? # Parquet - MR - Not released yet # Encryption With ORC files (similar to Parquet Modular encryption) - [https://jira.apache.org/jira/browse/ORC-14?jql=text%20~%20%22column%20level%20encryption%20to%20ORC%20files%22] *Some context*: My customer is using Apache Parquet with scala + Spark extensively. My customer is also planning to use Python with Parquet. Could you please recommend what would be the best choice? > Parquet modular encryption > -------------------------- > > Key: PARQUET-1178 > URL: https://issues.apache.org/jira/browse/PARQUET-1178 > Project: Parquet > Issue Type: New Feature > Reporter: Gidon Gershinsky > Assignee: Gidon Gershinsky > Priority: Major > > A mechanism for modular encryption and decryption of Parquet files. Allows to > keep data fully encrypted in the storage - while enabling efficient analytics > on the data, via reader-side extraction / authentication / decryption of data > subsets required by columnar projection and predicate push-down. > Enables fine-grained access control to column data by encrypting different > columns with different keys. > Supports a number of encryption algorithms, to account for different security > and performance requirements. -- This message was sent by Atlassian Jira (v8.3.4#803005)