yigal-rozenberg commented on issue #1582: URL: https://github.com/apache/iceberg-python/issues/1582#issuecomment-2620105176
Before posting this as a proper improvement request, I would like to come up with a POC that demonstrate the desired functionality/ The thread you provided talks about the need for proper data centric security, and I have some years of experience in this topic. IMHO the best way to secure data and centrally control access is to use data item encryption. In some cases this can also be referred to column level encryption, however, one can confuse this with file encryption in column based data. When data items are encrypted, the cipher text can be sent and shared/accessed across multiple systems and engines. The challenge is that cipher text by itself does not include metadata such as the key-id used to encrypt it, and the original data type of the clear text. I am trying to understand, as a first phase, how in Iceberg Python interface I can crate a new Data Type, which has a different behavior when it stores and reads the data from the table storage, and a different behavior when data is inserted/updated/selected. Do I need to implement different __str__ / __repr__ methods in the new data type, or I need to do it elsewhere? Where to implement the operators to support operations between 2 encrypted types, and operations between encrypted and clear text? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
