[ https://issues.apache.org/jira/browse/HBASE-7544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13551699#comment-13551699 ]
Andrew Purtell commented on HBASE-7544: --------------------------------------- {quote} bq. Should we do compression at the HDFS layer? IMO yes, probably {quote} As a longer term project, I wouldn't mind looking at pushing both compression and encryption down into HDFS somehow. I haven't really thought it through. It seems higher risk because of the externialities. > Transparent table/CF encryption > ------------------------------- > > Key: HBASE-7544 > URL: https://issues.apache.org/jira/browse/HBASE-7544 > Project: HBase > Issue Type: New Feature > Components: HFile, io > Affects Versions: 0.96.0 > Reporter: Andrew Purtell > Assignee: Andrew Purtell > > Introduce transparent encryption of HBase on disk data. > Depends on a separate contribution of an encryption codec framework to Hadoop > core and an AES-NI (native code) codec. This is work done in the context of > MAPREDUCE-4491 but I'd gather there will be additional JIRAs for common and > HDFS parts of it. > Requirements: > - Transparent encryption at the CF or table level > - Protect against all data leakage from files at rest > - Two-tier key architecture for consistency with best practices for this > feature in the RDBMS world > - Built-in key management > - Flexible and non-intrusive key rotation > - Mechanisms not exposed to or modifiable by users > - Hardware security module integration (via Java KeyStore) > - HBCK support for transparently encrypted files (+ plugin architecture for > HBCK) > Additional goals: > - Shell support for administrative functions > - Avoid performance impact for the null crypto codec case > - Play nicely with other changes underway: in HFile, block coding, etc. > We're aiming for rough parity with Oracle's transparent tablespace encryption > feature, described in > http://www.oracle.com/technetwork/database/owp-security-advanced-security-11gr-133411.pdf > as > {quote} > “Transparent Data Encryption uses a 2-tier key architecture for flexible and > non-intrusive key rotation and least operational and performance impact: Each > application table with at least one encrypted column has its own table key, > which is applied to all encrypted columns in that table. Equally, each > encrypted tablespace has its own tablespace key. Table keys are stored in the > data dictionary of the database, while tablespace keys are stored in the > header of the tablespace and additionally, the header of each underlying OS > file that makes up the tablespace. Each of these keys is encrypted with the > TDE master encryption key, which is stored outside of the database in an > external security module: either the Oracle Wallet (a PKCS#12 formatted file > that is encrypted using a passphrase supplied either by the designated > security administrator or DBA during setup), or a Hardware Security Module > (HSM) device for higher assurance […]” > {quote} > Further design details forthcoming in a design document and patch as soon as > we have all of the clearances in place. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira