[
https://issues.apache.org/jira/browse/PARQUET-1396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16969043#comment-16969043
]
Gabor Szadovszky commented on PARQUET-1396:
-------------------------------------------
As it is related to encryption which is not targeted to 1.11.0 I'm removing
target from here.
> Cryptodata Interface for Schema Activation of Parquet Encryption
> ----------------------------------------------------------------
>
> Key: PARQUET-1396
> URL: https://issues.apache.org/jira/browse/PARQUET-1396
> Project: Parquet
> Issue Type: New Feature
> Components: parquet-mr
> Affects Versions: 1.10.0, 1.10.1
> Reporter: Xinli Shang
> Priority: Major
> Fix For: 1.11.0
>
>
> This JIRA is an extension to Parquet Modular Encryption Jira(PARQUET-1178)
> that will provide the basic building blocks and APIs for the encryption
> support.
> This JIRA provides a crypto data interface for schema activation of Parquet
> encryption and serves as a high-level layer on top of PARQUET-1178 to make
> the adoption of Parquet-1178 easier, with pluggable key access module,
> without a need to use the low-level encryption APIs. Also, this feature will
> enable seamless integration with existing clients.
> No change to specifications (Parquet-format), no new Parquet APIs, and no
> changes in existing Parquet APIs. All current applications, tests, etc, will
> work.
> From developer perspective, they can just implement the interface into a
> plugin which can be attached any Parquet application like Hive/Spark etc.
> This decouples the complexity of dealing with KMS and schema from Parquet
> applications. In large organization, they may have hundreds or even thousands
> of Parquet applications and pipelines. The decoupling would make Parquet
> encryption easier to be adopted.
> From end user(for example data owner) perspective, if they think a column is
> sensitive, they can just set that column’s schema as sensitive and then the
> Parquet application just encrypt that column automatically. This makes end
> user easy to manage the encryptions of their columns.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
