[
https://issues.apache.org/jira/browse/MAPREDUCE-5890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13998426#comment-13998426
]
Alejandro Abdelnur commented on MAPREDUCE-5890:
-----------------------------------------------
HADOOP-10603 introduces crypto streams to be used by for filesystem
encryption. We could leverage it for encrypting map output data, the Reducer
shuffle would decrypt it (no need for network encryption as data would be
encrypted in transit). The reducer, when writing spills to disk woudl encrypt
and it would decrypt while reading the spills.
It may make sense to do this JIRA as part of fs-encryption branch.
> Support for encrypting Intermediate data and spills in local filesystem
> -----------------------------------------------------------------------
>
> Key: MAPREDUCE-5890
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5890
> Project: Hadoop Map/Reduce
> Issue Type: New Feature
> Components: security
> Affects Versions: 2.4.0
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
>
> For some sensitive data, encryption while in flight (network) is not
> sufficient, it is required that while at rest it should be encrypted.
> HADOOP-10150 & HDFS-6134 bring encryption at rest for data in filesystem
> using Hadoop FileSystem API. MapReduce intermediate data and spills should
> also be encrypted while at rest.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
