[
https://issues.apache.org/jira/browse/MAPREDUCE-5890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14043926#comment-14043926
]
Vinod Kumar Vavilapalli commented on MAPREDUCE-5890:
----------------------------------------------------
bq. Asking to install additional soft to encrypt local FS means installing
Kernel modules.
Agreed, I wasn't trying to make it look a simple step.
bq. Also, this would mean that ALL MR jobs are going to pay the penalty of
encrypted intermediate data. That is not reasonable.
That's a fair argument, but see below..
bq. I don't agree on the statement that this is "adding a lot of complexity",
it is simply wrapping the streams where necessary.
bq. This patch is not touching anything in Yarn, but in MapReduce,
private/evolving classes of it.
I didn't mean that we are touching YARN here. I meant that we will have to keep
implementing this for _every_ framework that runs on YARN - MR, YARN, Storm,
Spark, HBase and the list goes on an on. I am trying to trade off that
complexity in software with an admin prerequisite to install one or few
disks/partitions that selective users can chose to use via their
job-configuration.
> Support for encrypting Intermediate data and spills in local filesystem
> -----------------------------------------------------------------------
>
> Key: MAPREDUCE-5890
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5890
> Project: Hadoop Map/Reduce
> Issue Type: New Feature
> Components: security
> Affects Versions: 2.4.0
> Reporter: Alejandro Abdelnur
> Assignee: Arun Suresh
> Labels: encryption
> Attachments: MAPREDUCE-5890.3.patch, MAPREDUCE-5890.4.patch,
> MAPREDUCE-5890.5.patch, MAPREDUCE-5890.6.patch, MAPREDUCE-5890.7.patch,
> MAPREDUCE-5890.8.patch,
> org.apache.hadoop.mapred.TestMRIntermediateDataEncryption-output.txt,
> syslog.tar.gz
>
>
> For some sensitive data, encryption while in flight (network) is not
> sufficient, it is required that while at rest it should be encrypted.
> HADOOP-10150 & HDFS-6134 bring encryption at rest for data in filesystem
> using Hadoop FileSystem API. MapReduce intermediate data and spills should
> also be encrypted while at rest.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
