Re: [DISCUSS] What to do about encryption at rest?

Can we file some JIRAs to build out a suite to test this and run the necessary tests? On Thu, Nov 5, 2015 at 11:17 AM, Christopher wrote: > My main concern using HDFS encryption vs. built-in Accumulo implementation > is possibly performance with respect to seeks. If we

Re: [DISCUSS] What to do about encryption at rest?

+1 I think this is the right step. My hunch is that some of the common data access patterns that we have in Accumulo (over HBase) is that the per-colfam encryption isn't quick as common a design pattern as it is for HBase (please tell me I'm wrong if anyone disagrees -- this is mostly a gut

Re: [DISCUSS] What to do about encryption at rest?

My main concern using HDFS encryption vs. built-in Accumulo implementation is possibly performance with respect to seeks. If we encrypt our indexed blocks independently (as we do now), I suspect our seeks would be more performant than relying on HDFS encryption, whose encrypted blocks may not fall

Re: [DISCUSS] What to do about encryption at rest?

JIRAs are fine, but I thought this thread was mostly addressing the fact that there doesn't seem to be a sustained interest in actually working on any of the JIRAs addressing that area of code. Am I wrong? Is there willingness from anybody to expend effort on this code? Even if not, we can still

Re: [DISCUSS] What to do about encryption at rest?

I think you have misidentified the two camps. There is a camp that believes we should phase out the code in favour of the HDFS encryption, and a camp that believes the code is sufficiently mature. I don't think there is a group that is interested in improving the state of things. On Thu, Nov 5,

Re: [DISCUSS] What to do about encryption at rest?

On Thu, Nov 5, 2015 at 12:17 PM, Christopher wrote: > My main concern using HDFS encryption vs. built-in Accumulo implementation > is possibly performance with respect to seeks. If we encrypt our indexed > blocks independently (as we do now), I suspect our seeks would be

Re: [DISCUSS] What to do about encryption at rest?

> Does anybody have a good diagram showing the architecture of HDFS encryption? Related: Can we collect the digram and design docs from the various implementation JIRAs and put them up on the Accumulo website? Every time that I've needed to reference them it's been a giant pain to go find them.

Re: [DISCUSS] What to do about encryption at rest?

Perhaps. I had interpreted some of Adam's comments ("The only thing that doesn't get encrypted is a temporary WAL recovery file. That is a project we should take on..."), as favoring improvements to the current state of things. As that has also been the focus of previous conversations about the

Re: [DISCUSS] What to do about encryption at rest?

With regards to adding features, it probably makes sense to talk about adding table/namespace crypto configuration separately from column-level encryption. Column-level encryption would require big changes to how we partition data, how we organize configuration information, and how we handle

Re: [DISCUSS] What to do about encryption at rest?

Just to moonwalk back a bit, I see a few things happening concurrently now. First is trying to get a consensus on where we want to go with the encryption at rest story in Accumulo. I see us having established that what we have is scoped down to working for WALs and RFiles, and if you happen to

Re: [DISCUSS] What to do about encryption at rest?

SGTM William Slacum wrote: Just to moonwalk back a bit, I see a few things happening concurrently now. First is trying to get a consensus on where we want to go with the encryption at rest story in Accumulo. I see us having established that what we have is scoped down to working for WALs and