[
https://issues.apache.org/jira/browse/HADOOP-15006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16320663#comment-16320663
]
Steve Moist commented on HADOOP-15006:
--------------------------------------
{quote}
you are going to have to handle the fact that its only on the final post where
things are manifest later, with a separate process/host doing the final POST.
that's when any info would have to be persisted to DDB
{quote}
That seems reasonable. I had thought to store the OEMI first and have it act
as a the lock to prevent multiple uploads from confusing who's OEMI is theirs.
By inserting it first, it's hard to tell whether or not the object is still
being uploaded or the host/process/jvm/etc has crashed and abandoned the
upload. There's more to discuss than I originally had thought.
{quote}
One thing to consider is that posix seek() lets you do a relative-to-EOF seek.
Could we offer that, or at least an API call to get the length of a data
source, in our input stream. And then modify the core file formats (Sequence,
ORC, Parquet...) to use the length returned in the open call, rather than a
previously cached value?
{quote}
Can you give me a sample scenario to help visualize what your trying to solve
by this.
> Encrypt S3A data client-side with Hadoop libraries & Hadoop KMS
> ---------------------------------------------------------------
>
> Key: HADOOP-15006
> URL: https://issues.apache.org/jira/browse/HADOOP-15006
> Project: Hadoop Common
> Issue Type: New Feature
> Components: fs/s3, kms
> Reporter: Steve Moist
> Priority: Minor
> Attachments: S3-CSE Proposal.pdf
>
>
> This is for the proposal to introduce Client Side Encryption to S3 in such a
> way that it can leverage HDFS transparent encryption, use the Hadoop KMS to
> manage keys, use the `hdfs crypto` command line tools to manage encryption
> zones in the cloud, and enable distcp to copy from HDFS to S3 (and
> vice-versa) with data still encrypted.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
