Hello Udi, I took a look at your PR and let some coments.Please take a look at them because I think you can achieve what you want in a simpler way and without radical changes to the FileSystem (and File-related) interface(s).
Let's continue the discussion in the PR and then we bring back the subject to the mailing list. Regards, Ismaël On Fri, Jan 18, 2019 at 7:06 PM Udi Meiri <eh...@google.com> wrote: > > Hi Ismaël, > I'd like your feedback, especially from the AWS perspective. > I wasn't aware of BEAM-3821, but I did create a JIRA for Cloud KMS support on > GCS: https://issues.apache.org/jira/browse/BEAM-5959 > > Some details of my plan for KMS support: > 1. Add KMS settings to sources and sinks. > 2. Add a --kmsKey flag that is passed to the runner and applies to pipeline > state. > > On Fri, Jan 18, 2019 at 8:24 AM Ismaël Mejía <ieme...@gmail.com> wrote: >> >> Hello Udi, >> >> I implemented the support for KMS in Amazon and I am really interested >> in check your PR. However I won't have time to do it until next >> monday. I hope waiting a bit is ok with you if you want some feedback >> from me. >> >> I am curious if you considered or are aware of this issue: >> BEAM-3821 Support a pluggable key management system (KMS) >> https://issues.apache.org/jira/browse/BEAM-3821 >> >> >> On Fri, Jan 18, 2019 at 1:51 AM Udi Meiri <eh...@google.com> wrote: >> > >> > Hi, >> > I'd like to add support for creating files using a cloud Key Management >> > System. >> > A KMS allows you to audit, create, rotate, and disable encryption keys. >> > Both AWS and GCP have such a service.. >> > >> > I wanted to show the community what I've been working on and see if there >> > are any comments or objection before submitting a PR. >> > https://github.com/udim/beam/commit/d29f1ef26c58489416a2d413eb029596d96e1f25 >> > >> > Reference docs: >> > AWS S3: >> > https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingKMSEncryption.html >> > GCP GCS: >> > https://cloud.google.com/storage/docs/encryption/using-customer-managed-keys#add-object-key
