HOD may be too heavy weight for us with small cluster and small number of users.
From hadoop summit, I heard Kerberos authentication is in the pipeline, is there a place I can check on the progress? It seems that authentication/authorization work is from the perspective of file system, but not from the perspective of file encryption and key distribution among map reduce tasks. "If and when something like kerberos user authentication exists, then kerberos tickets may be the reasonable alternative for opening the keyring." Ted, Do you mean instead of "insert an auth key to the job conf", we can insert the ticket to the job conf? even though the job conf itself can be compromised, since the ticket is short lived, other people can't use the ticket to decrypt the file later. Is my understanding right? Thanks Haijun -----Original Message----- From: Ted Dunning [mailto:[EMAIL PROTECTED] Sent: Thursday, June 05, 2008 11:58 AM To: core-user@hadoop.apache.org Subject: Re: compressed/encrypted file Security and hadoop are not particularly compatible concepts. Things may improve when user authentication exists. The lack of security on job confs is the major motivation for making sure the auth is time limited. If and when something like kerberos user authentication exists, then kerberos tickets may be the reasonable alternative for opening the keyring. Can you suggest an alternative way to communicate a secret to hadoop tasks short of embedding it into source code? On Thu, Jun 5, 2008 at 11:46 AM, Allen Wittenauer <[EMAIL PROTECTED]> wrote: > On 6/5/08 11:38 AM, "Ted Dunning" <[EMAIL PROTECTED]> wrote: > > We use encryption on log files using standard AES. I wrote an input > format > > to deal with it. > > > > Key distribution should be done better than we do it. My preference > would > > be to insert an auth key into the job conf which is then used by the > input > > to open a well known keyring via an API that prevents auths from > surviving > > for long term. > > This sounds like it opens the door for key stealing in a > multi-user/static job tracker system, since the job conf is readable by all > jobs running on the same machine. > > -- ted
