All, I'm in the process of setting up encryption at rest on a cluster, but I want to make sure that everything else remains permissive - otherwise it will break existing processes that we have in place. I'm very close to getting this working - the last piece is that webhdfs is not permissive:
In my local setup where I have things working, webhdfs reports the following when trying to create a file (note t=simple): $ curl -i -X PUT 'localhost:50070/webhdfs/v1/tmp/foo?op=CREATE&overwrite=true&user.name=yarn' HTTP/1.1 307 TEMPORARY_REDIRECT Cache-Control: no-cache Expires: Tue, 27 Sep 2016 14:52:06 GMT Date: Tue, 27 Sep 2016 14:52:06 GMT Pragma: no-cache Expires: Tue, 27 Sep 2016 14:52:06 GMT Date: Tue, 27 Sep 2016 14:52:06 GMT Pragma: no-cache Content-Type: application/octet-stream Set-Cookie: hadoop.auth="u=yarn&p=yarn&t=simple&e=1475023926231&s=0wqlgqLNm50k/mN66qZwyCb4xUs="; Path=/; HttpOnly Location: http://localhost:50075/webhdfs/v1/tmp/foo?op=CREATE&user.name=yarn&namenoderpcaddress=localhost:9000&createflag=&createparent=true&overwrite=true Content-Length: 0 Server: Jetty(6.1.26.hwx) On the cluster, however, it reports the following (note t=simple-dt) $ curl -i -X PUT 'http://10.41.1.6:14000/webhdfs/v1/tmp/foo?op=CREATE&overwrite=true&user.name=yarn' HTTP/1.1 307 Temporary Redirect Server: Apache-Coyote/1.1 Set-Cookie: hadoop.auth="u=yarn&p=yarn&t=simple-dt&e=1475023818932&s=9FteGx9VW06bh5dD1L9J+1ENWtY="; Path=/; HttpOnly Location: http://10.41.1.6:14000/webhdfs/v1/tmp/foo?op=CREATE&user.name=yarn&overwrite=true&data=true Content-Type: application/json Content-Length: 0 Date: Tue, 27 Sep 2016 14:50:18 GMT Note that my local setup reports the authentication type as simple whereas the cluster reports simple-dt. This is the reason why I'm getting an authentication failure when trying to write a file to the cluster. I don't want Keberos or delegation tokens enabled. Does anyone know what I need to change so that this becomes simple again? Thanks in advance, Ben This message has been scanned for malware by Websense. www.websense.com
