Hi,
After upgrading Hadoop from 3.2.4 to 3.4.1 it seems I have lost Kerberos authentication on webhdfs - I can request everything as long as I provide a 'user.name' parameter (during testing I thought that 'user.name' was now mandatory and modified our webhdfs-client accordingly). I probably configured something wrong, but I can't figure out what. The documentation says: > When security is off, the authenticated user is the username > specified in the user.name query parameter. If the user.name > parameter is not set, the server may either set the authenticated > user to a default web user, if there is any, or return an error > response. > When security is on, authentication is performed by either Hadoop > delegation token or Kerberos SPNEGO. · https://hadoop.apache.org/docs/r3.4.1/hadoop-project-dist/hadoop-hdfs/WebHDFS.html#Authentication The behaviour I'm seeing matches "security is off": $ curl 'http://SERVER:50070/webhdfs/v1/apps/hbase?op=LISTSTATUS&user.name=testuser' {"FileStatuses":{"FileStatus":[ {"accessTime":0,"blockSize":0,"childrenNum":4,"fileId":20349,"group":"hadoop","length":0,"modificationTime":1629663445936,"owner":"hbase","pathSuffix":".hbase-snapshot","permission":"755","replication":0,"storagePolicy":0,"type":"DIRECTORY"}, {"accessTime":0,"blockSize":0,"childrenNum":1,"fileId":20618,"group":"hadoop","length":0,"modificationTime":1629662466786,"owner":"hbase","pathSuffix":"archive","permission":"755","replication":0,"storagePolicy":0,"type":"DIRECTORY"}, {"accessTime":0,"blockSize":0,"childrenNum":14,"fileId":16917,"group":"hadoop","length":0,"modificationTime":1757327019008,"owner":"hbase","pathSuffix":"data","permission":"755","replication":0,"storagePolicy":0,"type":"DIRECTORY"}, {"accessTime":0,"blockSize":0,"childrenNum":1,"fileId":23012,"group":"hadoop","length":0,"modificationTime":1629662701362,"owner":"hbase","pathSuffix":"staging","permission":"755","replication":0,"storagePolicy":0,"type":"DIRECTORY"} ]}} How do I turn Kerberos SPNEGO security on for webhdfs? In hdfs-site.xml I have: = = = <property> <name>dfs.web.authentication.kerberos.keytab</name> <value>{{ KEYTAB_DIR }}/spnego.service.keytab</value> </property> <property> <name>dfs.web.authentication.kerberos.principal</name> <value>HTTP/_HOST@REALM</value> </property> <property> <name>dfs.webhdfs.enabled</name> <value>true</value> </property> = = = and in core-site.xml: = = = <property> <name>hadoop.security.authentication</name> <value>kerberos</value> </property> <property> <name>hadoop.security.authorization</name> <value>true</value> </property> = = = Best regards, Adam -- "No more than that, but very powerful all the Adam Sjøgren same; simple things are good." [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
