[
https://issues.apache.org/jira/browse/HDFS-15753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
lujie updated HDFS-15753:
-------------------------
Description:
while we enable kerberos, we found that "hdfs fsck /path -move" always failed.
After checking the log, we find a WARN message:
2020-12-25 13:51:30,485 WARN org.apache.hadoop.ipc.Client: Exception
encountered while connecting to the server :
org.apache.hadoop.security.AccessControlException: Client cannot authenticate
via:[TOKEN, KERBEROS]
2020-12-25 13:51:30,490 WARN org.apache.hadoop.hdfs.server.namenode.NameNode:
Cannot initialize /lost+found.
2020-12-25 13:51:30,491 ERROR org.apache.hadoop.hdfs.server.namenode.NameNode:
copyBlocksToLostFound: error processing /private/file_name_sensitive.txt
java.io.IOException: failed to initialize lost+found
at
org.apache.hadoop.hdfs.server.namenode.NamenodeFsck.copyBlocksToLostFound(NamenodeFsck.java:772)
at
org.apache.hadoop.hdfs.server.namenode.NamenodeFsck.collectBlocksSummary(NamenodeFsck.java:718)
I guess the root cause is that Fsck use DFSClient to do operation like mkdir or
create. But once kerberos is enabled, the client can't do authentication due to
it is now on NameNode, not the client node.
Fixing the root cause is hard, my suggestion we should disable it while
KERBEROS is enabled, or enable it only when authorization is SIMPLE.
was:
while we enable kerberos, we found that "hdfs fsck /path -move" always failed.
After checking the log, we find a WARN message:
2020-12-25 13:51:30,485 WARN org.apache.hadoop.ipc.Client: Exception
encountered while connecting to the server :
org.apache.hadoop.security.AccessControlException: Client cannot authenticate
via:[TOKEN, KERBEROS]
2020-12-25 13:51:30,490 WARN org.apache.hadoop.hdfs.server.namenode.NameNode:
Cannot initialize /lost+found.
2020-12-25 13:51:30,491 ERROR org.apache.hadoop.hdfs.server.namenode.NameNode:
copyBlocksToLostFound: error processing /private/file_name_sensitive.txt
java.io.IOException: failed to initialize lost+found
at
org.apache.hadoop.hdfs.server.namenode.NamenodeFsck.copyBlocksToLostFound(NamenodeFsck.java:772)
at
org.apache.hadoop.hdfs.server.namenode.NamenodeFsck.collectBlocksSummary(NamenodeFsck.java:718)
I guess root cause is that Fsck use DFSClient to do operation like mkdir or
create. But once kerberos is enabled, the client can't do authentication due to
it is now on NameNode, not the client node.
Fixing the root cause is hard, my suggestion we should disable it while
KERBEROS is enabled, or enable it only when authorization is SIMPLE.
> "fsck -move" or "fsck -delete"does not work while enable kerberos
> -----------------------------------------------------------------
>
> Key: HDFS-15753
> URL: https://issues.apache.org/jira/browse/HDFS-15753
> Project: Hadoop HDFS
> Issue Type: Bug
> Reporter: lujie
> Priority: Major
>
> while we enable kerberos, we found that "hdfs fsck /path -move" always failed.
> After checking the log, we find a WARN message:
>
> 2020-12-25 13:51:30,485 WARN org.apache.hadoop.ipc.Client: Exception
> encountered while connecting to the server :
> org.apache.hadoop.security.AccessControlException: Client cannot authenticate
> via:[TOKEN, KERBEROS]
> 2020-12-25 13:51:30,490 WARN
> org.apache.hadoop.hdfs.server.namenode.NameNode: Cannot initialize
> /lost+found.
> 2020-12-25 13:51:30,491 ERROR
> org.apache.hadoop.hdfs.server.namenode.NameNode: copyBlocksToLostFound: error
> processing /private/file_name_sensitive.txt
> java.io.IOException: failed to initialize lost+found
> at
> org.apache.hadoop.hdfs.server.namenode.NamenodeFsck.copyBlocksToLostFound(NamenodeFsck.java:772)
> at
> org.apache.hadoop.hdfs.server.namenode.NamenodeFsck.collectBlocksSummary(NamenodeFsck.java:718)
>
> I guess the root cause is that Fsck use DFSClient to do operation like mkdir
> or create. But once kerberos is enabled, the client can't do authentication
> due to it is now on NameNode, not the client node.
> Fixing the root cause is hard, my suggestion we should disable it while
> KERBEROS is enabled, or enable it only when authorization is SIMPLE.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
