[
https://issues.apache.org/jira/browse/HDFS-15753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
lujie updated HDFS-15753:
-------------------------
Description:
while we enable kerberos, we found that "hdfs fsck /path -move" always failed.
After checking the log, we find a WARN message:
2020-12-25 13:51:30,485 WARN org.apache.hadoop.ipc.Client: Exception
encountered while connecting to the server :
org.apache.hadoop.security.AccessControlException: Client cannot authenticate
via:[TOKEN, KERBEROS]
2020-12-25 13:51:30,490 WARN org.apache.hadoop.hdfs.server.namenode.NameNode:
Cannot initialize /lost+found.
2020-12-25 13:51:30,491 ERROR org.apache.hadoop.hdfs.server.namenode.NameNode:
copyBlocksToLostFound: error processing /private/file_name_sensitive.txt
java.io.IOException: failed to initialize lost+found
at
org.apache.hadoop.hdfs.server.namenode.NamenodeFsck.copyBlocksToLostFound(NamenodeFsck.java:772)
at
org.apache.hadoop.hdfs.server.namenode.NamenodeFsck.collectBlocksSummary(NamenodeFsck.java:718)
The root cause is Fsck use DFSClient to do operation like mkdir or create. But
once kerberos is enabled, the client can't do authentication due to it is now
on NameNode, not the client node.
Fixing the root cause is hard, so when should disable it while KERBEROS is
enabled, or enable it while authorization is SIMPLE.
was:
while we enable kerberos, we found that "hdfs fsck /path -move" always failed.
After checking the log, we find a WARN message:
2020-12-25 13:51:30,485 WARN org.apache.hadoop.ipc.Client: Exception
encountered while connecting to the server :
org.apache.hadoop.security.AccessControlException: Client cannot authenticate
via:[TOKEN, KERBEROS]
2020-12-25 13:51:30,490 WARN org.apache.hadoop.hdfs.server.namenode.NameNode:
Cannot initialize /lost+found.
2020-12-25 13:51:30,491 ERROR org.apache.hadoop.hdfs.server.namenode.NameNode:
copyBlocksToLostFound: error processing /private/file_name_sensitive.txt
java.io.IOException: failed to initialize lost+found
at
org.apache.hadoop.hdfs.server.namenode.NamenodeFsck.copyBlocksToLostFound(NamenodeFsck.java:772)
at
org.apache.hadoop.hdfs.server.namenode.NamenodeFsck.collectBlocksSummary(NamenodeFsck.java:718)
The root cause is Fsck use DFSClient to do operation like mkdir or create. But
once kerberos is enabled, the client can't do authentication due to it is now
on NameNode, not the client node.
Fixing the root cause is hard, so when should disable it while kerberos is
enabled.
> "fsck -move" does not work while enable kerberos
> ------------------------------------------------
>
> Key: HDFS-15753
> URL: https://issues.apache.org/jira/browse/HDFS-15753
> Project: Hadoop HDFS
> Issue Type: Bug
> Reporter: lujie
> Priority: Major
>
> while we enable kerberos, we found that "hdfs fsck /path -move" always failed.
> After checking the log, we find a WARN message:
>
> 2020-12-25 13:51:30,485 WARN org.apache.hadoop.ipc.Client: Exception
> encountered while connecting to the server :
> org.apache.hadoop.security.AccessControlException: Client cannot authenticate
> via:[TOKEN, KERBEROS]
> 2020-12-25 13:51:30,490 WARN
> org.apache.hadoop.hdfs.server.namenode.NameNode: Cannot initialize
> /lost+found.
> 2020-12-25 13:51:30,491 ERROR
> org.apache.hadoop.hdfs.server.namenode.NameNode: copyBlocksToLostFound: error
> processing /private/file_name_sensitive.txt
> java.io.IOException: failed to initialize lost+found
> at
> org.apache.hadoop.hdfs.server.namenode.NamenodeFsck.copyBlocksToLostFound(NamenodeFsck.java:772)
> at
> org.apache.hadoop.hdfs.server.namenode.NamenodeFsck.collectBlocksSummary(NamenodeFsck.java:718)
>
> The root cause is Fsck use DFSClient to do operation like mkdir or create.
> But once kerberos is enabled, the client can't do authentication due to it is
> now on NameNode, not the client node.
> Fixing the root cause is hard, so when should disable it while KERBEROS is
> enabled, or enable it while authorization is SIMPLE.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
