Jose Luis López created MAPREDUCE-7548:
------------------------------------------
Summary: MapReduce tasks are rejected by the AM when
hadoop.security.authorization is enabled: TaskUmbilicalProtocol is not known
Key: MAPREDUCE-7548
URL: https://issues.apache.org/jira/browse/MAPREDUCE-7548
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mr-am, security
Affects Versions: 3.6.0
Reporter: Jose Luis López
*Problem*: on a Kerberos cluster with service-level authorization
({{hadoop.security.authorization=true}}), every MapReduce job fails. Each
task's connection to its application master is refused, and the task exits:
{noformat}
WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child :
java.io.UncheckedIOException:
org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException):
Protocol interface org.apache.hadoop.mapred.TaskUmbilicalProtocol is not known.
{noformat}
*Cause*: the protocol name and the ACL key do not match.
* {{TaskUmbilicalProtocolPB}} declares {{@ProtocolInfo(protocolName =
"org.apache.hadoop.mapred.TaskUmbilicalProtocol")}}, so the RPC server resolves
an incoming call to the {{TaskUmbilicalProtocol}} interface and authorizes it
against that class.
* {{MRAMPolicyProvider}} registers {{security.job.task.protocol.acl}} under
{{TaskUmbilicalProtocolPB.class}}, so {{ServiceAuthorizationManager}} has no
ACL for {{TaskUmbilicalProtocol}} and rejects the call.
HDFS uses the other convention, which works: {{HDFSPolicyProvider}} lists
{{ClientProtocol.class}} for {{ClientNamenodeProtocolPB}}, whose protocol name
is {{org.apache.hadoop.hdfs.protocol.ClientProtocol}}. {{MRClientProtocolPB}}
is consistent too, since its protocol name is its own class name.
*To reproduce*: a secure single-node cluster (Kerberos,
{{hadoop.security.authorization=true}}), then any job, e.g. {{yarn jar
hadoop-mapreduce-examples-*.jar pi 2 100}}. All task attempts fail with the
exception above. Seen on trunk (3.6.0-SNAPSHOT).
Found by an end-to-end test of a secure single-node cluster built from trunk.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
