[
https://issues.apache.org/jira/browse/YARN-4764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15184440#comment-15184440
]
Sunil G commented on YARN-4764:
-------------------------------
Thanks [~bibinchundatt] for the analysis and suggestions..
I will try summarizing the discussion so far. Agreeing to the fact that the
current behavior is inconsistent, we have to give similar way of handling for
queue-non-existent scenario (with and w/o ACL). So we had 2 options.
1. For CS alone, queue-non-existent check could be done in
{{createAndPopulateRMApp}}. Being said this, it will be common for apps with or
w/o ACL enabled. This will make App to be rejected before even RMApp is
created, hence audit logging is needed.
2. Or we could skip the ACL check if queue is non-existent and can pass to
Scheduler inside so that it can send {{APP_REJECT}}. This will be inline with
the old behavior. A minor drawback will be like, we know queue is not existing,
and still we send scheduler for a know failure handling.
I had an offline talk with [~jianhe] also on this. May be we can go with Option
2 for now. This will make a consistent behavior. But we need to improve here.
So I can raise an improvement ticket and all queue related validation check can
be done in a new YarnScheduler api. We can see how much we can make it common
for Fair and CS too.
Thoughts?
> Application submission fails when submitted queue is not available in
> scheduler xml
> -----------------------------------------------------------------------------------
>
> Key: YARN-4764
> URL: https://issues.apache.org/jira/browse/YARN-4764
> Project: Hadoop YARN
> Issue Type: Bug
> Reporter: Bibin A Chundatt
> Assignee: Bibin A Chundatt
> Attachments: 0001-YARN-4764.patch, 0002-YARN-4764.patch
>
>
> Available queues in capacity scheduler
> -root
> --queue1
> --queue2
> Submit application with queue3
> {noformat}
> 16/03/04 16:40:08 INFO mapreduce.JobSubmitter: Submitting tokens for job:
> job_1457077554812_1901
> 16/03/04 16:40:08 INFO mapreduce.JobSubmitter: Kind: HDFS_DELEGATION_TOKEN,
> Service: ha-hdfs:hacluster, Ident: (HDFS_DELEGATION_TOKEN token 3938 for
> mapred with renewer yarn)
> 16/03/04 16:40:08 WARN retry.RetryInvocationHandler: Exception while invoking
> class
> org.apache.hadoop.yarn.api.impl.pb.client.ApplicationClientProtocolPBClientImpl.submitApplication
> over rm2. Not retrying because try once and fail.
> java.lang.NullPointerException: java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.server.resourcemanager.RMAppManager.createAndPopulateNewRMApp(RMAppManager.java:366)
> at
> org.apache.hadoop.yarn.server.resourcemanager.RMAppManager.submitApplication(RMAppManager.java:289)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.submitApplication(ClientRMService.java:618)
> at
> org.apache.hadoop.yarn.api.impl.pb.service.ApplicationClientProtocolPBServiceImpl.submitApplication(ApplicationClientProtocolPBServiceImpl.java:252)
> at
> org.apache.hadoop.yarn.proto.ApplicationClientProtocol$ApplicationClientProtocolService$2.callBlockingMethod(ApplicationClientProtocol.java:483)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:637)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:989)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2305)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2301)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1742)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2301)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
> at
> org.apache.hadoop.yarn.ipc.RPCUtil.instantiateException(RPCUtil.java:53)
> at
> org.apache.hadoop.yarn.ipc.RPCUtil.instantiateRuntimeException(RPCUtil.java:85)
> at
> org.apache.hadoop.yarn.ipc.RPCUtil.unwrapAndThrowException(RPCUtil.java:122)
> at
> org.apache.hadoop.yarn.api.impl.pb.client.ApplicationClientProtocolPBClientImpl.submitApplication(ApplicationClientProtocolPBClientImpl.java:272)
> {noformat}
> Should be queue doesnt exist
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
