[ https://issues.apache.org/jira/browse/YARN-2230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14269432#comment-14269432 ]
Hudson commented on YARN-2230: ------------------------------ FAILURE: Integrated in Hadoop-Mapreduce-trunk #2018 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2018/]) YARN-2230. Fixed few configs description in yarn-default.xml. Contributed by Vijay Bhat (jianhe: rev fe8d2bd74175e7ad521bc310c41a367c0946d6ec) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml * hadoop-yarn-project/CHANGES.txt > Fix description of yarn.scheduler.maximum-allocation-vcores in > yarn-default.xml (or code) > ----------------------------------------------------------------------------------------- > > Key: YARN-2230 > URL: https://issues.apache.org/jira/browse/YARN-2230 > Project: Hadoop YARN > Issue Type: Bug > Components: client, documentation, scheduler > Affects Versions: 2.4.0 > Reporter: Adam Kawa > Assignee: Vijay Bhat > Priority: Minor > Fix For: 2.7.0 > > Attachments: YARN-2230.001.patch, YARN-2230.002.patch > > > When a user requests more vcores than the allocation limit (e.g. > mapreduce.map.cpu.vcores is larger than > yarn.scheduler.maximum-allocation-vcores), then > InvalidResourceRequestException is thrown - > https://svn.apache.org/repos/asf/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerUtils.java > {code} > if (resReq.getCapability().getVirtualCores() < 0 || > resReq.getCapability().getVirtualCores() > > maximumResource.getVirtualCores()) { > throw new InvalidResourceRequestException("Invalid resource request" > + ", requested virtual cores < 0" > + ", or requested virtual cores > max configured" > + ", requestedVirtualCores=" > + resReq.getCapability().getVirtualCores() > + ", maxVirtualCores=" + maximumResource.getVirtualCores()); > } > {code} > According to documentation - yarn-default.xml > http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-common/yarn-default.xml, > the request should be capped to the allocation limit. > {code} > <property> > <description>The maximum allocation for every container request at the RM, > in terms of virtual CPU cores. Requests higher than this won't take > effect, > and will get capped to this value.</description> > <name>yarn.scheduler.maximum-allocation-vcores</name> > <value>32</value> > </property> > {code} > This means that: > * Either documentation or code should be corrected (unless this exception is > handled elsewhere accordingly, but it looks that it is not). > This behavior is confusing, because when such a job (with > mapreduce.map.cpu.vcores is larger than > yarn.scheduler.maximum-allocation-vcores) is submitted, it does not make any > progress. The warnings/exceptions are thrown at the scheduler (RM) side e.g. > {code} > 2014-06-29 00:34:51,469 WARN > org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService: > Invalid resource ask by application appattempt_1403993411503_0002_000001 > org.apache.hadoop.yarn.exceptions.InvalidResourceRequestException: Invalid > resource request, requested virtual cores < 0, or requested virtual cores > > max configured, requestedVirtualCores=32, maxVirtualCores=3 > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerUtils.validateResourceRequest(SchedulerUtils.java:237) > at > org.apache.hadoop.yarn.server.resourcemanager.RMServerUtils.validateResourceRequests(RMServerUtils.java:80) > at > org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService.allocate(ApplicationMasterService.java:420) > ..... > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:585) > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1026) > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1986) > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1982) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:416) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548) > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1980) > {code} > * IMHO, such an exception should be forwarded to client. Otherwise, it is non > obvious to discover why a job does not make any progress. > The same looks to be related to memory. -- This message was sent by Atlassian JIRA (v6.3.4#6332)