[jira] [Commented] (YARN-3489) RMServerUtils.validateResourceRequests should only obtain queue info once
[ https://issues.apache.org/jira/browse/YARN-3489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14564505#comment-14564505 ] Varun Saxena commented on YARN-3489: Thanks [~leftnoteasy] for the review and commit > RMServerUtils.validateResourceRequests should only obtain queue info once > - > > Key: YARN-3489 > URL: https://issues.apache.org/jira/browse/YARN-3489 > Project: Hadoop YARN > Issue Type: Improvement > Components: resourcemanager >Affects Versions: 2.6.0 >Reporter: Jason Lowe >Assignee: Varun Saxena > Labels: BB2015-05-RFC > Fix For: 2.7.1. > > Attachments: YARN-3489-branch-2.7.02.patch, > YARN-3489-branch-2.7.03.patch, YARN-3489-branch-2.7.04.patch, > YARN-3489-branch-2.7.patch, YARN-3489.01.patch, YARN-3489.02.patch, > YARN-3489.03.patch > > > Since the label support was added we now get the queue info for each request > being validated in SchedulerUtils.validateResourceRequest. If > validateResourceRequests needs to validate a lot of requests at a time (e.g.: > large cluster with lots of varied locality in the requests) then it will get > the queue info for each request. Since we build the queue info this > generates a lot of unnecessary garbage, as the queue isn't changing between > requests. We should grab the queue info once and pass it down rather than > building it again for each request. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3489) RMServerUtils.validateResourceRequests should only obtain queue info once
[ https://issues.apache.org/jira/browse/YARN-3489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14562290#comment-14562290 ] Hadoop QA commented on YARN-3489: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:red}-1{color} | patch | 0m 0s | The patch command could not apply the patch during dryrun. | \\ \\ || Subsystem || Report/Notes || | Patch URL | http://issues.apache.org/jira/secure/attachment/12735791/YARN-3489-branch-2.7.04.patch | | Optional Tests | javadoc javac unit findbugs checkstyle | | git revision | trunk / 5450413 | | Console output | https://builds.apache.org/job/PreCommit-YARN-Build/8111/console | This message was automatically generated. > RMServerUtils.validateResourceRequests should only obtain queue info once > - > > Key: YARN-3489 > URL: https://issues.apache.org/jira/browse/YARN-3489 > Project: Hadoop YARN > Issue Type: Improvement > Components: resourcemanager >Affects Versions: 2.6.0 >Reporter: Jason Lowe >Assignee: Varun Saxena > Labels: BB2015-05-RFC > Attachments: YARN-3489-branch-2.7.02.patch, > YARN-3489-branch-2.7.03.patch, YARN-3489-branch-2.7.04.patch, > YARN-3489-branch-2.7.patch, YARN-3489.01.patch, YARN-3489.02.patch, > YARN-3489.03.patch > > > Since the label support was added we now get the queue info for each request > being validated in SchedulerUtils.validateResourceRequest. If > validateResourceRequests needs to validate a lot of requests at a time (e.g.: > large cluster with lots of varied locality in the requests) then it will get > the queue info for each request. Since we build the queue info this > generates a lot of unnecessary garbage, as the queue isn't changing between > requests. We should grab the queue info once and pass it down rather than > building it again for each request. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3489) RMServerUtils.validateResourceRequests should only obtain queue info once
[ https://issues.apache.org/jira/browse/YARN-3489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14560518#comment-14560518 ] Varun Saxena commented on YARN-3489: [~leftnoteasy], sorry missed your comment...Will have a look. > RMServerUtils.validateResourceRequests should only obtain queue info once > - > > Key: YARN-3489 > URL: https://issues.apache.org/jira/browse/YARN-3489 > Project: Hadoop YARN > Issue Type: Improvement > Components: resourcemanager >Affects Versions: 2.6.0 >Reporter: Jason Lowe >Assignee: Varun Saxena > Labels: BB2015-05-RFC > Attachments: YARN-3489-branch-2.7.02.patch, > YARN-3489-branch-2.7.03.patch, YARN-3489-branch-2.7.patch, > YARN-3489.01.patch, YARN-3489.02.patch, YARN-3489.03.patch > > > Since the label support was added we now get the queue info for each request > being validated in SchedulerUtils.validateResourceRequest. If > validateResourceRequests needs to validate a lot of requests at a time (e.g.: > large cluster with lots of varied locality in the requests) then it will get > the queue info for each request. Since we build the queue info this > generates a lot of unnecessary garbage, as the queue isn't changing between > requests. We should grab the queue info once and pass it down rather than > building it again for each request. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3489) RMServerUtils.validateResourceRequests should only obtain queue info once
[ https://issues.apache.org/jira/browse/YARN-3489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14548460#comment-14548460 ] Wangda Tan commented on YARN-3489: -- [~varun_saxena], Tx for updating, I tried to run RM tests locally after applied the patch, a couple of test failures: Results : Failed tests: TestRMApplicationHistoryWriter.testRMWritingMassiveHistoryForCapacitySche:383->testRMWritingMassiveHistory:441 null Tests in error: TestAppManager.testRMAppSubmitDuplicateApplicationId:531 » NullPointer TestAppManager.testRMAppSubmitMaxAppAttempts:506 » NullPointer TestAppManager.testRMAppSubmit:463 » NullPointer TestClientRMService.testAppSubmit:859 » NullPointer TestClientRMService.testGetApplications:959 » NullPointer TestClientRMService.testConcurrentAppSubmit:1115 » test timed out after 4000 ... Could you look at them? > RMServerUtils.validateResourceRequests should only obtain queue info once > - > > Key: YARN-3489 > URL: https://issues.apache.org/jira/browse/YARN-3489 > Project: Hadoop YARN > Issue Type: Improvement > Components: resourcemanager >Affects Versions: 2.6.0 >Reporter: Jason Lowe >Assignee: Varun Saxena > Labels: BB2015-05-RFC > Attachments: YARN-3489-branch-2.7.02.patch, > YARN-3489-branch-2.7.03.patch, YARN-3489-branch-2.7.patch, > YARN-3489.01.patch, YARN-3489.02.patch, YARN-3489.03.patch > > > Since the label support was added we now get the queue info for each request > being validated in SchedulerUtils.validateResourceRequest. If > validateResourceRequests needs to validate a lot of requests at a time (e.g.: > large cluster with lots of varied locality in the requests) then it will get > the queue info for each request. Since we build the queue info this > generates a lot of unnecessary garbage, as the queue isn't changing between > requests. We should grab the queue info once and pass it down rather than > building it again for each request. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3489) RMServerUtils.validateResourceRequests should only obtain queue info once
[ https://issues.apache.org/jira/browse/YARN-3489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14546691#comment-14546691 ] Hadoop QA commented on YARN-3489: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:red}-1{color} | patch | 0m 0s | The patch command could not apply the patch during dryrun. | \\ \\ || Subsystem || Report/Notes || | Patch URL | http://issues.apache.org/jira/secure/attachment/12733319/YARN-3489-branch-2.7.03.patch | | Optional Tests | javadoc javac unit findbugs checkstyle | | git revision | trunk / b0ad644 | | Console output | https://builds.apache.org/job/PreCommit-YARN-Build/7960/console | This message was automatically generated. > RMServerUtils.validateResourceRequests should only obtain queue info once > - > > Key: YARN-3489 > URL: https://issues.apache.org/jira/browse/YARN-3489 > Project: Hadoop YARN > Issue Type: Improvement > Components: resourcemanager >Affects Versions: 2.6.0 >Reporter: Jason Lowe >Assignee: Varun Saxena > Labels: BB2015-05-RFC > Attachments: YARN-3489-branch-2.7.02.patch, > YARN-3489-branch-2.7.03.patch, YARN-3489-branch-2.7.patch, > YARN-3489.01.patch, YARN-3489.02.patch, YARN-3489.03.patch > > > Since the label support was added we now get the queue info for each request > being validated in SchedulerUtils.validateResourceRequest. If > validateResourceRequests needs to validate a lot of requests at a time (e.g.: > large cluster with lots of varied locality in the requests) then it will get > the queue info for each request. Since we build the queue info this > generates a lot of unnecessary garbage, as the queue isn't changing between > requests. We should grab the queue info once and pass it down rather than > building it again for each request. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3489) RMServerUtils.validateResourceRequests should only obtain queue info once
[ https://issues.apache.org/jira/browse/YARN-3489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14542586#comment-14542586 ] Wangda Tan commented on YARN-3489: -- [~varun_saxena], patch doesn't apply to latest branch-2.7, could you please check? > RMServerUtils.validateResourceRequests should only obtain queue info once > - > > Key: YARN-3489 > URL: https://issues.apache.org/jira/browse/YARN-3489 > Project: Hadoop YARN > Issue Type: Improvement > Components: resourcemanager >Affects Versions: 2.6.0 >Reporter: Jason Lowe >Assignee: Varun Saxena > Labels: BB2015-05-RFC > Attachments: YARN-3489-branch-2.7.02.patch, > YARN-3489-branch-2.7.patch, YARN-3489.01.patch, YARN-3489.02.patch, > YARN-3489.03.patch > > > Since the label support was added we now get the queue info for each request > being validated in SchedulerUtils.validateResourceRequest. If > validateResourceRequests needs to validate a lot of requests at a time (e.g.: > large cluster with lots of varied locality in the requests) then it will get > the queue info for each request. Since we build the queue info this > generates a lot of unnecessary garbage, as the queue isn't changing between > requests. We should grab the queue info once and pass it down rather than > building it again for each request. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3489) RMServerUtils.validateResourceRequests should only obtain queue info once
[ https://issues.apache.org/jira/browse/YARN-3489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14541892#comment-14541892 ] Hadoop QA commented on YARN-3489: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:red}-1{color} | patch | 0m 0s | The patch command could not apply the patch during dryrun. | \\ \\ || Subsystem || Report/Notes || | Patch URL | http://issues.apache.org/jira/secure/attachment/12732569/YARN-3489-branch-2.7.02.patch | | Optional Tests | javadoc javac unit findbugs checkstyle | | git revision | trunk / 065d8f2 | | Console output | https://builds.apache.org/job/PreCommit-YARN-Build/7919/console | This message was automatically generated. > RMServerUtils.validateResourceRequests should only obtain queue info once > - > > Key: YARN-3489 > URL: https://issues.apache.org/jira/browse/YARN-3489 > Project: Hadoop YARN > Issue Type: Improvement > Components: resourcemanager >Affects Versions: 2.6.0 >Reporter: Jason Lowe >Assignee: Varun Saxena > Labels: BB2015-05-RFC > Attachments: YARN-3489-branch-2.7.02.patch, > YARN-3489-branch-2.7.patch, YARN-3489.01.patch, YARN-3489.02.patch, > YARN-3489.03.patch > > > Since the label support was added we now get the queue info for each request > being validated in SchedulerUtils.validateResourceRequest. If > validateResourceRequests needs to validate a lot of requests at a time (e.g.: > large cluster with lots of varied locality in the requests) then it will get > the queue info for each request. Since we build the queue info this > generates a lot of unnecessary garbage, as the queue isn't changing between > requests. We should grab the queue info once and pass it down rather than > building it again for each request. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3489) RMServerUtils.validateResourceRequests should only obtain queue info once
[ https://issues.apache.org/jira/browse/YARN-3489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14541471#comment-14541471 ] Varun Saxena commented on YARN-3489: Ok...Will do so > RMServerUtils.validateResourceRequests should only obtain queue info once > - > > Key: YARN-3489 > URL: https://issues.apache.org/jira/browse/YARN-3489 > Project: Hadoop YARN > Issue Type: Improvement > Components: resourcemanager >Affects Versions: 2.6.0 >Reporter: Jason Lowe >Assignee: Varun Saxena > Labels: BB2015-05-RFC > Attachments: YARN-3489-branch-2.7.patch, YARN-3489.01.patch, > YARN-3489.02.patch, YARN-3489.03.patch > > > Since the label support was added we now get the queue info for each request > being validated in SchedulerUtils.validateResourceRequest. If > validateResourceRequests needs to validate a lot of requests at a time (e.g.: > large cluster with lots of varied locality in the requests) then it will get > the queue info for each request. Since we build the queue info this > generates a lot of unnecessary garbage, as the queue isn't changing between > requests. We should grab the queue info once and pass it down rather than > building it again for each request. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3489) RMServerUtils.validateResourceRequests should only obtain queue info once
[ https://issues.apache.org/jira/browse/YARN-3489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14540010#comment-14540010 ] Hudson commented on YARN-3489: -- SUCCESS: Integrated in Hadoop-Mapreduce-trunk-Java8 #193 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/193/]) YARN-3489. RMServerUtils.validateResourceRequests should only obtain queue info once. (Varun Saxena via wangda) (wangda: rev d6f6741296639a73f5306e3ebefec84a40ca03e5) * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerUtils.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMServerUtils.java > RMServerUtils.validateResourceRequests should only obtain queue info once > - > > Key: YARN-3489 > URL: https://issues.apache.org/jira/browse/YARN-3489 > Project: Hadoop YARN > Issue Type: Improvement > Components: resourcemanager >Affects Versions: 2.6.0 >Reporter: Jason Lowe >Assignee: Varun Saxena > Labels: BB2015-05-RFC > Attachments: YARN-3489-branch-2.7.patch, YARN-3489.01.patch, > YARN-3489.02.patch, YARN-3489.03.patch > > > Since the label support was added we now get the queue info for each request > being validated in SchedulerUtils.validateResourceRequest. If > validateResourceRequests needs to validate a lot of requests at a time (e.g.: > large cluster with lots of varied locality in the requests) then it will get > the queue info for each request. Since we build the queue info this > generates a lot of unnecessary garbage, as the queue isn't changing between > requests. We should grab the queue info once and pass it down rather than > building it again for each request. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3489) RMServerUtils.validateResourceRequests should only obtain queue info once
[ https://issues.apache.org/jira/browse/YARN-3489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14539957#comment-14539957 ] Hudson commented on YARN-3489: -- FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #183 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/183/]) YARN-3489. RMServerUtils.validateResourceRequests should only obtain queue info once. (Varun Saxena via wangda) (wangda: rev d6f6741296639a73f5306e3ebefec84a40ca03e5) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerUtils.java * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMServerUtils.java > RMServerUtils.validateResourceRequests should only obtain queue info once > - > > Key: YARN-3489 > URL: https://issues.apache.org/jira/browse/YARN-3489 > Project: Hadoop YARN > Issue Type: Improvement > Components: resourcemanager >Affects Versions: 2.6.0 >Reporter: Jason Lowe >Assignee: Varun Saxena > Labels: BB2015-05-RFC > Attachments: YARN-3489-branch-2.7.patch, YARN-3489.01.patch, > YARN-3489.02.patch, YARN-3489.03.patch > > > Since the label support was added we now get the queue info for each request > being validated in SchedulerUtils.validateResourceRequest. If > validateResourceRequests needs to validate a lot of requests at a time (e.g.: > large cluster with lots of varied locality in the requests) then it will get > the queue info for each request. Since we build the queue info this > generates a lot of unnecessary garbage, as the queue isn't changing between > requests. We should grab the queue info once and pass it down rather than > building it again for each request. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3489) RMServerUtils.validateResourceRequests should only obtain queue info once
[ https://issues.apache.org/jira/browse/YARN-3489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14539935#comment-14539935 ] Hudson commented on YARN-3489: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #2123 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/2123/]) YARN-3489. RMServerUtils.validateResourceRequests should only obtain queue info once. (Varun Saxena via wangda) (wangda: rev d6f6741296639a73f5306e3ebefec84a40ca03e5) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerUtils.java * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMServerUtils.java > RMServerUtils.validateResourceRequests should only obtain queue info once > - > > Key: YARN-3489 > URL: https://issues.apache.org/jira/browse/YARN-3489 > Project: Hadoop YARN > Issue Type: Improvement > Components: resourcemanager >Affects Versions: 2.6.0 >Reporter: Jason Lowe >Assignee: Varun Saxena > Labels: BB2015-05-RFC > Attachments: YARN-3489-branch-2.7.patch, YARN-3489.01.patch, > YARN-3489.02.patch, YARN-3489.03.patch > > > Since the label support was added we now get the queue info for each request > being validated in SchedulerUtils.validateResourceRequest. If > validateResourceRequests needs to validate a lot of requests at a time (e.g.: > large cluster with lots of varied locality in the requests) then it will get > the queue info for each request. Since we build the queue info this > generates a lot of unnecessary garbage, as the queue isn't changing between > requests. We should grab the queue info once and pass it down rather than > building it again for each request. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3489) RMServerUtils.validateResourceRequests should only obtain queue info once
[ https://issues.apache.org/jira/browse/YARN-3489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14539905#comment-14539905 ] Hudson commented on YARN-3489: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk #2141 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2141/]) YARN-3489. RMServerUtils.validateResourceRequests should only obtain queue info once. (Varun Saxena via wangda) (wangda: rev d6f6741296639a73f5306e3ebefec84a40ca03e5) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMServerUtils.java * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerUtils.java > RMServerUtils.validateResourceRequests should only obtain queue info once > - > > Key: YARN-3489 > URL: https://issues.apache.org/jira/browse/YARN-3489 > Project: Hadoop YARN > Issue Type: Improvement > Components: resourcemanager >Affects Versions: 2.6.0 >Reporter: Jason Lowe >Assignee: Varun Saxena > Labels: BB2015-05-RFC > Attachments: YARN-3489-branch-2.7.patch, YARN-3489.01.patch, > YARN-3489.02.patch, YARN-3489.03.patch > > > Since the label support was added we now get the queue info for each request > being validated in SchedulerUtils.validateResourceRequest. If > validateResourceRequests needs to validate a lot of requests at a time (e.g.: > large cluster with lots of varied locality in the requests) then it will get > the queue info for each request. Since we build the queue info this > generates a lot of unnecessary garbage, as the queue isn't changing between > requests. We should grab the queue info once and pass it down rather than > building it again for each request. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3489) RMServerUtils.validateResourceRequests should only obtain queue info once
[ https://issues.apache.org/jira/browse/YARN-3489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14539747#comment-14539747 ] Hudson commented on YARN-3489: -- FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #194 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/194/]) YARN-3489. RMServerUtils.validateResourceRequests should only obtain queue info once. (Varun Saxena via wangda) (wangda: rev d6f6741296639a73f5306e3ebefec84a40ca03e5) * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMServerUtils.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerUtils.java > RMServerUtils.validateResourceRequests should only obtain queue info once > - > > Key: YARN-3489 > URL: https://issues.apache.org/jira/browse/YARN-3489 > Project: Hadoop YARN > Issue Type: Improvement > Components: resourcemanager >Affects Versions: 2.6.0 >Reporter: Jason Lowe >Assignee: Varun Saxena > Labels: BB2015-05-RFC > Attachments: YARN-3489-branch-2.7.patch, YARN-3489.01.patch, > YARN-3489.02.patch, YARN-3489.03.patch > > > Since the label support was added we now get the queue info for each request > being validated in SchedulerUtils.validateResourceRequest. If > validateResourceRequests needs to validate a lot of requests at a time (e.g.: > large cluster with lots of varied locality in the requests) then it will get > the queue info for each request. Since we build the queue info this > generates a lot of unnecessary garbage, as the queue isn't changing between > requests. We should grab the queue info once and pass it down rather than > building it again for each request. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3489) RMServerUtils.validateResourceRequests should only obtain queue info once
[ https://issues.apache.org/jira/browse/YARN-3489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14539716#comment-14539716 ] Hudson commented on YARN-3489: -- SUCCESS: Integrated in Hadoop-Yarn-trunk #925 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/925/]) YARN-3489. RMServerUtils.validateResourceRequests should only obtain queue info once. (Varun Saxena via wangda) (wangda: rev d6f6741296639a73f5306e3ebefec84a40ca03e5) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMServerUtils.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerUtils.java * hadoop-yarn-project/CHANGES.txt > RMServerUtils.validateResourceRequests should only obtain queue info once > - > > Key: YARN-3489 > URL: https://issues.apache.org/jira/browse/YARN-3489 > Project: Hadoop YARN > Issue Type: Improvement > Components: resourcemanager >Affects Versions: 2.6.0 >Reporter: Jason Lowe >Assignee: Varun Saxena > Labels: BB2015-05-RFC > Attachments: YARN-3489-branch-2.7.patch, YARN-3489.01.patch, > YARN-3489.02.patch, YARN-3489.03.patch > > > Since the label support was added we now get the queue info for each request > being validated in SchedulerUtils.validateResourceRequest. If > validateResourceRequests needs to validate a lot of requests at a time (e.g.: > large cluster with lots of varied locality in the requests) then it will get > the queue info for each request. Since we build the queue info this > generates a lot of unnecessary garbage, as the queue isn't changing between > requests. We should grab the queue info once and pass it down rather than > building it again for each request. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3489) RMServerUtils.validateResourceRequests should only obtain queue info once
[ https://issues.apache.org/jira/browse/YARN-3489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14539332#comment-14539332 ] Hadoop QA commented on YARN-3489: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:red}-1{color} | patch | 0m 0s | The patch command could not apply the patch during dryrun. | \\ \\ || Subsystem || Report/Notes || | Patch URL | http://issues.apache.org/jira/secure/attachment/12732137/YARN-3489-branch-2.7.patch | | Optional Tests | javadoc javac unit findbugs checkstyle | | git revision | branch-2 / 12584ac | | Console output | https://builds.apache.org/job/PreCommit-YARN-Build/7877/console | This message was automatically generated. > RMServerUtils.validateResourceRequests should only obtain queue info once > - > > Key: YARN-3489 > URL: https://issues.apache.org/jira/browse/YARN-3489 > Project: Hadoop YARN > Issue Type: Improvement > Components: resourcemanager >Affects Versions: 2.6.0 >Reporter: Jason Lowe >Assignee: Varun Saxena > Labels: BB2015-05-RFC > Attachments: YARN-3489-branch-2.7.patch, YARN-3489.01.patch, > YARN-3489.02.patch, YARN-3489.03.patch > > > Since the label support was added we now get the queue info for each request > being validated in SchedulerUtils.validateResourceRequest. If > validateResourceRequests needs to validate a lot of requests at a time (e.g.: > large cluster with lots of varied locality in the requests) then it will get > the queue info for each request. Since we build the queue info this > generates a lot of unnecessary garbage, as the queue isn't changing between > requests. We should grab the queue info once and pass it down rather than > building it again for each request. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3489) RMServerUtils.validateResourceRequests should only obtain queue info once
[ https://issues.apache.org/jira/browse/YARN-3489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14539104#comment-14539104 ] Wangda Tan commented on YARN-3489: -- Committed to trunk/branch-2, branch-2.7 needs rebase, running tests locally. Will commit once all RM related tests can get passed. > RMServerUtils.validateResourceRequests should only obtain queue info once > - > > Key: YARN-3489 > URL: https://issues.apache.org/jira/browse/YARN-3489 > Project: Hadoop YARN > Issue Type: Improvement > Components: resourcemanager >Affects Versions: 2.6.0 >Reporter: Jason Lowe >Assignee: Varun Saxena > Labels: BB2015-05-RFC > Attachments: YARN-3489.01.patch, YARN-3489.02.patch, > YARN-3489.03.patch > > > Since the label support was added we now get the queue info for each request > being validated in SchedulerUtils.validateResourceRequest. If > validateResourceRequests needs to validate a lot of requests at a time (e.g.: > large cluster with lots of varied locality in the requests) then it will get > the queue info for each request. Since we build the queue info this > generates a lot of unnecessary garbage, as the queue isn't changing between > requests. We should grab the queue info once and pass it down rather than > building it again for each request. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3489) RMServerUtils.validateResourceRequests should only obtain queue info once
[ https://issues.apache.org/jira/browse/YARN-3489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14539007#comment-14539007 ] Hudson commented on YARN-3489: -- FAILURE: Integrated in Hadoop-trunk-Commit #7800 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/7800/]) YARN-3489. RMServerUtils.validateResourceRequests should only obtain queue info once. (Varun Saxena via wangda) (wangda: rev d6f6741296639a73f5306e3ebefec84a40ca03e5) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMServerUtils.java * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerUtils.java > RMServerUtils.validateResourceRequests should only obtain queue info once > - > > Key: YARN-3489 > URL: https://issues.apache.org/jira/browse/YARN-3489 > Project: Hadoop YARN > Issue Type: Improvement > Components: resourcemanager >Affects Versions: 2.6.0 >Reporter: Jason Lowe >Assignee: Varun Saxena > Labels: BB2015-05-RFC > Attachments: YARN-3489.01.patch, YARN-3489.02.patch, > YARN-3489.03.patch > > > Since the label support was added we now get the queue info for each request > being validated in SchedulerUtils.validateResourceRequest. If > validateResourceRequests needs to validate a lot of requests at a time (e.g.: > large cluster with lots of varied locality in the requests) then it will get > the queue info for each request. Since we build the queue info this > generates a lot of unnecessary garbage, as the queue isn't changing between > requests. We should grab the queue info once and pass it down rather than > building it again for each request. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3489) RMServerUtils.validateResourceRequests should only obtain queue info once
[ https://issues.apache.org/jira/browse/YARN-3489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14538961#comment-14538961 ] Wangda Tan commented on YARN-3489: -- Committing. > RMServerUtils.validateResourceRequests should only obtain queue info once > - > > Key: YARN-3489 > URL: https://issues.apache.org/jira/browse/YARN-3489 > Project: Hadoop YARN > Issue Type: Improvement > Components: resourcemanager >Affects Versions: 2.6.0 >Reporter: Jason Lowe >Assignee: Varun Saxena > Labels: BB2015-05-RFC > Attachments: YARN-3489.01.patch, YARN-3489.02.patch, > YARN-3489.03.patch > > > Since the label support was added we now get the queue info for each request > being validated in SchedulerUtils.validateResourceRequest. If > validateResourceRequests needs to validate a lot of requests at a time (e.g.: > large cluster with lots of varied locality in the requests) then it will get > the queue info for each request. Since we build the queue info this > generates a lot of unnecessary garbage, as the queue isn't changing between > requests. We should grab the queue info once and pass it down rather than > building it again for each request. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3489) RMServerUtils.validateResourceRequests should only obtain queue info once
[ https://issues.apache.org/jira/browse/YARN-3489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14536332#comment-14536332 ] Varun Saxena commented on YARN-3489: [~leftnoteasy], test case passes in local. Failure seems unrelated. > RMServerUtils.validateResourceRequests should only obtain queue info once > - > > Key: YARN-3489 > URL: https://issues.apache.org/jira/browse/YARN-3489 > Project: Hadoop YARN > Issue Type: Improvement > Components: resourcemanager >Affects Versions: 2.6.0 >Reporter: Jason Lowe >Assignee: Varun Saxena > Labels: BB2015-05-RFC > Attachments: YARN-3489.01.patch, YARN-3489.02.patch, > YARN-3489.03.patch > > > Since the label support was added we now get the queue info for each request > being validated in SchedulerUtils.validateResourceRequest. If > validateResourceRequests needs to validate a lot of requests at a time (e.g.: > large cluster with lots of varied locality in the requests) then it will get > the queue info for each request. Since we build the queue info this > generates a lot of unnecessary garbage, as the queue isn't changing between > requests. We should grab the queue info once and pass it down rather than > building it again for each request. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3489) RMServerUtils.validateResourceRequests should only obtain queue info once
[ https://issues.apache.org/jira/browse/YARN-3489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14536095#comment-14536095 ] Hadoop QA commented on YARN-3489: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | pre-patch | 14m 41s | Pre-patch trunk compilation is healthy. | | {color:green}+1{color} | @author | 0m 0s | The patch does not contain any @author tags. | | {color:red}-1{color} | tests included | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | | {color:green}+1{color} | javac | 7m 32s | There were no new javac warning messages. | | {color:green}+1{color} | javadoc | 9m 41s | There were no new javadoc warning messages. | | {color:green}+1{color} | release audit | 0m 23s | The applied patch does not increase the total number of release audit warnings. | | {color:red}-1{color} | checkstyle | 0m 52s | The applied patch generated 1 new checkstyle issues (total was 76, now 77). | | {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that end in whitespace. | | {color:green}+1{color} | install | 1m 38s | mvn install still works. | | {color:green}+1{color} | eclipse:eclipse | 0m 32s | The patch built with eclipse:eclipse. | | {color:green}+1{color} | findbugs | 1m 16s | The patch does not introduce any new Findbugs (version 2.0.3) warnings. | | {color:red}-1{color} | yarn tests | 52m 6s | Tests failed in hadoop-yarn-server-resourcemanager. | | | | 88m 45s | | \\ \\ || Reason || Tests || | Failed unit tests | hadoop.yarn.server.resourcemanager.TestRMRestart | \\ \\ || Subsystem || Report/Notes || | Patch URL | http://issues.apache.org/jira/secure/attachment/12731613/YARN-3489.03.patch | | Optional Tests | javadoc javac unit findbugs checkstyle | | git revision | trunk / ed0f4db | | checkstyle | https://builds.apache.org/job/PreCommit-YARN-Build/7833/artifact/patchprocess/diffcheckstylehadoop-yarn-server-resourcemanager.txt | | hadoop-yarn-server-resourcemanager test log | https://builds.apache.org/job/PreCommit-YARN-Build/7833/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt | | Test Results | https://builds.apache.org/job/PreCommit-YARN-Build/7833/testReport/ | | Java | 1.7.0_55 | | uname | Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux | | Console output | https://builds.apache.org/job/PreCommit-YARN-Build/7833/console | This message was automatically generated. > RMServerUtils.validateResourceRequests should only obtain queue info once > - > > Key: YARN-3489 > URL: https://issues.apache.org/jira/browse/YARN-3489 > Project: Hadoop YARN > Issue Type: Improvement > Components: resourcemanager >Affects Versions: 2.6.0 >Reporter: Jason Lowe >Assignee: Varun Saxena > Labels: BB2015-05-RFC > Attachments: YARN-3489.01.patch, YARN-3489.02.patch, > YARN-3489.03.patch > > > Since the label support was added we now get the queue info for each request > being validated in SchedulerUtils.validateResourceRequest. If > validateResourceRequests needs to validate a lot of requests at a time (e.g.: > large cluster with lots of varied locality in the requests) then it will get > the queue info for each request. Since we build the queue info this > generates a lot of unnecessary garbage, as the queue isn't changing between > requests. We should grab the queue info once and pass it down rather than > building it again for each request. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3489) RMServerUtils.validateResourceRequests should only obtain queue info once
[ https://issues.apache.org/jira/browse/YARN-3489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14535645#comment-14535645 ] Wangda Tan commented on YARN-3489: -- Patch LGTM, will commit once Jenkins get back. > RMServerUtils.validateResourceRequests should only obtain queue info once > - > > Key: YARN-3489 > URL: https://issues.apache.org/jira/browse/YARN-3489 > Project: Hadoop YARN > Issue Type: Improvement > Components: resourcemanager >Affects Versions: 2.6.0 >Reporter: Jason Lowe >Assignee: Varun Saxena > Labels: BB2015-05-RFC > Attachments: YARN-3489.01.patch, YARN-3489.02.patch, > YARN-3489.03.patch > > > Since the label support was added we now get the queue info for each request > being validated in SchedulerUtils.validateResourceRequest. If > validateResourceRequests needs to validate a lot of requests at a time (e.g.: > large cluster with lots of varied locality in the requests) then it will get > the queue info for each request. Since we build the queue info this > generates a lot of unnecessary garbage, as the queue isn't changing between > requests. We should grab the queue info once and pass it down rather than > building it again for each request. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3489) RMServerUtils.validateResourceRequests should only obtain queue info once
[ https://issues.apache.org/jira/browse/YARN-3489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14535083#comment-14535083 ] Wangda Tan commented on YARN-3489: -- [~varun_saxena], {{SchedulerUtils}} itself is marked as @private so you can modify it without any issue. I think the new method you added to SchedulerUtils is duplicated logic, could you add QueueInfo instead of creating new method? > RMServerUtils.validateResourceRequests should only obtain queue info once > - > > Key: YARN-3489 > URL: https://issues.apache.org/jira/browse/YARN-3489 > Project: Hadoop YARN > Issue Type: Improvement > Components: resourcemanager >Affects Versions: 2.6.0 >Reporter: Jason Lowe >Assignee: Varun Saxena > Attachments: YARN-3489.01.patch, YARN-3489.02.patch > > > Since the label support was added we now get the queue info for each request > being validated in SchedulerUtils.validateResourceRequest. If > validateResourceRequests needs to validate a lot of requests at a time (e.g.: > large cluster with lots of varied locality in the requests) then it will get > the queue info for each request. Since we build the queue info this > generates a lot of unnecessary garbage, as the queue isn't changing between > requests. We should grab the queue info once and pass it down rather than > building it again for each request. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3489) RMServerUtils.validateResourceRequests should only obtain queue info once
[ https://issues.apache.org/jira/browse/YARN-3489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14535095#comment-14535095 ] Varun Saxena commented on YARN-3489: [~leftnoteasy], oh didn't notice that SchedulerUtils is marked as private. We can change the current API then because my major concern was changing a public API. I will update new patch as per your suggestion then. > RMServerUtils.validateResourceRequests should only obtain queue info once > - > > Key: YARN-3489 > URL: https://issues.apache.org/jira/browse/YARN-3489 > Project: Hadoop YARN > Issue Type: Improvement > Components: resourcemanager >Affects Versions: 2.6.0 >Reporter: Jason Lowe >Assignee: Varun Saxena > Attachments: YARN-3489.01.patch, YARN-3489.02.patch > > > Since the label support was added we now get the queue info for each request > being validated in SchedulerUtils.validateResourceRequest. If > validateResourceRequests needs to validate a lot of requests at a time (e.g.: > large cluster with lots of varied locality in the requests) then it will get > the queue info for each request. Since we build the queue info this > generates a lot of unnecessary garbage, as the queue isn't changing between > requests. We should grab the queue info once and pass it down rather than > building it again for each request. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3489) RMServerUtils.validateResourceRequests should only obtain queue info once
[ https://issues.apache.org/jira/browse/YARN-3489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14535050#comment-14535050 ] Varun Saxena commented on YARN-3489: Its quite similar to what I have done. Added a new queueInfo parameter in a new API. I did not change the old API {{SchedulerUtils#normalizeAndvalidateRequest}} (which is without queueInfo) because it is marked as public. > RMServerUtils.validateResourceRequests should only obtain queue info once > - > > Key: YARN-3489 > URL: https://issues.apache.org/jira/browse/YARN-3489 > Project: Hadoop YARN > Issue Type: Improvement > Components: resourcemanager >Affects Versions: 2.6.0 >Reporter: Jason Lowe >Assignee: Varun Saxena > Attachments: YARN-3489.01.patch, YARN-3489.02.patch > > > Since the label support was added we now get the queue info for each request > being validated in SchedulerUtils.validateResourceRequest. If > validateResourceRequests needs to validate a lot of requests at a time (e.g.: > large cluster with lots of varied locality in the requests) then it will get > the queue info for each request. Since we build the queue info this > generates a lot of unnecessary garbage, as the queue isn't changing between > requests. We should grab the queue info once and pass it down rather than > building it again for each request. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3489) RMServerUtils.validateResourceRequests should only obtain queue info once
[ https://issues.apache.org/jira/browse/YARN-3489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14535038#comment-14535038 ] Wangda Tan commented on YARN-3489: -- [~varun_saxena], I think we should pass queueinfo only once for a list of resource request. When {{RMServerUtils.normalizeAndValidateRequests}} is called, all ResourceRequests from same application, so they will from same queue as well. Then you can pass it down to {{normalizeAndvalidateRequest}}, In {code} public static void normalizeAndValidateRequest(ResourceRequest resReq, Resource maximumResource, String queueName, YarnScheduler scheduler, boolean isRecovery) {code} You can add a queue info as parameter, only get queue info from scheduler when it's not null. Thoughts? > RMServerUtils.validateResourceRequests should only obtain queue info once > - > > Key: YARN-3489 > URL: https://issues.apache.org/jira/browse/YARN-3489 > Project: Hadoop YARN > Issue Type: Improvement > Components: resourcemanager >Affects Versions: 2.6.0 >Reporter: Jason Lowe >Assignee: Varun Saxena > Attachments: YARN-3489.01.patch, YARN-3489.02.patch > > > Since the label support was added we now get the queue info for each request > being validated in SchedulerUtils.validateResourceRequest. If > validateResourceRequests needs to validate a lot of requests at a time (e.g.: > large cluster with lots of varied locality in the requests) then it will get > the queue info for each request. Since we build the queue info this > generates a lot of unnecessary garbage, as the queue isn't changing between > requests. We should grab the queue info once and pass it down rather than > building it again for each request. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3489) RMServerUtils.validateResourceRequests should only obtain queue info once
[ https://issues.apache.org/jira/browse/YARN-3489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14535014#comment-14535014 ] Karthik Kambatla commented on YARN-3489: The patch doesn't apply anymore. > RMServerUtils.validateResourceRequests should only obtain queue info once > - > > Key: YARN-3489 > URL: https://issues.apache.org/jira/browse/YARN-3489 > Project: Hadoop YARN > Issue Type: Improvement > Components: resourcemanager >Affects Versions: 2.6.0 >Reporter: Jason Lowe >Assignee: Varun Saxena > Attachments: YARN-3489.01.patch, YARN-3489.02.patch > > > Since the label support was added we now get the queue info for each request > being validated in SchedulerUtils.validateResourceRequest. If > validateResourceRequests needs to validate a lot of requests at a time (e.g.: > large cluster with lots of varied locality in the requests) then it will get > the queue info for each request. Since we build the queue info this > generates a lot of unnecessary garbage, as the queue isn't changing between > requests. We should grab the queue info once and pass it down rather than > building it again for each request. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3489) RMServerUtils.validateResourceRequests should only obtain queue info once
[ https://issues.apache.org/jira/browse/YARN-3489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14534238#comment-14534238 ] Varun Saxena commented on YARN-3489: [~vinodkv], as the current API is called in places where there is only one resource request, hence queue info need not be passed. Some sort of API change will be required to pass the queue info only once for a list of resource requests. Thoughts ? > RMServerUtils.validateResourceRequests should only obtain queue info once > - > > Key: YARN-3489 > URL: https://issues.apache.org/jira/browse/YARN-3489 > Project: Hadoop YARN > Issue Type: Improvement > Components: resourcemanager >Affects Versions: 2.6.0 >Reporter: Jason Lowe >Assignee: Varun Saxena > Attachments: YARN-3489.01.patch, YARN-3489.02.patch > > > Since the label support was added we now get the queue info for each request > being validated in SchedulerUtils.validateResourceRequest. If > validateResourceRequests needs to validate a lot of requests at a time (e.g.: > large cluster with lots of varied locality in the requests) then it will get > the queue info for each request. Since we build the queue info this > generates a lot of unnecessary garbage, as the queue isn't changing between > requests. We should grab the queue info once and pass it down rather than > building it again for each request. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3489) RMServerUtils.validateResourceRequests should only obtain queue info once
[ https://issues.apache.org/jira/browse/YARN-3489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14504437#comment-14504437 ] Hadoop QA commented on YARN-3489: - {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12726790/YARN-3489.02.patch against trunk revision d52de61. {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:red}-1 tests included{color}. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 javadoc{color}. There were no new javadoc warning messages. {color:green}+1 eclipse:eclipse{color}. The patch built with eclipse:eclipse. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 2.0.3) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:red}-1 core tests{color}. The patch failed these unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: org.apache.hadoop.yarn.server.resourcemanager.security.TestRMDelegationTokens Test results: https://builds.apache.org/job/PreCommit-YARN-Build/7417//testReport/ Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7417//console This message is automatically generated. > RMServerUtils.validateResourceRequests should only obtain queue info once > - > > Key: YARN-3489 > URL: https://issues.apache.org/jira/browse/YARN-3489 > Project: Hadoop YARN > Issue Type: Improvement > Components: resourcemanager >Affects Versions: 2.6.0 >Reporter: Jason Lowe >Assignee: Varun Saxena > Attachments: YARN-3489.01.patch, YARN-3489.02.patch > > > Since the label support was added we now get the queue info for each request > being validated in SchedulerUtils.validateResourceRequest. If > validateResourceRequests needs to validate a lot of requests at a time (e.g.: > large cluster with lots of varied locality in the requests) then it will get > the queue info for each request. Since we build the queue info this > generates a lot of unnecessary garbage, as the queue isn't changing between > requests. We should grab the queue info once and pass it down rather than > building it again for each request. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3489) RMServerUtils.validateResourceRequests should only obtain queue info once
[ https://issues.apache.org/jira/browse/YARN-3489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14502350#comment-14502350 ] Hadoop QA commented on YARN-3489: - {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12726511/YARN-3489.01.patch against trunk revision 8511d80. {color:red}-1 patch{color}. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7394//console This message is automatically generated. > RMServerUtils.validateResourceRequests should only obtain queue info once > - > > Key: YARN-3489 > URL: https://issues.apache.org/jira/browse/YARN-3489 > Project: Hadoop YARN > Issue Type: Improvement > Components: resourcemanager >Affects Versions: 2.6.0 >Reporter: Jason Lowe >Assignee: Varun Saxena > Attachments: YARN-3489.01.patch > > > Since the label support was added we now get the queue info for each request > being validated in SchedulerUtils.validateResourceRequest. If > validateResourceRequests needs to validate a lot of requests at a time (e.g.: > large cluster with lots of varied locality in the requests) then it will get > the queue info for each request. Since we build the queue info this > generates a lot of unnecessary garbage, as the queue isn't changing between > requests. We should grab the queue info once and pass it down rather than > building it again for each request. -- This message was sent by Atlassian JIRA (v6.3.4#6332)