[jira] [Commented] (YARN-10136) In Secure Federation, Router About UI page failed to display the subclusters information appropriately.
[ https://issues.apache.org/jira/browse/YARN-10136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17036731#comment-17036731 ] Hadoop QA commented on YARN-10136: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 40s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s{color} | {color:red} The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 19m 46s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 22s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 17s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 23s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 14m 40s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 32s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 21s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 21s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 17s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 17s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 12s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 19s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 14m 6s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 36s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 20s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 33s{color} | {color:green} hadoop-yarn-server-router in the patch passed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 26s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 55m 42s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Docker | Client=19.03.6 Server=19.03.6 Image:yetus/hadoop:c44943d1fc3 | | JIRA Issue | YARN-10136 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12993454/YARN-10136.001.patch | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux 616c16248d84 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/patchprocess/precommit/personality/provided.sh | | git revision | trunk / 56dee66 | | maven | version: Apache Maven 3.3.9 | | Default Java | 1.8.0_242 | | findbugs | v3.1.0-RC1 | | Test Results | https://builds.apache.org/job/PreCommit-YARN-Build/25524/testReport/ | | Max. process+thread count | 799 (vs. ulimit of 5500) | | modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router | | Console output | https://builds.apache.org/job/PreCommit-YARN-Build/25524/console | | Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
[jira] [Updated] (YARN-10136) In Secure Federation, Router About UI page failed to display the subclusters information appropriately.
[ https://issues.apache.org/jira/browse/YARN-10136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilwa S T updated YARN-10136: - Attachment: YARN-10136.001.patch > In Secure Federation, Router About UI page failed to display the subclusters > information appropriately. > --- > > Key: YARN-10136 > URL: https://issues.apache.org/jira/browse/YARN-10136 > Project: Hadoop YARN > Issue Type: Bug > Components: federation, yarn >Reporter: Sushanta Sen >Assignee: Bilwa S T >Priority: Major > Attachments: YARN-10136.001.patch > > > In Secure Federation, Router About UI page failed to display the subclusters > information appropriately. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9879) Allow multiple leaf queues with the same name in CS
[ https://issues.apache.org/jira/browse/YARN-9879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17036703#comment-17036703 ] Hadoop QA commented on YARN-9879: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 42s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 7 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 20m 12s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 43s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 51s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 46s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 15m 42s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 35s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 29s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 43s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 38s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 38s{color} | {color:green} the patch passed {color} | | {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange} 0m 46s{color} | {color:orange} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 72 new + 1168 unchanged - 5 fixed = 1240 total (was 1173) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 40s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 14m 13s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 37s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 28s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:red}-1{color} | {color:red} unit {color} | {color:red} 87m 9s{color} | {color:red} hadoop-yarn-server-resourcemanager in the patch passed. {color} | | {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m 28s{color} | {color:red} The patch generated 2 ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}147m 24s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.yarn.server.resourcemanager.scheduler.capacity.TestLeafQueue | | | hadoop.yarn.server.resourcemanager.reservation.TestCapacitySchedulerPlanFollower | | | hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerNodeLabelUpdate | | | hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesSchedulerActivities | | | hadoop.yarn.server.resourcemanager.monitor.capacity.TestProportionalCapacityPreemptionPolicyIntraQueueUserLimit | | | hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerAutoQueueCreation | | | hadoop.yarn.server.resourcemanager.scheduler.capacity.TestQueueManagementDynamicEditPolicy | | | hadoop.yarn.server.resourcemanager.scheduler.capacity.TestQueueStateManager | | | hadoop.yarn.server.resourcemanager.placement.TestUserGroupMappingPlacementRule | | | hadoop.yarn.server.resourcemanager.monitor.capacity.TestProportionalCapacityPreemptionPolicyIntraQueueFairOrdering | | | hadoop.yarn.server.resourcemanager.monitor.capacity.TestProportionalCapacityPreemptionPolicy | | |
[jira] [Commented] (YARN-9879) Allow multiple leaf queues with the same name in CS
[ https://issues.apache.org/jira/browse/YARN-9879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17036663#comment-17036663 ] Gergely Pollak commented on YARN-9879: -- In this latest patch I've checked all CSQueue.getQueueName calls, and replaced them with getQueuePath, also checked how those values were used, and changed the code if it was necessary. The biggest change was around the placement rules, but I think I've managed to solve the issues with minimal impact and code changes. The problem with placement rules is they can dynamically create colliding queue names, and at configuration time it is impossible to determine whether the rule will collide with an already existing queue or not. So in this implementation, I changed the config parsing methods to be aware of the the possible name collisions and also added an extra ambiguity check just before creating the placement context, this way the user will be able to see in the logs why the application fails. The expected behaviour for name collisions is to refuse the application submission if the matching placement rule results in an ambiguous queue name. This is consistent with the app submission behaviour when providing a queue name, but it is a leaf name only. Next step is to check the application submission side, and make sure the submitted queue names are parsed properly and are compatible with the internal full queue name usage. Also marked a few parts of the code for cleanup later, we are using 3 classes with very similar behaviour and it was really confusing to understand the difference between them (practically there is none), but since this patch is already growing in size, and I haven't even touched the tests, I'll file separate Jiras about those later. > Allow multiple leaf queues with the same name in CS > --- > > Key: YARN-9879 > URL: https://issues.apache.org/jira/browse/YARN-9879 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Gergely Pollak >Assignee: Gergely Pollak >Priority: Major > Labels: fs2cs > Attachments: CSQueue.getQueueUsage.txt, DesignDoc_v1.pdf, > YARN-9879.POC001.patch, YARN-9879.POC002.patch, YARN-9879.POC003.patch > > > Currently the leaf queue's name must be unique regardless of its position in > the queue hierarchy. > Design doc and first proposal is being made, I'll attach it as soon as it's > done. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9879) Allow multiple leaf queues with the same name in CS
[ https://issues.apache.org/jira/browse/YARN-9879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gergely Pollak updated YARN-9879: - Attachment: YARN-9879.POC003.patch > Allow multiple leaf queues with the same name in CS > --- > > Key: YARN-9879 > URL: https://issues.apache.org/jira/browse/YARN-9879 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Gergely Pollak >Assignee: Gergely Pollak >Priority: Major > Labels: fs2cs > Attachments: CSQueue.getQueueUsage.txt, DesignDoc_v1.pdf, > YARN-9879.POC001.patch, YARN-9879.POC002.patch, YARN-9879.POC003.patch > > > Currently the leaf queue's name must be unique regardless of its position in > the queue hierarchy. > Design doc and first proposal is being made, I'll attach it as soon as it's > done. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10119) Cannot reset the AM failure count for YARN Service
[ https://issues.apache.org/jira/browse/YARN-10119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17036621#comment-17036621 ] kyungwan nam commented on YARN-10119: - [~prabhujoseph], Can you please take a look at this? Thanks! > Cannot reset the AM failure count for YARN Service > -- > > Key: YARN-10119 > URL: https://issues.apache.org/jira/browse/YARN-10119 > Project: Hadoop YARN > Issue Type: Bug >Affects Versions: 3.1.2 >Reporter: kyungwan nam >Assignee: kyungwan nam >Priority: Major > Attachments: YARN-10119.001.patch > > > Currently, YARN Service does not support to reset AM failure count, which > introduced in YARN-611 > Since the AM failure count is never reset, eventually that will reach > yarn.service.am-restart.max-attempts and the app will be stopped. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-8959) TestContainerResizing fails randomly
[ https://issues.apache.org/jira/browse/YARN-8959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ahmed Hussein reassigned YARN-8959: --- Assignee: Ahmed Hussein (was: Bilwa S T) > TestContainerResizing fails randomly > > > Key: YARN-8959 > URL: https://issues.apache.org/jira/browse/YARN-8959 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Bibin Chundatt >Assignee: Ahmed Hussein >Priority: Minor > > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerResizing.testSimpleDecreaseContainer > {code} > testSimpleDecreaseContainer(org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerResizing) > Time elapsed: 0.348 s <<< FAILURE! > java.lang.AssertionError: expected:<1024> but was:<3072> > at org.junit.Assert.fail(Assert.java:88) > at org.junit.Assert.failNotEquals(Assert.java:834) > at org.junit.Assert.assertEquals(Assert.java:645) > at org.junit.Assert.assertEquals(Assert.java:631) > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerResizing.checkUsedResource(TestContainerResizing.java:1011) > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerResizing.testSimpleDecreaseContainer(TestContainerResizing.java:210) > {code} > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerResizing.testIncreaseContainerUnreservedWhenContainerCompleted > {code} > testIncreaseContainerUnreservedWhenContainerCompleted(org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerResizing) > Time elapsed: 0.445 s <<< FAILURE! > java.lang.AssertionError: expected:<1024> but was:<7168> > at org.junit.Assert.fail(Assert.java:88) > at org.junit.Assert.failNotEquals(Assert.java:834) > at org.junit.Assert.assertEquals(Assert.java:645) > at org.junit.Assert.assertEquals(Assert.java:631) > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerResizing.checkUsedResource(TestContainerResizing.java:1011) > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerResizing.testIncreaseContainerUnreservedWhenContainerCompleted(TestContainerResizing.java:729) > {code} > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerResizing.testExcessiveReservationWhenDecreaseSameContainer > {code} > testExcessiveReservationWhenDecreaseSameContainer(org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerResizing) > Time elapsed: 0.321 s <<< FAILURE! > java.lang.AssertionError: expected:<1024> but was:<2048> > at org.junit.Assert.fail(Assert.java:88) > at org.junit.Assert.failNotEquals(Assert.java:834) > at org.junit.Assert.assertEquals(Assert.java:645) > at org.junit.Assert.assertEquals(Assert.java:631) > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerResizing.checkUsedResource(TestContainerResizing.java:1015) > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerResizing.testExcessiveReservationWhenDecreaseSameContainer(TestContainerResizing.java:623) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10043) FairOrderingPolicy Improvements
[ https://issues.apache.org/jira/browse/YARN-10043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17036453#comment-17036453 ] Hadoop QA commented on YARN-10043: -- | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 42s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 2 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 19m 27s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 42s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 33s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 47s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 15m 20s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 33s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 29s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 43s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 37s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 37s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 28s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 40s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 14m 3s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 37s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 27s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 89m 1s{color} | {color:green} hadoop-yarn-server-resourcemanager in the patch passed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 24s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}147m 28s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Docker | Client=19.03.6 Server=19.03.6 Image:yetus/hadoop:c44943d1fc3 | | JIRA Issue | YARN-10043 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12993396/YARN-10043.004.patch | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux 4864fc79d35f 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/patchprocess/precommit/personality/provided.sh | | git revision | trunk / da99ac7 | | maven | version: Apache Maven 3.3.9 | | Default Java | 1.8.0_242 | | findbugs | v3.1.0-RC1 | | Test Results | https://builds.apache.org/job/PreCommit-YARN-Build/25522/testReport/ | | Max. process+thread count | 812 (vs. ulimit of 5500) | | modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager | | Console output | https://builds.apache.org/job/PreCommit-YARN-Build/25522/console | | Powered by | Apache Yetus 0.8.0 http://yetus.apache.org | This message was automatically generated. > FairOrderingPolicy Improvements >
[jira] [Commented] (YARN-8529) Add timeout to RouterWebServiceUtil#invokeRMWebService
[ https://issues.apache.org/jira/browse/YARN-8529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17036426#comment-17036426 ] Íñigo Goiri commented on YARN-8529: --- TestYarnConfigurationFields.testCompareXmlAgainstConfigurationClass is failing with: {code} java.lang.AssertionError: yarn-default.xml has 2 properties missing in class org.apache.hadoop.yarn.conf.YarnConfiguration Entries: yarn.router.webapp.connect-timeout yarn.router.webapp.read-timeout expected:<0> but was:<2> {code} We are adding as ignore to TestYarnConfigurationFields while defining it. Either we define it in yarn-default.xml or we don't tweak TestYarnConfigurationFields. > Add timeout to RouterWebServiceUtil#invokeRMWebService > -- > > Key: YARN-8529 > URL: https://issues.apache.org/jira/browse/YARN-8529 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Íñigo Goiri >Assignee: Minni Mittal >Priority: Major > Attachments: YARN-8529.v1.patch, YARN-8529.v2.patch, > YARN-8529.v3.patch, YARN-8529.v4.patch, YARN-8529.v5.patch, > YARN-8529.v6.patch, YARN-8529.v7.patch > > > {{RouterWebServiceUtil#invokeRMWebService}} currently has a fixed timeout. > This should be configurable. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-9338) Timeline related testcases are failing
[ https://issues.apache.org/jira/browse/YARN-9338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ahmed Hussein reassigned YARN-9338: --- Assignee: Abhishek Modi (was: Ahmed Hussein) > Timeline related testcases are failing > -- > > Key: YARN-9338 > URL: https://issues.apache.org/jira/browse/YARN-9338 > Project: Hadoop YARN > Issue Type: Test >Reporter: Prabhu Joseph >Assignee: Abhishek Modi >Priority: Major > Fix For: 3.3.0 > > Attachments: YARN-9338.001.patch, YARN-9338.002.patch, > YARN-9338.003.patch, YARN-9338.004.patch > > > Timeline related testcases are failing > {code} > [ERROR] Failures: > [ERROR] > TestCombinedSystemMetricsPublisher.testTimelineServiceEventPublishingV2Enabled:262->runTest:245->validateV2:382->verifyEntity:417 > Expected 2 events to be published expected:<2> but was:<1> > [ERROR] > TestSystemMetricsPublisherForV2.testPublishAppAttemptMetrics:259->verifyEntity:332 > Expected 2 events to be published expected:<2> but was:<1> > [ERROR] > TestSystemMetricsPublisherForV2.testPublishApplicationMetrics:224->verifyEntity:332 > Expected 4 events to be published expected:<4> but was:<1> > [ERROR] > TestSystemMetricsPublisherForV2.testPublishContainerMetrics:291->verifyEntity:332 > Expected 2 events to be published expected:<2> but was:<1> > [ERROR] Errors: > [ERROR] > TestCombinedSystemMetricsPublisher.testTimelineServiceEventPublishingV1V2Enabled:252->runTest:242->testSetup:123 > » YarnRuntime > [ERROR] Failures: > [ERROR] > TestTimelineAuthFilterForV2.testPutTimelineEntities:343->access$000:87->publishAndVerifyEntity:307 > expected:<1> but was:<2> > [ERROR] > TestTimelineAuthFilterForV2.testPutTimelineEntities:352->publishWithRetries:320->publishAndVerifyEntity:307 > expected:<1> but was:<2> > [ERROR] > TestTimelineAuthFilterForV2.testPutTimelineEntities:352->publishWithRetries:320->publishAndVerifyEntity:307 > expected:<1> but was:<2> > [ERROR] > TestTimelineAuthFilterForV2.testPutTimelineEntities:343->access$000:87->publishAndVerifyEntity:307 > expected:<1> but was:<2> > [INFO] > [ERROR] Failures: > [ERROR] > TestDistributedShell.testDSShellWithoutDomainV2:313->testDSShell:317->testDSShell:458->checkTimelineV2:557->verifyEntityForTimelineV2:710 > Unexpected number of DS_APP_ATTEMPT_START event published. expected:<1> but > was:<0> > [ERROR] > TestDistributedShell.testDSShellWithoutDomainV2CustomizedFlow:329->testDSShell:458->checkTimelineV2:557->verifyEntityForTimelineV2:710 > Unexpected number of DS_APP_ATTEMPT_START event published. expected:<1> but > was:<0> > [ERROR] > TestDistributedShell.testDSShellWithoutDomainV2DefaultFlow:323->testDSShell:458->checkTimelineV2:557->verifyEntityForTimelineV2:710 > Unexpected number of DS_APP_ATTEMPT_START event published. expected:<1> but > was:<0> > [ERROR] Failures: > [ERROR] > TestMRTimelineEventHandling.testMRNewTimelineServiceEventHandling:240->checkNewTimelineEvent:304->verifyEntity:462 > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-9338) Timeline related testcases are failing
[ https://issues.apache.org/jira/browse/YARN-9338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ahmed Hussein reassigned YARN-9338: --- Assignee: Ahmed Hussein (was: Abhishek Modi) > Timeline related testcases are failing > -- > > Key: YARN-9338 > URL: https://issues.apache.org/jira/browse/YARN-9338 > Project: Hadoop YARN > Issue Type: Test >Reporter: Prabhu Joseph >Assignee: Ahmed Hussein >Priority: Major > Fix For: 3.3.0 > > Attachments: YARN-9338.001.patch, YARN-9338.002.patch, > YARN-9338.003.patch, YARN-9338.004.patch > > > Timeline related testcases are failing > {code} > [ERROR] Failures: > [ERROR] > TestCombinedSystemMetricsPublisher.testTimelineServiceEventPublishingV2Enabled:262->runTest:245->validateV2:382->verifyEntity:417 > Expected 2 events to be published expected:<2> but was:<1> > [ERROR] > TestSystemMetricsPublisherForV2.testPublishAppAttemptMetrics:259->verifyEntity:332 > Expected 2 events to be published expected:<2> but was:<1> > [ERROR] > TestSystemMetricsPublisherForV2.testPublishApplicationMetrics:224->verifyEntity:332 > Expected 4 events to be published expected:<4> but was:<1> > [ERROR] > TestSystemMetricsPublisherForV2.testPublishContainerMetrics:291->verifyEntity:332 > Expected 2 events to be published expected:<2> but was:<1> > [ERROR] Errors: > [ERROR] > TestCombinedSystemMetricsPublisher.testTimelineServiceEventPublishingV1V2Enabled:252->runTest:242->testSetup:123 > » YarnRuntime > [ERROR] Failures: > [ERROR] > TestTimelineAuthFilterForV2.testPutTimelineEntities:343->access$000:87->publishAndVerifyEntity:307 > expected:<1> but was:<2> > [ERROR] > TestTimelineAuthFilterForV2.testPutTimelineEntities:352->publishWithRetries:320->publishAndVerifyEntity:307 > expected:<1> but was:<2> > [ERROR] > TestTimelineAuthFilterForV2.testPutTimelineEntities:352->publishWithRetries:320->publishAndVerifyEntity:307 > expected:<1> but was:<2> > [ERROR] > TestTimelineAuthFilterForV2.testPutTimelineEntities:343->access$000:87->publishAndVerifyEntity:307 > expected:<1> but was:<2> > [INFO] > [ERROR] Failures: > [ERROR] > TestDistributedShell.testDSShellWithoutDomainV2:313->testDSShell:317->testDSShell:458->checkTimelineV2:557->verifyEntityForTimelineV2:710 > Unexpected number of DS_APP_ATTEMPT_START event published. expected:<1> but > was:<0> > [ERROR] > TestDistributedShell.testDSShellWithoutDomainV2CustomizedFlow:329->testDSShell:458->checkTimelineV2:557->verifyEntityForTimelineV2:710 > Unexpected number of DS_APP_ATTEMPT_START event published. expected:<1> but > was:<0> > [ERROR] > TestDistributedShell.testDSShellWithoutDomainV2DefaultFlow:323->testDSShell:458->checkTimelineV2:557->verifyEntityForTimelineV2:710 > Unexpected number of DS_APP_ATTEMPT_START event published. expected:<1> but > was:<0> > [ERROR] Failures: > [ERROR] > TestMRTimelineEventHandling.testMRNewTimelineServiceEventHandling:240->checkNewTimelineEvent:304->verifyEntity:462 > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10043) FairOrderingPolicy Improvements
[ https://issues.apache.org/jira/browse/YARN-10043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikandan R updated YARN-10043: Attachment: YARN-10043.004.patch > FairOrderingPolicy Improvements > --- > > Key: YARN-10043 > URL: https://issues.apache.org/jira/browse/YARN-10043 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Manikandan R >Assignee: Manikandan R >Priority: Major > Attachments: YARN-10043.001.patch, YARN-10043.002.patch, > YARN-10043.003.patch, YARN-10043.004.patch > > > FairOrderingPolicy can be improved by using some of the approaches (only > relevant) implemented in FairSharePolicy of FS. This improvement has > significance in FS to CS migration context. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10043) FairOrderingPolicy Improvements
[ https://issues.apache.org/jira/browse/YARN-10043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17036349#comment-17036349 ] Manikandan R commented on YARN-10043: - Thanks [~snemeth]. Addressed all comments. {quote}You can remove the type argument to FairOrderingPolicy. {quote} Are you suggesting FairOrderingPolicy policy = new FairOrderingPolicy(); ? If so, we will need to do suppress warnings etc. > FairOrderingPolicy Improvements > --- > > Key: YARN-10043 > URL: https://issues.apache.org/jira/browse/YARN-10043 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Manikandan R >Assignee: Manikandan R >Priority: Major > Attachments: YARN-10043.001.patch, YARN-10043.002.patch, > YARN-10043.003.patch > > > FairOrderingPolicy can be improved by using some of the approaches (only > relevant) implemented in FairSharePolicy of FS. This improvement has > significance in FS to CS migration context. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10137) UIv2 build is broken in trunk
[ https://issues.apache.org/jira/browse/YARN-10137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17036336#comment-17036336 ] Hadoop QA commented on YARN-10137: -- | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 41s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 19m 57s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 34m 12s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 12s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 15m 18s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 26s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 51m 29s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Docker | Client=19.03.6 Server=19.03.6 Image:yetus/hadoop:c44943d1fc3 | | JIRA Issue | YARN-10137 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12993388/YARN-10137.001.patch | | Optional Tests | dupname asflicense shadedclient | | uname | Linux 17b5e509db33 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/patchprocess/precommit/personality/provided.sh | | git revision | trunk / a98352c | | maven | version: Apache Maven 3.3.9 | | Max. process+thread count | 324 (vs. ulimit of 5500) | | modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui | | Console output | https://builds.apache.org/job/PreCommit-YARN-Build/25521/console | | Powered by | Apache Yetus 0.8.0 http://yetus.apache.org | This message was automatically generated. > UIv2 build is broken in trunk > - > > Key: YARN-10137 > URL: https://issues.apache.org/jira/browse/YARN-10137 > Project: Hadoop YARN > Issue Type: Improvement > Components: yarn, yarn-ui-v2 >Affects Versions: 3.3.0 >Reporter: Adam Antal >Assignee: Adam Antal >Priority: Critical > Attachments: YARN-10137.001.patch > > > YARN-10029 broke the UI build. > {noformat} > mvn clean package -DskipTests -Pyarn-ui > [...] > File: yarn-ui/controllers/yarn-app/logs.js > yarn-ui/controllers/yarn-app/logs.js: Unexpected token (78:49) > SyntaxError: yarn-ui/controllers/yarn-app/logs.js: Unexpected token (78:49) > let historyContainers = []; > historyProvider.get("content").forEach(function(o) { > if(!containerIdArr[o[fieldName]])) { > historyContainers.push(o); > } > }.bind(this)); > [...] > {noformat} -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10137) UIv2 build is broken in trunk
[ https://issues.apache.org/jira/browse/YARN-10137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17036320#comment-17036320 ] Hudson commented on YARN-10137: --- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #17952 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/17952/]) YARN-10137. UIv2 build is broken in trunk. Contributed by Adam Antal (snemeth: rev da99ac7e93722bedd4fca1dd6c992140fcf927e1) * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/app/controllers/yarn-app/logs.js > UIv2 build is broken in trunk > - > > Key: YARN-10137 > URL: https://issues.apache.org/jira/browse/YARN-10137 > Project: Hadoop YARN > Issue Type: Improvement > Components: yarn, yarn-ui-v2 >Affects Versions: 3.3.0 >Reporter: Adam Antal >Assignee: Adam Antal >Priority: Critical > Attachments: YARN-10137.001.patch > > > YARN-10029 broke the UI build. > {noformat} > mvn clean package -DskipTests -Pyarn-ui > [...] > File: yarn-ui/controllers/yarn-app/logs.js > yarn-ui/controllers/yarn-app/logs.js: Unexpected token (78:49) > SyntaxError: yarn-ui/controllers/yarn-app/logs.js: Unexpected token (78:49) > let historyContainers = []; > historyProvider.get("content").forEach(function(o) { > if(!containerIdArr[o[fieldName]])) { > historyContainers.push(o); > } > }.bind(this)); > [...] > {noformat} -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10137) UIv2 build is broken in trunk
[ https://issues.apache.org/jira/browse/YARN-10137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17036306#comment-17036306 ] Szilard Nemeth commented on YARN-10137: --- Thanks [~adam.antal] for fixing this quickly! Given that UI2 is broken and you have a successful build I didn't wait for a green Jenkins build and pushed your fix to trunk. Green jenkins didn't mean too much anyways as for YARN-10029 we had it but the code was broken. > UIv2 build is broken in trunk > - > > Key: YARN-10137 > URL: https://issues.apache.org/jira/browse/YARN-10137 > Project: Hadoop YARN > Issue Type: Improvement > Components: yarn, yarn-ui-v2 >Affects Versions: 3.3.0 >Reporter: Adam Antal >Assignee: Adam Antal >Priority: Critical > Attachments: YARN-10137.001.patch > > > YARN-10029 broke the UI build. > {noformat} > mvn clean package -DskipTests -Pyarn-ui > [...] > File: yarn-ui/controllers/yarn-app/logs.js > yarn-ui/controllers/yarn-app/logs.js: Unexpected token (78:49) > SyntaxError: yarn-ui/controllers/yarn-app/logs.js: Unexpected token (78:49) > let historyContainers = []; > historyProvider.get("content").forEach(function(o) { > if(!containerIdArr[o[fieldName]])) { > historyContainers.push(o); > } > }.bind(this)); > [...] > {noformat} -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10137) UIv2 build is broken in trunk
[ https://issues.apache.org/jira/browse/YARN-10137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17036298#comment-17036298 ] Adam Antal commented on YARN-10137: --- Passes on my local using the {{mvn clean package -DskipTests -Pyarn-ui}} command. {noformat} [...] [INFO] Apache Hadoop Cloud Storage Project SUCCESS [ 0.035 s] [INFO] [INFO] BUILD SUCCESS [INFO] [INFO] Total time: 10:50 min [INFO] Finished at: 2020-02-13T16:20:16+01:00 [INFO] {noformat} > UIv2 build is broken in trunk > - > > Key: YARN-10137 > URL: https://issues.apache.org/jira/browse/YARN-10137 > Project: Hadoop YARN > Issue Type: Improvement > Components: yarn, yarn-ui-v2 >Affects Versions: 3.3.0 >Reporter: Adam Antal >Assignee: Adam Antal >Priority: Critical > Attachments: YARN-10137.001.patch > > > YARN-10029 broke the UI build. > {noformat} > mvn clean package -DskipTests -Pyarn-ui > [...] > File: yarn-ui/controllers/yarn-app/logs.js > yarn-ui/controllers/yarn-app/logs.js: Unexpected token (78:49) > SyntaxError: yarn-ui/controllers/yarn-app/logs.js: Unexpected token (78:49) > let historyContainers = []; > historyProvider.get("content").forEach(function(o) { > if(!containerIdArr[o[fieldName]])) { > historyContainers.push(o); > } > }.bind(this)); > [...] > {noformat} -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10137) UIv2 build is broken in trunk
[ https://issues.apache.org/jira/browse/YARN-10137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Antal updated YARN-10137: -- Attachment: YARN-10137.001.patch > UIv2 build is broken in trunk > - > > Key: YARN-10137 > URL: https://issues.apache.org/jira/browse/YARN-10137 > Project: Hadoop YARN > Issue Type: Improvement > Components: yarn, yarn-ui-v2 >Affects Versions: 3.3.0 >Reporter: Adam Antal >Assignee: Adam Antal >Priority: Critical > Attachments: YARN-10137.001.patch > > > YARN-10029 broke the UI build. > {noformat} > mvn clean package -DskipTests -Pyarn-ui > [...] > File: yarn-ui/controllers/yarn-app/logs.js > yarn-ui/controllers/yarn-app/logs.js: Unexpected token (78:49) > SyntaxError: yarn-ui/controllers/yarn-app/logs.js: Unexpected token (78:49) > let historyContainers = []; > historyProvider.get("content").forEach(function(o) { > if(!containerIdArr[o[fieldName]])) { > historyContainers.push(o); > } > }.bind(this)); > [...] > {noformat} -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-10137) UIv2 build is broken in trunk
Adam Antal created YARN-10137: - Summary: UIv2 build is broken in trunk Key: YARN-10137 URL: https://issues.apache.org/jira/browse/YARN-10137 Project: Hadoop YARN Issue Type: Improvement Components: yarn, yarn-ui-v2 Affects Versions: 3.3.0 Reporter: Adam Antal Assignee: Adam Antal YARN-10029 broke the UI build. {noformat} mvn clean package -DskipTests -Pyarn-ui [...] File: yarn-ui/controllers/yarn-app/logs.js yarn-ui/controllers/yarn-app/logs.js: Unexpected token (78:49) SyntaxError: yarn-ui/controllers/yarn-app/logs.js: Unexpected token (78:49) let historyContainers = []; historyProvider.get("content").forEach(function(o) { if(!containerIdArr[o[fieldName]])) { historyContainers.push(o); } }.bind(this)); [...] {noformat} -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-10136) In Secure Federation, Router About UI page failed to display the subclusters information appropriately.
[ https://issues.apache.org/jira/browse/YARN-10136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilwa S T reassigned YARN-10136: Assignee: Bilwa S T > In Secure Federation, Router About UI page failed to display the subclusters > information appropriately. > --- > > Key: YARN-10136 > URL: https://issues.apache.org/jira/browse/YARN-10136 > Project: Hadoop YARN > Issue Type: Bug > Components: federation, yarn >Reporter: Sushanta Sen >Assignee: Bilwa S T >Priority: Major > > In Secure Federation, Router About UI page failed to display the subclusters > information appropriately. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-10136) In Secure Federation, Router About UI page failed to display the subclusters information appropriately.
Sushanta Sen created YARN-10136: --- Summary: In Secure Federation, Router About UI page failed to display the subclusters information appropriately. Key: YARN-10136 URL: https://issues.apache.org/jira/browse/YARN-10136 Project: Hadoop YARN Issue Type: Bug Components: federation, yarn Reporter: Sushanta Sen In Secure Federation, Router About UI page failed to display the subclusters information appropriately. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8529) Add timeout to RouterWebServiceUtil#invokeRMWebService
[ https://issues.apache.org/jira/browse/YARN-8529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17036176#comment-17036176 ] Hadoop QA commented on YARN-8529: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m 11s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 24s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 23m 48s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m 55s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 25s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m 7s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 18m 56s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 26s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 58s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 15s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 37s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m 42s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m 42s{color} | {color:green} the patch passed {color} | | {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange} 1m 21s{color} | {color:orange} hadoop-yarn-project/hadoop-yarn: The patch generated 2 new + 214 unchanged - 1 fixed = 216 total (was 215) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m 1s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 2s{color} | {color:green} The patch has no ill-formed XML file. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 14m 48s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 30s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 47s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 52s{color} | {color:red} hadoop-yarn-api in the patch passed. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m 48s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 34s{color} | {color:green} hadoop-yarn-server-router in the patch passed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 38s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}102m 3s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.yarn.conf.TestYarnConfigurationFields | \\ \\ || Subsystem || Report/Notes || | Docker | Client=19.03.6 Server=19.03.6 Image:yetus/hadoop:c44943d1fc3 | | JIRA Issue | YARN-8529 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12993353/YARN-8529.v7.patch | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle xml | | uname | Linux f0bb21b729cf
[jira] [Updated] (YARN-10135) FS-CS converter tool: issue warning on dynamic auto-create mapping rules
[ https://issues.apache.org/jira/browse/YARN-10135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Bacsko updated YARN-10135: Labels: fs2cs (was: ) > FS-CS converter tool: issue warning on dynamic auto-create mapping rules > > > Key: YARN-10135 > URL: https://issues.apache.org/jira/browse/YARN-10135 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Peter Bacsko >Assignee: Peter Bacsko >Priority: Major > Labels: fs2cs > > The converter tool should issue a warning whenever the conversion results in > mapping rules similar to these: > {{u:%user:[managedParentQueueName].[queueName]}} > {{u:%user:[managedParentQueueName].%user}} > {{u:%user:[managedParentQueueName].%primary_group}} > {{u:%user:[managedParentQueueName].%secondary_group}} > {{u:%user:%primary_group.%user}} > {{u:%user:%secondary_group.%user}} > {{u:%user:[managedParentQueuePath].%user}} > > The reason is that right now it's fully clear how we'll handle a case like > "u:%user:%primary_group.%user", where "%primary_group.%user" might result in > something like "users.john". > In case of "u:%user:[managedParentQueuePath].%user" , the > [managedParentQueuePath] is a result of a full path from Fair Scheduler. > Therefore it's not going to be a leaf queue. > The user might be required to do some fine tuning and adjust the property > "auto-create-child-queues". We should display a warning about these > additional steps. > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-10135) FS-CS converter tool: issue warning on dynamic auto-create mapping rules
Peter Bacsko created YARN-10135: --- Summary: FS-CS converter tool: issue warning on dynamic auto-create mapping rules Key: YARN-10135 URL: https://issues.apache.org/jira/browse/YARN-10135 Project: Hadoop YARN Issue Type: Sub-task Reporter: Peter Bacsko Assignee: Peter Bacsko The converter tool should issue a warning whenever the conversion results in mapping rules similar to these: {{u:%user:[managedParentQueueName].[queueName]}} {{u:%user:[managedParentQueueName].%user}} {{u:%user:[managedParentQueueName].%primary_group}} {{u:%user:[managedParentQueueName].%secondary_group}} {{u:%user:%primary_group.%user}} {{u:%user:%secondary_group.%user}} {{u:%user:[managedParentQueuePath].%user}} The reason is that right now it's fully clear how we'll handle a case like "u:%user:%primary_group.%user", where "%primary_group.%user" might result in something like "users.john". In case of "u:%user:[managedParentQueuePath].%user" , the [managedParentQueuePath] is a result of a full path from Fair Scheduler. Therefore it's not going to be a leaf queue. The user might be required to do some fine tuning and adjust the property "auto-create-child-queues". We should display a warning about these additional steps. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10029) Add option to UIv2 to get container logs from the new JHS API
[ https://issues.apache.org/jira/browse/YARN-10029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17036141#comment-17036141 ] Hudson commented on YARN-10029: --- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #17950 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/17950/]) YARN-10029. Add option to UIv2 to get container logs from the new JHS (snemeth: rev f1b1b332f5d5b873398e4fa7319e967747674b45) * (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/app/adapters/yarn-jhs-container.js * (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/app/models/jhs-health.js * (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/app/adapters/jhs-health.js * (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/app/adapters/yarn-jhs-log.js * (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/app/models/yarn-jhs-log.js * (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/app/serializers/yarn-jhs-log.js * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/config/default-config.js * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/app/initializers/loader.js * (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/app/serializers/jhs-health.js * (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/app/serializers/yarn-jhs-container.js * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/app/controllers/yarn-app/logs.js * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/config/configs.env * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/app/templates/yarn-app/logs.hbs * (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/app/models/yarn-jhs-container.js * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/app/routes/application.js * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/app/routes/yarn-app/logs.js * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/app/services/hosts.js * (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/app/models/yarn-app-jhs-log.js * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/app/adapters/yarn-app-log.js * (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/app/serializers/yarn-app-jhs-log.js * (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/app/adapters/yarn-app-jhs-log.js > Add option to UIv2 to get container logs from the new JHS API > - > > Key: YARN-10029 > URL: https://issues.apache.org/jira/browse/YARN-10029 > Project: Hadoop YARN > Issue Type: Sub-task > Components: yarn >Affects Versions: 3.2.1 >Reporter: Adam Antal >Assignee: Adam Antal >Priority: Major > Fix For: 3.3.0 > > Attachments: YARN-10029.001.patch, YARN-10029.002.patch, > YARN-10029.003.patch, YARN-10029.004.patch > > > Provided the new API is ready to use (also integrated into JHS in > YARN-10028), we can add a new config option to UIv2 that would make the UIv2 > to request logs from the JHS API similarly as the ATSv2. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10029) Add option to UIv2 to get container logs from the new JHS API
[ https://issues.apache.org/jira/browse/YARN-10029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17036138#comment-17036138 ] Szilard Nemeth commented on YARN-10029: --- Thanks [~adam.antal] for this patch, Latest patch looks good to me, committed to trunk! Resolving jira. > Add option to UIv2 to get container logs from the new JHS API > - > > Key: YARN-10029 > URL: https://issues.apache.org/jira/browse/YARN-10029 > Project: Hadoop YARN > Issue Type: Sub-task > Components: yarn >Affects Versions: 3.2.1 >Reporter: Adam Antal >Assignee: Adam Antal >Priority: Major > Attachments: YARN-10029.001.patch, YARN-10029.002.patch, > YARN-10029.003.patch, YARN-10029.004.patch > > > Provided the new API is ready to use (also integrated into JHS in > YARN-10028), we can add a new config option to UIv2 that would make the UIv2 > to request logs from the JHS API similarly as the ATSv2. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8529) Add timeout to RouterWebServiceUtil#invokeRMWebService
[ https://issues.apache.org/jira/browse/YARN-8529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Minni Mittal updated YARN-8529: --- Attachment: YARN-8529.v7.patch > Add timeout to RouterWebServiceUtil#invokeRMWebService > -- > > Key: YARN-8529 > URL: https://issues.apache.org/jira/browse/YARN-8529 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Íñigo Goiri >Assignee: Minni Mittal >Priority: Major > Attachments: YARN-8529.v1.patch, YARN-8529.v2.patch, > YARN-8529.v3.patch, YARN-8529.v4.patch, YARN-8529.v5.patch, > YARN-8529.v6.patch, YARN-8529.v7.patch > > > {{RouterWebServiceUtil#invokeRMWebService}} currently has a fixed timeout. > This should be configurable. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10029) Add option to UIv2 to get container logs from the new JHS API
[ https://issues.apache.org/jira/browse/YARN-10029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17036086#comment-17036086 ] Adam Antal commented on YARN-10029: --- Got +1 from jenkins. Branch-3.2 is not applicable here as a few commits like YARN-8964 is missing from that branch. > Add option to UIv2 to get container logs from the new JHS API > - > > Key: YARN-10029 > URL: https://issues.apache.org/jira/browse/YARN-10029 > Project: Hadoop YARN > Issue Type: Sub-task > Components: yarn >Affects Versions: 3.2.1 >Reporter: Adam Antal >Assignee: Adam Antal >Priority: Major > Attachments: YARN-10029.001.patch, YARN-10029.002.patch, > YARN-10029.003.patch, YARN-10029.004.patch > > > Provided the new API is ready to use (also integrated into JHS in > YARN-10028), we can add a new config option to UIv2 that would make the UIv2 > to request logs from the JHS API similarly as the ATSv2. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-8529) Add timeout to RouterWebServiceUtil#invokeRMWebService
[ https://issues.apache.org/jira/browse/YARN-8529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17035328#comment-17035328 ] Minni Mittal edited comment on YARN-8529 at 2/13/20 9:30 AM: - Client created is not getting closed as there is missing close object in com.sun.jersey.api.client.Client. So, added patch that destroys client on completion of REST call. was (Author: minni31): Added patch with handling leak in Router. > Add timeout to RouterWebServiceUtil#invokeRMWebService > -- > > Key: YARN-8529 > URL: https://issues.apache.org/jira/browse/YARN-8529 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Íñigo Goiri >Assignee: Minni Mittal >Priority: Major > Attachments: YARN-8529.v1.patch, YARN-8529.v2.patch, > YARN-8529.v3.patch, YARN-8529.v4.patch, YARN-8529.v5.patch, YARN-8529.v6.patch > > > {{RouterWebServiceUtil#invokeRMWebService}} currently has a fixed timeout. > This should be configurable. > Currently, client is only getting created. So, leak in Router should be > handle. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8529) Add timeout to RouterWebServiceUtil#invokeRMWebService
[ https://issues.apache.org/jira/browse/YARN-8529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Minni Mittal updated YARN-8529: --- Description: {{RouterWebServiceUtil#invokeRMWebService}} currently has a fixed timeout. This should be configurable. (was: {{RouterWebServiceUtil#invokeRMWebService}} currently has a fixed timeout. This should be configurable. Currently, client is only getting created. So, leak in Router should be handle.) > Add timeout to RouterWebServiceUtil#invokeRMWebService > -- > > Key: YARN-8529 > URL: https://issues.apache.org/jira/browse/YARN-8529 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Íñigo Goiri >Assignee: Minni Mittal >Priority: Major > Attachments: YARN-8529.v1.patch, YARN-8529.v2.patch, > YARN-8529.v3.patch, YARN-8529.v4.patch, YARN-8529.v5.patch, YARN-8529.v6.patch > > > {{RouterWebServiceUtil#invokeRMWebService}} currently has a fixed timeout. > This should be configurable. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-8529) Add timeout to RouterWebServiceUtil#invokeRMWebService
[ https://issues.apache.org/jira/browse/YARN-8529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17035328#comment-17035328 ] Minni Mittal edited comment on YARN-8529 at 2/13/20 9:30 AM: - Client created is not getting closed as there is missing close object in com.sun.jersey.api.client.Client. So, added patch that destroys client on completion of REST call in remote RMWebService. was (Author: minni31): Client created is not getting closed as there is missing close object in com.sun.jersey.api.client.Client. So, added patch that destroys client on completion of REST call. > Add timeout to RouterWebServiceUtil#invokeRMWebService > -- > > Key: YARN-8529 > URL: https://issues.apache.org/jira/browse/YARN-8529 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Íñigo Goiri >Assignee: Minni Mittal >Priority: Major > Attachments: YARN-8529.v1.patch, YARN-8529.v2.patch, > YARN-8529.v3.patch, YARN-8529.v4.patch, YARN-8529.v5.patch, YARN-8529.v6.patch > > > {{RouterWebServiceUtil#invokeRMWebService}} currently has a fixed timeout. > This should be configurable. > Currently, client is only getting created. So, leak in Router should be > handle. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org