[jira] [Commented] (YARN-11560) Fix NPE bug when multi-node enabled with schedule asynchronously
[ https://issues.apache.org/jira/browse/YARN-11560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17883742#comment-17883742 ] Ayush Saxena commented on YARN-11560: - Committed to trunk. Thanx [~zhongwei11] for the contribution!!! > Fix NPE bug when multi-node enabled with schedule asynchronously > > > Key: YARN-11560 > URL: https://issues.apache.org/jira/browse/YARN-11560 > Project: Hadoop YARN > Issue Type: Bug > Components: capacity scheduler >Affects Versions: 3.3.3 >Reporter: wangzhongwei >Assignee: wangzhongwei >Priority: Blocker > Labels: pull-request-available > > when multiNodePlacementEnabled,using global scheduler,NPE may happend when > commit thread calling allocateFromReservedContainer with param > reservedContainer ,while the container may be unreserved by the judgment > thread in tryCommit->apply function -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11471) FederationStateStoreFacade Cache Support Caffeine
[ https://issues.apache.org/jira/browse/YARN-11471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17854248#comment-17854248 ] Ayush Saxena commented on YARN-11471: - [~slfan1989]/[~elgoiri] There are some 180 test failing in Yarn https://ci-hadoop.apache.org/view/Hadoop/job/hadoop-qbt-trunk-java8-linux-x86_64/1604/testReport/junit/ One such example: https://ci-hadoop.apache.org/view/Hadoop/job/hadoop-qbt-trunk-java8-linux-x86_64/1604/testReport/junit/org.apache.hadoop.yarn.server.federation.policies/TestRouterPolicyFacade/testGetHomeSubcluster/ I think the NPE is coming from the code introduced here: {noformat} public void clearCache() { this.cache.clear(); this.cache = null; } {noformat} I think the cache is {{null}} when we call clear, I think adding a {{null}} check before calling clear might solve the problem. Can you folks Please check, if this is related > FederationStateStoreFacade Cache Support Caffeine > - > > Key: YARN-11471 > URL: https://issues.apache.org/jira/browse/YARN-11471 > Project: Hadoop YARN > Issue Type: Improvement > Components: federation >Affects Versions: 3.4.0 >Reporter: Shilun Fan >Assignee: Shilun Fan >Priority: Major > Labels: pull-request-available > Fix For: 3.4.1, 3.5.0 > > > FederationStateStoreFacade Cache Support Caffeine -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-11671) Tests in hadoop-yarn-server-router are not running
Ayush Saxena created YARN-11671: --- Summary: Tests in hadoop-yarn-server-router are not running Key: YARN-11671 URL: https://issues.apache.org/jira/browse/YARN-11671 Project: Hadoop YARN Issue Type: Bug Reporter: Ayush Saxena [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1549/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-router.txt] {noformat} [INFO] --- maven-surefire-plugin:3.0.0-M1:test (default-test) @ hadoop-yarn-server-router --- [INFO] [INFO] --- [INFO] T E S T S [INFO] --- [INFO] [INFO] Results: [INFO] [INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0 [INFO] [INFO] [INFO] BUILD SUCCESS [INFO] {noformat} -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-11563) Fix typo in AbstractContainerAllocator from CSAssignemnt to CSAssignment
[ https://issues.apache.org/jira/browse/YARN-11563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena updated YARN-11563: Summary: Fix typo in AbstractContainerAllocator from CSAssignemnt to CSAssignment (was: Fix word misspellings from CSAssignemnt to CSAssignment) > Fix typo in AbstractContainerAllocator from CSAssignemnt to CSAssignment > > > Key: YARN-11563 > URL: https://issues.apache.org/jira/browse/YARN-11563 > Project: Hadoop YARN > Issue Type: Improvement > Components: capacityscheduler >Affects Versions: 3.3.3, 3.3.4, 3.3.6 >Reporter: wangzhongwei >Assignee: wangzhongwei >Priority: Trivial > Attachments: image-2023-12-11-17-55-21-930.png > > > misspellings of CSAssignemnt which shoud be CSAssignment in > AbstractContainerAllocator.java > !image-2023-12-11-17-55-21-930.png|width=389,height=159! -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11563) Fix typo in AbstractContainerAllocator from CSAssignemnt to CSAssignment
[ https://issues.apache.org/jira/browse/YARN-11563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17798662#comment-17798662 ] Ayush Saxena commented on YARN-11563: - Committed to trunk. Thanx [~zhongwei11] for the contribution!!! > Fix typo in AbstractContainerAllocator from CSAssignemnt to CSAssignment > > > Key: YARN-11563 > URL: https://issues.apache.org/jira/browse/YARN-11563 > Project: Hadoop YARN > Issue Type: Improvement > Components: capacityscheduler >Affects Versions: 3.3.3, 3.3.4, 3.3.6 >Reporter: wangzhongwei >Assignee: wangzhongwei >Priority: Trivial > Labels: pull-request-available > Attachments: image-2023-12-11-17-55-21-930.png > > > misspellings of CSAssignemnt which shoud be CSAssignment in > AbstractContainerAllocator.java > !image-2023-12-11-17-55-21-930.png|width=389,height=159! -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-11563) Fix word misspellings from CSAssignemnt to CSAssignment
[ https://issues.apache.org/jira/browse/YARN-11563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena reassigned YARN-11563: --- Assignee: wangzhongwei > Fix word misspellings from CSAssignemnt to CSAssignment > --- > > Key: YARN-11563 > URL: https://issues.apache.org/jira/browse/YARN-11563 > Project: Hadoop YARN > Issue Type: Improvement > Components: capacityscheduler >Affects Versions: 3.3.3, 3.3.4, 3.3.6 >Reporter: wangzhongwei >Assignee: wangzhongwei >Priority: Trivial > Attachments: image-2023-12-11-17-55-21-930.png > > > misspellings of CSAssignemnt which shoud be CSAssignment in > AbstractContainerAllocator.java > !image-2023-12-11-17-55-21-930.png|width=389,height=159! -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9568) NPE in MiniYarnCluster during FileSystemNodeAttributeStore.recover
[ https://issues.apache.org/jira/browse/YARN-9568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17783707#comment-17783707 ] Ayush Saxena commented on YARN-9568: I believe this fix breaks the Windows test(HDFS-17246) > NPE in MiniYarnCluster during FileSystemNodeAttributeStore.recover > -- > > Key: YARN-9568 > URL: https://issues.apache.org/jira/browse/YARN-9568 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager, test >Affects Versions: 3.3.0 > Environment: macos >Reporter: Steve Loughran >Assignee: Steve Loughran >Priority: Minor > Fix For: 3.3.0 > > Attachments: YARN-9568.001.patch, YARN-9568.002.patch, npe.log > > > This seems new in trunk. As in "wasn't happening a couple of weeks ago". Its > surfacing in the S3A committer tests which are trying to create > MiniYarnClusters: all such tests are failing as the mini yarn cluster won't > come up with an NPE in {{FileSystemNodeAttributeStore.recover}} > I'm not sure why node labels are needed on test clusters; the default implies > they should be off anyway. > At the same time, I can't seem to find one specific change in the git log to > say "this is causing the problem". -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-11607) TestTimelineAuthFilterForV2 fails intermittently
Ayush Saxena created YARN-11607: --- Summary: TestTimelineAuthFilterForV2 fails intermittently Key: YARN-11607 URL: https://issues.apache.org/jira/browse/YARN-11607 Project: Hadoop YARN Issue Type: Improvement Reporter: Ayush Saxena Ref: https://ci-hadoop.apache.org/view/Hadoop/job/hadoop-qbt-trunk-java8-linux-x86_64/1398/testReport/junit/org.apache.hadoop.yarn.server.timelineservice.security/TestTimelineAuthFilterForV2/testPutTimelineEntities_boolean__boolean__3_/ {noformat} org.opentest4j.AssertionFailedError: expected: <2> but was: <1> at org.junit.jupiter.api.AssertionUtils.fail(AssertionUtils.java:55) at org.junit.jupiter.api.AssertionUtils.failNotEqual(AssertionUtils.java:62) at org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:150) at org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:145) at org.junit.jupiter.api.Assertions.assertEquals(Assertions.java:527) at org.apache.hadoop.yarn.server.timelineservice.security.TestTimelineAuthFilterForV2.publishAndVerifyEntity(TestTimelineAuthFilterForV2.java:324) at org.apache.hadoop.yarn.server.timelineservice.security.TestTimelineAuthFilterForV2.publishWithRetries(TestTimelineAuthFilterForV2.java:337) at org.apache.hadoop.yarn.server.timelineservice.security.TestTimelineAuthFilterForV2.testPutTimelineEntities(TestTimelineAuthFilterForV2.java:383) {noformat} -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Resolved] (YARN-11540) Fix typo: form -> from
[ https://issues.apache.org/jira/browse/YARN-11540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena resolved YARN-11540. - Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Fix typo: form -> from > -- > > Key: YARN-11540 > URL: https://issues.apache.org/jira/browse/YARN-11540 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Seokchan Yoon >Assignee: Seokchan Yoon >Priority: Trivial > Labels: pull-request-available > Fix For: 3.4.0 > > > There is a typo in a log message that NodeManager prints: > > Retrieved credentials form RM for {}: {} > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11540) Fix typo: form -> from
[ https://issues.apache.org/jira/browse/YARN-11540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17745234#comment-17745234 ] Ayush Saxena commented on YARN-11540: - Committed to trunk. Thanx [~yoonsch217] for the contribution & [~tomscut] for the review!!! > Fix typo: form -> from > -- > > Key: YARN-11540 > URL: https://issues.apache.org/jira/browse/YARN-11540 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Seokchan Yoon >Assignee: Seokchan Yoon >Priority: Trivial > Labels: pull-request-available > > There is a typo in a log message that NodeManager prints: > > Retrieved credentials form RM for {}: {} > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-11540) Fix typo: form -> from
[ https://issues.apache.org/jira/browse/YARN-11540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena updated YARN-11540: Priority: Trivial (was: Minor) > Fix typo: form -> from > -- > > Key: YARN-11540 > URL: https://issues.apache.org/jira/browse/YARN-11540 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Seokchan Yoon >Assignee: Seokchan Yoon >Priority: Trivial > Labels: pull-request-available > > There is a typo in a log message that NodeManager prints: > > Retrieved credentials form RM for {}: {} > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-11540) Fix typo: form -> from
[ https://issues.apache.org/jira/browse/YARN-11540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena reassigned YARN-11540: --- Assignee: Seokchan Yoon > Fix typo: form -> from > -- > > Key: YARN-11540 > URL: https://issues.apache.org/jira/browse/YARN-11540 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Seokchan Yoon >Assignee: Seokchan Yoon >Priority: Minor > Labels: pull-request-available > > There is a typo in a log message that NodeManager prints: > > Retrieved credentials form RM for {}: {} > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Moved] (YARN-11540) Fix typo: form -> from
[ https://issues.apache.org/jira/browse/YARN-11540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena moved HADOOP-18813 to YARN-11540: -- Component/s: (was: common) Key: YARN-11540 (was: HADOOP-18813) Project: Hadoop YARN (was: Hadoop Common) > Fix typo: form -> from > -- > > Key: YARN-11540 > URL: https://issues.apache.org/jira/browse/YARN-11540 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Seokchan Yoon >Priority: Minor > Labels: pull-request-available > > There is a typo in a log message that NodeManager prints: > > Retrieved credentials form RM for {}: {} > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11528) Lock triple-beam to the version compatible with node.js 12 to avoid compilation error
[ https://issues.apache.org/jira/browse/YARN-11528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17741872#comment-17741872 ] Ayush Saxena commented on YARN-11528: - Committed to trunk & branch-3.3 Thanx [~iwasakims] for the contribution & [~slfan1989] for the review!!! > Lock triple-beam to the version compatible with node.js 12 to avoid > compilation error > - > > Key: YARN-11528 > URL: https://issues.apache.org/jira/browse/YARN-11528 > Project: Hadoop YARN > Issue Type: Bug > Components: build >Affects Versions: 3.4.0 >Reporter: Ayush Saxena >Assignee: Masatake Iwasaki >Priority: Major > Labels: pull-request-available > > {code:java} > 2023-07-10T07:32:20.1069856Z [INFO] yarn install v1.22.5 > 2023-07-10T07:32:20.1435049Z [INFO] info No lockfile found. > 2023-07-10T07:32:20.1489685Z [INFO] [1/4] Resolving packages... > 2023-07-10T07:32:20.9138642Z [INFO] warning angular-route@1.6.10: For the > actively supported Angular, see https://www.npmjs.com/package/@angular/core. > AngularJS support has officially ended. For extended AngularJS support > options, see https://goo.gle/angularjs-path-forward. > 2023-07-10T07:32:20.9433934Z [INFO] warning angular@1.6.10: For the actively > supported Angular, see https://www.npmjs.com/package/@angular/core. AngularJS > support has officially ended. For extended AngularJS support options, see > https://goo.gle/angularjs-path-forward. > 2023-07-10T07:32:21.5051267Z [INFO] [2/4] Fetching packages... > 2023-07-10T07:32:23.2554282Z [INFO] error triple-beam@1.4.0: The engine > "node" is incompatible with this module. Expected version ">= 16.0.0". Got > "12.22.1" > 2023-07-10T07:32:23.2651719Z [INFO] error Found incompatible module. > 2023-07-10T07:32:23.2664307Z [INFO] info Visit > https://yarnpkg.com/en/docs/cli/install for documentation about this command. > {code} > Ref: > https://pipelines.actions.githubusercontent.com/serviceHosts/a3203f99-c5b3-4def-b81d-c5a8ebd0356d/_apis/pipelines/1/runs/294/signedlogcontent/2?urlExpires=2023-07-10T09%3A08%3A57.5185178Z&urlSigningMethod=HMACV1&urlSignature=OHImbS%2FdsTBFKRxwRnV6jFPnumE0GxbB%2BalrVpAgmyw%3D -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Resolved] (YARN-11506) The formatted yarn queue list is displayed on the command line
[ https://issues.apache.org/jira/browse/YARN-11506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena resolved YARN-11506. - Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > The formatted yarn queue list is displayed on the command line > -- > > Key: YARN-11506 > URL: https://issues.apache.org/jira/browse/YARN-11506 > Project: Hadoop YARN > Issue Type: Improvement >Affects Versions: 3.3.4 >Reporter: Lu Yuan >Assignee: Lu Yuan >Priority: Minor > Labels: pull-request-available > Fix For: 3.4.0 > > > h4. In the issue list YARN-11474, the printed information is not aligned,This > issue fixes the yarn queue display formatting problem -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11506) The formatted yarn queue list is displayed on the command line
[ https://issues.apache.org/jira/browse/YARN-11506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17733883#comment-17733883 ] Ayush Saxena commented on YARN-11506: - Committed to trunk. Thanx [~shaqyl09099] for the contribution & [~slfan1989] for the review!!! > The formatted yarn queue list is displayed on the command line > -- > > Key: YARN-11506 > URL: https://issues.apache.org/jira/browse/YARN-11506 > Project: Hadoop YARN > Issue Type: Improvement >Affects Versions: 3.3.4 >Reporter: Lu Yuan >Assignee: Lu Yuan >Priority: Minor > Labels: pull-request-available > > h4. In the issue list YARN-11474, the printed information is not aligned,This > issue fixes the yarn queue display formatting problem -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-11506) The formatted yarn queue list is displayed on the command line
[ https://issues.apache.org/jira/browse/YARN-11506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena reassigned YARN-11506: --- Assignee: Lu Yuan > The formatted yarn queue list is displayed on the command line > -- > > Key: YARN-11506 > URL: https://issues.apache.org/jira/browse/YARN-11506 > Project: Hadoop YARN > Issue Type: Improvement >Affects Versions: 3.3.4 >Reporter: Lu Yuan >Assignee: Lu Yuan >Priority: Minor > Labels: pull-request-available > > h4. In the issue list YARN-11474, the printed information is not aligned,This > issue fixes the yarn queue display formatting problem -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Resolved] (YARN-11492) Improve createJerseyClient#setConnectTimeout Code
[ https://issues.apache.org/jira/browse/YARN-11492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena resolved YARN-11492. - Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Improve createJerseyClient#setConnectTimeout Code > - > > Key: YARN-11492 > URL: https://issues.apache.org/jira/browse/YARN-11492 > Project: Hadoop YARN > Issue Type: Improvement > Components: federation >Affects Versions: 3.4.0 >Reporter: Shilun Fan >Assignee: Shilun Fan >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > > createJerseyClient#setConnectTimeout There is a type conversion code that > converts long to int, we should be careful with this part of code, I added > some verification rules. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11492) Improve createJerseyClient#setConnectTimeout Code
[ https://issues.apache.org/jira/browse/YARN-11492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17729277#comment-17729277 ] Ayush Saxena commented on YARN-11492: - Committed to trunk. Thanx [~slfan1989] for the contribution & [~elgoiri] for the review!!! > Improve createJerseyClient#setConnectTimeout Code > - > > Key: YARN-11492 > URL: https://issues.apache.org/jira/browse/YARN-11492 > Project: Hadoop YARN > Issue Type: Improvement > Components: federation >Affects Versions: 3.4.0 >Reporter: Shilun Fan >Assignee: Shilun Fan >Priority: Major > Labels: pull-request-available > > createJerseyClient#setConnectTimeout There is a type conversion code that > converts long to int, we should be careful with this part of code, I added > some verification rules. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11498) Exclude Jettison from jersey-json artifact in hadoop-yarn-common's pom.xml
[ https://issues.apache.org/jira/browse/YARN-11498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17725172#comment-17725172 ] Ayush Saxena commented on YARN-11498: - code change is not in Hadoop Common but in Yarn, have moved it to Yarn. Good to link the tickets which induced this and the affected hadoop version > Exclude Jettison from jersey-json artifact in hadoop-yarn-common's pom.xml > -- > > Key: YARN-11498 > URL: https://issues.apache.org/jira/browse/YARN-11498 > Project: Hadoop YARN > Issue Type: Task > Components: build >Reporter: Devaspati Krishnatri >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Moved] (YARN-11498) Exclude Jettison from jersey-json artifact in hadoop-yarn-common's pom.xml
[ https://issues.apache.org/jira/browse/YARN-11498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena moved HADOOP-18732 to YARN-11498: -- Component/s: build (was: build) Key: YARN-11498 (was: HADOOP-18732) Project: Hadoop YARN (was: Hadoop Common) > Exclude Jettison from jersey-json artifact in hadoop-yarn-common's pom.xml > -- > > Key: YARN-11498 > URL: https://issues.apache.org/jira/browse/YARN-11498 > Project: Hadoop YARN > Issue Type: Task > Components: build >Reporter: Devaspati Krishnatri >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11496) Improve TimelineService log format
[ https://issues.apache.org/jira/browse/YARN-11496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17724517#comment-17724517 ] Ayush Saxena commented on YARN-11496: - Committed to trunk. Thanx [~15217812...@163.com] for the contribution & [~slfan1989] for the review!!! > Improve TimelineService log format > -- > > Key: YARN-11496 > URL: https://issues.apache.org/jira/browse/YARN-11496 > Project: Hadoop YARN > Issue Type: Improvement > Components: timelineservice >Affects Versions: 3.3.4 >Reporter: Xianming Lei >Assignee: Xianming Lei >Priority: Minor > Labels: pull-request-available > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Resolved] (YARN-11496) Improve TimelineService log format
[ https://issues.apache.org/jira/browse/YARN-11496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena resolved YARN-11496. - Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Improve TimelineService log format > -- > > Key: YARN-11496 > URL: https://issues.apache.org/jira/browse/YARN-11496 > Project: Hadoop YARN > Issue Type: Improvement > Components: timelineservice >Affects Versions: 3.3.4 >Reporter: Xianming Lei >Assignee: Xianming Lei >Priority: Minor > Labels: pull-request-available > Fix For: 3.4.0 > > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-11496) Improve TimelineService log format
[ https://issues.apache.org/jira/browse/YARN-11496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena reassigned YARN-11496: --- Assignee: Xianming Lei > Improve TimelineService log format > -- > > Key: YARN-11496 > URL: https://issues.apache.org/jira/browse/YARN-11496 > Project: Hadoop YARN > Issue Type: Improvement > Components: timelineservice >Affects Versions: 3.3.4 >Reporter: Xianming Lei >Assignee: Xianming Lei >Priority: Minor > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11495) Fix typos in hadoop-yarn-server-web-proxy
[ https://issues.apache.org/jira/browse/YARN-11495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17722329#comment-17722329 ] Ayush Saxena commented on YARN-11495: - Committed to trunk. Thanx [~slfan1989] for the contribution!!! > Fix typos in hadoop-yarn-server-web-proxy > - > > Key: YARN-11495 > URL: https://issues.apache.org/jira/browse/YARN-11495 > Project: Hadoop YARN > Issue Type: Improvement > Components: webapp >Affects Versions: 3.4.0 >Reporter: Shilun Fan >Assignee: Shilun Fan >Priority: Minor > > While reading the code, I found some typo issues. This JIRA will fix these > issues in hadoop-yarn-server-web-proxy. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Resolved] (YARN-11495) Fix typos in hadoop-yarn-server-web-proxy
[ https://issues.apache.org/jira/browse/YARN-11495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena resolved YARN-11495. - Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Fix typos in hadoop-yarn-server-web-proxy > - > > Key: YARN-11495 > URL: https://issues.apache.org/jira/browse/YARN-11495 > Project: Hadoop YARN > Issue Type: Improvement > Components: webapp >Affects Versions: 3.4.0 >Reporter: Shilun Fan >Assignee: Shilun Fan >Priority: Minor > Fix For: 3.4.0 > > > While reading the code, I found some typo issues. This JIRA will fix these > issues in hadoop-yarn-server-web-proxy. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11465) Improved YarnClient Log Format
[ https://issues.apache.org/jira/browse/YARN-11465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17714729#comment-17714729 ] Ayush Saxena commented on YARN-11465: - Added [~shaqyl09099] as Yarn contributor to assign the ticket > Improved YarnClient Log Format > -- > > Key: YARN-11465 > URL: https://issues.apache.org/jira/browse/YARN-11465 > Project: Hadoop YARN > Issue Type: Improvement > Components: client >Affects Versions: 3.4.0 >Reporter: Lu Yuan >Assignee: Lu Yuan >Priority: Minor > Labels: pull-request-available > Fix For: 3.4.0 > > > When reading the YarnClient module code, I found some logging irregularities. > This PR fixes the problem -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-11465) Improved YarnClient Log Format
[ https://issues.apache.org/jira/browse/YARN-11465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena reassigned YARN-11465: --- Assignee: Lu Yuan > Improved YarnClient Log Format > -- > > Key: YARN-11465 > URL: https://issues.apache.org/jira/browse/YARN-11465 > Project: Hadoop YARN > Issue Type: Improvement > Components: client >Affects Versions: 3.4.0 >Reporter: Lu Yuan >Assignee: Lu Yuan >Priority: Minor > Labels: pull-request-available > Fix For: 3.4.0 > > > When reading the YarnClient module code, I found some logging irregularities. > This PR fixes the problem -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-11472) Normalizes log records on the Yarn client
[ https://issues.apache.org/jira/browse/YARN-11472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena updated YARN-11472: Fix Version/s: (was: 3.3.4) > Normalizes log records on the Yarn client > - > > Key: YARN-11472 > URL: https://issues.apache.org/jira/browse/YARN-11472 > Project: Hadoop YARN > Issue Type: Improvement > Components: client >Affects Versions: 3.3.4 >Reporter: Lu Yuan >Priority: Major > Labels: pull-request-available > > Modify the code format based on YARN-11465 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-11472) Normalizes log records on the Yarn client
[ https://issues.apache.org/jira/browse/YARN-11472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena reassigned YARN-11472: --- Assignee: Lu Yuan > Normalizes log records on the Yarn client > - > > Key: YARN-11472 > URL: https://issues.apache.org/jira/browse/YARN-11472 > Project: Hadoop YARN > Issue Type: Improvement > Components: client >Affects Versions: 3.3.4 >Reporter: Lu Yuan >Assignee: Lu Yuan >Priority: Major > Labels: pull-request-available > > Modify the code format based on YARN-11465 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-11404) Add junit5 dependency to hadoop-mapreduce-client-app to fix few unit test failure
[ https://issues.apache.org/jira/browse/YARN-11404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena updated YARN-11404: Fix Version/s: (was: 3.4.0) > Add junit5 dependency to hadoop-mapreduce-client-app to fix few unit test > failure > - > > Key: YARN-11404 > URL: https://issues.apache.org/jira/browse/YARN-11404 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn >Reporter: Susheel Gupta >Assignee: Susheel Gupta >Priority: Major > Labels: pull-request-available > Attachments: > patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt > > > We need to add Junit 5 dependency in > {code:java} > /hadoop/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/pom.xml{code} > as the testcase TestAMWebServicesJobConf, TestAMWebServicesJobs, > TestAMWebServices, TestAMWebServicesAttempts, TestAMWebServicesTasks were > passing locally but failed at jenkins build in this > [link|https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5119/7/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt] > for YARN-5607 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Reopened] (YARN-11404) Add junit5 dependency to hadoop-mapreduce-client-app to fix few unit test failure
[ https://issues.apache.org/jira/browse/YARN-11404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena reopened YARN-11404: - > Add junit5 dependency to hadoop-mapreduce-client-app to fix few unit test > failure > - > > Key: YARN-11404 > URL: https://issues.apache.org/jira/browse/YARN-11404 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn >Reporter: Susheel Gupta >Assignee: Susheel Gupta >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > Attachments: > patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt > > > We need to add Junit 5 dependency in > {code:java} > /hadoop/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/pom.xml{code} > as the testcase TestAMWebServicesJobConf, TestAMWebServicesJobs, > TestAMWebServices, TestAMWebServicesAttempts, TestAMWebServicesTasks were > passing locally but failed at jenkins build in this > [link|https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5119/7/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt] > for YARN-5607 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Resolved] (YARN-11380) Fix hadoop-yarn-api module Java Doc Errors
[ https://issues.apache.org/jira/browse/YARN-11380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena resolved YARN-11380. - Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Fix hadoop-yarn-api module Java Doc Errors > -- > > Key: YARN-11380 > URL: https://issues.apache.org/jira/browse/YARN-11380 > Project: Hadoop YARN > Issue Type: Improvement > Components: yarn >Affects Versions: 3.4.0 >Reporter: Shilun Fan >Assignee: Shilun Fan >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > > When finishing YARN-11373, ps has java-doc compilation errors, I will fix all > java-doc compilation errors of hadoop-yarn-api module. I will pay attention > to the java doc compilation of jdk8 & jdk11 at the same time. > {code:java} > [ERROR] Failed to execute goal > org.apache.maven.plugins:maven-javadoc-plugin:3.0.1:javadoc-no-fork > (default-cli) on project hadoop-yarn-api: An error has occurred in Javadoc > report generation: > [ERROR] Exit code: 1 - javadoc: warning - You have specified the HTML version > as HTML 4.01 by using the -html4 option. > [ERROR] The default is currently HTML5 and the support for HTML 4.01 will be > removed > [ERROR] in a future release. To suppress this warning, please ensure that any > HTML constructs > [ERROR] in your comments are valid in HTML5, and remove the -html4 option. > [ERROR] > /home/jenkins/jenkins-home/workspace/hadoop-multibranch_PR-5146/ubuntu-focal/src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ApplicationBaseProtocol.java:97: > warning: no description for @throws > [ERROR]* @throws YarnException > [ERROR] ^ > [ERROR] > /home/jenkins/jenkins-home/workspace/hadoop-multibranch_PR-5146/ubuntu-focal/src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ApplicationBaseProtocol.java:98: > warning: no description for @throws > [ERROR]* @throws IOException > [ERROR] ^ > [ERROR] > /home/jenkins/jenkins-home/workspace/hadoop-multibranch_PR-5146/ubuntu-focal/src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ApplicationBaseProtocol.java:129: > warning: no description for @throws > [ERROR]* @throws YarnException > [ERROR] ^ > [ERROR] > /home/jenkins/jenkins-home/workspace/hadoop-multibranch_PR-5146/ubuntu-focal/src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ApplicationBaseProtocol.java:130: > warning: no description for @throws > [ERROR]* @throws IOException {code} -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11380) Fix hadoop-yarn-api module Java Doc Errors
[ https://issues.apache.org/jira/browse/YARN-11380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17640033#comment-17640033 ] Ayush Saxena commented on YARN-11380: - Committed to trunk. Thanx [~slfan1989] for the contribution!!! > Fix hadoop-yarn-api module Java Doc Errors > -- > > Key: YARN-11380 > URL: https://issues.apache.org/jira/browse/YARN-11380 > Project: Hadoop YARN > Issue Type: Improvement > Components: yarn >Affects Versions: 3.4.0 >Reporter: Shilun Fan >Assignee: Shilun Fan >Priority: Major > Labels: pull-request-available > > When finishing YARN-11373, ps has java-doc compilation errors, I will fix all > java-doc compilation errors of hadoop-yarn-api module. I will pay attention > to the java doc compilation of jdk8 & jdk11 at the same time. > {code:java} > [ERROR] Failed to execute goal > org.apache.maven.plugins:maven-javadoc-plugin:3.0.1:javadoc-no-fork > (default-cli) on project hadoop-yarn-api: An error has occurred in Javadoc > report generation: > [ERROR] Exit code: 1 - javadoc: warning - You have specified the HTML version > as HTML 4.01 by using the -html4 option. > [ERROR] The default is currently HTML5 and the support for HTML 4.01 will be > removed > [ERROR] in a future release. To suppress this warning, please ensure that any > HTML constructs > [ERROR] in your comments are valid in HTML5, and remove the -html4 option. > [ERROR] > /home/jenkins/jenkins-home/workspace/hadoop-multibranch_PR-5146/ubuntu-focal/src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ApplicationBaseProtocol.java:97: > warning: no description for @throws > [ERROR]* @throws YarnException > [ERROR] ^ > [ERROR] > /home/jenkins/jenkins-home/workspace/hadoop-multibranch_PR-5146/ubuntu-focal/src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ApplicationBaseProtocol.java:98: > warning: no description for @throws > [ERROR]* @throws IOException > [ERROR] ^ > [ERROR] > /home/jenkins/jenkins-home/workspace/hadoop-multibranch_PR-5146/ubuntu-focal/src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ApplicationBaseProtocol.java:129: > warning: no description for @throws > [ERROR]* @throws YarnException > [ERROR] ^ > [ERROR] > /home/jenkins/jenkins-home/workspace/hadoop-multibranch_PR-5146/ubuntu-focal/src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ApplicationBaseProtocol.java:130: > warning: no description for @throws > [ERROR]* @throws IOException {code} -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Resolved] (YARN-11381) Fix hadoop-yarn-common module Java Doc Errors
[ https://issues.apache.org/jira/browse/YARN-11381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena resolved YARN-11381. - Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Fix hadoop-yarn-common module Java Doc Errors > - > > Key: YARN-11381 > URL: https://issues.apache.org/jira/browse/YARN-11381 > Project: Hadoop YARN > Issue Type: Improvement > Components: yarn >Affects Versions: 3.4.0 >Reporter: Shilun Fan >Assignee: Shilun Fan >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > > When finishing YARN-11373, ps has java-doc compilation errors, I will fix > all java-doc compilation errors of hadoop-yarn-common module. I will pay > attention to the java doc compilation of jdk8 & jdk11 at the same time. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11381) Fix hadoop-yarn-common module Java Doc Errors
[ https://issues.apache.org/jira/browse/YARN-11381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17639484#comment-17639484 ] Ayush Saxena commented on YARN-11381: - Committed to trunk. Thanx [~slfan1989] for the contribution!!! > Fix hadoop-yarn-common module Java Doc Errors > - > > Key: YARN-11381 > URL: https://issues.apache.org/jira/browse/YARN-11381 > Project: Hadoop YARN > Issue Type: Improvement > Components: yarn >Affects Versions: 3.4.0 >Reporter: Shilun Fan >Assignee: Shilun Fan >Priority: Major > Labels: pull-request-available > > When finishing YARN-11373, ps has java-doc compilation errors, I will fix > all java-doc compilation errors of hadoop-yarn-common module. I will pay > attention to the java doc compilation of jdk8 & jdk11 at the same time. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Resolved] (YARN-11334) [Federation] Improve SubClusterState#fromString parameter and LogMessage
[ https://issues.apache.org/jira/browse/YARN-11334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena resolved YARN-11334. - Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > [Federation] Improve SubClusterState#fromString parameter and LogMessage > > > Key: YARN-11334 > URL: https://issues.apache.org/jira/browse/YARN-11334 > Project: Hadoop YARN > Issue Type: Improvement > Components: federation >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Trivial > Labels: pull-request-available > Fix For: 3.4.0 > > > The SubClusterState object has a fromString method, which can be improved in > the following 2 places > {code:java} > /** > * Convert a string into {@code SubClusterState}. > * > * @param x the string to convert in SubClusterState > * @return the respective {@code SubClusterState} > */ > public static SubClusterState fromString(String x) { > try { > return SubClusterState.valueOf(x); > } catch (Exception e) { > LOG.error("Invalid SubCluster State value in the StateStore does not" > + " match with the YARN Federation standard."); > return null; > } > } {code} > * The parameter is named x, which cannot well express the meaning of the > input parameter. > * The error log does not print the error value. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11334) [Federation] Improve SubClusterState#fromString parameter and LogMessage
[ https://issues.apache.org/jira/browse/YARN-11334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616804#comment-17616804 ] Ayush Saxena commented on YARN-11334: - Committed to trunk. Thanx [~slfan1989] for the contribution!!! > [Federation] Improve SubClusterState#fromString parameter and LogMessage > > > Key: YARN-11334 > URL: https://issues.apache.org/jira/browse/YARN-11334 > Project: Hadoop YARN > Issue Type: Improvement > Components: federation >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Trivial > Labels: pull-request-available > > The SubClusterState object has a fromString method, which can be improved in > the following 2 places > {code:java} > /** > * Convert a string into {@code SubClusterState}. > * > * @param x the string to convert in SubClusterState > * @return the respective {@code SubClusterState} > */ > public static SubClusterState fromString(String x) { > try { > return SubClusterState.valueOf(x); > } catch (Exception e) { > LOG.error("Invalid SubCluster State value in the StateStore does not" > + " match with the YARN Federation standard."); > return null; > } > } {code} > * The parameter is named x, which cannot well express the meaning of the > input parameter. > * The error log does not print the error value. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-11334) [Federation] Improve SubClusterState#fromString parameter and LogMessage
[ https://issues.apache.org/jira/browse/YARN-11334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena updated YARN-11334: Priority: Trivial (was: Major) > [Federation] Improve SubClusterState#fromString parameter and LogMessage > > > Key: YARN-11334 > URL: https://issues.apache.org/jira/browse/YARN-11334 > Project: Hadoop YARN > Issue Type: Improvement > Components: federation >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Trivial > Labels: pull-request-available > > The SubClusterState object has a fromString method, which can be improved in > the following 2 places > {code:java} > /** > * Convert a string into {@code SubClusterState}. > * > * @param x the string to convert in SubClusterState > * @return the respective {@code SubClusterState} > */ > public static SubClusterState fromString(String x) { > try { > return SubClusterState.valueOf(x); > } catch (Exception e) { > LOG.error("Invalid SubCluster State value in the StateStore does not" > + " match with the YARN Federation standard."); > return null; > } > } {code} > * The parameter is named x, which cannot well express the meaning of the > input parameter. > * The error log does not print the error value. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Resolved] (YARN-11240) Fix incorrect placeholder in yarn-module
[ https://issues.apache.org/jira/browse/YARN-11240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena resolved YARN-11240. - Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Fix incorrect placeholder in yarn-module > > > Key: YARN-11240 > URL: https://issues.apache.org/jira/browse/YARN-11240 > Project: Hadoop YARN > Issue Type: Improvement >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Minor > Labels: pull-request-available > Fix For: 3.4.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Try to deal with the moudle problem at a time. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Resolved] (YARN-11305) Fix TestLogAggregationService#testLocalFileDeletionAfterUpload Failed After YARN-11241(#4703)
[ https://issues.apache.org/jira/browse/YARN-11305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena resolved YARN-11305. - Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed Committed to trunk. Thanx [~slfan1989] for the contribution!!! > Fix TestLogAggregationService#testLocalFileDeletionAfterUpload Failed After > YARN-11241(#4703) > - > > Key: YARN-11305 > URL: https://issues.apache.org/jira/browse/YARN-11305 > Project: Hadoop YARN > Issue Type: Bug > Components: log-aggregation >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > > Yarn's unit tests, with parallel mode enabled > {code:java} > /usr/bin/mvn --batch-mode > -Dmaven.repo.local=/home/jenkins/jenkins-home/workspace/hadoop-multibranch_PR-4846/yetus-m2/hadoop-trunk-patch-0 > -Ptest-patch -Dsurefire.rerunFailingTestsCount=2 -Pparallel-tests > -P!shelltest -Pnative -Drequire.fuse -Drequire.openssl -Drequire.snappy > -Drequire.valgrind -Drequire.zstd -Drequire.test.libhadoop -Pyarn-ui clean > test -fae{code} > Because the automatically generated application_id is the same, > *testLocalFileDeletionAfterUpload* is affected by > *testLocalFileRemainsAfterUploadOnCleanupDisable* > testLocalFileDeletionAfterUpload needs to check that the log of an > application is deleted, and testLocalFileRemainsAfterUploadOnCleanupDisable > regenerates the log of the same application. > Error Message > {code:java} > Directory > [/home/jenkins/jenkins-home/workspace/hadoop-multibranch_PR-4846/ubuntu-focal/src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/TestLogAggregationService-localLogDir/application_1234_0001] > was not deleted {code} > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Resolved] (YARN-11286) Make AsyncDispatcher#printEventDetailsExecutor thread pool parameter configurable
[ https://issues.apache.org/jira/browse/YARN-11286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena resolved YARN-11286. - Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Make AsyncDispatcher#printEventDetailsExecutor thread pool parameter > configurable > - > > Key: YARN-11286 > URL: https://issues.apache.org/jira/browse/YARN-11286 > Project: Hadoop YARN > Issue Type: Improvement > Components: resourcemanager >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Minor > Labels: pull-request-available > Fix For: 3.4.0 > > > AsyncDispatcher#printEventDetailsExecutor thread pool parameters are > hard-coded, extract this part of hard-coded configuration parameters to the > configuration file. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11286) Make AsyncDispatcher#printEventDetailsExecutor thread pool parameter configurable
[ https://issues.apache.org/jira/browse/YARN-11286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17602736#comment-17602736 ] Ayush Saxena commented on YARN-11286: - Committed to trunk. Thanx [~slfan1989] for the contribution!!! > Make AsyncDispatcher#printEventDetailsExecutor thread pool parameter > configurable > - > > Key: YARN-11286 > URL: https://issues.apache.org/jira/browse/YARN-11286 > Project: Hadoop YARN > Issue Type: Improvement > Components: resourcemanager >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Minor > Labels: pull-request-available > > AsyncDispatcher#printEventDetailsExecutor thread pool parameters are > hard-coded, extract this part of hard-coded configuration parameters to the > configuration file. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Resolved] (YARN-11274) Improve Nodemanager#NodeStatusUpdaterImpl Log
[ https://issues.apache.org/jira/browse/YARN-11274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena resolved YARN-11274. - Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Improve Nodemanager#NodeStatusUpdaterImpl Log > - > > Key: YARN-11274 > URL: https://issues.apache.org/jira/browse/YARN-11274 > Project: Hadoop YARN > Issue Type: Improvement > Components: nodemanager >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Minor > Labels: pull-request-available > Fix For: 3.4.0 > > > 1.When RM performs master-slave switching, NM will re-register, report the > list of running apps, and print the number of running apps > 2.registerWithRM#containerReports is enough to print the number of prints, > because the above calling process has already printed the container list, so > there is no need to repeat printing > 3.Use placeholders to display logs to avoid string concatenation -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11274) Improve Nodemanager#NodeStatusUpdaterImpl Log
[ https://issues.apache.org/jira/browse/YARN-11274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17602734#comment-17602734 ] Ayush Saxena commented on YARN-11274: - Committed to trunk. Thanx [~slfan1989] for the contribution!!! > Improve Nodemanager#NodeStatusUpdaterImpl Log > - > > Key: YARN-11274 > URL: https://issues.apache.org/jira/browse/YARN-11274 > Project: Hadoop YARN > Issue Type: Improvement > Components: nodemanager >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Minor > Labels: pull-request-available > > 1.When RM performs master-slave switching, NM will re-register, report the > list of running apps, and print the number of running apps > 2.registerWithRM#containerReports is enough to print the number of prints, > because the above calling process has already printed the container list, so > there is no need to repeat printing > 3.Use placeholders to display logs to avoid string concatenation -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9127) Create more tests to verify GpuDeviceInformationParser
[ https://issues.apache.org/jira/browse/YARN-9127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17598654#comment-17598654 ] Ayush Saxena commented on YARN-9127: [~pbacsko] / [~snemeth] There were some xml added as part of this, seems they create issues in the daily build: {noformat} Reason | Tests XML | Parsing Error(s): | hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-excerpt.xml | hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-missing-tags.xml | hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-missing-tags2.xml | hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-sample-output.xml | hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/fair-scheduler-invalid.xml | hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/yarn-site-with-invalid-allocation-file-ref.xml {noformat} Do you folks have any idea about this? Can check here: [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/965/console] At the very end > Create more tests to verify GpuDeviceInformationParser > -- > > Key: YARN-9127 > URL: https://issues.apache.org/jira/browse/YARN-9127 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Szilard Nemeth >Assignee: Peter Bacsko >Priority: Major > Fix For: 3.3.0, 3.2.1, 3.1.3 > > Attachments: YARN-9127.001.patch, YARN-9127.002.patch, > YARN-9127.003.patch, YARN-9127.004.patch, YARN-9127.005.patch, > YARN-9127.branch-3.1.001.patch, YARN-9127.branch-3.2.001.patch > > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11240) Fix incorrect placeholder in yarn-module
[ https://issues.apache.org/jira/browse/YARN-11240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17584288#comment-17584288 ] Ayush Saxena commented on YARN-11240: - Committed to trunk. Thanx [~slfan1989] for the contribution!!! > Fix incorrect placeholder in yarn-module > > > Key: YARN-11240 > URL: https://issues.apache.org/jira/browse/YARN-11240 > Project: Hadoop YARN > Issue Type: Improvement >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Minor > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > Try to deal with the moudle problem at a time. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9425) Make initialDelay configurable for FederationStateStoreService#scheduledExecutorService
[ https://issues.apache.org/jira/browse/YARN-9425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17582641#comment-17582641 ] Ayush Saxena commented on YARN-9425: Committed to trunk. Thanx [~groot] and [~shenyinjie] for the contribution!!! > Make initialDelay configurable for > FederationStateStoreService#scheduledExecutorService > --- > > Key: YARN-9425 > URL: https://issues.apache.org/jira/browse/YARN-9425 > Project: Hadoop YARN > Issue Type: Bug > Components: federation >Affects Versions: 3.1.0 >Reporter: Shen Yinjie >Assignee: groot >Priority: Major > Labels: pull-request-available > Attachments: YARN-9425_1.patch, YARN-9425_2.patch, YARN-9425_3.patch, > YARN-9425_4.patch > > > When enable YARN federation, subclusters info in Router Web UI cannot be > loaded immediately, and client cannot find any active subclusters after 5mins > by default ,which is configured by > "yarn.federation.state-store.heartbeat-interval-secs". > IMA,we should seperate 'initialDely' and 'delay' for > FederationStateStoreService#scheduledExecutorService. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10793) Upgrade Junit from 4 to 5 in hadoop-yarn-server-applicationhistoryservice
[ https://issues.apache.org/jira/browse/YARN-10793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17582204#comment-17582204 ] Ayush Saxena commented on YARN-10793: - Hi [~aajisaka] , Can you give a check if these failures are related to this patch: [https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/957/testReport/junit/org.apache.hadoop.yarn.server.timeline/TestLevelDBCacheTimelineStore/testGetDomains/] The error seems to be NoClassDef for the import introduced in this patch: {noformat} java.lang.NoClassDefFoundError: org/junit/jupiter/api/Assertions at org.apache.hadoop.yarn.server.timeline.TimelineStoreTestUtils.loadTestEntityData(TimelineStoreTestUtils.java:148){noformat} > Upgrade Junit from 4 to 5 in hadoop-yarn-server-applicationhistoryservice > - > > Key: YARN-10793 > URL: https://issues.apache.org/jira/browse/YARN-10793 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: ANANDA G B >Assignee: groot >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > Time Spent: 1h > Remaining Estimate: 0h > > Upgrade Junit from 4 to 5 in hadoop-yarn-server-applicationhistoryservice -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Resolved] (YARN-11203) Fix typo in hadoop-yarn-server-router module
[ https://issues.apache.org/jira/browse/YARN-11203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena resolved YARN-11203. - Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Fix typo in hadoop-yarn-server-router module > > > Key: YARN-11203 > URL: https://issues.apache.org/jira/browse/YARN-11203 > Project: Hadoop YARN > Issue Type: Sub-task > Components: federation >Affects Versions: 3.4.0, 3.3.4 >Reporter: fanshilun >Assignee: fanshilun >Priority: Minor > Labels: pull-request-available > Fix For: 3.4.0 > > Time Spent: 1h 50m > Remaining Estimate: 0h > > Fix typo in hadoop-yarn-server-router module. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11203) Fix typo in hadoop-yarn-server-router module
[ https://issues.apache.org/jira/browse/YARN-11203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17570318#comment-17570318 ] Ayush Saxena commented on YARN-11203: - Committed to trunk. Thanx [~slfan1989] for the contribution!!! > Fix typo in hadoop-yarn-server-router module > > > Key: YARN-11203 > URL: https://issues.apache.org/jira/browse/YARN-11203 > Project: Hadoop YARN > Issue Type: Sub-task > Components: federation >Affects Versions: 3.4.0, 3.3.4 >Reporter: fanshilun >Assignee: fanshilun >Priority: Minor > Labels: pull-request-available > Time Spent: 1h 50m > Remaining Estimate: 0h > > Fix typo in hadoop-yarn-server-router module. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Resolved] (YARN-11192) TestRouterWebServicesREST failing after YARN-9827
[ https://issues.apache.org/jira/browse/YARN-11192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena resolved YARN-11192. - Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > TestRouterWebServicesREST failing after YARN-9827 > - > > Key: YARN-11192 > URL: https://issues.apache.org/jira/browse/YARN-11192 > Project: Hadoop YARN > Issue Type: Bug > Components: federation >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > Time Spent: 1h > Remaining Estimate: 0h > > In YARN-9827, the following modifications: > {code:java} > GenericExceptionHandler should respond with SERVICE_UNAVAILABLE in case of > connection and service unavailable exception instead of > INTERNAL_SERVICE_ERROR. {code} > This modification caused all of YARN Federation's TestRouterWebServicesREST > unit tests to fail > {code:java} > [ERROR] Tests run: 201, Failures: 15, Errors: 0, Skipped: 0, Flakes: 2 > . > [ERROR] > org.apache.hadoop.yarn.server.router.webapp.TestRouterWebServicesREST.testUpdateAppStateXML(org.apache.hadoop.yarn.server.router.webapp.TestRouterWebServicesREST) > [ERROR] Run 1: TestRouterWebServicesREST.testUpdateAppStateXML:774 > expected:<500> but was:<503> > [ERROR] Run 2: TestRouterWebServicesREST.testUpdateAppStateXML:774 > expected:<500> but was:<503> > [ERROR] Run 3: TestRouterWebServicesREST.testUpdateAppStateXML:774 > expected:<500> but was:<503> {code} > Report-URL: > https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4464/5/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-router.txt -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11192) TestRouterWebServicesREST failing after YARN-9827
[ https://issues.apache.org/jira/browse/YARN-11192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17557872#comment-17557872 ] Ayush Saxena commented on YARN-11192: - Committed to trunk. Thanx [~slfan1989] for the contribution!!! > TestRouterWebServicesREST failing after YARN-9827 > - > > Key: YARN-11192 > URL: https://issues.apache.org/jira/browse/YARN-11192 > Project: Hadoop YARN > Issue Type: Bug > Components: federation >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Major > Labels: pull-request-available > Time Spent: 1h > Remaining Estimate: 0h > > In YARN-9827, the following modifications: > {code:java} > GenericExceptionHandler should respond with SERVICE_UNAVAILABLE in case of > connection and service unavailable exception instead of > INTERNAL_SERVICE_ERROR. {code} > This modification caused all of YARN Federation's TestRouterWebServicesREST > unit tests to fail > {code:java} > [ERROR] Tests run: 201, Failures: 15, Errors: 0, Skipped: 0, Flakes: 2 > . > [ERROR] > org.apache.hadoop.yarn.server.router.webapp.TestRouterWebServicesREST.testUpdateAppStateXML(org.apache.hadoop.yarn.server.router.webapp.TestRouterWebServicesREST) > [ERROR] Run 1: TestRouterWebServicesREST.testUpdateAppStateXML:774 > expected:<500> but was:<503> > [ERROR] Run 2: TestRouterWebServicesREST.testUpdateAppStateXML:774 > expected:<500> but was:<503> > [ERROR] Run 3: TestRouterWebServicesREST.testUpdateAppStateXML:774 > expected:<500> but was:<503> {code} > Report-URL: > https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4464/5/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-router.txt -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11142) Remove unused Imports in Hadoop YARN project
[ https://issues.apache.org/jira/browse/YARN-11142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17544533#comment-17544533 ] Ayush Saxena commented on YARN-11142: - Merged PR to trunk. Thanx [~groot] for the contribution!!! > Remove unused Imports in Hadoop YARN project > > > Key: YARN-11142 > URL: https://issues.apache.org/jira/browse/YARN-11142 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Ashutosh Gupta >Assignee: Ashutosh Gupta >Priority: Minor > Labels: pull-request-available > Time Spent: 50m > Remaining Estimate: 0h > > h3. Optimize Imports to keep code clean > # Remove any unused imports -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Resolved] (YARN-11142) Remove unused Imports in Hadoop YARN project
[ https://issues.apache.org/jira/browse/YARN-11142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena resolved YARN-11142. - Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Remove unused Imports in Hadoop YARN project > > > Key: YARN-11142 > URL: https://issues.apache.org/jira/browse/YARN-11142 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Ashutosh Gupta >Assignee: Ashutosh Gupta >Priority: Minor > Labels: pull-request-available > Fix For: 3.4.0 > > Time Spent: 50m > Remaining Estimate: 0h > > h3. Optimize Imports to keep code clean > # Remove any unused imports -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-11130) RouterClientRMService Has Unused import
[ https://issues.apache.org/jira/browse/YARN-11130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena reassigned YARN-11130: --- Assignee: fanshilun > RouterClientRMService Has Unused import > --- > > Key: YARN-11130 > URL: https://issues.apache.org/jira/browse/YARN-11130 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: fanshilun >Assignee: fanshilun >Priority: Minor > Labels: pull-request-available > Attachments: RouterClientRMService Has Unused Import.png, > YARN-11130.01.patch > > Time Spent: 1.5h > Remaining Estimate: 0h > > RouterClientRMService Has Unused import, keep the code clean and remove > unused. -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11130) RouterClientRMService Has Unused import
[ https://issues.apache.org/jira/browse/YARN-11130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17534706#comment-17534706 ] Ayush Saxena commented on YARN-11130: - Committed to trunk. Thanx [~slfan1989] for the contribution!!! Note: Added [~slfan1989] as Yarn-Contributor-1 to assign this jira > RouterClientRMService Has Unused import > --- > > Key: YARN-11130 > URL: https://issues.apache.org/jira/browse/YARN-11130 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: fanshilun >Assignee: fanshilun >Priority: Minor > Labels: pull-request-available > Attachments: RouterClientRMService Has Unused Import.png, > YARN-11130.01.patch > > Time Spent: 1.5h > Remaining Estimate: 0h > > RouterClientRMService Has Unused import, keep the code clean and remove > unused. -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11026) Make AppPlacementAllocator configurable in AppSchedulingInfo
[ https://issues.apache.org/jira/browse/YARN-11026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17501147#comment-17501147 ] Ayush Saxena commented on YARN-11026: - This breaks TestYarnConfigurationFields#testCompareConfigurationClassAgainstXml {{yarn.scheduler.app-placement-allocator.class}} is added in the YarnConfiguration, but not in {{yarn-default.xml}} Ref: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/798/testReport/org.apache.hadoop.yarn.conf/TestYarnConfigurationFields/testCompareConfigurationClassAgainstXml/ > Make AppPlacementAllocator configurable in AppSchedulingInfo > > > Key: YARN-11026 > URL: https://issues.apache.org/jira/browse/YARN-11026 > Project: Hadoop YARN > Issue Type: Task >Reporter: Minni Mittal >Assignee: Minni Mittal >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > Time Spent: 2h 50m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10788) TestCsiClient fails
[ https://issues.apache.org/jira/browse/YARN-10788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17487495#comment-17487495 ] Ayush Saxena commented on YARN-10788: - >From the error it looks OS specific: {noformat} Caused by: io.netty.channel.unix.Errors$NativeIoException: bind(..) failed: File name too long {noformat} Reproduces in the docker: {noformat} [INFO] --- [INFO] T E S T S [INFO] --- [INFO] Running org.apache.hadoop.yarn.csi.client.TestCsiClient [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 2.041 s <<< FAILURE! - in org.apache.hadoop.yarn.csi.client.TestCsiClient [ERROR] testIdentityService(org.apache.hadoop.yarn.csi.client.TestCsiClient) Time elapsed: 1.701 s <<< ERROR! java.io.IOException: Failed to bind at io.grpc.netty.NettyServer.start(NettyServer.java:257) at io.grpc.internal.ServerImpl.start(ServerImpl.java:184) at io.grpc.internal.ServerImpl.start(ServerImpl.java:90) at org.apache.hadoop.yarn.csi.client.FakeCsiDriver.start(FakeCsiDriver.java:56) at org.apache.hadoop.yarn.csi.client.TestCsiClient.testIdentityService(TestCsiClient.java:72) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365) at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273) at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238) at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159) at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418) Caused by: io.netty.channel.unix.Errors$NativeIoException: bind(..) failed: File name too long {noformat} So, the problem is with the path. I just added a line to set the path to {{/tmp}} to confirm the root cause like: {code:java} public static void setUp() throws IOException { System.setProperty(GenericTestUtils.SYSPROP_TEST_DATA_DIR, "/tmp"); {code} And the test passed, {noformat} [INFO] --- [INFO] T E S T S [INFO] --- [INFO] Running org.apache.hadoop.yarn.csi.client.TestCsiClient [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.982 s - in org.apache.hadoop.yarn.csi.client.TestCsiClient [INFO] [INFO] Results: [INFO] [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0 {noformat} Just tried to find the root cause, Didn't work on finding the best solution, if someone is interested can take this ahead, I wil
[jira] [Commented] (YARN-10540) Node page is broken in YARN UI1 and UI2 including RMWebService api for nodes
[ https://issues.apache.org/jira/browse/YARN-10540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17252338#comment-17252338 ] Ayush Saxena commented on YARN-10540: - I too gave it a try, On Mac the UI seems to be broken : !Mac-Yarn-UI.png! But on Ubuntu-20, everything works fine : !Yarn-UI-Ubuntu.png! Even the http link is working for me, seems a mac only issue > Node page is broken in YARN UI1 and UI2 including RMWebService api for nodes > > > Key: YARN-10540 > URL: https://issues.apache.org/jira/browse/YARN-10540 > Project: Hadoop YARN > Issue Type: Task > Components: webapp >Affects Versions: 3.2.2 >Reporter: Sunil G >Priority: Critical > Attachments: Mac-Yarn-UI.png, Screenshot 2020-12-19 at 11.01.43 > PM.png, Screenshot 2020-12-19 at 11.02.14 PM.png, Yarn-UI-Ubuntu.png, > yarnodes.png, yarnui2onubuntu.png > > > YARN-10450 added changes in NodeInfo class. > Various exceptions are showing while accessing UI2 and UI1 NODE pages. > {code:java} > Caused by: java.lang.NullPointerException > at > org.apache.hadoop.yarn.server.resourcemanager.webapp.dao.NodeInfo.(NodeInfo.java:103) > at > org.apache.hadoop.yarn.server.resourcemanager.webapp.NodesPage$NodesBlock.render(NodesPage.java:164) > at > org.apache.hadoop.yarn.webapp.view.HtmlBlock.render(HtmlBlock.java:69) > at > org.apache.hadoop.yarn.webapp.view.HtmlBlock.renderPartial(HtmlBlock.java:79) > at org.apache.hadoop.yarn.webapp.View.render(View.java:243) > at > org.apache.hadoop.yarn.webapp.view.HtmlPage$Page.subView(HtmlPage.java:49) > at > org.apache.hadoop.yarn.webapp.hamlet2.HamletImpl$EImp._v(HamletImpl.java:117) > at org.apache.hadoop.yarn.webapp.hamlet2.Hamlet$TD.__(Hamlet.java:848) > at > org.apache.hadoop.yarn.webapp.view.TwoColumnLayout.render(TwoColumnLayout.java:71) > at > org.apache.hadoop.yarn.webapp.view.HtmlPage.render(HtmlPage.java:82) > at > org.apache.hadoop.yarn.webapp.Controller.render(Controller.java:216) > at > org.apache.hadoop.yarn.server.resourcemanager.webapp.RmController.nodes(RmController.java:70) > {code} > {code:java} > 2020-12-19 22:55:54,846 WARN > org.apache.hadoop.yarn.webapp.GenericExceptionHandler: INTERNAL_SERVER_ERROR > java.lang.NullPointerException > at > org.apache.hadoop.yarn.server.resourcemanager.webapp.dao.NodeInfo.(NodeInfo.java:103) > at > org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices.getNodes(RMWebServices.java:450) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10540) Node page is broken in YARN UI1 and UI2 including RMWebService api for nodes
[ https://issues.apache.org/jira/browse/YARN-10540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena updated YARN-10540: Attachment: Mac-Yarn-UI.png > Node page is broken in YARN UI1 and UI2 including RMWebService api for nodes > > > Key: YARN-10540 > URL: https://issues.apache.org/jira/browse/YARN-10540 > Project: Hadoop YARN > Issue Type: Task > Components: webapp >Affects Versions: 3.2.2 >Reporter: Sunil G >Priority: Critical > Attachments: Mac-Yarn-UI.png, Screenshot 2020-12-19 at 11.01.43 > PM.png, Screenshot 2020-12-19 at 11.02.14 PM.png, Yarn-UI-Ubuntu.png, > yarnodes.png, yarnui2onubuntu.png > > > YARN-10450 added changes in NodeInfo class. > Various exceptions are showing while accessing UI2 and UI1 NODE pages. > {code:java} > Caused by: java.lang.NullPointerException > at > org.apache.hadoop.yarn.server.resourcemanager.webapp.dao.NodeInfo.(NodeInfo.java:103) > at > org.apache.hadoop.yarn.server.resourcemanager.webapp.NodesPage$NodesBlock.render(NodesPage.java:164) > at > org.apache.hadoop.yarn.webapp.view.HtmlBlock.render(HtmlBlock.java:69) > at > org.apache.hadoop.yarn.webapp.view.HtmlBlock.renderPartial(HtmlBlock.java:79) > at org.apache.hadoop.yarn.webapp.View.render(View.java:243) > at > org.apache.hadoop.yarn.webapp.view.HtmlPage$Page.subView(HtmlPage.java:49) > at > org.apache.hadoop.yarn.webapp.hamlet2.HamletImpl$EImp._v(HamletImpl.java:117) > at org.apache.hadoop.yarn.webapp.hamlet2.Hamlet$TD.__(Hamlet.java:848) > at > org.apache.hadoop.yarn.webapp.view.TwoColumnLayout.render(TwoColumnLayout.java:71) > at > org.apache.hadoop.yarn.webapp.view.HtmlPage.render(HtmlPage.java:82) > at > org.apache.hadoop.yarn.webapp.Controller.render(Controller.java:216) > at > org.apache.hadoop.yarn.server.resourcemanager.webapp.RmController.nodes(RmController.java:70) > {code} > {code:java} > 2020-12-19 22:55:54,846 WARN > org.apache.hadoop.yarn.webapp.GenericExceptionHandler: INTERNAL_SERVER_ERROR > java.lang.NullPointerException > at > org.apache.hadoop.yarn.server.resourcemanager.webapp.dao.NodeInfo.(NodeInfo.java:103) > at > org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices.getNodes(RMWebServices.java:450) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10540) Node page is broken in YARN UI1 and UI2 including RMWebService api for nodes
[ https://issues.apache.org/jira/browse/YARN-10540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena updated YARN-10540: Attachment: Yarn-UI-Ubuntu.png > Node page is broken in YARN UI1 and UI2 including RMWebService api for nodes > > > Key: YARN-10540 > URL: https://issues.apache.org/jira/browse/YARN-10540 > Project: Hadoop YARN > Issue Type: Task > Components: webapp >Affects Versions: 3.2.2 >Reporter: Sunil G >Priority: Critical > Attachments: Mac-Yarn-UI.png, Screenshot 2020-12-19 at 11.01.43 > PM.png, Screenshot 2020-12-19 at 11.02.14 PM.png, Yarn-UI-Ubuntu.png, > yarnodes.png, yarnui2onubuntu.png > > > YARN-10450 added changes in NodeInfo class. > Various exceptions are showing while accessing UI2 and UI1 NODE pages. > {code:java} > Caused by: java.lang.NullPointerException > at > org.apache.hadoop.yarn.server.resourcemanager.webapp.dao.NodeInfo.(NodeInfo.java:103) > at > org.apache.hadoop.yarn.server.resourcemanager.webapp.NodesPage$NodesBlock.render(NodesPage.java:164) > at > org.apache.hadoop.yarn.webapp.view.HtmlBlock.render(HtmlBlock.java:69) > at > org.apache.hadoop.yarn.webapp.view.HtmlBlock.renderPartial(HtmlBlock.java:79) > at org.apache.hadoop.yarn.webapp.View.render(View.java:243) > at > org.apache.hadoop.yarn.webapp.view.HtmlPage$Page.subView(HtmlPage.java:49) > at > org.apache.hadoop.yarn.webapp.hamlet2.HamletImpl$EImp._v(HamletImpl.java:117) > at org.apache.hadoop.yarn.webapp.hamlet2.Hamlet$TD.__(Hamlet.java:848) > at > org.apache.hadoop.yarn.webapp.view.TwoColumnLayout.render(TwoColumnLayout.java:71) > at > org.apache.hadoop.yarn.webapp.view.HtmlPage.render(HtmlPage.java:82) > at > org.apache.hadoop.yarn.webapp.Controller.render(Controller.java:216) > at > org.apache.hadoop.yarn.server.resourcemanager.webapp.RmController.nodes(RmController.java:70) > {code} > {code:java} > 2020-12-19 22:55:54,846 WARN > org.apache.hadoop.yarn.webapp.GenericExceptionHandler: INTERNAL_SERVER_ERROR > java.lang.NullPointerException > at > org.apache.hadoop.yarn.server.resourcemanager.webapp.dao.NodeInfo.(NodeInfo.java:103) > at > org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices.getNodes(RMWebServices.java:450) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10040) DistributedShell test failure on X86 and ARM
[ https://issues.apache.org/jira/browse/YARN-10040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17227548#comment-17227548 ] Ayush Saxena commented on YARN-10040: - [~abmodi] any pointers about this? Is the code only broken or just the test. If the functionality itself has some issue we should consider reverting YARN-9697, else if this is only a test issue, we should wrap this up, if there isn't a fix available we can disable this test for time being. Let me know what is the actual situation. I can try help in whichever way possible. [~aajisaka] [~elgoiri] any pointers about this. Or can you point to someone who has some context about this part of code. > DistributedShell test failure on X86 and ARM > > > Key: YARN-10040 > URL: https://issues.apache.org/jira/browse/YARN-10040 > Project: Hadoop YARN > Issue Type: Bug > Components: applications/distributed-shell > Environment: X86/ARM > OS: ubuntu1804 > Java 8 >Reporter: zhao bo >Assignee: Abhishek Modi >Priority: Major > Attachments: YARN-10040.001.patch > > > * > org.apache.hadoop.yarn.applications.distributedshell.TestDistributedShell.testDSShellWithOpportunisticContainers > * > org.apache.hadoop.yarn.applications.distributedshell.TestDistributedShell.testDSShellWithEnforceExecutionType > Please see the Apache Jenkins Test result: > [https://builds.apache.org/job/hadoop-multibranch/job/PR-1767/1/testReport/] > > These 2 tests are failed on both X86 and ARM platform. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10347) Fix double locking in CapacityScheduler#reinitialize in branch-3.1
[ https://issues.apache.org/jira/browse/YARN-10347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17154062#comment-17154062 ] Ayush Saxena commented on YARN-10347: - hmm, need to see what triggered this at jenkins. Anyway the fix seems pretty straight forward. The original trunk patch didn't had it in YARN-10022, got added during backport. +1, Thanx [~iwasakims] for the find. This should ideally unblock 3.1.4 release as well? cc. [~gabor.bota] > Fix double locking in CapacityScheduler#reinitialize in branch-3.1 > -- > > Key: YARN-10347 > URL: https://issues.apache.org/jira/browse/YARN-10347 > Project: Hadoop YARN > Issue Type: Bug > Components: capacity scheduler >Affects Versions: 3.1.4 >Reporter: Masatake Iwasaki >Assignee: Masatake Iwasaki >Priority: Critical > Attachments: YARN-10347-branch-3.1.001.patch > > > Double locking blocks another threads in ResourceManager waiting for the lock. > I found the issue on testing hadoop-3.1.4-RC2 with RM-HA enabled deployment. > ResourceManager blocks on {{submitApplication}} waiting for the lock when I > run example MR applications. > {noformat} > "IPC Server handler 45 on default port 8032" #211 daemon prio=5 os_prio=0 > tid=0x7f0e45a40200 nid=0x418 waiting on condition [0x7f0e14abe000] >java.lang.Thread.State: WAITING (parking) > at sun.misc.Unsafe.park(Native Method) > - parking to wait for <0x85d56510> (a > java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync) > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireShared(AbstractQueuedSynchronizer.java:967) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(AbstractQueuedSynchronizer.java:1283) > at > java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock(ReentrantReadWriteLock.java:727) > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.checkAndGetApplicationPriority(CapacityScheduler.java:2521) > at > org.apache.hadoop.yarn.server.resourcemanager.RMAppManager.createAndPopulateNewRMApp(RMAppManager.java:417) > at > org.apache.hadoop.yarn.server.resourcemanager.RMAppManager.submitApplication(RMAppManager.java:342) > at > org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.submitApplication(ClientRMService.java:678) > at > org.apache.hadoop.yarn.api.impl.pb.service.ApplicationClientProtocolPBServiceImpl.submitApplication(ApplicationClientProtocolPBServiceImpl.java:277) > at > org.apache.hadoop.yarn.proto.ApplicationClientProtocol$ApplicationClientProtocolService$2.callBlockingMethod(ApplicationClientProtocol.java:563) > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:527) > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1036) > at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:1015) > at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:943) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729) > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2943) > {noformat} > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-10347) Fix double locking in CapacityScheduler#reinitialize in branch-3.1
[ https://issues.apache.org/jira/browse/YARN-10347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17154009#comment-17154009 ] Ayush Saxena edited comment on YARN-10347 at 7/8/20, 8:59 PM: -- Thanx [~iwasakims] for the fix. Changes seems fair enough, but the jenkins doesn't seems behaving as expected. Any idea? was (Author: ayushtkn): Thanx [~iwasakims] for the fix. Changes seems fair enough, but the jenkins doesn't seems behaving expected. Any idea? > Fix double locking in CapacityScheduler#reinitialize in branch-3.1 > -- > > Key: YARN-10347 > URL: https://issues.apache.org/jira/browse/YARN-10347 > Project: Hadoop YARN > Issue Type: Bug > Components: capacity scheduler >Affects Versions: 3.1.4 >Reporter: Masatake Iwasaki >Assignee: Masatake Iwasaki >Priority: Critical > Attachments: YARN-10347-branch-3.1.001.patch > > > Double locking blocks another threads in ResourceManager waiting for the lock. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10347) Fix double locking in CapacityScheduler#reinitialize in branch-3.1
[ https://issues.apache.org/jira/browse/YARN-10347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17154009#comment-17154009 ] Ayush Saxena commented on YARN-10347: - Thanx [~iwasakims] for the fix. Changes seems fair enough, but the jenkins doesn't seems behaving expected. Any idea? > Fix double locking in CapacityScheduler#reinitialize in branch-3.1 > -- > > Key: YARN-10347 > URL: https://issues.apache.org/jira/browse/YARN-10347 > Project: Hadoop YARN > Issue Type: Bug > Components: capacity scheduler >Affects Versions: 3.1.4 >Reporter: Masatake Iwasaki >Assignee: Masatake Iwasaki >Priority: Critical > Attachments: YARN-10347-branch-3.1.001.patch > > > Double locking blocks another threads in ResourceManager waiting for the lock. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10265) Upgrade Netty-all dependency to latest version 4.1.50 to fix ARM support issue
[ https://issues.apache.org/jira/browse/YARN-10265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17107092#comment-17107092 ] Ayush Saxena commented on YARN-10265: - +1 > Upgrade Netty-all dependency to latest version 4.1.50 to fix ARM support issue > -- > > Key: YARN-10265 > URL: https://issues.apache.org/jira/browse/YARN-10265 > Project: Hadoop YARN > Issue Type: Bug >Reporter: liusheng >Priority: Major > Attachments: YARN-10265.001.patch > > > In the issue: https://issues.apache.org/jira/browse/YARN-9898 we have added a > workaround using an non-official released netty-4.1.48 to fix the ARM support > issue. but just a few hours agon, Netty has release the 4.1.50 version which > is officially support ARM platform, please see: > [https://github.com/netty/netty/pull/9804] > > netty-4.1.50.Final release: > [https://github.com/netty/netty/releases/tag/netty-4.1.50.Final] > commits from netty-4.1.48 to netty-4.1.50: > [https://github.com/netty/netty/compare/netty-4.1.48.Final...netty-4.1.50.Final] > So, now it is better to upgrade the netty-dependency version of Hadoop to > 4.1.50 version. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9898) Dependency netty-all-4.1.27.Final doesn't support ARM platform
[ https://issues.apache.org/jira/browse/YARN-9898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17106603#comment-17106603 ] Ayush Saxena commented on YARN-9898: Committed to trunk, branch-3.3 and 3.3.0 Thanx [~seanlau] for the contribution!!! > Dependency netty-all-4.1.27.Final doesn't support ARM platform > -- > > Key: YARN-9898 > URL: https://issues.apache.org/jira/browse/YARN-9898 > Project: Hadoop YARN > Issue Type: Bug >Reporter: liusheng >Assignee: liusheng >Priority: Major > Attachments: YARN-9898.001.patch, YARN-9898.002.patch, > YARN-9898.003.patch, YARN-9898.004.patch > > > Hadoop dependent the Netty package, but the *netty-all-4.1.27.Final* of > io.netty maven repo, cannot support ARM platform. > When run the test *TestCsiClient.testIdentityService* on ARM server, it will > raise error like following: > {code:java} > Caused by: java.io.FileNotFoundException: > META-INF/native/libnetty_transport_native_epoll_aarch_64.so > at > io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:161) > ... 45 more > Suppressed: java.lang.UnsatisfiedLinkError: no > netty_transport_native_epoll_aarch_64 in java.library.path > at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867) > at java.lang.Runtime.loadLibrary0(Runtime.java:870) > at java.lang.System.loadLibrary(System.java:1122) > at > io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38) > at > io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:243) > at > io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:124) > ... 45 more > Suppressed: java.lang.UnsatisfiedLinkError: no > netty_transport_native_epoll_aarch_64 in java.library.path > at > java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867) > at java.lang.Runtime.loadLibrary0(Runtime.java:870) > at java.lang.System.loadLibrary(System.java:1122) > at > io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38) > at > sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > io.netty.util.internal.NativeLibraryLoader$1.run(NativeLibraryLoader.java:263) > at java.security.AccessController.doPrivileged(Native > Method) > at > io.netty.util.internal.NativeLibraryLoader.loadLibraryByHelper(NativeLibraryLoader.java:255) > at > io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:233) > ... 46 more > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9898) Dependency netty-all-4.1.27.Final doesn't support ARM platform
[ https://issues.apache.org/jira/browse/YARN-9898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17105425#comment-17105425 ] Ayush Saxena commented on YARN-9898: Thanx [~seanlau] for the update. The build seems happy now. v004 LGTM +1 if no further comments, will commit by tomorrow EOD. > Dependency netty-all-4.1.27.Final doesn't support ARM platform > -- > > Key: YARN-9898 > URL: https://issues.apache.org/jira/browse/YARN-9898 > Project: Hadoop YARN > Issue Type: Bug >Reporter: liusheng >Assignee: liusheng >Priority: Major > Attachments: YARN-9898.001.patch, YARN-9898.002.patch, > YARN-9898.003.patch, YARN-9898.004.patch > > > Hadoop dependent the Netty package, but the *netty-all-4.1.27.Final* of > io.netty maven repo, cannot support ARM platform. > When run the test *TestCsiClient.testIdentityService* on ARM server, it will > raise error like following: > {code:java} > Caused by: java.io.FileNotFoundException: > META-INF/native/libnetty_transport_native_epoll_aarch_64.so > at > io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:161) > ... 45 more > Suppressed: java.lang.UnsatisfiedLinkError: no > netty_transport_native_epoll_aarch_64 in java.library.path > at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867) > at java.lang.Runtime.loadLibrary0(Runtime.java:870) > at java.lang.System.loadLibrary(System.java:1122) > at > io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38) > at > io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:243) > at > io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:124) > ... 45 more > Suppressed: java.lang.UnsatisfiedLinkError: no > netty_transport_native_epoll_aarch_64 in java.library.path > at > java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867) > at java.lang.Runtime.loadLibrary0(Runtime.java:870) > at java.lang.System.loadLibrary(System.java:1122) > at > io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38) > at > sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > io.netty.util.internal.NativeLibraryLoader$1.run(NativeLibraryLoader.java:263) > at java.security.AccessController.doPrivileged(Native > Method) > at > io.netty.util.internal.NativeLibraryLoader.loadLibraryByHelper(NativeLibraryLoader.java:255) > at > io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:233) > ... 46 more > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9898) Dependency netty-all-4.1.27.Final doesn't support ARM platform
[ https://issues.apache.org/jira/browse/YARN-9898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17103669#comment-17103669 ] Ayush Saxena commented on YARN-9898: Well the second build also crashed with the same errors : Is this related : {code:java} [ERROR] Plugin org.apache.hadoop:hadoop-maven-plugins:3.4.0-SNAPSHOT or one of its dependencies could not be resolved: Failed to read artifact descriptor for org.apache.hadoop:hadoop-maven-plugins:jar:3.4.0-SNAPSHOT: 1 problem was encountered while building the effective model for org.apache.hadoop:hadoop-maven-plugins:3.4.0-SNAPSHOT [ERROR] [ERROR] 'dependencyManagement.dependencies.dependency.groupId' for ${netty4.group}:netty-all:jar with value '${netty4.group}' does not match a valid id pattern. @ {code} https://builds.apache.org/job/PreCommit-YARN-Build/26014/artifact/out/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs.txt The errors are due to dependencies, and doesn't fail at my local. Can you split the HDFS part of changes into a separate HDFS jira? > Dependency netty-all-4.1.27.Final doesn't support ARM platform > -- > > Key: YARN-9898 > URL: https://issues.apache.org/jira/browse/YARN-9898 > Project: Hadoop YARN > Issue Type: Bug >Reporter: liusheng >Assignee: liusheng >Priority: Major > Attachments: YARN-9898.001.patch, YARN-9898.002.patch > > > Hadoop dependent the Netty package, but the *netty-all-4.1.27.Final* of > io.netty maven repo, cannot support ARM platform. > When run the test *TestCsiClient.testIdentityService* on ARM server, it will > raise error like following: > {code:java} > Caused by: java.io.FileNotFoundException: > META-INF/native/libnetty_transport_native_epoll_aarch_64.so > at > io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:161) > ... 45 more > Suppressed: java.lang.UnsatisfiedLinkError: no > netty_transport_native_epoll_aarch_64 in java.library.path > at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867) > at java.lang.Runtime.loadLibrary0(Runtime.java:870) > at java.lang.System.loadLibrary(System.java:1122) > at > io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38) > at > io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:243) > at > io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:124) > ... 45 more > Suppressed: java.lang.UnsatisfiedLinkError: no > netty_transport_native_epoll_aarch_64 in java.library.path > at > java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867) > at java.lang.Runtime.loadLibrary0(Runtime.java:870) > at java.lang.System.loadLibrary(System.java:1122) > at > io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38) > at > sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > io.netty.util.internal.NativeLibraryLoader$1.run(NativeLibraryLoader.java:263) > at java.security.AccessController.doPrivileged(Native > Method) > at > io.netty.util.internal.NativeLibraryLoader.loadLibraryByHelper(NativeLibraryLoader.java:255) > at > io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:233) > ... 46 more > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9898) Dependency netty-all-4.1.27.Final doesn't support ARM platform
[ https://issues.apache.org/jira/browse/YARN-9898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17102750#comment-17102750 ] Ayush Saxena commented on YARN-9898: Thanx [~seanlau] for the patch. I have re triggered the build here, I don't think the failure is related to the patch : https://builds.apache.org/job/PreCommit-YARN-Build/26007/ Have verified the fix in AARCH-64 : {noformat} [INFO] --- [INFO] T E S T S [INFO] --- [INFO] Running org.apache.hadoop.yarn.csi.client.TestCsiClient [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.835 s - in org.apache.hadoop.yarn.csi.client.TestCsiClient [INFO] [INFO] Results: [INFO] [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0 {noformat} {noformat} [INFO] [INFO] Detecting the operating system and CPU architecture [INFO] [INFO] os.detected.name: linux [INFO] os.detected.arch: aarch_64 [INFO] os.detected.version: 4.15 [INFO] os.detected.version.major: 4 [INFO] os.detected.version.minor: 15 [INFO] os.detected.release: ubuntu [INFO] os.detected.release.version: 16.04 [INFO] os.detected.release.like.ubuntu: true [INFO] os.detected.release.like.debian: true [INFO] os.detected.classifier: linux-aarch_64 [INFO] {noformat} [~weichiu] Let me know for any concerns, will hold this up for you a couple of days to have a look. > Dependency netty-all-4.1.27.Final doesn't support ARM platform > -- > > Key: YARN-9898 > URL: https://issues.apache.org/jira/browse/YARN-9898 > Project: Hadoop YARN > Issue Type: Bug >Reporter: liusheng >Assignee: liusheng >Priority: Major > Attachments: YARN-9898.001.patch, YARN-9898.002.patch > > > Hadoop dependent the Netty package, but the *netty-all-4.1.27.Final* of > io.netty maven repo, cannot support ARM platform. > When run the test *TestCsiClient.testIdentityService* on ARM server, it will > raise error like following: > {code:java} > Caused by: java.io.FileNotFoundException: > META-INF/native/libnetty_transport_native_epoll_aarch_64.so > at > io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:161) > ... 45 more > Suppressed: java.lang.UnsatisfiedLinkError: no > netty_transport_native_epoll_aarch_64 in java.library.path > at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867) > at java.lang.Runtime.loadLibrary0(Runtime.java:870) > at java.lang.System.loadLibrary(System.java:1122) > at > io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38) > at > io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:243) > at > io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:124) > ... 45 more > Suppressed: java.lang.UnsatisfiedLinkError: no > netty_transport_native_epoll_aarch_64 in java.library.path > at > java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867) > at java.lang.Runtime.loadLibrary0(Runtime.java:870) > at java.lang.System.loadLibrary(System.java:1122) > at > io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38) > at > sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > io.netty.util.internal.NativeLibraryLoader$1.run(NativeLibraryLoader.java:263) > at java.security.AccessController.doPrivileged(Native > Method) > at > io.netty.util.internal.NativeLibraryLoader.loadLibraryByHelper(NativeLibraryLoader.java:255) > at > io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:233) > ... 46 more > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10040) DistributedShell test failure on X86 and ARM
[ https://issues.apache.org/jira/browse/YARN-10040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17064057#comment-17064057 ] Ayush Saxena commented on YARN-10040: - Seems one test got fixed but {{testDSShellWithEnforceExecutionType}} still stays > DistributedShell test failure on X86 and ARM > > > Key: YARN-10040 > URL: https://issues.apache.org/jira/browse/YARN-10040 > Project: Hadoop YARN > Issue Type: Bug > Components: applications/distributed-shell > Environment: X86/ARM > OS: ubuntu1804 > Java 8 >Reporter: zhao bo >Assignee: Abhishek Modi >Priority: Major > Attachments: YARN-10040.001.patch > > > * > org.apache.hadoop.yarn.applications.distributedshell.TestDistributedShell.testDSShellWithOpportunisticContainers > * > org.apache.hadoop.yarn.applications.distributedshell.TestDistributedShell.testDSShellWithEnforceExecutionType > Please see the Apache Jenkins Test result: > [https://builds.apache.org/job/hadoop-multibranch/job/PR-1767/1/testReport/] > > These 2 tests are failed on both X86 and ARM platform. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9831) NMTokenSecretManagerInRM#createNMToken blocks ApplicationMasterService allocate flow
[ https://issues.apache.org/jira/browse/YARN-9831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17046489#comment-17046489 ] Ayush Saxena commented on YARN-9831: Thanx [~BilwaST] for the patch. On a quick look. Looks good Please fix the checkstyle warnings. [~maniraj...@gmail.com] Give a check, if this is fine with you.. > NMTokenSecretManagerInRM#createNMToken blocks ApplicationMasterService > allocate flow > > > Key: YARN-9831 > URL: https://issues.apache.org/jira/browse/YARN-9831 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Bibin Chundatt >Assignee: Bilwa S T >Priority: Critical > Attachments: YARN-9831.001.patch, YARN-9831.002.patch > > > Currently attempt's NMToken cannot be generated independently. > Each attempts allocate flow blocks each other. We should improve the same -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8982) [Router] Add locality policy
[ https://issues.apache.org/jira/browse/YARN-8982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena updated YARN-8982: --- Fix Version/s: (was: 3.4.0) 3.3.0 > [Router] Add locality policy > - > > Key: YARN-8982 > URL: https://issues.apache.org/jira/browse/YARN-8982 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Giovanni Matteo Fumarola >Assignee: Young Chen >Priority: Major > Fix For: 3.3.0 > > Attachments: YARN-8982.v1.patch, YARN-8982.v2.patch, > YARN-8982.v3.patch, YARN-8982.v4.patch, YARN-8982.v5.patch > > > This jira tracks the effort to add a new policy in the Router. > This policy will allow the Router to pick the SubCluster based on the node > that the client requested. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Moved] (YARN-10146) [Federation] Add missing REST APIs for Router
[ https://issues.apache.org/jira/browse/YARN-10146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena moved HDFS-15179 to YARN-10146: Key: YARN-10146 (was: HDFS-15179) Project: Hadoop YARN (was: Hadoop HDFS) > [Federation] Add missing REST APIs for Router > - > > Key: YARN-10146 > URL: https://issues.apache.org/jira/browse/YARN-10146 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Bilwa S T >Priority: Major > > In FederationInterceptorREST following API are not implemented: > |getClusterInfo| > |getSchedulerInfo| > |dumpSchedulerLogs| > |getActivities| > |getAppActivities| > |getAppStatistics| > |getAppState| > |getNodeToLabels| > |getLabelsToNodes| > |replaceLabelsOnNodes| > |replaceLabelsOnNode| > |getClusterNodeLabels| > |addToClusterNodeLabels| > |removeFromCluserNodeLabels| > |getLabelsOnNode| > |getAppPriority| > |updateApplicationPriority| > |getAppQueue| > |updateAppQueue| > |postDelegationToken| > |postDelegationTokenExpiration| > |cancelDelegationToken| > |createNewReservation| > |submitReservation| > |updateReservation| > |deleteReservation| > |listReservation| > |getAppTimeout| > |getAppTimeouts| > |updateApplicationTimeout| > |getAppAttempts| > |checkUserAccessToQueue| > |getAppAttempt| > |getContainers| > |getContainer| -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9624) Use switch case for ProtoUtils#convertFromProtoFormat containerState
[ https://issues.apache.org/jira/browse/YARN-9624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17026427#comment-17026427 ] Ayush Saxena commented on YARN-9624: +1 > Use switch case for ProtoUtils#convertFromProtoFormat containerState > > > Key: YARN-9624 > URL: https://issues.apache.org/jira/browse/YARN-9624 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Bibin Chundatt >Assignee: Bilwa S T >Priority: Major > Labels: performance > Attachments: YARN-9624.001.patch, YARN-9624.002.patch, > YARN-9624.003.patch > > > On large cluster with 100K+ containers on every heartbeat > {{ContainerState.valueOf(e.name().replace(CONTAINER_STATE_PREFIX, ""))}} will > be too costly. Update with switch case. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10040) DistributedShell test failure on X86 and ARM
[ https://issues.apache.org/jira/browse/YARN-10040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17004554#comment-17004554 ] Ayush Saxena commented on YARN-10040: - Thanx [~bzhaoopenstack] for the report. Seems YARN-9697, could be the reason, I reverted it and the tests passes for me. [~abmodi] [~bibinchundatt] [~elgoiri] you worked on YARN-9697, mind giving a check. > DistributedShell test failure on X86 and ARM > > > Key: YARN-10040 > URL: https://issues.apache.org/jira/browse/YARN-10040 > Project: Hadoop YARN > Issue Type: Bug > Components: applications/distributed-shell > Environment: X86/ARM > OS: ubuntu1804 > Java 8 >Reporter: zhao bo >Priority: Major > > * > org.apache.hadoop.yarn.applications.distributedshell.TestDistributedShell.testDSShellWithOpportunisticContainers > * > org.apache.hadoop.yarn.applications.distributedshell.TestDistributedShell.testDSShellWithEnforceExecutionType > Please see the Apache Jenkins Test result: > [https://builds.apache.org/job/hadoop-multibranch/job/PR-1767/1/testReport/] > > These 2 tests are failed on both X86 and ARM platform. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-10042) Uupgrade grpc-xxx depdencies to 1.26.0
[ https://issues.apache.org/jira/browse/YARN-10042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena reassigned YARN-10042: --- Assignee: liusheng > Uupgrade grpc-xxx depdencies to 1.26.0 > -- > > Key: YARN-10042 > URL: https://issues.apache.org/jira/browse/YARN-10042 > Project: Hadoop YARN > Issue Type: Bug >Reporter: liusheng >Assignee: liusheng >Priority: Major > Fix For: 3.3.0 > > Attachments: YARN-10042.001.patch, > hadoop_build_aarch64_grpc_1.26.0.log, hadoop_build_x86_64_grpc_1.26.0.log, > yarn_csi_tests_aarch64_grpc_1.26.0.log, yarn_csi_tests_x86_64_grpc_1.26.0.log > > > For now, Hadoop YARN use grpc-context, grpc-core, grpc-netty, grpc-protobuf, > grpc-protobuf-lite, grpc-stub and protoc-gen-grpc-java of version 1.15.1, but > the "protoc-gen-grpc-java" cannot support on aarch64 platform. Now the > grpc-java repo has support aarch64 platform and release in 1.26.0 in maven > central. > see: > [https://github.com/grpc/grpc-java/pull/6496] > [https://search.maven.org/search?q=g:io.grpc] > It is better to upgrade the version of grpc-xxx dependencies to 1.26.0 > version. both x86_64 and aarch64 server are building OK accroding to my > testing, please see the attachment, they are: log of building on aarch64, log > of building on x86_64, log of running tests of yarn csi on aarch64, log of > running tests of yarn csi on x86_64. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9985) Unsupported "transitionToObserver" option displaying for rmadmin command
[ https://issues.apache.org/jira/browse/YARN-9985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16986089#comment-16986089 ] Ayush Saxena commented on YARN-9985: [~aajisaka] [~tasanuma] can you help review. > Unsupported "transitionToObserver" option displaying for rmadmin command > > > Key: YARN-9985 > URL: https://issues.apache.org/jira/browse/YARN-9985 > Project: Hadoop YARN > Issue Type: Bug > Components: RM, yarn >Affects Versions: 3.2.1 >Reporter: Souryakanta Dwivedy >Assignee: Ayush Saxena >Priority: Minor > Attachments: YARN-9985-01.patch, YARN-9985-02.patch, > image-2019-11-18-18-31-17-755.png, image-2019-11-18-18-35-54-688.png > > > Unsupported "transitionToObserver" option displaying for rmadmin command > Check the options for Yarn rmadmin command > It will display the "-transitionToObserver " option which is not > supported > by yarn rmadmin command which is wrong behavior. > But if you check the yarn rmadmin -help it will not display any option > "-transitionToObserver " > > !image-2019-11-18-18-31-17-755.png! > > == > install/hadoop/resourcemanager/bin> ./yarn rmadmin -help > rmadmin is the command to execute YARN administrative commands. > The full syntax is: > yarn rmadmin [-refreshQueues] [-refreshNodes [-g|graceful [timeout in > seconds] -client|server]] [-refreshNodesResources] > [-refreshSuperUserGroupsConfiguration] [-refreshUserToGroupsMappings] > [-refreshAdminAcls] [-refreshServiceAcl] [-getGroup [username]] > [-addToClusterNodeLabels > <"label1(exclusive=true),label2(exclusive=false),label3">] > [-removeFromClusterNodeLabels ] [-replaceLabelsOnNode > <"node1[:port]=label1,label2 node2[:port]=label1"> [-failOnUnknownNodes]] > [-directlyAccessNodeLabelStore] [-refreshClusterMaxPriority] > [-updateNodeResource [NodeID] [MemSize] [vCores] ([OvercommitTimeout]) or > -updateNodeResource [NodeID] [ResourceTypes] ([OvercommitTimeout])] > *{color:#FF}[-transitionToActive [--forceactive] ]{color} > {color:#FF}[-transitionToStandby ]{color}* [-getServiceState > ] [-getAllServiceState] [-checkHealth ] [-help [cmd]] > -refreshQueues: Reload the queues' acls, states and scheduler specific > properties. > ResourceManager will reload the mapred-queues configuration file. > -refreshNodes [-g|graceful [timeout in seconds] -client|server]: Refresh the > hosts information at the ResourceManager. Here [-g|graceful [timeout in > seconds] -client|server] is optional, if we specify the timeout then > ResourceManager will wait for timeout before marking the NodeManager as > decommissioned. The -client|server indicates if the timeout tracking should > be handled by the client or the ResourceManager. The client-side tracking is > blocking, while the server-side tracking is not. Omitting the timeout, or a > timeout of -1, indicates an infinite timeout. Known Issue: the server-side > tracking will immediately decommission if an RM HA failover occurs. > -refreshNodesResources: Refresh resources of NodeManagers at the > ResourceManager. > -refreshSuperUserGroupsConfiguration: Refresh superuser proxy groups mappings > -refreshUserToGroupsMappings: Refresh user-to-groups mappings > -refreshAdminAcls: Refresh acls for administration of ResourceManager > -refreshServiceAcl: Reload the service-level authorization policy file. > ResourceManager will reload the authorization policy file. > -getGroups [username]: Get the groups which given user belongs to. > -addToClusterNodeLabels > <"label1(exclusive=true),label2(exclusive=false),label3">: add to cluster > node labels. Default exclusivity is true > -removeFromClusterNodeLabels (label splitted by ","): > remove from cluster node labels > -replaceLabelsOnNode <"node1[:port]=label1,label2 > node2[:port]=label1,label2"> [-failOnUnknownNodes] : replace labels on nodes > (please note that we do not support specifying multiple labels on a single > host for now.) > [-failOnUnknownNodes] is optional, when we set this option, it will fail if > specified nodes are unknown. > -directlyAccessNodeLabelStore: This is DEPRECATED, will be removed in future > releases. Directly access node label store, with this option, all node label > related operations will not connect RM. Instead, they will access/modify > stored node labels directly. By default, it is false (access via RM). AND > PLEASE NOTE: if you configured yarn.node-labels.fs-store.root-dir to a local > directory (instead of NFS or HDFS), this option will only work when the > command run on the machine where RM is running. > -refreshClusterMaxPriority: Refresh cluster max priority > -updateNodeResource [NodeID] [MemSize] [vCores] ([OvercommitTimeout]) > or > [Node
[jira] [Commented] (YARN-9985) Unsupported "transitionToObserver" option displaying for rmadmin command
[ https://issues.apache.org/jira/browse/YARN-9985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16984999#comment-16984999 ] Ayush Saxena commented on YARN-9985: Found Failover too mentioned in two places along with transitionToObserver, Removed this as that too wasn't supposed to be there as per YARN-3397 > Unsupported "transitionToObserver" option displaying for rmadmin command > > > Key: YARN-9985 > URL: https://issues.apache.org/jira/browse/YARN-9985 > Project: Hadoop YARN > Issue Type: Bug > Components: RM, yarn >Affects Versions: 3.2.1 >Reporter: Souryakanta Dwivedy >Assignee: Ayush Saxena >Priority: Minor > Attachments: YARN-9985-01.patch, YARN-9985-02.patch, > image-2019-11-18-18-31-17-755.png, image-2019-11-18-18-35-54-688.png > > > Unsupported "transitionToObserver" option displaying for rmadmin command > Check the options for Yarn rmadmin command > It will display the "-transitionToObserver " option which is not > supported > by yarn rmadmin command which is wrong behavior. > But if you check the yarn rmadmin -help it will not display any option > "-transitionToObserver " > > !image-2019-11-18-18-31-17-755.png! > > == > install/hadoop/resourcemanager/bin> ./yarn rmadmin -help > rmadmin is the command to execute YARN administrative commands. > The full syntax is: > yarn rmadmin [-refreshQueues] [-refreshNodes [-g|graceful [timeout in > seconds] -client|server]] [-refreshNodesResources] > [-refreshSuperUserGroupsConfiguration] [-refreshUserToGroupsMappings] > [-refreshAdminAcls] [-refreshServiceAcl] [-getGroup [username]] > [-addToClusterNodeLabels > <"label1(exclusive=true),label2(exclusive=false),label3">] > [-removeFromClusterNodeLabels ] [-replaceLabelsOnNode > <"node1[:port]=label1,label2 node2[:port]=label1"> [-failOnUnknownNodes]] > [-directlyAccessNodeLabelStore] [-refreshClusterMaxPriority] > [-updateNodeResource [NodeID] [MemSize] [vCores] ([OvercommitTimeout]) or > -updateNodeResource [NodeID] [ResourceTypes] ([OvercommitTimeout])] > *{color:#FF}[-transitionToActive [--forceactive] ]{color} > {color:#FF}[-transitionToStandby ]{color}* [-getServiceState > ] [-getAllServiceState] [-checkHealth ] [-help [cmd]] > -refreshQueues: Reload the queues' acls, states and scheduler specific > properties. > ResourceManager will reload the mapred-queues configuration file. > -refreshNodes [-g|graceful [timeout in seconds] -client|server]: Refresh the > hosts information at the ResourceManager. Here [-g|graceful [timeout in > seconds] -client|server] is optional, if we specify the timeout then > ResourceManager will wait for timeout before marking the NodeManager as > decommissioned. The -client|server indicates if the timeout tracking should > be handled by the client or the ResourceManager. The client-side tracking is > blocking, while the server-side tracking is not. Omitting the timeout, or a > timeout of -1, indicates an infinite timeout. Known Issue: the server-side > tracking will immediately decommission if an RM HA failover occurs. > -refreshNodesResources: Refresh resources of NodeManagers at the > ResourceManager. > -refreshSuperUserGroupsConfiguration: Refresh superuser proxy groups mappings > -refreshUserToGroupsMappings: Refresh user-to-groups mappings > -refreshAdminAcls: Refresh acls for administration of ResourceManager > -refreshServiceAcl: Reload the service-level authorization policy file. > ResourceManager will reload the authorization policy file. > -getGroups [username]: Get the groups which given user belongs to. > -addToClusterNodeLabels > <"label1(exclusive=true),label2(exclusive=false),label3">: add to cluster > node labels. Default exclusivity is true > -removeFromClusterNodeLabels (label splitted by ","): > remove from cluster node labels > -replaceLabelsOnNode <"node1[:port]=label1,label2 > node2[:port]=label1,label2"> [-failOnUnknownNodes] : replace labels on nodes > (please note that we do not support specifying multiple labels on a single > host for now.) > [-failOnUnknownNodes] is optional, when we set this option, it will fail if > specified nodes are unknown. > -directlyAccessNodeLabelStore: This is DEPRECATED, will be removed in future > releases. Directly access node label store, with this option, all node label > related operations will not connect RM. Instead, they will access/modify > stored node labels directly. By default, it is false (access via RM). AND > PLEASE NOTE: if you configured yarn.node-labels.fs-store.root-dir to a local > directory (instead of NFS or HDFS), this option will only work when the > command run on the machine where RM is running. > -refreshClusterMaxPriority: Refresh cluste
[jira] [Updated] (YARN-9985) Unsupported "transitionToObserver" option displaying for rmadmin command
[ https://issues.apache.org/jira/browse/YARN-9985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena updated YARN-9985: --- Attachment: YARN-9985-02.patch > Unsupported "transitionToObserver" option displaying for rmadmin command > > > Key: YARN-9985 > URL: https://issues.apache.org/jira/browse/YARN-9985 > Project: Hadoop YARN > Issue Type: Bug > Components: RM, yarn >Affects Versions: 3.2.1 >Reporter: Souryakanta Dwivedy >Assignee: Ayush Saxena >Priority: Minor > Attachments: YARN-9985-01.patch, YARN-9985-02.patch, > image-2019-11-18-18-31-17-755.png, image-2019-11-18-18-35-54-688.png > > > Unsupported "transitionToObserver" option displaying for rmadmin command > Check the options for Yarn rmadmin command > It will display the "-transitionToObserver " option which is not > supported > by yarn rmadmin command which is wrong behavior. > But if you check the yarn rmadmin -help it will not display any option > "-transitionToObserver " > > !image-2019-11-18-18-31-17-755.png! > > == > install/hadoop/resourcemanager/bin> ./yarn rmadmin -help > rmadmin is the command to execute YARN administrative commands. > The full syntax is: > yarn rmadmin [-refreshQueues] [-refreshNodes [-g|graceful [timeout in > seconds] -client|server]] [-refreshNodesResources] > [-refreshSuperUserGroupsConfiguration] [-refreshUserToGroupsMappings] > [-refreshAdminAcls] [-refreshServiceAcl] [-getGroup [username]] > [-addToClusterNodeLabels > <"label1(exclusive=true),label2(exclusive=false),label3">] > [-removeFromClusterNodeLabels ] [-replaceLabelsOnNode > <"node1[:port]=label1,label2 node2[:port]=label1"> [-failOnUnknownNodes]] > [-directlyAccessNodeLabelStore] [-refreshClusterMaxPriority] > [-updateNodeResource [NodeID] [MemSize] [vCores] ([OvercommitTimeout]) or > -updateNodeResource [NodeID] [ResourceTypes] ([OvercommitTimeout])] > *{color:#FF}[-transitionToActive [--forceactive] ]{color} > {color:#FF}[-transitionToStandby ]{color}* [-getServiceState > ] [-getAllServiceState] [-checkHealth ] [-help [cmd]] > -refreshQueues: Reload the queues' acls, states and scheduler specific > properties. > ResourceManager will reload the mapred-queues configuration file. > -refreshNodes [-g|graceful [timeout in seconds] -client|server]: Refresh the > hosts information at the ResourceManager. Here [-g|graceful [timeout in > seconds] -client|server] is optional, if we specify the timeout then > ResourceManager will wait for timeout before marking the NodeManager as > decommissioned. The -client|server indicates if the timeout tracking should > be handled by the client or the ResourceManager. The client-side tracking is > blocking, while the server-side tracking is not. Omitting the timeout, or a > timeout of -1, indicates an infinite timeout. Known Issue: the server-side > tracking will immediately decommission if an RM HA failover occurs. > -refreshNodesResources: Refresh resources of NodeManagers at the > ResourceManager. > -refreshSuperUserGroupsConfiguration: Refresh superuser proxy groups mappings > -refreshUserToGroupsMappings: Refresh user-to-groups mappings > -refreshAdminAcls: Refresh acls for administration of ResourceManager > -refreshServiceAcl: Reload the service-level authorization policy file. > ResourceManager will reload the authorization policy file. > -getGroups [username]: Get the groups which given user belongs to. > -addToClusterNodeLabels > <"label1(exclusive=true),label2(exclusive=false),label3">: add to cluster > node labels. Default exclusivity is true > -removeFromClusterNodeLabels (label splitted by ","): > remove from cluster node labels > -replaceLabelsOnNode <"node1[:port]=label1,label2 > node2[:port]=label1,label2"> [-failOnUnknownNodes] : replace labels on nodes > (please note that we do not support specifying multiple labels on a single > host for now.) > [-failOnUnknownNodes] is optional, when we set this option, it will fail if > specified nodes are unknown. > -directlyAccessNodeLabelStore: This is DEPRECATED, will be removed in future > releases. Directly access node label store, with this option, all node label > related operations will not connect RM. Instead, they will access/modify > stored node labels directly. By default, it is false (access via RM). AND > PLEASE NOTE: if you configured yarn.node-labels.fs-store.root-dir to a local > directory (instead of NFS or HDFS), this option will only work when the > command run on the machine where RM is running. > -refreshClusterMaxPriority: Refresh cluster max priority > -updateNodeResource [NodeID] [MemSize] [vCores] ([OvercommitTimeout]) > or > [NodeID] [resourcetypes] ([OvercommitTimeout]). : Update resource on > s
[jira] [Updated] (YARN-9985) Unsupported "transitionToObserver" option displaying for rmadmin command
[ https://issues.apache.org/jira/browse/YARN-9985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena updated YARN-9985: --- Summary: Unsupported "transitionToObserver" option displaying for rmadmin command (was: YARN: Unsupported "transitionToObserver" option displaying for rmadmin command) > Unsupported "transitionToObserver" option displaying for rmadmin command > > > Key: YARN-9985 > URL: https://issues.apache.org/jira/browse/YARN-9985 > Project: Hadoop YARN > Issue Type: Bug > Components: RM, yarn >Affects Versions: 3.2.1 >Reporter: Souryakanta Dwivedy >Assignee: Ayush Saxena >Priority: Minor > Attachments: YARN-9985-01.patch, image-2019-11-18-18-31-17-755.png, > image-2019-11-18-18-35-54-688.png > > > Unsupported "transitionToObserver" option displaying for rmadmin command > Check the options for Yarn rmadmin command > It will display the "-transitionToObserver " option which is not > supported > by yarn rmadmin command which is wrong behavior. > But if you check the yarn rmadmin -help it will not display any option > "-transitionToObserver " > > !image-2019-11-18-18-31-17-755.png! > > == > install/hadoop/resourcemanager/bin> ./yarn rmadmin -help > rmadmin is the command to execute YARN administrative commands. > The full syntax is: > yarn rmadmin [-refreshQueues] [-refreshNodes [-g|graceful [timeout in > seconds] -client|server]] [-refreshNodesResources] > [-refreshSuperUserGroupsConfiguration] [-refreshUserToGroupsMappings] > [-refreshAdminAcls] [-refreshServiceAcl] [-getGroup [username]] > [-addToClusterNodeLabels > <"label1(exclusive=true),label2(exclusive=false),label3">] > [-removeFromClusterNodeLabels ] [-replaceLabelsOnNode > <"node1[:port]=label1,label2 node2[:port]=label1"> [-failOnUnknownNodes]] > [-directlyAccessNodeLabelStore] [-refreshClusterMaxPriority] > [-updateNodeResource [NodeID] [MemSize] [vCores] ([OvercommitTimeout]) or > -updateNodeResource [NodeID] [ResourceTypes] ([OvercommitTimeout])] > *{color:#FF}[-transitionToActive [--forceactive] ]{color} > {color:#FF}[-transitionToStandby ]{color}* [-getServiceState > ] [-getAllServiceState] [-checkHealth ] [-help [cmd]] > -refreshQueues: Reload the queues' acls, states and scheduler specific > properties. > ResourceManager will reload the mapred-queues configuration file. > -refreshNodes [-g|graceful [timeout in seconds] -client|server]: Refresh the > hosts information at the ResourceManager. Here [-g|graceful [timeout in > seconds] -client|server] is optional, if we specify the timeout then > ResourceManager will wait for timeout before marking the NodeManager as > decommissioned. The -client|server indicates if the timeout tracking should > be handled by the client or the ResourceManager. The client-side tracking is > blocking, while the server-side tracking is not. Omitting the timeout, or a > timeout of -1, indicates an infinite timeout. Known Issue: the server-side > tracking will immediately decommission if an RM HA failover occurs. > -refreshNodesResources: Refresh resources of NodeManagers at the > ResourceManager. > -refreshSuperUserGroupsConfiguration: Refresh superuser proxy groups mappings > -refreshUserToGroupsMappings: Refresh user-to-groups mappings > -refreshAdminAcls: Refresh acls for administration of ResourceManager > -refreshServiceAcl: Reload the service-level authorization policy file. > ResourceManager will reload the authorization policy file. > -getGroups [username]: Get the groups which given user belongs to. > -addToClusterNodeLabels > <"label1(exclusive=true),label2(exclusive=false),label3">: add to cluster > node labels. Default exclusivity is true > -removeFromClusterNodeLabels (label splitted by ","): > remove from cluster node labels > -replaceLabelsOnNode <"node1[:port]=label1,label2 > node2[:port]=label1,label2"> [-failOnUnknownNodes] : replace labels on nodes > (please note that we do not support specifying multiple labels on a single > host for now.) > [-failOnUnknownNodes] is optional, when we set this option, it will fail if > specified nodes are unknown. > -directlyAccessNodeLabelStore: This is DEPRECATED, will be removed in future > releases. Directly access node label store, with this option, all node label > related operations will not connect RM. Instead, they will access/modify > stored node labels directly. By default, it is false (access via RM). AND > PLEASE NOTE: if you configured yarn.node-labels.fs-store.root-dir to a local > directory (instead of NFS or HDFS), this option will only work when the > command run on the machine where RM is running. > -refreshClusterMaxPriority: Refresh cluster max priority > -updateNodeResource [NodeID] [Me
[jira] [Commented] (YARN-9985) Unsupported "transitionToObserver" option displaying for rmadmin command
[ https://issues.apache.org/jira/browse/YARN-9985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16984617#comment-16984617 ] Ayush Saxena commented on YARN-9985: Thanx [~SouryakantaDwivedy] for the report. Uploaded patch with the fix, Pls Review!!! > Unsupported "transitionToObserver" option displaying for rmadmin command > > > Key: YARN-9985 > URL: https://issues.apache.org/jira/browse/YARN-9985 > Project: Hadoop YARN > Issue Type: Bug > Components: RM, yarn >Affects Versions: 3.2.1 >Reporter: Souryakanta Dwivedy >Assignee: Ayush Saxena >Priority: Minor > Attachments: YARN-9985-01.patch, image-2019-11-18-18-31-17-755.png, > image-2019-11-18-18-35-54-688.png > > > Unsupported "transitionToObserver" option displaying for rmadmin command > Check the options for Yarn rmadmin command > It will display the "-transitionToObserver " option which is not > supported > by yarn rmadmin command which is wrong behavior. > But if you check the yarn rmadmin -help it will not display any option > "-transitionToObserver " > > !image-2019-11-18-18-31-17-755.png! > > == > install/hadoop/resourcemanager/bin> ./yarn rmadmin -help > rmadmin is the command to execute YARN administrative commands. > The full syntax is: > yarn rmadmin [-refreshQueues] [-refreshNodes [-g|graceful [timeout in > seconds] -client|server]] [-refreshNodesResources] > [-refreshSuperUserGroupsConfiguration] [-refreshUserToGroupsMappings] > [-refreshAdminAcls] [-refreshServiceAcl] [-getGroup [username]] > [-addToClusterNodeLabels > <"label1(exclusive=true),label2(exclusive=false),label3">] > [-removeFromClusterNodeLabels ] [-replaceLabelsOnNode > <"node1[:port]=label1,label2 node2[:port]=label1"> [-failOnUnknownNodes]] > [-directlyAccessNodeLabelStore] [-refreshClusterMaxPriority] > [-updateNodeResource [NodeID] [MemSize] [vCores] ([OvercommitTimeout]) or > -updateNodeResource [NodeID] [ResourceTypes] ([OvercommitTimeout])] > *{color:#FF}[-transitionToActive [--forceactive] ]{color} > {color:#FF}[-transitionToStandby ]{color}* [-getServiceState > ] [-getAllServiceState] [-checkHealth ] [-help [cmd]] > -refreshQueues: Reload the queues' acls, states and scheduler specific > properties. > ResourceManager will reload the mapred-queues configuration file. > -refreshNodes [-g|graceful [timeout in seconds] -client|server]: Refresh the > hosts information at the ResourceManager. Here [-g|graceful [timeout in > seconds] -client|server] is optional, if we specify the timeout then > ResourceManager will wait for timeout before marking the NodeManager as > decommissioned. The -client|server indicates if the timeout tracking should > be handled by the client or the ResourceManager. The client-side tracking is > blocking, while the server-side tracking is not. Omitting the timeout, or a > timeout of -1, indicates an infinite timeout. Known Issue: the server-side > tracking will immediately decommission if an RM HA failover occurs. > -refreshNodesResources: Refresh resources of NodeManagers at the > ResourceManager. > -refreshSuperUserGroupsConfiguration: Refresh superuser proxy groups mappings > -refreshUserToGroupsMappings: Refresh user-to-groups mappings > -refreshAdminAcls: Refresh acls for administration of ResourceManager > -refreshServiceAcl: Reload the service-level authorization policy file. > ResourceManager will reload the authorization policy file. > -getGroups [username]: Get the groups which given user belongs to. > -addToClusterNodeLabels > <"label1(exclusive=true),label2(exclusive=false),label3">: add to cluster > node labels. Default exclusivity is true > -removeFromClusterNodeLabels (label splitted by ","): > remove from cluster node labels > -replaceLabelsOnNode <"node1[:port]=label1,label2 > node2[:port]=label1,label2"> [-failOnUnknownNodes] : replace labels on nodes > (please note that we do not support specifying multiple labels on a single > host for now.) > [-failOnUnknownNodes] is optional, when we set this option, it will fail if > specified nodes are unknown. > -directlyAccessNodeLabelStore: This is DEPRECATED, will be removed in future > releases. Directly access node label store, with this option, all node label > related operations will not connect RM. Instead, they will access/modify > stored node labels directly. By default, it is false (access via RM). AND > PLEASE NOTE: if you configured yarn.node-labels.fs-store.root-dir to a local > directory (instead of NFS or HDFS), this option will only work when the > command run on the machine where RM is running. > -refreshClusterMaxPriority: Refresh cluster max priority > -updateNodeResource [NodeID] [MemSize] [vCores] ([OvercommitTi
[jira] [Updated] (YARN-9985) YARN: Unsupported "transitionToObserver" option displaying for rmadmin command
[ https://issues.apache.org/jira/browse/YARN-9985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena updated YARN-9985: --- Attachment: YARN-9985-01.patch > YARN: Unsupported "transitionToObserver" option displaying for rmadmin command > -- > > Key: YARN-9985 > URL: https://issues.apache.org/jira/browse/YARN-9985 > Project: Hadoop YARN > Issue Type: Bug > Components: RM, yarn >Affects Versions: 3.2.1 >Reporter: Souryakanta Dwivedy >Priority: Minor > Attachments: YARN-9985-01.patch, image-2019-11-18-18-31-17-755.png, > image-2019-11-18-18-35-54-688.png > > > Unsupported "transitionToObserver" option displaying for rmadmin command > Check the options for Yarn rmadmin command > It will display the "-transitionToObserver " option which is not > supported > by yarn rmadmin command which is wrong behavior. > But if you check the yarn rmadmin -help it will not display any option > "-transitionToObserver " > > !image-2019-11-18-18-31-17-755.png! > > == > install/hadoop/resourcemanager/bin> ./yarn rmadmin -help > rmadmin is the command to execute YARN administrative commands. > The full syntax is: > yarn rmadmin [-refreshQueues] [-refreshNodes [-g|graceful [timeout in > seconds] -client|server]] [-refreshNodesResources] > [-refreshSuperUserGroupsConfiguration] [-refreshUserToGroupsMappings] > [-refreshAdminAcls] [-refreshServiceAcl] [-getGroup [username]] > [-addToClusterNodeLabels > <"label1(exclusive=true),label2(exclusive=false),label3">] > [-removeFromClusterNodeLabels ] [-replaceLabelsOnNode > <"node1[:port]=label1,label2 node2[:port]=label1"> [-failOnUnknownNodes]] > [-directlyAccessNodeLabelStore] [-refreshClusterMaxPriority] > [-updateNodeResource [NodeID] [MemSize] [vCores] ([OvercommitTimeout]) or > -updateNodeResource [NodeID] [ResourceTypes] ([OvercommitTimeout])] > *{color:#FF}[-transitionToActive [--forceactive] ]{color} > {color:#FF}[-transitionToStandby ]{color}* [-getServiceState > ] [-getAllServiceState] [-checkHealth ] [-help [cmd]] > -refreshQueues: Reload the queues' acls, states and scheduler specific > properties. > ResourceManager will reload the mapred-queues configuration file. > -refreshNodes [-g|graceful [timeout in seconds] -client|server]: Refresh the > hosts information at the ResourceManager. Here [-g|graceful [timeout in > seconds] -client|server] is optional, if we specify the timeout then > ResourceManager will wait for timeout before marking the NodeManager as > decommissioned. The -client|server indicates if the timeout tracking should > be handled by the client or the ResourceManager. The client-side tracking is > blocking, while the server-side tracking is not. Omitting the timeout, or a > timeout of -1, indicates an infinite timeout. Known Issue: the server-side > tracking will immediately decommission if an RM HA failover occurs. > -refreshNodesResources: Refresh resources of NodeManagers at the > ResourceManager. > -refreshSuperUserGroupsConfiguration: Refresh superuser proxy groups mappings > -refreshUserToGroupsMappings: Refresh user-to-groups mappings > -refreshAdminAcls: Refresh acls for administration of ResourceManager > -refreshServiceAcl: Reload the service-level authorization policy file. > ResourceManager will reload the authorization policy file. > -getGroups [username]: Get the groups which given user belongs to. > -addToClusterNodeLabels > <"label1(exclusive=true),label2(exclusive=false),label3">: add to cluster > node labels. Default exclusivity is true > -removeFromClusterNodeLabels (label splitted by ","): > remove from cluster node labels > -replaceLabelsOnNode <"node1[:port]=label1,label2 > node2[:port]=label1,label2"> [-failOnUnknownNodes] : replace labels on nodes > (please note that we do not support specifying multiple labels on a single > host for now.) > [-failOnUnknownNodes] is optional, when we set this option, it will fail if > specified nodes are unknown. > -directlyAccessNodeLabelStore: This is DEPRECATED, will be removed in future > releases. Directly access node label store, with this option, all node label > related operations will not connect RM. Instead, they will access/modify > stored node labels directly. By default, it is false (access via RM). AND > PLEASE NOTE: if you configured yarn.node-labels.fs-store.root-dir to a local > directory (instead of NFS or HDFS), this option will only work when the > command run on the machine where RM is running. > -refreshClusterMaxPriority: Refresh cluster max priority > -updateNodeResource [NodeID] [MemSize] [vCores] ([OvercommitTimeout]) > or > [NodeID] [resourcetypes] ([OvercommitTimeout]). : Update resource on > specific node. > -transitionToActive [--forc
[jira] [Assigned] (YARN-9985) YARN: Unsupported "transitionToObserver" option displaying for rmadmin command
[ https://issues.apache.org/jira/browse/YARN-9985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena reassigned YARN-9985: -- Assignee: Ayush Saxena > YARN: Unsupported "transitionToObserver" option displaying for rmadmin command > -- > > Key: YARN-9985 > URL: https://issues.apache.org/jira/browse/YARN-9985 > Project: Hadoop YARN > Issue Type: Bug > Components: RM, yarn >Affects Versions: 3.2.1 >Reporter: Souryakanta Dwivedy >Assignee: Ayush Saxena >Priority: Minor > Attachments: YARN-9985-01.patch, image-2019-11-18-18-31-17-755.png, > image-2019-11-18-18-35-54-688.png > > > Unsupported "transitionToObserver" option displaying for rmadmin command > Check the options for Yarn rmadmin command > It will display the "-transitionToObserver " option which is not > supported > by yarn rmadmin command which is wrong behavior. > But if you check the yarn rmadmin -help it will not display any option > "-transitionToObserver " > > !image-2019-11-18-18-31-17-755.png! > > == > install/hadoop/resourcemanager/bin> ./yarn rmadmin -help > rmadmin is the command to execute YARN administrative commands. > The full syntax is: > yarn rmadmin [-refreshQueues] [-refreshNodes [-g|graceful [timeout in > seconds] -client|server]] [-refreshNodesResources] > [-refreshSuperUserGroupsConfiguration] [-refreshUserToGroupsMappings] > [-refreshAdminAcls] [-refreshServiceAcl] [-getGroup [username]] > [-addToClusterNodeLabels > <"label1(exclusive=true),label2(exclusive=false),label3">] > [-removeFromClusterNodeLabels ] [-replaceLabelsOnNode > <"node1[:port]=label1,label2 node2[:port]=label1"> [-failOnUnknownNodes]] > [-directlyAccessNodeLabelStore] [-refreshClusterMaxPriority] > [-updateNodeResource [NodeID] [MemSize] [vCores] ([OvercommitTimeout]) or > -updateNodeResource [NodeID] [ResourceTypes] ([OvercommitTimeout])] > *{color:#FF}[-transitionToActive [--forceactive] ]{color} > {color:#FF}[-transitionToStandby ]{color}* [-getServiceState > ] [-getAllServiceState] [-checkHealth ] [-help [cmd]] > -refreshQueues: Reload the queues' acls, states and scheduler specific > properties. > ResourceManager will reload the mapred-queues configuration file. > -refreshNodes [-g|graceful [timeout in seconds] -client|server]: Refresh the > hosts information at the ResourceManager. Here [-g|graceful [timeout in > seconds] -client|server] is optional, if we specify the timeout then > ResourceManager will wait for timeout before marking the NodeManager as > decommissioned. The -client|server indicates if the timeout tracking should > be handled by the client or the ResourceManager. The client-side tracking is > blocking, while the server-side tracking is not. Omitting the timeout, or a > timeout of -1, indicates an infinite timeout. Known Issue: the server-side > tracking will immediately decommission if an RM HA failover occurs. > -refreshNodesResources: Refresh resources of NodeManagers at the > ResourceManager. > -refreshSuperUserGroupsConfiguration: Refresh superuser proxy groups mappings > -refreshUserToGroupsMappings: Refresh user-to-groups mappings > -refreshAdminAcls: Refresh acls for administration of ResourceManager > -refreshServiceAcl: Reload the service-level authorization policy file. > ResourceManager will reload the authorization policy file. > -getGroups [username]: Get the groups which given user belongs to. > -addToClusterNodeLabels > <"label1(exclusive=true),label2(exclusive=false),label3">: add to cluster > node labels. Default exclusivity is true > -removeFromClusterNodeLabels (label splitted by ","): > remove from cluster node labels > -replaceLabelsOnNode <"node1[:port]=label1,label2 > node2[:port]=label1,label2"> [-failOnUnknownNodes] : replace labels on nodes > (please note that we do not support specifying multiple labels on a single > host for now.) > [-failOnUnknownNodes] is optional, when we set this option, it will fail if > specified nodes are unknown. > -directlyAccessNodeLabelStore: This is DEPRECATED, will be removed in future > releases. Directly access node label store, with this option, all node label > related operations will not connect RM. Instead, they will access/modify > stored node labels directly. By default, it is false (access via RM). AND > PLEASE NOTE: if you configured yarn.node-labels.fs-store.root-dir to a local > directory (instead of NFS or HDFS), this option will only work when the > command run on the machine where RM is running. > -refreshClusterMaxPriority: Refresh cluster max priority > -updateNodeResource [NodeID] [MemSize] [vCores] ([OvercommitTimeout]) > or > [NodeID] [resourcetypes] ([OvercommitTimeout]). : Update resource on > specific n
[jira] [Commented] (YARN-9624) Use switch case for ProtoUtils#convertFromProtoFormat containerState
[ https://issues.apache.org/jira/browse/YARN-9624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16958057#comment-16958057 ] Ayush Saxena commented on YARN-9624: For the test instead of having if-else checks and explicitly passing one elemt, it can be done in one line using streams, Something like this : {code:java} @Test public void testConvertFromOrToProtoFormat() { // Check if utility has all enum values Stream.of(ContainerState.values()) .forEach(a -> ProtoUtils.convertToProtoFormat(a)); Stream.of(ContainerSubState.values()) .forEach(a -> ProtoUtils.convertToProtoFormat(a)); Stream.of(ContainerSubStateProto.values()) .forEach(a -> ProtoUtils.convertFromProtoFormat(a)); Stream.of(ContainerStateProto.values()) .forEach(a -> ProtoUtils.convertFromProtoFormat(a)); } {code} Give a check if it sounds better. > Use switch case for ProtoUtils#convertFromProtoFormat containerState > > > Key: YARN-9624 > URL: https://issues.apache.org/jira/browse/YARN-9624 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Bibin Chundatt >Assignee: Bilwa S T >Priority: Major > Labels: performance > Attachments: YARN-9624.001.patch, YARN-9624.002.patch > > > On large cluster with 100K+ containers on every heartbeat > {{ContainerState.valueOf(e.name().replace(CONTAINER_STATE_PREFIX, ""))}} will > be too costly. Update with switch case. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9287) Consecutive String Builder Append Should Reuse
[ https://issues.apache.org/jira/browse/YARN-9287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16776636#comment-16776636 ] Ayush Saxena commented on YARN-9287: Test Failures Seems Not Related. [~bibinchundatt] [~giovanni.fumarola] Can you give a look. :) > Consecutive String Builder Append Should Reuse > -- > > Key: YARN-9287 > URL: https://issues.apache.org/jira/browse/YARN-9287 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Attachments: YARN-9287-01.patch, YARN-9287-02.patch, > YARN-9287-03.patch, YARN-9287-04.patch, YARN-9287-05.patch, > YARN-9287-05.patch, YARN-9287-06.patch > > > Consecutive calls to StringBuffer/StringBuilder .append should be chained, > reusing the target object. This can improve the performance by producing a > smaller bytecode, reducing overhead and improving inlining. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9287) Consecutive String Builder Append Should Reuse
[ https://issues.apache.org/jira/browse/YARN-9287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena updated YARN-9287: --- Attachment: YARN-9287-06.patch > Consecutive String Builder Append Should Reuse > -- > > Key: YARN-9287 > URL: https://issues.apache.org/jira/browse/YARN-9287 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Attachments: YARN-9287-01.patch, YARN-9287-02.patch, > YARN-9287-03.patch, YARN-9287-04.patch, YARN-9287-05.patch, > YARN-9287-05.patch, YARN-9287-06.patch > > > Consecutive calls to StringBuffer/StringBuilder .append should be chained, > reusing the target object. This can improve the performance by producing a > smaller bytecode, reducing overhead and improving inlining. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9287) Consecutive String Builder Append Should Reuse
[ https://issues.apache.org/jira/browse/YARN-9287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena updated YARN-9287: --- Attachment: YARN-9287-05.patch > Consecutive String Builder Append Should Reuse > -- > > Key: YARN-9287 > URL: https://issues.apache.org/jira/browse/YARN-9287 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Attachments: YARN-9287-01.patch, YARN-9287-02.patch, > YARN-9287-03.patch, YARN-9287-04.patch, YARN-9287-05.patch, YARN-9287-05.patch > > > Consecutive calls to StringBuffer/StringBuilder .append should be chained, > reusing the target object. This can improve the performance by producing a > smaller bytecode, reducing overhead and improving inlining. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9287) Consecutive String Builder Append Should Reuse
[ https://issues.apache.org/jira/browse/YARN-9287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16775310#comment-16775310 ] Ayush Saxena commented on YARN-9287: Thanx [~giovanni.fumarola] for the review. Have uploaded the rebased version. Pls Review :) > Consecutive String Builder Append Should Reuse > -- > > Key: YARN-9287 > URL: https://issues.apache.org/jira/browse/YARN-9287 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Attachments: YARN-9287-01.patch, YARN-9287-02.patch, > YARN-9287-03.patch, YARN-9287-04.patch, YARN-9287-05.patch > > > Consecutive calls to StringBuffer/StringBuilder .append should be chained, > reusing the target object. This can improve the performance by producing a > smaller bytecode, reducing overhead and improving inlining. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9287) Consecutive String Builder Append Should Reuse
[ https://issues.apache.org/jira/browse/YARN-9287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena updated YARN-9287: --- Attachment: YARN-9287-05.patch > Consecutive String Builder Append Should Reuse > -- > > Key: YARN-9287 > URL: https://issues.apache.org/jira/browse/YARN-9287 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Attachments: YARN-9287-01.patch, YARN-9287-02.patch, > YARN-9287-03.patch, YARN-9287-04.patch, YARN-9287-05.patch > > > Consecutive calls to StringBuffer/StringBuilder .append should be chained, > reusing the target object. This can improve the performance by producing a > smaller bytecode, reducing overhead and improving inlining. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9287) Consecutive String Builder Append Should Reuse
[ https://issues.apache.org/jira/browse/YARN-9287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16773139#comment-16773139 ] Ayush Saxena commented on YARN-9287: Ping [~bibinchundatt] [~giovanni.fumarola] Can Someone Help With The Review :) > Consecutive String Builder Append Should Reuse > -- > > Key: YARN-9287 > URL: https://issues.apache.org/jira/browse/YARN-9287 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Attachments: YARN-9287-01.patch, YARN-9287-02.patch, > YARN-9287-03.patch, YARN-9287-04.patch > > > Consecutive calls to StringBuffer/StringBuilder .append should be chained, > reusing the target object. This can improve the performance by producing a > smaller bytecode, reducing overhead and improving inlining. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9287) Consecutive String Builder Append Should Reuse
[ https://issues.apache.org/jira/browse/YARN-9287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16770297#comment-16770297 ] Ayush Saxena commented on YARN-9287: Fixed CheckStyle in v4. Other Warnings for pathlen and all seems not related. Pls Review :) > Consecutive String Builder Append Should Reuse > -- > > Key: YARN-9287 > URL: https://issues.apache.org/jira/browse/YARN-9287 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Attachments: YARN-9287-01.patch, YARN-9287-02.patch, > YARN-9287-03.patch, YARN-9287-04.patch > > > Consecutive calls to StringBuffer/StringBuilder .append should be chained, > reusing the target object. This can improve the performance by producing a > smaller bytecode, reducing overhead and improving inlining. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9287) Consecutive String Builder Append Should Reuse
[ https://issues.apache.org/jira/browse/YARN-9287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena updated YARN-9287: --- Attachment: YARN-9287-04.patch > Consecutive String Builder Append Should Reuse > -- > > Key: YARN-9287 > URL: https://issues.apache.org/jira/browse/YARN-9287 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Attachments: YARN-9287-01.patch, YARN-9287-02.patch, > YARN-9287-03.patch, YARN-9287-04.patch > > > Consecutive calls to StringBuffer/StringBuilder .append should be chained, > reusing the target object. This can improve the performance by producing a > smaller bytecode, reducing overhead and improving inlining. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9287) Consecutive String Builder Append Should Reuse
[ https://issues.apache.org/jira/browse/YARN-9287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena updated YARN-9287: --- Attachment: YARN-9287-03.patch > Consecutive String Builder Append Should Reuse > -- > > Key: YARN-9287 > URL: https://issues.apache.org/jira/browse/YARN-9287 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Attachments: YARN-9287-01.patch, YARN-9287-02.patch, > YARN-9287-03.patch > > > Consecutive calls to StringBuffer/StringBuilder .append should be chained, > reusing the target object. This can improve the performance by producing a > smaller bytecode, reducing overhead and improving inlining. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9287) Consecutive String Builder Append Should Reuse
[ https://issues.apache.org/jira/browse/YARN-9287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena updated YARN-9287: --- Attachment: YARN-9287-02.patch > Consecutive String Builder Append Should Reuse > -- > > Key: YARN-9287 > URL: https://issues.apache.org/jira/browse/YARN-9287 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Attachments: YARN-9287-01.patch, YARN-9287-02.patch > > > Consecutive calls to StringBuffer/StringBuilder .append should be chained, > reusing the target object. This can improve the performance by producing a > smaller bytecode, reducing overhead and improving inlining. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9287) Consecutive String Builder Append Should Reuse
[ https://issues.apache.org/jira/browse/YARN-9287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16762986#comment-16762986 ] Ayush Saxena commented on YARN-9287: Hi [~giovanni.fumarola]!!! Can you help review. :) > Consecutive String Builder Append Should Reuse > -- > > Key: YARN-9287 > URL: https://issues.apache.org/jira/browse/YARN-9287 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Attachments: YARN-9287-01.patch > > > Consecutive calls to StringBuffer/StringBuilder .append should be chained, > reusing the target object. This can improve the performance by producing a > smaller bytecode, reducing overhead and improving inlining. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9287) Consecutive String Builder Append Should Reuse
[ https://issues.apache.org/jira/browse/YARN-9287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena updated YARN-9287: --- Attachment: YARN-9287-01.patch > Consecutive String Builder Append Should Reuse > -- > > Key: YARN-9287 > URL: https://issues.apache.org/jira/browse/YARN-9287 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Attachments: YARN-9287-01.patch > > > Consecutive calls to StringBuffer/StringBuilder .append should be chained, > reusing the target object. This can improve the performance by producing a > smaller bytecode, reducing overhead and improving inlining. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-9287) Consecutive String Builder Append Should Reuse
Ayush Saxena created YARN-9287: -- Summary: Consecutive String Builder Append Should Reuse Key: YARN-9287 URL: https://issues.apache.org/jira/browse/YARN-9287 Project: Hadoop YARN Issue Type: Improvement Reporter: Ayush Saxena Assignee: Ayush Saxena Consecutive calls to StringBuffer/StringBuilder .append should be chained, reusing the target object. This can improve the performance by producing a smaller bytecode, reducing overhead and improving inlining. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-8498) Yarn NodeManager OOM Listener Fails Compilation on Ubuntu 18.04
[ https://issues.apache.org/jira/browse/YARN-8498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16748635#comment-16748635 ] Ayush Saxena edited comment on YARN-8498 at 1/23/19 9:43 AM: - [~vinayrpet] [~sunilg] [~surendrasingh] Mind giving a check was (Author: ayushtkn): [~vinayrpet] [~sunilg] Mind giving a check > Yarn NodeManager OOM Listener Fails Compilation on Ubuntu 18.04 > --- > > Key: YARN-8498 > URL: https://issues.apache.org/jira/browse/YARN-8498 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Jack Bearden >Assignee: Ayush Saxena >Priority: Blocker > Attachments: YARN-8498-02.patch, YARN-8948-01.patch > > > While building this project, I ran into a few compilation errors here. The > first one was in this file: > hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/oom-listener/impl/oom_listener_main.c > At the very end, during the compilation of the OOM test, it fails again: > > hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/oom-listener/test/oom_listener_test_main.cc > hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/oom-listener/test/oom_listener_test_main.cc:256:7: > error: ‘__WAIT_STATUS’ was not declared in this scope > __WAIT_STATUS mem_hog_status = {}; > hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/oom-listener/test/oom_listener_test_main.cc:257:30: > error: ‘mem_hog_status’ was not declared in this scope > __pid_t exited0 = wait(mem_hog_status); > hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/oom-listener/test/oom_listener_test_main.cc:275:21: > error: expected ‘;’ before ‘oom_listener_status’ > __WAIT_STATUS oom_listener_status = {}; > hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/oom-listener/test/oom_listener_test_main.cc:276:30: > error: ‘oom_listener_status’ was not declared in this scope > __pid_t exited1 = wait(oom_listener_status); > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org