[jira] [Updated] (YARN-4957) Add getNewReservation in ApplicationClientProtocol
[ https://issues.apache.org/jira/browse/YARN-4957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Po updated YARN-4957: -- Attachment: YARN-4957.v10.patch YARN-4957.v10.patch fixes remaining Javadoc issues, and has been tested on a single node. > Add getNewReservation in ApplicationClientProtocol > -- > > Key: YARN-4957 > URL: https://issues.apache.org/jira/browse/YARN-4957 > Project: Hadoop YARN > Issue Type: Sub-task > Components: applications, client, resourcemanager >Affects Versions: 2.8.0 >Reporter: Subru Krishnan >Assignee: Sean Po > Labels: api-breaking > Attachments: YARN-4957.v0.patch, YARN-4957.v1.patch, > YARN-4957.v10.patch, YARN-4957.v2.patch, YARN-4957.v3.patch, > YARN-4957.v4.patch, YARN-4957.v5.1.patch, YARN-4957.v5.2.patch, > YARN-4957.v5.patch, YARN-4957.v7.patch, YARN-4957.v8.patch, > YARN-4957.v9.1.patch, YARN-4957.v9.patch > > > Currently submitReservation returns a ReservationId if sucessful. This JIRA > propose adding a getNewReservation in ApplicationClientProtocol for the > following reasons: > * Prevent zombie reservations in the face of client and/or network failures > post submitReservation > * Align reservation submission with application submission -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-4957) Add getNewReservation in ApplicationClientProtocol
[ https://issues.apache.org/jira/browse/YARN-4957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Po updated YARN-4957: -- Attachment: YARN-4957.v9.1.patch Re-uploading version 9 because patch creation failed. > Add getNewReservation in ApplicationClientProtocol > -- > > Key: YARN-4957 > URL: https://issues.apache.org/jira/browse/YARN-4957 > Project: Hadoop YARN > Issue Type: Sub-task > Components: applications, client, resourcemanager >Affects Versions: 2.8.0 >Reporter: Subru Krishnan >Assignee: Sean Po > Labels: api-breaking > Attachments: YARN-4957.v0.patch, YARN-4957.v1.patch, > YARN-4957.v2.patch, YARN-4957.v3.patch, YARN-4957.v4.patch, > YARN-4957.v5.1.patch, YARN-4957.v5.2.patch, YARN-4957.v5.patch, > YARN-4957.v7.patch, YARN-4957.v8.patch, YARN-4957.v9.1.patch, > YARN-4957.v9.patch > > > Currently submitReservation returns a ReservationId if sucessful. This JIRA > propose adding a getNewReservation in ApplicationClientProtocol for the > following reasons: > * Prevent zombie reservations in the face of client and/or network failures > post submitReservation > * Align reservation submission with application submission -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-4957) Add getNewReservation in ApplicationClientProtocol
[ https://issues.apache.org/jira/browse/YARN-4957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Po updated YARN-4957: -- Attachment: YARN-4957.v9.patch Refactored TestYarnClient and TestClientRMService to reduce the boilerplate cluster and client setup code. > Add getNewReservation in ApplicationClientProtocol > -- > > Key: YARN-4957 > URL: https://issues.apache.org/jira/browse/YARN-4957 > Project: Hadoop YARN > Issue Type: Sub-task > Components: applications, client, resourcemanager >Affects Versions: 2.8.0 >Reporter: Subru Krishnan >Assignee: Sean Po > Labels: api-breaking > Attachments: YARN-4957.v0.patch, YARN-4957.v1.patch, > YARN-4957.v2.patch, YARN-4957.v3.patch, YARN-4957.v4.patch, > YARN-4957.v5.1.patch, YARN-4957.v5.2.patch, YARN-4957.v5.patch, > YARN-4957.v7.patch, YARN-4957.v8.patch, YARN-4957.v9.patch > > > Currently submitReservation returns a ReservationId if sucessful. This JIRA > propose adding a getNewReservation in ApplicationClientProtocol for the > following reasons: > * Prevent zombie reservations in the face of client and/or network failures > post submitReservation > * Align reservation submission with application submission -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-4957) Add getNewReservation in ApplicationClientProtocol
[ https://issues.apache.org/jira/browse/YARN-4957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Po updated YARN-4957: -- Attachment: YARN-4957.v8.patch Fixed the remaining checkstyle issues in this patch. > Add getNewReservation in ApplicationClientProtocol > -- > > Key: YARN-4957 > URL: https://issues.apache.org/jira/browse/YARN-4957 > Project: Hadoop YARN > Issue Type: Sub-task > Components: applications, client, resourcemanager >Affects Versions: 2.8.0 >Reporter: Subru Krishnan >Assignee: Sean Po > Labels: api-breaking > Attachments: YARN-4957.v0.patch, YARN-4957.v1.patch, > YARN-4957.v2.patch, YARN-4957.v3.patch, YARN-4957.v4.patch, > YARN-4957.v5.1.patch, YARN-4957.v5.2.patch, YARN-4957.v5.patch, > YARN-4957.v7.patch, YARN-4957.v8.patch > > > Currently submitReservation returns a ReservationId if sucessful. This JIRA > propose adding a getNewReservation in ApplicationClientProtocol for the > following reasons: > * Prevent zombie reservations in the face of client and/or network failures > post submitReservation > * Align reservation submission with application submission -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-4957) Add getNewReservation in ApplicationClientProtocol
[ https://issues.apache.org/jira/browse/YARN-4957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Po updated YARN-4957: -- Attachment: YARN-4957.v7.patch YARN-4957.v7.patch addresses Carlo's comments and fixes the checkstyle and javadoc issues. Two check-style issues with regards to method length will remain - these are pre-existing, and a refactor will be out of the scope of this JIRA. > Add getNewReservation in ApplicationClientProtocol > -- > > Key: YARN-4957 > URL: https://issues.apache.org/jira/browse/YARN-4957 > Project: Hadoop YARN > Issue Type: Sub-task > Components: applications, client, resourcemanager >Affects Versions: 2.8.0 >Reporter: Subru Krishnan >Assignee: Sean Po > Labels: api-breaking > Attachments: YARN-4957.v0.patch, YARN-4957.v1.patch, > YARN-4957.v2.patch, YARN-4957.v3.patch, YARN-4957.v4.patch, > YARN-4957.v5.1.patch, YARN-4957.v5.2.patch, YARN-4957.v5.patch, > YARN-4957.v7.patch > > > Currently submitReservation returns a ReservationId if sucessful. This JIRA > propose adding a getNewReservation in ApplicationClientProtocol for the > following reasons: > * Prevent zombie reservations in the face of client and/or network failures > post submitReservation > * Align reservation submission with application submission -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-4957) Add getNewReservation in ApplicationClientProtocol
[ https://issues.apache.org/jira/browse/YARN-4957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Po updated YARN-4957: -- Attachment: YARN-4957.v5.2.patch Remade patch based on latest trunk. > Add getNewReservation in ApplicationClientProtocol > -- > > Key: YARN-4957 > URL: https://issues.apache.org/jira/browse/YARN-4957 > Project: Hadoop YARN > Issue Type: Sub-task > Components: applications, client, resourcemanager >Affects Versions: 2.8.0 >Reporter: Subru Krishnan >Assignee: Sean Po > Labels: api-breaking > Attachments: YARN-4957.v0.patch, YARN-4957.v1.patch, > YARN-4957.v2.patch, YARN-4957.v3.patch, YARN-4957.v4.patch, > YARN-4957.v5.1.patch, YARN-4957.v5.2.patch, YARN-4957.v5.patch > > > Currently submitReservation returns a ReservationId if sucessful. This JIRA > propose adding a getNewReservation in ApplicationClientProtocol for the > following reasons: > * Prevent zombie reservations in the face of client and/or network failures > post submitReservation > * Align reservation submission with application submission -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-4957) Add getNewReservation in ApplicationClientProtocol
[ https://issues.apache.org/jira/browse/YARN-4957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Po updated YARN-4957: -- Attachment: YARN-4957.v5.1.patch Re-uploading patch to rekick jenkins. > Add getNewReservation in ApplicationClientProtocol > -- > > Key: YARN-4957 > URL: https://issues.apache.org/jira/browse/YARN-4957 > Project: Hadoop YARN > Issue Type: Sub-task > Components: applications, client, resourcemanager >Affects Versions: 2.8.0 >Reporter: Subru Krishnan >Assignee: Sean Po > Labels: api-breaking > Attachments: YARN-4957.v0.patch, YARN-4957.v1.patch, > YARN-4957.v2.patch, YARN-4957.v3.patch, YARN-4957.v4.patch, > YARN-4957.v5.1.patch, YARN-4957.v5.patch > > > Currently submitReservation returns a ReservationId if sucessful. This JIRA > propose adding a getNewReservation in ApplicationClientProtocol for the > following reasons: > * Prevent zombie reservations in the face of client and/or network failures > post submitReservation > * Align reservation submission with application submission -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-4957) Add getNewReservation in ApplicationClientProtocol
[ https://issues.apache.org/jira/browse/YARN-4957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Po updated YARN-4957: -- Attachment: YARN-4957.v5.patch Thanks [~subru] for your comments. V5 of the patch addresses each of them. > Add getNewReservation in ApplicationClientProtocol > -- > > Key: YARN-4957 > URL: https://issues.apache.org/jira/browse/YARN-4957 > Project: Hadoop YARN > Issue Type: Sub-task > Components: applications, client, resourcemanager >Affects Versions: 2.8.0 >Reporter: Subru Krishnan >Assignee: Sean Po > Labels: api-breaking > Attachments: YARN-4957.v0.patch, YARN-4957.v1.patch, > YARN-4957.v2.patch, YARN-4957.v3.patch, YARN-4957.v4.patch, YARN-4957.v5.patch > > > Currently submitReservation returns a ReservationId if sucessful. This JIRA > propose adding a getNewReservation in ApplicationClientProtocol for the > following reasons: > * Prevent zombie reservations in the face of client and/or network failures > post submitReservation > * Align reservation submission with application submission -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-4957) Add getNewReservation in ApplicationClientProtocol
[ https://issues.apache.org/jira/browse/YARN-4957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Po updated YARN-4957: -- Attachment: YARN-4957.v4.patch Added description as to why IOException and YarnException might be thrown when calling createNewReservation. > Add getNewReservation in ApplicationClientProtocol > -- > > Key: YARN-4957 > URL: https://issues.apache.org/jira/browse/YARN-4957 > Project: Hadoop YARN > Issue Type: Sub-task > Components: applications, client, resourcemanager >Affects Versions: 2.8.0 >Reporter: Subru Krishnan >Assignee: Sean Po > Labels: api-breaking > Attachments: YARN-4957.v0.patch, YARN-4957.v1.patch, > YARN-4957.v2.patch, YARN-4957.v3.patch, YARN-4957.v4.patch > > > Currently submitReservation returns a ReservationId if sucessful. This JIRA > propose adding a getNewReservation in ApplicationClientProtocol for the > following reasons: > * Prevent zombie reservations in the face of client and/or network failures > post submitReservation > * Align reservation submission with application submission -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-4957) Add getNewReservation in ApplicationClientProtocol
[ https://issues.apache.org/jira/browse/YARN-4957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Po updated YARN-4957: -- Attachment: YARN-4957.v3.patch Fixed javadoc issues. Ignoring request to remove unused constructor for NewReservation because it is required for using record factory for instantiation. > Add getNewReservation in ApplicationClientProtocol > -- > > Key: YARN-4957 > URL: https://issues.apache.org/jira/browse/YARN-4957 > Project: Hadoop YARN > Issue Type: Sub-task > Components: applications, client, resourcemanager >Affects Versions: 2.8.0 >Reporter: Subru Krishnan >Assignee: Sean Po > Labels: api-breaking > Attachments: YARN-4957.v0.patch, YARN-4957.v1.patch, > YARN-4957.v2.patch, YARN-4957.v3.patch > > > Currently submitReservation returns a ReservationId if sucessful. This JIRA > propose adding a getNewReservation in ApplicationClientProtocol for the > following reasons: > * Prevent zombie reservations in the face of client and/or network failures > post submitReservation > * Align reservation submission with application submission -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-4957) Add getNewReservation in ApplicationClientProtocol
[ https://issues.apache.org/jira/browse/YARN-4957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Po updated YARN-4957: -- Attachment: YARN-4957.v2.patch In YARN-4957.v2.patch I fixed the findbugs and Javadoc issues. The findbugs related to unused empty constructor is incorrect because the empty constructor is actually used in RecordFactoryPBImpl. *+Test Failures:+* +Test:+ [testNodeLabelContainerAllocation.testQueueUsedCapacitiesUpdate] Existing JIRA: https://issues.apache.org/jira/browse/YARN-4890?jql=text%20~%20%22TestNodeLabelContainerAllocation%22 This test passes locally. +Tests:+[TestClientRMTokens.testShortCircuitRenewCancelDifferentHostSamePort, TestClientRMTokens.testShortCircuitRenewCancelDifferentHostDifferentPort, TestClientRMTokens.testShortCircuitRenewCancelSameHostDifferentPort, TestClientRMTokens.testShortCircuitRenewCancel:285->checkShortCircuitRenewCancel, TestClientRMTokens.testShortCircuitRenewCancelWildcardAddress, TestAMAuthorization.testUnauthorizedAccess] Existing JIRA: https://issues.apache.org/jira/browse/HADOOP-12687 These tests pass locally. +Tests:+ [TestAMRestart.testAMRestartNotLostContainerCompleteMsg] This test passes locally if I change the timeout to 40 seconds. Right now, it is set at 30, but takes 37 seconds to complete. The result of this test seems dependent on the machine on which it is run. +Tests:+ [TestNodeLabelContainerAllocation.testQueueUsedCapacitiesUpdate] Existing JIRA: https://issues.apache.org/jira/browse/YARN-4890?jql=text%20~%20%22TestNodeLabelContainerAllocation%22 This test passes locally. +Tests:+ [TestSchedulingPolicy.testIsApplicableTo] This test passes locally if I change the timeout to 3 seconds. Right now, it is set to 1 second, but takes 2.9 seconds to complete.he result of this test seems dependent on the machine on which it is run. +Tests:+ [TestUberAM.testSleepJob, TestUberAM.testSleepJobWithMultipleReducers, TestUberAM.testRandomWriter, TestUberAM>TestMRJobs.testSleepJobWithRemoteJar] Existing JIRA: https://issues.apache.org/jira/browse/MAPREDUCE-6681?jql=text%20~%20%22TestUberAM%22 This test fails locally. It also fails locally on trunk. +Tests:+ [TestAMRMProxy.testAMRMProxyE2E, TestAMRMProxy.testE2ETokenSwap, TestAMRMProxy.testE2ETokenRenewal] Existing JIRA: There was no existing JIRA, but I have found evidence that this test fails on other open JIRAs as well. https://builds.apache.org/job/PreCommit-YARN-Build/11016/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.8.0_77.txt These tests pass locally. +Tests:+ [TestYarnClient.testShouldNotRetryForeverForNonNetworkExceptions, TestYarnClient.testAMMRTokens, TestYarnClient.testSubmitIncorrectQueue] Existing JIRAs: [https://issues.apache.org/jira/browse/YARN-4935?jql=text%20~%20%22TestAMRMProxy%22, https://builds.apache.org/job/PreCommit-YARN-Build/11016/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.8.0_77.txt]. The second link is evidence of other opened JIRA having the same failure. These tests pass locally. +Tests:+ [TestGetGroups>GetGroupsTestBase.testMultipleNonExistingUsers, TestGetGroups>GetGroupsTestBase.testExistingInterleavedWithNonExistentUsers, TestGetGroups>GetGroupsTestBase.testNoUserGiven, TestGetGroups>GetGroupsTestBase.testExistingUser] Existing JIRA: https://issues.apache.org/jira/browse/YARN-4351?jql=text%20~%20%22TestGetGroups%22 These tests pass locally. > Add getNewReservation in ApplicationClientProtocol > -- > > Key: YARN-4957 > URL: https://issues.apache.org/jira/browse/YARN-4957 > Project: Hadoop YARN > Issue Type: Sub-task > Components: applications, client, resourcemanager >Affects Versions: 2.8.0 >Reporter: Subru Krishnan >Assignee: Sean Po > Labels: api-breaking > Attachments: YARN-4957.v0.patch, YARN-4957.v1.patch, > YARN-4957.v2.patch > > > Currently submitReservation returns a ReservationId if sucessful. This JIRA > propose adding a getNewReservation in ApplicationClientProtocol for the > following reasons: > * Prevent zombie reservations in the face of client and/or network failures > post submitReservation > * Align reservation submission with application submission -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-4957) Add getNewReservation in ApplicationClientProtocol
[ https://issues.apache.org/jira/browse/YARN-4957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Po updated YARN-4957: -- Labels: api-breaking (was: ) > Add getNewReservation in ApplicationClientProtocol > -- > > Key: YARN-4957 > URL: https://issues.apache.org/jira/browse/YARN-4957 > Project: Hadoop YARN > Issue Type: Sub-task > Components: applications, client, resourcemanager >Affects Versions: 2.8.0 >Reporter: Subru Krishnan >Assignee: Sean Po > Labels: api-breaking > Attachments: YARN-4957.v0.patch, YARN-4957.v1.patch > > > Currently submitReservation returns a ReservationId if sucessful. This JIRA > propose adding a getNewReservation in ApplicationClientProtocol for the > following reasons: > * Prevent zombie reservations in the face of client and/or network failures > post submitReservation > * Align reservation submission with application submission -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-4957) Add getNewReservation in ApplicationClientProtocol
[ https://issues.apache.org/jira/browse/YARN-4957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Po updated YARN-4957: -- Attachment: YARN-4957.v1.patch > Add getNewReservation in ApplicationClientProtocol > -- > > Key: YARN-4957 > URL: https://issues.apache.org/jira/browse/YARN-4957 > Project: Hadoop YARN > Issue Type: Sub-task > Components: applications, client, resourcemanager >Affects Versions: 2.8.0 >Reporter: Subru Krishnan >Assignee: Sean Po > Attachments: YARN-4957.v0.patch, YARN-4957.v1.patch > > > Currently submitReservation returns a ReservationId if sucessful. This JIRA > propose adding a getNewReservation in ApplicationClientProtocol for the > following reasons: > * Prevent zombie reservations in the face of client and/or network failures > post submitReservation > * Align reservation submission with application submission -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-4957) Add getNewReservation in ApplicationClientProtocol
[ https://issues.apache.org/jira/browse/YARN-4957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Po updated YARN-4957: -- Attachment: YARN-4957.v0.patch > Add getNewReservation in ApplicationClientProtocol > -- > > Key: YARN-4957 > URL: https://issues.apache.org/jira/browse/YARN-4957 > Project: Hadoop YARN > Issue Type: Sub-task > Components: applications, client, resourcemanager >Affects Versions: 2.8.0 >Reporter: Subru Krishnan >Assignee: Sean Po > Attachments: YARN-4957.v0.patch > > > Currently submitReservation returns a ReservationId if sucessful. This JIRA > propose adding a getNewReservation in ApplicationClientProtocol for the > following reasons: > * Prevent zombie reservations in the face of client and/or network failures > post submitReservation > * Align reservation submission with application submission -- This message was sent by Atlassian JIRA (v6.3.4#6332)
