[jira] [Commented] (MAPREDUCE-6324) Uber jobs fail to update AMRM token when it rolls over
[ https://issues.apache.org/jira/browse/MAPREDUCE-6324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14505767#comment-14505767 ] Robert Kanter commented on MAPREDUCE-6324: -- Thanks for finding this issue and posting it; this'll be helpful for anyone using Uber mode. LGTM +1 on the change. It seems like this might be tricky to do, but would it be possible to add a unit test? > Uber jobs fail to update AMRM token when it rolls over > -- > > Key: MAPREDUCE-6324 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6324 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mr-am >Affects Versions: 2.6.0 >Reporter: Jason Lowe >Assignee: Jason Lowe >Priority: Blocker > Attachments: MAPREDUCE-6324.001.patch > > > When the RM rolls a new AMRM master key the AMs are supposed to receive a new > AMRM token on subsequent heartbeats between the time when the new key is > rolled and when it is activated. This is not occurring for uber jobs. If > the connection to the RM needs to be re-established after the new key is > activated (e.g.: RM restart or network hiccup) then the uber job AM will be > unable to reconnect to the RM. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6324) Uber jobs fail to update AMRM token when it rolls over
[ https://issues.apache.org/jira/browse/MAPREDUCE-6324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14505805#comment-14505805 ] Jason Lowe commented on MAPREDUCE-6324: --- Thanks for the review, Robert! Yes, this should have a unit test. Wanted to post a version without since I was able to manually test it with an uberized Oozie launcher job and verify it fixes the issue. I'm busy with a bunch of other stuff right now, but I'll try to take a crack at adding a unit test later this week. > Uber jobs fail to update AMRM token when it rolls over > -- > > Key: MAPREDUCE-6324 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6324 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mr-am >Affects Versions: 2.6.0 >Reporter: Jason Lowe >Assignee: Jason Lowe >Priority: Blocker > Attachments: MAPREDUCE-6324.001.patch > > > When the RM rolls a new AMRM master key the AMs are supposed to receive a new > AMRM token on subsequent heartbeats between the time when the new key is > rolled and when it is activated. This is not occurring for uber jobs. If > the connection to the RM needs to be re-established after the new key is > activated (e.g.: RM restart or network hiccup) then the uber job AM will be > unable to reconnect to the RM. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6324) Uber jobs fail to update AMRM token when it rolls over
[
https://issues.apache.org/jira/browse/MAPREDUCE-6324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14505812#comment-14505812
]
Hadoop QA commented on MAPREDUCE-6324:
--
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 14m 38s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:red}-1{color} | tests included | 0m 0s | The patch doesn't appear
to include any new or modified tests. Please justify why no new tests are
needed for this patch. Also please list what manual steps were performed to
verify this patch. |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | javac | 7m 25s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 33s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 22s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 5m 27s | The applied patch generated 1
additional checkstyle issues. |
| {color:green}+1{color} | install | 1m 34s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 0m 56s | The patch does not introduce
any new Findbugs (version 2.0.3) warnings. |
| {color:green}+1{color} | mapreduce tests | 8m 44s | Tests passed in
hadoop-mapreduce-client-app. |
| | | 49m 15s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12726980/MAPREDUCE-6324.001.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 73ddb6b |
| checkstyle |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5426/artifact/patchprocess/checkstyle-result-diff.txt
|
| hadoop-mapreduce-client-app test log |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5426/artifact/patchprocess/testrun_hadoop-mapreduce-client-app.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5426/testReport/ |
| Console output |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5426//console |
This message was automatically generated.
> Uber jobs fail to update AMRM token when it rolls over
> --
>
> Key: MAPREDUCE-6324
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6324
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mr-am
>Affects Versions: 2.6.0
>Reporter: Jason Lowe
>Assignee: Jason Lowe
>Priority: Blocker
> Attachments: MAPREDUCE-6324.001.patch
>
>
> When the RM rolls a new AMRM master key the AMs are supposed to receive a new
> AMRM token on subsequent heartbeats between the time when the new key is
> rolled and when it is activated. This is not occurring for uber jobs. If
> the connection to the RM needs to be re-established after the new key is
> activated (e.g.: RM restart or network hiccup) then the uber job AM will be
> unable to reconnect to the RM.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6324) Uber jobs fail to update AMRM token when it rolls over
[
https://issues.apache.org/jira/browse/MAPREDUCE-6324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14509871#comment-14509871
]
Hadoop QA commented on MAPREDUCE-6324:
--
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 14m 58s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | javac | 7m 30s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 35s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 22s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 4m 1s | The applied patch generated 1
additional checkstyle issues. |
| {color:green}+1{color} | install | 1m 33s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 32s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 0m 55s | The patch does not introduce
any new Findbugs (version 2.0.3) warnings. |
| {color:red}-1{color} | mapreduce tests | 9m 30s | Tests failed in
hadoop-mapreduce-client-app. |
| | | 48m 59s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests | hadoop.mapreduce.v2.app.webapp.TestAMWebServicesTasks |
| | hadoop.mapreduce.v2.app.webapp.TestAppController |
| | hadoop.mapreduce.jobhistory.TestEvents |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12727702/MAPREDUCE-6324.002.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 19262d9 |
| checkstyle |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5436/artifact/patchprocess/checkstyle-result-diff.txt
|
| hadoop-mapreduce-client-app test log |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5436/artifact/patchprocess/testrun_hadoop-mapreduce-client-app.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5436/testReport/ |
| Console output |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5436/console |
This message was automatically generated.
> Uber jobs fail to update AMRM token when it rolls over
> --
>
> Key: MAPREDUCE-6324
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6324
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mr-am
>Affects Versions: 2.6.0
>Reporter: Jason Lowe
>Assignee: Jason Lowe
>Priority: Blocker
> Attachments: MAPREDUCE-6324.001.patch, MAPREDUCE-6324.002.patch
>
>
> When the RM rolls a new AMRM master key the AMs are supposed to receive a new
> AMRM token on subsequent heartbeats between the time when the new key is
> rolled and when it is activated. This is not occurring for uber jobs. If
> the connection to the RM needs to be re-established after the new key is
> activated (e.g.: RM restart or network hiccup) then the uber job AM will be
> unable to reconnect to the RM.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6324) Uber jobs fail to update AMRM token when it rolls over
[ https://issues.apache.org/jira/browse/MAPREDUCE-6324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14510086#comment-14510086 ] Vinod Kumar Vavilapalli commented on MAPREDUCE-6324: Looks good to me. One nit: testAllocResponseId is not really validating that responseIDs are changing? > Uber jobs fail to update AMRM token when it rolls over > -- > > Key: MAPREDUCE-6324 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6324 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mr-am >Affects Versions: 2.6.0 >Reporter: Jason Lowe >Assignee: Jason Lowe >Priority: Blocker > Attachments: MAPREDUCE-6324.001.patch, MAPREDUCE-6324.002.patch > > > When the RM rolls a new AMRM master key the AMs are supposed to receive a new > AMRM token on subsequent heartbeats between the time when the new key is > rolled and when it is activated. This is not occurring for uber jobs. If > the connection to the RM needs to be re-established after the new key is > activated (e.g.: RM restart or network hiccup) then the uber job AM will be > unable to reconnect to the RM. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6324) Uber jobs fail to update AMRM token when it rolls over
[ https://issues.apache.org/jira/browse/MAPREDUCE-6324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1457#comment-1457 ] Jason Lowe commented on MAPREDUCE-6324: --- bq. One nit: testAllocResponseId is not really validating that responseIDs are changing? It does because it fails if LocalContainerAllocator doesn't have the patch. The assert is in the MockScheduler class that increments the response ID with each allocate call and sends it back in the response, just like the real ApplicationMasterService. The MockScheduler asserts that the response ID of allocate requests match the last response ID sent. > Uber jobs fail to update AMRM token when it rolls over > -- > > Key: MAPREDUCE-6324 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6324 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mr-am >Affects Versions: 2.6.0 >Reporter: Jason Lowe >Assignee: Jason Lowe >Priority: Blocker > Attachments: MAPREDUCE-6324.001.patch, MAPREDUCE-6324.002.patch > > > When the RM rolls a new AMRM master key the AMs are supposed to receive a new > AMRM token on subsequent heartbeats between the time when the new key is > rolled and when it is activated. This is not occurring for uber jobs. If > the connection to the RM needs to be re-established after the new key is > activated (e.g.: RM restart or network hiccup) then the uber job AM will be > unable to reconnect to the RM. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6324) Uber jobs fail to update AMRM token when it rolls over
[ https://issues.apache.org/jira/browse/MAPREDUCE-6324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14515077#comment-14515077 ] Vinod Kumar Vavilapalli commented on MAPREDUCE-6324: Makes sense. Looks good, +1. Checking this in. > Uber jobs fail to update AMRM token when it rolls over > -- > > Key: MAPREDUCE-6324 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6324 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mr-am >Affects Versions: 2.6.0 >Reporter: Jason Lowe >Assignee: Jason Lowe >Priority: Blocker > Attachments: MAPREDUCE-6324.001.patch, MAPREDUCE-6324.002.patch > > > When the RM rolls a new AMRM master key the AMs are supposed to receive a new > AMRM token on subsequent heartbeats between the time when the new key is > rolled and when it is activated. This is not occurring for uber jobs. If > the connection to the RM needs to be re-established after the new key is > activated (e.g.: RM restart or network hiccup) then the uber job AM will be > unable to reconnect to the RM. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6324) Uber jobs fail to update AMRM token when it rolls over
[ https://issues.apache.org/jira/browse/MAPREDUCE-6324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14515109#comment-14515109 ] Hudson commented on MAPREDUCE-6324: --- FAILURE: Integrated in Hadoop-trunk-Commit #7689 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/7689/]) MAPREDUCE-6324. Fixed MapReduce uber jobs to not fail the udpate of AM-RM tokens when they roll-over. Contributed by Jason Lowe. (vinodkv: rev 9fc32c5c4d1d5f50c605bdb0e3b13f44c86660c8) * hadoop-mapreduce-project/CHANGES.txt * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/local/TestLocalContainerAllocator.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/local/LocalContainerAllocator.java > Uber jobs fail to update AMRM token when it rolls over > -- > > Key: MAPREDUCE-6324 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6324 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mr-am >Affects Versions: 2.6.0 >Reporter: Jason Lowe >Assignee: Jason Lowe >Priority: Blocker > Fix For: 2.7.1 > > Attachments: MAPREDUCE-6324.001.patch, MAPREDUCE-6324.002.patch > > > When the RM rolls a new AMRM master key the AMs are supposed to receive a new > AMRM token on subsequent heartbeats between the time when the new key is > rolled and when it is activated. This is not occurring for uber jobs. If > the connection to the RM needs to be re-established after the new key is > activated (e.g.: RM restart or network hiccup) then the uber job AM will be > unable to reconnect to the RM. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6324) Uber jobs fail to update AMRM token when it rolls over
[ https://issues.apache.org/jira/browse/MAPREDUCE-6324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14516856#comment-14516856 ] Hudson commented on MAPREDUCE-6324: --- FAILURE: Integrated in Hadoop-Hdfs-trunk #2109 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/2109/]) MAPREDUCE-6324. Fixed MapReduce uber jobs to not fail the udpate of AM-RM tokens when they roll-over. Contributed by Jason Lowe. (vinodkv: rev 9fc32c5c4d1d5f50c605bdb0e3b13f44c86660c8) * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/local/LocalContainerAllocator.java * hadoop-mapreduce-project/CHANGES.txt * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/local/TestLocalContainerAllocator.java > Uber jobs fail to update AMRM token when it rolls over > -- > > Key: MAPREDUCE-6324 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6324 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mr-am >Affects Versions: 2.6.0 >Reporter: Jason Lowe >Assignee: Jason Lowe >Priority: Blocker > Fix For: 2.7.1 > > Attachments: MAPREDUCE-6324.001.patch, MAPREDUCE-6324.002.patch > > > When the RM rolls a new AMRM master key the AMs are supposed to receive a new > AMRM token on subsequent heartbeats between the time when the new key is > rolled and when it is activated. This is not occurring for uber jobs. If > the connection to the RM needs to be re-established after the new key is > activated (e.g.: RM restart or network hiccup) then the uber job AM will be > unable to reconnect to the RM. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6324) Uber jobs fail to update AMRM token when it rolls over
[ https://issues.apache.org/jira/browse/MAPREDUCE-6324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14516870#comment-14516870 ] Hudson commented on MAPREDUCE-6324: --- FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #168 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/168/]) MAPREDUCE-6324. Fixed MapReduce uber jobs to not fail the udpate of AM-RM tokens when they roll-over. Contributed by Jason Lowe. (vinodkv: rev 9fc32c5c4d1d5f50c605bdb0e3b13f44c86660c8) * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/local/LocalContainerAllocator.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/local/TestLocalContainerAllocator.java * hadoop-mapreduce-project/CHANGES.txt > Uber jobs fail to update AMRM token when it rolls over > -- > > Key: MAPREDUCE-6324 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6324 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mr-am >Affects Versions: 2.6.0 >Reporter: Jason Lowe >Assignee: Jason Lowe >Priority: Blocker > Fix For: 2.7.1 > > Attachments: MAPREDUCE-6324.001.patch, MAPREDUCE-6324.002.patch > > > When the RM rolls a new AMRM master key the AMs are supposed to receive a new > AMRM token on subsequent heartbeats between the time when the new key is > rolled and when it is activated. This is not occurring for uber jobs. If > the connection to the RM needs to be re-established after the new key is > activated (e.g.: RM restart or network hiccup) then the uber job AM will be > unable to reconnect to the RM. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6324) Uber jobs fail to update AMRM token when it rolls over
[ https://issues.apache.org/jira/browse/MAPREDUCE-6324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14516886#comment-14516886 ] Hudson commented on MAPREDUCE-6324: --- FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #177 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/177/]) MAPREDUCE-6324. Fixed MapReduce uber jobs to not fail the udpate of AM-RM tokens when they roll-over. Contributed by Jason Lowe. (vinodkv: rev 9fc32c5c4d1d5f50c605bdb0e3b13f44c86660c8) * hadoop-mapreduce-project/CHANGES.txt * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/local/LocalContainerAllocator.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/local/TestLocalContainerAllocator.java > Uber jobs fail to update AMRM token when it rolls over > -- > > Key: MAPREDUCE-6324 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6324 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mr-am >Affects Versions: 2.6.0 >Reporter: Jason Lowe >Assignee: Jason Lowe >Priority: Blocker > Fix For: 2.7.1 > > Attachments: MAPREDUCE-6324.001.patch, MAPREDUCE-6324.002.patch > > > When the RM rolls a new AMRM master key the AMs are supposed to receive a new > AMRM token on subsequent heartbeats between the time when the new key is > rolled and when it is activated. This is not occurring for uber jobs. If > the connection to the RM needs to be re-established after the new key is > activated (e.g.: RM restart or network hiccup) then the uber job AM will be > unable to reconnect to the RM. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6324) Uber jobs fail to update AMRM token when it rolls over
[ https://issues.apache.org/jira/browse/MAPREDUCE-6324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14516908#comment-14516908 ] Hudson commented on MAPREDUCE-6324: --- FAILURE: Integrated in Hadoop-Yarn-trunk #911 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/911/]) MAPREDUCE-6324. Fixed MapReduce uber jobs to not fail the udpate of AM-RM tokens when they roll-over. Contributed by Jason Lowe. (vinodkv: rev 9fc32c5c4d1d5f50c605bdb0e3b13f44c86660c8) * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/local/LocalContainerAllocator.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/local/TestLocalContainerAllocator.java * hadoop-mapreduce-project/CHANGES.txt > Uber jobs fail to update AMRM token when it rolls over > -- > > Key: MAPREDUCE-6324 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6324 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mr-am >Affects Versions: 2.6.0 >Reporter: Jason Lowe >Assignee: Jason Lowe >Priority: Blocker > Fix For: 2.7.1 > > Attachments: MAPREDUCE-6324.001.patch, MAPREDUCE-6324.002.patch > > > When the RM rolls a new AMRM master key the AMs are supposed to receive a new > AMRM token on subsequent heartbeats between the time when the new key is > rolled and when it is activated. This is not occurring for uber jobs. If > the connection to the RM needs to be re-established after the new key is > activated (e.g.: RM restart or network hiccup) then the uber job AM will be > unable to reconnect to the RM. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6324) Uber jobs fail to update AMRM token when it rolls over
[ https://issues.apache.org/jira/browse/MAPREDUCE-6324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14517162#comment-14517162 ] Hudson commented on MAPREDUCE-6324: --- FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #178 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/178/]) MAPREDUCE-6324. Fixed MapReduce uber jobs to not fail the udpate of AM-RM tokens when they roll-over. Contributed by Jason Lowe. (vinodkv: rev 9fc32c5c4d1d5f50c605bdb0e3b13f44c86660c8) * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/local/LocalContainerAllocator.java * hadoop-mapreduce-project/CHANGES.txt * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/local/TestLocalContainerAllocator.java > Uber jobs fail to update AMRM token when it rolls over > -- > > Key: MAPREDUCE-6324 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6324 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mr-am >Affects Versions: 2.6.0 >Reporter: Jason Lowe >Assignee: Jason Lowe >Priority: Blocker > Fix For: 2.7.1 > > Attachments: MAPREDUCE-6324.001.patch, MAPREDUCE-6324.002.patch > > > When the RM rolls a new AMRM master key the AMs are supposed to receive a new > AMRM token on subsequent heartbeats between the time when the new key is > rolled and when it is activated. This is not occurring for uber jobs. If > the connection to the RM needs to be re-established after the new key is > activated (e.g.: RM restart or network hiccup) then the uber job AM will be > unable to reconnect to the RM. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6324) Uber jobs fail to update AMRM token when it rolls over
[ https://issues.apache.org/jira/browse/MAPREDUCE-6324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14517230#comment-14517230 ] Hudson commented on MAPREDUCE-6324: --- SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2127 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2127/]) MAPREDUCE-6324. Fixed MapReduce uber jobs to not fail the udpate of AM-RM tokens when they roll-over. Contributed by Jason Lowe. (vinodkv: rev 9fc32c5c4d1d5f50c605bdb0e3b13f44c86660c8) * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/local/LocalContainerAllocator.java * hadoop-mapreduce-project/CHANGES.txt * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/local/TestLocalContainerAllocator.java > Uber jobs fail to update AMRM token when it rolls over > -- > > Key: MAPREDUCE-6324 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6324 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mr-am >Affects Versions: 2.6.0 >Reporter: Jason Lowe >Assignee: Jason Lowe >Priority: Blocker > Fix For: 2.7.1 > > Attachments: MAPREDUCE-6324.001.patch, MAPREDUCE-6324.002.patch > > > When the RM rolls a new AMRM master key the AMs are supposed to receive a new > AMRM token on subsequent heartbeats between the time when the new key is > rolled and when it is activated. This is not occurring for uber jobs. If > the connection to the RM needs to be re-established after the new key is > activated (e.g.: RM restart or network hiccup) then the uber job AM will be > unable to reconnect to the RM. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
