[jira] [Commented] (MAPREDUCE-4901) JobHistoryEventHandler errors should be fatal
[
https://issues.apache.org/jira/browse/MAPREDUCE-4901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14538233#comment-14538233
]
Hadoop QA commented on MAPREDUCE-4901:
--
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | patch | 0m 1s | The patch file was not named
according to hadoop's naming conventions. Please see
https://wiki.apache.org/hadoop/HowToContribute for instructions. |
| {color:red}-1{color} | patch | 0m 0s | The patch command could not apply
the patch during dryrun. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12562403/MR-4901-trunk.txt |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / b9cebfc |
| Console output |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5702/console |
This message was automatically generated.
> JobHistoryEventHandler errors should be fatal
> -
>
> Key: MAPREDUCE-4901
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4901
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
>Affects Versions: 0.23.0, 2.0.0-alpha
>Reporter: Robert Joseph Evans
>Assignee: Robert Joseph Evans
> Labels: BB2015-05-TBR
> Attachments: MR-4901-trunk.txt
>
>
> To be able to truly fix issues like MAPREDUCE-4819 and MAPREDUCE-4832, we
> need a 2 phase commit where a subsequent AM can be sure that at a specific
> point in time it knows exactly if any tasks/jobs are committing. The job
> history log is already used for similar functionality so we would like to
> reuse this, but we need to be sure that errors while writing out to the job
> history log are now fatal.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-4901) JobHistoryEventHandler errors should be fatal
[ https://issues.apache.org/jira/browse/MAPREDUCE-4901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14538213#comment-14538213 ] Robert Joseph Evans commented on MAPREDUCE-4901: [~mitdesai], I really don't have a lot of time to devote to this right now. If you are willing to wait I can try to take a look at upmerging, but it would probably be faster for someone else to take it if it is important to them. > JobHistoryEventHandler errors should be fatal > - > > Key: MAPREDUCE-4901 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-4901 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0, 2.0.0-alpha >Reporter: Robert Joseph Evans >Assignee: Robert Joseph Evans > Labels: BB2015-05-TBR > Attachments: MR-4901-trunk.txt > > > To be able to truly fix issues like MAPREDUCE-4819 and MAPREDUCE-4832, we > need a 2 phase commit where a subsequent AM can be sure that at a specific > point in time it knows exactly if any tasks/jobs are committing. The job > history log is already used for similar functionality so we would like to > reuse this, but we need to be sure that errors while writing out to the job > history log are now fatal. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-4901) JobHistoryEventHandler errors should be fatal
[ https://issues.apache.org/jira/browse/MAPREDUCE-4901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14535407#comment-14535407 ] Mit Desai commented on MAPREDUCE-4901: -- Looks like a very stale patch. [~revans2], can you see if the patch still applies or can you upload a newer patch so we can get a +1 from HadoopQA? > JobHistoryEventHandler errors should be fatal > - > > Key: MAPREDUCE-4901 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-4901 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0, 2.0.0-alpha >Reporter: Robert Joseph Evans >Assignee: Robert Joseph Evans > Labels: BB2015-05-TBR > Attachments: MR-4901-trunk.txt > > > To be able to truly fix issues like MAPREDUCE-4819 and MAPREDUCE-4832, we > need a 2 phase commit where a subsequent AM can be sure that at a specific > point in time it knows exactly if any tasks/jobs are committing. The job > history log is already used for similar functionality so we would like to > reuse this, but we need to be sure that errors while writing out to the job > history log are now fatal. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-4901) JobHistoryEventHandler errors should be fatal
[
https://issues.apache.org/jira/browse/MAPREDUCE-4901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13971930#comment-13971930
]
Hadoop QA commented on MAPREDUCE-4901:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12562403/MR-4901-trunk.txt
against trunk revision .
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4528//console
This message is automatically generated.
> JobHistoryEventHandler errors should be fatal
> -
>
> Key: MAPREDUCE-4901
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4901
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
>Affects Versions: 0.23.0, 2.0.0-alpha
>Reporter: Robert Joseph Evans
>Assignee: Robert Joseph Evans
> Attachments: MR-4901-trunk.txt
>
>
> To be able to truly fix issues like MAPREDUCE-4819 and MAPREDUCE-4832, we
> need a 2 phase commit where a subsequent AM can be sure that at a specific
> point in time it knows exactly if any tasks/jobs are committing. The job
> history log is already used for similar functionality so we would like to
> reuse this, but we need to be sure that errors while writing out to the job
> history log are now fatal.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (MAPREDUCE-4901) JobHistoryEventHandler errors should be fatal
[
https://issues.apache.org/jira/browse/MAPREDUCE-4901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13837324#comment-13837324
]
Hadoop QA commented on MAPREDUCE-4901:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12562403/MR-4901-trunk.txt
against trunk revision .
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4238//console
This message is automatically generated.
> JobHistoryEventHandler errors should be fatal
> -
>
> Key: MAPREDUCE-4901
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4901
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
>Affects Versions: 0.23.0, 2.0.0-alpha
>Reporter: Robert Joseph Evans
>Assignee: Robert Joseph Evans
> Attachments: MR-4901-trunk.txt
>
>
> To be able to truly fix issues like MAPREDUCE-4819 and MAPREDUCE-4832, we
> need a 2 phase commit where a subsequent AM can be sure that at a specific
> point in time it knows exactly if any tasks/jobs are committing. The job
> history log is already used for similar functionality so we would like to
> reuse this, but we need to be sure that errors while writing out to the job
> history log are now fatal.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (MAPREDUCE-4901) JobHistoryEventHandler errors should be fatal
[
https://issues.apache.org/jira/browse/MAPREDUCE-4901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13539681#comment-13539681
]
Hadoop QA commented on MAPREDUCE-4901:
--
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12562403/MR-4901-trunk.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3174//testReport/
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3174//console
This message is automatically generated.
> JobHistoryEventHandler errors should be fatal
> -
>
> Key: MAPREDUCE-4901
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4901
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
>Affects Versions: 0.23.0, 2.0.0-alpha
>Reporter: Robert Joseph Evans
>Assignee: Robert Joseph Evans
> Attachments: MR-4901-trunk.txt
>
>
> To be able to truly fix issues like MAPREDUCE-4819 and MAPREDUCE-4832, we
> need a 2 phase commit where a subsequent AM can be sure that at a specific
> point in time it knows exactly if any tasks/jobs are committing. The job
> history log is already used for similar functionality so we would like to
> reuse this, but we need to be sure that errors while writing out to the job
> history log are now fatal.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4901) JobHistoryEventHandler errors should be fatal
[ https://issues.apache.org/jira/browse/MAPREDUCE-4901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13539581#comment-13539581 ] Robert Joseph Evans commented on MAPREDUCE-4901: Oh so looking at the code a bit more, it also appears that if an error does happen the thread that writes to the log dies and the event queue then grows without bound, so technically there is a memory leak involved here too. > JobHistoryEventHandler errors should be fatal > - > > Key: MAPREDUCE-4901 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-4901 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0, 2.0.0-alpha >Reporter: Robert Joseph Evans >Assignee: Robert Joseph Evans > > To be able to truly fix issues like MAPREDUCE-4819 and MAPREDUCE-4832, we > need a 2 phase commit where a subsequent AM can be sure that at a specific > point in time it knows exactly if any tasks/jobs are committing. The job > history log is already used for similar functionality so we would like to > reuse this, but we need to be sure that errors while writing out to the job > history log are now fatal. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
