[jira] [Updated] (MAPREDUCE-3872) event handling races in ContainerLauncherImpl and TestContainerLauncher
[
https://issues.apache.org/jira/browse/MAPREDUCE-3872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Siddharth Seth updated MAPREDUCE-3872:
--
Resolution: Fixed
Fix Version/s: 2.0.5-beta
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Committed to trunk and branch-2.
> event handling races in ContainerLauncherImpl and TestContainerLauncher
> ---
>
> Key: MAPREDUCE-3872
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3872
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: client, mrv2
>Affects Versions: 0.23.1
>Reporter: Patrick Hunt
>Assignee: Robert Kanter
> Fix For: 2.0.5-beta
>
> Attachments: MAPREDUCE-3872.2.patch, MAPREDUCE-3872.patch,
> MAPREDUCE-3872.patch, MAPREDUCE-3872.patch
>
>
> TestContainerLauncher is failing intermittently for me.
> {noformat}
> junit.framework.AssertionFailedError: Expected: but was: Expected 22
> but found 21
> at junit.framework.Assert.fail(Assert.java:47)
> at junit.framework.Assert.assertTrue(Assert.java:20)
> at junit.framework.Assert.assertNull(Assert.java:233)
> at junit.framework.Assert.assertNull(Assert.java:226)
> at
> org.apache.hadoop.mapreduce.v2.app.launcher.TestContainerLauncher.testPoolSize(TestContainerLauncher.java:117)
> {noformat}
> Patch momentarily.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3872) event handling races in ContainerLauncherImpl and TestContainerLauncher
[
https://issues.apache.org/jira/browse/MAPREDUCE-3872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Nauroth updated MAPREDUCE-3872:
-
Attachment: MAPREDUCE-3872.2.patch
+1 for the patch.
I'm attaching a slightly rebased version of the same code just so the patch
applies cleanly to current trunk. The race condition around {{allHosts}}
happens pretty easily on Windows and causes frequent failures in
{{TestContainerLauncher}}. I confirmed that this patch fixes the failures on
Windows and also works on Mac.
> event handling races in ContainerLauncherImpl and TestContainerLauncher
> ---
>
> Key: MAPREDUCE-3872
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3872
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: client, mrv2
>Affects Versions: 0.23.1
>Reporter: Patrick Hunt
>Assignee: Robert Kanter
> Attachments: MAPREDUCE-3872.2.patch, MAPREDUCE-3872.patch,
> MAPREDUCE-3872.patch, MAPREDUCE-3872.patch
>
>
> TestContainerLauncher is failing intermittently for me.
> {noformat}
> junit.framework.AssertionFailedError: Expected: but was: Expected 22
> but found 21
> at junit.framework.Assert.fail(Assert.java:47)
> at junit.framework.Assert.assertTrue(Assert.java:20)
> at junit.framework.Assert.assertNull(Assert.java:233)
> at junit.framework.Assert.assertNull(Assert.java:226)
> at
> org.apache.hadoop.mapreduce.v2.app.launcher.TestContainerLauncher.testPoolSize(TestContainerLauncher.java:117)
> {noformat}
> Patch momentarily.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3872) event handling races in ContainerLauncherImpl and TestContainerLauncher
[
https://issues.apache.org/jira/browse/MAPREDUCE-3872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Kanter updated MAPREDUCE-3872:
-
Attachment: MAPREDUCE-3872.patch
I recently ran into the issue and this patch fixed it; uploading refreshed
patch that applies cleanly now.
> event handling races in ContainerLauncherImpl and TestContainerLauncher
> ---
>
> Key: MAPREDUCE-3872
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3872
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: client, mrv2
>Affects Versions: 0.23.1
>Reporter: Patrick Hunt
>Assignee: Robert Kanter
> Attachments: MAPREDUCE-3872.patch, MAPREDUCE-3872.patch,
> MAPREDUCE-3872.patch
>
>
> TestContainerLauncher is failing intermittently for me.
> {noformat}
> junit.framework.AssertionFailedError: Expected: but was: Expected 22
> but found 21
> at junit.framework.Assert.fail(Assert.java:47)
> at junit.framework.Assert.assertTrue(Assert.java:20)
> at junit.framework.Assert.assertNull(Assert.java:233)
> at junit.framework.Assert.assertNull(Assert.java:226)
> at
> org.apache.hadoop.mapreduce.v2.app.launcher.TestContainerLauncher.testPoolSize(TestContainerLauncher.java:117)
> {noformat}
> Patch momentarily.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3872) event handling races in ContainerLauncherImpl and TestContainerLauncher
[
https://issues.apache.org/jira/browse/MAPREDUCE-3872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Patrick Hunt updated MAPREDUCE-3872:
Attachment: MAPREDUCE-3872.patch
Refreshing the patch. Looks like MAPREDUCE-3634 fixed a number of the issues I
had originally seen/fixed in this patch.
The latest version of this patch fixes the obvious concurrency bug in updating
allNodes. This patch is currently tested by the unit tests, I don't see a way
to trigger the bad case given it's non-deterministic. However by inspection you
can see the obvious concurrency bug that exists in the current code.
> event handling races in ContainerLauncherImpl and TestContainerLauncher
> ---
>
> Key: MAPREDUCE-3872
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3872
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: client, mrv2
>Affects Versions: 0.23.1
>Reporter: Patrick Hunt
> Attachments: MAPREDUCE-3872.patch, MAPREDUCE-3872.patch
>
>
> TestContainerLauncher is failing intermittently for me.
> {noformat}
> junit.framework.AssertionFailedError: Expected: but was: Expected 22
> but found 21
> at junit.framework.Assert.fail(Assert.java:47)
> at junit.framework.Assert.assertTrue(Assert.java:20)
> at junit.framework.Assert.assertNull(Assert.java:233)
> at junit.framework.Assert.assertNull(Assert.java:226)
> at
> org.apache.hadoop.mapreduce.v2.app.launcher.TestContainerLauncher.testPoolSize(TestContainerLauncher.java:117)
> {noformat}
> Patch momentarily.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3872) event handling races in ContainerLauncherImpl and TestContainerLauncher
[
https://issues.apache.org/jira/browse/MAPREDUCE-3872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mahadev konar updated MAPREDUCE-3872:
-
Component/s: mrv2
> event handling races in ContainerLauncherImpl and TestContainerLauncher
> ---
>
> Key: MAPREDUCE-3872
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3872
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: client, mrv2
>Affects Versions: 0.23.1
>Reporter: Patrick Hunt
> Attachments: MAPREDUCE-3872.patch
>
>
> TestContainerLauncher is failing intermittently for me.
> {noformat}
> junit.framework.AssertionFailedError: Expected: but was: Expected 22
> but found 21
> at junit.framework.Assert.fail(Assert.java:47)
> at junit.framework.Assert.assertTrue(Assert.java:20)
> at junit.framework.Assert.assertNull(Assert.java:233)
> at junit.framework.Assert.assertNull(Assert.java:226)
> at
> org.apache.hadoop.mapreduce.v2.app.launcher.TestContainerLauncher.testPoolSize(TestContainerLauncher.java:117)
> {noformat}
> Patch momentarily.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3872) event handling races in ContainerLauncherImpl and TestContainerLauncher
[
https://issues.apache.org/jira/browse/MAPREDUCE-3872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Patrick Hunt updated MAPREDUCE-3872:
Description:
TestContainerLauncher is failing intermittently for me.
{noformat}
junit.framework.AssertionFailedError: Expected: but was: Expected 22 but
found 21
at junit.framework.Assert.fail(Assert.java:47)
at junit.framework.Assert.assertTrue(Assert.java:20)
at junit.framework.Assert.assertNull(Assert.java:233)
at junit.framework.Assert.assertNull(Assert.java:226)
at
org.apache.hadoop.mapreduce.v2.app.launcher.TestContainerLauncher.testPoolSize(TestContainerLauncher.java:117)
{noformat}
Patch momentarily.
was:
TestContainerLauncher is failing intermittently for me.
{noformat}
> event handling races in ContainerLauncherImpl and TestContainerLauncher
> ---
>
> Key: MAPREDUCE-3872
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3872
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: client
>Affects Versions: 0.23.1
>Reporter: Patrick Hunt
>
> TestContainerLauncher is failing intermittently for me.
> {noformat}
> junit.framework.AssertionFailedError: Expected: but was: Expected 22
> but found 21
> at junit.framework.Assert.fail(Assert.java:47)
> at junit.framework.Assert.assertTrue(Assert.java:20)
> at junit.framework.Assert.assertNull(Assert.java:233)
> at junit.framework.Assert.assertNull(Assert.java:226)
> at
> org.apache.hadoop.mapreduce.v2.app.launcher.TestContainerLauncher.testPoolSize(TestContainerLauncher.java:117)
> {noformat}
> Patch momentarily.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3872) event handling races in ContainerLauncherImpl and TestContainerLauncher
[
https://issues.apache.org/jira/browse/MAPREDUCE-3872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Patrick Hunt updated MAPREDUCE-3872:
Status: Patch Available (was: Open)
> event handling races in ContainerLauncherImpl and TestContainerLauncher
> ---
>
> Key: MAPREDUCE-3872
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3872
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: client
>Affects Versions: 0.23.1
>Reporter: Patrick Hunt
> Attachments: MAPREDUCE-3872.patch
>
>
> TestContainerLauncher is failing intermittently for me.
> {noformat}
> junit.framework.AssertionFailedError: Expected: but was: Expected 22
> but found 21
> at junit.framework.Assert.fail(Assert.java:47)
> at junit.framework.Assert.assertTrue(Assert.java:20)
> at junit.framework.Assert.assertNull(Assert.java:233)
> at junit.framework.Assert.assertNull(Assert.java:226)
> at
> org.apache.hadoop.mapreduce.v2.app.launcher.TestContainerLauncher.testPoolSize(TestContainerLauncher.java:117)
> {noformat}
> Patch momentarily.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3872) event handling races in ContainerLauncherImpl and TestContainerLauncher
[
https://issues.apache.org/jira/browse/MAPREDUCE-3872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Patrick Hunt updated MAPREDUCE-3872:
Attachment: MAPREDUCE-3872.patch
This patch fixes three issues:
1) ContainerLauncherImpl had a race in updating "allNodes". This might result
in the pool size being incorrectly calculated.
2) test had an issues where waitForEvent could run forever
3) there's a race btw eventHandlingThread in ContainerLauncherImpl and the test
code. In some cases the event would be processed, and the pool size increased,
before all events had been processed (and allNodes updated with all new hosts).
I believe this is the result of the test failure I documented in the
description.
> event handling races in ContainerLauncherImpl and TestContainerLauncher
> ---
>
> Key: MAPREDUCE-3872
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3872
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: client
>Affects Versions: 0.23.1
>Reporter: Patrick Hunt
> Attachments: MAPREDUCE-3872.patch
>
>
> TestContainerLauncher is failing intermittently for me.
> {noformat}
> junit.framework.AssertionFailedError: Expected: but was: Expected 22
> but found 21
> at junit.framework.Assert.fail(Assert.java:47)
> at junit.framework.Assert.assertTrue(Assert.java:20)
> at junit.framework.Assert.assertNull(Assert.java:233)
> at junit.framework.Assert.assertNull(Assert.java:226)
> at
> org.apache.hadoop.mapreduce.v2.app.launcher.TestContainerLauncher.testPoolSize(TestContainerLauncher.java:117)
> {noformat}
> Patch momentarily.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
