[jira] [Commented] (MAPREDUCE-2364) Shouldn't hold lock on rjob while localizing resources.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13081086#comment-13081086 ] Devaraj Das commented on MAPREDUCE-2364: Subroto, I see a significant difference in the patches attached to MAPREDUCE-2209 and the last one here. I'll need to look at the details but if you have time could you please take a look at the patch attached here and see if this makes sense (given this patch predates the patch on MAPREDUCE-2209; I am sorry that I didn't look at the patch here earlier). > Shouldn't hold lock on rjob while localizing resources. > --- > > Key: MAPREDUCE-2364 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2364 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: tasktracker >Affects Versions: 0.20.203.0 >Reporter: Owen O'Malley >Assignee: Devaraj Das > Fix For: 0.20.204.0 > > Attachments: MAPREDUCE-2364.patch, > no-lock-localize-branch-0.20-security.patch, no-lock-localize-trunk.patch > > > There is a deadlock while localizing resources on the TaskTracker. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2364) Shouldn't hold lock on rjob while localizing resources.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13080732#comment-13080732 ] Subroto Sanyal commented on MAPREDUCE-2364: --- Hi Devraj, MAPREDUCE-2209 also resolves the same issue. MAPREDUCE-2209 targets to solve one more thread blocking. Request you to look into MAPREDUCE-2209 patch. The patch provided in the issue is for 0.23 version. > Shouldn't hold lock on rjob while localizing resources. > --- > > Key: MAPREDUCE-2364 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2364 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: tasktracker >Affects Versions: 0.20.203.0 >Reporter: Owen O'Malley >Assignee: Devaraj Das > Fix For: 0.20.204.0 > > Attachments: MAPREDUCE-2364.patch, > no-lock-localize-branch-0.20-security.patch, no-lock-localize-trunk.patch > > > There is a deadlock while localizing resources on the TaskTracker. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2364) Shouldn't hold lock on rjob while localizing resources.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13078631#comment-13078631 ] Liyin Liang commented on MAPREDUCE-2364: I think this issue is the same with MAPREDUCE-2209. > Shouldn't hold lock on rjob while localizing resources. > --- > > Key: MAPREDUCE-2364 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2364 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: tasktracker >Affects Versions: 0.20.203.0 >Reporter: Owen O'Malley >Assignee: Devaraj Das > Fix For: 0.20.204.0 > > Attachments: MAPREDUCE-2364.patch, > no-lock-localize-branch-0.20-security.patch, no-lock-localize-trunk.patch > > > There is a deadlock while localizing resources on the TaskTracker. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2364) Shouldn't hold lock on rjob while localizing resources.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13044318#comment-13044318 ] Hadoop QA commented on MAPREDUCE-2364: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12481449/no-lock-localize-trunk.patch against trunk revision 1131265. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these core unit tests: org.apache.hadoop.cli.TestMRCLI +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://builds.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/348//testReport/ Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/348//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/348//console This message is automatically generated. > Shouldn't hold lock on rjob while localizing resources. > --- > > Key: MAPREDUCE-2364 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2364 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: tasktracker >Affects Versions: 0.20.203.0 >Reporter: Owen O'Malley >Assignee: Devaraj Das > Fix For: 0.20.203.0 > > Attachments: MAPREDUCE-2364.patch, > no-lock-localize-branch-0.20-security.patch, no-lock-localize-trunk.patch > > > There is a deadlock while localizing resources on the TaskTracker. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2364) Shouldn't hold lock on rjob while localizing resources.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13044287#comment-13044287 ] Binglin Chang commented on MAPREDUCE-2364: -- trunk patch > Shouldn't hold lock on rjob while localizing resources. > --- > > Key: MAPREDUCE-2364 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2364 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: tasktracker >Affects Versions: 0.20.203.0 >Reporter: Owen O'Malley >Assignee: Devaraj Das > Fix For: 0.20.203.0 > > Attachments: MAPREDUCE-2364.patch, > no-lock-localize-branch-0.20-security.patch, no-lock-localize-trunk.patch > > > There is a deadlock while localizing resources on the TaskTracker. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2364) Shouldn't hold lock on rjob while localizing resources.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13044103#comment-13044103 ] Hadoop QA commented on MAPREDUCE-2364: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12481414/no-lock-localize-branch-0.20-security.patch against trunk revision 1131265. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/345//console This message is automatically generated. > Shouldn't hold lock on rjob while localizing resources. > --- > > Key: MAPREDUCE-2364 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2364 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: tasktracker >Affects Versions: 0.20.203.0 >Reporter: Owen O'Malley >Assignee: Devaraj Das > Fix For: 0.20.203.0 > > Attachments: MAPREDUCE-2364.patch, > no-lock-localize-branch-0.20-security.patch > > > There is a deadlock while localizing resources on the TaskTracker. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2364) Shouldn't hold lock on rjob while localizing resources.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13042265#comment-13042265 ] Hadoop QA commented on MAPREDUCE-2364: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12480768/MAPREDUCE-2364.patch against trunk revision 1129771. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these core unit tests: org.apache.hadoop.cli.TestMRCLI -1 contrib tests. The patch failed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://builds.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/328//testReport/ Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/328//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/328//console This message is automatically generated. > Shouldn't hold lock on rjob while localizing resources. > --- > > Key: MAPREDUCE-2364 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2364 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: tasktracker >Affects Versions: 0.20.203.0 >Reporter: Owen O'Malley >Assignee: Devaraj Das > Fix For: 0.20.203.0 > > Attachments: MAPREDUCE-2364.patch > > > There is a deadlock while localizing resources on the TaskTracker. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2364) Shouldn't hold lock on rjob while localizing resources.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13040755#comment-13040755 ] Binglin Chang commented on MAPREDUCE-2364: -- We encounter the same problem, when TaskTracker download & unJar a very big job.jar in localizeJob(), it stops sending heartbeat and web service hangs too. Our solution for this issue is to add a new lock in RunningJob class called localizing. Instead of holding the whole rjob lock, rjob.localizing is locked. > Shouldn't hold lock on rjob while localizing resources. > --- > > Key: MAPREDUCE-2364 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2364 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: tasktracker >Affects Versions: 0.20.203.0 >Reporter: Owen O'Malley >Assignee: Devaraj Das > Fix For: 0.20.203.0 > > > There is a deadlock while localizing resources on the TaskTracker. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
