[jira] Updated: (MAPREDUCE-2249) Better to check the reflexive property of the object while overriding equals method of it
[ https://issues.apache.org/jira/browse/MAPREDUCE-2249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bhallamudi Venkata Siva Kamesh updated MAPREDUCE-2249: -- Attachment: MAPREDUCE-2249-1.patch Better to check the reflexive property of the object while overriding equals method of it - Key: MAPREDUCE-2249 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2249 Project: Hadoop Map/Reduce Issue Type: Improvement Components: jobtracker, tasktracker Affects Versions: 0.21.0 Environment: NA Reporter: Bhallamudi Venkata Siva Kamesh Attachments: MAPREDUCE-2249-1.patch Original Estimate: 48h Remaining Estimate: 48h It is better to check the reflexive property of the object while overriding equals method of it. It improves the performance when a heavy object is compared to itself. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-2249) Better to check the reflexive property of the object while overriding equals method of it
[ https://issues.apache.org/jira/browse/MAPREDUCE-2249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bhallamudi Venkata Siva Kamesh updated MAPREDUCE-2249: -- Affects Version/s: (was: 0.21.0) 0.22.0 Status: Patch Available (was: Open) Added reflexive check in the equals method. Better to check the reflexive property of the object while overriding equals method of it - Key: MAPREDUCE-2249 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2249 Project: Hadoop Map/Reduce Issue Type: Improvement Components: jobtracker, tasktracker Affects Versions: 0.22.0 Environment: NA Reporter: Bhallamudi Venkata Siva Kamesh Attachments: MAPREDUCE-2249-1.patch Original Estimate: 48h Remaining Estimate: 48h It is better to check the reflexive property of the object while overriding equals method of it. It improves the performance when a heavy object is compared to itself. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-2271) TestSetupTaskScheduling failing in trunk
[ https://issues.apache.org/jira/browse/MAPREDUCE-2271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12985881#action_12985881 ] Todd Lipcon commented on MAPREDUCE-2271: Sorry if I'm being dense, but I don't quite follow the logic of this test. Could you help me understand what's supposed to be going on here? It seems createJob() already makes the two cleanup tasks with status as FAILED_UNCLEAN from that tracker. Is the change in MAPREDUCE-2207 only such that the cleanup-tasks won't be rescheduled in response to the same heartbeat that marked them failed? So in this test we get them scheduled because it's a new heartbeat with no task statuses, even though it previously had failed on that tracker? TestSetupTaskScheduling failing in trunk Key: MAPREDUCE-2271 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2271 Project: Hadoop Map/Reduce Issue Type: Bug Components: jobtracker Affects Versions: 0.23.0 Reporter: Todd Lipcon Priority: Blocker Attachments: 2271-1.diff This test case is failing in trunk after the commit of MAPREDUCE-2207 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Assigned: (MAPREDUCE-2271) TestSetupTaskScheduling failing in trunk
[ https://issues.apache.org/jira/browse/MAPREDUCE-2271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon reassigned MAPREDUCE-2271: -- Assignee: Liyin Liang TestSetupTaskScheduling failing in trunk Key: MAPREDUCE-2271 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2271 Project: Hadoop Map/Reduce Issue Type: Bug Components: jobtracker Affects Versions: 0.23.0 Reporter: Todd Lipcon Assignee: Liyin Liang Priority: Blocker Attachments: 2271-1.diff This test case is failing in trunk after the commit of MAPREDUCE-2207 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-2249) Better to check the reflexive property of the object while overriding equals method of it
[ https://issues.apache.org/jira/browse/MAPREDUCE-2249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12985921#action_12985921 ] Todd Lipcon commented on MAPREDUCE-2249: Hi Bhallamudi. Looks like your patch introduces some tab characters. The style for Hadoop is whitespace-only, 2-indent. Could you please reformat the change? Better to check the reflexive property of the object while overriding equals method of it - Key: MAPREDUCE-2249 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2249 Project: Hadoop Map/Reduce Issue Type: Improvement Components: jobtracker, tasktracker Affects Versions: 0.22.0 Environment: NA Reporter: Bhallamudi Venkata Siva Kamesh Attachments: MAPREDUCE-2249-1.patch Original Estimate: 48h Remaining Estimate: 48h It is better to check the reflexive property of the object while overriding equals method of it. It improves the performance when a heavy object is compared to itself. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-2249) Better to check the reflexive property of the object while overriding equals method of it
[ https://issues.apache.org/jira/browse/MAPREDUCE-2249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon updated MAPREDUCE-2249: --- Status: Open (was: Patch Available) Better to check the reflexive property of the object while overriding equals method of it - Key: MAPREDUCE-2249 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2249 Project: Hadoop Map/Reduce Issue Type: Improvement Components: jobtracker, tasktracker Affects Versions: 0.22.0 Environment: NA Reporter: Bhallamudi Venkata Siva Kamesh Attachments: MAPREDUCE-2249-1.patch Original Estimate: 48h Remaining Estimate: 48h It is better to check the reflexive property of the object while overriding equals method of it. It improves the performance when a heavy object is compared to itself. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-2249) Better to check the reflexive property of the object while overriding equals method of it
[
https://issues.apache.org/jira/browse/MAPREDUCE-2249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12985922#action_12985922
]
Owen O'Malley commented on MAPREDUCE-2249:
--
We should also change the check for classes to ensure commutativity. Could you
change your patch to insist on equality between the classes too?. It should
look like:
{code}
if (this == obj) {
return true;
} else if (obj == null || obj.getClass() != getClass()) {
return false;
} else {
...
}
{code}
Better to check the reflexive property of the object while overriding equals
method of it
-
Key: MAPREDUCE-2249
URL: https://issues.apache.org/jira/browse/MAPREDUCE-2249
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: jobtracker, tasktracker
Affects Versions: 0.22.0
Environment: NA
Reporter: Bhallamudi Venkata Siva Kamesh
Attachments: MAPREDUCE-2249-1.patch
Original Estimate: 48h
Remaining Estimate: 48h
It is better to check the reflexive property of the object while overriding
equals method of it.
It improves the performance when a heavy object is compared to itself.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1754) Replace mapred.persmissions.supergroup with an acl : mapreduce.cluster.administrators
[ https://issues.apache.org/jira/browse/MAPREDUCE-1754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12985927#action_12985927 ] Todd Lipcon commented on MAPREDUCE-1754: +1 for putting this in 0.22 if tests still pass. Can someone test-patch it? Replace mapred.persmissions.supergroup with an acl : mapreduce.cluster.administrators - Key: MAPREDUCE-1754 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1754 Project: Hadoop Map/Reduce Issue Type: Bug Components: jobtracker Reporter: Amareshwari Sriramadasu Assignee: Amareshwari Sriramadasu Attachments: patch-1754-1.txt, patch-1754-ydist.txt, patch-1754.txt mapred.permissions.supergroup should be replaced with an acl so that it does not restrict the admins to a single group. See more details on MAPREDUCE-1542. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1382) MRAsyncDiscService should tolerate missing local.dir
[ https://issues.apache.org/jira/browse/MAPREDUCE-1382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12985935#action_12985935 ] Todd Lipcon commented on MAPREDUCE-1382: +1, seems like a good fix. MRAsyncDiscService should tolerate missing local.dir Key: MAPREDUCE-1382 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1382 Project: Hadoop Map/Reduce Issue Type: Improvement Reporter: Scott Chen Assignee: Zheng Shao Attachments: MAPREDUCE-1382.1.patch, MAPREDUCE-1382.2.patch, MAPREDUCE-1382.3.patch, MAPREDUCE-1382.4.patch, MAPREDUCE-1382.5.patch, MAPREDUCE-1382.branch-0.20.on.top.of.MAPREDUCE-1302.1.patch, MAPREDUCE-1382.patch Currently when some of the local.dir do not exist, MRAsyncDiscService will fail. It should only fail when all directories don't work. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-2193) 13 Findbugs warnings on trunk and branch-0.22
[ https://issues.apache.org/jira/browse/MAPREDUCE-2193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12985946#action_12985946 ] Todd Lipcon commented on MAPREDUCE-2193: Hey Harsh, thanks for making the changes. A few more notes: - Holder needs the Apache license and an interface annotation - probably Stable and Private. It should also move into the o.a.h.mapreduce.util package - o.a.h.mapreduce is for public APIs as far as I understand - I'd like someone else to take a look at adding 'synchronized' to getTasks() in addition to me - it's just scary enough that I think an extra reviewer is prudent. 13 Findbugs warnings on trunk and branch-0.22 - Key: MAPREDUCE-2193 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2193 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 0.22.0, 0.23.0 Reporter: Nigel Daley Assignee: Harsh J Chouraria Priority: Blocker Fix For: 0.22.0, 0.23.0 Attachments: findbugsWarnings.html, hadoop-findbugs-report.html, hadoop-findbugs-report.html, mapreduce.trunk.findbugs.r1.diff, mapreduce.trunk.findbugs.r2.diff, mapreduce.trunk.findbugs.r3.diff There are 13 findbugs warnings on trunk. See attached html file. These must be fixed or filtered out to get back to 0 warnings. The OK_FINDBUGS_WARNINGS property in src/test/test-patch.properties should also be set to 0 in the patch that fixes this issue. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-2250) Fix logging in raid code.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ramkumar Vadali updated MAPREDUCE-2250: --- Attachment: MAPREDUCE-2250.2.patch Pulling in a test fix from another jira. Fix logging in raid code. - Key: MAPREDUCE-2250 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2250 Project: Hadoop Map/Reduce Issue Type: Improvement Components: contrib/raid Reporter: Ramkumar Vadali Assignee: Ramkumar Vadali Priority: Trivial Attachments: MAPREDUCE-2250.1.patch, MAPREDUCE-2250.2.patch, MAPREDUCE-2250.patch There are quite a few error messages being logged with a log level of info. That should be fixed to help debugging. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-2250) Fix logging in raid code.
[
https://issues.apache.org/jira/browse/MAPREDUCE-2250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12986049#action_12986049
]
Ramkumar Vadali commented on MAPREDUCE-2250:
TEST RESULTS:
{code}
[exec]
[exec] +1 overall.
[exec]
[exec] +1 @author. The patch does not contain any @author tags.
[exec]
[exec] +1 tests included. The patch appears to include 3 new or
modified tests.
[exec]
[exec] +1 javadoc. The javadoc tool did not generate any warning
messages.
[exec]
[exec] +1 javac. The applied patch does not increase the total number
of javac compiler warnings.
[exec]
[exec] +1 findbugs. The patch does not introduce any new Findbugs
(version 1.3.9) warnings.
[exec]
[exec] +1 release audit. The applied patch does not increase the
total number of release audit warnings.
[exec]
[exec] +1 system test framework. The patch passed system test
framework compile.
[exec]
[exec]
[exec]
[exec]
[exec]
==
[exec]
==
[exec] Finished build.
[exec]
==
[exec]
==
[exec]
[exec]
{code}
{code}
test-junit:
[junit] WARNING: multiple versions of ant detected in path for junit
[junit]
jar:file:/home/rvadali/local/external/ant/lib/ant.jar!/org/apache/tools/ant/Project.class
[junit] and
jar:file:/home/rvadali/.ivy2/cache/ant/ant/jars/ant-1.6.5.jar!/org/apache/tools/ant/Project.class
[junit] Running org.apache.hadoop.hdfs.TestRaidDfs
[junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 498.696 sec
[junit] Running
org.apache.hadoop.hdfs.server.namenode.TestBlockPlacementPolicyRaid
[junit] Tests run: 6, Failures: 0, Errors: 0, Time elapsed: 153.311 sec
[junit] Running org.apache.hadoop.raid.TestBlockFixer
[junit] Tests run: 14, Failures: 0, Errors: 0, Time elapsed: 969.737 sec
[junit] Running org.apache.hadoop.raid.TestDirectoryTraversal
[junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 12.785 sec
[junit] Running org.apache.hadoop.raid.TestErasureCodes
[junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 17.575 sec
[junit] Running org.apache.hadoop.raid.TestGaloisField
[junit] Tests run: 7, Failures: 0, Errors: 0, Time elapsed: 0.295 sec
[junit] Running org.apache.hadoop.raid.TestHarIndexParser
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.038 sec
[junit] Running org.apache.hadoop.raid.TestRaidFilter
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 14.459 sec
[junit] Running org.apache.hadoop.raid.TestRaidHar
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 65.327 sec
[junit] Running org.apache.hadoop.raid.TestRaidNode
[junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 512.67 sec
[junit] Running org.apache.hadoop.raid.TestRaidPurge
[junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 254.251 sec
[junit] Running org.apache.hadoop.raid.TestRaidShell
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 41.865 sec
[junit] Running org.apache.hadoop.raid.TestRaidShellFsck
[junit] Tests run: 11, Failures: 0, Errors: 0, Time elapsed: 257.72 sec
[junit] Running org.apache.hadoop.raid.TestReedSolomonDecoder
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 12.654 sec
[junit] Running org.apache.hadoop.raid.TestReedSolomonEncoder
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 3.779 sec
test:
BUILD SUCCESSFUL
{code}
Fix logging in raid code.
-
Key: MAPREDUCE-2250
URL: https://issues.apache.org/jira/browse/MAPREDUCE-2250
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: contrib/raid
Reporter: Ramkumar Vadali
Assignee: Ramkumar Vadali
Priority: Trivial
Attachments: MAPREDUCE-2250.1.patch, MAPREDUCE-2250.2.patch,
MAPREDUCE-2250.patch
There are quite a few error messages being logged with a log level of info.
That should be fixed to help debugging.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1382) MRAsyncDiscService should tolerate missing local.dir
[ https://issues.apache.org/jira/browse/MAPREDUCE-1382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tom White updated MAPREDUCE-1382: - Resolution: Fixed Fix Version/s: 0.22.0 Hadoop Flags: [Reviewed] Status: Resolved (was: Patch Available) I've just committed this. Thanks, Zheng. MRAsyncDiscService should tolerate missing local.dir Key: MAPREDUCE-1382 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1382 Project: Hadoop Map/Reduce Issue Type: Improvement Reporter: Scott Chen Assignee: Zheng Shao Fix For: 0.22.0 Attachments: MAPREDUCE-1382.1.patch, MAPREDUCE-1382.2.patch, MAPREDUCE-1382.3.patch, MAPREDUCE-1382.4.patch, MAPREDUCE-1382.5.patch, MAPREDUCE-1382.branch-0.20.on.top.of.MAPREDUCE-1302.1.patch, MAPREDUCE-1382.patch Currently when some of the local.dir do not exist, MRAsyncDiscService will fail. It should only fail when all directories don't work. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-2250) Fix logging in raid code.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Chen updated MAPREDUCE-2250: -- Resolution: Fixed Fix Version/s: 0.23.0 Status: Resolved (was: Patch Available) I just committed this. Thanks Ram :) Fix logging in raid code. - Key: MAPREDUCE-2250 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2250 Project: Hadoop Map/Reduce Issue Type: Improvement Components: contrib/raid Reporter: Ramkumar Vadali Assignee: Ramkumar Vadali Priority: Trivial Fix For: 0.23.0 Attachments: MAPREDUCE-2250.1.patch, MAPREDUCE-2250.2.patch, MAPREDUCE-2250.patch There are quite a few error messages being logged with a log level of info. That should be fixed to help debugging. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1382) MRAsyncDiscService should tolerate missing local.dir
[ https://issues.apache.org/jira/browse/MAPREDUCE-1382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12986091#action_12986091 ] Hudson commented on MAPREDUCE-1382: --- Integrated in Hadoop-Mapreduce-trunk-Commit #581 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/581/]) MAPREDUCE-1382. MRAsyncDiscService should tolerate missing local.dir. Contributed by Zheng Shao and tomwhite. MRAsyncDiscService should tolerate missing local.dir Key: MAPREDUCE-1382 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1382 Project: Hadoop Map/Reduce Issue Type: Improvement Reporter: Scott Chen Assignee: Zheng Shao Fix For: 0.22.0 Attachments: MAPREDUCE-1382.1.patch, MAPREDUCE-1382.2.patch, MAPREDUCE-1382.3.patch, MAPREDUCE-1382.4.patch, MAPREDUCE-1382.5.patch, MAPREDUCE-1382.branch-0.20.on.top.of.MAPREDUCE-1302.1.patch, MAPREDUCE-1382.patch Currently when some of the local.dir do not exist, MRAsyncDiscService will fail. It should only fail when all directories don't work. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-2239) BlockPlacementPolicyRaid should call getBlockLocations only when necessary
[ https://issues.apache.org/jira/browse/MAPREDUCE-2239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Chen updated MAPREDUCE-2239: -- Attachment: MAPREDUCE-2239-2.txt BlockPlacementPolicyRaid should call getBlockLocations only when necessary -- Key: MAPREDUCE-2239 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2239 Project: Hadoop Map/Reduce Issue Type: Improvement Components: contrib/raid Affects Versions: 0.23.0 Reporter: Scott Chen Assignee: Scott Chen Fix For: 0.23.0 Attachments: MAPREDUCE-2239-1.txt, MAPREDUCE-2239-2.txt, MAPREDUCE-2239.txt Currently BlockPlacementPolicyRaid calls getBlockLocations for every chooseTarget(). This puts pressure on NameNode. We should avoid calling if this file is not raided or a parity file. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-2239) BlockPlacementPolicyRaid should call getBlockLocations only when necessary
[ https://issues.apache.org/jira/browse/MAPREDUCE-2239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Chen updated MAPREDUCE-2239: -- Status: Open (was: Patch Available) BlockPlacementPolicyRaid should call getBlockLocations only when necessary -- Key: MAPREDUCE-2239 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2239 Project: Hadoop Map/Reduce Issue Type: Improvement Components: contrib/raid Affects Versions: 0.23.0 Reporter: Scott Chen Assignee: Scott Chen Fix For: 0.23.0 Attachments: MAPREDUCE-2239-1.txt, MAPREDUCE-2239-2.txt, MAPREDUCE-2239.txt Currently BlockPlacementPolicyRaid calls getBlockLocations for every chooseTarget(). This puts pressure on NameNode. We should avoid calling if this file is not raided or a parity file. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-2239) BlockPlacementPolicyRaid should call getBlockLocations only when necessary
[ https://issues.apache.org/jira/browse/MAPREDUCE-2239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Chen updated MAPREDUCE-2239: -- Status: Patch Available (was: Open) BlockPlacementPolicyRaid should call getBlockLocations only when necessary -- Key: MAPREDUCE-2239 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2239 Project: Hadoop Map/Reduce Issue Type: Improvement Components: contrib/raid Affects Versions: 0.23.0 Reporter: Scott Chen Assignee: Scott Chen Fix For: 0.23.0 Attachments: MAPREDUCE-2239-1.txt, MAPREDUCE-2239-2.txt, MAPREDUCE-2239.txt Currently BlockPlacementPolicyRaid calls getBlockLocations for every chooseTarget(). This puts pressure on NameNode. We should avoid calling if this file is not raided or a parity file. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-2239) BlockPlacementPolicyRaid should call getBlockLocations only when necessary
[ https://issues.apache.org/jira/browse/MAPREDUCE-2239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12986113#action_12986113 ] Scott Chen commented on MAPREDUCE-2239: --- Rebased BlockPlacementPolicyRaid should call getBlockLocations only when necessary -- Key: MAPREDUCE-2239 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2239 Project: Hadoop Map/Reduce Issue Type: Improvement Components: contrib/raid Affects Versions: 0.23.0 Reporter: Scott Chen Assignee: Scott Chen Fix For: 0.23.0 Attachments: MAPREDUCE-2239-1.txt, MAPREDUCE-2239-2.txt, MAPREDUCE-2239.txt Currently BlockPlacementPolicyRaid calls getBlockLocations for every chooseTarget(). This puts pressure on NameNode. We should avoid calling if this file is not raided or a parity file. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-2238) Undeletable build directories
[ https://issues.apache.org/jira/browse/MAPREDUCE-2238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon updated MAPREDUCE-2238: --- Resolution: Fixed Hadoop Flags: [Reviewed] Status: Resolved (was: Patch Available) Committed to trunk and branch, thanks for review Eli, and thanks to Greg for helping brainstorm. Undeletable build directories -- Key: MAPREDUCE-2238 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2238 Project: Hadoop Map/Reduce Issue Type: Bug Components: build, test Affects Versions: 0.22.0 Reporter: Eli Collins Assignee: Todd Lipcon Priority: Critical Fix For: 0.22.0 Attachments: mapreduce-2238.txt, mapreduce-2238.txt, mapreduce-2238.txt The MR hudson job is failing, looks like it's due to a test chmod'ing a build directory so the checkout can't clean the build dir. https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/549/console Building remotely on hadoop7 hudson.util.IOException2: remote file operation failed: /grid/0/hudson/hudson-slave/workspace/Hadoop-Mapreduce-trunk at hudson.remoting.Channel@2545938c:hadoop7 at hudson.FilePath.act(FilePath.java:749) at hudson.FilePath.act(FilePath.java:735) at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:589) at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:537) at hudson.model.AbstractProject.checkout(AbstractProject.java:1116) at hudson.model.AbstractBuild$AbstractRunner.checkout(AbstractBuild.java:479) at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:411) at hudson.model.Run.run(Run.java:1324) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:139) Caused by: java.io.IOException: Unable to delete /grid/0/hudson/hudson-slave/workspace/Hadoop-Mapreduce-trunk/trunk/build/test/logs/userlogs/job_20101230131139886_0001/attempt_20101230131139886_0001_m_00_0 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-2238) Undeletable build directories
[ https://issues.apache.org/jira/browse/MAPREDUCE-2238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12986169#action_12986169 ] Hudson commented on MAPREDUCE-2238: --- Integrated in Hadoop-Mapreduce-trunk-Commit #583 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/583/]) MAPREDUCE-2238. Fix permissions handling to avoid leaving undeletable directories in local dirs. Contributed by Todd Lipcon Undeletable build directories -- Key: MAPREDUCE-2238 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2238 Project: Hadoop Map/Reduce Issue Type: Bug Components: build, test Affects Versions: 0.22.0 Reporter: Eli Collins Assignee: Todd Lipcon Priority: Critical Fix For: 0.22.0 Attachments: mapreduce-2238.txt, mapreduce-2238.txt, mapreduce-2238.txt The MR hudson job is failing, looks like it's due to a test chmod'ing a build directory so the checkout can't clean the build dir. https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/549/console Building remotely on hadoop7 hudson.util.IOException2: remote file operation failed: /grid/0/hudson/hudson-slave/workspace/Hadoop-Mapreduce-trunk at hudson.remoting.Channel@2545938c:hadoop7 at hudson.FilePath.act(FilePath.java:749) at hudson.FilePath.act(FilePath.java:735) at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:589) at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:537) at hudson.model.AbstractProject.checkout(AbstractProject.java:1116) at hudson.model.AbstractBuild$AbstractRunner.checkout(AbstractBuild.java:479) at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:411) at hudson.model.Run.run(Run.java:1324) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:139) Caused by: java.io.IOException: Unable to delete /grid/0/hudson/hudson-slave/workspace/Hadoop-Mapreduce-trunk/trunk/build/test/logs/userlogs/job_20101230131139886_0001/attempt_20101230131139886_0001_m_00_0 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-981) TaskTracker.purgeJob fails with NPE if the job is partially localized.
[ https://issues.apache.org/jira/browse/MAPREDUCE-981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Priyo Mustafi updated MAPREDUCE-981: Attachment: patch-981-4.txt The current patch is old and doesn't match. Created a new patch (same code). TaskTracker.purgeJob fails with NPE if the job is partially localized. -- Key: MAPREDUCE-981 URL: https://issues.apache.org/jira/browse/MAPREDUCE-981 Project: Hadoop Map/Reduce Issue Type: Bug Components: tasktracker Reporter: Karam Singh Assignee: Amareshwari Sriramadasu Attachments: patch-981-1.txt, patch-981-2.txt, patch-981-3.txt, patch-981-4.txt, patch-981.txt TaskTracker does not cleans up job directory when LinuxTraskController is used with wrong permissions and setup task of job fails -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1562) TestBadRecords fails sometimes
[ https://issues.apache.org/jira/browse/MAPREDUCE-1562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12986254#action_12986254 ] Priyo Mustafi commented on MAPREDUCE-1562: -- Hi, I am not seeing this error in many of the recent Hudson builds. Also I ran it several time without failure. Is it possible this issue is fixed because of other changes? TestBadRecords fails sometimes -- Key: MAPREDUCE-1562 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1562 Project: Hadoop Map/Reduce Issue Type: Bug Components: test Reporter: Vinod Kumar Vavilapalli Attachments: consoleText TestBadRecords.testMapRed fails sometimes. One instance of this was seen by Hudson while testing MAPREDUCE-890: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/342/testReport/. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
