[jira] [Updated] (MAPREDUCE-2425) Distributed simulator for stressing JobTracker and NameNode
[ https://issues.apache.org/jira/browse/MAPREDUCE-2425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Min Zhou updated MAPREDUCE-2425: Component/s: (was: benchmarks) contrib/mumak > Distributed simulator for stressing JobTracker and NameNode > --- > > Key: MAPREDUCE-2425 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2425 > Project: Hadoop Map/Reduce > Issue Type: New Feature > Components: contrib/mumak >Reporter: Min Zhou > Labels: benchmark, hadoop > Fix For: 0.22.0 > > Attachments: .jpg, screenshot-1.jpg > > > Hadoop need a tool for stressing JobTracker and NameNode. Mumak introduced a > simulated JobTracker, whose behavior doesn't exactly like that of the real > JobTracker. Even more, mumak can't simulate a large cluster with quite a lot > of jobs run on it. On the other hand, Gridmix v3 need hundreds of physical > nodes to replay job stories. > You can think this tool a complementation of mumak and gridmix v3. We > successfully used this tool to simulate a 12000 nodes cluster through 4 real > machines. > I've talk to Hong Tang and Scott Chen offline, they suggested me contributing > this tool to the hadoop community. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2425) Distributed simulator for stressing JobTracker and NameNode
[ https://issues.apache.org/jira/browse/MAPREDUCE-2425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13017315#comment-13017315 ] Min Zhou commented on MAPREDUCE-2425: - btw, this tool can stress RPC as well. > Distributed simulator for stressing JobTracker and NameNode > --- > > Key: MAPREDUCE-2425 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2425 > Project: Hadoop Map/Reduce > Issue Type: New Feature > Components: benchmarks >Reporter: Min Zhou > Labels: benchmark, hadoop > Fix For: 0.22.0 > > Attachments: .jpg, screenshot-1.jpg > > > Hadoop need a tool for stressing JobTracker and NameNode. Mumak introduced a > simulated JobTracker, whose behavior doesn't exactly like that of the real > JobTracker. Even more, mumak can't simulate a large cluster with quite a lot > of jobs run on it. On the other hand, Gridmix v3 need hundreds of physical > nodes to replay job stories. > You can think this tool a complementation of mumak and gridmix v3. We > successfully used this tool to simulate a 12000 nodes cluster through 4 real > machines. > I've talk to Hong Tang and Scott Chen offline, they suggested me contributing > this tool to the hadoop community. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2425) Distributed simulator for stressing JobTracker and NameNode
[ https://issues.apache.org/jira/browse/MAPREDUCE-2425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13017312#comment-13017312 ] Min Zhou commented on MAPREDUCE-2425: - Amar, see MAPREDUCE-1261. 1. Mumak couldn't stress JT. It's JobClient, SimulatorJobTracker & SimulatorTaskTrackers are all run in the same JVM. 2. Mumak doesn't use the real JT. > Distributed simulator for stressing JobTracker and NameNode > --- > > Key: MAPREDUCE-2425 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2425 > Project: Hadoop Map/Reduce > Issue Type: New Feature > Components: benchmarks >Reporter: Min Zhou > Labels: benchmark, hadoop > Fix For: 0.22.0 > > Attachments: .jpg, screenshot-1.jpg > > > Hadoop need a tool for stressing JobTracker and NameNode. Mumak introduced a > simulated JobTracker, whose behavior doesn't exactly like that of the real > JobTracker. Even more, mumak can't simulate a large cluster with quite a lot > of jobs run on it. On the other hand, Gridmix v3 need hundreds of physical > nodes to replay job stories. > You can think this tool a complementation of mumak and gridmix v3. We > successfully used this tool to simulate a 12000 nodes cluster through 4 real > machines. > I've talk to Hong Tang and Scott Chen offline, they suggested me contributing > this tool to the hadoop community. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2425) Distributed simulator for stressing JobTracker and NameNode
[ https://issues.apache.org/jira/browse/MAPREDUCE-2425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13017296#comment-13017296 ] Amar Kamat commented on MAPREDUCE-2425: --- Min, Thanks for the quick reply. I guess testing (stress/functional) JT is also the primary goal of Mumak. Hence I see this as an enhancement (maybe rewrite) of Mumak. So the goal is to support existing functionality of Mumak and add new features to support the use cases that you are interested in. bq. Mumak uses a simulated JT.. I guess Mumak only wanted to work with certain features of JT and hence the design. I believe we can enhance Mumak to instantiate the "real" JT if needed. bq. I should uses new MR API before merging into mumak. Yes. For now, can you quickly point out the major highlights of the simulator that you are planning to contribute? It would be nice to also compare/contrast it with Mumak. > Distributed simulator for stressing JobTracker and NameNode > --- > > Key: MAPREDUCE-2425 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2425 > Project: Hadoop Map/Reduce > Issue Type: New Feature > Components: benchmarks >Reporter: Min Zhou > Labels: benchmark, hadoop > Fix For: 0.22.0 > > Attachments: .jpg, screenshot-1.jpg > > > Hadoop need a tool for stressing JobTracker and NameNode. Mumak introduced a > simulated JobTracker, whose behavior doesn't exactly like that of the real > JobTracker. Even more, mumak can't simulate a large cluster with quite a lot > of jobs run on it. On the other hand, Gridmix v3 need hundreds of physical > nodes to replay job stories. > You can think this tool a complementation of mumak and gridmix v3. We > successfully used this tool to simulate a 12000 nodes cluster through 4 real > machines. > I've talk to Hong Tang and Scott Chen offline, they suggested me contributing > this tool to the hadoop community. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2425) Distributed simulator for stressing JobTracker and NameNode
[ https://issues.apache.org/jira/browse/MAPREDUCE-2425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13017273#comment-13017273 ] Min Zhou commented on MAPREDUCE-2425: - Amar, This simulator is developed majorly for stressing JT and NN. It can also verify JT's runtime behavior as mumak does. Actually, we use v0.19.1 where rumen and mumak havenot been introduced at that time, so I developed this tool independent from them. Now that I am planning to merge my code into mumak. But before that, I must do 2 things list below 1. Mumak uses a simulated JT for telling TT some informations of a task-attempt reproduced by rumen through heartbeat. I perfer using the real JT when stressing it. 2. I should uses new MR API before merging into mumak. > Distributed simulator for stressing JobTracker and NameNode > --- > > Key: MAPREDUCE-2425 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2425 > Project: Hadoop Map/Reduce > Issue Type: New Feature > Components: benchmarks >Reporter: Min Zhou > Labels: benchmark, hadoop > Fix For: 0.22.0 > > Attachments: .jpg, screenshot-1.jpg > > > Hadoop need a tool for stressing JobTracker and NameNode. Mumak introduced a > simulated JobTracker, whose behavior doesn't exactly like that of the real > JobTracker. Even more, mumak can't simulate a large cluster with quite a lot > of jobs run on it. On the other hand, Gridmix v3 need hundreds of physical > nodes to replay job stories. > You can think this tool a complementation of mumak and gridmix v3. We > successfully used this tool to simulate a 12000 nodes cluster through 4 real > machines. > I've talk to Hong Tang and Scott Chen offline, they suggested me contributing > this tool to the hadoop community. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2337) Remove dependence of public MapReduce API on classes in server package
[ https://issues.apache.org/jira/browse/MAPREDUCE-2337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13017268#comment-13017268 ] Hudson commented on MAPREDUCE-2337: --- Integrated in Hadoop-Mapreduce-trunk-Commit #639 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/639/]) MAPREDUCE-2337. Remove dependence of public MapReduce API on classes in server package. > Remove dependence of public MapReduce API on classes in server package > -- > > Key: MAPREDUCE-2337 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2337 > Project: Hadoop Map/Reduce > Issue Type: Improvement >Reporter: Tom White >Assignee: Tom White > Fix For: 0.22.0 > > Attachments: MAPREDUCE-2337.patch, MAPREDUCE-2337.patch, > MAPREDUCE-2337.patch, MAPREDUCE-2337.patch > > > Cluster#getJobTrackerState() returns a > org.apache.hadoop.mapreduce.server.jobtracker.State enum, which makes the API > in o.a.h.mapreduce have a dependency on the server package. It would be > better to make the public API self-contained by using an equivalent enum in > the Cluster class. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2337) Remove dependence of public MapReduce API on classes in server package
[ https://issues.apache.org/jira/browse/MAPREDUCE-2337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tom White updated MAPREDUCE-2337: - Resolution: Fixed Fix Version/s: 0.22.0 Hadoop Flags: [Reviewed] Status: Resolved (was: Patch Available) I've just committed this. > Remove dependence of public MapReduce API on classes in server package > -- > > Key: MAPREDUCE-2337 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2337 > Project: Hadoop Map/Reduce > Issue Type: Improvement >Reporter: Tom White >Assignee: Tom White > Fix For: 0.22.0 > > Attachments: MAPREDUCE-2337.patch, MAPREDUCE-2337.patch, > MAPREDUCE-2337.patch, MAPREDUCE-2337.patch > > > Cluster#getJobTrackerState() returns a > org.apache.hadoop.mapreduce.server.jobtracker.State enum, which makes the API > in o.a.h.mapreduce have a dependency on the server package. It would be > better to make the public API self-contained by using an equivalent enum in > the Cluster class. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2425) Distributed simulator for stressing JobTracker and NameNode
[ https://issues.apache.org/jira/browse/MAPREDUCE-2425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13017266#comment-13017266 ] Amar Kamat commented on MAPREDUCE-2425: --- Min, Can you kindly post some more details as to what this new simulation intends to do (as opposed to Mumak) and the basic design? Also is it possible to modify Mumak to do what your tool is doing. If the only difference is about scalability then I would prefer fixing Mumak and making it better. Thoughts? > Distributed simulator for stressing JobTracker and NameNode > --- > > Key: MAPREDUCE-2425 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2425 > Project: Hadoop Map/Reduce > Issue Type: New Feature > Components: benchmarks >Reporter: Min Zhou > Labels: benchmark, hadoop > Fix For: 0.22.0 > > Attachments: .jpg, screenshot-1.jpg > > > Hadoop need a tool for stressing JobTracker and NameNode. Mumak introduced a > simulated JobTracker, whose behavior doesn't exactly like that of the real > JobTracker. Even more, mumak can't simulate a large cluster with quite a lot > of jobs run on it. On the other hand, Gridmix v3 need hundreds of physical > nodes to replay job stories. > You can think this tool a complementation of mumak and gridmix v3. We > successfully used this tool to simulate a 12000 nodes cluster through 4 real > machines. > I've talk to Hong Tang and Scott Chen offline, they suggested me contributing > this tool to the hadoop community. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2425) Distributed simulator for stressing JobTracker and NameNode
[ https://issues.apache.org/jira/browse/MAPREDUCE-2425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Min Zhou updated MAPREDUCE-2425: Attachment: screenshot-1.jpg screenshot > Distributed simulator for stressing JobTracker and NameNode > --- > > Key: MAPREDUCE-2425 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2425 > Project: Hadoop Map/Reduce > Issue Type: New Feature > Components: benchmarks >Reporter: Min Zhou > Labels: benchmark, hadoop > Fix For: 0.22.0 > > Attachments: .jpg, screenshot-1.jpg > > > Hadoop need a tool for stressing JobTracker and NameNode. Mumak introduced a > simulated JobTracker, whose behavior doesn't exactly like that of the real > JobTracker. Even more, mumak can't simulate a large cluster with quite a lot > of jobs run on it. On the other hand, Gridmix v3 need hundreds of physical > nodes to replay job stories. > You can think this tool a complementation of mumak and gridmix v3. We > successfully used this tool to simulate a 12000 nodes cluster through 4 real > machines. > I've talk to Hong Tang and Scott Chen offline, they suggested me contributing > this tool to the hadoop community. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2425) Distributed simulator for stressing JobTracker and NameNode
[ https://issues.apache.org/jira/browse/MAPREDUCE-2425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Min Zhou updated MAPREDUCE-2425: Attachment: .jpg A screenshot of this tool. we are using hadoop 0.19.1 > Distributed simulator for stressing JobTracker and NameNode > --- > > Key: MAPREDUCE-2425 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2425 > Project: Hadoop Map/Reduce > Issue Type: New Feature > Components: benchmarks >Reporter: Min Zhou > Labels: benchmark, hadoop > Fix For: 0.22.0 > > Attachments: .jpg > > > Hadoop need a tool for stressing JobTracker and NameNode. Mumak introduced a > simulated JobTracker, whose behavior doesn't exactly like that of the real > JobTracker. Even more, mumak can't simulate a large cluster with quite a lot > of jobs run on it. On the other hand, Gridmix v3 need hundreds of physical > nodes to replay job stories. > You can think this tool a complementation of mumak and gridmix v3. We > successfully used this tool to simulate a 12000 nodes cluster through 4 real > machines. > I've talk to Hong Tang and Scott Chen offline, they suggested me contributing > this tool to the hadoop community. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-2425) Distributed simulator for stressing JobTracker and NameNode
Distributed simulator for stressing JobTracker and NameNode --- Key: MAPREDUCE-2425 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2425 Project: Hadoop Map/Reduce Issue Type: New Feature Components: benchmarks Reporter: Min Zhou Fix For: 0.22.0 Hadoop need a tool for stressing JobTracker and NameNode. Mumak introduced a simulated JobTracker, whose behavior doesn't exactly like that of the real JobTracker. Even more, mumak can't simulate a large cluster with quite a lot of jobs run on it. On the other hand, Gridmix v3 need hundreds of physical nodes to replay job stories. You can think this tool a complementation of mumak and gridmix v3. We successfully used this tool to simulate a 12000 nodes cluster through 4 real machines. I've talk to Hong Tang and Scott Chen offline, they suggested me contributing this tool to the hadoop community. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2424) MR-279: counters/UI/etc. for uber-AppMaster (in-cluster LocalJobRunner for MRv2)
[
https://issues.apache.org/jira/browse/MAPREDUCE-2424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Greg Roelofs updated MAPREDUCE-2424:
Description:
Polish uber-AM (MAPREDUCE-2405). Specifically:
* uber-specific counters ("command-line UI")
* GUI indicators
** RM all-containers level
** multi-job app level [if exists]
** single-job level
* fix uber-decision ("is this a small job?"):
** memory criterion
** input-bytes criterion
* disable speculation
* isUber() method (somewhere) for unit tests to use
* delete (most of) old UberTask code (MAPREDUCE-1220; came in with initial
MR-279 branch)
* implement non-RPC, local version of umbilical
* AM restart (default 4 tries) on another node on any task-attempt failure
* uber-specific metrics?
* rename configurables? (still "ubertask"-based)
was:"Port" MAPREDUCE-1220 to MRv2. This is an optimization for small jobs
wherein all tasks run on the same node in the same JVM/container.
> MR-279: counters/UI/etc. for uber-AppMaster (in-cluster LocalJobRunner for
> MRv2)
>
>
> Key: MAPREDUCE-2424
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2424
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
>Reporter: Greg Roelofs
>Assignee: Greg Roelofs
> Fix For: 0.23.0
>
>
> Polish uber-AM (MAPREDUCE-2405). Specifically:
> * uber-specific counters ("command-line UI")
> * GUI indicators
> ** RM all-containers level
> ** multi-job app level [if exists]
> ** single-job level
> * fix uber-decision ("is this a small job?"):
> ** memory criterion
> ** input-bytes criterion
> * disable speculation
> * isUber() method (somewhere) for unit tests to use
> * delete (most of) old UberTask code (MAPREDUCE-1220; came in with initial
> MR-279 branch)
> * implement non-RPC, local version of umbilical
> * AM restart (default 4 tries) on another node on any task-attempt failure
> * uber-specific metrics?
> * rename configurables? (still "ubertask"-based)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-2424) MR-279: counters/UI/etc. for uber-AppMaster (in-cluster LocalJobRunner for MRv2)
MR-279: counters/UI/etc. for uber-AppMaster (in-cluster LocalJobRunner for MRv2) Key: MAPREDUCE-2424 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2424 Project: Hadoop Map/Reduce Issue Type: Improvement Components: mrv2 Reporter: Greg Roelofs Assignee: Greg Roelofs Fix For: 0.23.0 "Port" MAPREDUCE-1220 to MRv2. This is an optimization for small jobs wherein all tasks run on the same node in the same JVM/container. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2405) MR-279: Implement uber-AppMaster (in-cluster LocalJobRunner for MRv2)
[ https://issues.apache.org/jira/browse/MAPREDUCE-2405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13017249#comment-13017249 ] Greg Roelofs commented on MAPREDUCE-2405: - Thanks, Mahadev! You're clearly a person of outstanding character, intelligence, and better-than-average looks. :-) (I'll just note here that there's further work/fixes to be done, particularly including counters, UI, the memory and input-size uber-decision criteria, and AM-restart on task-attempt failures, but I'll file a follow-up JIRA for that.) > MR-279: Implement uber-AppMaster (in-cluster LocalJobRunner for MRv2) > - > > Key: MAPREDUCE-2405 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2405 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: mrv2 >Reporter: Mahadev konar >Assignee: Greg Roelofs > Fix For: 0.23.0 > > Attachments: MR-2405-MR-1220-yarn.v10.MR-279-hadoop-yarn.patch.txt, > MR-2405-MR-1220-yarn.v11.MR-279-hadoop-yarn.patch.txt, > MR-2405-MR-1220-yarn.v8.MR-279-hadoop-yarn.patch.txt > > > "Port" MAPREDUCE-1220 to MRv2. This is an optimization for small jobs > wherein all tasks run on the same node in the same JVM/container. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (MAPREDUCE-2405) MR-279: Implement uber-AppMaster (in-cluster LocalJobRunner for MRv2)
[ https://issues.apache.org/jira/browse/MAPREDUCE-2405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar resolved MAPREDUCE-2405. -- Resolution: Fixed I just pushed this to MR-279 branch. thanks greg. > MR-279: Implement uber-AppMaster (in-cluster LocalJobRunner for MRv2) > - > > Key: MAPREDUCE-2405 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2405 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: mrv2 >Reporter: Mahadev konar >Assignee: Greg Roelofs > Fix For: 0.23.0 > > Attachments: MR-2405-MR-1220-yarn.v10.MR-279-hadoop-yarn.patch.txt, > MR-2405-MR-1220-yarn.v11.MR-279-hadoop-yarn.patch.txt, > MR-2405-MR-1220-yarn.v8.MR-279-hadoop-yarn.patch.txt > > > "Port" MAPREDUCE-1220 to MRv2. This is an optimization for small jobs > wherein all tasks run on the same node in the same JVM/container. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2337) Remove dependence of public MapReduce API on classes in server package
[ https://issues.apache.org/jira/browse/MAPREDUCE-2337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13017220#comment-13017220 ] Hadoop QA commented on MAPREDUCE-2337: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12475752/MAPREDUCE-2337.patch against trunk revision 1089686. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 12 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. -1 contrib tests. The patch failed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://hudson.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/158//testReport/ Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/158//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://hudson.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/158//console This message is automatically generated. > Remove dependence of public MapReduce API on classes in server package > -- > > Key: MAPREDUCE-2337 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2337 > Project: Hadoop Map/Reduce > Issue Type: Improvement >Reporter: Tom White >Assignee: Tom White > Attachments: MAPREDUCE-2337.patch, MAPREDUCE-2337.patch, > MAPREDUCE-2337.patch, MAPREDUCE-2337.patch > > > Cluster#getJobTrackerState() returns a > org.apache.hadoop.mapreduce.server.jobtracker.State enum, which makes the API > in o.a.h.mapreduce have a dependency on the server package. It would be > better to make the public API self-contained by using an equivalent enum in > the Cluster class. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2337) Remove dependence of public MapReduce API on classes in server package
[ https://issues.apache.org/jira/browse/MAPREDUCE-2337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13017183#comment-13017183 ] Todd Lipcon commented on MAPREDUCE-2337: +1 > Remove dependence of public MapReduce API on classes in server package > -- > > Key: MAPREDUCE-2337 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2337 > Project: Hadoop Map/Reduce > Issue Type: Improvement >Reporter: Tom White >Assignee: Tom White > Attachments: MAPREDUCE-2337.patch, MAPREDUCE-2337.patch, > MAPREDUCE-2337.patch, MAPREDUCE-2337.patch > > > Cluster#getJobTrackerState() returns a > org.apache.hadoop.mapreduce.server.jobtracker.State enum, which makes the API > in o.a.h.mapreduce have a dependency on the server package. It would be > better to make the public API self-contained by using an equivalent enum in > the Cluster class. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2337) Remove dependence of public MapReduce API on classes in server package
[ https://issues.apache.org/jira/browse/MAPREDUCE-2337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tom White updated MAPREDUCE-2337: - Attachment: MAPREDUCE-2337.patch Fixed. Thanks for the review. > Remove dependence of public MapReduce API on classes in server package > -- > > Key: MAPREDUCE-2337 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2337 > Project: Hadoop Map/Reduce > Issue Type: Improvement >Reporter: Tom White >Assignee: Tom White > Attachments: MAPREDUCE-2337.patch, MAPREDUCE-2337.patch, > MAPREDUCE-2337.patch, MAPREDUCE-2337.patch > > > Cluster#getJobTrackerState() returns a > org.apache.hadoop.mapreduce.server.jobtracker.State enum, which makes the API > in o.a.h.mapreduce have a dependency on the server package. It would be > better to make the public API self-contained by using an equivalent enum in > the Cluster class. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2422) Removed unused internal methods from DistributedCache
[ https://issues.apache.org/jira/browse/MAPREDUCE-2422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13017151#comment-13017151 ] Hadoop QA commented on MAPREDUCE-2422: -- +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12475726/MAPREDUCE-2422.patch against trunk revision 1089686. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 6 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://hudson.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/157//testReport/ Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/157//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://hudson.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/157//console This message is automatically generated. > Removed unused internal methods from DistributedCache > - > > Key: MAPREDUCE-2422 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2422 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: client >Reporter: Tom White >Assignee: Tom White > Attachments: MAPREDUCE-2422.patch, MAPREDUCE-2422.patch > > > DistributedCache has a number of deprecated methods that are no longer used > ever since TrackerDistributedCacheManager was introduced in MAPREDUCE-476. > Removing these methods (which are not user-facing) will make it possible to > complete MAPREDUCE-1638 by keeping DistributedCache in the API tree, and > TrackerDistributedCacheManager, TaskDistributedCacheManager in the > implementation tree. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2337) Remove dependence of public MapReduce API on classes in server package
[ https://issues.apache.org/jira/browse/MAPREDUCE-2337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13017103#comment-13017103 ] Todd Lipcon commented on MAPREDUCE-2337: In o.a.h.mapreduce.Cluster#getJobTrackerStatus(), can you delegate that directly to client.getJobTrackerStatus(), rather than calling the now-deprecated getJobTrackerState()? > Remove dependence of public MapReduce API on classes in server package > -- > > Key: MAPREDUCE-2337 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2337 > Project: Hadoop Map/Reduce > Issue Type: Improvement >Reporter: Tom White >Assignee: Tom White > Attachments: MAPREDUCE-2337.patch, MAPREDUCE-2337.patch, > MAPREDUCE-2337.patch > > > Cluster#getJobTrackerState() returns a > org.apache.hadoop.mapreduce.server.jobtracker.State enum, which makes the API > in o.a.h.mapreduce have a dependency on the server package. It would be > better to make the public API self-contained by using an equivalent enum in > the Cluster class. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Moved] (MAPREDUCE-2423) Monitoring the job tracker ui of hadoop using other open source monitoring tools like Nagios
[ https://issues.apache.org/jira/browse/MAPREDUCE-2423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer moved HDFS-1816 to MAPREDUCE-2423: --- Component/s: (was: scripts) jobtracker Key: MAPREDUCE-2423 (was: HDFS-1816) Project: Hadoop Map/Reduce (was: Hadoop HDFS) > Monitoring the job tracker ui of hadoop using other open source monitoring > tools like Nagios > > > Key: MAPREDUCE-2423 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2423 > Project: Hadoop Map/Reduce > Issue Type: Wish > Components: jobtracker >Reporter: Saurabh Mishra > > I just wish if there is a way I can write monitors to check my hadoop job > tracker UI using my existing Nagios infrastructure. As this would help me in > keeping everything centrally located and hence under manageable limits. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2423) Monitoring the job tracker ui of hadoop using other open source monitoring tools like Nagios
[ https://issues.apache.org/jira/browse/MAPREDUCE-2423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13017052#comment-13017052 ] Allen Wittenauer commented on MAPREDUCE-2423: - (or hitting the JMX ports or ...) Is the request that Hadoop should come with a Nagios plug-in for lazy people? > Monitoring the job tracker ui of hadoop using other open source monitoring > tools like Nagios > > > Key: MAPREDUCE-2423 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2423 > Project: Hadoop Map/Reduce > Issue Type: Wish > Components: jobtracker >Reporter: Saurabh Mishra >Priority: Trivial > > I just wish if there is a way I can write monitors to check my hadoop job > tracker UI using my existing Nagios infrastructure. As this would help me in > keeping everything centrally located and hence under manageable limits. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2423) Monitoring the job tracker ui of hadoop using other open source monitoring tools like Nagios
[ https://issues.apache.org/jira/browse/MAPREDUCE-2423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated MAPREDUCE-2423: Priority: Trivial (was: Major) > Monitoring the job tracker ui of hadoop using other open source monitoring > tools like Nagios > > > Key: MAPREDUCE-2423 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2423 > Project: Hadoop Map/Reduce > Issue Type: Wish > Components: jobtracker >Reporter: Saurabh Mishra >Priority: Trivial > > I just wish if there is a way I can write monitors to check my hadoop job > tracker UI using my existing Nagios infrastructure. As this would help me in > keeping everything centrally located and hence under manageable limits. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2422) Removed unused internal methods from DistributedCache
[ https://issues.apache.org/jira/browse/MAPREDUCE-2422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tom White updated MAPREDUCE-2422: - Attachment: MAPREDUCE-2422.patch Minor update to remove unused imports. > Removed unused internal methods from DistributedCache > - > > Key: MAPREDUCE-2422 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2422 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: client >Reporter: Tom White >Assignee: Tom White > Attachments: MAPREDUCE-2422.patch, MAPREDUCE-2422.patch > > > DistributedCache has a number of deprecated methods that are no longer used > ever since TrackerDistributedCacheManager was introduced in MAPREDUCE-476. > Removing these methods (which are not user-facing) will make it possible to > complete MAPREDUCE-1638 by keeping DistributedCache in the API tree, and > TrackerDistributedCacheManager, TaskDistributedCacheManager in the > implementation tree. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2260) Remove auto-generated native build files
[ https://issues.apache.org/jira/browse/MAPREDUCE-2260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016998#comment-13016998 ] Hudson commented on MAPREDUCE-2260: --- Integrated in Hadoop-Mapreduce-trunk #643 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/]) > Remove auto-generated native build files > > > Key: MAPREDUCE-2260 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2260 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: build >Reporter: Roman Shaposhnik >Assignee: Roman Shaposhnik > Fix For: 0.22.0, 0.23.0 > > Attachments: MAPREDUCE-2260.diff > > Original Estimate: 24h > Remaining Estimate: 24h > > The repo currently includes the automake and autoconf generated files for the > native build. Per discussion on HADOOP-6421 let's remove them and use the > host's automake and autoconf. We should also do this for libhdfs and > fuse-dfs. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2126) JobQueueJobInProgressListener's javadoc is inconsistent with source code
[
https://issues.apache.org/jira/browse/MAPREDUCE-2126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016997#comment-13016997
]
Hudson commented on MAPREDUCE-2126:
---
Integrated in Hadoop-Mapreduce-trunk #643 (See
[https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/])
> JobQueueJobInProgressListener's javadoc is inconsistent with source code
>
>
> Key: MAPREDUCE-2126
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2126
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobtracker
>Affects Versions: 0.20.2
>Reporter: Jingguo Yao
>Assignee: Jingguo Yao
>Priority: Minor
> Fix For: 0.22.0
>
> Attachments: MAPREDUCE-2126.patch
>
> Original Estimate: 5m
> Remaining Estimate: 5m
>
> JobQueueJobInProgressListener.java has {@link
> #JobQueueJobInProgressListener(Collection)} in Javadoc. But it does not have
> the corresponding constructor. It has constructor
> JobQueueJobInProgressListener(Map
> jobQueue). So {@link JobQueueJobInProgressListener(Collection)} should be
> {@link #JobQueueJobInProgressListener(Map)}.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-1831) BlockPlacement policy for RAID
[ https://issues.apache.org/jira/browse/MAPREDUCE-1831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13017000#comment-13017000 ] Hudson commented on MAPREDUCE-1831: --- Integrated in Hadoop-Mapreduce-trunk #643 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/]) > BlockPlacement policy for RAID > -- > > Key: MAPREDUCE-1831 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1831 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: contrib/raid >Affects Versions: 0.23.0 >Reporter: Scott Chen >Assignee: Scott Chen > Fix For: 0.23.0 > > Attachments: MAPREDUCE-1831-v2.txt, MAPREDUCE-1831.20100610.txt, > MAPREDUCE-1831.txt, MAPREDUCE-1831.v1.1.txt, test.result > > > Raid introduce the new dependency between blocks within a file. > The blocks help decode each other. Therefore we should avoid put them on the > same machine. > The proposed BlockPlacementPolicy does the following > 1. When writing parity blocks, it avoid the parity blocks and source blocks > sit together. > 2. When reducing replication number, it deletes the blocks that sits with > other dependent blocks. > 3. It does not change the way we write normal files. It only has different > behavior when processing raid files. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-1974) FairScheduler can preempt the same task many times
[ https://issues.apache.org/jira/browse/MAPREDUCE-1974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016996#comment-13016996 ] Hudson commented on MAPREDUCE-1974: --- Integrated in Hadoop-Mapreduce-trunk #643 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/]) > FairScheduler can preempt the same task many times > -- > > Key: MAPREDUCE-1974 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1974 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: contrib/fair-share >Affects Versions: 0.21.0, 0.21.1, 0.22.0, 0.23.0 >Reporter: Scott Chen >Assignee: Scott Chen > Fix For: 0.22.0 > > Attachments: MAPREDUCE-1974.txt > > > In FairScheduler.preemptTasks(), tasks are collected from > JobInProgress.runningMapCache. > But tasks repeat multiple times in JobInProgress.runningMapCache (on rack, > node and cluster). > This makes FairScheduler preempt the same task many times. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2263) MapReduce side of HADOOP-6904
[ https://issues.apache.org/jira/browse/MAPREDUCE-2263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016994#comment-13016994 ] Hudson commented on MAPREDUCE-2263: --- Integrated in Hadoop-Mapreduce-trunk #643 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/]) > MapReduce side of HADOOP-6904 > - > > Key: MAPREDUCE-2263 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2263 > Project: Hadoop Map/Reduce > Issue Type: Improvement >Reporter: Hairong Kuang >Assignee: Hairong Kuang > Fix For: 0.23.0 > > Attachments: mapreduceRPC.patch > > > Make changes in Map/Reduce to incorporate HADOOP-6904. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-1970) Reed-Solomon code implementation to be used in raid
[ https://issues.apache.org/jira/browse/MAPREDUCE-1970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016991#comment-13016991 ] Hudson commented on MAPREDUCE-1970: --- Integrated in Hadoop-Mapreduce-trunk #643 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/]) > Reed-Solomon code implementation to be used in raid > --- > > Key: MAPREDUCE-1970 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1970 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: contrib/raid >Reporter: Scott Chen >Assignee: Scott Chen > Fix For: 0.22.0 > > Attachments: MAPREDUCE-1970-v2.txt, MAPREDUCE-1970.txt, test-result > > > A Reed-Solomon erasure code implementation. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2132) Need a command line option in RaidShell to fix blocks using raid
[ https://issues.apache.org/jira/browse/MAPREDUCE-2132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016990#comment-13016990 ] Hudson commented on MAPREDUCE-2132: --- Integrated in Hadoop-Mapreduce-trunk #643 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/]) > Need a command line option in RaidShell to fix blocks using raid > > > Key: MAPREDUCE-2132 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2132 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: contrib/raid >Reporter: Ramkumar Vadali >Assignee: Ramkumar Vadali > Fix For: 0.22.0 > > Attachments: MAPREDUCE-2132.2.patch, MAPREDUCE-2132.3.patch, > MAPREDUCE-2132.patch > > > RaidShell currently has an option to recover a file and return the path to > the recovered file. The administrator can then rename the recovered file to > the damaged file. > The problem with this is that the file metadata is altered, specifically the > modification time. Instead we need a way to just repair the damaged blocks > and send the fixed blocks to a data node. > Once this is done, we can put automation around it. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2348) TestSimulator* failed on trunk
[
https://issues.apache.org/jira/browse/MAPREDUCE-2348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016992#comment-13016992
]
Hudson commented on MAPREDUCE-2348:
---
Integrated in Hadoop-Mapreduce-trunk #643 (See
[https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/])
> TestSimulator* failed on trunk
> --
>
> Key: MAPREDUCE-2348
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2348
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: contrib/mumak
>Reporter: Scott Chen
>Assignee: Todd Lipcon
>Priority: Blocker
>
> All Failed Tests
> {code}
> org.apache.hadoop.mapred.TestSimulatorJobTracker.testTrackerInteraction
> org.apache.hadoop.mapred.TestSimulatorDeterministicReplay.testMain
> org.apache.hadoop.mapred.TestSimulatorEndToEnd.testMain
> org.apache.hadoop.mapred.TestSimulatorSerialJobSubmission.testMain
> org.apache.hadoop.mapred.TestSimulatorStressJobSubmission.testMain
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2271) TestSetupTaskScheduling failing in trunk
[ https://issues.apache.org/jira/browse/MAPREDUCE-2271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016989#comment-13016989 ] Hudson commented on MAPREDUCE-2271: --- Integrated in Hadoop-Mapreduce-trunk #643 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/]) > TestSetupTaskScheduling failing in trunk > > > Key: MAPREDUCE-2271 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2271 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: jobtracker >Affects Versions: 0.23.0 >Reporter: Todd Lipcon >Assignee: Liyin Liang >Priority: Blocker > Fix For: 0.23.0 > > Attachments: 2271-1.diff > > > This test case is failing in trunk after the commit of MAPREDUCE-2207 -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2134) ant binary-system is broken in mapreduce project.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016987#comment-13016987 ] Hudson commented on MAPREDUCE-2134: --- Integrated in Hadoop-Mapreduce-trunk #643 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/]) > ant binary-system is broken in mapreduce project. > - > > Key: MAPREDUCE-2134 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2134 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: build >Affects Versions: 0.21.1 >Reporter: Vinay Kumar Thota >Assignee: Konstantin Boudnik > Fix For: 0.21.1 > > Attachments: MAPREDUCE-2134.patch > > > Build failed due to unable to copy the commons instrumented jar. I could see > the following error in the log. > binary-system: > [copy] Copying 5 files to > /home/vinay/mapreduce/build-fi/system/hadoop-mapred-0.22.0-SNAPSHOT > BUILD FAILED > /home/vinay/mapreduce/build.xml:1307: Warning: Could not find file > /home/vinay/mapreduce/build-fi/ivy/lib/Hadoop/system/hadoop-common-instrumented-0.22.0-SNAPSHOT.jar > to copy. > It's pointing to the wrong path to copy the file. Actually the correct path > is, > /home/vinay/mapreduce/build-fi/system/ivy/Hadoop/system -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2272) Job ACL file should not be executable
[ https://issues.apache.org/jira/browse/MAPREDUCE-2272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016988#comment-13016988 ] Hudson commented on MAPREDUCE-2272: --- Integrated in Hadoop-Mapreduce-trunk #643 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/]) > Job ACL file should not be executable > - > > Key: MAPREDUCE-2272 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2272 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: tasktracker >Affects Versions: 0.22.0 >Reporter: Todd Lipcon >Assignee: Harsh J Chouraria >Priority: Trivial > Fix For: 0.22.0 > > Attachments: mapreduce.2272.r1.diff, mapreduce.2272.r2.diff > > > For some reason the job ACL file is localized with permissions 700. This > doesn't make sense, since it's not executable. It should be 600. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2351) mapred.job.tracker.history.completed.location should support an arbitrary filesystem URI
[ https://issues.apache.org/jira/browse/MAPREDUCE-2351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016983#comment-13016983 ] Hudson commented on MAPREDUCE-2351: --- Integrated in Hadoop-Mapreduce-trunk #643 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/]) > mapred.job.tracker.history.completed.location should support an arbitrary > filesystem URI > > > Key: MAPREDUCE-2351 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2351 > Project: Hadoop Map/Reduce > Issue Type: Improvement >Reporter: Tom White >Assignee: Tom White > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2351.patch > > > Currently, mapred.job.tracker.history.completed.location is resolved relative > to the default filesystem. If not set it defaults to history/done in the > local log directory. There is no way to set it to another local filesystem > location (with a file:// URI) or an arbitrary Hadoop filesystem. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2073) TestTrackerDistributedCacheManager should be up-front about requirements on build environment
[ https://issues.apache.org/jira/browse/MAPREDUCE-2073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016976#comment-13016976 ] Hudson commented on MAPREDUCE-2073: --- Integrated in Hadoop-Mapreduce-trunk #643 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/]) > TestTrackerDistributedCacheManager should be up-front about requirements on > build environment > - > > Key: MAPREDUCE-2073 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2073 > Project: Hadoop Map/Reduce > Issue Type: Test > Components: distributed-cache, test >Affects Versions: 0.22.0 >Reporter: Todd Lipcon >Assignee: Todd Lipcon >Priority: Trivial > Fix For: 0.22.0 > > Attachments: mapreduce-2073-0.20.txt, mapreduce-2073.txt > > > TestTrackerDistributedCacheManager will fail on a system where the build > directory is in any path where an ancestor doesn't have a+x permissions. On > one of our hudson boxes, for example, hudson's workspace had 700 permissions > and caused this test to fail reliably, but not in an obvious manner. It would > be helpful if the test failed with a more obvious error message during > setUp() when the build environment is misconfigured. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-1980) TaskAttemptUnsuccessfulCompletionEvent.java incorrectly logs MAP_ATTEMPT_KILLED as event type for reduce tasks
[
https://issues.apache.org/jira/browse/MAPREDUCE-1980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016977#comment-13016977
]
Hudson commented on MAPREDUCE-1980:
---
Integrated in Hadoop-Mapreduce-trunk #643 (See
[https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/])
> TaskAttemptUnsuccessfulCompletionEvent.java incorrectly logs
> MAP_ATTEMPT_KILLED as event type for reduce tasks
> --
>
> Key: MAPREDUCE-1980
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1980
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Amar Kamat
>Assignee: Amar Kamat
> Fix For: 0.21.0
>
> Attachments: mapreduce-1980-v1.0.patch, mapreduce-1980-v1.1.patch
>
>
> TaskAttemptUnsuccessfulCompletionEvent is used to log unsuccessful map and
> reduce task attempts to JobHistory. Following is the implementation of
> getEventType() method of TaskAttemptUnsuccessfulCompletionEvent
> /** Get the event type */
> public EventType getEventType() {
> return EventType.MAP_ATTEMPT_KILLED;
> }
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2077) Name clash in the deprecated o.a.h.util.MemoryCalculatorPlugin
[ https://issues.apache.org/jira/browse/MAPREDUCE-2077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016978#comment-13016978 ] Hudson commented on MAPREDUCE-2077: --- Integrated in Hadoop-Mapreduce-trunk #643 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/]) > Name clash in the deprecated o.a.h.util.MemoryCalculatorPlugin > -- > > Key: MAPREDUCE-2077 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2077 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Luke Lu >Assignee: Luke Lu > Fix For: 0.22.0 > > Attachments: mr-2077-trunk-v1.patch > > > Name clash compile error in the deprecated > org.apache.hadoop.util.MemoryCalculatorPlugin due to JLS3 8.4.8.3 (cf. > http://bugs.sun.com/view_bug.do?bug_id=6182950) > The bug doesn't manifest in jdk 1.6 up to 20, but shows up in NetBeans 6.9+ > due to its bundled (conforming) compiler. Fix is trivial: just remove the > offending method in the deprecated subclass as its equivalent erasure is > inherited from the parent class anyway. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2277) TestCapacitySchedulerWithJobTracker fails sometimes
[ https://issues.apache.org/jira/browse/MAPREDUCE-2277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016979#comment-13016979 ] Hudson commented on MAPREDUCE-2277: --- Integrated in Hadoop-Mapreduce-trunk #643 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/]) > TestCapacitySchedulerWithJobTracker fails sometimes > --- > > Key: MAPREDUCE-2277 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2277 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: contrib/capacity-sched >Affects Versions: 0.22.0 >Reporter: Todd Lipcon >Assignee: Todd Lipcon >Priority: Minor > Fix For: 0.22.0 > > Attachments: mapreduce-2277.txt > > > Sometimes the testJobTrackerIntegration test fails on my Hudson. It seems the > issue is that it doesn't ever wait for the first job to complete before > checking its success status. Since the two jobs are in different queues, the > first job may complete after the second job. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-1592) Generate Eclipse's .classpath file from Ivy config
[ https://issues.apache.org/jira/browse/MAPREDUCE-1592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016980#comment-13016980 ] Hudson commented on MAPREDUCE-1592: --- Integrated in Hadoop-Mapreduce-trunk #643 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/]) > Generate Eclipse's .classpath file from Ivy config > -- > > Key: MAPREDUCE-1592 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1592 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: build >Reporter: Tom White >Assignee: Tom White > Labels: ant, eclipse, ivy > Fix For: 0.22.0 > > Attachments: MAPREDUCE-1592.patch, MAPREDUCE-1592.patch, > MAPREDUCE-1592.patch, MAPREDUCE-1592.patch > > > MapReduce companion issue for HADOOP-6407. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2074) Task should fail when symlink creation fail
[
https://issues.apache.org/jira/browse/MAPREDUCE-2074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016975#comment-13016975
]
Hudson commented on MAPREDUCE-2074:
---
Integrated in Hadoop-Mapreduce-trunk #643 (See
[https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/])
> Task should fail when symlink creation fail
> ---
>
> Key: MAPREDUCE-2074
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2074
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: distributed-cache
>Affects Versions: 0.20.2
>Reporter: Koji Noguchi
>Assignee: Priyo Mustafi
>Priority: Minor
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2074.txt, MAPREDUCE-2074.txt,
> MAPREDUCE-2074.txt
>
>
> If I pass an invalid symlink as
> -Dmapred.cache.files=/user/knoguchi/onerecord.txt#abc/abc
> Task only reports a WARN and goes on.
> {noformat}
> 2010-09-16 21:38:49,782 INFO org.apache.hadoop.mapred.TaskRunner: Creating
> symlink:
> /0/tmp/mapred-local/taskTracker/knoguchi/distcache/-5031501808205559510_-128488332_1354038698/abc-nn1.def.com/user/knoguchi/onerecord.txt
> <-
> /0/tmp/mapred-local/taskTracker/knoguchi/jobcache/job_201008310107_15105/attempt_201008310107_15105_m_00_0/work/./abc/abc
> 2010-09-16 21:38:49,789 WARN org.apache.hadoop.mapred.TaskRunner: Failed to
> create symlink:
> /0/tmp/mapred-local/taskTracker/knoguchi/distcache/-5031501808205559510_-128488332_1354038698/abc-nn1.def.com/user/knoguchi/onerecord.txt
> <-
> /0/tmp/mapred-local/taskTracker/knoguchi/jobcache/job_201008310107_15105/attempt_201008310107_15105_m_00_0/work/./abc/abc
> {noformat}
> I believe we should fail the task at this point.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2146) Raid should not affect access time of a source file
[ https://issues.apache.org/jira/browse/MAPREDUCE-2146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016974#comment-13016974 ] Hudson commented on MAPREDUCE-2146: --- Integrated in Hadoop-Mapreduce-trunk #643 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/]) > Raid should not affect access time of a source file > --- > > Key: MAPREDUCE-2146 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2146 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: contrib/raid >Reporter: Ramkumar Vadali >Assignee: Ramkumar Vadali >Priority: Minor > Fix For: 0.22.0 > > Attachments: MAPREDUCE-2146.2.patch, MAPREDUCE-2146.patch > > > After a file is read for creating a raid parity file, the access time should > be set back to the value before the read. The read by RAID code is not an > application read and should not affect the access time. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2141) Add an "extra data" field to Task for use by Mesos
[ https://issues.apache.org/jira/browse/MAPREDUCE-2141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016969#comment-13016969 ] Hudson commented on MAPREDUCE-2141: --- Integrated in Hadoop-Mapreduce-trunk #643 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/]) > Add an "extra data" field to Task for use by Mesos > -- > > Key: MAPREDUCE-2141 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2141 > Project: Hadoop Map/Reduce > Issue Type: Improvement >Reporter: Matei Zaharia >Assignee: Matei Zaharia >Priority: Minor > Fix For: 0.22.0 > > Attachments: mapreduce-2141-v1.patch, mapreduce-2141-v2.patch > > > In order to support running Hadoop on the Mesos cluster manager > (http://mesos.berkeley.edu/), I'd like to add an extra String field to the > Task class to allow extra data (a Mesos task ID) to be associated with each > task. This should have no impact on normal operation other than making the > serialized form of Task a few bytes longer. In the Mesos support patch for > Hadoop, this field is set by a pluggable Hadoop scheduler implementation to > allow code on the TaskTracker side to see which Mesos task corresponds to > each Hadoop task. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2142) Refactor RaidNode to remove dependence on map reduce
[ https://issues.apache.org/jira/browse/MAPREDUCE-2142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016972#comment-13016972 ] Hudson commented on MAPREDUCE-2142: --- Integrated in Hadoop-Mapreduce-trunk #643 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/]) > Refactor RaidNode to remove dependence on map reduce > > > Key: MAPREDUCE-2142 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2142 > Project: Hadoop Map/Reduce > Issue Type: Task >Reporter: Patrick Kling >Assignee: Patrick Kling > Fix For: 0.22.0 > > Attachments: MAPREDUCE-2142.2.patch, MAPREDUCE-2142.3.patch, > MAPREDUCE-2142.patch > > > I am refactoring the RaidNode code as follows: The base class RaidNode will > contain the common functionality needed for raiding files. The derived class > LocalRaidNode contains an implementation of RaidNode that performs raiding > locally. The derived class DistRaidNode performs raiding using map reduce > jobs. This way, only DistRaidNode has a dependency on map reduce code and > RaidNode and LocalRaidNode can be moved to HDFS. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2147) JobInProgress has some redundant lines in its ctor
[ https://issues.apache.org/jira/browse/MAPREDUCE-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016973#comment-13016973 ] Hudson commented on MAPREDUCE-2147: --- Integrated in Hadoop-Mapreduce-trunk #643 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/]) > JobInProgress has some redundant lines in its ctor > -- > > Key: MAPREDUCE-2147 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2147 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: jobtracker >Affects Versions: 0.21.0 > Environment: Not environment related. >Reporter: Harsh J Chouraria >Assignee: Harsh J Chouraria >Priority: Trivial > Fix For: 0.22.0 > > Attachments: repeated.lines.JIP.r1.diff > > Original Estimate: 0h > Remaining Estimate: 0h > > In the ctor of JobInProgress class that's used by the JT, lines that create > the various lists of TIPs are repeated for no purpose. Might've been due to > an overlook I think. > Attaching a patch that removes these unnecessary repeats of re-init. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2143) HarFileSystem is not able to handle spaces in its path
[ https://issues.apache.org/jira/browse/MAPREDUCE-2143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016971#comment-13016971 ] Hudson commented on MAPREDUCE-2143: --- Integrated in Hadoop-Mapreduce-trunk #643 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/]) > HarFileSystem is not able to handle spaces in its path > -- > > Key: MAPREDUCE-2143 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2143 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: harchive >Reporter: Ramkumar Vadali >Assignee: Ramkumar Vadali > Fix For: 0.22.0 > > Attachments: MAPREDUCE-2143.patch > > > If the Path to the HAR contains spaces, Path.getFileSystem() fails. The > problem is in HarFileSystem.initialize(), which uses URI.toString() to get a > string for getting to the .har suffix. URI.toString() returns a > percent-encoded string when the path contains spaces. When this string is > subsequently used to get the _index file, we get a FileNotFoundException. The > fix is to use URI.getPath(). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2140) Re-generate fair scheduler design doc PDF
[ https://issues.apache.org/jira/browse/MAPREDUCE-2140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016970#comment-13016970 ] Hudson commented on MAPREDUCE-2140: --- Integrated in Hadoop-Mapreduce-trunk #643 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/]) > Re-generate fair scheduler design doc PDF > - > > Key: MAPREDUCE-2140 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2140 > Project: Hadoop Map/Reduce > Issue Type: Improvement >Reporter: Matei Zaharia >Assignee: Matei Zaharia >Priority: Trivial > Fix For: 0.21.1, 0.22.0 > > Attachments: fair_scheduler_design_doc.pdf > > > The Fair Scheduler contains a design document in > src/contrib/fairscheduler/designdoc that is included both as a Latex file and > as a PDF. However, the PDF that's currently there is not generated properly > and has some question marks for section references. I'd like to regenerate it > and commit the new one. There is no patch to attach because this just > requires running pdflatex and committing a binary file. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2248) DistributedRaidFileSystem should unraid only the corrupt block
[ https://issues.apache.org/jira/browse/MAPREDUCE-2248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016967#comment-13016967 ] Hudson commented on MAPREDUCE-2248: --- Integrated in Hadoop-Mapreduce-trunk #643 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/]) > DistributedRaidFileSystem should unraid only the corrupt block > -- > > Key: MAPREDUCE-2248 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2248 > Project: Hadoop Map/Reduce > Issue Type: Improvement >Affects Versions: 0.23.0 >Reporter: Ramkumar Vadali >Assignee: Ramkumar Vadali > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2248.1.patch, MAPREDUCE-2248.patch > > > DistributedRaidFileSystem unraids the entire file if it hits a corrupt block. > It is better to unraid just the corrupt block and use the rest of the file as > normal. This becomes really important when we have tera-byte sized files. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2241) ClusterWithLinuxTaskController should accept relative path on the command line
[ https://issues.apache.org/jira/browse/MAPREDUCE-2241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016968#comment-13016968 ] Hudson commented on MAPREDUCE-2241: --- Integrated in Hadoop-Mapreduce-trunk #643 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/]) > ClusterWithLinuxTaskController should accept relative path on the command line > -- > > Key: MAPREDUCE-2241 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2241 > Project: Hadoop Map/Reduce > Issue Type: Test > Components: task-controller, test >Affects Versions: 0.23.0 >Reporter: Todd Lipcon >Assignee: Todd Lipcon >Priority: Trivial > Fix For: 0.22.0 > > Attachments: mr-2241.txt > > > Currently if you pass a relative path for the -Dtaskcontroller-path option > when running these tests, it fails in a fairly unintuitive way. We should > absolutize it inside the tests to make it easier for people to run them. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-1334) contrib/index - test - TestIndexUpdater fails due to an additional presence of file _SUCCESS in hdfs
[ https://issues.apache.org/jira/browse/MAPREDUCE-1334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016966#comment-13016966 ] Hudson commented on MAPREDUCE-1334: --- Integrated in Hadoop-Mapreduce-trunk #643 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/]) > contrib/index - test - TestIndexUpdater fails due to an additional presence > of file _SUCCESS in hdfs > - > > Key: MAPREDUCE-1334 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1334 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: contrib/index >Reporter: Kay Kay >Assignee: Kay Kay > Fix For: 0.23.0 > > Attachments: MAPREDUCE-1334.patch, MAPREDUCE-1334.patch > > > $ cd src/contrib/index > $ ant clean test > This fails the test TestIndexUpdater due to a mismatch in the - doneFileNames > - data structure, when it is being run with different parameters. > (ArrayIndexOutOfBoundsException raised when inserting elements in > doneFileNames, array ). > Debugging further - there seems to be an additional file called as - > hdfs://localhost:36021/myoutput/_SUCCESS , taken into consideration in > addition to those that begins with done* . The presence of the extra file > causes the error. > Attaching a patch that would circumvent this by increasing the array length > of shards by 1 . > But longer term the test fixtures need to be probably revisited to see if the > presence of _SUCCESS as a file is a good thing to begin with before we even > get to this test case. > Any comments / suggestions on the same welcome. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2250) Fix logging in raid code.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016961#comment-13016961 ] Hudson commented on MAPREDUCE-2250: --- Integrated in Hadoop-Mapreduce-trunk #643 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/]) > Fix logging in raid code. > - > > Key: MAPREDUCE-2250 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2250 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: contrib/raid >Reporter: Ramkumar Vadali >Assignee: Ramkumar Vadali >Priority: Trivial > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2250.1.patch, MAPREDUCE-2250.2.patch, > MAPREDUCE-2250.patch > > > There are quite a few error messages being logged with a log level of info. > That should be fixed to help debugging. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2253) Servlets should specify content type
[ https://issues.apache.org/jira/browse/MAPREDUCE-2253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016956#comment-13016956 ] Hudson commented on MAPREDUCE-2253: --- Integrated in Hadoop-Mapreduce-trunk #643 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/]) > Servlets should specify content type > > > Key: MAPREDUCE-2253 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2253 > Project: Hadoop Map/Reduce > Issue Type: Bug >Affects Versions: 0.22.0 >Reporter: Todd Lipcon >Assignee: Todd Lipcon >Priority: Critical > Fix For: 0.22.0 > > Attachments: mapreduce-2253.txt, mapreduce-2253.txt, > mapreduce-2253.txt > > > HADOOP-7093 will change the default content-type to text/plain. So > TaskLogServlet, which outputs HTML, needs to change to specify this content > type. I believe the other HTML servlets already correctly specify a content > type. The MapOutputServlet appears to specify no content type and work fine > without one, but to be "correct" we may as well specify > application/octet-stream -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2155) RaidNode should optionally dispatch map reduce jobs to fix corrupt blocks (instead of fixing locally)
[ https://issues.apache.org/jira/browse/MAPREDUCE-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016963#comment-13016963 ] Hudson commented on MAPREDUCE-2155: --- Integrated in Hadoop-Mapreduce-trunk #643 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/]) > RaidNode should optionally dispatch map reduce jobs to fix corrupt blocks > (instead of fixing locally) > - > > Key: MAPREDUCE-2155 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2155 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: contrib/raid >Affects Versions: 0.23.0 >Reporter: Patrick Kling >Assignee: Patrick Kling > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2155.2.patch, MAPREDUCE-2155.patch > > > Recomputing blocks based on parity information is expensive. Rather than > doing this locally at the RaidNode, we should run map reduce jobs. This will > allow us to quickly fix a large number of corrupt or missing blocks. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-1085) For tasks, "ulimit -v -1" is being run when user doesn't specify mapred.child.ulimit
[ https://issues.apache.org/jira/browse/MAPREDUCE-1085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016964#comment-13016964 ] Hudson commented on MAPREDUCE-1085: --- Integrated in Hadoop-Mapreduce-trunk #643 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/]) > For tasks, "ulimit -v -1" is being run when user doesn't specify > mapred.child.ulimit > > > Key: MAPREDUCE-1085 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1085 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: tasktracker >Affects Versions: 0.21.0, 0.22.0 >Reporter: Ravi Gummadi >Assignee: Todd Lipcon >Priority: Minor > Fix For: 0.22.0 > > Attachments: mapreduce-1085.txt > > > For tasks, "ulimit -v -1" is being run when user doesn't specify > mapred.child.ulimit. Taking -1 as default value and using it in building the > command is not right. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2334) Update BlockPlacementPolicyRaid
[
https://issues.apache.org/jira/browse/MAPREDUCE-2334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016957#comment-13016957
]
Hudson commented on MAPREDUCE-2334:
---
Integrated in Hadoop-Mapreduce-trunk #643 (See
[https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/])
> Update BlockPlacementPolicyRaid
> ---
>
> Key: MAPREDUCE-2334
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2334
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: contrib/raid
>Reporter: Tsz Wo (Nicholas), SZE
>Assignee: Tsz Wo (Nicholas), SZE
>Priority: Trivial
> Attachments: m2334_20110217.patch
>
>
> Update {{BlockPlacementPolicyRaid}} for the recent changes of
> {{BlockPlacementPolicy}}.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2251) Remove mapreduce.job.userhistorylocation config
[ https://issues.apache.org/jira/browse/MAPREDUCE-2251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016958#comment-13016958 ] Hudson commented on MAPREDUCE-2251: --- Integrated in Hadoop-Mapreduce-trunk #643 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/]) > Remove mapreduce.job.userhistorylocation config > --- > > Key: MAPREDUCE-2251 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2251 > Project: Hadoop Map/Reduce > Issue Type: Bug >Affects Versions: 0.22.0 >Reporter: Todd Lipcon >Assignee: Harsh J Chouraria > Fix For: 0.22.0 > > Attachments: mapreduce.2251.jobhistorylocremove.r1.diff > > > Best I can tell, this config parameter is no longer used as of MAPREDUCE-157 > but still exists in the code and in mapred-default.xml. We should remove it > to avoid user confusion. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2156) Raid-aware FSCK
[ https://issues.apache.org/jira/browse/MAPREDUCE-2156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016959#comment-13016959 ] Hudson commented on MAPREDUCE-2156: --- Integrated in Hadoop-Mapreduce-trunk #643 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/]) > Raid-aware FSCK > --- > > Key: MAPREDUCE-2156 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2156 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: contrib/raid >Affects Versions: 0.23.0 >Reporter: Patrick Kling >Assignee: Patrick Kling > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2156.2.patch, MAPREDUCE-2156.3.patch, > MAPREDUCE-2156.patch > > > Currently, FSCK reports files as corrupt even if they can be fixed using > parity blocks. We need a tool that only reports files that are irreparably > corrupt (i.e., files for which too many data or parity blocks belonging to > the same stripe have been lost or corrupted). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-1867) Remove unused methods in org.apache.hadoop.streaming.StreamUtil
[ https://issues.apache.org/jira/browse/MAPREDUCE-1867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016960#comment-13016960 ] Hudson commented on MAPREDUCE-1867: --- Integrated in Hadoop-Mapreduce-trunk #643 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/]) > Remove unused methods in org.apache.hadoop.streaming.StreamUtil > --- > > Key: MAPREDUCE-1867 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1867 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: contrib/streaming >Reporter: Amareshwari Sriramadasu >Assignee: Amareshwari Sriramadasu >Priority: Minor > Fix For: 0.22.0 > > Attachments: MAPREDUCE-1867.patch, patch-1867-1.txt, patch-1867.txt > > > There are many unused methods in org.apache.hadoop.streaming.StreamUtil. They > should be removed from the class for maintainability. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2254) Allow setting of end-of-record delimiter for TextInputFormat
[ https://issues.apache.org/jira/browse/MAPREDUCE-2254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016955#comment-13016955 ] Hudson commented on MAPREDUCE-2254: --- Integrated in Hadoop-Mapreduce-trunk #643 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/]) > Allow setting of end-of-record delimiter for TextInputFormat > > > Key: MAPREDUCE-2254 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2254 > Project: Hadoop Map/Reduce > Issue Type: Improvement >Reporter: Ahmed Radwan >Assignee: Ahmed Radwan > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2245.patch, MAPREDUCE-2254_r2.patch, > MAPREDUCE-2254_r3.patch > > > It will be useful to allow setting the end-of-record delimiter for > TextInputFormat. The current implementation hardcodes '\n', '\r' or '\r\n' as > the only possible record delimiters. This is a problem if users have embedded > newlines in their data fields (which is pretty common). This is also a > problem for other tools using this TextInputFormat (See for example: > https://issues.apache.org/jira/browse/PIG-836 and > https://issues.cloudera.org/browse/SQOOP-136). > I have wrote a patch to address this issue. This patch allows users to > specify any custom end-of-record delimiter using a new added configuration > property. For backward compatibility, if this new configuration property is > absent, then the same exact previous delimiters are used (i.e., '\n', '\r' or > '\r\n'). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2336) Tool-related packages should be in the Tool javadoc group
[ https://issues.apache.org/jira/browse/MAPREDUCE-2336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016954#comment-13016954 ] Hudson commented on MAPREDUCE-2336: --- Integrated in Hadoop-Mapreduce-trunk #643 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/]) > Tool-related packages should be in the Tool javadoc group > - > > Key: MAPREDUCE-2336 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2336 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: documentation >Reporter: Tom White >Assignee: Tom White > Fix For: 0.22.0 > > Attachments: MAPREDUCE-2336.patch > > > Some of the tool packages are mistakenly in the general group. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2054) Hierarchical queue implementation broke dynamic queue addition in Dynamic Scheduler
[ https://issues.apache.org/jira/browse/MAPREDUCE-2054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016951#comment-13016951 ] Hudson commented on MAPREDUCE-2054: --- Integrated in Hadoop-Mapreduce-trunk #643 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/]) > Hierarchical queue implementation broke dynamic queue addition in Dynamic > Scheduler > --- > > Key: MAPREDUCE-2054 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2054 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: contrib/dynamic-scheduler >Affects Versions: 0.21.0 >Reporter: Thomas Sandholm >Assignee: Thomas Sandholm > Fix For: 0.22.0 > > Attachments: MAPREDUCE-2054.patch > > > Queue names were returned from the queue manager as an immutable set after > the hierarchical queuname feature which breaks the dynamic priority scheduler -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2331) Add coverage of task graph servlet to fair scheduler system test
[ https://issues.apache.org/jira/browse/MAPREDUCE-2331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016952#comment-13016952 ] Hudson commented on MAPREDUCE-2331: --- Integrated in Hadoop-Mapreduce-trunk #643 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/]) > Add coverage of task graph servlet to fair scheduler system test > > > Key: MAPREDUCE-2331 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2331 > Project: Hadoop Map/Reduce > Issue Type: Test >Affects Versions: 0.22.0 >Reporter: Todd Lipcon >Assignee: Todd Lipcon > Fix For: 0.23.0 > > Attachments: mapreduce-2331.txt > > > Would be useful to hit the TaskGraph servlet in the fair scheduler system > test. This way, when run under JCarder, it will check for any lock inversions > in this code. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2256) FairScheduler fairshare preemption from multiple pools may preempt all tasks from one pool causing that pool to go below fairshare.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016953#comment-13016953 ] Hudson commented on MAPREDUCE-2256: --- Integrated in Hadoop-Mapreduce-trunk #643 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/]) > FairScheduler fairshare preemption from multiple pools may preempt all tasks > from one pool causing that pool to go below fairshare. > --- > > Key: MAPREDUCE-2256 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2256 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: contrib/fair-share >Affects Versions: 0.21.1, 0.22.0 >Reporter: Priyo Mustafi >Assignee: Priyo Mustafi > Fix For: 0.22.0 > > Attachments: mapreduce-2256_0_22.txt > > > Scenarios: > You have a cluster with 600 map slots and 3 pools. Fairshare for each pool > is 200 to start with. Fairsharepreemption timeout is 5 mins. > 1) Pool1 schedules 300 map tasks first > 2) Pool2 then schedules another 300 map tasks > 3) Pool3 demands 300 map tasks but doesn't get any slot as all slots are > taken. > 4) After 5 mins pool3 should preempt 200 map-slots. Instead of peempting > 100 slots each from pool1 and pool2, the bug would cause it to preempt all > 200 slots from pool2 (last started) causing it to go below fairshare. This > is happening because the preemptTask method is not reducing the tasks left > from a pool while preempting the tasks. > The above scenario could be an extreme case but some amount of excess > preemption would happen because of this bug. > The patch I created was for 0.22.0 but the code fix should work on 0.21 as > well as looks like it has the same bug. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-1878) Add MRUnit documentation
[ https://issues.apache.org/jira/browse/MAPREDUCE-1878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016947#comment-13016947 ] Hudson commented on MAPREDUCE-1878: --- Integrated in Hadoop-Mapreduce-trunk #643 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/]) > Add MRUnit documentation > > > Key: MAPREDUCE-1878 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1878 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: contrib/mrunit >Reporter: Aaron Kimball >Assignee: Aaron Kimball > Fix For: 0.22.0 > > Attachments: MAPREDUCE-1878.2.patch, MAPREDUCE-1878.3.patch, > MAPREDUCE-1878.patch > > > A short user guide for MRUnit, written in asciidoc. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2167) Faster directory traversal for raid node
[ https://issues.apache.org/jira/browse/MAPREDUCE-2167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016948#comment-13016948 ] Hudson commented on MAPREDUCE-2167: --- Integrated in Hadoop-Mapreduce-trunk #643 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/]) > Faster directory traversal for raid node > > > Key: MAPREDUCE-2167 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2167 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: contrib/raid >Reporter: Ramkumar Vadali >Assignee: Ramkumar Vadali > Fix For: 0.22.0 > > Attachments: MAPREDUCE-2167.2.patch, MAPREDUCE-2167.3.patch, > MAPREDUCE-2167.4.patch, MAPREDUCE-2167.patch > > > The RaidNode currently iterates over the directory structure to figure out > which files to RAID. With millions of files, this can take a long time - > especially if some files are already RAIDed and the RaidNode needs to look at > parity files / parity file HARs to determine if the file needs to be RAIDed. > The directory traversal is encapsulated inside the class DirectoryTraversal, > which examines one file at a time, using the caller's thread. > My proposal is to make this multi-threaded as follows: > * use a pool of threads inside DirectoryTraversal > * The caller's thread is used to retrieve directories, and each new > directory is assigned to a thread in the pool. The worker thread examines all > the files the directory. > * If there sub-directories, those are added back as workitems to the pool. > Comments? -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2300) TestUmbilicalProtocolWithJobToken failing
[ https://issues.apache.org/jira/browse/MAPREDUCE-2300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016949#comment-13016949 ] Hudson commented on MAPREDUCE-2300: --- Integrated in Hadoop-Mapreduce-trunk #643 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/]) > TestUmbilicalProtocolWithJobToken failing > - > > Key: MAPREDUCE-2300 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2300 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Todd Lipcon >Assignee: Todd Lipcon >Priority: Blocker > Attachments: mapreduce-2300.txt > > > Testcase: testJobTokenRpc took 0.678 sec > Caused an ERROR > null > java.lang.NullPointerException > at > org.apache.hadoop.ipc.WritableRpcEngine.getProxy(WritableRpcEngine.java:241) > at org.apache.hadoop.ipc.RPC.getProtocolProxy(RPC.java:422) > at org.apache.hadoop.ipc.RPC.getProtocolProxy(RPC.java:368) > at org.apache.hadoop.ipc.RPC.getProtocolProxy(RPC.java:333) > at org.apache.hadoop.ipc.RPC.getProtocolProxy(RPC.java:461) > at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:442) > at > org.apache.hadoop.mapreduce.security.TestUmbilicalProtocolWithJobToken$1.run(TestUmbilicalProtocolWithJobToken.java:102) -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2302) Add static factory methods in GaloisField
[ https://issues.apache.org/jira/browse/MAPREDUCE-2302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016950#comment-13016950 ] Hudson commented on MAPREDUCE-2302: --- Integrated in Hadoop-Mapreduce-trunk #643 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/]) > Add static factory methods in GaloisField > - > > Key: MAPREDUCE-2302 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2302 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: contrib/raid >Reporter: Scott Chen >Assignee: Scott Chen > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2302.2.txt, MAPREDUCE-2302.3.txt, > MAPREDUCE-2302.txt > > > GaloisField is immutable and should be kept reuse after creation to avoid > redundant calculation of the multiplication and division tables. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2238) Undeletable build directories
[ https://issues.apache.org/jira/browse/MAPREDUCE-2238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016946#comment-13016946 ] Hudson commented on MAPREDUCE-2238: --- Integrated in Hadoop-Mapreduce-trunk #643 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/]) > Undeletable build directories > -- > > Key: MAPREDUCE-2238 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2238 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: build, test >Affects Versions: 0.22.0 >Reporter: Eli Collins >Assignee: Todd Lipcon >Priority: Critical > Fix For: 0.22.0 > > Attachments: mapreduce-2238.txt, mapreduce-2238.txt, > mapreduce-2238.txt > > > The MR hudson job is failing, looks like it's due to a test chmod'ing a build > directory so the checkout can't clean the build dir. > https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/549/console > Building remotely on hadoop7 > hudson.util.IOException2: remote file operation failed: > /grid/0/hudson/hudson-slave/workspace/Hadoop-Mapreduce-trunk at > hudson.remoting.Channel@2545938c:hadoop7 > at hudson.FilePath.act(FilePath.java:749) > at hudson.FilePath.act(FilePath.java:735) > at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:589) > at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:537) > at hudson.model.AbstractProject.checkout(AbstractProject.java:1116) > at > hudson.model.AbstractBuild$AbstractRunner.checkout(AbstractBuild.java:479) > at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:411) > at hudson.model.Run.run(Run.java:1324) > at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46) > at hudson.model.ResourceController.execute(ResourceController.java:88) > at hudson.model.Executor.run(Executor.java:139) > Caused by: java.io.IOException: Unable to delete > /grid/0/hudson/hudson-slave/workspace/Hadoop-Mapreduce-trunk/trunk/build/test/logs/userlogs/job_20101230131139886_0001/attempt_20101230131139886_0001_m_00_0 -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2232) Add missing methods to TestMapredGroupMappingServiceRefresh
[ https://issues.apache.org/jira/browse/MAPREDUCE-2232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016943#comment-13016943 ] Hudson commented on MAPREDUCE-2232: --- Integrated in Hadoop-Mapreduce-trunk #643 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/]) > Add missing methods to TestMapredGroupMappingServiceRefresh > --- > > Key: MAPREDUCE-2232 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2232 > Project: Hadoop Map/Reduce > Issue Type: Bug >Affects Versions: 0.23.0 >Reporter: Todd Lipcon >Assignee: Todd Lipcon > Attachments: mapreduce-2232.txt > > > HADOOP-6864 added new methods to the GroupMappingServiceProvider interface, > so MR trunk no longer compiles. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2169) Integrated Reed-Solomon code with RaidNode
[ https://issues.apache.org/jira/browse/MAPREDUCE-2169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016944#comment-13016944 ] Hudson commented on MAPREDUCE-2169: --- Integrated in Hadoop-Mapreduce-trunk #643 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/]) > Integrated Reed-Solomon code with RaidNode > -- > > Key: MAPREDUCE-2169 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2169 > Project: Hadoop Map/Reduce > Issue Type: Task > Components: contrib/raid >Reporter: Ramkumar Vadali >Assignee: Ramkumar Vadali > Fix For: 0.22.0 > > Attachments: MAPREDUCE-2169.2.patch, MAPREDUCE-2169.patch > > > Scott Chen recently checked in an implementation of the Reed Solomon code. > This task will track the integration of the code with RaidNode. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2239) BlockPlacementPolicyRaid should call getBlockLocations only when necessary
[ https://issues.apache.org/jira/browse/MAPREDUCE-2239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016945#comment-13016945 ] Hudson commented on MAPREDUCE-2239: --- Integrated in Hadoop-Mapreduce-trunk #643 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/]) > BlockPlacementPolicyRaid should call getBlockLocations only when necessary > -- > > Key: MAPREDUCE-2239 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2239 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: contrib/raid >Affects Versions: 0.23.0 >Reporter: Scott Chen >Assignee: Scott Chen > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2239-1.txt, MAPREDUCE-2239-2.txt, > MAPREDUCE-2239-3.txt, MAPREDUCE-2239.txt > > > Currently BlockPlacementPolicyRaid calls getBlockLocations for every > chooseTarget(). > This puts pressure on NameNode. We should avoid calling if this file is not > raided or a parity file. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2234) If Localizer can't create task log directory, it should fail on the spot
[ https://issues.apache.org/jira/browse/MAPREDUCE-2234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016941#comment-13016941 ] Hudson commented on MAPREDUCE-2234: --- Integrated in Hadoop-Mapreduce-trunk #643 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/]) > If Localizer can't create task log directory, it should fail on the spot > > > Key: MAPREDUCE-2234 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2234 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: tasktracker >Affects Versions: 0.22.0 >Reporter: Todd Lipcon >Assignee: Todd Lipcon > Fix For: 0.22.0 > > Attachments: mapreduce-2234.txt, mapreduce-2234.txt > > > Currently, it simply emits a warning. Then, when the taskjvm.sh tries to pipe > its output into this directory, it fails with a strange error code like "exit > code: 1" which is not intuitive to ops. Instead it should simply throw an > exception at initialization time rather than attempting to run the task. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-1929) Allow artifacts to be published to the staging Apache Nexus Maven Repository
[ https://issues.apache.org/jira/browse/MAPREDUCE-1929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016942#comment-13016942 ] Hudson commented on MAPREDUCE-1929: --- Integrated in Hadoop-Mapreduce-trunk #643 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/]) > Allow artifacts to be published to the staging Apache Nexus Maven Repository > > > Key: MAPREDUCE-1929 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1929 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: build >Reporter: Tom White >Assignee: Tom White >Priority: Blocker > Fix For: 0.21.1 > > Attachments: MAPREDUCE-1929.patch > > > MapReduce companion issue to HADOOP-6847. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-1927) unit test for HADOOP-6835 (concatenated gzip support)
[ https://issues.apache.org/jira/browse/MAPREDUCE-1927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016940#comment-13016940 ] Hudson commented on MAPREDUCE-1927: --- Integrated in Hadoop-Mapreduce-trunk #643 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/]) > unit test for HADOOP-6835 (concatenated gzip support) > - > > Key: MAPREDUCE-1927 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1927 > Project: Hadoop Map/Reduce > Issue Type: Test > Components: test >Affects Versions: 0.22.0 >Reporter: Greg Roelofs >Assignee: Greg Roelofs >Priority: Minor > Fix For: 0.23.0 > > Attachments: MR-1927.v12.trunk-hadoop-mapreduce.patch, > MR-1927.v13.trunk-hadoop-mapreduce.patch.txt, > MR-1927.v9.trunk-hadoop-mapreduce.patch, > construct-all-gz-header-types-20100812.zip > > > More extensive test of concatenated gzip (and bzip2) decoding support for > HADOOP-6835 (and HADOOP-4012 and HADOOP-6852). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2304) TestMRCLI fails when hostname has a hyphen (-)
[ https://issues.apache.org/jira/browse/MAPREDUCE-2304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016936#comment-13016936 ] Hudson commented on MAPREDUCE-2304: --- Integrated in Hadoop-Mapreduce-trunk #643 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/]) > TestMRCLI fails when hostname has a hyphen (-) > -- > > Key: MAPREDUCE-2304 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2304 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: test >Affects Versions: 0.22.0 > Environment: Linux >Reporter: Priyo Mustafi >Assignee: Priyo Mustafi >Priority: Minor > Fix For: 0.22.0 > > Attachments: MAPREDUCE-2304.patch > > > TestMRCLI fails with below > Comparator: [RegexpComparator] > Comparision result: [fail] > Expected output: [mv: Wrong FS: har:/dest/dir0.har/dir0/file0, expected: > hdfs://\w+[.a-z]*:[0-9]+] > Actual output: [mv: Wrong FS: har:/dest/dir0.har/dir0/file0, expected: > hdfs://lab-something.host.com:34039 -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2307) Exception thrown in Jobtracker logs, when the Scheduler configured is FairScheduler.
[
https://issues.apache.org/jira/browse/MAPREDUCE-2307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016937#comment-13016937
]
Hudson commented on MAPREDUCE-2307:
---
Integrated in Hadoop-Mapreduce-trunk #643 (See
[https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/])
> Exception thrown in Jobtracker logs, when the Scheduler configured is
> FairScheduler.
>
>
> Key: MAPREDUCE-2307
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2307
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: contrib/fair-share
>Affects Versions: 0.23.0
>Reporter: Devaraj K
>Priority: Minor
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2307.patch
>
>
> If we try to start the job tracker with fair scheduler using the default
> configuration, It is giving the below exception.
> {code:xml}
> 2010-07-03 10:18:27,142 INFO org.apache.hadoop.ipc.Server: IPC Server handler
> 2 on 9001: starting
> 2010-07-03 10:18:27,143 INFO org.apache.hadoop.ipc.Server: IPC Server handler
> 3 on 9001: starting
> 2010-07-03 10:18:27,143 INFO org.apache.hadoop.ipc.Server: IPC Server handler
> 4 on 9001: starting
> 2010-07-03 10:18:27,143 INFO org.apache.hadoop.ipc.Server: IPC Server handler
> 5 on 9001: starting
> 2010-07-03 10:18:27,143 INFO org.apache.hadoop.ipc.Server: IPC Server handler
> 6 on 9001: starting
> 2010-07-03 10:18:27,143 INFO org.apache.hadoop.ipc.Server: IPC Server handler
> 7 on 9001: starting
> 2010-07-03 10:18:27,143 INFO org.apache.hadoop.ipc.Server: IPC Server handler
> 8 on 9001: starting
> 2010-07-03 10:18:27,143 INFO org.apache.hadoop.mapred.JobTracker: Starting
> RUNNING
> 2010-07-03 10:18:27,143 INFO org.apache.hadoop.ipc.Server: IPC Server handler
> 9 on 9001: starting
> 2010-07-03 10:18:28,037 INFO org.apache.hadoop.net.NetworkTopology: Adding a
> new node: /default-rack/linux172.site
> 2010-07-03 10:18:28,090 INFO org.apache.hadoop.net.NetworkTopology: Adding a
> new node: /default-rack/linux177.site
> 2010-07-03 10:18:40,074 ERROR org.apache.hadoop.mapred.PoolManager: Failed to
> reload allocations file - will use existing allocations.
> java.lang.NullPointerException
> at java.io.File.(File.java:222)
> at
> org.apache.hadoop.mapred.PoolManager.reloadAllocsIfNecessary(PoolManager.java:127)
> at org.apache.hadoop.mapred.FairScheduler.assignTasks(FairScheduler.java:234)
> at org.apache.hadoop.mapred.JobTracker.heartbeat(JobTracker.java:2785)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:513)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:984)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:980)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:978)
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2394) JUnit output format doesn't propagate into some contrib builds
[
https://issues.apache.org/jira/browse/MAPREDUCE-2394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016938#comment-13016938
]
Hudson commented on MAPREDUCE-2394:
---
Integrated in Hadoop-Mapreduce-trunk #643 (See
[https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/])
> JUnit output format doesn't propagate into some contrib builds
> --
>
> Key: MAPREDUCE-2394
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2394
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Affects Versions: 0.22.0
>Reporter: Todd Lipcon
>Assignee: Todd Lipcon
>Priority: Blocker
> Fix For: 0.22.0
>
> Attachments: mapreduce-2394.txt
>
>
> Some of the contribs seem to have an issue where the test.junit.output.format
> property isn't propagating down into their builds. So, Hudson is unable to
> parse the test output, and we see failed builds with no actual parsed test
> results showing what failed.
> This is at least true for {{contrib/raid}} but maybe others as well.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2395) TestBlockFixer timing out on trunk
[ https://issues.apache.org/jira/browse/MAPREDUCE-2395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016939#comment-13016939 ] Hudson commented on MAPREDUCE-2395: --- Integrated in Hadoop-Mapreduce-trunk #643 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/]) MAPREDUCE-2395. TestBlockFixer timing out on trunk. Contributed by Ramkumar Vadali. > TestBlockFixer timing out on trunk > -- > > Key: MAPREDUCE-2395 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2395 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: contrib/raid >Affects Versions: 0.23.0 >Reporter: Todd Lipcon >Assignee: Ramkumar Vadali >Priority: Critical > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2395.patch > > > In recent Hudson builds, TestBlockFixer has been timing out. Not clear how > long it has been broken since MAPREDUCE-2394 was hiding the RAID tests from > Hudson's test result parsing. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2026) JobTracker.getJobCounters() should not hold JobTracker lock while calling JobInProgress.getCounters()
[ https://issues.apache.org/jira/browse/MAPREDUCE-2026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016934#comment-13016934 ] Hudson commented on MAPREDUCE-2026: --- Integrated in Hadoop-Mapreduce-trunk #643 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/]) > JobTracker.getJobCounters() should not hold JobTracker lock while calling > JobInProgress.getCounters() > - > > Key: MAPREDUCE-2026 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2026 > Project: Hadoop Map/Reduce > Issue Type: Improvement >Reporter: Scott Chen >Assignee: Joydeep Sen Sarma > Fix For: 0.22.0, 0.23.0 > > Attachments: 2026.1.patch, MAPREDUCE-2026.txt > > > JobTracker.getJobCounter() will lock JobTracker and call > JobInProgress.getCounters(). > JobInProgress.getCounters() can be very expensive because it aggregates all > the task counters. > We found that from the JobTracker jstacks that this method is one of the > bottleneck of the JobTracker performance. > JobInProgress.getCounters() should be able to be called out side the > JobTracker lock because it already has JobInProgress lock. > For example, it is used by jobdetails.jsp without a JobTracker lock. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2392) TaskTracker shutdown in the tests sometimes take 60s
[
https://issues.apache.org/jira/browse/MAPREDUCE-2392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016935#comment-13016935
]
Hudson commented on MAPREDUCE-2392:
---
Integrated in Hadoop-Mapreduce-trunk #643 (See
[https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/])
> TaskTracker shutdown in the tests sometimes take 60s
>
>
> Key: MAPREDUCE-2392
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2392
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Tom White
>Assignee: Tom White
> Fix For: 0.22.0
>
> Attachments: MAPREDUCE-2392.patch
>
>
> There are a lot of the following in the test logs:
> {noformat}
> 2011-03-16 13:47:02,267 INFO mapred.TaskTracker
> (TaskTracker.java:shutdown(1275)) - Shutting down StatusHttpServer
> 2011-03-16 13:48:02,349 ERROR mapred.TaskTracker
> (TaskTracker.java:offerService(1609)) - Caught exception:
> java.io.IOException: Call to localhost/127.0.0.1:57512 failed on local
> exception: java.nio.channels.ClosedByInterruptException
> {noformat}
> Note there is over one minute between the first line and the second.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2173) Race condition in TestBlockFixer causes intermittent failure
[ https://issues.apache.org/jira/browse/MAPREDUCE-2173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016929#comment-13016929 ] Hudson commented on MAPREDUCE-2173: --- Integrated in Hadoop-Mapreduce-trunk #643 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/]) > Race condition in TestBlockFixer causes intermittent failure > > > Key: MAPREDUCE-2173 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2173 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Patrick Kling >Assignee: Patrick Kling > Fix For: 0.22.0 > > Attachments: MAPREDUCE-2173.patch > > > TestBlockFixer sometimes fails in reportCorruptBlocks because a corrupt block > is deleted before in.readFully is called. This causes a BlockMissingException > instead of the expected ChecksumException. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2311) TestFairScheduler failing on trunk
[ https://issues.apache.org/jira/browse/MAPREDUCE-2311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016931#comment-13016931 ] Hudson commented on MAPREDUCE-2311: --- Integrated in Hadoop-Mapreduce-trunk #643 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/]) > TestFairScheduler failing on trunk > -- > > Key: MAPREDUCE-2311 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2311 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: contrib/fair-share >Affects Versions: 0.22.0, 0.23.0 >Reporter: Todd Lipcon >Assignee: Scott Chen >Priority: Blocker > Fix For: 0.22.0, 0.23.0 > > Attachments: MAPREDUCE-2311.txt, test.result > > > Most of the test cases in this test are failing on trunk, unclear how long > since the contrib tests weren't running while the core tests were failed. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-993) bin/hadoop job -events <#-of-events> help message is confusing
[ https://issues.apache.org/jira/browse/MAPREDUCE-993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016933#comment-13016933 ] Hudson commented on MAPREDUCE-993: -- Integrated in Hadoop-Mapreduce-trunk #643 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/]) > bin/hadoop job -events <#-of-events> help message is > confusing > - > > Key: MAPREDUCE-993 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-993 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: jobtracker >Affects Versions: 0.21.0 >Reporter: Iyappan Srinivasan >Assignee: Harsh J Chouraria >Priority: Minor > Fix For: 0.23.0 > > Attachments: mapreduce.993.r1.diff > > > More explanation needs to be there like a) events always start from 1 b) the > message could be like where > starts from 1. This will give teh end user idea as to > what to enter. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2172) test-patch.properties contains incorrect/version-dependent values of OK_FINDBUGS_WARNINGS and OK_RELEASEAUDIT_WARNINGS
[
https://issues.apache.org/jira/browse/MAPREDUCE-2172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016932#comment-13016932
]
Hudson commented on MAPREDUCE-2172:
---
Integrated in Hadoop-Mapreduce-trunk #643 (See
[https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/])
> test-patch.properties contains incorrect/version-dependent values of
> OK_FINDBUGS_WARNINGS and OK_RELEASEAUDIT_WARNINGS
> --
>
> Key: MAPREDUCE-2172
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2172
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Environment: FindBugs 1.3.4
>Reporter: Patrick Kling
>Assignee: Nigel Daley
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2172.patch
>
>
> Running ant test-patch with an empty patch yields 25 findbugs warning and 3
> release audit warnings (rather than the 0 findbugs warnings and 1 release
> audit warning specified in test-patch.properties):
> {code}
> [exec] -1 overall.
> [exec]
> [exec] +1 @author. The patch does not contain any @author tags.
> [exec]
> [exec] -1 tests included. The patch doesn't appear to include any new or
> modified tests.
> [exec] Please justify why no new tests are needed for
> this patch.
> [exec] Also please list what manual steps were
> performed to verify this patch.
> [exec]
> [exec] +1 javadoc. The javadoc tool did not generate any warning
> messages.
> [exec]
> [exec] +1 javac. The applied patch does not increase the total number of
> javac compiler warnings.
> [exec]
> [exec] -1 findbugs. The patch appears to introduce 25 new Findbugs
> warnings.
> [exec]
> [exec] -1 release audit. The applied patch generated 3 release audit
> warnings (more than the trunk's current 1 warnings).
> [exec]
> [exec] +1 system test framework. The patch passed system test framework
> compile.
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-1752) Implement getFileBlockLocations in HarFilesystem
[ https://issues.apache.org/jira/browse/MAPREDUCE-1752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016927#comment-13016927 ] Hudson commented on MAPREDUCE-1752: --- Integrated in Hadoop-Mapreduce-trunk #643 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/]) > Implement getFileBlockLocations in HarFilesystem > > > Key: MAPREDUCE-1752 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1752 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: harchive >Reporter: Dmytro Molkov >Assignee: Dmytro Molkov > Fix For: 0.23.0 > > Attachments: MAPREDUCE-1752.2.patch, MAPREDUCE-1752.3.patch, > MR-1752.patch > > > To efficiently run map reduce on the data that has been HAR'ed it will be > great to actually implement getFileBlockLocations for a given filename. > This way the JobTracker will have information about data locality and will > schedule tasks appropriately. > I believe the overhead introduced by doing lookups in the index files can be > smaller than that of copying data over the wire. > Will upload the patch shortly, but would love to get some feedback on this. > And any ideas on how to test it are very welcome. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-1809) Ant build changes for Streaming system tests in contrib projects.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016928#comment-13016928 ] Hudson commented on MAPREDUCE-1809: --- Integrated in Hadoop-Mapreduce-trunk #643 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/]) > Ant build changes for Streaming system tests in contrib projects. > - > > Key: MAPREDUCE-1809 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1809 > Project: Hadoop Map/Reduce > Issue Type: Task > Components: build >Affects Versions: 0.21.0 >Reporter: Vinay Kumar Thota >Assignee: Vinay Kumar Thota > Fix For: 0.21.1 > > Attachments: 1809-ydist-security.patch, 1809-ydist-security.patch, > MAPREDUCE-1809.patch, MAPREDUCE-1809.patch, MAPREDUCE-1809.patch, > MAPREDUCE-1809.patch, MAPREDUCE-1809.patch, MAPREDUCE-1809.patch, > MAPREDUCE-1809.patch, MAPREDUCE-1809.patch, MAPREDUCE-1809.patch > > > Implementing new target( test-system) in build-contrib.xml file for executing > the system test that are in contrib projects. Also adding 'subant' target in > aop.xml that calls the build-contrib.xml file for system tests. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2314) configure files that are generated as part of the released tarball need to have executable bit set
[ https://issues.apache.org/jira/browse/MAPREDUCE-2314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016930#comment-13016930 ] Hudson commented on MAPREDUCE-2314: --- Integrated in Hadoop-Mapreduce-trunk #643 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/]) > configure files that are generated as part of the released tarball need to > have executable bit set > -- > > Key: MAPREDUCE-2314 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2314 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: build >Affects Versions: 0.22.0 >Reporter: Roman Shaposhnik >Assignee: Roman Shaposhnik > Fix For: 0.22.0 > > Attachments: MAPREDUCE-2314.patch > > > Currently the configure files that are packaged in a tarball are -rw-rw-r-- -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2179) RaidBlockSender.java compilation fails
[
https://issues.apache.org/jira/browse/MAPREDUCE-2179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016925#comment-13016925
]
Hudson commented on MAPREDUCE-2179:
---
Integrated in Hadoop-Mapreduce-trunk #643 (See
[https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/])
> RaidBlockSender.java compilation fails
> --
>
> Key: MAPREDUCE-2179
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2179
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: contrib/raid
>Affects Versions: 0.22.0
>Reporter: Giridharan Kesavan
>Assignee: Ramkumar Vadali
>Priority: Blocker
> Fix For: 0.22.0
>
> Attachments: MAPREDUCE-2179.patch
>
>
> https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/490/consoleFull
> Mapreduce trunk compilation is broken with
> compile:
> [echo] contrib: raid
> [javac] Compiling 27 source files to
> /grid/0/hudson/hudson-slave/workspace/Hadoop-Mapreduce-trunk/trunk/build/contrib/raid/classes
> [javac]
> /grid/0/hudson/hudson-slave/workspace/Hadoop-Mapreduce-trunk/trunk/src/contrib/raid/src/java/org/apache/hadoop/hdfs/server/datanode/RaidBlockSender.java:71:
> cannot find symbol
> [javac] symbol : class BlockTransferThrottler
> [javac] location: class
> org.apache.hadoop.hdfs.server.datanode.RaidBlockSender
> [javac] private BlockTransferThrottler throttler;
> [javac] ^
> [javac]
> /grid/0/hudson/hudson-slave/workspace/Hadoop-Mapreduce-trunk/trunk/src/contrib/raid/src/java/org/apache/hadoop/hdfs/server/datanode/RaidBlockSender.java:377:
> cannot find symbol
> [javac] symbol : class BlockTransferThrottler
> [javac] location: class
> org.apache.hadoop.hdfs.server.datanode.RaidBlockSender
> [javac] BlockTransferThrottler throttler) throws
> IOException {
> [javac] ^
> [javac] Note: Some input files use or override a deprecated API.
> [javac] Note: Recompile with -Xlint:deprecation for details.
> [javac] Note: Some input files use unchecked or unsafe operations.
> [javac] Note: Recompile with -Xlint:unchecked for details.
> [javac] 2 errors
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-1706) Log RAID recoveries on HDFS
[ https://issues.apache.org/jira/browse/MAPREDUCE-1706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016926#comment-13016926 ] Hudson commented on MAPREDUCE-1706: --- Integrated in Hadoop-Mapreduce-trunk #643 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/]) > Log RAID recoveries on HDFS > --- > > Key: MAPREDUCE-1706 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1706 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: contrib/raid >Affects Versions: 0.22.0 >Reporter: Rodrigo Schmidt >Assignee: Scott Chen > Fix For: 0.23.0 > > Attachments: MAPREDUCE-1706.rebased.txt, MAPREDUCE-1706.txt, > test.result, test.result > > > It would be good to have a way to centralize all the recovery logs, since > recovery can be executed by any hdfs client. The best place to store this > information is HDFS itself. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-1915) IndexCache - getIndexInformation - check reduce index Out Of Bounds
[ https://issues.apache.org/jira/browse/MAPREDUCE-1915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016918#comment-13016918 ] Hudson commented on MAPREDUCE-1915: --- Integrated in Hadoop-Mapreduce-trunk #643 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/]) > IndexCache - getIndexInformation - check reduce index Out Of Bounds > --- > > Key: MAPREDUCE-1915 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1915 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: tasktracker >Affects Versions: 0.20.1, 0.20.2, 0.20.3, 0.21.0, 0.22.0 >Reporter: Rares Vernica >Assignee: Priyo Mustafi >Priority: Trivial > Fix For: 0.22.0 > > Attachments: MAPREDUCE-1915.patch, MAPREDUCE-1915_1.patch > > > When checking if the "reduce" index is out of bounds the check should be > info.mapSpillRecord.size() <= reduce > instead of: > info.mapSpillRecord.size() < reduce > Not a big bug since an Out Of Bounds is thrown downstream anyway. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-1754) Replace mapred.persmissions.supergroup with an acl : mapreduce.cluster.administrators
[ https://issues.apache.org/jira/browse/MAPREDUCE-1754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016923#comment-13016923 ] Hudson commented on MAPREDUCE-1754: --- Integrated in Hadoop-Mapreduce-trunk #643 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/]) > Replace mapred.persmissions.supergroup with an acl : > mapreduce.cluster.administrators > - > > Key: MAPREDUCE-1754 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1754 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: jobtracker >Affects Versions: 0.22.0 >Reporter: Amareshwari Sriramadasu >Assignee: Amareshwari Sriramadasu > Fix For: 0.22.0 > > Attachments: patch-1754-1.txt, patch-1754-1.txt, > patch-1754-ydist.txt, patch-1754.txt > > > mapred.permissions.supergroup should be replaced with an acl so that it does > not restrict the admins to a single group. > See more details on MAPREDUCE-1542. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2224) Synchronization bugs in JvmManager
[ https://issues.apache.org/jira/browse/MAPREDUCE-2224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016924#comment-13016924 ] Hudson commented on MAPREDUCE-2224: --- Integrated in Hadoop-Mapreduce-trunk #643 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/]) > Synchronization bugs in JvmManager > -- > > Key: MAPREDUCE-2224 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2224 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: tasktracker >Affects Versions: 0.22.0 >Reporter: Todd Lipcon >Assignee: Todd Lipcon >Priority: Critical > Fix For: 0.22.0 > > Attachments: mapreduce-2224-cdh3.txt, mapreduce-2224.txt, > mapreduce-2224.txt, mapreduce-2224.txt, mapreduce-2224.txt > > > JvmManager.JvmManagerForType has several HashMap members that are > inconsistently synchronized. I've seen sporadic NPEs in the 0.20 version of > this code which has similar bugs. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2225) MultipleOutputs should not require the use of 'Writable'
[ https://issues.apache.org/jira/browse/MAPREDUCE-2225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016920#comment-13016920 ] Hudson commented on MAPREDUCE-2225: --- Integrated in Hadoop-Mapreduce-trunk #643 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/]) > MultipleOutputs should not require the use of 'Writable' > > > Key: MAPREDUCE-2225 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2225 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: job submission >Affects Versions: 0.20.1 > Environment: Linux >Reporter: Harsh J Chouraria >Assignee: Harsh J Chouraria >Priority: Blocker > Fix For: 0.23.0 > > Attachments: multipleoutputs.nowritables.r1.diff, > multipleoutputs.nowritables.r2.diff, multipleoutputs.nowritables.r2.diff > > Original Estimate: 1m > Remaining Estimate: 1m > > MultipleOutputs right now requires for Key/Value classes to utilize the > Writable and WritableComparable interfaces, and fails if the associated > key/value classes aren't doing so. > With support for alternates like Avro serialization, using Writables isn't > necessary and thus the MO class must not strictly check for them. > And since comparators may be given separately, key class doesn't need to be > checked for implementing a comparable (although it is good design if the key > class does implement Comparable at least). > Am not sure if this brings about an incompatible change (does Java have BIC? > No idea). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-1159) Limit Job name on jobtracker.jsp to be 80 char long
[ https://issues.apache.org/jira/browse/MAPREDUCE-1159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016921#comment-13016921 ] Hudson commented on MAPREDUCE-1159: --- Integrated in Hadoop-Mapreduce-trunk #643 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/]) > Limit Job name on jobtracker.jsp to be 80 char long > --- > > Key: MAPREDUCE-1159 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1159 > Project: Hadoop Map/Reduce > Issue Type: Improvement >Affects Versions: 0.22.0 >Reporter: Zheng Shao >Assignee: Harsh J Chouraria >Priority: Trivial > Fix For: 0.22.0, 0.23.0 > > Attachments: MAPREDUCE-1159.r1.patch, MAPREDUCE-1159.r2.patch, > MAPREDUCE-1159.trunk.patch > > > Sometimes a user submits a job with a very long job name. That made > jobtracker.jsp very hard to read. > We should limit the size of the job name. User can see the full name when > they click on the job. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2223) TestMRCLI might fail on Ubuntu with default /etc/hosts
[ https://issues.apache.org/jira/browse/MAPREDUCE-2223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016922#comment-13016922 ] Hudson commented on MAPREDUCE-2223: --- Integrated in Hadoop-Mapreduce-trunk #643 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/]) > TestMRCLI might fail on Ubuntu with default /etc/hosts > -- > > Key: MAPREDUCE-2223 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2223 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: test >Affects Versions: 0.21.1 >Reporter: Konstantin Boudnik >Assignee: Konstantin Boudnik >Priority: Minor > Fix For: 0.21.1 > > Attachments: MAPREDUCE-2223.patch > > > Depending on the order of entries in /etc/hosts, TestCLI can fail. This is > because it sets fs.default.name to "localhost", and then the bound IPC socket > on the NN side reports its hostname as "foobar-host" if the entry for > 127.0.0.1 lists "foobar-host" before "localhost". This seems to be the > default in some versions of Ubuntu. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-1996) API: Reducer.reduce() method detail misstatement
[ https://issues.apache.org/jira/browse/MAPREDUCE-1996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016914#comment-13016914 ] Hudson commented on MAPREDUCE-1996: --- Integrated in Hadoop-Mapreduce-trunk #643 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/]) > API: Reducer.reduce() method detail misstatement > > > Key: MAPREDUCE-1996 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1996 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: documentation >Affects Versions: 0.20.1 > Environment: Seen in Hadoop 0.20.2 API and Hadoop 0.19.x API >Reporter: Glynn Durham >Assignee: Harsh J Chouraria >Priority: Trivial > Fix For: 0.22.0, 0.23.0 > > Attachments: MAPREDUCE-1996.r1.diff > > Original Estimate: 5m > Remaining Estimate: 5m > > method detail for Reducer.reduce() method has paragraph starting: > "Applications can use the Reporter provided to report progress or just > indicate that they are alive. In scenarios where the application takes an > insignificant amount of time to process individual key/value pairs, this is > crucial since the framework might assume that the task has timed-out and kill > that task. " > s/an insignificant amount of time/a significant amount of time/ -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2184) Port DistRaid.java to new mapreduce API
[ https://issues.apache.org/jira/browse/MAPREDUCE-2184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016916#comment-13016916 ] Hudson commented on MAPREDUCE-2184: --- Integrated in Hadoop-Mapreduce-trunk #643 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/]) > Port DistRaid.java to new mapreduce API > --- > > Key: MAPREDUCE-2184 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2184 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: contrib/raid >Reporter: Ramkumar Vadali >Assignee: Ramkumar Vadali > Fix For: 0.22.0 > > Attachments: MAPREDUCE-2184.patch > > > DistRaid.java was implemented with the older mapred API, this task is for > porting it to the new API -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2034) TestSubmitJob triggers NPE instead of permissions error
[ https://issues.apache.org/jira/browse/MAPREDUCE-2034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016917#comment-13016917 ] Hudson commented on MAPREDUCE-2034: --- Integrated in Hadoop-Mapreduce-trunk #643 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/]) > TestSubmitJob triggers NPE instead of permissions error > --- > > Key: MAPREDUCE-2034 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2034 > Project: Hadoop Map/Reduce > Issue Type: Test > Components: test >Affects Versions: 0.22.0 >Reporter: Todd Lipcon >Assignee: Todd Lipcon >Priority: Trivial > Fix For: 0.22.0 > > Attachments: mapreduce-2034.txt > > > TestSubmitJob.testSecureJobExecution catches _any_ IOException and assumes a > permissions error has been caught. In fact, it was passing an invalid path > name to the NameNode and triggering an NPE, not a Permission denied error, in > one case, but the test was not specific enough to detect this. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2285) MiniMRCluster does not start after ant test-patch
[
https://issues.apache.org/jira/browse/MAPREDUCE-2285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016915#comment-13016915
]
Hudson commented on MAPREDUCE-2285:
---
Integrated in Hadoop-Mapreduce-trunk #643 (See
[https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/])
> MiniMRCluster does not start after ant test-patch
> -
>
> Key: MAPREDUCE-2285
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2285
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
>Reporter: Ramkumar Vadali
>Assignee: Todd Lipcon
>Priority: Blocker
> Fix For: 0.22.0
>
> Attachments: cp-bad, cp-good, fix-build.diff
>
>
> Any test using MiniMRCluster hangs in the MiniMRCluster constructor after
> running ant test-patch. Steps to reproduce:
> 1. ant -Dpatch.file= -Dforrest.home= forrest> -Dfindbugs.home= -Dscratch.dir=/tmp/testpatch
> -Djava5.home= test-patch
> 2. Run any test that creates MiniMRCluster, say ant test
> -Dtestcase=TestFileArgs (contrib/streaming)
> Expected result: Test should succeed
> Actual result: Test hangs in MiniMRCluster.. This does not happen if
> we run ant clean after ant test-patch
> Test output:
> {code}
> [junit] 11/01/27 12:11:43 INFO ipc.Server: IPC Server handler 3 on 58675:
> starting
> [junit] 11/01/27 12:11:43 INFO mapred.TaskTracker: TaskTracker up at:
> localhost.localdomain/127.0.0.1:58675
> [junit] 11/01/27 12:11:43 INFO mapred.TaskTracker: Starting tracker
> tracker_host0.foo.com:localhost.localdomain/127.0.0.1:58675
> [junit] 11/01/27 12:11:44 INFO ipc.Client: Retrying connect to server:
> localhost/127.0.0.1:0. Already tried 0 time(s).
> [junit] 11/01/27 12:11:45 INFO ipc.Client: Retrying connect to server:
> localhost/127.0.0.1:0. Already tried 1 time(s).
> [junit] 11/01/27 12:11:46 INFO ipc.Client: Retrying connect to server:
> localhost/127.0.0.1:0. Already tried 2 time(s).
> [junit] 11/01/27 12:11:47 INFO ipc.Client: Retrying connect to server:
> localhost/127.0.0.1:0. Already tried 3 time(s).
> [junit] 11/01/27 12:11:48 INFO ipc.Client: Retrying connect to server:
> localhost/127.0.0.1:0. Already tried 4 time(s).
> [junit] 11/01/27 12:11:49 INFO ipc.Client: Retrying connect to server:
> localhost/127.0.0.1:0. Already tried 5 time(s).
> [junit] 11/01/27 12:11:50 INFO ipc.Client: Retrying connect to server:
> localhost/127.0.0.1:0. Already tried 6 time(s).
> [junit] 11/01/27 12:11:51 INFO ipc.Client: Retrying connect to server:
> localhost/127.0.0.1:0. Already tried 7 time(s).
> [junit] 11/01/27 12:11:52 INFO ipc.Client: Retrying connect to server:
> localhost/127.0.0.1:0. Already tried 8 time(s).
> [junit] 11/01/27 12:11:53 INFO ipc.Client: Retrying connect to server:
> localhost/127.0.0.1:0. Already tried 9 time(s).
> [junit] 11/01/27 12:11:53 INFO ipc.RPC: Server at localhost/127.0.0.1:0
> not available yet, Z...
> {code}
> Stack trace:
> {code}
> at java.lang.Thread.sleep(Native Method)
> at
> org.apache.hadoop.ipc.Client$Connection.handleConnectionFailure(Client.java:611)
> at
> org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:429)
> - locked <0x7f3b8dc08700> (a
> org.apache.hadoop.ipc.Client$Connection)
> at
> org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:504)
> - locked <0x7f3b8dc08700> (a
> org.apache.hadoop.ipc.Client$Connection)
> at
> org.apache.hadoop.ipc.Client$Connection.access$2000(Client.java:206)
> at org.apache.hadoop.ipc.Client.getConnection(Client.java:1164)
> at org.apache.hadoop.ipc.Client.call(Client.java:1008)
> at
> org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:198)
> at org.apache.hadoop.mapred.$Proxy11.getProtocolVersion(Unknown
> Source)
> at
> org.apache.hadoop.ipc.WritableRpcEngine.getProxy(WritableRpcEngine.java:235)
> at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:275)
> at org.apache.hadoop.ipc.RPC.waitForProxy(RPC.java:206)
> at org.apache.hadoop.ipc.RPC.waitForProxy(RPC.java:185)
> at org.apache.hadoop.ipc.RPC.waitForProxy(RPC.java:169)
> at org.apache.hadoop.mapred.TaskTracker$2.run(TaskTracker.java:699)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1142)
> at
> org.apache.hadoop.mapred.TaskTracker.initialize(TaskTracker.java:695)
> - locked <0x7f3b8ccc3870> (a org.apache.hadoop.mapred.TaskTracker)
> at org.apache.hadoop.mapred.TaskTracker.(TaskTracker.java:139
