Re: Heads up - 2.0.5-beta

2013-04-26 Thread Arun C Murthy
On Apr 25, 2013, at 6:36 PM, Suresh Srinivas wrote: On Thu, Apr 25, 2013 at 6:34 PM, Arun C Murthy a...@hortonworks.com wrote: Similarly on HDFS side, can someone please help out by tagging features, bug-fixes, protocol/API changes etc.? This way we can ensure HDFS APIs protocols are

Cannot find JobTracker and TaskTracker classes in Hadoop 2.0.2-alpha

2013-04-26 Thread Thoihen Maibam
Hi, Can anyone help me out where can I find JobTracker and TaskTracker classes for the above releases, it's not present in hadoop-mapreduce-project. I was tracing through the source code from JobSubmission but lost the flow as I could not find the JobTracker and TaskTracker. Are these classes

Re: Heads up - 2.0.5-beta

2013-04-26 Thread Arun C Murthy
On Apr 25, 2013, at 7:31 PM, Roman Shaposhnik wrote: On Thu, Apr 25, 2013 at 6:34 PM, Arun C Murthy a...@hortonworks.com wrote: With that in mind, I really want to make a serious push to lock down APIs and wire-protocols for hadoop-2.0.5-beta. Thus, we can confidently support hadoop-2.x

[jira] [Resolved] (MAPREDUCE-5167) Update MR App after YARN-562

2013-04-26 Thread Vinod Kumar Vavilapalli (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-5167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kumar Vavilapalli resolved MAPREDUCE-5167. Resolution: Fixed Fix Version/s: 2.0.5-beta Hadoop

Re: Heads up - 2.0.5-beta

2013-04-26 Thread Eli Collins
On Fri, Apr 26, 2013 at 11:15 AM, Arun C Murthy a...@hortonworks.com wrote: On Apr 25, 2013, at 7:31 PM, Roman Shaposhnik wrote: On Thu, Apr 25, 2013 at 6:34 PM, Arun C Murthy a...@hortonworks.com wrote: With that in mind, I really want to make a serious push to lock down APIs and

[jira] [Resolved] (MAPREDUCE-5180) Running wordcount with -Ddfs.client.read.shortcircuit=true/false fails to get proper message on syslogs

2013-04-26 Thread yeshavora (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-5180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yeshavora resolved MAPREDUCE-5180. -- Resolution: Invalid Running wordcount with -Ddfs.client.read.shortcircuit=true/false

Environment setup for testing a patch submitting jobs from eclipse.

2013-04-26 Thread Thoihen Maibam
Hi All, Apology to everyone in case my question has been dealt before.If the question has been answered before please do provide me the link . Basically, I want to start contributing to Hadoop by submitting patches mostly from Map Reduce issues. 1. Assuming I create a patch, now I want to test

Re: Heads up - 2.0.5-beta

2013-04-26 Thread Luke Lu
If protocol compatibility of v2 and v3 is a goal, HADOOP-8990 should be a blocker for v2. __Luke On Fri, Apr 26, 2013 at 12:07 PM, Eli Collins e...@cloudera.com wrote: On Fri, Apr 26, 2013 at 11:15 AM, Arun C Murthy a...@hortonworks.com wrote: On Apr 25, 2013, at 7:31 PM, Roman Shaposhnik

Re: Versions - Confusion

2013-04-26 Thread Robert Evans
It is kind of complex. Up until 0.20 everything was fairly regular like you would expect. In 0.20 there was a split where security was added in to a branch and started to be numbered as 0.20.20X. But the other releases went on without security and became 0.21 and 0.22. 0.23 was created when

Re: Heads up - 2.0.5-beta

2013-04-26 Thread Suresh Srinivas
Eli, I will post a more detailed reply soon. But one small correction: I'm also not sure there's currently consensus on what an incompatible change is. For example, I think HADOOP-9151 is incompatible because it broke client/server wire compatibility with previous releases and any change that

Re: Heads up - 2.0.5-beta

2013-04-26 Thread Eli Collins
On Fri, Apr 26, 2013 at 2:42 PM, Suresh Srinivas sur...@hortonworks.com wrote: Eli, I will post a more detailed reply soon. But one small correction: I'm also not sure there's currently consensus on what an incompatible change is. For example, I think HADOOP-9151 is incompatible because it

Re: Environment setup for testing a patch submitting jobs from eclipse.

2013-04-26 Thread maisnam ns
Hi thoihen, It is extremely tough to debug the Hadoop core code, but it is not impossible.In fact debugging through eclipse and setting breakpoints may help considerably in understanding the flow of hadoop core logic . Steps are given below: 1. You may have to run all you hadoop daemons in your

Re: Heads up - 2.0.5-beta

2013-04-26 Thread Konstantin Shvachko
Arun, Could you please define the release plan and put it into vote. In accordance with the ByLaws. After this discussion of course. http://hadoop.apache.org/bylaws.html Release Plan Defines the timetable and actions for a release. The plan also nominates a Release Manager. Lazy majority of

Re: Heads up - 2.0.5-beta

2013-04-26 Thread Konstantin Shvachko
Arun, Suresh, Very exciting to hear about this final push to stable Hadoop 2. But I have a problem. Either with the plan or with the version number. I'll be arguing for the number change below rather than the plan. 1. Based on features listed by Suresh it looks that you plan a heavy feature-full

[jira] [Created] (MAPREDUCE-5185) When log aggregation not enabled, message should point to NM HTTP port, not IPC port

2013-04-26 Thread Sandy Ryza (JIRA)
Sandy Ryza created MAPREDUCE-5185: - Summary: When log aggregation not enabled, message should point to NM HTTP port, not IPC port Key: MAPREDUCE-5185 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5185

[jira] [Created] (MAPREDUCE-5186) mapreduce.job.max.split.locations causes some splits created by CombineFileInputFormat to fail

2013-04-26 Thread Sangjin Lee (JIRA)
Sangjin Lee created MAPREDUCE-5186: -- Summary: mapreduce.job.max.split.locations causes some splits created by CombineFileInputFormat to fail Key: MAPREDUCE-5186 URL:

Re: Heads up - 2.0.5-beta

2013-04-26 Thread Arun C Murthy
Konstantin, On Apr 26, 2013, at 4:34 PM, Konstantin Shvachko wrote: Do you think we can call the version you proposed to release 2.1.0 or 2.1.0-beta? The proposed new features imho do not exactly conform with the idea of dot-dot release, but definitely qualify for a major number change. I

Re: Heads up - 2.0.5-beta

2013-04-26 Thread Arun C Murthy
On Apr 26, 2013, at 12:07 PM, Eli Collins wrote: Arun, Suresh, Mind reviewing the following page Karthik put together on compatibility? http://wiki.apache.org/hadoop/Compatibility Sure. Will do. I just opened https://issues.apache.org/jira/browse/HADOOP-9517 to ensure we capture it for

[jira] [Created] (MAPREDUCE-5187) Create mapreduce command scripts on Windows

2013-04-26 Thread Chuan Liu (JIRA)
Chuan Liu created MAPREDUCE-5187: Summary: Create mapreduce command scripts on Windows Key: MAPREDUCE-5187 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5187 Project: Hadoop Map/Reduce

[jira] [Resolved] (MAPREDUCE-5158) Cleanup required when mapreduce.job.restart.recover is set to false

2013-04-26 Thread Arun C Murthy (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-5158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy resolved MAPREDUCE-5158. -- Resolution: Fixed Fix Version/s: 1.2.0 I just committed this after running

[jira] [Created] (MAPREDUCE-5188) error when verify FileType of RS_SOURCE in getCompanionBlocks in BlockPlacementPolicyRaid.java

2013-04-26 Thread junjin (JIRA)
junjin created MAPREDUCE-5188: - Summary: error when verify FileType of RS_SOURCE in getCompanionBlocks in BlockPlacementPolicyRaid.java Key: MAPREDUCE-5188 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5188

[jira] [Created] (MAPREDUCE-5189) Basic AM changes to support preemption requests (per YARN-45)

2013-04-26 Thread Carlo Curino (JIRA)
Carlo Curino created MAPREDUCE-5189: --- Summary: Basic AM changes to support preemption requests (per YARN-45) Key: MAPREDUCE-5189 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5189 Project: