Re: API incompatibility between Hadoop 1.0 and Hadoop 2.0

2013-02-08 Thread Harsh J
Ah ok, then that is probably the only base incompatibility you'll encounter (it was captured by jdiff IIRC) and is almost the only reason we've demanded jobs from 1.0 be recompiled before running on 2.0 (its a binary incompatible change made around ~0.21 time). On Sat, Feb 9, 2013 at 11:23 AM, lo

Re: API incompatibility between Hadoop 1.0 and Hadoop 2.0

2013-02-08 Thread lohit
Thanks. What I am more interested is list of APIs where users have to change code. I know that on YARN I can run jobs which still use org.apache.hadoop.mapred code. But recently we hit a case where we had to change code for one case to start with. For example JobContext was class in 1.0 while Inter

Re: mapreduce.jobtracker.address vs. yarn.resourcemanager.address

2013-02-08 Thread Harsh J
Yes, there isn't a "classic" version of mapreduce.framework.name now either. Its mostly being used as a pseudonym. MR1 no longer exists in 2.x releases/trunk. Note though that Oozie still relies on that specific property (JT address) and 0.22 clients may also perhaps be using it. We can deprecate

Re: API incompatibility between Hadoop 1.0 and Hadoop 2.0

2013-02-08 Thread Harsh J
Hey Lohit, It may not be exactly in the form you're looking for, but take a look at Dave's API Evolution page http://dbeech.github.com/hadoop-api-evolution.html that presents a matrix. Its psychedelic to look at, I know, but you asked for it ;) We also generate jdiff reports as part of the regula

Re: pre-historic record IO stuff, is this used anywhere?

2013-02-08 Thread Harsh J
Hadoop streaming is also tied to recordio as it is today: https://issues.apache.org/jira/browse/MAPREDUCE-3303, but it can be removed per Klaas. On Sat, Feb 9, 2013 at 6:48 AM, Alejandro Abdelnur wrote: > This seems to be used only in tests in common and in a standalone class in > streaming tests

pre-historic record IO stuff, is this used anywhere?

2013-02-08 Thread Alejandro Abdelnur
This seems to be used only in tests in common and in a standalone class in streaming tests. What is the purpose of these classes as they don't seem to be used in the any of the source that ends up in Hadoop? hadoop-common-project/hadoop-common/src/test/ddl/buffer.jr hadoop-common-project/hadoop-c

Re: [VOTE] Release hadoop-2.0.3-alpha

2013-02-08 Thread lohit
+1 Deployed on more than 100 nodes. Ran 30TB teragen/terasort. Will run few more over the weekend to test scheduler. Things looks stable. I do see few failures, but those I believe are hardware problems. Thanks, @lohitvijayarenu 2013/2/8 Konstantin Boudnik > The issue with the configuration is

Re: [VOTE] Release hadoop-2.0.3-alpha

2013-02-08 Thread Konstantin Boudnik
The issue with the configuration is raised (and adressed) in https://issues.apache.org/jira/browse/BIGTOP-841 Cos On Fri, Feb 08, 2013 at 04:25PM, Aaron T. Myers wrote: > +1 (binding) > > I downloaded the src tar ball, built it with the native bits enabled, > started up a little cluster, and

Re: [VOTE] Release hadoop-2.0.3-alpha

2013-02-08 Thread Aaron T. Myers
+1 (binding) I downloaded the src tar ball, built it with the native bits enabled, started up a little cluster, and ran some sample jobs. Things worked as expected. I also verified the signatures on the source artifact. I did bump into one little issue, but I don't think it should be considered a

[jira] [Resolved] (MAPREDUCE-4982) AM hung with one pending map task

2013-02-08 Thread Jason Lowe (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-4982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe resolved MAPREDUCE-4982. --- Resolution: Duplicate This is fixed by the changes in MAPREDUCE-4893. That has since bee

[jira] [Resolved] (MAPREDUCE-4922) Request with multiple data local nodes can cause NPE in AppSchedulingInfo

2013-02-08 Thread Sandy Ryza (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-4922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza resolved MAPREDUCE-4922. --- Resolution: Won't Fix This is the expected behavior > Request with multi

Re: mapreduce.jobtracker.address vs. yarn.resourcemanager.address

2013-02-08 Thread Alejandro Abdelnur
MR1 code is already gone from trunk and branch-2 Thx On Fri, Feb 8, 2013 at 2:27 PM, Vinod Kumar Vavilapalli < vino...@hortonworks.com> wrote: > > There was a time when there were thoughts of being able to run MR1 in > parallel with YARN while the later took off. So they were supposed to > coex

Re: mapreduce.jobtracker.address vs. yarn.resourcemanager.address

2013-02-08 Thread Vinod Kumar Vavilapalli
There was a time when there were thoughts of being able to run MR1 in parallel with YARN while the later took off. So they were supposed to coexist, hence the non-deprecation. I am not sure we formally did, but once it is cast in stone that MR1 cannot be run with Hadoop 2.0 *(by which time we

mapreduce.jobtracker.address vs. yarn.resourcemanager.address

2013-02-08 Thread Sandy Ryza
Is there a reason why mapreduce.jobtracker.address is not deprecated to yarn.resourcemanager.address? thanks, Sandy

[jira] [Created] (MAPREDUCE-4995) task-controller fails compilation on Mac

2013-02-08 Thread Chris Nauroth (JIRA)
Chris Nauroth created MAPREDUCE-4995: Summary: task-controller fails compilation on Mac Key: MAPREDUCE-4995 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4995 Project: Hadoop Map/Reduce

[jira] [Created] (MAPREDUCE-4994) Can't submit local job with hadoop jar -jt local

2013-02-08 Thread Sandy Ryza (JIRA)
Sandy Ryza created MAPREDUCE-4994: - Summary: Can't submit local job with hadoop jar -jt local Key: MAPREDUCE-4994 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4994 Project: Hadoop Map/Reduce

Re: API incompatibility between Hadoop 1.0 and Hadoop 2.0

2013-02-08 Thread lohit
Thanks Vinod. 2013/2/8 Vinod Kumar Vavilapalli > > It isn't complete yet, but Tom did some initial work comparing with 23 > which (preceeds 2.0): https://issues.apache.org/jira/browse/HADOOP-7738 > > We need to take that to the finish line. > > Thanks, > +Vinod > > On Feb 8, 2013, at 10:21 AM, l

Re: API incompatibility between Hadoop 1.0 and Hadoop 2.0

2013-02-08 Thread Zhijie Shen
It seems that there're no documented API changes since 2.0. I'm studying the backward compatibility, if there's anything I can help, please let me know. Thanks, Zhijie On Fri, Feb 8, 2013 at 10:37 AM, Vinod Kumar Vavilapalli < vino...@hortonworks.com> wrote: > > It isn't complete yet, but Tom d

Re: API incompatibility between Hadoop 1.0 and Hadoop 2.0

2013-02-08 Thread Vinod Kumar Vavilapalli
It isn't complete yet, but Tom did some initial work comparing with 23 which (preceeds 2.0): https://issues.apache.org/jira/browse/HADOOP-7738 We need to take that to the finish line. Thanks, +Vinod On Feb 8, 2013, at 10:21 AM, lohit wrote: > Hi Devs, > > Is there a documented list of API in

[jira] [Created] (MAPREDUCE-4993) AM thinks it was killed when an error occurs setting up a task container launch context

2013-02-08 Thread Jason Lowe (JIRA)
Jason Lowe created MAPREDUCE-4993: - Summary: AM thinks it was killed when an error occurs setting up a task container launch context Key: MAPREDUCE-4993 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4993

Re: Doubt about map reduce version 2

2013-02-08 Thread Robert Evans
Suresh, The 1.0 line is still the stable line and improvements there can have a large impact on existing users. That being said I think there will be a lot of movement to Yarn/MRv2 starting in the second half of this year and all of next year. Also YARN scheduling is a larger area for study beca

[jira] [Created] (MAPREDUCE-4992) AM hung in RecoveryService

2013-02-08 Thread Robert Parker (JIRA)
Robert Parker created MAPREDUCE-4992: Summary: AM hung in RecoveryService Key: MAPREDUCE-4992 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4992 Project: Hadoop Map/Reduce Issue Ty

Hadoop-Mapreduce-trunk - Build # 1338 - Failure

2013-02-08 Thread Apache Jenkins Server
See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1338/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 28871 lines...] Tests run: 3, Failures: 0, Errors: 0

[jira] [Created] (MAPREDUCE-4991) coverage for grid mix

2013-02-08 Thread Aleksey Gorshkov (JIRA)
Aleksey Gorshkov created MAPREDUCE-4991: --- Summary: coverage for grid mix Key: MAPREDUCE-4991 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4991 Project: Hadoop Map/Reduce Issue T