Re: JobConf and MiniMapRedCluster

2013-04-17 Thread Harsh J
Ah yes, we ended up un-deprecating the old API and are currently supporting both across all releases. 1.x releases too got a few patches un-deprecating the API classes. On Thu, Apr 18, 2013 at 11:25 AM, Jay Vyas wrote: > Okay, thanks Ill look into this JIRA . > > it is clear from some light g

Re: JobConf and MiniMapRedCluster

2013-04-17 Thread Jay Vyas
Okay, thanks Ill look into this JIRA . it is clear from some light googling that, at some point ** some version of JobConf was deprecated, and that maybe again it was undeprecated or maybe moved** Will have to look into this more formally to really determine what's going on. On Apr

Re: JobConf and MiniMapRedCluster

2013-04-17 Thread Harsh J
Am not sure I totally understand yet. JobConf isn't deprecated, and is still a (and the only) valid way to use the older mapred.* API. If you mean we should shift the tests over to the new API (mapreduce.*, and Job) then am all for it. The Tool+Configured extensions are good for ToolRunner.run(…)

Re: JobConf and MiniMapRedCluster

2013-04-17 Thread Jay Vyas
Only one response, inline below... Certainly I will file a JIRA and some updates if this makes sense :) Would love to bring the minimrcluster class up to date! On Apr 18, 2013, at 12:49 AM, Harsh J wrote: > Why do you imagine a test case would need the Configured and Tool > interfaces, wh

Re: JobConf and MiniMapRedCluster

2013-04-17 Thread Harsh J
Why do you imagine a test case would need the Configured and Tool interfaces, which are more useful for actual client apps? Or did you mean these should support running Tool apps? > Any plans to update this or should file a JIRA? No plans as far as I'm aware; please do file a JIRA with a patch i

[jira] [Created] (MAPREDUCE-5163) Update MR App after YARN-441

2013-04-17 Thread Vinod Kumar Vavilapalli (JIRA)
Vinod Kumar Vavilapalli created MAPREDUCE-5163: -- Summary: Update MR App after YARN-441 Key: MAPREDUCE-5163 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5163 Project: Hadoop Map/Red

[jira] [Created] (MAPREDUCE-5162) Add ability to change the queue a job has been assigned to with JobClient

2013-04-17 Thread Jeff Lord (JIRA)
Jeff Lord created MAPREDUCE-5162: Summary: Add ability to change the queue a job has been assigned to with JobClient Key: MAPREDUCE-5162 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5162 Proje

Re: [VOTE] Release Apache Hadoop 0.23.7

2013-04-17 Thread Sandy Ryza
+1 (non-binding) Built from source and ran a couple of MR examples on a single node cluster. -Sandy On Wed, Apr 17, 2013 at 12:03 PM, Siddharth Seth wrote: > +1 (binding). > Verified checksums and signature. > Built from source tar, deployed a single node cluster (CapacityScheduler) > and tried

Re: [VOTE] Release Apache Hadoop 2.0.4-alpha

2013-04-17 Thread Robert Evans
+1 (binding) Downloaded the tar ball and ran some simple jobs. --Bobby Evans On 4/17/13 2:01 PM, "Siddharth Seth" wrote: >+1 (binding) >Verified checksums and signatures. >Built from the source tar, deployed a single node cluster and tested a >couple of simple MR jobs. > >- Sid > > >On Fri, Ap

[jira] [Created] (MAPREDUCE-5161) CombineFileInputFormat fix for paths not on default FS merge from branch-1 to branch-1-win

2013-04-17 Thread Chris Nauroth (JIRA)
Chris Nauroth created MAPREDUCE-5161: Summary: CombineFileInputFormat fix for paths not on default FS merge from branch-1 to branch-1-win Key: MAPREDUCE-5161 URL: https://issues.apache.org/jira/browse/MAPREDUC

[jira] [Resolved] (MAPREDUCE-5140) MR part of YARN-514

2013-04-17 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-5140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bikas Saha resolved MAPREDUCE-5140. --- Resolution: Fixed Committed to trunk and branch-2 > MR part of YARN-514

Re: combiner without reducer

2013-04-17 Thread Radim Kolar
Dne 16.4.2013 11:08, Arpan Rajani napsal(a): Radim, Is it not happening if you set reducers = 0 ? if you set reducers = 0, then class set as combiner is ignored. My proposal is if you set reducers = 0 and combiner, then combiner is run. No API change is needed.

Re: [VOTE] Release Apache Hadoop 0.23.7

2013-04-17 Thread Siddharth Seth
+1 (binding). Verified checksums and signature. Built from source tar, deployed a single node cluster (CapacityScheduler) and tried a couple of simple MR jobs. - Sid On Thu, Apr 11, 2013 at 12:55 PM, Thomas Graves wrote: > I've created a release candidate (RC0) for hadoop-0.23.7 that I would li

Re: [VOTE] Release Apache Hadoop 2.0.4-alpha

2013-04-17 Thread Siddharth Seth
+1 (binding) Verified checksums and signatures. Built from the source tar, deployed a single node cluster and tested a couple of simple MR jobs. - Sid On Fri, Apr 12, 2013 at 2:56 PM, Arun C Murthy wrote: > Folks, > > I've created a release candidate (RC2) for hadoop-2.0.4-alpha that I would >

[jira] [Created] (MAPREDUCE-5160) Aggregatewordcount and aggregatewordhist in hadoop-1 examples can not find their inner classes when running on Yarn

2013-04-17 Thread Zhijie Shen (JIRA)
Zhijie Shen created MAPREDUCE-5160: -- Summary: Aggregatewordcount and aggregatewordhist in hadoop-1 examples can not find their inner classes when running on Yarn Key: MAPREDUCE-5160 URL: https://issues.apache.org

Re: [VOTE] Release Apache Hadoop 2.0.4-alpha

2013-04-17 Thread Thomas Graves
+1 (binding). I downloaded, verified signatures and checksums, built, and ran small jobs on a single node unsecure cluster. Tom On 4/12/13 4:56 PM, "Arun C Murthy" wrote: >Folks, > >I've created a release candidate (RC2) for hadoop-2.0.4-alpha that I >would like to release. > >The RC is avail

JobConf and MiniMapRedCluster

2013-04-17 Thread Jay Vyas
Hi guys: the MiniMapRedCluster seems like a very useful tool which I just discovered in this blog post : http://grepalex.com/2012/10/20/hadoop-unit-testing-with-minimrcluster/ But it looks like MiniMapRedCluster http://svn.apache.org/viewvc/hadoop/common/tags/release-1.0.3/src/test/org/apache/h

[jira] [Created] (MAPREDUCE-5159) Aggregatewordcount and aggregatewordhist in hadoop-1 examples are not binary compatible with hadoop-2 mapred.lib.aggregate

2013-04-17 Thread Zhijie Shen (JIRA)
Zhijie Shen created MAPREDUCE-5159: -- Summary: Aggregatewordcount and aggregatewordhist in hadoop-1 examples are not binary compatible with hadoop-2 mapred.lib.aggregate Key: MAPREDUCE-5159 URL: https://issues.apa

[jira] [Created] (MAPREDUCE-5158) Cleanup required when mapreduce.job.restart.recover is set to false

2013-04-17 Thread yeshavora (JIRA)
yeshavora created MAPREDUCE-5158: Summary: Cleanup required when mapreduce.job.restart.recover is set to false Key: MAPREDUCE-5158 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5158 Project: Ha

Re: [VOTE] Release Apache Hadoop 2.0.4-alpha

2013-04-17 Thread Chris Nauroth
+1 (non-binding) - downloaded binary tarball - verified signatures and checksums - deployed to 3 Ubuntu VMs: 1xNN, 1xRM, 2xDN, 2xNM, 1x2NN - tested multiple HDFS operations - ran wordcount MR job - verified that 2NN can take a checkpoint On Wed, Apr 17, 2013 at 1:24 AM, Chris Douglas wrote: >

[jira] [Created] (MAPREDUCE-5157) Sort in hadoop-1 examples is not binary compatible with hadoop-2 mapred.lib

2013-04-17 Thread Zhijie Shen (JIRA)
Zhijie Shen created MAPREDUCE-5157: -- Summary: Sort in hadoop-1 examples is not binary compatible with hadoop-2 mapred.lib Key: MAPREDUCE-5157 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5157

Re: [VOTE] Release Apache Hadoop 0.23.7

2013-04-17 Thread Chris Nauroth
+1 (non-binding) - downloaded binary tarball - verified signatures and checksums - deployed to 3 Ubuntu VMs: 1xNN, 1xRM, 2xDN, 2xNM, 1x2NN - tested multiple HDFS operations - ran wordcount MR job - verified that 2NN can take a checkpoint On Wed, Apr 17, 2013 at 1:17 AM, Chris Douglas wrote: >

Re: [VOTE] Release Apache Hadoop 2.0.4-alpha

2013-04-17 Thread Chris Douglas
+1 Verified checksum, signatures. Ran some tests, built the package. -C On Fri, Apr 12, 2013 at 2:56 PM, Arun C Murthy wrote: > Folks, > > I've created a release candidate (RC2) for hadoop-2.0.4-alpha that I would > like to release. > > The RC is available at: > http://people.apache.org/~acmur

Re: [VOTE] Release Apache Hadoop 0.23.7

2013-04-17 Thread Chris Douglas
+1 Verified checksums and signatures, ran some tests, built the tarball. -C On Thu, Apr 11, 2013 at 12:55 PM, Thomas Graves wrote: > I've created a release candidate (RC0) for hadoop-0.23.7 that I would like > to release. > > This release is a sustaining release with several important bug fixes

[jira] [Created] (MAPREDUCE-5156) Hadoop-examples-1.x.x.jar cannot run on Yarn

2013-04-17 Thread Zhijie Shen (JIRA)
Zhijie Shen created MAPREDUCE-5156: -- Summary: Hadoop-examples-1.x.x.jar cannot run on Yarn Key: MAPREDUCE-5156 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5156 Project: Hadoop Map/Reduce