[jira] [Created] (MAPREDUCE-5177) Move to common utils FileUtil#setReadable/Writable/Executable and FileUtil#canRead/Write/Execute

2013-04-23 Thread Ivan Mitic (JIRA)
Ivan Mitic created MAPREDUCE-5177: - Summary: Move to common utils FileUtil#setReadable/Writable/Executable and FileUtil#canRead/Write/Execute Key: MAPREDUCE-5177 URL: https://issues.apache.org/jira/browse/MAPREDUC

[jira] [Resolved] (MAPREDUCE-5175) Update MR App to not set envs that will be set by NMs anyways after YARN-561

2013-04-23 Thread Vinod Kumar Vavilapalli (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-5175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kumar Vavilapalli resolved MAPREDUCE-5175. Resolution: Fixed Fix Version/s: 2.0.5-beta Hadoop F

[jira] [Created] (MAPREDUCE-5176) Preemptable annotations (to support preemption in MR)

2013-04-23 Thread Carlo Curino (JIRA)
Carlo Curino created MAPREDUCE-5176: --- Summary: Preemptable annotations (to support preemption in MR) Key: MAPREDUCE-5176 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5176 Project: Hadoop Map/

[jira] [Created] (MAPREDUCE-5175) Update MR App to not set envs that will be set by NMs anyways after YARN-561

2013-04-23 Thread Vinod Kumar Vavilapalli (JIRA)
Vinod Kumar Vavilapalli created MAPREDUCE-5175: -- Summary: Update MR App to not set envs that will be set by NMs anyways after YARN-561 Key: MAPREDUCE-5175 URL: https://issues.apache.org/jira/browse/MA

CfP 2013 Workshop on Middleware for HPC and Big Data Systems (MHPC'13)

2013-04-23 Thread MHPC 2013
we apologize if you receive multiple copies of this message === CALL FOR PAPERS 2013 Workshop on Middleware for HPC and Big Data Systems MHPC '13 as part of Euro-Par 2013, Aachen, Germany

Re: [VOTE] Release Apache Hadoop 2.0.4-alpha

2013-04-23 Thread Arun C Murthy
With 15 +1s (8 binding), 1 -0 and no -1s the vote passes. Thanks to all who verified the bits, I'll push them out. thanks, Arun On Apr 12, 2013, at 2:56 PM, Arun C Murthy wrote: > Folks, > > I've created a release candidate (RC2) for hadoop-2.0.4-alpha that I would > like to release. > > The

Re: Why number of reducers should be less than number of reducer slots.

2013-04-23 Thread sudhakara st
Hi, "*mapred.job.reduce* "- number of reduce (map) tasks your job will has is depends on *mapred.tasktracker.reduce.tasks.maximum*( many reduce slot(s) you can have on each tasktracker, which decide number of total number reducer slots) property, recommendation for setting slightly fewer reducers

[jira] [Created] (MAPREDUCE-5174) History server gives wrong no of total maps and reducers

2013-04-23 Thread Devaraj K (JIRA)
Devaraj K created MAPREDUCE-5174: Summary: History server gives wrong no of total maps and reducers Key: MAPREDUCE-5174 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5174 Project: Hadoop Map/Red

Re: MapTask.java -sort part

2013-04-23 Thread Samaneh Shokuhi
Currently i am doing some experiments with WordCount example ,so considering this TestMerge.java ,it seems i need to change WordCount example and add MapOutputCopier and etc like what Mariappan done in TestMerge to nullify sort, right ? Samaneh On Tue, Apr 23, 2013 at 10:49 AM, Harsh J w

Re: MapTask.java -sort part

2013-04-23 Thread Harsh J
Yes as Mariappan has already pointed out; the sort is pluggable so the plug-in can also nullify it. See [1] for an example implementation of the MapOutputCollector interface (class MapOutputCopier) to start with. Per Mariappan, this does not do any sorting, but only merges. [1] - http://svn.apach

Re: MapTask.java -sort part

2013-04-23 Thread Samaneh Shokuhi
Hi Harsh, That is nice if branch-2 included that .Just one thing i want to be sure about that .In this patch MAPREDUCE-4807 (branch-2) , sort in mapper is avoided ? Becouse i need a version of hadoop without sorting part in mapper and thats the reason i wanted to modify MapTask to disable the mapp

Re: MapTask.java -sort part

2013-04-23 Thread Harsh J
Note that the Reducer also does a merge sort over the acquired map data; but am not sure if you looked there as well. Mariappan's pointed JIRA (MAPREDUCE-4807) is already available in the 2.0.3+ releases out today and in the current branch-2. It would be simpler to reuse that than do these modific