[jira] [Created] (MAPREDUCE-3226) Few reduce tasks hanging in a gridmix-run

2011-10-19 Thread Vinod Kumar Vavilapalli (Created) (JIRA)
Few reduce tasks hanging in a gridmix-run - Key: MAPREDUCE-3226 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3226 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2, task

[jira] [Resolved] (MAPREDUCE-3211) Fetch failures if used ephemeral port for property "mapreduce.shuffle.port".

2011-10-19 Thread Prashant Sharma (Resolved) (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-3211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Sharma resolved MAPREDUCE-3211. Resolution: Duplicate Hadoop Flags: Reviewed MAPREDUCE-2986

[jira] [Created] (MAPREDUCE-3225) Killing an unkown job throws NPE.

2011-10-19 Thread Mahadev konar (Created) (JIRA)
Killing an unkown job throws NPE. - Key: MAPREDUCE-3225 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3225 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.2

[jira] [Reopened] (MAPREDUCE-2629) Class loading quirk prevents inner class method compilation

2011-10-19 Thread Todd Lipcon (Reopened) (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-2629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon reopened MAPREDUCE-2629: I'm still seeing this in MRv2 > Class loading quirk prevents inner class m

Re: Map-Reduce without sorting

2011-10-19 Thread Arun C Murthy
I'm talking about the Hadoop impl of map-task. On Oct 19, 2011, at 3:45 PM, wrote: > Arun, > > From the Tenzing paper: > > > Hash table based aggregation is common in RDBMS sys- > tems. However, it is impossible to implement eciently > on the basic MapReduce framework, since the reducer al

[jira] [Created] (MAPREDUCE-3223) Remove MR1 configs from mapred-default.xml

2011-10-19 Thread Todd Lipcon (Created) (JIRA)
Remove MR1 configs from mapred-default.xml -- Key: MAPREDUCE-3223 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3223 Project: Hadoop Map/Reduce Issue Type: Bug Components: documentatio

[jira] [Created] (MAPREDUCE-3224) Remove MR1 configs from mapred-default.xml

2011-10-19 Thread Todd Lipcon (Created) (JIRA)
Remove MR1 configs from mapred-default.xml -- Key: MAPREDUCE-3224 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3224 Project: Hadoop Map/Reduce Issue Type: Bug Components: documentatio

Re: Map-Reduce without sorting

2011-10-19 Thread Milind.Bhandarkar
Arun, >From the Tenzing paper: Hash table based aggregation is common in RDBMS sys- tems. However, it is impossible to implement eciently on the basic MapReduce framework, since the reducer al- ways unnecessarily sorts the data by key. We enhanced the MapReduce framework to relax this restricti

Re: Map-Reduce without sorting

2011-10-19 Thread Arun C Murthy
Milind the map-side sort uses the partiion as the primary key. So, you still sort. See MR-1639 for more details. On Oct 19, 2011, at 3:22 PM, wrote: > How is that different from specifying a comparator that always returns > that k1 and k2 are equal regardless of k1 and k2 ? So, you will get on

Re: Map-Reduce without sorting

2011-10-19 Thread Arun C Murthy
Lets move this discussion to MR-1639, MR2 would allow for a really cool optimization there (which I hope works, see details on the jira). On Oct 19, 2011, at 2:42 PM, Zheng Shao wrote: > Google's Tenzing paper mentioned that they modified MR to make sorting in > reducer optional: > http://static

Re: Map-Reduce without sorting

2011-10-19 Thread Milind.Bhandarkar
How is that different from specifying a comparator that always returns that k1 and k2 are equal regardless of k1 and k2 ? So, you will get only partitioning, and not sorting. - Milind On 10/19/11 2:42 PM, "Zheng Shao" wrote: >Google's Tenzing paper mentioned that they modified MR to make sort

Re: Map-Reduce without sorting

2011-10-19 Thread Todd Lipcon
For historical interest, the JIRAs where this was discussed were MAPREDUCE-326 and MAPREDUCE-1639 On Wed, Oct 19, 2011 at 2:44 PM, Todd Lipcon wrote: > On Wed, Oct 19, 2011 at 2:42 PM, Zheng Shao wrote: >> Google's Tenzing paper mentioned that they modified MR to make sorting in >> reducer optio

Re: Map-Reduce without sorting

2011-10-19 Thread Todd Lipcon
On Wed, Oct 19, 2011 at 2:42 PM, Zheng Shao wrote: > Google's Tenzing paper mentioned that they modified MR to make sorting in > reducer optional: > http://static.googleusercontent.com/external_content/untrusted_dlcp/research.google.com/en/us/pubs/archive/37200.pdf > > Is there any plan to support

Map-Reduce without sorting

2011-10-19 Thread Zheng Shao
Google's Tenzing paper mentioned that they modified MR to make sorting in reducer optional: http://static.googleusercontent.com/external_content/untrusted_dlcp/research.google.com/en/us/pubs/archive/37200.pdf Is there any plan to support that in MR 2.0? Zheng

[jira] [Created] (MAPREDUCE-3222) ant test TestTaskContext failing on trunk

2011-10-19 Thread Hitesh Shah (Created) (JIRA)
ant test TestTaskContext failing on trunk - Key: MAPREDUCE-3222 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3222 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2

[jira] [Created] (MAPREDUCE-3221) ant test TestSubmitJob failing on trunk

2011-10-19 Thread Hitesh Shah (Created) (JIRA)
ant test TestSubmitJob failing on trunk --- Key: MAPREDUCE-3221 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3221 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2 Affe

[jira] [Created] (MAPREDUCE-3220) ant test TestCombineOutputCollector failing on trunk

2011-10-19 Thread Hitesh Shah (Created) (JIRA)
ant test TestCombineOutputCollector failing on trunk Key: MAPREDUCE-3220 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3220 Project: Hadoop Map/Reduce Issue Type: Sub-task

[jira] [Created] (MAPREDUCE-3219) ant test TestDelegationToken failing on trunk

2011-10-19 Thread Hitesh Shah (Created) (JIRA)
ant test TestDelegationToken failing on trunk - Key: MAPREDUCE-3219 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3219 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: m

[jira] [Created] (MAPREDUCE-3218) ant test TestTokenCache failing on trunk

2011-10-19 Thread Hitesh Shah (Created) (JIRA)
ant test TestTokenCache failing on trunk Key: MAPREDUCE-3218 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3218 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2 Af

[jira] [Created] (MAPREDUCE-3217) ant test TestAuditLogger fails on trunk

2011-10-19 Thread Hitesh Shah (Created) (JIRA)
ant test TestAuditLogger fails on trunk --- Key: MAPREDUCE-3217 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3217 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2 Affe

[jira] [Created] (MAPREDUCE-3216) ant test TestNoDefaultsJobConf fails on trunk

2011-10-19 Thread Hitesh Shah (Created) (JIRA)
ant test TestNoDefaultsJobConf fails on trunk - Key: MAPREDUCE-3216 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3216 Project: Hadoop Map/Reduce Issue Type: Sub-task Components: m

[jira] [Created] (MAPREDUCE-3215) org.apache.hadoop.mapreduce.TestNoJobSetupCleanup failing on trunk

2011-10-19 Thread Hitesh Shah (Created) (JIRA)
org.apache.hadoop.mapreduce.TestNoJobSetupCleanup failing on trunk -- Key: MAPREDUCE-3215 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3215 Project: Hadoop Map/Reduce Is

[jira] [Created] (MAPREDUCE-3214) ant mapreduce tests failing

2011-10-19 Thread Hitesh Shah (Created) (JIRA)
ant mapreduce tests failing --- Key: MAPREDUCE-3214 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3214 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0

Re: Delay scheduling in MR2?

2011-10-19 Thread Todd Lipcon
I filed MR-3210. Thanks On Tue, Oct 18, 2011 at 10:30 PM, Todd Lipcon wrote: > On Tue, Oct 18, 2011 at 6:31 PM, Arun C Murthy wrote: >> The CS has always treated rack-local same as node-local... so yes, the >> current behaviour is expected. > > Hrm - never knew that -- I'm usually a fairschedul

[jira] [Resolved] (MAPREDUCE-3199) TestJobMonitorAndPrint is broken on trunk

2011-10-19 Thread Vinod Kumar Vavilapalli (Resolved) (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-3199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kumar Vavilapalli resolved MAPREDUCE-3199. Resolution: Fixed Okay, apparently the protocol for HADOOP-7741

[jira] [Resolved] (MAPREDUCE-3003) Publish MR JARs to Maven snapshot repository

2011-10-19 Thread Vinod Kumar Vavilapalli (Resolved) (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-3003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kumar Vavilapalli resolved MAPREDUCE-3003. Resolution: Fixed Release Note: Yeah, sure. Done. > P

[jira] [Created] (MAPREDUCE-3213) AM should kill all running tasks on a node when it gets marked as blacklisted

2011-10-19 Thread Hitesh Shah (Created) (JIRA)
AM should kill all running tasks on a node when it gets marked as blacklisted - Key: MAPREDUCE-3213 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3213 Project: Hadoop M

[jira] [Created] (MAPREDUCE-3212) Message displays while executing yarn command should be proper

2011-10-19 Thread Bhallamudi Venkata Siva Kamesh (Created) (JIRA)
Message displays while executing yarn command should be proper -- Key: MAPREDUCE-3212 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3212 Project: Hadoop Map/Reduce Issue Type

[jira] [Resolved] (MAPREDUCE-2875) NM does not communicate Container crash to RM

2011-10-19 Thread Vinod Kumar Vavilapalli (Resolved) (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-2875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kumar Vavilapalli resolved MAPREDUCE-2875. Resolution: Duplicate Assignee: Siddharth Seth (was: Deva

[jira] [Created] (MAPREDUCE-3211) Fetch failures if used ephemeral port for property "mapreduce.shuffle.port".

2011-10-19 Thread Prashant Sharma (Created) (JIRA)
Fetch failures if used ephemeral port for property "mapreduce.shuffle.port". - Key: MAPREDUCE-3211 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3211 Project: Hadoop M

[jira] [Created] (MAPREDUCE-3210) Support delay scheduling for node locality in MR2's capacity scheduler

2011-10-19 Thread Todd Lipcon (Created) (JIRA)
Support delay scheduling for node locality in MR2's capacity scheduler -- Key: MAPREDUCE-3210 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3210 Project: Hadoop Map/Reduce