Re: Heads up: branch-2.1-beta

2013-06-19 Thread Roman Shaposhnik
On Tue, Jun 18, 2013 at 11:58 PM, Arun C Murthy wrote: > Ping. Any luck? Unfortunately I've ran into: https://issues.apache.org/jira/browse/HADOOP-9654 which was caused by the unrelated memory pressure on the NN, but it had an unfortunate side effect of making the rest of the testing stuck.

[jira] [Created] (MAPREDUCE-5334) TestContainerLauncherImpl is failing

2013-06-19 Thread Vinod Kumar Vavilapalli (JIRA)
Vinod Kumar Vavilapalli created MAPREDUCE-5334: -- Summary: TestContainerLauncherImpl is failing Key: MAPREDUCE-5334 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5334 Project: Hadoop

[jira] [Created] (MAPREDUCE-5333) Add test that verifies MRAM works correctly when sending requests with non-normalized capabilities

2013-06-19 Thread Alejandro Abdelnur (JIRA)
Alejandro Abdelnur created MAPREDUCE-5333: - Summary: Add test that verifies MRAM works correctly when sending requests with non-normalized capabilities Key: MAPREDUCE-5333 URL: https://issues.apache.org/ji

Re: Heads up: branch-2.1-beta

2013-06-19 Thread Roman Shaposhnik
On Tue, Jun 18, 2013 at 11:58 PM, Arun C Murthy wrote: > Ping. Any luck? The full bigtop build failed because of Giraph: http://bigtop01.cloudera.org:8080/view/Upstream-tests/job/Hadoop-2.1.0/label=centos5/4/console Now, I'm pretty sure that's an issue with Giraph itself (or more precisely

Re: InputFormat to regroup splits of underlying InputFormat to control number of map tasks

2013-06-19 Thread Nicolae Marasoiu
Hi, Our intention is to solve this in a generic context, not just file input. Thus the split class should be generic (very similar to CompositeInputSplit from mapred). We also already implement getRecordReader by iterating over record readers created by the decorated input format (this method i

Re: mapred.child.ulimit in MR2

2013-06-19 Thread Robert Evans
Sandy, I think it was something that was missed in the port to YARN and the dead code was cleaned up as part of HADOOP-8288. If you have a use case for it or are worried about backwards compatibility we can add it back in. It is not that hard, all it did was add 'ulimt -v ' to the shell script t

Re: InputFormat to regroup splits of underlying InputFormat to control number of map tasks

2013-06-19 Thread Robert Evans
This sounds similar to MultiFileInputFormat http://svn.apache.org/viewvc/hadoop/common/trunk/hadoop-mapreduce-project/h adoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apach e/hadoop/mapred/MultiFileInputFormat.java?revision=1239482&view=markup It would be nice if you could

Hadoop-Mapreduce-trunk - Build # 1462 - Failure

2013-06-19 Thread Apache Jenkins Server
See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1462/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 29128 lines...] Running org.apache.hadoop.mapreduce.

[jira] [Created] (MAPREDUCE-5332) Support token-preserving restart of history server

2013-06-19 Thread Jason Lowe (JIRA)
Jason Lowe created MAPREDUCE-5332: - Summary: Support token-preserving restart of history server Key: MAPREDUCE-5332 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5332 Project: Hadoop Map/Reduce

RE: Heads up: branch-2.1-beta

2013-06-19 Thread Devaraj k
Hi Arun, Is there any possibility of including YARN-41 in this release. Thanks Devaraj K -Original Message- From: Arun C Murthy [mailto:a...@hortonworks.com] Sent: 19 June 2013 12:29 To: mapreduce-dev@hadoop.apache.org; common-...@hadoop.apache.org; hdfs-...@hadoop.apache.org;

InputFormat to regroup splits of underlying InputFormat to control number of map tasks

2013-06-19 Thread Nicolae Marasoiu
Hi, When running map-reduce with many splits it would be nice from a performance perspective to have fewer splits while maintaining data locality, so that the overhead of running a map task (jvm creation, map executor ramp-up e.g. spring context, etc) be less impactful when frequently running m

Re: Heads up: branch-2.1-beta

2013-06-19 Thread Arun C Murthy
Ping. Any luck? On Jun 17, 2013, at 4:06 PM, Roman Shaposhnik wrote: > On Sun, Jun 16, 2013 at 5:14 PM, Arun C Murthy wrote: >> Roman, >> >> Is there a chance you can run the tests with the full stack built against >> branch-2.1-beta and help us know where we are? > > I will try to kick off