Re: Thinking about the next hadoop mainline release

2011-06-21 Thread Steve Loughran
On 17/06/2011 21:27, Allen Wittenauer wrote: On Jun 17, 2011, at 10:31 AM, Allen Wittenauer wrote: On Jun 17, 2011, at 12:17 AM, Eric Baldeschwieler wrote: Yahoo stands ready to help us (the Apache Hadoop Community) turn this new release into a stable release by running it through its 9

Re: [DISCUSSION] Thinking about 20.204 and beyond

2011-06-21 Thread Steve Loughran
On 18/06/2011 21:22, Roy T. Fielding wrote: olutely no reason that trunk cannot be packaged for release tomorrow as 0.23. There may be many reasons why it won't pass a release vote, but we probably aren't going to find them until somebody tries. One limitation with releases has always been

Re: [DISCUSSION] Thinking about 20.204 and beyond

2011-06-21 Thread Roy T. Fielding
On Jun 21, 2011, at 4:39 AM, Steve Loughran wrote: On 18/06/2011 21:22, Roy T. Fielding wrote: olutely no reason that trunk cannot be packaged for release tomorrow as 0.23. There may be many reasons why it won't pass a release vote, but we probably aren't going to find them until somebody

Re: [DISCUSSION] Thinking about 20.204 and beyond

2011-06-21 Thread Allen Wittenauer
On Jun 21, 2011, at 4:39 AM, Steve Loughran wrote: If I can actually bring up a heterogenous cluster here I believe there was a post in one of the mailing lists in the past 6 months where someone tried a mixed endian grid. It blew up big time.

Re: [DISCUSSION] Thinking about 20.204 and beyond

2011-06-21 Thread Ted Dunning
For the pain in doing this, it is probably better to just drop $10 and bring up a nice EC2 cluster with 10 m1.large instances using spot pricing for 5 hours. On Tue, Jun 21, 2011 at 11:43 AM, Allen Wittenauer a...@apache.org wrote: On Jun 21, 2011, at 4:39 AM, Steve Loughran wrote: If I can

Re: [DISCUSSION] Thinking about 20.204 and beyond

2011-06-21 Thread Allen Wittenauer
On Jun 21, 2011, at 12:04 PM, Ted Dunning wrote: For the pain in doing this, it is probably better to just drop $10 and bring up a nice EC2 cluster with 10 m1.large instances using spot pricing for 5 hours. Testing on non-Intel, non-Linux is something we need to do more of.

Re: [DISCUSSION] Thinking about 20.204 and beyond

2011-06-21 Thread Marcos Ortiz
For example in Solaris and BSD´s systems Right? regards On 06/21/2011 02:39 PM, Allen Wittenauer wrote: On Jun 21, 2011, at 12:04 PM, Ted Dunning wrote: For the pain in doing this, it is probably better to just drop $10 and bring up a nice EC2 cluster with 10 m1.large instances using spot

Re: [DISCUSSION] Thinking about 20.204 and beyond

2011-06-21 Thread Ted Dunning
Amazon can help with the non-linux, but not with the non-Intel. On Tue, Jun 21, 2011 at 12:09 PM, Allen Wittenauer a...@apache.org wrote: On Jun 21, 2011, at 12:04 PM, Ted Dunning wrote: For the pain in doing this, it is probably better to just drop $10 and bring up a nice EC2 cluster

Re: Large startup time in remote MapReduce job

2011-06-21 Thread Harsh J
Gabor, If your jar does not contain code changes that need to get transmitted every time, you can consider placing them on the JT/TT classpaths and not do any jar registration in your job submission code. You'll see a related WARN but it should be OK to ignore that. If not, work on other ways to

Re: Large startup time in remote MapReduce job

2011-06-21 Thread Harsh J
Allen, On Wed, Jun 22, 2011 at 2:28 AM, Allen Wittenauer a...@apache.org wrote: On Jun 21, 2011, at 1:31 PM, Harsh J wrote: Gabor, If your jar does not contain code changes that need to get transmitted every time, you can consider placing them on the JT/TT classpaths        ... which

Re: [DISCUSSION] Thinking about 20.204 and beyond

2011-06-21 Thread Scott Carey
On 6/17/11 11:50 PM, Eric Baldeschwieler eri...@yahoo-inc.com wrote: Producing a stable release of Hadoop is a long, hard and expensive process. Historically Y! has produced all such releases. Other releases of Hadoop have either not been stable (Hadoop 0.19 and Hadoop 0.21) You lost me