Re: questions about hadoop map reduce and compute intensive related applications

2011-04-30 Thread elton sky
Ted, MPI supports node-to-node communications in ways that map-reduce does not, > however, which requires that you iterate map-reduce steps for many > algorithms. With Hadoop's current implementation, this is horrendously > slow (minimum 20-30 seconds per iteration). > > Sometimes you can avoid

Re: Serving Media Streaming

2011-04-30 Thread Luiz Fernando Figueiredo
Great link. Thanks a lot. Em 30/04/11 17:31, Ted Dunning escreveu: Check out S4 http://s4.io/ On Fri, Apr 29, 2011 at 10:13 PM, Luiz Fernando Figueiredo< luiz.figueir...@auctorita.com.br> wrote: Hi guys. Hadoop is well known to process large amounts of data but we think that we can do much

Re: Applications creates bigger output than input?

2011-04-30 Thread Ted Dunning
Cooccurrence analysis is commonly used in recommendations. These produce large intermediates. Come on over to the Mahout project if you would like to talk to a bunch of people who work on these problems. On Fri, Apr 29, 2011 at 9:31 PM, elton sky wrote: > Thank you for suggestions: > > Weblog

Re: Serving Media Streaming

2011-04-30 Thread Ted Dunning
Check out S4 http://s4.io/ On Fri, Apr 29, 2011 at 10:13 PM, Luiz Fernando Figueiredo < luiz.figueir...@auctorita.com.br> wrote: > Hi guys. > > Hadoop is well known to process large amounts of data but we think that we > can do much more than it. Our goal is try to serve pseudo-streaming near of

Re: questions about hadoop map reduce and compute intensive related applications

2011-04-30 Thread Ted Dunning
On Sat, Apr 30, 2011 at 12:18 AM, elton sky wrote: > I got 2 questions: > > 1. I am wondering how hadoop MR performs when it runs compute intensive > applications, e.g. Monte carlo method compute PI. There's a example in > 0.21, > QuasiMonteCarlo, but that example doesn't use random number and it

Re: ant version problem

2011-04-30 Thread Ravi
Daniel McEnnis writes: > > Dear Hadoop, > > Which version of ant do I need to keep the hadoop build from failing. > Netbeans ant works as well as eclipse ant works. However, ant 1.8.2 > does not, nor does the > default ant from Ubuntu 10.10. Snippet from failure to follow: > > BUILD FAILED >

questions about hadoop map reduce and compute intensive related applications

2011-04-30 Thread elton sky
I got 2 questions: 1. I am wondering how hadoop MR performs when it runs compute intensive applications, e.g. Monte carlo method compute PI. There's a example in 0.21, QuasiMonteCarlo, but that example doesn't use random number and it generates psudo input upfront. If we use distributed random num