Re: [PHP-DB] big table / hadoop / map reduce

2010-10-30 Thread Artur Ejsmont
sure that was a bit more helpful, thanks :) i was still wondering to what other use cases would that apply. This is a good article (best so far i guess): http://code.google.com/edu/parallel/mapreduce-tutorial.html The thing is that reduce has to aggregate data or it would be impractical. So i am

Re: [PHP-DB] big table / hadoop / map reduce

2010-10-30 Thread Andrés G . Montañez
Hi Artur, in your IPs examples, lets supouse you have ten access log files (from ten different servers), there you already have the mapping part done. Then you reduce each log into anonther new file, indicating the IP address and the times it's repeated. At this stage you have a reduced version

Re: [PHP-DB] big table / hadoop / map reduce

2010-10-30 Thread Artur Ejsmont
yeah i think that would make sense. if you find more good examples from different areas let me know ... i think i get the basic idea ... will try to apply it some time :) cheers :) art On 30 October 2010 18:58, Andrés G. Montañez andresmonta...@gmail.com wrote: Hi Artur, in your IPs

[PHP-DB] big table / hadoop / map reduce

2010-10-22 Thread Artur Ejsmont
Hi there guys and girls Have anyone came across any reasonable explanation / articles on how hadoop and map reduce work in practice? i have read a few articles now and then and i must say i am puzzled am i stupid or they just cant find an easy way to explain it? :P What i would hope for is

Re: [PHP-DB] big table / hadoop / map reduce

2010-10-22 Thread Andrés G . Montañez
Hi Artur, Here is an article on wikipedia: http://en.wikipedia.org/wiki/MapReduce And here are the native implementations in php: http://www.php.net/manual/en/function.array-map.php http://www.php.net/manual/en/function.array-reduce.php The basic idea is to gather a lot of data, from several

Re: [PHP-DB] big table / hadoop / map reduce

2010-10-22 Thread Artur Ejsmont
hehe sorry but this does not help :-) i can google for wikipedia definitions. I was hoping for some really good articles/examples that would put it into enough context. I would like to have good idea when it could be useful. So far had no luck with that. Its like with design patterns ...

Re: [PHP-DB] big table / hadoop / map reduce

2010-10-22 Thread Andrés G . Montañez
Imagine you have to get track of some kind of traffic, for example, ad impressions; lets supose that you have millions of those hits; you will have to have a few servers to receive the notifications of the impression of an ad. After the end of the day, you will have that info across a bunch of