Song Liu wrote:
Hi, Tonci, Actually, I am taking a Master's thesis by developing algorithms
on hadoop.
My project is to extend algorithms into mapreduce fasion and to discover
whether there is a optimal choice. Most of them belong to the Machine
Learning area. Personally, I think this is a fresh area, and if you search
the main academic database, you may find few literature about this.
I recently made an proposal about my study on Hadoop, and I would like to
discuss this with you in depth if you wish.
Another interesting topic is to discover the limit of hadoop. We have a very
large cluster at a very high rank among TOP500, so I'm wondering whether
hadoop can perform as we expected.
A lot of the big clusters have premium network infrastructure and SAN
mounted storage whose access times are independent of location.
MapReduce is designed to work on lower-cost storage/network
infrastructure, saving money there that you can spend on more servers
and storage. But it does require algorithms to work on local data only,
or the LAN becomes a bottleneck, fast.