On 10/18/2012 02:21 AM, Pamecha, Abhishek wrote:
Tom
Do you mean you are using GPFS instead of HDFS? Also, if you can share,
are you deploying it as DAS set up or a SAN?
Thanks,
Abhishek
Though I don't think I'd buy a SAN for a new Hadoop cluster, we have a
SAN and are using it *instead of HDFS* with a small/medium Hadoop
MapReduce cluster (up to 100 nodes or so, depending on our need). We
still use the local node disks for intermediate data (mapred local
storage). Although this set-up does limit our possibility to scale to a
large number of nodes, that's not a concern for us. On the plus, we
gain the flexibility to be able to share our cluster with non-Hadoop
users at our centre.
--
Luca Pireddu
CRS4 - Distributed Computing Group
Loc. Pixina Manna Edificio 1
09010 Pula (CA), Italy
Tel: +39 0709250452
