On Jun 10, 2010, at 2:37 AM, Vikas Ashok Patil wrote: > In my cluster I have the lustre parallel file system which mainly exposes > storage over a network. Also there is some local space on each node of the > cluster. This space is not part of the lustre file system. My hadoop > installation currently makes use of this local file system. However, I would > like to make use of the space available through lustre(exposed over the > network). Hence I was thinking of a way to integrate HDFS and lustre, where > HDFS would manage the local storage and lustre would provide the storage > over the network. > > Please let me know your thoughts on this.
Your local storage should get used for MR. Use Lustre via file:// (LocalFileSystem, iirc) instead of HDFS via hdfs:// (DistributedFileSystem, irrc) as the default file system type.
