I keep asking because I wasn't able to use a XFS filesystem larger than 3-4TB. If the XFS file system is larger than 4TB hdfs won't recognize the space. I am on a 64bit RHEL 5.3 host.
On Tue, May 10, 2011 at 6:30 AM, Will Maier <wcma...@hep.wisc.edu> wrote: > On Tue, May 10, 2011 at 12:03:09AM -0400, Rita wrote: > > what filesystem are they using and what is the size of each filesystem? > > It sounds nuts, but each disk has its own ext3 filesystem. Beyond switching > to > the deadline IO scheduler, we haven't done much tuning/tweaking. A script > runs > every ten minutes to test all of the data mounts and reconfigure > hdfs-site.xml > and restart the datanode if necessary. So far, this approach has allowed us > to > avoid loss of space to RAID without correlating the risk of disk failure by > building larger RAID0s. > > In the future, we expect to deprecate the script and rely on the datanode > process > itself to handle missing/failing disks. > > -- > > Will Maier - UW High Energy Physics > cel: 608.438.6162 > tel: 608.263.9692 > web: http://www.hep.wisc.edu/~wcmaier/ > -- --- Get your facts first, then you can distort them as you please.--
