Re: using multiple disks for HDFS

2010-02-09 Thread Todd Lipcon
Hi Vasilis, Two things: 1) You're missing a matching } in your hadoop.tmp.dir setting 2) When you use ${hadoop.tmp.dir}/dfs/data, it does a literal string interpolation. Thus, it's not adding dfs/data to each of the hadoop.tmp.dir directories, but rather just the last one. I'd recommend setting

Re: using multiple disks for HDFS

2010-02-09 Thread Allen Wittenauer
On 2/9/10 8:49 AM, "Vasilis Liaskovitis" wrote: > > dfs.data.dir > ${hadoop.tmp.dir}/dfs/data > > > > hadoop.tmp.dir > > /local/user2/hdfs/hadoop-${user.name},/local2/user2/hdfs/hadoop-${user. > name},/local3/user2/hdfs/hadoop-${user.name,/local4/user2/hdfs/hadoop-${user.n > a

using multiple disks for HDFS

2010-02-09 Thread Vasilis Liaskovitis
Hi, I am trying to use 4 SATA disks per node in my hadoop cluster. This is a JBOD configuration, no RAID is involved. There is one single xfs partition per disk, each one mounted as /local/, /local2/, /local3, /local4 - with sufficient privileges for running hadoop jobs. HDFS is setup across the 4