Re: [Gluster-users] Optimal XFS formatting?
mount -t xfs -o rw,noatime,nodiratime,logbufs=8 nodiratime is redundant, noatime will do both. ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
[Gluster-users] Optimal XFS formatting?
Hi, Are there any 'optimal' settings for XFS formatting under GlusterFS? The storage will be used for Virtual Disk storage, virtual disk size from 8GB to 100 GB in size. One of the VM's (separate gluster volume) will be running MSSQL server (4K reads and writes). The other will be running file servers, etc). Thanks, Gerald ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Re: [Gluster-users] Optimal XFS formatting?
Hi, I've seen that EXT4 has better random I/O performance than XFS, especially on small reads and writes. For large sequential reads and writes XFS is a little bit better. For really large sequential reads and write EXT4 and XFS are about the same. I used to format XFS using this: mkfs.xfs -l size=64m (notes from http://everything2.com/title/Filesystem+performance+tweaking+with+XFS+on+Linux) but realized that it doesn't seem to effect performance for me. You should definitely try mounting with this : mount -t xfs -o rw,noatime,nodiratime,logbufs=8 HTH, Sabuj On Thu, Oct 20, 2011 at 8:18 AM, Gerald Brandt g...@majentis.com wrote: Hi, Are there any 'optimal' settings for XFS formatting under GlusterFS? The storage will be used for Virtual Disk storage, virtual disk size from 8GB to 100 GB in size. One of the VM's (separate gluster volume) will be running MSSQL server (4K reads and writes). The other will be running file servers, etc). Thanks, Gerald ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Re: [Gluster-users] Optimal XFS formatting?
Try using ext4 if you can. Small file read performance will be MUCH better than xfs. On the other hand, you might wanna run some benchmark tests which resemble your workload, to compare xfs vs ext4 both with and without glusterfs. On 10/20/2011 03:36 PM, Sabuj Pattanayek wrote: Hi, I've seen that EXT4 has better random I/O performance than XFS, especially on small reads and writes. For large sequential reads and writes XFS is a little bit better. For really large sequential reads and write EXT4 and XFS are about the same. I used to format XFS using this: mkfs.xfs -l size=64m (notes from http://everything2.com/title/Filesystem+performance+tweaking+with+XFS+on+Linux) but realized that it doesn't seem to effect performance for me. You should definitely try mounting with this : mount -t xfs -o rw,noatime,nodiratime,logbufs=8 HTH, Sabuj On Thu, Oct 20, 2011 at 8:18 AM, Gerald Brandt g...@majentis.com wrote: Hi, Are there any 'optimal' settings for XFS formatting under GlusterFS? The storage will be used for Virtual Disk storage, virtual disk size from 8GB to 100 GB in size. One of the VM's (separate gluster volume) will be running MSSQL server (4K reads and writes). The other will be running file servers, etc). Thanks, Gerald ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Re: [Gluster-users] Optimal XFS formatting?
Thanks for the ext4 comments. My issues with ext4 (and 3) are the long fsck times. In case of a reboot, w need to be up, and not waiting hours for 6 TB to fsck. Gerald - Original Message - From: Robert Krig rob...@bitcaster.de To: gluster-users@gluster.org Sent: Thursday, October 20, 2011 8:53:50 AM Subject: Re: [Gluster-users] Optimal XFS formatting? Try using ext4 if you can. Small file read performance will be MUCH better than xfs. On the other hand, you might wanna run some benchmark tests which resemble your workload, to compare xfs vs ext4 both with and without glusterfs. On 10/20/2011 03:36 PM, Sabuj Pattanayek wrote: Hi, I've seen that EXT4 has better random I/O performance than XFS, especially on small reads and writes. For large sequential reads and writes XFS is a little bit better. For really large sequential reads and write EXT4 and XFS are about the same. I used to format XFS using this: mkfs.xfs -l size=64m (notes from http://everything2.com/title/Filesystem+performance+tweaking+with+XFS+on+Linux) but realized that it doesn't seem to effect performance for me. You should definitely try mounting with this : mount -t xfs -o rw,noatime,nodiratime,logbufs=8 HTH, Sabuj On Thu, Oct 20, 2011 at 8:18 AM, Gerald Brandt g...@majentis.com wrote: Hi, Are there any 'optimal' settings for XFS formatting under GlusterFS? The storage will be used for Virtual Disk storage, virtual disk size from 8GB to 100 GB in size. One of the VM's (separate gluster volume) will be running MSSQL server (4K reads and writes). The other will be running file servers, etc). Thanks, Gerald ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Re: [Gluster-users] Optimal XFS formatting?
IIRC XFS also has long fsck times. I don't know of any fs's which don't, but I guess you've seen different behavior with respect to ext4 vs xfs on that issue. One thing I like about XFS is the short mkfs time, on the order of a few seconds vs minutes or hours with EXT3/4 for large fs's. On Thu, Oct 20, 2011 at 9:57 AM, Gerald Brandt g...@majentis.com wrote: Thanks for the ext4 comments. My issues with ext4 (and 3) are the long fsck times. In case of a reboot, w need to be up, and not waiting hours for 6 TB to fsck. Gerald - Original Message - From: Robert Krig rob...@bitcaster.de To: gluster-users@gluster.org Sent: Thursday, October 20, 2011 8:53:50 AM Subject: Re: [Gluster-users] Optimal XFS formatting? Try using ext4 if you can. Small file read performance will be MUCH better than xfs. On the other hand, you might wanna run some benchmark tests which resemble your workload, to compare xfs vs ext4 both with and without glusterfs. On 10/20/2011 03:36 PM, Sabuj Pattanayek wrote: Hi, I've seen that EXT4 has better random I/O performance than XFS, especially on small reads and writes. For large sequential reads and writes XFS is a little bit better. For really large sequential reads and write EXT4 and XFS are about the same. I used to format XFS using this: mkfs.xfs -l size=64m (notes from http://everything2.com/title/Filesystem+performance+tweaking+with+XFS+on+Linux) but realized that it doesn't seem to effect performance for me. You should definitely try mounting with this : mount -t xfs -o rw,noatime,nodiratime,logbufs=8 HTH, Sabuj On Thu, Oct 20, 2011 at 8:18 AM, Gerald Brandt g...@majentis.com wrote: Hi, Are there any 'optimal' settings for XFS formatting under GlusterFS? The storage will be used for Virtual Disk storage, virtual disk size from 8GB to 100 GB in size. One of the VM's (separate gluster volume) will be running MSSQL server (4K reads and writes). The other will be running file servers, etc). Thanks, Gerald ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Re: [Gluster-users] Optimal XFS formatting?
The issues with both random-access performance and fsck times vary a lot according to *exactly* which version of each you're using. I'm in the same Yup, our tests recently were done directly to XFS using bonnie, iozone, fio, and tiobench on centos6, which is not using the most bleeding edge version of XFS by default. We're still using gluster 3.0.x so many of those benchmarking programs blow up just trying to start. ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Re: [Gluster-users] Optimal XFS formatting?
On 10/20/2011 03:18 PM, Gerald Brandt wrote:
Hi,
Are there any 'optimal' settings for XFS formatting under GlusterFS? The
storage will be used for Virtual Disk storage, virtual disk size from 8GB to
100 GB in size.
One of the VM's (separate gluster volume) will be running MSSQL server (4K
reads and writes). The other will be running file servers, etc).
hi,
Why do you use VM images for these IO demanding tasks?
Anyway, we use this script:
#!/bin/bash
BLOCKSIZE=4096 # Make sure this is in bytes
CHUNKSIZE=128 # Make sure this is in KiB
NUMSPINDLES=8
RAID_TYPE=6
RAID_DEVICE_NAME=/dev/sda4 # Specify device name for your RAID device
FSLABEL=gluster # specify filesystem label for generating mkfs line here
case $RAID_TYPE in
0)
RAID_DISKS=${NUMSPINDLES};
;;
1)
RAID_DISKS=${NUMSPINDLES};
;;
10)
RAID_DISKS=${NUMSPINDLES};
;;
5)
RAID_DISKS=`echo ${NUMSPINDLES} - 1 | bc`;
;;
6)
RAID_DISKS=`echo ${NUMSPINDLES} - 2 | bc`;
;;
*)
echo Please specify RAID_TYPE as one of: 0, 1, 10, 5, or 6.
exit
;;
esac
SUNIT=`echo ${CHUNKSIZE} * 1024 / 512 | bc`
SWIDTH=`echo $RAID_DISKS * ${SUNIT} | bc`
echo System blocksize=${BLOCKSIZE}
echo Chunk Size=${CHUNKSIZE} KiB
echo NumSpindles=${NUMSPINDLES}
echo RAID Type=${RAID_TYPE}
echo RAID Disks (usable for data)=${RAID_DISKS}
echo Calculated values:
echo Stripe Unit=${SUNIT}
echo -e Stripe Width=${SWIDTH}\n
echo mkfs line:
echo -e mkfs.xfs -b size=${BLOCKSIZE} -d
sunit=${SUNIT},swidth=${SWIDTH} -L ${FSLABEL} ${RAID_DEVICE_NAME}\n
echo mount line:
echo -e mount -o remount,sunit=${SUNIT},swidth=${SWIDTH}\n
echo Add these options to your /etc/fstab to make permanent:
echo sunit=${SUNIT},swidth=${SWIDTH}
Corresponding fstab:
/dev/sda4/mnt/brick1xfs
delaylog,sunit=256,swidth=1536,logbufs=8,logbsize=256k,noatime,nodiratime
00
tamas
___
Gluster-users mailing list
Gluster-users@gluster.org
http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Re: [Gluster-users] Optimal XFS formatting?
Basic thumb rule RAID 5 , 64k stripe size, use the following. If you have larger files go further. # xfs, 5 disks, 64K stripe, units in 512-byte sectors mkfs -txfs -d sunit=$((64*2)) -d swidth=$((5*64*2)) Use for better memory alignment with Scheduler echo 16 /proc/sys/vm/page-cluster - and also don't forget to change echo 256 /sys/block/disk/queue/nr_requests - which should be twice the queue_depth. -Harsha ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
