2008/1/28, Greg Freemyer <[EMAIL PROTECTED]>:
> On Jan 28, 2008 11:25 AM, Ciro Iriarte <[EMAIL PROTECTED]> wrote:
> > Hi, anybody has some notes about tuning md raid5, lvn and xfs?. I'm
> > getting 20mb/s with dd and I think it can be improved. I'll add config
> > parameters as soon as i get home. I'm using md raid5 on a motherboard
> > with nvidia sata controller, 4x500gb samsung sata2 disks and lvm with
> > OpenSUSE [EMAIL PROTECTED]
> >
> > Regards,
> > Ciro
> > --
>
> I have not done any raid 5 perf. testing: 20 mb/sec seems pretty bad,
> but not outrageous I suppose. I can get about 4-5GB/min from new sata
> drives. So about 75 MB/sec from a single raw drive (ie. dd
> if=/dev/zero of=/dev/sdb bs=4k)
>
> You don't say how your invoking dd. The default bs is only 512 bytes
> I think and that is totally inefficient with the linux kernel.
>
> I typically use 4k which maps to what the kernel uses. ie. dd
> if=/dev/zero of=big-file bs=4k count=1000 should give you a simple but
> meaningful test..
>
> I think the default stride is 64k per drive, so if your writing 3x 64K
> at a time, you may get perfect alignment and miss the overhead of
> having to recalculate the checksum all the time.
>
> As another data point, I would bump that up to 30x 64K and see if you
> continue to get speed improvements.
>
> So tell us the write speed for
> bs=512
> bs=4k
> bs=192k
> bs=1920k
>
> And the read speeds for the same. ie. dd if=big-file of=/dev/null bs=4k,
> etc.
>
> I would expect the write speed to go up with each increase in bs, but
> the read speed to be more or less constant. Then you need to figure
> out what sort of real world block sizes your going to be using. Once
> you have a bs, or collection of bs sizes that match your needs, then
> you can start tuning your stack.
>
> Greg
Hi, posted the first mail from my cell phone, so couldn't add more info....
- I created the raid with chunk size= 256k.
mainwks:~ # mdadm --misc --detail /dev/md2
/dev/md2:
Version : 01.00.03
Creation Time : Sun Jan 27 20:08:48 2008
Raid Level : raid5
Array Size : 1465151232 (1397.28 GiB 1500.31 GB)
Used Dev Size : 976767488 (465.76 GiB 500.10 GB)
Raid Devices : 4
Total Devices : 4
Preferred Minor : 2
Persistence : Superblock is persistent
Intent Bitmap : Internal
Update Time : Mon Jan 28 17:42:51 2008
State : active
Active Devices : 4
Working Devices : 4
Failed Devices : 0
Spare Devices : 0
Layout : left-symmetric
Chunk Size : 256K
Name : 2
UUID : 65cb16de:d89af60e:6cac47da:88828cfe
Events : 12
Number Major Minor RaidDevice State
0 8 33 0 active sync /dev/sdc1
1 8 49 1 active sync /dev/sdd1
2 8 65 2 active sync /dev/sde1
4 8 81 3 active sync /dev/sdf1
- Speed reported by hdparm:
mainwks:~ # hdparm -tT /dev/sdc
/dev/sdc:
Timing cached reads: 1754 MB in 2.00 seconds = 877.60 MB/sec
Timing buffered disk reads: 226 MB in 3.02 seconds = 74.76 MB/sec
mainwks:~ # hdparm -tT /dev/md2
/dev/md2:
Timing cached reads: 1250 MB in 2.00 seconds = 624.82 MB/sec
Timing buffered disk reads: 620 MB in 3.01 seconds = 206.09 MB/sec
- LVM:
mainwks:~ # vgdisplay data
Incorrect metadata area header checksum
--- Volume group ---
VG Name data
System ID
Format lvm2
Metadata Areas 1
Metadata Sequence No 5
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 2
Open LV 2
Max PV 0
Cur PV 1
Act PV 1
VG Size 1.36 TB
PE Size 4.00 MB
Total PE 357702
Alloc PE / Size 51200 / 200.00 GB
Free PE / Size 306502 / 1.17 TB
VG UUID KpUAeN-mPjO-2K8t-hiLX-FF0C-93R2-IP3aFI
mainwks:~ # pvdisplay /dev/sdc1
Incorrect metadata area header checksum
--- Physical volume ---
PV Name /dev/md2
VG Name data
PV Size 1.36 TB / not usable 3.75 MB
Allocatable yes
PE Size (KByte) 4096
Total PE 357702
Free PE 306502
Allocated PE 51200
PV UUID Axl2c0-RP95-WwO0-inHP-aJEF-6SYJ-Fqhnga
- XFS:
mainwks:~ # xfs_info /dev/data/test
meta-data=/dev/mapper/data-test isize=256 agcount=16, agsize=1638400 blks
= sectsz=512 attr=0
data = bsize=4096 blocks=26214400, imaxpct=25
= sunit=16 swidth=48 blks, unwritten=1
naming =version 2 bsize=4096
log =internal bsize=4096 blocks=16384, version=1
= sectsz=512 sunit=0 blks, lazy-count=0
realtime =none extsz=4096 blocks=0, rtextents=0
- The reported dd
mainwks:~ # dd if=/dev/zero bs=1024k count=100 of=/mnt/custom/t3
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 5.11596 s, 20.5 MB/s
- New dd (seems to give better result)
mainwks:~ # dd if=/dev/zero bs=1024k count=1000 of=/mnt/custom/t0
1000+0 records in
1000+0 records out
1048576000 bytes (1.0 GB) copied, 13.6218 s, 77.0 MB/s
Ciro
--
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
