Hello,
I am new to raid and running disk benchmarks. I am not getting the
performance from my raid I was expecting. This machine is going to be
running apache/php/mysql in a very high load environment and I want to
make sure things are running well before I put it into production.
Any input would be greatly appreciated!!!
Thanks!
Jacob Martinson
Here is what I did at first (naive I am sure):
(/home is on the system disk and /raid is the raid-5)
% root@sybil:/home
05:24 $ time dd if=/dev/zero of=100M bs=512k count=200
200+0 records in
200+0 records out
real 0m9.626s
user 0m0.000s
sys 0m0.590s
% root@sybil:/home
05:24 $ time dd if=100M of=/dev/null
204800+0 records in
204800+0 records out
real 0m11.541s
user 0m0.190s
sys 0m0.750s
% root@sybil:/raid
05:34 $ time dd if=/dev/zero of=100M bs=512k count=200
200+0 records in
200+0 records out
real 0m4.974s
user 0m0.000s
sys 0m0.690s
% root@sybil:/raid
05:35 $ time dd if=100M of=/dev/null
204800+0 records in
204800+0 records out
real 0m14.365s
user 0m0.180s
sys 0m1.010s
bonnie -d /raid -s 500
-------Sequential Output-------- ---Sequential Input-- --Random--
-Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks---
Machine MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU /sec %CPU
500 5385 91.0 19089 12.6 5484 8.3 3784 65.5 7570 4.5 231.7 1.9
I have also noticed that raid5d takes between 3& and 20% of the cpu
(which surprised me with a modern processor)
Quick summary of my setup:
Athlon/Duron 600 cpu
Gigabyte GA-7ZX
256MB PC133 ram
2 UW scsi disks on a buslogic/flashpoint controller (system and log
disk)
4 LVD 9GB scsi disks on a 2940U2W (raid-5 with one hot spare)
Kernel is 2.2.16 with the patch:
http://people.redhat.com/mingo/raid-patches/raid-2.2.16-A0
Here's my /etc/raidtab:
raiddev /dev/md0
raid-level 5
nr-raid-disks 3
nr-spare-disks 1
persistent-superblock 1
parity-algorithm left-symmetric
chunk-size 128
device /dev/sdd1
raid-disk 0
device /dev/sde1
raid-disk 1
device /dev/sdf1
raid-disk 2
device /dev/sdg1
spare-disk 0
Here's my /proc/mdstat:
Personalities : [linear] [raid0] [raid1] [raid5] [translucent]
read_ahead 1024 sectors
md0 : active raid5 sdg1[3] sdf1[2] sde1[1] sdd1[0] 17863936 blocks
level 5, 128k chunk, algorithm 2 [3/3] [UUU] resync=31% finish=87.5min
unused devices: <none>
I did mke2fs like this:
mke2fs -b 4096 -R stride=32 /dev/md0
df output:
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 137M 20M 109M 16% /
/dev/sda6 2.8G 26M 2.6G 1% /var
/dev/sda7 2.8G 606M 2.0G 23% /usr
/dev/sda8 1.1G 200M 870M 19% /home
/dev/sda9 258M 14k 245M 0% /tmp
/dev/sdb1 17G 20k 16G 0% /var/log/httpd
/dev/md0 17G 20k 16G 0% /raid
I get thousands of these lines in dmesg:
set_blocksize: b_count 1, dev md(9,0), block 4553720, from 8013e430
set_blocksize: b_count 1, dev md(9,0), block 4553721, from 8013e430
set_blocksize: b_count 1, dev md(9,0), block 4553722, from 8013e430
set_blocksize: b_count 1, dev md(9,0), block 4553723, from 8013e430
set_blocksize: b_count 1, dev md(9,0), block 4553724, from 8013e430
set_blocksize: b_count 1, dev md(9,0), block 4553725, from 8013e430
set_blocksize: b_count 1, dev md(9,0), block 4553726, from 8013e430
set_blocksize: b_count 2, dev md(9,0), block 4553727, from 8013e430
md0: blocksize changed during read
nr_blocks changed to 256 (blocksize 4096, j 1138176, max_blocks
4465984)
Here's my initial system dmesg:
Linux version 2.2.16-RAID (root@sybil) (gcc version 2.95.2 20000220
(Debian GNU/Linux)) #1 Tue S
ep 5 13:57:35 CDT 2000
Detected 600042 kHz processor.
Console: colour VGA+ 80x25
Calibrating delay loop... 1196.03 BogoMIPS
Memory: 63812k/66556k available (1328k kernel code, 416k reserved,
912k data, 88k init)
Dentry hash table entries: 16384 (order 5, 128k)
Buffer cache hash table entries: 65536 (order 6, 256k)
Page cache hash table entries: 16384 (order 4, 64k)
CPU: L1 I Cache: 64K L1 D Cache: 64K
CPU: L2 Cache: 1K
CPU: AMD AMD Athlon(tm) Processor stepping 00
Checking 386/387 coupling... OK, FPU using exception 16 error
reporting.
Checking 'hlt' instruction... OK.
POSIX conformance testing by UNIFIX
PCI: PCI BIOS revision 2.10 entry at 0xfdb71
PCI: Using configuration type 1
PCI: Probing PCI hardware
Linux NET4.0 for Linux 2.2
Based upon Swansea University Computer Society NET3.039
NET4: Unix domain sockets 1.0 for Linux NET4.0.
NET4: Linux TCP/IP 1.0 for NET4.0
IP Protocols: ICMP, UDP, TCP
TCP: Hash tables configured (ehash 65536 bhash 65536)
Starting kswapd v 1.5
Detected PS/2 Mouse Port.
Serial driver version 4.27 with no serial options enabled
ttyS00 at 0x03f8 (irq = 4) is a 16550A
ttyS01 at 0x02f8 (irq = 3) is a 16550A
pty: 256 Unix98 ptys configured
VP_IDE: IDE controller on PCI bus 00 dev 39
VP_IDE: not 100% native mode: will probe irqs later
ide0: BM-DMA at 0xffa0-0xffa7, BIOS settings: hda:DMA, hdb:pio
ide1: BM-DMA at 0xffa8-0xffaf, BIOS settings: hdc:pio, hdd:pio
hda: ATAPI CDROM, ATAPI CDROM drive
ide0 at 0x1f0-0x1f7,0x3f6 on irq 14
hda: ATAPI 24X CD-ROM drive, 120kB Cache
Uniform CD-ROM driver Revision: 3.09
Floppy drive(s): fd0 is 1.44M
FDC 0 is a post-1991 82077
md driver 0.90.0 MAX_MD_DEVS=256, MAX_REAL=12
translucent personality registered
linear personality registered
raid0 personality registered
raid1 personality registered
raid5 personality registered
raid5: measuring checksumming speed
raid5: MMX detected, trying high-speed MMX checksum routines
pII_mmx : 1392.936 MB/sec
p5_mmx : 1748.409 MB/sec
8regs : 805.434 MB/sec
32regs : 651.510 MB/sec
using fastest function: p5_mmx (1748.409 MB/sec)
scsi: ***** BusLogic SCSI Driver Version 2.1.15 of 17 August 1998
*****
scsi: Copyright 1995-1998 by Leonard N. Zubkoff <[EMAIL PROTECTED]>
scsi0: Configuring BusLogic Model BT-958 PCI Wide Ultra SCSI Host
Adapter
scsi0: Firmware Version: 5.07B, I/O Address: 0xD000, IRQ Channel:
12/Level
scsi0: PCI Bus: 0, Device: 9, Address: 0xDFFFF000, Host Adapter SCSI
ID: 7
scsi0: Parity Checking: Enabled, Extended Translation: Enabled
scsi0: Synchronous Negotiation: Fast, Wide Negotiation: Enabled
scsi0: Disconnect/Reconnect: Enabled, Tagged Queuing: Enabled
scsi0: Scatter/Gather Limit: 128 of 8192 segments, Mailboxes: 211
scsi0: Driver Queue Depth: 211, Host Adapter Queue Depth: 192
scsi0: Tagged Queue Depth: Automatic, Untagged Queue Depth: 3
scsi0: Error Recovery Strategy: Default, SCSI Bus Reset: Enabled
scsi0: SCSI Bus Termination: Both Enabled, SCAM: Disabled
scsi0: *** BusLogic BT-958 Initialized Successfully ***
(scsi1) <Adaptec AHA-294X Ultra2 SCSI host adapter> found at PCI
0/15/0
(scsi1) Wide Channel, SCSI ID=7, 32/255 SCBs
(scsi1) Downloading sequencer code... 396 instructions downloaded
scsi0 : BusLogic BT-958
scsi1 : Adaptec AHA274x/284x/294x (EISA/VLB/PCI-Fast SCSI)
5.1.28/3.2.4
<Adaptec AHA-294X Ultra2 SCSI host adapter>
scsi : 2 hosts.
Vendor: QUANTUM Model: VIKING II 9.1WLS Rev: 5520
Type: Direct-Access ANSI SCSI revision: 02
Detected scsi disk sda at scsi0, channel 0, id 0, lun 0
Vendor: WDIGTL Model: WDE18300 ULTRA2 Rev: 1.30
Type: Direct-Access ANSI SCSI revision: 02
Detected scsi disk sdb at scsi0, channel 0, id 1, lun 0
Vendor: IOMEGA Model: ZIP 100 Rev: J.03
Type: Direct-Access ANSI SCSI revision: 02
Detected scsi removable disk sdc at scsi0, channel 0, id 6, lun 0
scsi0: Target 0: Queue Depth 28, Wide Synchronous at 20.0 MB/sec,
offset 15
scsi0: Target 1: Queue Depth 28, Wide Synchronous at 20.0 MB/sec,
offset 15
scsi0: Target 6: Queue Depth 3, Asynchronous
(scsi1:0:0:0) Synchronous at 80.0 Mbyte/sec, offset 31.
Vendor: WDIGTL Model: WDE9180 ULTRA2 Rev: 1.30
Type: Direct-Access ANSI SCSI revision: 02
Detected scsi disk sdd at scsi1, channel 0, id 0, lun 0
(scsi1:0:1:0) Synchronous at 80.0 Mbyte/sec, offset 31.
Vendor: WDIGTL Model: WDE9180 ULTRA2 Rev: 1.30
Type: Direct-Access ANSI SCSI revision: 02
Detected scsi disk sde at scsi1, channel 0, id 1, lun 0
(scsi1:0:2:0) Synchronous at 80.0 Mbyte/sec, offset 31.
Vendor: SEAGATE Model: ST39236LC Rev: 0004
Type: Direct-Access ANSI SCSI revision: 03
Detected scsi disk sdf at scsi1, channel 0, id 2, lun 0
(scsi1:0:3:0) Synchronous at 80.0 Mbyte/sec, offset 31.
Vendor: WDIGTL Model: WDE9180 ULTRA2 Rev: 1.30
Type: Direct-Access ANSI SCSI revision: 02
Detected scsi disk sdg at scsi1, channel 0, id 3, lun 0
scsi : detected 7 SCSI generics 7 SCSI disks total.
SCSI device sda: hdwr sector= 512 bytes. Sectors= 17836668 [8709 MB]
[8.7 GB]
SCSI device sdb: hdwr sector= 512 bytes. Sectors= 35761710 [17461 MB]
[17.5 GB]
SCSI device sdc: hdwr sector= 512 bytes. Sectors= 196608 [96 MB] [0.1
GB]
sdc: Write Protect is off
SCSI device sdd: hdwr sector= 512 bytes. Sectors= 17873040 [8727 MB]
[8.7 GB]
SCSI device sde: hdwr sector= 512 bytes. Sectors= 17873040 [8727 MB]
[8.7 GB]
SCSI device sdf: hdwr sector= 512 bytes. Sectors= 17942584 [8761 MB]
[8.8 GB]
SCSI device sdg: hdwr sector= 512 bytes. Sectors= 17873040 [8727 MB]
[8.7 GB]
Partition check:
sda: sda1 sda2 < sda5 sda6 sda7 sda8 sda9 >
sdb: sdb1
sdc: sdc4
sdd: sdd1
sde: sde1
sdf: sdf1
sdg: sdg1
md.c: sizeof(mdp_super_t) = 4096
autodetecting RAID arrays
autorun ...
... autorun DONE.
VFS: Mounted root (ext2 filesystem) readonly.
Freeing unused kernel memory: 88k freed
scsi0: Tagged Queuing now active for Target 0
Adding Swap: 1461872k swap-space (priority -1)
tulip.c:v0.91g-ppc 7/16/99 [EMAIL PROTECTED]
eth0: Lite-On 82c168 PNIC rev 32 at 0xb800, 00:A0:CC:D1:6A:FA, IRQ 12.
eth0: MII transceiver #1 config 3000 status 7829 advertising 01e1.
scsi0: Tagged Queuing now active for Target 1
Here's my kernel config:
CONFIG_EXPERIMENTAL=y
CONFIG_M686=y
CONFIG_X86_WP_WORKS_OK=y
CONFIG_X86_INVLPG=y
CONFIG_X86_BSWAP=y
CONFIG_X86_POPAD_OK=y
CONFIG_X86_TSC=y
CONFIG_X86_GOOD_APIC=y
CONFIG_2GB=y
CONFIG_MODULES=y
CONFIG_MODVERSIONS=y
CONFIG_KMOD=y
CONFIG_NET=y
CONFIG_PCI=y
CONFIG_PCI_GOANY=y
CONFIG_PCI_BIOS=y
CONFIG_PCI_DIRECT=y
CONFIG_PCI_QUIRKS=y
CONFIG_PCI_OLD_PROC=y
CONFIG_SYSVIPC=y
CONFIG_SYSCTL=y
CONFIG_BINFMT_AOUT=y
CONFIG_BINFMT_ELF=y
CONFIG_BINFMT_MISC=y
CONFIG_BLK_DEV_FD=y
CONFIG_BLK_DEV_IDE=y
CONFIG_BLK_DEV_IDEDISK=y
CONFIG_BLK_DEV_IDECD=y
CONFIG_BLK_DEV_CMD640=y
CONFIG_BLK_DEV_RZ1000=y
CONFIG_BLK_DEV_IDEPCI=y
CONFIG_BLK_DEV_IDEDMA=y
CONFIG_IDEDMA_AUTO=y
CONFIG_BLK_DEV_LOOP=m
CONFIG_BLK_DEV_MD=y
CONFIG_AUTODETECT_RAID=y
CONFIG_MD_LINEAR=y
CONFIG_MD_STRIPED=y
CONFIG_MD_MIRRORING=y
CONFIG_MD_RAID5=y
CONFIG_MD_TRANSLUCENT=y
CONFIG_MD_BOOT=y
CONFIG_BLK_DEV_RAM=m
CONFIG_BLK_DEV_RAM_SIZE=4096
CONFIG_PARIDE_PARPORT=y
CONFIG_PACKET=y
CONFIG_FIREWALL=y
CONFIG_UNIX=y
CONFIG_INET=y
CONFIG_IP_FIREWALL=y
CONFIG_IP_ALIAS=y
CONFIG_SYN_COOKIES=y
CONFIG_SKB_LARGE=y
CONFIG_SCSI=y
CONFIG_BLK_DEV_SD=y
CONFIG_CHR_DEV_ST=y
CONFIG_CHR_DEV_SG=y
CONFIG_SCSI_MULTI_LUN=y
CONFIG_SCSI_CONSTANTS=y
CONFIG_SCSI_AHA152X=y
CONFIG_SCSI_AHA1542=y
CONFIG_SCSI_AHA1740=y
CONFIG_SCSI_AIC7XXX=y
CONFIG_AIC7XXX_CMDS_PER_DEVICE=8
CONFIG_AIC7XXX_RESET_DELAY=5
CONFIG_SCSI_ADVANSYS=y
CONFIG_SCSI_BUSLOGIC=y
CONFIG_SCSI_SYM53C8XX=y
CONFIG_SCSI_NCR53C8XX_DEFAULT_TAGS=4
CONFIG_SCSI_NCR53C8XX_MAX_TAGS=32
CONFIG_SCSI_NCR53C8XX_SYNC=20
CONFIG_NETDEVICES=y
CONFIG_DUMMY=m
CONFIG_NET_ETHERNET=y
CONFIG_NET_VENDOR_3COM=y
CONFIG_EL1=m
CONFIG_EL2=m
CONFIG_ELPLUS=m
CONFIG_EL16=m
CONFIG_EL3=m
CONFIG_3C515=m
CONFIG_VORTEX=m
CONFIG_RTL8139=m
CONFIG_NET_EISA=y
CONFIG_DEC_ELCP=m
CONFIG_EEXPRESS_PRO100=m
CONFIG_VT=y
CONFIG_VT_CONSOLE=y
CONFIG_SERIAL=y
CONFIG_UNIX98_PTYS=y
CONFIG_UNIX98_PTY_COUNT=256
CONFIG_MOUSE=y
CONFIG_PSMOUSE=y
CONFIG_82C710_MOUSE=y
CONFIG_AUTOFS_FS=y
CONFIG_ISO9660_FS=y
CONFIG_PROC_FS=y
CONFIG_DEVPTS_FS=y
CONFIG_EXT2_FS=y
CONFIG_NFS_FS=y
CONFIG_SUNRPC=y
CONFIG_LOCKD=y
CONFIG_VGA_CONSOLE=y
-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to [EMAIL PROTECTED]
