Re: degraded RAID performance after OS upgrade, and drives added.

2006-11-30 Thread Chuck Swiger

On Nov 30, 2006, at 11:40 AM, Derrick MacPherson wrote:
That seems like a pretty crazy drop in performance, more than one  
would expect. The machine is busy but not busy enough to warrant  
this.. Imo.. Is there a way to test to confirm?


Using dd is a trivial benchmark, and not especially precise but good  
enough to give rough answers; otherwise, there are lots of I/O  
benchmarks in ports like iozone or even "diskinfo -t".  Try using  
different block sizes while reading and writing with dd, and you'll  
probably find some useful info.


While it is likely that you can adjust the RAID-5 stripe size, change  
the write-caching from write-thru to write-back (if your RAID has a  
battery, anyway, otherwise this is dangerous), etc, using a 3-disk  
RAID-5 volume is just not a great idea-- RAID-5 is happier with more  
drives than the minimum of 3 to get better parallelism and reduce the  
overhead for parity info.


--
-Chuck

___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "[EMAIL PROTECTED]"


RE: degraded RAID performance after OS upgrade, and drives added.

2006-11-30 Thread Derrick MacPherson
That seems like a pretty crazy drop in performance, more than one would expect. 
The machine is busy but not busy enough to warrant this.. Imo.. Is there a way 
to test to confirm?

-Original Message-
From: "Chuck Swiger" <[EMAIL PROTECTED]>
To: "Derrick MacPherson" <[EMAIL PROTECTED]>
Cc: freebsd-questions@freebsd.org
Sent: 11/30/06 10:39
Subject: Re: degraded RAID performance after OS upgrade, and drives added.

On Nov 29, 2006, at 6:18 PM, Derrick MacPherson wrote:
> We updated to 6.1 this weekend and added 3 300gb drives to the  
> external raid cabinet, they were to go on a seprate controller but  
> the server happens to have a few other boxes on top making it  
> impossible at that time, so we put the 3x300  (RAID5) , upgraded  
> the OS and  performance is very poor. When I run systat I see  
> upward of 300 tps on the problematic array (da2) and under systat - 
> vmstat :

It's normal for RAID-5 to perform worse than a single drive-- and  
sometimes it performs much worse, as in nearly an order of magnitude  
slower, for the case of very small writes.  If you value performance,  
choose another RAID level like RAID-0, RAID-1, or RAID-10.

-- 
-Chuck


___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "[EMAIL PROTECTED]"


Re: degraded RAID performance after OS upgrade, and drives added.

2006-11-30 Thread Chuck Swiger

On Nov 29, 2006, at 6:18 PM, Derrick MacPherson wrote:
We updated to 6.1 this weekend and added 3 300gb drives to the  
external raid cabinet, they were to go on a seprate controller but  
the server happens to have a few other boxes on top making it  
impossible at that time, so we put the 3x300  (RAID5) , upgraded  
the OS and  performance is very poor. When I run systat I see  
upward of 300 tps on the problematic array (da2) and under systat - 
vmstat :


It's normal for RAID-5 to perform worse than a single drive-- and  
sometimes it performs much worse, as in nearly an order of magnitude  
slower, for the case of very small writes.  If you value performance,  
choose another RAID level like RAID-0, RAID-1, or RAID-10.


--
-Chuck

___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "[EMAIL PROTECTED]"


degraded RAID performance after OS upgrade, and drives added.

2006-11-29 Thread Derrick MacPherson
We updated to 6.1 this weekend and added 3 300gb drives to the  
external raid cabinet, they were to go on a seprate controller but  
the server happens to have a few other boxes on top making it  
impossible at that time, so we put the 3x300  (RAID5) , upgraded the  
OS and  performance is very poor. When I run systat I see upward of  
300 tps on the problematic array (da2) and under systat -vmstat :


Disks   da0   da1   da2 pass0 pass1 pass2
KB/t 0.00  0.00 15.00  0.00  0.00  0.00
tps   0  0  341 0 0 0
MB/s   0.00  0.00  4.99  0.00  0.00  0.00
% busy0 095 0 0 0

the busy percentage is consistenly over 90% sometimes over 100%. I'm  
not sure where to go next with providing info for someone to look at  
or testing to run..


camcontrol devlist
 at scbus0 target 0 lun 0 (da0,pass0)
 at scbus0 target 1 lun 0 (da1,pass1)
 at scbus0 target 2 lun 0 (da2,pass2)

out of dmesg:

ciss0:  port 0x3000-0x30ff mem  
0xf05c-0xf05f,0xf04f-0xf04f3fff irq 30 at device 3.0 on

pci1
ciss0: [GIANT-LOCKED]


da0 at ciss0 bus 0 target 0 lun 0
da0:  Fixed Direct Access SCSI-0 device
da0: 135.168MB/s transfers
da0: 140006MB (286734240 512 byte sectors: 255H 32S/T 35139C)
da1 at ciss0 bus 0 target 1 lun 0
da1:  Fixed Direct Access SCSI-0 device
da1: 135.168MB/s transfers
da1: 140006MB (286734240 512 byte sectors: 255H 32S/T 35139C)
da2 at ciss0 bus 0 target 2 lun 0
da2:  Fixed Direct Access SCSI-0 device
da2: 135.168MB/s transfers
da2: 572195MB (1171856412 512 byte sectors: 255H 63S/T 65535C)



dmesg
Copyright (c) 1992-2006 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights  
reserved.

FreeBSD 6.1-RELEASE-p10 #0: Sat Nov 25 15:52:17 PST 2006
[EMAIL PROTECTED]:/usr/src/sys/i386/compile/SMP
Timecounter "i8254" frequency 1193182 Hz quality 0
CPU: Intel(R) Xeon(TM) CPU 2.80GHz (2786.87-MHz 686-class CPU)
  Origin = "GenuineIntel"  Id = 0xf25  Stepping = 5
   
Features=0xbfebfbff,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>

  Features2=0x4400>
  Logical CPUs per core: 2
real memory  = 2147459072 (2047 MB)
avail memory = 2096439296 (1999 MB)
ACPI APIC Table: 
FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
cpu0 (BSP): APIC ID:  0
cpu1 (AP): APIC ID:  1
cpu2 (AP): APIC ID:  6
cpu3 (AP): APIC ID:  7
MADT: Forcing active-low polarity and level trigger for SCI
ioapic0  irqs 0-15 on motherboard
ioapic1  irqs 16-31 on motherboard
ioapic2  irqs 32-47 on motherboard
ioapic3  irqs 48-63 on motherboard
kbd1 at kbdmux0
acpi0:  on motherboard
acpi0: Power Button (fixed)
Timecounter "ACPI-safe" frequency 3579545 Hz quality 1000
acpi_timer0: <32-bit timer at 3.579545MHz> port 0x920-0x923 on acpi0
cpu0:  on acpi0
cpu1:  on acpi0
cpu2:  on acpi0
cpu3:  on acpi0
pcib0:  on acpi0
pci0:  on pcib0
pci0:  at device 3.0 (no driver attached)
pci0:  at device 4.0 (no driver attached)
pci0:  at device 4.2 (no driver attached)
isab0:  at device 15.0 on pci0
isa0:  on isab0
atapci0:  port  
0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x2000-0x200f at device 15.1 on pci0

ata0:  on atapci0
ata1:  on atapci0
ohci0:  mem 0xeeef-0xeeef0fff irq  
7 at device 15.2 on pci0

ohci0: [GIANT-LOCKED]
usb0: OHCI version 1.0, legacy support
usb0: SMM does not respond, resetting
usb0:  on ohci0
usb0: USB revision 1.0
uhub0: (0x1166) OHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub0: 4 ports with 4 removable, self powered
pcib1:  on acpi0
pci1:  on pcib1
ciss0:  port 0x3000-0x30ff mem  
0xf05c-0xf05f,0xf04f-0xf04f3fff irq 30 at device 3.0 on pci1

ciss0: [GIANT-LOCKED]
pcib2:  on acpi0
pci2:  on pcib2
bge0:  mem  
0xf06f-0xf06f irq 29 at device 1.0 on pci2

miibus0:  on bge0
brgphy0:  on miibus0
brgphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseTX,  
1000baseTX-FDX, auto

bge0: Ethernet address: 00:11:0a:9e:2f:49
bge1:  mem  
0xf06e-0xf06e irq 31 at device 2.0 on pci2

miibus1:  on bge1
brgphy1:  on miibus1
brgphy1:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseTX,  
1000baseTX-FDX, auto

bge1: Ethernet address: 00:11:0a:9e:2f:48
pcib3:  on acpi0
pci3:  on pcib3
pcib4:  on acpi0
pci6:  on pcib4
pci6:  at device 30.0 (no  
driver attached)

acpi_tz0:  on acpi0
atkbdc0:  port 0x60,0x64 irq 1 on acpi0
atkbd0:  irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
psm0:  irq 12 on atkbdc0
psm0: [GIANT-LOCKED]
psm0: model IntelliMouse, device ID 3
sio0:  port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
sio0: type 16550A
fdc0:  port 0x3f2-0x3f5 irq 6 drq 2 on  
acpi0

fdc0: [FAST]
fd0: <1440-KB 3.5" drive> on fdc0 drive 0
pmtimer0 on isa0
orm0:  at iomem 0xc-0xc7fff,0xc8000-0xcbfff, 
0xcc000-0xcd7ff,0xee000-0xe on isa0

ppc0: parallel port not found.
sc0:  at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
sio1: configured irq 3 not in bitmap of probed irqs 0
sio1: port may not be