Well, I know I'm not getting the performance I want out of the Mylex
DAC1164P. I was only getting 26MB/s on write throughput with 2 RAID 5
chains (5+p). I'm certain that either the card is not optimized, the
driver is not optimized, or both. One of the things that we have
found here at LLNL is that there MUST be synchronization between the
firmware on the controller, the driver, the host OS VFS layer, and the
file system.
There seems to be a "sweet spot" in RAID 5 systems where the file
system block size and the stripe size are the same size (or the block
size is larger than and a multiple of the stripe size). Not only does
this need to happen, but the firmware on the controller needs to be
aware of some sort of "transaction" or block transfer. This will
allow for the driver to send only full blocks to the drives in a
complete stripe. This will eliminate all read-modify-write sequences
when writing to the drives.
As for the VFS layer, I'm not sure what would be needed in Linux to
help out in finding the "sweet spot" of the RAID system. I'll try to
see if I can get more info on what needed to be done to AIX and GFS
(IBM's Global File System) to make this happen. I'm also going to try
to get in touch with some engineers at Mylex to work with me on this.
I'm going start putting together a high-performance RAID how-to based
on our experiences with this sort of thing at LLNL. There are quite a
few people here that have a great deal of knowledge in this area.
Another thing that I can't seem to convince people of is that for
high-performance computing, larger block sizes are better. This will
increase latency but it will also increase I/O throughput. If your
application is primarily writes (as is the case with most simulations
here at the lab), then the greater throughput is better. Again, I'll
see if I can get some numbers on this. I think we have all sorts of
graphs and benchmark results for Blue Pacific (the big IBM cluster),
this information could be valuable for optimizing Linux.
BAPper
On Wed, Mar 08, 2000 at 04:05:17AM -0000, Matthew Clark wrote:
> Hey guys.. I just installed and ran iozone.. neat tool..
>
> However.... I am very concerned about the "write" performance of my
> system...
>
> When the file size reaches 32Mb, I see a huge drop from around 129Mb/sec
> (obviously caching effects) right down to 10Mb/sec... then at 64Mb it drops
> to between 2.5 and 6.7 Mb/sec depending on record/block size...
>
> I have a Dual Intel PIII 500 system with 256Mb of main Memory... It has a
> Hardware RAID 5 system on 5 18 Gb Seagate Barracuda drives spread over 3 LVD
> SCSI channels on a Megaraid controller. I have the latest megaraid source
> (1.05) from ami.com.
>
> Any ideas how I can get more performance out of this machine? What is the
> best configuration when creating the filesystem? I used mke2fs -b 4096 -i
> 8192 ...
>
> Any help would be most appreciated!!
>
> Regards,
>
> Matthew Clark.
