On Mon, Aug 29, 2005 at 06:20:56PM +0000, Holger Kiehl wrote: > Hello > > I have a system with the following setup: > > Board is Tyan S4882 with AMD 8131 Chipset > 4 Opterons 848 (2.2GHz) > 8 GB DDR400 Ram (2GB for each CPU) > 1 onboard Symbios Logic 53c1030 dual channel U320 controller > 2 SATA disks put together as a SW Raid1 for system, swap and spares > 8 SCSI U320 (15000 rpm) disks where 4 disks (sdc, sdd, sde, sdf) > are on one channel and the other four (sdg, sdh, sdi, sdj) on > the other channel. > > The U320 SCSI controller has a 64 bit PCI-X bus for itself, there is > no other device on that bus. Unfortunatly I was unable to determine at > what speed it is running, here the output from lspci -vv:
> How does one determine the PCI-X bus speed? Usually only the card (in your case the Symbios SCSI controller) can tell. If it does, it'll be most likely in 'dmesg'. > Anyway, I thought with this system I would get theoretically 640 MB/s using > both channels. You can never use the full theoretical bandwidth of the channel for data. A lot of overhead remains for other signalling. Similarly for PCI. > I tested several software raid setups to get the best possible write > speeds for this system. But testing shows that the absolute maximum I > can reach with software raid is only approx. 270 MB/s for writting. > Which is very disappointing. I'd expect somewhat better (in the 300-400 MB/s range), but this is not too bad. To find where the bottleneck is, I'd suggest trying without the filesystem at all, and just filling a large part of the block device using the 'dd' command. Also, trying without the RAID, and just running 4 (and 8) concurrent dd's to the separate drives could show whether it's the RAID that's slowing things down. > The tests where done with 2.6.12.5 kernel from kernel.org, scheduler > is the deadline and distribution is fedora core 4 x86_64 with all > updates. Chunksize is always the default from mdadm (64k). Filesystem > was always created with the command mke2fs -j -b4096 -O dir_index > /dev/mdx. > > I also have tried with 2.6.13-rc7, but here the speed was much lower, > the maximum there was approx. 140 MB/s for writting. Now that's very low. -- Vojtech Pavlik SuSE Labs, SuSE CR - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/