Gregory,
On Thu, 27 Jul 2000, Gregory Leblanc wrote:
> > 6) Use tiotest, NOT bonnie! Try multiple threads (I use 1, 2,
> > 4, 8, 16,
> > 32, 64, 128, 256 threads - this is perhaps excessive!)
>
> What size datasets are you using?
I use 1G if I'm feeling like making absolutely sure it's fair, or else
something like 256M if I'm trying to get it done quickly.
> Bonnie++ is still a good benchmark, although it stresses things
> differently.
I haven't used bonnie++ actually...
> The maximum number of threads that you should need to
> (or probably even want to) run is between 2x and 3x the number of
> disks that you have installed. That should ensure that every drive is
> pulling 1 piece of data, and that there is another thread that is
> waiting for data while that one is being retrieved.
I believe in seeing how the performance breaks down under extreme
stress. With a threaded database like mysql (one of the primary uses for
our RAID arrays) you could quite easily have numerous threads all trying
to read and write from the array simultaneously.
When I was comparing performance of RAID0+1 to RAID5 there was a big
difference in how quickly (as per number of threads) they ground to a
halt. Here's an example:
./tiobench.pl --size 256 --dir /mnt/md3/ --block 4096 --threads 1
--threads 2 --threads 4 --threads 16 --threads 32 --threads 64
--threads 128 --threads 256
Linux Kernel 2.2.14, RAID 0+1
Dir Size BlkSz Thr# Read (CPU%) Write (CPU%) Seeks (CPU%)
----- ------ ------- ---- ------------- -------------- --------------
/mnt/ 256 4096 1 46.3288 25.6% 40.3105 47.2% 165.171 0.66%
/mnt/ 256 4096 2 35.3465 21.9% 39.5187 45.9% 193.171 0.67%
/mnt/ 256 4096 4 29.1810 18.0% 38.7580 45.0% 214.686 0.89%
/mnt/ 256 4096 16 26.9373 17.3% 36.5620 42.2% 220.682 0.93%
/mnt/ 256 4096 32 21.4527 24.1% 34.7506 40.0% 216.958 0.97%
/mnt/ 256 4096 64 12.7891 47.4% 31.7158 36.1% 202.744 1.05%
/mnt/ 256 4096 128 8.65209 80.6% 27.8459 31.2% 200.230 3.27%
/mnt/ 256 4096 256 5.41081 131.% 24.6386 27.3% 193.811 16.1%
Linux Kernel 2.2.14 with Mika's read-balance patch, RAID 0+1
Dir Size BlkSz Thr# Read (CPU%) Write (CPU%) Seeks (CPU%)
----- ------ ------- ---- ------------- -------------- --------------
/mnt/ 256 4096 1 46.6853 24.6% 38.2826 44.2% 176.209 0.39%
/mnt/ 256 4096 2 59.6558 40.3% 38.7603 43.6% 221.300 0.69%
/mnt/ 256 4096 4 60.6616 43.6% 38.2311 42.9% 263.113 0.89%
/mnt/ 256 4096 16 51.5140 37.6% 37.1443 42.1% 302.154 1.05%
/mnt/ 256 4096 32 47.0307 34.9% 35.1884 40.1% 329.017 1.33%
/mnt/ 256 4096 64 42.1452 33.2% 33.0139 37.3% 341.591 1.41%
/mnt/ 256 4096 128 27.4339 36.0% 30.8700 34.3% 332.434 1.53%
/mnt/ 256 4096 256 15.5834 76.4% 28.2604 31.1% 321.990 13.2%
Linux Kernel 2.2.14 with Mika's read-balance patch, RAID 5
Dir Size BlkSz Thr# Read (CPU%) Write (CPU%) Seeks (CPU%)
----- ------ ------- ---- ------------- -------------- --------------
/mnt/ 256 4096 1 67.5911 38.8% 24.3309 34.9% 167.331 0.41%
/mnt/ 256 4096 2 60.4156 49.0% 24.5966 37.1% 208.991 0.67%
/mnt/ 256 4096 4 46.5667 38.1% 24.4007 37.2% 247.676 0.90%
/mnt/ 256 4096 16 27.7189 32.6% 24.3155 37.5% 282.041 1.12%
/mnt/ 256 4096 32 14.4717 45.2% 23.9831 36.8% 301.291 1.32%
/mnt/ 256 4096 64 8.39616 82.4% 22.5777 34.1% 299.902 1.67%
/mnt/ 256 4096 128 6.77856 103.% 20.8036 30.6% 276.423 16.7%
/mnt/ 256 4096 256 6.14939 115.% 19.0964 27.6% 266.183 35.5%
This shows the quite interesting result that (for reads) RAID-5 starts
off with 1 thread out-performing RAID-0+1 (68 vs 47), drops to the same
level with 2 threads (60 vs 60), and rapidly decreases thereafter, eg
at 64 threads it's 8 vs 42.
Of course, because of that slight hiccup, RAID-0+1 arrays will fail
(recoverably, but still bring the machine down) with one faulty
disk. So we had to go with RAID-5 anyway...
> Heh, I'm using it because it provides redundancy, the added speed from
> Mika's RAID 1 read balancing patch is just a perk... HTH,
Yeah, maybe I was being slightly unrealistic. But the performance is
still mighty nice...
Regards,
Corin
/------------------------+-------------------------------------\
| Corin Hartland-Swann | Direct: +44 (0) 20 7544 4676 |
| Commerce Internet Ltd | Mobile: +44 (0) 79 5854 0027 |
| 22 Cavendish Buildings | Tel: +44 (0) 20 7491 2000 |
| Gilbert Street | Fax: +44 (0) 20 7491 2010 |
| Mayfair | Web: http://www.commerce.uk.net/ |
| London W1K 5HJ | E-Mail: [EMAIL PROTECTED] |
\------------------------+-------------------------------------/
