Matthew Roberts wrote: > I have been experimenting with a softraid mirror, using two cheap SATA > disks. > In general the performance is very good - except when rebuilding. A quick > set of sums suggests that the problem is seek time. > > The disks are 7200rpm, therefore one can hope for 120 seeks per second. > "systat iostat" (while rebuilding) gives this (edited to fit in an email): > > DEVICE READ WRITE RTPS WTPS SEC > wd0 3801088 3827302 58 59 1.0 > wd1 0 3801088 0 58 0.0 > sd0 0 0 0 0 0.0 > Totals 3801088 7628390 58 117 1.0 > > When not rebuilding, the disks can stream data at ~100MB/s. Whereas while > rebuilding the speed is closer to 3.8MB/s - which seems to be cause by the > 64KB softraid rebuild blocksize and the seek time of the disk. > (3.8MB/s = 64KB/block * 60block/s). > > Interestingly, even though I have a mirror, the rebuild process is writing > to both disks during the rebuild. > > --- > > So... given that my raid will take ~5days to rebuild when I expect a few > hours I have started to thing about how to 'fix' this problem. > > My instinct says that there are two ways to attack this problem. > > 1) I could modify /src/sys/dev/softraid.c so that it consumes much more > kernel memory and rebuilds in blocks of (say) 1MB, rather than 64k. Which > would work for anyone else having rebuild performance problems on any raid > level. > > 2) I could, instead, modify /src/sys/dev/softraid_raid1.c so that it > doesn't > write back to source chunk when rebuilding. Obviously I want 'normal' > IO to > be written to all disks when rebuilding, so I'll need to be careful. > > --- > > My question is: has anyone else come across this issue before, and if so > what > did they do? Google fails me... so please don't say "google it".
I've come across this. I yanked a disk to test reconstruction. When I realized it would take days (2T raid 1 over three disks), I gave up on it and instead tweaked my backup script to support multiple stores. while that in the end was a better (mostly as in "simpler") solution for me, I'd still love to see the issue you describe being solved. > My second question is: which of the two ideas above has most merit? - > and what should I do with the patch afterwards (assuming that it isn't > nasty)? First, let me say I'm delighted to read your conclusions on the subject. (1) Does not feel like a proper solution. dd(1) et al usually works dandy with a 64k buffer, so it should be sufficient here. (2) Sounds like it would fix the problem (no extra seeking needed if one disk reads a constant stream while the other disk writes a constant stream. Again, just like dd(1). However, I have no idea how complex it would be to actually implement. That said, I'm not a device nor a filesystem guy, so you need to accept the fact that I might be totally misleading you here. :-) /Alexander
