Tomasz, > On Sun, Jul 20, 2014 at 01:53:34PM +0000, Duncan wrote: >> TM posted on Sun, 20 Jul 2014 08:45:51 +0000 as excerpted: >> >>> One week for a raid10 rebuild 4x3TB drives is a very long time. >>> Any thoughts? >>> Can you share any statistics from your RAID10 rebuilds? >> >> At a week, that's nearly 5 MiB per second, which isn't great, but isn't >> entirely out of the realm of reason either, given all the processing >> it's >> doing. A day would be 33.11+, reasonable thruput for a straight copy, >> and a raid rebuild is rather more complex than a straight copy, so... > > Uhm, sorry, but 5MBps is _entirely_ unreasonable. It is > order-of-magnitude > unreasonable. And "all the processing" shouldn't even show as a blip > on modern CPUs. > This "speed" is undefendable.
Maybe and maybe not. It's already been discussed that BTRFS serializes its disk I/O, instead of performing it in parallel. That can have a significant impact on overall throughput. In addition, depending on how BTRFS does a reconstruct, it's possible that there's another problem. Most low-level RAID implementations (MD driver and RAID controller cards) reconstruct a segment at a time (or possibly multiple segments), and reconstruct the entire disk, even the unused portions. ZFS is different, in that it reconstructs only the used sections. This behavior will read/write less to the disk, but it will have more random I/O. The random I/O can result in more seeks (depending on how busy the system is), which could slow down the reconstruction, especially on an idle system. As ZFS was designed for high usage environments, where it's likely that a seek would be needed to get the head back to the data being reconstructed, the randomness of the reconstruct would have no significant impact on a reconstruct. The question implied above is which method of reconstruction BTRFS uses. Could someone please inform us? As BTRFS is verifying the checksum on the read, and possibly generating a new checksum on write, the CPU time used will be significantly greater than for a "dumb" RAID-10 array. If it's not a sequential segment reconstruction, the CPU time needed would be greater still. WRT CPU performance, we should remember that TM's system is far from a speed demon. Finally, TM didn't mention anything about other I/O activity on the array, which, regardless of the method of reconstruction, could have a significant impact on the speed of a reconstruction. There are a LOT of parameters here that could impact throughput. Some are designed in (the checksum computations), some are "temporary" (the single I/O path) and some are end-user issues (slow CPU and other activity on the array). I'm sure that there are other parameters, possibly including soft read errors on the other disks, that could be impacting the overall throughput. As all the information that could affect performance hasnt been provided yet, it is premature to make a blanket statement that the performance of a reconstruction is "unreasonable". For the circumstances, it's possible that the performance is just fine. We have not yet been provided with enough data to verify this, one way or another. Peter Ashford -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
