TM posted on Sun, 20 Jul 2014 08:45:51 +0000 as excerpted: > One week for a raid10 rebuild 4x3TB drives is a very long time. > Any thoughts? > Can you share any statistics from your RAID10 rebuilds?
Well, 3 TB is big and spinning rust is slow. Even using the smaller power-of-10 (1000) figures for TB that the device manufacturers use instead of the power-of-2 (1024, TiB) figures that are common in computing... TB GB MB KB B KiB MiB s/hr day wk 3*1000*1000*1000*1000/1024/1024/3600/24/7=4.73+ MiB/sec At a week, that's nearly 5 MiB per second, which isn't great, but isn't entirely out of the realm of reason either, given all the processing it's doing. A day would be 33.11+, reasonable thruput for a straight copy, and a raid rebuild is rather more complex than a straight copy, so... Which is one reason a lot of people are using partitioning to break down those huge numbers into something a bit more manageable in reasonable time, these days, or switching to much faster if also much more expensive per GiB (USD 50 cents to $1 per gig vs 5-10 cents per gig) SSDs. And btrfs is still under development so hasn't been really optimized yet and is thus slower than necessary -- in particular, it often serializes multi-device processing where given that the bottleneck is normally device IO, an optimized algorithm would parallel-process all devices at once. Just parallelizing the algorithm could give it a 2-4X speed increase on a 4-device raid10. So you're right, it /is/ slow. > If I shut down the system, before the rebuild, what is the proper > procedure to remount it? Again degraded? Or normally? Can the process of > rebuilding the raid continue after a reboot? Will it survive, and > continue rebuilding? Raid10 requires four devices for undegraded layout, but I /think/ once it has a forth device added back in, you should be able to mount it undegraded, as it can write changes to four devices at that point. Tho I'm not positive about that. I'd try mounting it undegraded here and if it worked, great, if not I'd mount it degraded again. Regardless of that, however, barring bugs, provided you shut down properly (umounting the filesystem, etc), a shutdown and reboot should be fine, and it should continue where it left off after the reboot, as internally it's simply doing a rebalance of existing data to include the new device, and btrfs is designed to gracefully shutdown in the middle of a rebalance and restart it on reboot, when necessary. Tho don't expect umount and shutdown to be instantaneous. After you issue the umount command, it shouldn't start any new chunk balances, but it could require a bit to finish the balances in-flight at the time you issued the shutdown. If it takes more than a few minutes, however, there's a bug. FWIW, data chunks are a GiB in size, which at the calculated rate of a bit under 5 MiB/sec, should be ~205 seconds or roughly 3.5 minutes. Doubling that and a bit more to be safe, I'd say wait 10 minutes or so. If it hasn't properly umounted after 10 minutes, you likely have a bug and may have to recover after a reboot. With btrfs still under heavy development backups are STRONGLY recommended so I hope you have them, but at this point anyway, while it's slow going there's no indication that you'll actually need to use those backups. Just expect the umount to take a bit. -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
