On Tue, Apr 5, 2016 at 4:37 AM, Duncan <1i5t5.dun...@cox.net> wrote: > Gareth Pye posted on Tue, 05 Apr 2016 09:36:48 +1000 as excerpted: > >> I've got a btrfs file system set up on 6 drbd disks running on 2Tb >> spinning disks. The server is moderately loaded with various regular >> tasks that use a fair bit of disk IO, but I've scheduled my weekly btrfs >> scrub for the best quiet time in the week. >> >> The command that is run is: >> /usr/local/bin/btrfs scrub start -Bd -c idle /data >> >> Which is my best attempt to try and get it to have a low impact on user >> operations >> >> But iotop shows me: >> >> 1765 be/4 root 14.84 M/s 0.00 B/s 0.00 % 96.65 % btrfs scrub >> start -Bd -c idle /data >> 1767 be/4 root 14.70 M/s 0.00 B/s 0.00 % 95.35 % btrfs >> scrub start -Bd -c idle /data >> 1768 be/4 root 13.47 M/s 0.00 B/s 0.00 % 92.59 % btrfs >> scrub start -Bd -c idle /data >> 1764 be/4 root 12.61 M/s 0.00 B/s 0.00 % 88.77 % btrfs >> scrub start -Bd -c idle /data >> 1766 be/4 root 11.24 M/s 0.00 B/s 0.00 % 85.18 % btrfs >> scrub start -Bd -c idle /data >> 1763 be/4 root 7.79 M/s 0.00 B/s 0.00 % 63.30 % btrfs >> scrub start -Bd -c idle /data >> 28858 be/4 root 0.00 B/s 810.50 B/s 0.00 % 61.32 % [kworker/ > u16:25] >> >> >> Which doesn't look like an idle priority to me. And the system sure >> feels like a system with a lot of heavy io going on. Is there something >> I'm doing wrong?
When I see the throughput numbers, it lets me think that the filesystem is raid5 or raid6. On single or raid1 or raid10 one easily gets around 100M/s without the notice/feeling of heavy IO ongoing, mostly independent of scrub options. > Two points: > > 1) It appears btrfs scrub start's -c option only takes numeric class, so > try -c3 instead of -c idle. Thanks to Duncan for pointing this out. I don't remember exactly, but I think I also had issues with this in the past, but did not realize or have a further look at it. > Works for me with the numeric class (same results as you with spelled out > class), tho I'm on ssd with multiple independent btrfs on partitions, the > biggest of which is 24 GiB, 18.something GiB used, which scrubs in all of > 20 seconds, so I don't need and hadn't tried the -c option at all until > now. > > 2) What a difference an ssd makes! > > $$ sudo btrfs scrub start -c3 /p > scrub started on /p, [...] > > $$ sudo iotop -obn1 > Total DISK READ : 626.53 M/s | Total DISK WRITE : 0.00 B/s > Actual DISK READ: 596.93 M/s | Actual DISK WRITE: 0.00 B/s > TID PRIO USER DISK READ DISK WRITE SWAPIN IO COMMAND > 872 idle root 268.40 M/s 0.00 B/s 0.00 % 0.00 % btrfs scrub > start -c3 /p > 873 idle root 358.13 M/s 0.00 B/s 0.00 % 0.00 % btrfs scrub > start -c3 /p > > CPU bound, 0% IOWait even at idle IO priority, in addition to the > hundreds of M/s values per thread/device, here. You OTOH are showing > under 20 M/s per thread/device on spinning rust, with an IOWait near 90%, > thus making it IO bound. This low M/s and high IOWait is the kind of behavior I noticed with 3x 2TB raid5 when scrubbing or balancing (no bcache activated, kernel 4.3.3). -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
