> On Jul 29, 2026, at 5:28 PM, Andy Smith <[email protected]> wrote:
>
>
> On Wed, Jul 29, 2026 at 04:38:55PM -0600, Casey Deccio wrote:
>> - /home is mounted on /dev/md125, which is a RAID 5 consisting of three
>> disks, formatted with ext4
>
> I did not see anywhere in your email the details of what hardware these
> disks are.
Thanks. Correct, I omitted that. These are all:
Seagate BarraCuda 5TB Internal Hard Drive HDD – 2.5 Inch SATA 6Gb/s 5400 RPM
128MB
>
>> I'm not really sure what else I might try, and I'm reaching out to the
>> community to see if anyone has any suggestions for identifying and fixing
>> the problem.
>
> I would start by a process of simplification. For example, does the sync
> still take a long time when /home is not actually mounted? If so then
> the issue is with your NVMe.
Thanks for the great idea. I just tested your proposed setup. I unmounted
/home and started the file transfer to /root (on the NVMe). While the transfer
was happening, I ran `time sync` again. The times were much, much lower than
what I was seeing when /home was mounted on /dev/md125.
> If it does occur when /home is mounted, then it is the disks in the RAID
> setup or the RAID setup itself. Does each individual disk perform
> anomalously?
>
> "fio" is a good tool,to carry out performance testing of storage
> devices. Have a look what the random 4k read and write I/O is like in
> terms of IOPS, and the sequential disk transfer rate in MB/s for reads
> or writes. You can try that on the array and then break the array and
> try it on each individual disk. Obviously this is destructive to the
> data.
>
> If the performance is as expected on a per-disk basis but terrible only
> with RAID, it may be time to post to the linux-raid mailing list as this
> is unlikely to be a Debian-specific issue. I would then also be checking
> if it happens in other RAID configurations like RAID-1.
>
> 150 IOPS of random reads (not coming from RAM) is reasonable for a
> single 7.2kRPM SATA disk. You could expect 2-3x this from a RAID-5 of
> three identical disks.
>
> Write IOPS is a bit more difficult to guess and will depend on whether
> you have disabled the write cache in the drives themselves. Due to the
> RAID-5 write penalty (read-modify-write) you may expect only 50-70% of a
> single disk's IOPS for random writes.
Thanks for that very helpful information. I've run fio on the still-assembled
array, and here's what I get for read/write:
read: IOPS=237
write: IOPS=79
I'm not sure exactly what to make of that...
> Of course, if your sd[abc] are SSDs then there should be plenty of IOPS
> available and those sync times are very weird.
Right. As mentioned above, these are not SSDs.
>
> "iostat -x 5" can give some more hints about what is going on.
>
> If there is other I/O happening that you aren't expecting then "iotop"
> can help identify it.
Things seem to be idle when the file transfer is not happening. When it is,
iotop shows about 100% utilization for /dev/md125 and about 50 - 75%
utilization of /dev/sd{a,b,c}. I haven't done comparisons with anything else
at this point to see how this compares. Thanks in the mean time for the idea.
Casey
