On 9/23/23 08:04, Dale wrote:
Howdy,
As most everyone knows, I redone my NAS box. Before I had Truenas on it
but switched to Ubuntu server thingy called Jimmy. Kinda like the
name. lol Anyway, Ubuntu has the same odd transfer pattern as the
Truenas box had. I'm not sure if the problem is on the Gentoo end or
the Ubuntu end or something else. I'm attaching a picture of Gkrellm so
you can see what I'm talking about. It transfers a bit, then seems to
stop for some reason, then start up again and this repeats over and
over. I'm expecting more of a consistent throughput instead of all the
idle time. The final throughput is only around 29.32MB/s according to
info from rsync. If it was not stopping all the time and passing data
through all the time, I think that would improve. Might even double.
A little more info. The set of drives this is being copied from use LVM
and are encrypted with dm-crypt. They are also set up the same way on
the NAS box. I also notice that on the NAS box, using htop, the CPUs
sit at idle for a bit then show heavy use, on Ubuntu about 60 or 70%,
then go back to idle. This seems to be the same thing I'm seeing on
htop with the data throughput. One may have something to do with the
other but I don't know what. I got so much stuff running on my main rig
that I can't really tell if that CPU has the same changes or not. By
the way, it showed the same when Truenas was on there. These things are
mounted using nfs. I don't know if that matters or not. In case this
is a routing issue, I have a Netgear router with 1GB ports. This is the
first part of the mount command:
mount -t nfs -o nolock
Has anyone ever seen something like this and know why it is idle for so
much of the time? Anyone know if this can be fixed so that it is more
consistent, and hopefully faster?
If you need more info, let me know. If you know the command, that might
help too. Just in case it is a command I'm not familiar with.
I can't add to what others have suggested to improve the throughput, but
the gkrellm pic you posted tells me something is handling the data in
batches. enp3s0 (your network interface) gets a peak of activity then
stops while crypt (the disk) has a peak of activity. Rinse and repeat.
I don't know if this is caused by the program invoked by the command you
issued or by some interaction of different pieces that get called to do
the work. My guess is that it is reading until it fills some buffer,
then writes it out. (Note, it doesn't matter which device is reading and
which is writing, the two just don't overlap) If encryption is
involved, it might be that there is actually a encrypt/decrypt which
takes place in between the disk and network flows. I don't know of any
way to change this, but it might explain why the network transfer rate
is as fast as it can get, but the overall throughput is lower.
