Re: System becomes unusable when copying a large file via USB into HDD

2020-04-02 Thread Sreyan Chakravarty
Thank you. On Thu, Apr 2, 2020 at 12:44 AM stan via users < users@lists.fedoraproject.org> wrote: > On Wed, 01 Apr 2020 06:02:07 - > "Sreyan Chakravarty" wrote: > > > Can you please explain what the bfq driver is ? Are you using a HDD > > and not a SSD. > > Budget Fair Queueing (BFQ)

Re: System becomes unusable when copying a large file via USB into HDD

2020-04-02 Thread Sreyan Chakravarty
It worked without any tweaks on my side too, once I updated Fedora 31 to the latest kernel that is 5.5, I was using 5.3. On Wed, Apr 1, 2020 at 6:27 PM Ed Greshko wrote: > On 2020-04-01 20:50, Roger Heflin wrote: > > the "cache" is in memory. My original reason for setting it was I did > >

Re: System becomes unusable when copying a large file via USB into HDD

2020-04-01 Thread stan via users
On Wed, 01 Apr 2020 06:02:07 - "Sreyan Chakravarty" wrote: > Can you please explain what the bfq driver is ? Are you using a HDD > and not a SSD. Budget Fair Queueing (BFQ) Storage-I/O Scheduler kernel documentation https://www.kernel.org/doc/html/latest/block/bfq-iosched.html some other

Re: System becomes unusable when copying a large file via USB into HDD

2020-04-01 Thread Ed Greshko
On 2020-04-01 20:50, Roger Heflin wrote: > the "cache" is in memory. My original reason for setting it was I did > usbstick and usb-sd cards, and both of those are really slow, and i > also did not have a lot of ram in those laptops such that an > additional 20% of memory going for writecache

Re: System becomes unusable when copying a large file via USB into HDD

2020-04-01 Thread Roger Heflin
I doubt this is going to be fixed upstream.This feature was put into the kernel sometime prior to 2004, and the first version was set even higher than 20%. And this becomes less of an issue if you have enough ram that you can give up the 20% of ram. On Wed, Apr 1, 2020 at 3:56 AM Sreyan

Re: System becomes unusable when copying a large file via USB into HDD

2020-04-01 Thread Roger Heflin
the "cache" is in memory. My original reason for setting it was I did usbstick and usb-sd cards, and both of those are really slow, and i also did not have a lot of ram in those laptops such that an additional 20% of memory going for writecache also made the system page horribly. Ed: How much

Re: System becomes unusable when copying a large file via USB into HDD

2020-04-01 Thread Sreyan Chakravarty
I have some good news. The workaround : vm.dirty_background_bytes = 300 vm.dirty_bytes = 500 worked wonders for me. I can now copy and watch YouTube at the same time. Thank you and hoped this is fixed in upstream soon. On Wed, Apr 1, 2020 at 1:30 PM Ed Greshko wrote: > On

Re: System becomes unusable when copying a large file via USB into HDD

2020-04-01 Thread Ed Greshko
On 2020-04-01 15:49, Sreyan Chakravarty wrote: > No, you are missing the point. The kernel needs to be on HDD, or rather the > cache used by the kernel needs to be on the HDD. I don't think it is I missing the point.  I already informed you of my system's configuration.  So, I found it odd

Re: System becomes unusable when copying a large file via USB into HDD

2020-04-01 Thread Sreyan Chakravarty
No, you are missing the point. The kernel needs to be on HDD, or rather the cache used by the kernel needs to be on the HDD. Anyways, can you tell me if Fedora 31 uses the BFQ scheduler by default now ? Because thats what I am reading here:

Re: System becomes unusable when copying a large file via USB into HDD

2020-04-01 Thread Ed Greshko
On 2020-04-01 13:58, Sreyan Chakravarty wrote: > Try : > USB -> HDD. > Well, as I said, the HDD is "an old laptop HDD in a USB 2.0 enclosure". So, I would be doing USB-Flash--->USB HDD. Is that what you would be interested in? -- The key to getting good answers is to ask good questions.

Re: System becomes unusable when copying a large file via USB into HDD

2020-04-01 Thread Sreyan Chakravarty
> Mine aren't missing a 0, I divided wrong and added a 0, so I am using > 5MB and 3MB, which would leave my spread at 2MB or at say 100MB/sec > 1/50sec. It has been a long time since I set them and my system works > decent during copying. Hold on let me try this.

Re: System becomes unusable when copying a large file via USB into HDD

2020-04-01 Thread Sreyan Chakravarty
> On Tue, 31 Mar 2020 19:23:53 -0400 > Tom Horsley > > A few versions ago the default disk drivers shifted to multi queue > versions from single queue. I don't know that this had any thing to do > with it, but it is a change in disk handling. > > I don't copy large files, so I haven't seen

Re: System becomes unusable when copying a large file via USB into HDD

2020-04-01 Thread Sreyan Chakravarty
Try : USB -> HDD. Please dont use an SSD. You are not testing correctly. On Wed, Apr 1, 2020 at 8:56 AM Ed Greshko wrote: > On 2020-04-01 09:14, Roger Heflin wrote: > > To have the issue it helps to have the reading disk be a quite a bit > > faster than the receiving disk that way the reading

Re: System becomes unusable when copying a large file via USB into HDD

2020-03-31 Thread Ed Greshko
On 2020-04-01 09:14, Roger Heflin wrote: > To have the issue it helps to have the reading disk be a quite a bit > faster than the receiving disk that way the reading disk can easily > get ahead and fill up the write cache faster than the reading disk can > process it (it backs up into the

Re: System becomes unusable when copying a large file via USB into HDD

2020-03-31 Thread Roger Heflin
To have the issue it helps to have the reading disk be a quite a bit faster than the receiving disk that way the reading disk can easily get ahead and fill up the write cache faster than the reading disk can process it (it backs up into the writecache). So USB3.0 3.5-7200rpm or an SSD will easily

Re: System becomes unusable when copying a large file via USB into HDD

2020-03-31 Thread Ed Greshko
On 2020-04-01 08:46, Samuel Sieb wrote: > On 3/31/20 5:42 PM, Ed Greshko wrote: >> On 2020-04-01 07:09, Sreyan Chakravarty wrote: >>> I have just installed the latest version of Fedora Gnome on my HP laptop. I >>> had to import a 50 GB OVA file from my USB HDD to my hard drive via USB 3.0. >> >>

Re: System becomes unusable when copying a large file via USB into HDD

2020-03-31 Thread Samuel Sieb
On 3/31/20 5:42 PM, Ed Greshko wrote: On 2020-04-01 07:09, Sreyan Chakravarty wrote: I have just installed the latest version of Fedora Gnome on my HP laptop. I had to import a 50 GB OVA file from my USB HDD to my hard drive via USB 3.0. I don't often copy large files to/from USB.  But

Re: System becomes unusable when copying a large file via USB into HDD

2020-03-31 Thread Ed Greshko
On 2020-04-01 07:09, Sreyan Chakravarty wrote: > I have just installed the latest version of Fedora Gnome on my HP laptop. I > had to import a 50 GB OVA file from my USB HDD to my hard drive via USB 3.0. I don't often copy large files to/from USB.  But figured I give it a go.  I created a file

Re: System becomes unusable when copying a large file via USB into HDD

2020-03-31 Thread stan via users
On Tue, 31 Mar 2020 19:23:53 -0400 Tom Horsley wrote: > On Wed, 1 Apr 2020 04:39:13 +0530 > Sreyan Chakravarty wrote: > > > Why does this happen ? > > I don't know why, but I've noticed it as well. It wasn't always this > way, but a few fedora versions ago it changed. A few versions ago the

Re: System becomes unusable when copying a large file via USB into HDD

2020-03-31 Thread Roger Heflin
I have never seen nice have any useful IO throttling effect (unless you are using 100% of your cpu with non-niced processes that is), so for exactly the reason you say, it takes very little cpu to do a massive amount of IO, so nice won't work. There are some kernel knobs that will control IO but

Re: System becomes unusable when copying a large file via USB into HDD

2020-03-31 Thread Roger Heflin
Mine aren't missing a 0, I divided wrong and added a 0, so I am using 5MB and 3MB, which would leave my spread at 2MB or at say 100MB/sec 1/50sec. It has been a long time since I set them and my system works decent during copying. On Tue, Mar 31, 2020 at 7:22 PM Samuel Sieb wrote: > > On

Re: System becomes unusable when copying a large file via USB into HDD

2020-03-31 Thread Samuel Sieb
On 3/31/20 5:01 PM, Mauricio Tavares wrote: This may not solve the problem but if you need it to work right now, what would happen if your reniced it? That doesn't help. There's no lack of CPU time and I don't think process priority has any effect on IO priority.

Re: System becomes unusable when copying a large file via USB into HDD

2020-03-31 Thread Samuel Sieb
On 3/31/20 5:00 PM, Roger Heflin wrote: These are what I set: vm.dirty_background_bytes = 300 vm.dirty_bytes = 500 I think you're missing a zero. If those are your actual numbers, you have a very small buffer. Even with your numbers that seems a little small. I have noticed that

Re: System becomes unusable when copying a large file via USB into HDD

2020-03-31 Thread Mauricio Tavares
This may not solve the problem but if you need it to work right now, what would happen if your reniced it? On Tue, Mar 31, 2020 at 7:40 PM Tom Horsley wrote: > > On Wed, 1 Apr 2020 04:55:23 +0530 > Sreyan Chakravarty wrote: > > > Marvelous. > > > > So now what do we do ? > > Well, the

Re: System becomes unusable when copying a large file via USB into HDD

2020-03-31 Thread Roger Heflin
These are what I set: vm.dirty_background_bytes = 300 vm.dirty_bytes = 500 That limits the to-be-written bytes to 50Mb, and when it hits 50MB it will clear the write cache down to 30MB and let writes happen again. Since 20MB of writes happens pretty fast on modern HD's this makes

Re: System becomes unusable when copying a large file via USB into HDD

2020-03-31 Thread Tom Horsley
On Wed, 1 Apr 2020 04:55:23 +0530 Sreyan Chakravarty wrote: > Marvelous. > > So now what do we do ? Well, the ubuntu bug had suggested sysctl settings, but I've mostly ignored it since I don't have to copy big files that often, so I don't know how well the ubuntu suggestions work.

Re: System becomes unusable when copying a large file via USB into HDD

2020-03-31 Thread Sreyan Chakravarty
Marvelous. So now what do we do ? On Wed, Apr 1, 2020 at 4:54 AM Tom Horsley wrote: > On Wed, 1 Apr 2020 04:39:13 +0530 > Sreyan Chakravarty wrote: > > > Why does this happen ? > > I don't know why, but I've noticed it as well. It wasn't always this > way, but a few fedora versions ago it

Re: System becomes unusable when copying a large file via USB into HDD

2020-03-31 Thread Tom Horsley
On Wed, 1 Apr 2020 04:39:13 +0530 Sreyan Chakravarty wrote: > Why does this happen ? I don't know why, but I've noticed it as well. It wasn't always this way, but a few fedora versions ago it changed. I always figured some kernel developer got a bee in his bonnet about some disk copying

System becomes unusable when copying a large file via USB into HDD

2020-03-31 Thread Sreyan Chakravarty
I have just installed the latest version of Fedora Gnome on my HP laptop. I had to import a 50 GB OVA file from my USB HDD to my hard drive via USB 3.0. When the copying was taking place through VirtualBox, the system hung almost completely, I was not able to even move my mouse. Why does this