Re: [arch-general] KDE plasma, baloo and UI/video playback freezes
So, I finally tried the BFQ scheduler as Carsten suggested. I tested it eg. using youtube-dl to download a big video from Vimeo (it creates/remove quickly a lot of temp files that baloo indexes every time causing a lot of IO), while Qbittorrent was downloading several big files... So 100% disk activity going on (monitored with iotop). Overloading the disk queue this way, with BFQ enabled, the result was that the video playback of a 1080p movie by VLC was smooth and didn't hang/freeze at all. I can assume BFQ was able to handle better all the stuff going on. So thanks Carsten for the suggestion :) This fixed (or at least worked around) the problem. It remains to understand why on Debian and derivs. this issue doesn't exist. Maybe the CONFIG_PREEMPT_VOLUNTARY=y set by default on kernel's config or some other block device related configs I don't have the skill (and the time) to inspect now. Cheers -- fp
Re: [arch-general] KDE plasma, baloo and UI/video playback freezes
In data martedì 29 maggio 2018 16:26:14 CEST, hai scritto: > Indeed. It does work properly if the sender is the mailing list address (eg. > arch-general@archlinux.org), not the private address of the author (eg > frapox at gmail dot com). Sorry I was imprecise... The "reply to mailing list" of KMail works fine if the mail you're replying to is coming from the list, not straight from the author. -- fp
Re: [arch-general] KDE plasma, baloo and UI/video playback freezes
In data martedì 29 maggio 2018 15:19:28 CEST, Jagannathan Tiruvallur Eachambadi via arch-general ha scritto: > On 29-05-18 09:10:35 -0400, Eli Schwartz via arch-general wrote: > >On 05/29/2018 08:01 AM, Francesco Porro via arch-general wrote: > >> Can you see you (or your Mua) added my address to CC? (since the first > >> mail I got). If I hit reply, Kmail autofills the "To" field with your > >> mailbox instead of the mailinglist address, and I have to switch to ml > >> address manually every time (and accidentally I forgot to do it once, so > >> you received a mail directly from me, you assumed was a private reply). > >> Btw, no problem. > > > >Sounds like a bug in kmail, my Thunderbird replaces the default reply > >button with "Reply to list" and only sends to the list. > > Kmail has a reply to list button and it appears in the context menu as > well. I don't remember kmail having a problem earlier when I used with > respect to sending mails to a list. But this has been discussed a lot > already :P Indeed. It does work properly if the sender is the mailing list address (eg. arch-general@archlinux.org), not the private address of the author (eg frapox at gmail dot com). -- fp
Re: [arch-general] KDE plasma, baloo and UI/video playback freezes
On 29-05-18 09:10:35 -0400, Eli Schwartz via arch-general wrote: On 05/29/2018 08:01 AM, Francesco Porro via arch-general wrote: Can you see you (or your Mua) added my address to CC? (since the first mail I got). If I hit reply, Kmail autofills the "To" field with your mailbox instead of the mailinglist address, and I have to switch to ml address manually every time (and accidentally I forgot to do it once, so you received a mail directly from me, you assumed was a private reply). Btw, no problem. Sounds like a bug in kmail, my Thunderbird replaces the default reply button with "Reply to list" and only sends to the list. Kmail has a reply to list button and it appears in the context menu as well. I don't remember kmail having a problem earlier when I used with respect to sending mails to a list. But this has been discussed a lot already :P -- Regards Jagan signature.asc Description: PGP signature
Re: [arch-general] KDE plasma, baloo and UI/video playback freezes
On 05/29/2018 08:01 AM, Francesco Porro via arch-general wrote: > Can you see you (or your Mua) added my address to CC? (since the first mail I > got). If I hit reply, Kmail autofills the "To" field with your mailbox > instead > of the mailinglist address, and I have to switch to ml address manually every > time (and accidentally I forgot to do it once, so you received a mail > directly > from me, you assumed was a private reply). Btw, no problem. Sounds like a bug in kmail, my Thunderbird replaces the default reply button with "Reply to list" and only sends to the list. I actually have to go into "More" ==> "Reply all" if I want to CC people automatically. Even the "Reply" option finds the mailinglist address as the *From* address to reply to. -- Eli Schwartz Bug Wrangler and Trusted User signature.asc Description: OpenPGP digital signature
Re: [arch-general] KDE plasma, baloo and UI/video playback freezes
In data martedì 29 maggio 2018 13:14:00 CEST, hai scritto: > I think there's some confusion. I checked the mail you sent me, and > the only recipient was my address, no list addres in either field. > Therefore I assumed you mailed me directly, off-list, and so I didn't > add back the list in my reply. Usually, when someone does this, it's > considered off-list and meant to be private. It's better to miss the > list than expose a potentially personal email to the public list. Can you see you (or your Mua) added my address to CC? (since the first mail I got). If I hit reply, Kmail autofills the "To" field with your mailbox instead of the mailinglist address, and I have to switch to ml address manually every time (and accidentally I forgot to do it once, so you received a mail directly from me, you assumed was a private reply). Btw, no problem. > Interesting. That sounds like a performance bug in the interaction of > Konversation and Baloo. But it's still odd that Debian doesn't have > this. I assume the KDE version tested was the same on the different > distros. At this point I don't know if it's a bug of baloo or konversation (or a combination of the two), since It doesn't happen on Debian or Kubuntu. And yes, the version of KDE and related apps were the same. -- fp
Re: [arch-general] KDE plasma, baloo and UI/video playback freezes
On 5/29/18, Francesco Porro via arch-general wrote: > You don't need to add my address since I'm subscribed to the mailing > list :) > > (it you do, and I hit reply, Kmail replys only to your email by > default). I think there's some confusion. I checked the mail you sent me, and the only recipient was my address, no list addres in either field. Therefore I assumed you mailed me directly, off-list, and so I didn't add back the list in my reply. Usually, when someone does this, it's considered off-list and meant to be private. It's better to miss the list than expose a potentially personal email to the public list. Anyway, let's focus on the topic. > Btw, today i'm going to try changing the scheduler to BFQ, even if I Do not expect magical fixes. It may or may not help the I/O load that's affecting you. > found a workaround that avoid the baloo scheduler activating every > 10 seconds: simply I changed the path where to save Konversation's > logs to his .config dir (baloo doen't indexes dotted files o dirs). Interesting. That sounds like a performance bug in the interaction of Konversation and Baloo. But it's still odd that Debian doesn't have this. I assume the KDE version tested was the same on the different distros.
Re: [arch-general] KDE plasma, baloo and UI/video playback freezes
In data domenica 27 maggio 2018 18:27:22 CEST, hai scritto: > On 5/27/18, Francesco Porro via arch-general wrote: > > You mailed me privately. Better replying to mailing list since this could > > be > > > > useful to others && please do not top quote. > > Agreed, but I only replied to your direct mail to me. Apologies if > I missed the CC somehow. You don't need to add my address since I'm subscribed to the mailing list :) (it you do, and I hit reply, Kmail replys only to your email by default). Btw, today i'm going to try changing the scheduler to BFQ, even if I found a workaround that avoid the baloo scheduler activating every 10 seconds: simply I changed the path where to save Konversation's logs to his .config dir (baloo doen't indexes dotted files o dirs). -- fp
Re: [arch-general] KDE plasma, baloo and UI/video playback freezes
On 5/27/18, Francesco Porro via arch-general wrote: > You mailed me privately. Better replying to mailing list since this could be > > useful to others && please do not top quote. Agreed, but I only replied to your direct mail to me. Apologies if I missed the CC somehow.
Re: [arch-general] KDE plasma, baloo and UI/video playback freezes
You mailed me privately. Better replying to mailing list since this could be useful to others && please do not top quote. In data domenica 27 maggio 2018 15:27:13 CEST, hai scritto: > Please try different i/o schedulers and maybe ck or zen kernel > for extra BFQ. But first try deadline scheduler. And, like I > suggested, check if there's a difference in process and i/o > priority between Ubuntu and Arch KDE. How can I achieve this? I never changed the default scheduler before. I'd prefer to stay on the default kernel, because I don't have enough knowledge (and time or will) to maintain a custom kernel. The priority of the Baloo scheduler is the same between Debian and derivatives and Arch, as far as I can see. > Also, please consider filing an Arch bug with all relevant info. > Since you tested this with the same kernel version on different > distros, it's likely to be a configuration of the kernel and > applications on each distro. Sure I'll be doing that, but before I need to get more useful data to file a bug. -- fp
Re: [arch-general] KDE plasma, baloo and UI/video playback freezes
On 5/8/18, Francesco Porro wrote:
> In data martedì 8 maggio 2018 21:08:35 CEST, Carsten Mattner ha scritto:
>> Linux block layer's writeback system was supposed to fix this,
>> but I've also noticed that the mechanism isn't perfect and
>> you can still have a "hanging" application when doing the
>> infamous USB-to-USB transfer that kills the VM subsystem.
>>
>> Another way I can reproduce it is when there SSD-to-thumb-drive
>> and you decide to some disk activity, too.
>>
>> https://lwn.net/Articles/682582/
>>
>> The problem is that VM gets pressured a lot and the whole
>> construct fails in a way, while working as designed,
>> manifesting as hanging programs.
>
> Ok, but I'm talking about HDD on Sata bus. No Usb-to-usb transfers
> involved. And, as said before, I've altready tweaked the
> vm.dirty{writeback,background_ratio} to partialy word around this,
> either on Debian or Arch linux.
It's easier to trigger with slow devices but ultimately the
same issue of block layer and vm subsystem getting overwhelmed.
>> First, to confirm, if you manage to run the indexer like
>> you would `ionice -c idle `, and it shows
>> less hangs, you know the issue is unfair I/O queuing.
>
> The indexer's process autostarts and rapidly kills himself in a bunch of
> seconds... its impossible to renice the process, however I saw (by
> Ksysguard) it's always 19 as nice value... meaning high fairness both
> for the CPU, and I think for the IO too.
>
>
>> You can compare the block layer kernel configuration of
>> Debian vs Arch.
>
> Sorry, how to? Which are the keyword for searching?
On both systems there should be /proc/config.gz of the
running kernel. If you have a suspicion which kernel
option is responsible, you can check if it's different
between Arch and Debian.
>> You can try deadline or bfq schedulers. One is dead simple
>> and the other optimizes for desktop responsiveness.
>
> As a last chance, I'll look into these alternatives schedulers. But now,
> either Debian or Arch are using the same one: Cfq. So I don't think this
> is the cause of the problem.
Sounds true. Is it only that one application or others as well
when they cause much I/O?
Re: [arch-general] KDE plasma, baloo and UI/video playback freezes
In data martedì 8 maggio 2018 21:08:35 CEST, Carsten Mattner ha scritto:
> Linux block layer's writeback system was supposed to fix this,
> but I've also noticed that the mechanism isn't perfect and
> you can still have a "hanging" application when doing the
> infamous USB-to-USB transfer that kills the VM subsystem.
>
> Another way I can reproduce it is when there SSD-to-thumb-drive
> and you decide to some disk activity, too.
>
> https://lwn.net/Articles/682582/
>
> The problem is that VM gets pressured a lot and the whole
> construct fails in a way, while working as designed,
> manifesting as hanging programs.
Ok, but I'm talking about HDD on Sata bus. No Usb-to-usb transfers
involved. And, as said before, I've altready tweaked the
vm.dirty{writeback,background_ratio} to partialy word around this,
either on Debian or Arch linux.
> First, to confirm, if you manage to run the indexer like
> you would `ionice -c idle `, and it shows
> less hangs, you know the issue is unfair I/O queuing.
The indexer's process autostarts and rapidly kills himself in a bunch of
seconds... its impossible to renice the process, however I saw (by
Ksysguard) it's always 19 as nice value... meaning high fairness both
for the CPU, and I think for the IO too.
> You can compare the block layer kernel configuration of
> Debian vs Arch.
Sorry, how to? Which are the keyword for searching?
> You can try deadline or bfq schedulers. One is dead simple
> and the other optimizes for desktop responsiveness.
As a last chance, I'll look into these alternatives schedulers. But now,
either Debian or Arch are using the same one: Cfq. So I don't think this
is the cause of the problem.
--
fp
Re: [arch-general] KDE plasma, baloo and UI/video playback freezes
On 5/8/18, Francesco Porro via arch-general wrote: > Hi, > > My problem is: when I'm watching a video, running Konversation in the > background with logging on disk enabled, the playback of video slows > down, freezes for a while. > > Running iotop and ksysguard I found that baloo_file_extractor is writing > on disk a big amount of data, filling the write queue; balooct monitor > shows baloo is indexing Konversation's logs file changes. Peaks of 60-90 > MB/s of data on mechanical hdd are generated and the playback of videos > (or audio, or other GUI-related tasks) slow down or freeze. Very very > annoying. > > I'm running updated versions of: KDE plasma (5.12.5), kernel (4.16.7), > baloo (5.45); CFQ I/O scheduler on sda. > > On the same machine, the same version of softwares, the same > configurations (of programs, mounts, IO scheduler) BUT on a Debian sid > install (home is shared between installs) is NOT showing the same issue. > On Debian, playback is running smootly even if baloo_file_extractor is > heavly loading IO writes queue. > > On each installation I added this tweaks, to improve Vm management by > the kernel: > > [frapox@tungsteno ~]$ cat /etc/sysctl.d/local.conf > vm.dirty_background_ratio = 2 > vm.dirty_ratio = 5 > vm.vfs_cache_pressure = 60 > > I also tried to recompile the Arch kernel with > CONFIG_PREEMPT_VOLUNTARY=y, to reflect the same default compile setting > of Debian, but the issue wasn't solved. > > So i'm wondering which other config files or settings/tweaks I can look > into to overcome this issue on Arch linux (I'd really love to keep on > use Arch as my main OS). Linux block layer's writeback system was supposed to fix this, but I've also noticed that the mechanism isn't perfect and you can still have a "hanging" application when doing the infamous USB-to-USB transfer that kills the VM subsystem. Another way I can reproduce it is when there SSD-to-thumb-drive and you decide to some disk activity, too. https://lwn.net/Articles/682582/ The problem is that VM gets pressured a lot and the whole construct fails in a way, while working as designed, manifesting as hanging programs. First, to confirm, if you manage to run the indexer like you would `ionice -c idle `, and it shows less hangs, you know the issue is unfair I/O queuing. You can compare the block layer kernel configuration of Debian vs Arch. You can try deadline or bfq schedulers. One is dead simple and the other optimizes for desktop responsiveness.
