https://bugs.kde.org/show_bug.cgi?id=475205
Bug ID: 475205 Summary: [UX/Perf] : UI freezes when handling big projects / adding lots of clips in the resource folder Classification: Applications Product: kdenlive Version: 23.08.0 Platform: Archlinux OS: Linux Status: REPORTED Severity: major Priority: NOR Component: User Interface Assignee: j...@kdenlive.org Reporter: benoittarr...@hotmail.fr Target Milestone: --- Created attachment 162080 --> https://bugs.kde.org/attachment.cgi?id=162080&action=edit Kdenlive starts to freeze right away (loading a big project) SUMMARY Kdenlive UI seems to be very tied to the underlying processes, thus making the UI event processing loop stuck for some reasons. This happens either when : * dropping a big folder into the resource explorer * opening a saved project which points to a lot of resources. In my context, I am working on a project where I have around 1500 items (video and pictures) for a total of around 33 GB. That's a lot of files, and kdenlive (or mlt!) takes sometimes close to 10 minutes scanning all of them, making the UI totally unresponsive. This happens on every load as well, so it seems caching is not sufficient (by the way, I'm having a lot of logs saying "GOT THUMBNAIL : -1 x -1" ; don't know what that means exactly but it seems that the cache contains thumbnails at least. => This is an I/O bound issue, Kdenlive lacks asynchronicity when it comes to opening files/and performing preprocessing tasks on them (parsing/loading content, computing thumbnails, etc.) STEPS TO REPRODUCE 1. Open a new project 2. Drop a big folder with few tens of clips in the resource explorer 3. Alternative : save a big project and reopen it later on, file indexing seems to be taking very long as well and if the amount of files is big, it takes forever (especially if data is stored on a not-brand-new HDD). OBSERVED RESULT The loading dialog appears (and is more or less frozen), most of the times the Desktop manager shows and asks if you want to kill the process (because Qt needs to ping the Destkop manager's watchdog periodically, otherwise it's considered a zombie process) Then wait. Eventually Kdenlive will get to the point where it finishes loading, or at least gets out of the work intensive loop it's stuck in and the little "xxx tasks" counter increases suddenly, and the UI comes back to life. EXPECTED RESULT The UI should remain responsive at all times, and the user shall be able to cancel the loading process if he/she needs to. Like "Whoops, I dropped the wrong folder ! I do I do know to stop it from loading this 4000 files rush folder ??". For now you need to kill the process, or make one good cup of coffee :) SOFTWARE/OS VERSIONS Linux/KDE Plasma: Linux arch-linux 6.1.55-1-lts #1 SMP PREEMPT_DYNAMIC /// Gnome 44.5 / X11 Qt Version: Qt 5.15.x ADDITIONAL INFORMATION > /!\ I haven't taken the time to read the source code yet, maybe what I'm > saying here is off (and in such a case I apologize), but I'm trying to lay > out some ideas here My gut feeling is that we need to defer (and push in background loading tasks) files loading as much as we can and prioritize the loading order so that the end user can start working as soon as possible ! * A simple stat should be enough to check if the file is reachable and valid * All I/O bound operations need to be as asynchronous as possible (maybe using the std::future for ease of use, or put in a background thread that'll wait the I/O and move on to the next one) * Thumbnails shall be computed only for the visible par of the viewport in the resources folder and cached for later reuse, I believe the caching is already written, but I think thumbnail generation really takes too much time (again, my gut feeling; I might really well be wrong) -- You are receiving this mail because: You are watching all bug changes.