[digikam] [Bug 405235] Slow startup and new file scan on low latency networks
https://bugs.kde.org/show_bug.cgi?id=405235 --- Comment #56 from Mark Stoughton --- Everything is fine on this end now. As far as I'm concerned this can be closed -- You are receiving this mail because: You are watching all bug changes.
[digikam] [Bug 405235] Slow startup and new file scan on low latency networks
https://bugs.kde.org/show_bug.cgi?id=405235 --- Comment #55 from Mark Stoughton --- I think I fixed it. Two different solutions, one for PC one for Mac. Both versions seem to be running fine now. For the PC -Uninstall and reinstall Digikam. For the Mac - Uninstall and reinstall Digikam - Create a new database form scratch and reimport the photos into discrete albums, one album per folder They were both really slow, but the reinstall worked by itself for the PC, but I'm lost as to why. So it got me thinking about how I built the database each one. n the PC, I imported folders one at a time into discrete albums. On the Mac I did it all at once into one huge master album. The existing database worked on the PC, but not on the Mac. So when I created a new database on the Mac, I imported one folder at a time into discrete albums. Digikam is now working perfectly on both platforms. -- You are receiving this mail because: You are watching all bug changes.
[digikam] [Bug 405235] Slow startup and new file scan on low latency networks
https://bugs.kde.org/show_bug.cgi?id=405235 Mark Stoughton changed: What|Removed |Added OS|All |macOS --- Comment #54 from Mark Stoughton --- I tried uninstalling and reinstalling Digikam on both my Mac nd PC. After that, the PC runs normally, and NAS performance is excellent. No change on the Mac, still slow as molasses. I've changed the platform to Mac only. Hope this helps. -- You are receiving this mail because: You are watching all bug changes.
[digikam] [Bug 405235] Slow startup and new file scan on low latency networks
https://bugs.kde.org/show_bug.cgi?id=405235 --- Comment #53 from Mark Stoughton --- I've found one more thing. I swapped databases between the one that uses local files and the one that uses files on the NAS. The only difference was the location of the photo files. The databases were both on local folders on the SSD attached to my Mac. Using photo files stored locally: Digicam scans the folders in about 2 seconds and starts normally Using photo files stored on the NAS Digicam scanned for over an hour at 0% before I cancelled it. -- You are receiving this mail because: You are watching all bug changes.
[digikam] [Bug 405235] Slow startup and new file scan on low latency networks
https://bugs.kde.org/show_bug.cgi?id=405235 Mark Stoughton changed: What|Removed |Added OS|Linux |All -- You are receiving this mail because: You are watching all bug changes.
[digikam] [Bug 405235] Slow startup and new file scan on low latency networks
https://bugs.kde.org/show_bug.cgi?id=405235 --- Comment #52 from Mark Stoughton --- I ran a few more tests. I think that I've isolated the issue to occur only when photos are stored on the NAS. Test 1: Move the database off the System drive to a Thunderbolt Attached SSD - Photos still on the NAS -Startup time reduced to 21 minutes -Changing folders took 3 minutes each time Test 2: Copied files to the SSD Drive and Created a new database on the SSD Drive -Startup time took about 10 seconds -Changing folders took 3-5 seconds if the folder had no thumbnails, 1 second if it had thumbnails Test 3: Read all database tables using SQL Lite - Nearly instant response in all cases for 118k photo records Test 4: Opening photo files using Finder/Windows Explorer from the NAS - File on SSD - 3 seconds - File on NAS - 3 seconds I can reproduce this behavior on both my Mac and my PC. -- You are receiving this mail because: You are watching all bug changes.
[digikam] [Bug 405235] Slow startup and new file scan on low latency networks
https://bugs.kde.org/show_bug.cgi?id=405235 --- Comment #51 from Mark Stoughton --- I ran a few more tests. I think that I've isolated the issue to occur only when photos are stored on the NAS. Test 1: Move the database off the System drive to a USB Attached SSD -Startup time reduced to 21 minutes -Changing folders took 3 minutes each time Test 2: Copied files to the SSD Drive and Created a new database on the SSD Drive -Startup time took about 10 seconds -Changing folders took 3-5 seconds if the folder had no thumbnails, 1 second if it had thumbnails Test 3: Read all database tables using SQL Lite - Nearly instant response in all cases for 118k photo records Test 4: Opening photo files using Finder/Windows Explorer from the NAS - File on SSD - 3 seconds - File on NAS - 3 seconds I can reproduce this behavior on both my Mac and my PC. -- You are receiving this mail because: You are watching all bug changes.
[digikam] [Bug 405235] Slow startup and new file scan on low latency networks
https://bugs.kde.org/show_bug.cgi?id=405235 --- Comment #49 from Mark Stoughton --- (In reply to Maik Qualmann from comment #48) > If you change an album it's a pure database operation. It doesn't matter > whether the album is on a NAS. As a test, I would move the digiKam databases > to a completely different directory. > > Maik Hi Maik, I moved the database to a thunderbolt attached external SSD. Digikam took 21 minutes to load, down from ~40, and the splash screen stayed visiblele the entire time. It still consistently took 3 minutes to change folders, regardless of the number of photos in the folder. I'm backing up the folders on the NAS to a USB hard drive. I'm going to try using a directly attached drive to see if that works. I'm thinking that if that works normally, the issue is somewhere between Digikam and files on the NAS. -- You are receiving this mail because: You are watching all bug changes.
[digikam] [Bug 405235] Slow startup and new file scan on low latency networks
https://bugs.kde.org/show_bug.cgi?id=405235 --- Comment #47 from Mark Stoughton --- Hi, That is correct, it takes about 3 minutes to change folders while using the program. While in a single folder the program works fine, but takes about a minute to read the metadata, particularly the EXIF data. I've checked the permissions, and Digikam has RW on all folders on the NAS. My database is on the system drive, which is an SSD. Querying the database using SQLite Studio returns results instantly. Just to investigate further, I duplicated Giles' setup and installed Digikam on a different Mac, put about 1000 photos on the system drive and created a new database on the system drive. Digikam performed flawlessly. This makes me think that there is some issue when files are stored on a NAS. I don't think it's a latency issue because I have been investigating DAM software and have installed several of them (ACDSee, Photo Mechanic, Apple Photos, Adobe Bridge, Luminar) on this machine, and Digikam is the only one that behaves this way. It's unfortunate because Digikam would be my favorite otherwise. The log that I have annotated shows where Digikam pauses during execution, and for how long. It pauses in the same places and for roughly the same amount of time whether monitoring for external changes is on or off. (In reply to Maik Qualmann from comment #44) > I understand you correctly that the change from an album takes 3 minutes > after the start? > This is a database operation, in principle there is no loss of time. You > must have very, very slow access to your database. Since there are no error > messages in the log, the behavior cannot be explained, check the rights as > Gilles already recommends. > > Maik -- You are receiving this mail because: You are watching all bug changes.
[digikam] [Bug 405235] Slow startup and new file scan on low latency networks
https://bugs.kde.org/show_bug.cgi?id=405235 --- Comment #46 from Mark Stoughton --- I checked the rights and digicam does have RW on the entire drive. The only thing that I can think of is that there may be some issue because the files are on a NAS. So this morning I installed it on another Mac, with photos on the system drive, and Digikam works perfectly. Hope this helps -- You are receiving this mail because: You are watching all bug changes.
[digikam] [Bug 405235] Slow startup and new file scan on low latency networks
https://bugs.kde.org/show_bug.cgi?id=405235 --- Comment #45 from Mark Stoughton --- (In reply to Maik Qualmann from comment #44) > I understand you correctly that the change from an album takes 3 minutes > after the start? > This is a database operation, in principle there is no loss of time. You > must have very, very slow access to your database. Since there are no error > messages in the log, the behavior cannot be explained, check the rights as > Gilles already recommends. > > Maik In both cases (Mac and PC), the database is on the system drive, which are SSDs in both cases. The photos themselves are stored on regular hard drives on a NAS. Each folder has anywhere between 100 to 1000 photos in it. When I query the database using SQLite studio, the results come back instantaneously. I can reproduce this exact behavior on both my Mac and PC installs. While the watching Digikam write the log, there are several long pauses as I documented in the first log that I sent. I didn't annotate the second one because the pauses were all in the same places and for similar times. It appears to me that Digikam is taking an abnormally long time to complete whatever it does at those points. I've tried several DAM packages and Digikam is the only one that behaves this way. -- You are receiving this mail because: You are watching all bug changes.
[digikam] [Bug 405235] Slow startup and new file scan on low latency networks
https://bugs.kde.org/show_bug.cgi?id=405235 --- Comment #43 from Mark Stoughton --- This is a run with the boxes unchecked. Load time was down to 30 minutes exactly Last login: Thu Feb 10 12:53:11 on ttys000 markstoughton@Marks-Mac-mini ~ % export QT_LOGGING_RULES="digikam*=true" /Applications/digiKam.org/digikam.app/Contents/MacOS/digikam digikam.widgets: Breeze icons resource file found digikam.widgets: Breeze-dark icons resource file found digikam.general: Qt standard translations removed: 0 digikam.general: Qt standard translations path: "/Applications/digiKam.org/digikam.app/Contents/Resources/translations" digikam.general: Loaded Qt standard translations "en_US" from catalog "qt" digikam.general: Loaded Qt standard translations "en_US" from catalog "qtbase" digikam.general: Loaded Qt standard translations "en_US" from catalog "qt_help" digikam.general: Loaded Qt standard translations "en_US" from catalog "qtdeclarative" digikam.general: Loaded Qt standard translations "en_US" from catalog "qtquickcontrols" digikam.general: Loaded Qt standard translations "en_US" from catalog "qtquickcontrols2" digikam.general: Loaded Qt standard translations "en_US" from catalog "qtmultimedia" digikam.general: Loaded Qt standard translations "en_US" from catalog "qtwebengine" digikam.general: Loaded Qt standard translations "en_US" from catalog "qtxmlpatterns" digikam.general: Loaded Qt ECM translations "en" from catalog "kcoreaddons5_qt" digikam.general: Loaded Qt ECM translations "en" from catalog "kwidgetsaddons5_qt" digikam.general: Switch to widget style: "Fusion" digikam.general: AlbumWatch is disabled digikam.general: Database Parameters: Type: "QSQLITE" DB Core Name: "/Users/markstoughton/Pictures/Digikam/digikam4.db" DB Thumbs Name: "/Users/markstoughton/Pictures/Digikam/thumbnails-digikam.db" DB Face Name: "/Users/markstoughton/Pictures/Digikam/recognition.db" DB Similarity Name: "/Users/markstoughton/Pictures/Digikam/similarity.db" Connect Options: "" Host Name: "" Host port: -1 Internal Server: false Internal Server Path: "" Internal Server Admin Cmd: "" Internal Server Serv Cmd: "" Internal Server Init Cmd: "" Username: "" Password: "" digikam.dbengine: Loading SQL code from config file "/Applications/digiKam.org/digikam.app/Contents/Resources/digikam/database/dbconfig.xml" digikam.dbengine: Checking XML version ID => expected: 3 found: 3 digikam.coredb: Core database: running schema update digikam.coredb: Core database: have a structure version 15 digikam.coredb: Core database: makeUpdates 15 to 15 digikam.database: Creating new Location "/" uuid "networkshareid:?mountpath=/Volumes/OurPhotos/Photo Library/2021" digikam.database: Creating new Location "/" uuid "networkshareid:?mountpath=/Volumes/OurPhotos/Photo Library/2020" digikam.database: Creating new Location "/" uuid "networkshareid:?mountpath=/Volumes/OurPhotos/Photo Library/2019" digikam.database: Creating new Location "/" uuid "networkshareid:?mountpath=/Volumes/OurPhotos/Photo Library/2018" digikam.database: Creating new Location "/" uuid "networkshareid:?mountpath=/Volumes/OurPhotos/Photo Library/2017" digikam.database: Creating new Location "/" uuid "networkshareid:?mountpath=/Volumes/OurPhotos/Photo Library/2016" digikam.database: Creating new Location "/" uuid "networkshareid:?mountpath=/Volumes/OurPhotos/Photo Library/2015" digikam.database: Creating new Location "/" uuid "networkshareid:?mountpath=/Volumes/OurPhotos/Photo Library/2014" digikam.database: location for "/Volumes/OurPhotos/Photo Library/2021" is available true digikam.database: location for "/Volumes/OurPhotos/Photo Library/2020" is available true digikam.database: location for "/Volumes/OurPhotos/Photo Library/2019" is available true digikam.database: location for "/Volumes/OurPhotos/Photo Library/2018" is available true digikam.database: location for "/Volumes/OurPhotos/Photo Library/2017" is available true digikam.database: location for "/Volumes/OurPhotos/Photo Library/2016" is available true digikam.database: location for "/Volumes/OurPhotos/Photo Library/2015" is available true digikam
[digikam] [Bug 405235] Slow startup and new file scan on low latency networks
https://bugs.kde.org/show_bug.cgi?id=405235 --- Comment #42 from Mark Stoughton --- I unchecked the Settings-Collections I unchecked the box that says "Monitor the ablbums for external changes(requires restart). Is that the right one? If not, please direct me to the right one. Thanks. P.S. The log that I sent was before I unchecked that box. I'm rerunning it now and will post when it's done -- You are receiving this mail because: You are watching all bug changes.
[digikam] [Bug 405235] Slow startup and new file scan on low latency networks
https://bugs.kde.org/show_bug.cgi?id=405235 --- Comment #38 from Mark Stoughton --- After some use, the album switching got down to. fairly repeatable three minutes, regardless of the number of photos in the album -- You are receiving this mail because: You are watching all bug changes.
[digikam] [Bug 405235] Slow startup and new file scan on low latency networks
https://bugs.kde.org/show_bug.cgi?id=405235 --- Comment #37 from Mark Stoughton --- I deactivated the monitor external changes and restarted. I saw no difference in either startup time (40 minutes) or album switching (15 minutes). -- You are receiving this mail because: You are watching all bug changes.
[digikam] [Bug 405235] Slow startup and new file scan on low latency networks
https://bugs.kde.org/show_bug.cgi?id=405235 --- Comment #36 from Mark Stoughton --- It was activated. I'll try again after deactivating it (In reply to Maik Qualmann from comment #34) > I assume you have activated the option to monitor for external changes in > the digiKam settings under Collection. This option must be deactivated under > MacOS, as it causes the albums to load extremely slowly. > > Maik -- You are receiving this mail because: You are watching all bug changes.
[digikam] [Bug 405235] Slow startup and new file scan on low latency networks
https://bugs.kde.org/show_bug.cgi?id=405235 --- Comment #35 from Mark Stoughton --- I ran the debug session. I've annotated it because digicam stopped for several minutes in different places along the way, and the log does not show the pauses. It took 41 minutes from start to finish markstoughton@Marks-Mac-mini ~ % export QT_LOGGING_RULES="digikam*=true" /Applications/digiKam.org/digikam.app/Contents/MacOS/digikam digikam.widgets: Breeze icons resource file found digikam.widgets: Breeze-dark icons resource file found digikam.general: Qt standard translations removed: 0 digikam.general: Qt standard translations path: "/Applications/digiKam.org/digikam.app/Contents/Resources/translations" digikam.general: Loaded Qt standard translations "en_US" from catalog "qt" digikam.general: Loaded Qt standard translations "en_US" from catalog "qtbase" digikam.general: Loaded Qt standard translations "en_US" from catalog "qt_help" digikam.general: Loaded Qt standard translations "en_US" from catalog "qtdeclarative" digikam.general: Loaded Qt standard translations "en_US" from catalog "qtquickcontrols" digikam.general: Loaded Qt standard translations "en_US" from catalog "qtquickcontrols2" digikam.general: Loaded Qt standard translations "en_US" from catalog "qtmultimedia" digikam.general: Loaded Qt standard translations "en_US" from catalog "qtwebengine" digikam.general: Loaded Qt standard translations "en_US" from catalog "qtxmlpatterns" digikam.general: Loaded Qt ECM translations "en" from catalog "kcoreaddons5_qt" digikam.general: Loaded Qt ECM translations "en" from catalog "kwidgetsaddons5_qt" digikam.general: Switch to widget style: "Fusion" digikam.general: AlbumWatch use QFileSystemWatcher digikam.general: Database Parameters: Type: "QSQLITE" DB Core Name: "/Users/markstoughton/Pictures/Digikam/digikam4.db" DB Thumbs Name: "/Users/markstoughton/Pictures/Digikam/thumbnails-digikam.db" DB Face Name: "/Users/markstoughton/Pictures/Digikam/recognition.db" DB Similarity Name: "/Users/markstoughton/Pictures/Digikam/similarity.db" Connect Options: "" Host Name: "" Host port: -1 Internal Server: false Internal Server Path: "" Internal Server Admin Cmd: "" Internal Server Serv Cmd: "" Internal Server Init Cmd: "" Username: "" Password: "" digikam.dbengine: Loading SQL code from config file "/Applications/digiKam.org/digikam.app/Contents/Resources/digikam/database/dbconfig.xml" digikam.dbengine: Checking XML version ID => expected: 3 found: 3 digikam.coredb: Core database: running schema update digikam.coredb: Core database: have a structure version 15 digikam.coredb: Core database: makeUpdates 15 to 15 digikam.database: Creating new Location "/" uuid "networkshareid:?mountpath=/Volumes/OurPhotos/Photo Library/2021" digikam.database: Creating new Location "/" uuid "networkshareid:?mountpath=/Volumes/OurPhotos/Photo Library/2020" digikam.database: Creating new Location "/" uuid "networkshareid:?mountpath=/Volumes/OurPhotos/Photo Library/2019" digikam.database: Creating new Location "/" uuid "networkshareid:?mountpath=/Volumes/OurPhotos/Photo Library/2018" digikam.database: Creating new Location "/" uuid "networkshareid:?mountpath=/Volumes/OurPhotos/Photo Library/2017" digikam.database: Creating new Location "/" uuid "networkshareid:?mountpath=/Volumes/OurPhotos/Photo Library/2016" digikam.database: Creating new Location "/" uuid "networkshareid:?mountpath=/Volumes/OurPhotos/Photo Library/2015" digikam.database: Creating new Location "/" uuid "networkshareid:?mountpath=/Volumes/OurPhotos/Photo Library/2014" digikam.database: location for "/Volumes/OurPhotos/Photo Library/2021" is available true digikam.database: location for "/Volumes/OurPhotos/Photo Library/2020" is available true digikam.database: location for "/Volumes/OurPhotos/Photo Library/2019" is available true digikam.database: location for "/Volumes/OurPhotos/Photo Library/2018" is available true digikam.database: location for "/Volumes/OurPhotos/Photo Library/2017" is available true digikam.database: location for "/Volumes/OurPhotos/Photo Library/2016" is available true digikam.database: locat
[digikam] [Bug 405235] Slow startup and new file scan on low latency networks
https://bugs.kde.org/show_bug.cgi?id=405235 --- Comment #33 from Mark Stoughton --- sorry for the slow reply. It's 10 minutes after the smash screen disappears. The app shows up as non-responsive in Task manager in Windows or in Force Quit on the Mac. -- You are receiving this mail because: You are watching all bug changes.
[digikam] [Bug 405235] Slow startup and new file scan on low latency networks
https://bugs.kde.org/show_bug.cgi?id=405235 --- Comment #32 from Mark Stoughton --- Hi, It's from the time the splash screen disappears until the interface shows up. During that time there is no indicator that Digikam is even running at all. The photos were already imported. It shows up as "not responding" in my Mac Force Quit Applications box or Windows Task Manager. It also takes several minutes of appearing to be frozen when I switch between folders. As long as I'm working in one folder, the performance is slow but useable. The minute I switch folders, I have to wait another 10 minutes or so for Digikam to wake up again. Latency on my network to the SAN is 0.5ms. Throughput is about 100MB/s. I don't have tools to test latency installed right now. I also have ACDSee installed on both the Mac and PC, and neither of those exhibits this issue. From: MarcP Sent: Saturday, January 22, 2022 1:23 PM To: geoph...@hotmail.com Subject: [digikam] [Bug 405235] Slow startup and new file scan on low latency networks https://bugs.kde.org/show_bug.cgi?id=405235 --- Comment #31 from MarcP --- Hi Mark, you mean 10-minutes since the digikam splash disappears and the interface shows up, or until the progress bar reaches 100%? Are the pictures in your library already imported or it's the first scan? I'm now in a high latency network (~120ms to my NAS), and it's not nearly as bad as it was with v6.x versions. The interface shows up rather quickly and it's true that the initial scan can take a while, but if there aren't many new pictures it's just a matter of minutes. -- You are receiving this mail because: You are on the CC list for the bug. -- You are receiving this mail because: You are watching all bug changes.
[digikam] [Bug 405235] Slow startup and new file scan on low latency networks
https://bugs.kde.org/show_bug.cgi?id=405235 Mark Stoughton changed: What|Removed |Added Version|7.4.0 |7.5.0 CC||geoph...@hotmail.com --- Comment #29 from Mark Stoughton --- I am seeing this on the 7.5.0 install. I have my photos stored on a Synology NAS in a shared set of folders. They contain approximately 50,000 photos. It takes 10-20 minutes for DigiKam to start up. The splash screen shows for about 20 seconds, then disappears. In fact, I started Digikam, went and got a cup of coffee, logged into the bugs site, searched for the problem, found this bug and am making this entry while waiting. I'm seeing this behavior on both my Mac and Windows installs. I'm willing to do whatever logging that you require, and to run experiments if you want. -- You are receiving this mail because: You are watching all bug changes.