[digikam] [Bug 405235] Slow startup and new file scan on low latency networks

2022-02-24 Thread Mark Stoughton
https://bugs.kde.org/show_bug.cgi?id=405235

--- Comment #56 from Mark Stoughton  ---
Everything is fine on this end now.  As far as I'm concerned this can be closed

-- 
You are receiving this mail because:
You are watching all bug changes.

[digikam] [Bug 405235] Slow startup and new file scan on low latency networks

2022-02-19 Thread Mark Stoughton
https://bugs.kde.org/show_bug.cgi?id=405235

--- Comment #55 from Mark Stoughton  ---
I think I fixed it.  Two different solutions, one for PC one for Mac. Both
versions seem to be running fine now.

For the PC
-Uninstall and reinstall Digikam.  

For the Mac
- Uninstall and reinstall Digikam
- Create a new database form scratch and reimport the photos into discrete
albums, one album per folder

They were both really slow, but the reinstall worked by itself for the PC, but
I'm lost as to why.  So it got me thinking about how I built the database each
one.  n the PC, I imported folders one at a time into discrete albums.  On the
Mac I did it all at once into one huge master album.  
The existing database worked on the PC, but not on the Mac.  So when I created
a new database on the Mac, I imported one folder at a time into discrete
albums.  Digikam is now working perfectly on both platforms.

-- 
You are receiving this mail because:
You are watching all bug changes.

[digikam] [Bug 405235] Slow startup and new file scan on low latency networks

2022-02-18 Thread Mark Stoughton
https://bugs.kde.org/show_bug.cgi?id=405235

Mark Stoughton  changed:

   What|Removed |Added

 OS|All |macOS

--- Comment #54 from Mark Stoughton  ---
I tried uninstalling and reinstalling Digikam on both my Mac nd PC.  After
that, the PC runs normally, and NAS performance is excellent.  No change on the
Mac, still slow as molasses.  I've changed the platform to Mac only.  Hope this
helps.

-- 
You are receiving this mail because:
You are watching all bug changes.

[digikam] [Bug 405235] Slow startup and new file scan on low latency networks

2022-02-17 Thread Mark Stoughton
https://bugs.kde.org/show_bug.cgi?id=405235

--- Comment #53 from Mark Stoughton  ---
I've found one more thing.  I swapped databases between the one that uses local
files and the one that uses files on the NAS.

The only difference was the location of the photo files.  The databases were
both on local folders on the SSD attached to my Mac.

Using photo files stored locally:
Digicam scans the folders in about 2 seconds and starts normally

Using photo files stored on the NAS
Digicam scanned for over an hour at 0% before I cancelled it.

-- 
You are receiving this mail because:
You are watching all bug changes.

[digikam] [Bug 405235] Slow startup and new file scan on low latency networks

2022-02-16 Thread Mark Stoughton
https://bugs.kde.org/show_bug.cgi?id=405235

Mark Stoughton  changed:

   What|Removed |Added

 OS|Linux   |All

-- 
You are receiving this mail because:
You are watching all bug changes.

[digikam] [Bug 405235] Slow startup and new file scan on low latency networks

2022-02-16 Thread Mark Stoughton
https://bugs.kde.org/show_bug.cgi?id=405235

--- Comment #52 from Mark Stoughton  ---
I ran a few more tests.  I think that I've isolated the issue to occur only
when photos are stored on the NAS.  

Test 1: Move the database off the System drive to a Thunderbolt Attached SSD -
Photos still on the NAS
-Startup time reduced to 21 minutes
-Changing folders took 3 minutes each time

Test 2: Copied files to the SSD Drive and Created a new database on the SSD
Drive
-Startup time took about 10 seconds
-Changing folders took 3-5 seconds if the folder had no thumbnails, 1 second if
it had thumbnails

Test 3: Read all database tables using SQL Lite
- Nearly instant response in all cases for 118k photo records

Test 4: Opening photo files using Finder/Windows Explorer from the NAS
- File on SSD - 3 seconds
- File on NAS - 3 seconds

I can reproduce this behavior on both my Mac and my PC.

-- 
You are receiving this mail because:
You are watching all bug changes.

[digikam] [Bug 405235] Slow startup and new file scan on low latency networks

2022-02-16 Thread Mark Stoughton
https://bugs.kde.org/show_bug.cgi?id=405235

--- Comment #51 from Mark Stoughton  ---
I ran a few more tests.  I think that I've isolated the issue to occur only
when photos are stored on the NAS.  

Test 1: Move the database off the System drive to a USB Attached SSD
-Startup time reduced to 21 minutes
-Changing folders took 3 minutes each time

Test 2: Copied files to the SSD Drive and Created a new database on the SSD
Drive
-Startup time took about 10 seconds
-Changing folders took 3-5 seconds if the folder had no thumbnails, 1 second if
it had thumbnails

Test 3: Read all database tables using SQL Lite
- Nearly instant response in all cases for 118k photo records

Test 4: Opening photo files using Finder/Windows Explorer from the NAS
- File on SSD - 3 seconds
- File on NAS - 3 seconds

I can reproduce this behavior on both my Mac and my PC.

-- 
You are receiving this mail because:
You are watching all bug changes.

[digikam] [Bug 405235] Slow startup and new file scan on low latency networks

2022-02-14 Thread Mark Stoughton
https://bugs.kde.org/show_bug.cgi?id=405235

--- Comment #49 from Mark Stoughton  ---
(In reply to Maik Qualmann from comment #48)
> If you change an album it's a pure database operation. It doesn't matter
> whether the album is on a NAS. As a test, I would move the digiKam databases
> to a completely different directory.
> 
> Maik

Hi Maik,

I moved the database to a thunderbolt attached external SSD.  Digikam took 21
minutes to load, down from ~40, and the splash screen stayed visiblele the
entire time.  It still consistently took 3 minutes to change folders,
regardless of the number of photos in the folder.  

I'm backing up the folders on the NAS to a USB hard drive.  I'm going to try
using a directly attached drive to see if that works.  I'm thinking that if
that works normally, the issue is somewhere between Digikam and files on the
NAS.

-- 
You are receiving this mail because:
You are watching all bug changes.

[digikam] [Bug 405235] Slow startup and new file scan on low latency networks

2022-02-11 Thread Mark Stoughton
https://bugs.kde.org/show_bug.cgi?id=405235

--- Comment #47 from Mark Stoughton  ---
Hi,

That is correct, it takes about 3 minutes to change folders while using the
program.  While in a single folder the program works fine, but takes about a
minute to read the metadata, particularly the EXIF data.

I've checked the permissions, and Digikam has RW on all folders on the NAS.  
My database is on the system drive, which is an SSD.  
Querying the database using SQLite Studio returns results instantly.  

Just to investigate further, I duplicated Giles' setup and installed Digikam on
a different Mac, put about 1000 photos on the system drive and created a new
database on the system drive. Digikam performed flawlessly.  This makes me
think that there is some issue when files are stored on a NAS.  I don't think
it's a latency issue because I have been investigating DAM software and have
installed several of them (ACDSee, Photo Mechanic, Apple Photos, Adobe Bridge,
Luminar) on this machine, and Digikam is the only one that behaves this way. 
It's unfortunate because Digikam would be my favorite otherwise.

The log that I have annotated shows where Digikam pauses during execution, and
for how long.  It pauses in the same places and for roughly the same amount of
time whether monitoring for external changes is on or off.

(In reply to Maik Qualmann from comment #44)
> I understand you correctly that the change from an album takes 3 minutes
> after the start?
> This is a database operation, in principle there is no loss of time. You
> must have very, very slow access to your database. Since there are no error
> messages in the log, the behavior cannot be explained, check the rights as
> Gilles already recommends.
> 
> Maik

-- 
You are receiving this mail because:
You are watching all bug changes.

[digikam] [Bug 405235] Slow startup and new file scan on low latency networks

2022-02-11 Thread Mark Stoughton
https://bugs.kde.org/show_bug.cgi?id=405235

--- Comment #46 from Mark Stoughton  ---
I checked the rights and digicam does have RW on the entire drive.  The only
thing that I can think of is that there may be some issue because the files are
on a NAS.  So this morning I installed it on another Mac, with photos on the
system drive, and Digikam works perfectly.  Hope this helps

-- 
You are receiving this mail because:
You are watching all bug changes.

[digikam] [Bug 405235] Slow startup and new file scan on low latency networks

2022-02-11 Thread Mark Stoughton
https://bugs.kde.org/show_bug.cgi?id=405235

--- Comment #45 from Mark Stoughton  ---
(In reply to Maik Qualmann from comment #44)
> I understand you correctly that the change from an album takes 3 minutes
> after the start?
> This is a database operation, in principle there is no loss of time. You
> must have very, very slow access to your database. Since there are no error
> messages in the log, the behavior cannot be explained, check the rights as
> Gilles already recommends.
> 
> Maik

In both cases (Mac and PC), the database is on the system drive, which are SSDs
in both cases.  The photos themselves are stored on regular hard drives on a
NAS.  Each folder has anywhere between 100 to 1000 photos in it. When I query
the database using SQLite studio, the results come back instantaneously. I can
reproduce this exact behavior on both my Mac and PC installs.  While the
watching Digikam write the log, there are several long pauses as I documented
in the first log that I sent.  I didn't annotate the second one because the
pauses were all in the same places and for similar times.  It appears to me
that Digikam is taking an abnormally long time to complete whatever it does at
those points.  I've tried several DAM packages and Digikam is the only one that
behaves this way.

-- 
You are receiving this mail because:
You are watching all bug changes.

[digikam] [Bug 405235] Slow startup and new file scan on low latency networks

2022-02-10 Thread Mark Stoughton
https://bugs.kde.org/show_bug.cgi?id=405235

--- Comment #43 from Mark Stoughton  ---
This is a run with the boxes unchecked.  Load time was down to 30 minutes
exactly

Last login: Thu Feb 10 12:53:11 on ttys000
markstoughton@Marks-Mac-mini ~ % export QT_LOGGING_RULES="digikam*=true"
/Applications/digiKam.org/digikam.app/Contents/MacOS/digikam
digikam.widgets: Breeze icons resource file found
digikam.widgets: Breeze-dark icons resource file found
digikam.general: Qt standard translations removed: 0
digikam.general: Qt standard translations path:
"/Applications/digiKam.org/digikam.app/Contents/Resources/translations"
digikam.general: Loaded Qt standard translations "en_US" from catalog "qt"
digikam.general: Loaded Qt standard translations "en_US" from catalog "qtbase"
digikam.general: Loaded Qt standard translations "en_US" from catalog "qt_help"
digikam.general: Loaded Qt standard translations "en_US" from catalog
"qtdeclarative"
digikam.general: Loaded Qt standard translations "en_US" from catalog
"qtquickcontrols"
digikam.general: Loaded Qt standard translations "en_US" from catalog
"qtquickcontrols2"
digikam.general: Loaded Qt standard translations "en_US" from catalog
"qtmultimedia"
digikam.general: Loaded Qt standard translations "en_US" from catalog
"qtwebengine"
digikam.general: Loaded Qt standard translations "en_US" from catalog
"qtxmlpatterns"
digikam.general: Loaded Qt ECM translations "en" from catalog "kcoreaddons5_qt"
digikam.general: Loaded Qt ECM translations "en" from catalog
"kwidgetsaddons5_qt"
digikam.general: Switch to widget style:  "Fusion"
digikam.general: AlbumWatch is disabled
digikam.general: Database Parameters:
   Type:  "QSQLITE"
   DB Core Name: 
"/Users/markstoughton/Pictures/Digikam/digikam4.db"
   DB Thumbs Name:   
"/Users/markstoughton/Pictures/Digikam/thumbnails-digikam.db"
   DB Face Name: 
"/Users/markstoughton/Pictures/Digikam/recognition.db"
   DB Similarity Name:   
"/Users/markstoughton/Pictures/Digikam/similarity.db"
   Connect Options:   ""
   Host Name: ""
   Host port: -1
   Internal Server:   false
   Internal Server Path:  ""
   Internal Server Admin Cmd: ""
   Internal Server Serv Cmd:  ""
   Internal Server Init Cmd:  ""
   Username:  ""
   Password:  ""

digikam.dbengine: Loading SQL code from config file
"/Applications/digiKam.org/digikam.app/Contents/Resources/digikam/database/dbconfig.xml"
digikam.dbengine: Checking XML version ID => expected:  3  found:  3
digikam.coredb: Core database: running schema update
digikam.coredb: Core database: have a structure version  15
digikam.coredb: Core database: makeUpdates  15  to  15
digikam.database: Creating new Location  "/"  uuid 
"networkshareid:?mountpath=/Volumes/OurPhotos/Photo Library/2021"
digikam.database: Creating new Location  "/"  uuid 
"networkshareid:?mountpath=/Volumes/OurPhotos/Photo Library/2020"
digikam.database: Creating new Location  "/"  uuid 
"networkshareid:?mountpath=/Volumes/OurPhotos/Photo Library/2019"
digikam.database: Creating new Location  "/"  uuid 
"networkshareid:?mountpath=/Volumes/OurPhotos/Photo Library/2018"
digikam.database: Creating new Location  "/"  uuid 
"networkshareid:?mountpath=/Volumes/OurPhotos/Photo Library/2017"
digikam.database: Creating new Location  "/"  uuid 
"networkshareid:?mountpath=/Volumes/OurPhotos/Photo Library/2016"
digikam.database: Creating new Location  "/"  uuid 
"networkshareid:?mountpath=/Volumes/OurPhotos/Photo Library/2015"
digikam.database: Creating new Location  "/"  uuid 
"networkshareid:?mountpath=/Volumes/OurPhotos/Photo Library/2014"
digikam.database: location for  "/Volumes/OurPhotos/Photo Library/2021"  is
available  true
digikam.database: location for  "/Volumes/OurPhotos/Photo Library/2020"  is
available  true
digikam.database: location for  "/Volumes/OurPhotos/Photo Library/2019"  is
available  true
digikam.database: location for  "/Volumes/OurPhotos/Photo Library/2018"  is
available  true
digikam.database: location for  "/Volumes/OurPhotos/Photo Library/2017"  is
available  true
digikam.database: location for  "/Volumes/OurPhotos/Photo Library/2016"  is
available  true
digikam.database: location for  "/Volumes/OurPhotos/Photo Library/2015"  is
available  true
digikam

[digikam] [Bug 405235] Slow startup and new file scan on low latency networks

2022-02-10 Thread Mark Stoughton
https://bugs.kde.org/show_bug.cgi?id=405235

--- Comment #42 from Mark Stoughton  ---
I unchecked the Settings-Collections I unchecked the box that says "Monitor the
ablbums for external changes(requires restart).  Is that the right one?  If
not, please direct me to the right one.  Thanks.  

P.S. The log that I sent was before I unchecked that box. I'm rerunning it now
and will post when it's done

-- 
You are receiving this mail because:
You are watching all bug changes.

[digikam] [Bug 405235] Slow startup and new file scan on low latency networks

2022-02-10 Thread Mark Stoughton
https://bugs.kde.org/show_bug.cgi?id=405235

--- Comment #38 from Mark Stoughton  ---
After some use, the album switching got down to. fairly repeatable three
minutes, regardless of the number of photos in the album

-- 
You are receiving this mail because:
You are watching all bug changes.

[digikam] [Bug 405235] Slow startup and new file scan on low latency networks

2022-02-10 Thread Mark Stoughton
https://bugs.kde.org/show_bug.cgi?id=405235

--- Comment #37 from Mark Stoughton  ---
I deactivated the monitor external changes and restarted.  I saw no difference
in either startup time (40 minutes) or album switching (15 minutes).

-- 
You are receiving this mail because:
You are watching all bug changes.

[digikam] [Bug 405235] Slow startup and new file scan on low latency networks

2022-02-10 Thread Mark Stoughton
https://bugs.kde.org/show_bug.cgi?id=405235

--- Comment #36 from Mark Stoughton  ---
It was activated.  I'll try again after deactivating it

(In reply to Maik Qualmann from comment #34)
> I assume you have activated the option to monitor for external changes in
> the digiKam settings under Collection. This option must be deactivated under
> MacOS, as it causes the albums to load extremely slowly.
> 
> Maik

-- 
You are receiving this mail because:
You are watching all bug changes.

[digikam] [Bug 405235] Slow startup and new file scan on low latency networks

2022-02-10 Thread Mark Stoughton
https://bugs.kde.org/show_bug.cgi?id=405235

--- Comment #35 from Mark Stoughton  ---
I ran the debug session.  I've annotated it because digicam stopped for several
minutes in different places along the way, and the log does not show the
pauses.

It took 41 minutes from start to finish

markstoughton@Marks-Mac-mini ~ % export QT_LOGGING_RULES="digikam*=true"
/Applications/digiKam.org/digikam.app/Contents/MacOS/digikam
digikam.widgets: Breeze icons resource file found
digikam.widgets: Breeze-dark icons resource file found
digikam.general: Qt standard translations removed: 0
digikam.general: Qt standard translations path:
"/Applications/digiKam.org/digikam.app/Contents/Resources/translations"
digikam.general: Loaded Qt standard translations "en_US" from catalog "qt"
digikam.general: Loaded Qt standard translations "en_US" from catalog "qtbase"
digikam.general: Loaded Qt standard translations "en_US" from catalog "qt_help"
digikam.general: Loaded Qt standard translations "en_US" from catalog
"qtdeclarative"
digikam.general: Loaded Qt standard translations "en_US" from catalog
"qtquickcontrols"
digikam.general: Loaded Qt standard translations "en_US" from catalog
"qtquickcontrols2"
digikam.general: Loaded Qt standard translations "en_US" from catalog
"qtmultimedia"
digikam.general: Loaded Qt standard translations "en_US" from catalog
"qtwebengine"
digikam.general: Loaded Qt standard translations "en_US" from catalog
"qtxmlpatterns"
digikam.general: Loaded Qt ECM translations "en" from catalog "kcoreaddons5_qt"
digikam.general: Loaded Qt ECM translations "en" from catalog
"kwidgetsaddons5_qt"
digikam.general: Switch to widget style:  "Fusion"
digikam.general: AlbumWatch use QFileSystemWatcher
digikam.general: Database Parameters:
   Type:  "QSQLITE"
   DB Core Name: 
"/Users/markstoughton/Pictures/Digikam/digikam4.db"
   DB Thumbs Name:   
"/Users/markstoughton/Pictures/Digikam/thumbnails-digikam.db"
   DB Face Name: 
"/Users/markstoughton/Pictures/Digikam/recognition.db"
   DB Similarity Name:   
"/Users/markstoughton/Pictures/Digikam/similarity.db"
   Connect Options:   ""
   Host Name: ""
   Host port: -1
   Internal Server:   false
   Internal Server Path:  ""
   Internal Server Admin Cmd: ""
   Internal Server Serv Cmd:  ""
   Internal Server Init Cmd:  ""
   Username:  ""
   Password:  ""

digikam.dbengine: Loading SQL code from config file
"/Applications/digiKam.org/digikam.app/Contents/Resources/digikam/database/dbconfig.xml"
digikam.dbengine: Checking XML version ID => expected:  3  found:  3
digikam.coredb: Core database: running schema update
digikam.coredb: Core database: have a structure version  15
digikam.coredb: Core database: makeUpdates  15  to  15
digikam.database: Creating new Location  "/"  uuid 
"networkshareid:?mountpath=/Volumes/OurPhotos/Photo Library/2021"
digikam.database: Creating new Location  "/"  uuid 
"networkshareid:?mountpath=/Volumes/OurPhotos/Photo Library/2020"
digikam.database: Creating new Location  "/"  uuid 
"networkshareid:?mountpath=/Volumes/OurPhotos/Photo Library/2019"
digikam.database: Creating new Location  "/"  uuid 
"networkshareid:?mountpath=/Volumes/OurPhotos/Photo Library/2018"
digikam.database: Creating new Location  "/"  uuid 
"networkshareid:?mountpath=/Volumes/OurPhotos/Photo Library/2017"
digikam.database: Creating new Location  "/"  uuid 
"networkshareid:?mountpath=/Volumes/OurPhotos/Photo Library/2016"
digikam.database: Creating new Location  "/"  uuid 
"networkshareid:?mountpath=/Volumes/OurPhotos/Photo Library/2015"
digikam.database: Creating new Location  "/"  uuid 
"networkshareid:?mountpath=/Volumes/OurPhotos/Photo Library/2014"
digikam.database: location for  "/Volumes/OurPhotos/Photo Library/2021"  is
available  true
digikam.database: location for  "/Volumes/OurPhotos/Photo Library/2020"  is
available  true
digikam.database: location for  "/Volumes/OurPhotos/Photo Library/2019"  is
available  true
digikam.database: location for  "/Volumes/OurPhotos/Photo Library/2018"  is
available  true
digikam.database: location for  "/Volumes/OurPhotos/Photo Library/2017"  is
available  true
digikam.database: location for  "/Volumes/OurPhotos/Photo Library/2016"  is
available  true
digikam.database: locat

[digikam] [Bug 405235] Slow startup and new file scan on low latency networks

2022-02-10 Thread Mark Stoughton
https://bugs.kde.org/show_bug.cgi?id=405235

--- Comment #33 from Mark Stoughton  ---
sorry for the slow reply.  It's 10 minutes after the smash screen disappears. 
The app shows up as non-responsive in Task manager in Windows or in Force Quit
on the Mac.

-- 
You are receiving this mail because:
You are watching all bug changes.

[digikam] [Bug 405235] Slow startup and new file scan on low latency networks

2022-01-22 Thread Mark Stoughton
https://bugs.kde.org/show_bug.cgi?id=405235

--- Comment #32 from Mark Stoughton  ---
Hi,

It's from the time the splash screen disappears until the interface shows up.
During that time there is no indicator that Digikam is even running at all. The
photos were already imported. It shows up as "not responding" in my Mac Force
Quit Applications box or Windows Task Manager.  It also takes several minutes
of appearing to be frozen when I switch between folders.  As long as I'm
working in one folder, the performance is slow but useable.  The minute I
switch folders, I have to wait another 10 minutes or so for Digikam to wake up
again.

Latency on my network to the SAN is 0.5ms. Throughput is about 100MB/s. I don't
have tools to test latency installed right now.

I also have ACDSee installed on both the Mac and PC, and neither of those
exhibits this issue.


From: MarcP 
Sent: Saturday, January 22, 2022 1:23 PM
To: geoph...@hotmail.com 
Subject: [digikam] [Bug 405235] Slow startup and new file scan on low latency
networks

https://bugs.kde.org/show_bug.cgi?id=405235

--- Comment #31 from MarcP  ---
Hi Mark, you mean 10-minutes since the digikam splash disappears and the
interface shows up,  or until the progress bar reaches 100%? Are the pictures
in your library already imported or it's the first scan?

I'm now in a high latency network (~120ms to my NAS), and it's not nearly as
bad as it was with v6.x versions. The interface shows up rather quickly and
it's true that the initial scan can take a while, but if there aren't many new
pictures it's just a matter of minutes.

--
You are receiving this mail because:
You are on the CC list for the bug.

-- 
You are receiving this mail because:
You are watching all bug changes.

[digikam] [Bug 405235] Slow startup and new file scan on low latency networks

2022-01-22 Thread Mark Stoughton
https://bugs.kde.org/show_bug.cgi?id=405235

Mark Stoughton  changed:

   What|Removed |Added

Version|7.4.0   |7.5.0
 CC||geoph...@hotmail.com

--- Comment #29 from Mark Stoughton  ---
I am seeing this on the 7.5.0 install.  I have my photos stored on a Synology
NAS in a shared set of folders. They contain approximately 50,000 photos. It
takes 10-20 minutes for DigiKam to start up. The splash screen shows for about
20 seconds, then disappears. In fact, I started Digikam, went and got a cup of
coffee, logged into the bugs site, searched for the problem, found this bug and
am making this entry while waiting.  I'm seeing this behavior on both my Mac
and Windows installs.

I'm willing to do whatever logging that you require, and to run experiments if
you want.

-- 
You are receiving this mail because:
You are watching all bug changes.