[digikam] [Bug 478554] Very slow maintenance process with mariadb
https://bugs.kde.org/show_bug.cgi?id=478554 --- Comment #13 from Maik Qualmann --- I have a comparison here with the AppImage digiKam-8.0.0 (older AppImages unfortunately no longer work here) and my current Qt6 version shows no difference when creating thumbnails and fingerprints. Maik -- You are receiving this mail because: You are watching all bug changes.
[digikam] [Bug 478554] Very slow maintenance process with mariadb
https://bugs.kde.org/show_bug.cgi?id=478554 --- Comment #12 from Maik Qualmann --- (In reply to maderios from comment #10) > I forgot to say i found other errors with an other digikam user account. I > have many messages like this: > digikam.metaengine: Exiv2 ( 2 ) : IPTC dataset Iptc.0x001c.0x0002 has > invalid size 16640; skipped. > digikam.metaengine: Exiv2 ( 2 ) : Failed to decode IPTC metadata. > I don't use IPTC Exiv2 throws an exception because it detected unresolvable problems/errors in the images. In these cases we then read the metadata using ExifTool, which is a little slower. But it's still better than having incorrect metadata or no metadata at all. Maik -- You are receiving this mail because: You are watching all bug changes.
[digikam] [Bug 478554] Very slow maintenance process with mariadb
https://bugs.kde.org/show_bug.cgi?id=478554 --- Comment #11 from Maik Qualmann --- Git commit bc4a263fefbbf58cffd5ad55d50387e4acd9a4e6 by Maik Qualmann. Committed on 16/12/2023 at 17:32. Pushed by mqualmann into branch 'master'. prevent double processed images when tags are also selected M +14 -2core/utilities/maintenance/autotagsassignment.cpp M +14 -2core/utilities/maintenance/fingerprintsgenerator.cpp M +14 -2core/utilities/maintenance/imagequalitysorter.cpp M +15 -2core/utilities/maintenance/thumbsgenerator.cpp https://invent.kde.org/graphics/digikam/-/commit/bc4a263fefbbf58cffd5ad55d50387e4acd9a4e6 -- You are receiving this mail because: You are watching all bug changes.
[digikam] [Bug 478554] Very slow maintenance process with mariadb
https://bugs.kde.org/show_bug.cgi?id=478554 --- Comment #10 from maderios --- I forgot to say i found other errors with an other digikam user account. I have many messages like this: digikam.metaengine: Exiv2 ( 2 ) : IPTC dataset Iptc.0x001c.0x0002 has invalid size 16640; skipped. digikam.metaengine: Exiv2 ( 2 ) : Failed to decode IPTC metadata. I don't use IPTC -- You are receiving this mail because: You are watching all bug changes.
[digikam] [Bug 478554] Very slow maintenance process with mariadb
https://bugs.kde.org/show_bug.cgi?id=478554 --- Comment #9 from maderios --- (In reply to Maik Qualmann from comment #8) > In fact, ExifTool has to intervene a few times to read metadata, with times > between 60-500-1100ms per image. Not really "A few times": if we say it takes only 200 ms per image, 20 000 images will take 4000 s to scan, more than one hour. I think it's much more because I have many big files like png, xcf, .nef, with many sidecars/xmp tags (write to XMP sidecar only) > But what is noticeable is that images are processed twice. You also selected > the “All Tags” option. Depending on the tag, this can actually lead to > images being processed twice. I will change this situation. > > Maik -- You are receiving this mail because: You are watching all bug changes.
[digikam] [Bug 478554] Very slow maintenance process with mariadb
https://bugs.kde.org/show_bug.cgi?id=478554 --- Comment #8 from Maik Qualmann --- In fact, ExifTool has to intervene a few times to read metadata, with times between 60-500-1100ms per image. But what is noticeable is that images are processed twice. You also selected the “All Tags” option. Depending on the tag, this can actually lead to images being processed twice. I will change this situation. Maik -- You are receiving this mail because: You are watching all bug changes.
[digikam] [Bug 478554] Very slow maintenance process with mariadb
https://bugs.kde.org/show_bug.cgi?id=478554 --- Comment #7 from maderios --- Created attachment 164223 --> https://bugs.kde.org/attachment.cgi?id=164223=edit rescan debug log See maintenance rescan debug log (about 400 images) I use a fresh account for these tests. -- You are receiving this mail because: You are watching all bug changes.
[digikam] [Bug 478554] Very slow maintenance process with mariadb
https://bugs.kde.org/show_bug.cgi?id=478554 --- Comment #6 from Maik Qualmann --- Yes, they may be scanned with ExifTool, because PNG is definitely a candidate where Exiv2 fails. A debug log from the terminal can quickly clarify things. Maik -- You are receiving this mail because: You are watching all bug changes.
[digikam] [Bug 478554] Very slow maintenance process with mariadb
https://bugs.kde.org/show_bug.cgi?id=478554 --- Comment #5 from maderios --- (In reply to Maik Qualmann from comment #1) > I haven't looked at the valgrind file yet, I'll do that late this evening. > But just a thought, ExifTool has been on board since digiKam-8.0.0. So if > you have a lot of files (videos or similar) or Exiv2 has an exception with > your images, ExifTool will automatically read the metadata. This of course > slows down the process. I definitely can't reproduce a drop in speed myself. > > Maik Did you try to reproduce it with big files, like png? They slow down process. It takes more one second to scan some of them. -- You are receiving this mail because: You are watching all bug changes.
[digikam] [Bug 478554] Very slow maintenance process with mariadb
https://bugs.kde.org/show_bug.cgi?id=478554 --- Comment #4 from maderios --- (In reply to Maik Qualmann from comment #3) > From other bug reports I know that you also have XCF files. > > Maik Yes, two XCF files in directory/album i used to test with valgrind today. They don't slow down but I deleted them. In an other account, I have no XCF files but big png files. I see scanning these big png takes about from 300 ms to 1100 ms. -- You are receiving this mail because: You are watching all bug changes.
[digikam] [Bug 478554] Very slow maintenance process with mariadb
https://bugs.kde.org/show_bug.cgi?id=478554 --- Comment #3 from Maik Qualmann --- >From other bug reports I know that you also have XCF files. Maik -- You are receiving this mail because: You are watching all bug changes.
[digikam] [Bug 478554] Very slow maintenance process with mariadb
https://bugs.kde.org/show_bug.cgi?id=478554 --- Comment #2 from maderios --- I have only 2 video files (2 .mkv) in my test directory/album, other files are .jpeg and .png (225 files) -- You are receiving this mail because: You are watching all bug changes.
[digikam] [Bug 478554] Very slow maintenance process with mariadb
https://bugs.kde.org/show_bug.cgi?id=478554 Maik Qualmann changed: What|Removed |Added CC||metzping...@gmail.com --- Comment #1 from Maik Qualmann --- I haven't looked at the valgrind file yet, I'll do that late this evening. But just a thought, ExifTool has been on board since digiKam-8.0.0. So if you have a lot of files (videos or similar) or Exiv2 has an exception with your images, ExifTool will automatically read the metadata. This of course slows down the process. I definitely can't reproduce a drop in speed myself. Maik -- You are receiving this mail because: You are watching all bug changes.
