[digikam] [Bug 432369] Wrong time for mp4 in _almost_ all places, metadata differs from filesystem/other apps. DB issues?
https://bugs.kde.org/show_bug.cgi?id=432369 --- Comment #13 from Maik Qualmann --- Git commit fea2a72f7f1bb6957638b741489b226537e50e92 by Maik Qualmann. Committed on 03/10/2022 at 06:12. Pushed by mqualmann into branch 'master'. merge metadata from FFMpeg and ExifTool for video Related: bug 459898, bug 459928 M +1-1core/libs/metadataengine/dmetadata/dmetadata.h M +9-2core/libs/metadataengine/dmetadata/dmetadata_exiftool.cpp M +11 -4core/libs/metadataengine/dmetadata/dmetadata_fileio.cpp M +59 -0core/libs/metadataengine/engine/metaengine.cpp M +7-0core/libs/metadataengine/engine/metaengine.h https://invent.kde.org/graphics/digikam/commit/fea2a72f7f1bb6957638b741489b226537e50e92 -- You are receiving this mail because: You are watching all bug changes.
[digikam] [Bug 432369] Wrong time for mp4 in _almost_ all places, metadata differs from filesystem/other apps. DB issues?
https://bugs.kde.org/show_bug.cgi?id=432369 Maik Qualmann changed: What|Removed |Added Resolution|--- |FIXED Status|REPORTED|RESOLVED Version Fixed In||8.0.0 --- Comment #12 from Maik Qualmann --- ExifTool is now the primary metadata reader for video files in digiKam-8.0.0. Maik -- You are receiving this mail because: You are watching all bug changes.
[digikam] [Bug 432369] Wrong time for mp4 in _almost_ all places, metadata differs from filesystem/other apps. DB issues?
https://bugs.kde.org/show_bug.cgi?id=432369 --- Comment #11 from pmgy...@gmail.com --- (In reply to Maik Qualmann from comment #10) > *** Bug 447371 has been marked as a duplicate of this bug. *** Hi, I am the reporter of this duplicated bug report. So I am just adding my findings here. After some investigation, I noticed that the files with issue does not contain "QuickTime:AndroidVersion" tag. After adding this tag with a value, the time is correctly displayed in digikam. So apparently, these files were not treated as videos from Android by digiKam. It does have a "QuickTime:AndroidCaptureFps" tag though. I am using gcam port by BSG. I wonder if the algorithm of identifying Android files can be a bit more flexible (e.g. scan metadata for tag names that start with Android?) I'm just saying, I know it will impact performance and may trigger potential issues that I am not aware of. Thanks again. At least I have a simple solution for my case. -- You are receiving this mail because: You are watching all bug changes.
[digikam] [Bug 432369] Wrong time for mp4 in _almost_ all places, metadata differs from filesystem/other apps. DB issues?
https://bugs.kde.org/show_bug.cgi?id=432369 Maik Qualmann changed: What|Removed |Added CC||pmgy...@gmail.com --- Comment #10 from Maik Qualmann --- *** Bug 447371 has been marked as a duplicate of this bug. *** -- You are receiving this mail because: You are watching all bug changes.
[digikam] [Bug 432369] Wrong time for mp4 in _almost_ all places, metadata differs from filesystem/other apps. DB issues?
https://bugs.kde.org/show_bug.cgi?id=432369 --- Comment #9 from Thomas --- Just to add for future readers: I solved this by first adjusting my dates to what I want them to be, then creating sidecars for all my phone video files (xmp files), and added metadata for xmp:CreateDate and xmp:ModifyDate using exiftool. Command, if it helps anyone, that I use for creating sidecars is (\ is just for splitting command over multiple lines): exiftool -P -progress -ext mp4 \ -tagsfromfile @ \ -filemodifydate \ -createdate \ -modifydate \ -datetimeoriginal \ -srcfile %d%f.xmp \ This creates .xmp for all mp4 files in . Perhaps this can be simplified, still figuring exiftool out, but it works for me. Apparently Digikam reads these sidecars instead of the files if they exist, so it ignores the dates of the files and sorts them accordingly. A bit annoying to have to do so, but I will most likely tag most of my photos anyway, so a sidecar would be created either way. Another annoying thing is it _ONLY_ reads the metadata from sidecars, not just the data present in the sidecar, and any other data from the file. This would be preferable from my point of view. Since I only added the two tags to my files, the metadata properties look like this: https://i.postimg.cc/T3MrXdz6/sidecar-exists.png While any file not having a sidecar is shown like this: https://i.postimg.cc/mgY7Zx4F/sidecar-non-existant.png Not even the duration is shown for my files with sidecars. This should, IMO, be read from the file if it is not in the sidecar. Perhaps there is a reason for this behavior? Might look into this later. For now it works. This can probably be done from within digikam as well, perhaps using the batch process or metadata editor to shift the time and add an xmp. But I want this automated when importing my files from my cameras/devices (and I do not use digikam to import my files). -- You are receiving this mail because: You are watching all bug changes.
[digikam] [Bug 432369] Wrong time for mp4 in _almost_ all places, metadata differs from filesystem/other apps. DB issues?
https://bugs.kde.org/show_bug.cgi?id=432369 --- Comment #8 from Thomas --- The problem is I am trying to fix many years of forgetting to change the DST setting on different cameras from time to time. So, I thought that if I identify the correct time and set all dates to that time it would show all files in order. But it appears that I will have to set my jpegs/raws correctly, and videos to different times depending on device, to have them line up in digikam. (In reply to Maik Qualmann from comment #7) > Your MOV even included 13:59 in the > filename, so it will be the time that was set. So digikam uses that information even though I specify to sort by createdate? I think I didn't read that right. I have edited the filename to match the metadata, the point is that all dates match, and so does digikam. For mp4, not so much. But as you say, all manufacturers do it differently. I will find a workaround somehow, unless I can make digikam always respect my set createdate. Judging from your answers, not possible atm. Maybe in the future there will be an option. Thank you for looking at this though! -- You are receiving this mail because: You are watching all bug changes.
[digikam] [Bug 432369] Wrong time for mp4 in _almost_ all places, metadata differs from filesystem/other apps. DB issues?
https://bugs.kde.org/show_bug.cgi?id=432369 --- Comment #7 from Maik Qualmann --- Hmm, I don't really understand your problem at the moment. DigiKam shows the correct time for video and image. Your MOV even included 13:59 in the filename, so it will be the time that was set. Image and videos from SLR cameras save their time information in local time. With phones, the images are saved in local time, with videos, however, in UTC. That is almost the rule. But there are exceptions, manufacturers save the local time in videos, but also set the UTC flag or vice versa. There is currently no final solution to this. DigiKam tries to determine the local time as best it can. Maik -- You are receiving this mail because: You are watching all bug changes.
[digikam] [Bug 432369] Wrong time for mp4 in _almost_ all places, metadata differs from filesystem/other apps. DB issues?
https://bugs.kde.org/show_bug.cgi?id=432369 --- Comment #6 from Thomas --- My bad for being unclear, yes digikam shows it correctly. I guess what bothers me is the fact that images and video are not treated the same. Created two test files, and the data is this: IMG_20210201_182358.jpg FileName: IMG_20210201_182358.jpg FileModifyDate : 2021:02:01 18:29:53+01:00 ModifyDate : 2021:02:01 18:23:58 DateTimeOriginal: 2021:02:01 18:23:58 CreateDate : 2021:02:01 18:23:58 VID_20210201_182353.mp4 FileName: VID_20210201_182353.mp4 FileModifyDate : 2021:02:01 18:30:01+01:00 ModifyDate : 2021:02:01 17:23:53 CreateDate : 2021:02:01 17:23:53 2 image files read And digikam shows them both correctly, the time is 18.23 just like the filenames say. But the metadata does not reflect this. I don't like the fact that it adds 1 hour to the mp4, but not the jpg... Especially when it is not consistent. If I use my Em10 it will not respect my timezone and just use the createdate for my .mov files. One file has this data, and with the above logic it should add 2hrs to it (+1+DST): FileName: 20200519_135904-VID-EM10_MkII.movFileModifyDate : 2020:05:19 13:59:04+02:00 ModifyDate : 2020:05:19 13:59:04 CreateDate : 2020:05:19 13:59:04 But it does not. It shows 13:59. Makes handling files harder. I understand that it is to make the files show correctly across devices, but a way to turn off the behavior would be nice, when metadata is handled elsewhere. Or do all apps do it this way? Now I have to keep track of how metadata is shown, wheter the file is from a phone or a SLR etc.. I have plenty of devices from 2005 and forward, that seem to use data in different ways. I thought that if I set all dates to the taken date, image or video, device etc doesn't matter, that it would show the date "By creation date" for everything. Guess not. :( -- You are receiving this mail because: You are watching all bug changes.
[digikam] [Bug 432369] Wrong time for mp4 in _almost_ all places, metadata differs from filesystem/other apps. DB issues?
https://bugs.kde.org/show_bug.cgi?id=432369 --- Comment #5 from Maik Qualmann --- I don't see any errors in the video sample, I'm guessing you're in the same time zone as me +1. The clock shows 8:58, digiKam also shows 8:58. The filename also says 8:58. The metadata in the image is "2021-02-01T07:58:01.00Z", with the Z at the end stands for UTC. Since we find the string "com.android.version" in the metadata, we assume the correct UTC date and do not remove the "Z" when converting the date to a Qt local time. Everything is correct for me. The camera manufacturers always store local time for images. Only in the current EXIF standard is there new metadata that will contain the time zone used when taking images. Maik -- You are receiving this mail because: You are watching all bug changes.
[digikam] [Bug 432369] Wrong time for mp4 in _almost_ all places, metadata differs from filesystem/other apps. DB issues?
https://bugs.kde.org/show_bug.cgi?id=432369 --- Comment #4 from Thomas --- I have tested a bit more. I transferred a fresh file from my phone and imported into digikam. Exiftool reports: [tomteverkstan] ~/digibug (13:44) (digitest:2) $ exiftool -time:all -G0:1 VID_20210201_085801.mp4 [ExifTool] Now : 2021:02:01 13:48:20+01:00 [File:System] File Modification Date/Time : 2021:02:01 09:03:52+01:00 [File:System] File Access Date/Time : 2021:02:01 09:04:44+01:00 [File:System] File Inode Change Date/Time : 2021:02:01 09:04:22+01:00 [QuickTime] Create Date : 2021:02:01 07:58:01 [QuickTime] Modify Date : 2021:02:01 07:58:01 [QuickTime:Track1] Track Create Date: 2021:02:01 07:58:01 [QuickTime:Track1] Track Modify Date: 2021:02:01 07:58:01 [QuickTime:Track1] Media Create Date: 2021:02:01 07:58:01 [QuickTime:Track1] Media Modify Date: 2021:02:01 07:58:01 [QuickTime:Track2] Track Create Date: 2021:02:01 07:58:01 [QuickTime:Track2] Track Modify Date: 2021:02:01 07:58:01 [QuickTime:Track2] Media Create Date: 2021:02:01 07:58:01 [QuickTime:Track2] Media Modify Date: 2021:02:01 07:58:01 The correct time is 08:58, and that is what digikam says. So, it appears files are recorded in UTC time, and digikam adds +1h based on my system timezone, like you said. But, why not for images then? This image: 20201003_163739-IMG-P20.jpg [ExifTool] Now : 2021:02:01 13:54:02+01:00 [File:System] File Modification Date/Time : 2020:10:03 16:37:39+02:00 [File:System] File Access Date/Time : 2021:02:01 13:44:12+01:00 [File:System] File Inode Change Date/Time : 2021:02:01 13:44:08+01:00 [EXIF:IFD0] Modify Date : 2020:10:03 16:37:39 [EXIF:ExifIFD] Date/Time Original : 2020:10:03 16:37:39 [EXIF:ExifIFD] Create Date : 2020:10:03 16:37:39 [EXIF:ExifIFD] Sub Sec Time: 220008 [EXIF:ExifIFD] Sub Sec Time Original : 220008 [EXIF:ExifIFD] Sub Sec Time Digitized : 220008 [Composite] Create Date : 2020:10:03 16:37:39.220008 [Composite] Date/Time Original : 2020:10:03 16:37:39.220008 [Composite] Modify Date : 2020:10:03 16:37:39.220008 is displayed with the correct time. 16:37. Not 17:37. So, why is +1 hour not added there? Are images not treated as UTC? I have the setting "View -> Sort Items -> By Creation Date". This led me to believe that it would use either EXIF:Create Date, or QuickTime:Create Date for images and movies respectively. But it appears it also edits the date for video files based on, like you say, if it is a phone. This seems odd to me given the setting. Can this be toggled somewhere? I would much rather just set all my file dates correctly, and adjust digikam accordingly, than the other way around. Especially when videos and images are treated differently. I uploaded my test file here: https://drive.google.com/file/d/1NvMmgS1JWxdjaPPhICXxJVZ_2Fqxh_8t/view?usp=sharing -- You are receiving this mail because: You are watching all bug changes.
[digikam] [Bug 432369] Wrong time for mp4 in _almost_ all places, metadata differs from filesystem/other apps. DB issues?
https://bugs.kde.org/show_bug.cgi?id=432369 --- Comment #3 from Thomas --- (In reply to Maik Qualmann from comment #1) > Well, I think the time the digiKam recognizes will be correct based on the > local time. > When we added reading video metadata, users provided us with a large number > of videos from various devices. Including documentation of when the video > was recorded, what digiKam is currently displaying and what users expect. > Unfortunately, the manufacturers of the devices store the video metadata for > time information differently. Some in local time, others in UTC with no time > zone, others with time zone, etc. > If we recognize an Android or Apple device based on the metadata, we assume > that the time in UTC and the time zones are correct and convert them. The > video files should be sorted in the time history together with the images in > the correct order. This was very important to the users. We should even add > the running time of the video, but we did not implement this because it is > not the standard for cameras and long exposure either. > > Maik If that was the case, then all my videos would be off since 90% are from the same time zone, only DST changes. The images as well. And, if the time was using the TZ of the filemodifydate, then all videos should be off, and all images as well? Not just the mp4s? What fields in an mp4 does it rely on? I should be able to find that data in the file, shouldn't I? I will try to reproduce it with a new file and report back, both my Huawei P20 and the wifes Sony XZ2C show the same issues. My E-m10 Mk II seems correct, but it records mov files. -- You are receiving this mail because: You are watching all bug changes.
[digikam] [Bug 432369] Wrong time for mp4 in _almost_ all places, metadata differs from filesystem/other apps. DB issues?
https://bugs.kde.org/show_bug.cgi?id=432369 --- Comment #2 from Maik Qualmann --- In order to investigate your problem, we need the recording of a test video, preferably a clock with the current local time. Maik -- You are receiving this mail because: You are watching all bug changes.
[digikam] [Bug 432369] Wrong time for mp4 in _almost_ all places, metadata differs from filesystem/other apps. DB issues?
https://bugs.kde.org/show_bug.cgi?id=432369 Maik Qualmann changed: What|Removed |Added CC||metzping...@gmail.com --- Comment #1 from Maik Qualmann --- Well, I think the time the digiKam recognizes will be correct based on the local time. When we added reading video metadata, users provided us with a large number of videos from various devices. Including documentation of when the video was recorded, what digiKam is currently displaying and what users expect. Unfortunately, the manufacturers of the devices store the video metadata for time information differently. Some in local time, others in UTC with no time zone, others with time zone, etc. If we recognize an Android or Apple device based on the metadata, we assume that the time in UTC and the time zones are correct and convert them. The video files should be sorted in the time history together with the images in the correct order. This was very important to the users. We should even add the running time of the video, but we did not implement this because it is not the standard for cameras and long exposure either. Maik -- You are receiving this mail because: You are watching all bug changes.
