No problem, Guillermo, and thanks for your help. It does seem a big issue, particularly looking at the db, and I do hope some more developer eyes see the post. In meantime, do you know which files I would find the relevant 'create duplicate' functionality in the source code so I can dig into the issue some more - I know nothing about how the dt code is structured?
Regards Dusenberg On 27/07/2023 11:46, Guillermo Rozas wrote:
Hi, sorry for the late reply, I was a bit busy. Yes, pixls.us <http://pixls.us> will probably give you more 'developer eyes' to check the problem. It's probably a big, so I would be prepared to file one on GitHub. Best regards, Guillermo On Thu, Jul 27, 2023, 06:21 Dusenberg <dusenb...@gmx.co.uk> wrote: I have posted this on pixls.us <http://pixls.us> now it is back up. Dusenberg On 26/07/2023 20:39, Dusenberg wrote:Guillermo Since my last post, I have extracted data from the dt db image table for the image and versions concerned from three backup instances of the dt database going back 3 years: a) 2020-12-14 - pre-dt3.4.0, closest I can get to the original shot date b) 2023-07-22 - dt4.2.1, before new duplicate created c) 2023-07-24 - dt4.2.1, after new duplicate created showing duplicate version '****' Hope this sheds more light on the issue. pre-dt3.4.0-library.db from backup archive 2020-12-14 ------------------------------------------------------ select key, value from db_info; key value version 30 select id, group_id, film_id,filename,version,max_version from images where filename = "20200325_BonallyWoods_NIK_1413.NEF" order by id; id group_id film_id filename version max_version 3574 3574 135 20200325_BonallyWoods_NIK_1413.NEF 0 7 3578 3574 135 20200325_BonallyWoods_NIK_1413.NEF 1 7 3582 3574 135 20200325_BonallyWoods_NIK_1413.NEF 2 7 3583 3574 135 20200325_BonallyWoods_NIK_1413.NEF 3 7 3789 3574 135 20200325_BonallyWoods_NIK_1413.NEF 4 7 3790 3574 135 20200325_BonallyWoods_NIK_1413.NEF 5 7 3791 3574 135 20200325_BonallyWoods_NIK_1413.NEF 6 7 3792 3574 135 20200325_BonallyWoods_NIK_1413.NEF 7 7 library.db-snp-20230722155434 from backup archive BEFORE new duplicate created ------------------------------------------------------------------------------ select key, value from db_info; key value version 37 select id, group_id, film_id,filename,version,max_version from images where filename = "20200325_BonallyWoods_NIK_1413.NEF" order by id; id group_id film_id filename version max_version 3574 3574 135 20200325_BonallyWoods_NIK_1413.NEF 0 7 3578 3574 135 20200325_BonallyWoods_NIK_1413.NEF 1 7 3582 3574 135 20200325_BonallyWoods_NIK_1413.NEF 2 7 3583 3574 135 20200325_BonallyWoods_NIK_1413.NEF 3 7 3789 3574 135 20200325_BonallyWoods_NIK_1413.NEF 4 7 3790 3574 135 20200325_BonallyWoods_NIK_1413.NEF 5 7 3791 3574 135 20200325_BonallyWoods_NIK_1413.NEF 6 7 3792 3574 135 20200325_BonallyWoods_NIK_1413.NEF 7 7 library.db-snp-20230724163328 from current backup AFTER new duplicate created ------------------------------------------------------------------------------- select key, value from db_info; key value version 37 select id, group_id, film_id,filename,version,max_version from images where filename = "20200325_BonallyWoods_NIK_1413.NEF" order by id; id group_id film_id filename version max_version 3574 3574 135 20200325_BonallyWoods_NIK_1413.NEF 0 0 3578 3574 135 20200325_BonallyWoods_NIK_1413.NEF 1 3 3582 3574 135 20200325_BonallyWoods_NIK_1413.NEF 2 3 3583 3574 135 20200325_BonallyWoods_NIK_1413.NEF 3 3 ****duplicate 3789 3574 135 20200325_BonallyWoods_NIK_1413.NEF 4 3 3790 3574 135 20200325_BonallyWoods_NIK_1413.NEF 5 3 3791 3574 135 20200325_BonallyWoods_NIK_1413.NEF 6 3 3792 3574 135 20200325_BonallyWoods_NIK_1413.NEF 7 3 9744 3574 135 20200325_BonallyWoods_NIK_1413.NEF 3 3 ****duplicate The dt database has an index, "images_filename_index" ON "images" ( "filename", "version" ); which means that a 'duplicate' is related to a particular image filename. There are two implications from this which may be relevant to the issue I raised: a) The index is is not unique, it allows duplicates. Therefore the database allows (and cannot trap) insertion of a new image table row with a version number that already exists for the given filename value. The fact I have a duplicate version for a filename suggests the dt code also does not trap this. b) This index assumes filenames are unique across the whole dt database, which probably is not realistic given how cameras from the same manufacturer can generate common filenames. While a unique id is given to each imported image by the dt db to ensure images with the same filename are permitted and can be handled, it seems the 'duplicate image' functionality does not recognise this potential. Regards Dusenberg On 26/07/2023 09:59, Dusenberg wrote:Guillermo, Answers to your questions: a) xmp's are named '<project name>_<camerafilename>_<$VERSION>.RAWextension.xmp' eg: original raw: '20200325_BonallyWoods_NIK_1413.NEF' version 3 xmp: '20200325_BonallyWoods_NIK_1413_03.NEF.xmp' b) I've checked the files and the new duplicate '3' has overwritten the existing xmp for the previous version 3. Also all xmp files in that group new have new modified dates - 24 July 2023, when I created the new duplicate Also I didn't mention that the NIK_1413 RAW and duplicates are in a single group. Thanks for your time. Dusenberg On 26/07/2023 03:41, Guillermo Rozas wrote:That sounds strange. How are the xmp files named? If the duplicate uses a previously used version number, does it also overwrites the corresponding xmp sidecar? Regards, Guillermo On Tue, Jul 25, 2023 at 1:06 PM Dusenberg <dusenb...@gmx.co.uk> wrote: Hi Guillermo Yes the original and all duplicates were in the database before making the new duplicate. Regards Dusenberg On 25/07/2023 15:03, Guillermo Rozas wrote:Hi, were the original and all the duplicates present in the database before making the duplicate? Regards, Guillermo On Tue, Jul 25, 2023 at 5:57 AM Dusenberg <dusenb...@gmx.co.uk> wrote: Originally posted to darktable-dev list in error. dt 4.2.1 (OBS), Linux Mint 21,Ubuntu 22.04 jammy I have an image from March 2020 developed in darktable. I went back to it today to try another edit on it (its a monochrome rendition that I just can't get 'right'). However, today when I created a duplicate of this 2020 image in dt 4.2.1, it was given version number '3' - which already exists for that image (there are seven pre-existing duplicates). I see that dt has also given the new duplicate a different 'image id' to the original RAW image. I've never seen this before, although its not often I go back in time like this. My workflow is that I always create a new version (duplicate) of the base RAW for a different edit so I can trace back any final output that may result. My filenaming system is '<filename>_<version number>_<colorspace>_<max size>' where filename is composed of '<YYYYMMDD_projectname_original camera filename>'. Original camera images are renamed during download onto my workstation via a bespoke script (ie outside dt). I use variables in the dt export module to ensure any output follows this format. This provides unique identification of every image and its derivatives across my libraries, even when intermediate tiffs are involved in say, focus stacks. This is critical for me - I can't have two different edits of a RAW with the same filename! Why has it happened and what can I do about it? Thanks ____________________________________________________________________________ darktable user mailing list to unsubscribe send a mail to darktable-user+unsubscr...@lists.darktable.org <mailto:darktable-user%2bunsubscr...@lists.darktable.org> ____________________________________________________________________________ darktable user mailing list to unsubscribe send a mail to darktable-user+unsubscr...@lists.darktable.org____________________________________________________________________________ darktable user mailing list to unsubscribe send a mail to darktable-user+unsubscr...@lists.darktable.org <mailto:darktable-user%2bunsubscr...@lists.darktable.org> ____________________________________________________________________________ darktable user mailing list to unsubscribe send a mail to darktable-user+unsubscr...@lists.darktable.org____________________________________________________________________________ darktable user mailing list to unsubscribe send a mail to darktable-user+unsubscr...@lists.darktable.org____________________________________________________________________________ darktable user mailing list to unsubscribe send a mail to darktable-user+unsubscr...@lists.darktable.org____________________________________________________________________________ darktable user mailing list to unsubscribe send a mail to darktable-user+unsubscr...@lists.darktable.org <mailto:darktable-user%2bunsubscr...@lists.darktable.org> ____________________________________________________________________________ darktable user mailing list to unsubscribe send a mail to darktable-user+unsubscr...@lists.darktable.org
____________________________________________________________________________ darktable user mailing list to unsubscribe send a mail to darktable-user+unsubscr...@lists.darktable.org
