Re: [darktable-user] cryptic icons
I currently use fswatch to update my web page. When I change a source file (mtime changes) it recompiles markdown into static pages. I'm not a good programmer outside a bit of perl hacking. I've been considering trying to do something with perl scripts, and have been considering how to write the event processor. At this point I would say, "don't hold your breath" Regards Sherwood On Thu, 12 Mar 2020 at 13:30, Patrick Shanahan wrote: > * Sherwood Botsford [03-12-20 15:26]: > > This to me is a serious flaw, but one that AFAIK all programs that > attempt > > to be a DAM suffer from. > > > > How can you make a system that is robust against changes made by external > > programs? > > > > In the Bad Old Days this would be impossible. Now it's merely difficult. > > > > The open source program fswatcher is available for Linux, *BSD, OS-X and > > Windows -- admittedly with somewhat different capabilities. > > > > fswatcher is invoked with a top level of a directory tree to monitor. It > > can monitor when a file is created, modified, moved, renamed deleted, and > > what process did the deed. > > > > So: run this tool as a helper application and have it watch whatever set > of > > directories you've given DT to take care of. > > > > Have DT also have a helper app that handles the created events. > > > > Event: User uses Finder or Explore to rename an image file and leaves it > > in same directory. > > FSWatcher: Detects the event, and queues it for action. > > Helper: Checks that metadata files are renamed to match the image file, > as > > well as thumbnails. Updates database with new name. > > FSWatcher sees Helper's actions, but it knows about Helper so doesn't log > > anything. > > > > Event: User uses finder and moves file to new directory, also under DT > > purview. > > FSWatcher sees the two changed directory entries. > > Helper: Updates the path information for the primary file. Moves the > > auxilary files accordingly. > > > > Event: User askes DT to open file in Photoshop or GIMP. > > FSWatcher sees file creation. > > Helper moves this event to it's own watch list, and waits for the file to > > be closed. > > FSWatcher sees file close. > > Helper checks that name is the same (except for suffix) as original > image. > > Helper extracts preview image from .PSD file and adds it to database. > > Helper notes that this file is derived from master file in > > appropiraite field in database. > > Photoshop crops and saves for web outside DT directories. > > FSWatcher doesn't see this, since it's not watching the destination. > > User realizes his mistake and saves again inside DT directories. > > FSWatcher sees the new file creation. > > Helper creates thumbnails, notes that this file is derived from master > file > > in appropiraite field in database. > > > > *** > > I suggest that fswatcher and helper be different programs for several > > reasons: > > * Due to implementation differences fswatcher will have to be > substantially > > different to parse the observed events into a standard format. > > * You want at least fswatcher to run all the time even when DT isn't > > running. This will build a large queue of changes to be monitored, but > it > > will keep DT in sync with the file system. > > * As a small tight program it's easy to keep debugged, quicker to update > > with changes in OS, and more likely not to encounter some race condition > > and miss events. > > > > > > Regards > > > > Sherwood > > > > > > > > On Wed, 11 Mar 2020 at 23:57, Juha Lintula > wrote: > > > > > Hi, > > > > > > I'm interested if something is missing. Those skulls represent > something > > > you have had and potentially worked on and now they have disappeared. > Is it > > > on purpose or by accident? It's like do you care where your wallet is > if > > > you still have money on your bank account. > > > > > > -Juha > > > > > > On Wed, 11 Mar 2020 at 23:10, Dr. A. Krebs wrote: > > > > > >> Dear August, Dear Gray Card, Dear Patrick: > > >> > > >> thank you for so quick response. I wasn't able to find this > explanation > > >> so quickly. > > >> > > >> If such visual representations for missing files as "skulls" seem not > to > > >> bear extremely relevant
Re: [darktable-user] cryptic icons
This to me is a serious flaw, but one that AFAIK all programs that attempt to be a DAM suffer from. How can you make a system that is robust against changes made by external programs? In the Bad Old Days this would be impossible. Now it's merely difficult. The open source program fswatcher is available for Linux, *BSD, OS-X and Windows -- admittedly with somewhat different capabilities. fswatcher is invoked with a top level of a directory tree to monitor. It can monitor when a file is created, modified, moved, renamed deleted, and what process did the deed. So: run this tool as a helper application and have it watch whatever set of directories you've given DT to take care of. Have DT also have a helper app that handles the created events. Event: User uses Finder or Explore to rename an image file and leaves it in same directory. FSWatcher: Detects the event, and queues it for action. Helper: Checks that metadata files are renamed to match the image file, as well as thumbnails. Updates database with new name. FSWatcher sees Helper's actions, but it knows about Helper so doesn't log anything. Event: User uses finder and moves file to new directory, also under DT purview. FSWatcher sees the two changed directory entries. Helper: Updates the path information for the primary file. Moves the auxilary files accordingly. Event: User askes DT to open file in Photoshop or GIMP. FSWatcher sees file creation. Helper moves this event to it's own watch list, and waits for the file to be closed. FSWatcher sees file close. Helper checks that name is the same (except for suffix) as original image. Helper extracts preview image from .PSD file and adds it to database. Helper notes that this file is derived from master file in appropiraite field in database. Photoshop crops and saves for web outside DT directories. FSWatcher doesn't see this, since it's not watching the destination. User realizes his mistake and saves again inside DT directories. FSWatcher sees the new file creation. Helper creates thumbnails, notes that this file is derived from master file in appropiraite field in database. *** I suggest that fswatcher and helper be different programs for several reasons: * Due to implementation differences fswatcher will have to be substantially different to parse the observed events into a standard format. * You want at least fswatcher to run all the time even when DT isn't running. This will build a large queue of changes to be monitored, but it will keep DT in sync with the file system. * As a small tight program it's easy to keep debugged, quicker to update with changes in OS, and more likely not to encounter some race condition and miss events. Regards Sherwood On Wed, 11 Mar 2020 at 23:57, Juha Lintula wrote: > Hi, > > I'm interested if something is missing. Those skulls represent something > you have had and potentially worked on and now they have disappeared. Is it > on purpose or by accident? It's like do you care where your wallet is if > you still have money on your bank account. > > -Juha > > On Wed, 11 Mar 2020 at 23:10, Dr. A. Krebs wrote: > >> Dear August, Dear Gray Card, Dear Patrick: >> >> thank you for so quick response. I wasn't able to find this explanation >> so quickly. >> >> If such visual representations for missing files as "skulls" seem not to >> bear extremely relevant information, I like to suggest to run those >> mentioned scripts (chapter 2.2.3.2, Gray Card, and the script, Patrick >> mentioned, automatically triggered in background. >> >> What would be the advantage to know, if there is s.th. missing? >> >> Instead of dealing with missing file, I prefer caring about existing ones. >> >> Thanks again, >> >> >> Axel >> - >> Am 11.03.20 um 21:33 schrieb Patrick Shanahan: >> > * Dr. A. Krebs [03-11-20 16:13]: >> >> Hi, >> >> >> >> I use darktable 3.0.1. 64 bit under Linux. >> >> >> >> Instead of picture-previews, I can see only cryptic icons (attachment). >> >> What do these icons mean? >> >> How can I avoid these? >> >> >> >> Is it necessary to "maintain" the darktable database? >> >> Or: Is this done automatically? >> > >> > they are not "cryptic icons" but representations of missing images in >> your >> > library. Images which you have relocated or deleted outside of dt and >> now >> > dt has no knowledge of them. Utilize dt to perform these actions and you >> > will not experience "cryptic icons". >> > >> > there exists a shell script to remove them from your library: >> >/usr/share/darktable/tools/purge_non_existing_images.sh >> > and from your cache: >> >/usr/share/darktable/tools/purge_from_cache.sh >> > >> > >> >> >> darktable user mailing list >> to unsubscribe send a mail to >> darktable-user+unsubscr...@lists.darktable.org >> >> > > darktable user mailing list to unsubscribe send a mail
Re: [darktable-user] Finding photos stored more then once
Not a DT expert. MANY photo apps will have duplicate names. There is usually good reason for this: * Thumbnails are created, sometimes in multiple sizes. * If there is some kind of album feature, these can be implemented as folders of links (aliases) to the actual image. * Some apps can/will duplicate an image before editing it. For non-destructive edits this amounts to duplicating the 'recipe' *** Generally you don't want a zillion files in a single directory. The optimum number for speed varies by operating system, but it's a much smaller number than most photo collections. A few hundred to a few thousand. *** If you have extensions turned off (E.g. Img_0006.JPEG shows in your file browser as Img_0006) then you can't tell the Raw from digital negative from tiff, from jpeg. *** AFAIK *NO* photo dam is robust against file changes happening outside of the program. You can use other tools to *search* for duplicates, but do your removal inside the app. (This doesn't have to be. Programs like fswatch can monitor directory trees and report changes. Clever code could uses these changes to update the picture database, and reconnect side car files.) Regards Sherwood On Sat, 15 Feb 2020 at 13:49, Tobias Krause wrote: > Hi, > > I just started using darktbale - in the first step mainly as DAM tool > trying to clean the mess of photos that have been spread over different > devices, folder, ... > > First things first: darktable is amazing and after just using it fore > some days I wonder how I could ever use something else. Thanks to all > developers! > > Unfortunately quite some of the the photos are stored more than once and > there is no "main" folder containing all images. > > Is there a way to find photos in the database which are the same by > filename and date? Knowing a photo has a twin would make live easier > when it comes to trashing duplicate files: trash all photos in a > specific folder which have a duplicate in some other folder. > > Regards > Tobias > > > darktable user mailing list > to unsubscribe send a mail to > darktable-user+unsubscr...@lists.darktable.org > > darktable user mailing list to unsubscribe send a mail to darktable-user+unsubscr...@lists.darktable.org
Re: [darktable-user] OFF LIST The difference between folder and directory,
There are a bunch of "grouping" terms. Folder is common usage on Windows and Mac. Most desktops on Linix/BSD at least use a folder for the icon. Discussion on technical boards are more likely to use directory. Generally from a user prospective you can use the two words interchangeably. In photography programs you have a bunch of other vocab surfacing, all of it non-standard. Collections Albums Projects Folders In Apple Aperture: A folder can contain other folders, or it can contain a project. All image files have to be in a project. An Album is a virtual construct that under the hood is a link/alias pointing to an image in a project. A smart album is a stored search. You can search within it. Albums I've not used software that included collections, except that I think a trial of Photo Supreme. *** There is no requirement that the organizational structure of what you see is reflected in the file structure underneath. Aperture makes accessing the actual image locations difficult. It takes a lot of extra work to make an image database robust against file browsers moving files around, and changing their names. I've felt for some time there was merit in having some finer grained divisions in organizing photos: I like the idea of project. A wedding. A birthday party. Vacation in Italy. Within a project you may have events: The wedding would have rehearsal, rehearsal dinner, pre wedding, wedding, churchyard, reception. A vacation could be split by days, or cities, Within an event you might have a shoot. This would be all the shots you took without taking the camera away from your face. E.g. the 12 shots of him putting the ring on her finger. The advantage for me for this is the potential to speed keywording, and to find something later. If you knew that Susan spilled wine on her dress in Dresden, then finding the Dresden event just sped your search up. Regards Sherwood On Mon, 10 Feb 2020 at 00:30, Coding Dave wrote: > Actually there is a concept behind and os' following that concept are > calling the thing folder as an abstraction from the technical directory to > map the real world to the computer world. The concept is called the desktop > metaphor and there even exists the folder metaphor. The idea is whenever > you are sitting on a desktop, then you use the same tools. Whether the > desktop is digital or analog you have a paper trash, the desktop, folders, > notepad, (sticky) notes, calculator, and many other things. > > So I guess talking on a usability level the term finder is used, whereas > on a technical level desktop is the word of choice. > > BTW, Microsoft is doing this, Macintosh is doing this, KDE, Gnome, and > others are doing this... I think this is common these days > > https://en.m.wikipedia.org/wiki/Desktop_metaphor > > https://en.m.wikipedia.org/wiki/Directory_(computing) > > https://github.com/darktable-org/darktable/pull/4074 > > > > Anton Aylward schrieb am Mo., 10. Feb. 2020, > 03:27: > >> On 2020-02-09 7:39 p.m., Аl Воgnеr wrote: >> > BTW, can you explain the difference between folder and directory, for a >> > non-native English speaker I cannot see a difference. >> >> I've always taken it as a the Real Thing(tm) is a 'directory' >> but Microsoft, since they had to have a different nomenclature for just >> about >> everything and alter standards in subtle ways and claim they were right >> and >> everyone else is wrong, called them 'folders' and created icons to match. >> >> Perhaps the best counterpoint is SUN's icon for a directory. >> OK so it looks like your smartphones 'contacts' icon. >> Back then, the popular concept was a 'phone directory'. >> Of which "Yellow pages", another bright concept from SUN, was an example. >> >> >> Of course it was UNIX that was The Real Thing(tm) and predated Microsoft, >> perhaps in the form of Multics, by more than a decade. The original >> MS-DOS, you >> recall, was a flat file system. So was the file system for the original >> Macintosh. >> >> But then if we turn the clock back a bit further, in the older IBM >> mainframe >> world a 'directory' was (and still is) the header of a Data Set that >> listed the >> entries in the that data set. >> >> https://www.ibm.com/support/knowledgecenter/en/SSLTBW_2.1.0/com.ibm.zos.v2r1.idad400/pdsd.htm >> >> >> >> However, in our world: >> >> Linux and other UNIX derivations:DIRECTORY >> Windows and other Microsoft derivations: FOLDER >> >> >> -- >> "Objectives are not fate; they are direction. They are not commands; they >> are >> commitments. They do not determine the future; they are a means to >> mobilize >> resources and energies of the business for the making of the future." >>-- Peter F. Drucker. >> >> >> darktable user mailing list >> to unsubscribe send a mail to >> darktable-user+unsubscr...@lists.darktable.org >> >> > ___
Re: [darktable-user] Semi-OT: setting orientation
yeah. In theory, not very often. My experience with Aperture is that I have to do a full rebuild of the database about twice a year, and partial about twice as often. That doesn't require reading the full image. I vaguely recall my time with Digikam was something similar. This is why I want belt and suspenders in my replacement software. In the event of the sort of crash that reduces me to File01, File02 I have a way to recover. That means either a unique ID in the image, or one that can be calculated from the image AND sidecar files AND database. AND metadata in derived images. Regards Sherwood On Sat, 1 Feb 2020 at 12:48, Guillermo Rozas wrote: > The downside of it is the time to recalculate the hash. > >> >> Quick test: md5 against a directory of NEF (nikon raw files) 135 seconds >> for 721 files at about 30M each. >> That's about 5 raw images at 30 M each. Suppose it's 1/5sec each. A >> library of 100,000 images would take about 5 hours to extract checksums. >> >> If the unique ID is in the file, then the program only has to read part >> of the file instead of all of it. >> > > The question is: how many times would you need to do a full recalculation? > I would expect it only to be necessary in case of a catastrophic loss of > all your data and having to recreate everything from zero, in which case I > would guess losing 5h will be the smallest off your problems. > > In any case, you can save a minimum sidecar file including the hash > together with your RAW in the backup. Or better yet, as Stefan suggested, > include the hash on the filename. You could probably include only small > part of it to avoid really long names, and then check the full hash to > verify if you need it. > > Regards, > Guillermo > >> > > darktable user mailing list to unsubscribe send a mail to > darktable-user+unsubscr...@lists.darktable.org > darktable user mailing list to unsubscribe send a mail to darktable-user+unsubscr...@lists.darktable.org
Re: [darktable-user] Semi-OT: setting orientation
This actually is a valid way, and would work to reconnect raw files with sidecar file/databases against the scenarios I've postulated. The downside of it is the time to recalculate the hash. Quick test: md5 against a directory of NEF (nikon raw files) 135 seconds for 721 files at about 30M each. That's about 5 raw images at 30 M each. Suppose it's 1/5sec each. A library of 100,000 images would take about 5 hours to extract checksums. If the unique ID is in the file, then the program only has to read part of the file instead of all of it. This may or may not be a win. I don't know how much of that time just positioning the read heads to slurp up the file. If your disks are capable of 6Gb/s that's roughly600B/s allowing 20% overhead for sector layout and error correction data. That's about 1/20 s per image, about half time time. Realistically, looking at a few drives, a 6Gb drive gets about 160MB/s with medium largish files. If the reads are short -- 4K, then the MB/s goes to hell, but you can do about 150-200 IOP/s A big downside of writing metadata back to the original file is the space it takes in the backup. If I change a file, the whole file gets backed up. You don't want to do this on a regular basis. In my book, a good photo management solution would give you the choice. Any combination of: * Keep the original untouched. * Calculate a hash of the original. * Insert the hash into the original * Back up the original to a temporary folder while spot checks are done with metadata updates. * Dump the first N sectors of the file to a backing store before making changes in that N sectors. If I had to choose today I would keep the original untouched but would want the photomanagement program to write all metadata (includeing the original image checksum) to all derived files. Regards Sherwood On Thu, 30 Jan 2020 at 10:17, Guillermo Rozas wrote: > > If I can do so safely, at a minimum I want a unique ID in the master > image that can be propagated with the image to all derived ones. > > At best I want all critical metadata -- the stuff that takes hours to > put in -- keywords, caption, description to reside in the image, and in the > database, and in the sidecar files, and in every derived image. > > An option to do that without touching the RAW file could be: > - calculate a cryptographic hash on the original RAW file when you > download it > - save the hash in a sidecar file, on an XMP tag which you know is > carried over by all the programs you use > > This also has the nice property of warning you if your RAW file is > corrupted in the future. > > Best regards, > Guillermo > > > darktable user mailing list > to unsubscribe send a mail to > darktable-user+unsubscr...@lists.darktable.org > > darktable user mailing list to unsubscribe send a mail to darktable-user+unsubscr...@lists.darktable.org
Re: [darktable-user] Opencl support for DarkTable
On suspension: On my Mac I have screens go to screen saver in 10 minutes, and shut down after 20. As a sysadmin for a university math department, I had a policy: "Leave them on" My mac uses 160 watts with the screen off, 270 with the screens on If use use an average of 200 w, then we are using about 5 kWh/day. My differential electrical rate is 8 c/kWh, so it costs me 40 cents a day to leave my computer on, or about $150/year 40 cents is worth it me JUST to not have to re-establish state. E.g. right now I have: * 3 Chorme windows up with about 50 tabs * 1 firefox window with 5 tabs. * two textwranger windows up (It remembers full state, though...) * Aperture * Affinity * Ms Word * Preview * Itunes * Messenger I do have a UPS for it to get us through the short power bumps. Regards Sherwood > darktable user mailing list to unsubscribe send a mail to darktable-user+unsubscr...@lists.darktable.org
Re: [darktable-user] Semi-OT: setting orientation
Future proofing. I've already used IMatch, Digikam, Photos, Aperture. I have had incidents with Aperture where it had to rebuild a library and it lost orientation changes on several directories of images. I responded to the orientation question because it's one of the cases that is very simple. If I can do so safely, at a minimum I want a unique ID in the master image that can be propagated with the image to all derived ones. At best I want all critical metadata -- the stuff that takes hours to put in -- keywords, caption, description to reside in the image, and in the database, and in the sidecar files, and in every derived image. Regards Sherwood On Wed, 29 Jan 2020 at 14:08, Patrick Shanahan wrote: > * Sherwood Botsford [01-29-20 15:48]: > > Hashes and other hazards: > > > > Camera makers understand that their cameras have to work with other > editing > > tools. Making a change that encrypts the image is not going to fly very > > well. A single bit flip results in a corrupt file. > > A hash value on part of the image could be useful for verifying that the > > image hasn't been modified since being downloaded from the camera. This > > would mean however that any camera maker has to come up with a unique way > > to hasn their image. Otherwise, the user only has to recalculate the > hash. > > > > Yes: There needs to be extensive checking with each firmware version to > > check that things don't break. At this point you need to decide how > > paranoid to be: > > > > * I will keep my raw images sacrosanct. Keep them in triplicate: One on > > my computer, one in the cloud, one in a disk in a fire/water proof data > > safe in the garage, one on a periodic backup disk stored at my dad's farm > > * I will keep the original images in a separate folder, process them once > > to give each one a unique ID. > > * I will keep original images in a separated folder, and add as much > > metadata as the file format supports to my images, figuring that images > > lost to corruption is a lower risk than images lost to bad indexing. At > > some point when I need disk space I discard the originals. > > * I will just keep my dozen memory cards in a box in my desk, figuring > that > > metadata induced corruption will show up before I start to recycle the > > cards. > > * I'll download the images, reformat the card, and when a problem shows > up, > > go out and reshoot the event. > > > > I'm probably about a #3 or #4 right now. > > > > Benefits: I have been bitten by the "I can't find the original" of this > > image several times, and I only have about 40,000 images. In some cases > I > > had to use a similar image. In a few I've had to reshoot an image. > > I have only once had a loss of images cause by a software malfunction -- > > and that was Nikon's own software. > > > > > > > > > > > > Regards > > > > Sherwood > > > > > > > > On Wed, 29 Jan 2020 at 13:00, wrote: > > > > > What would you do when camera makers decided to store a cryptographic > > > hash or even a signature created over the image and other metadata? > > > You'd invalidate the whole image. > > > > > > I'm not saying they do, but one day they might. Would be a nice > > > feature to certify to some extent how the image was taken. > > > > > > If you want to bet your images on the belief that some spare time > > > programmers can always keep up with each and every turn > > > multi-million-dollar companies do on their undocumented, proprietary > > > formats, be my guest. And bring popcorn. > > > > > > I see no benefit that would outweigh the risk. > > > > > > > > > -- > > > Dr. Stefan Klinger -- Informatiker, Mathematiker o/X > > > https://stefan-klinger.de /\/ > > > I prefer receiving plain text messages, not exceeding 32kB. \ > > > > > > > > > > darktable user mailing list > > > to unsubscribe send a mail to > > > darktable-user+unsubscr...@lists.darktable.org > > > > > > > > > > > > > darktable user mailing list > > to unsubscribe send a mail to > darktable-user+unsubscr...@lists.darktable.org > > > Then why not utilize darktable to perform the ori
Re: [darktable-user] Semi-OT: setting orientation
Hashes and other hazards: Camera makers understand that their cameras have to work with other editing tools. Making a change that encrypts the image is not going to fly very well. A single bit flip results in a corrupt file. A hash value on part of the image could be useful for verifying that the image hasn't been modified since being downloaded from the camera. This would mean however that any camera maker has to come up with a unique way to hasn their image. Otherwise, the user only has to recalculate the hash. Yes: There needs to be extensive checking with each firmware version to check that things don't break. At this point you need to decide how paranoid to be: * I will keep my raw images sacrosanct. Keep them in triplicate: One on my computer, one in the cloud, one in a disk in a fire/water proof data safe in the garage, one on a periodic backup disk stored at my dad's farm * I will keep the original images in a separate folder, process them once to give each one a unique ID. * I will keep original images in a separated folder, and add as much metadata as the file format supports to my images, figuring that images lost to corruption is a lower risk than images lost to bad indexing. At some point when I need disk space I discard the originals. * I will just keep my dozen memory cards in a box in my desk, figuring that metadata induced corruption will show up before I start to recycle the cards. * I'll download the images, reformat the card, and when a problem shows up, go out and reshoot the event. I'm probably about a #3 or #4 right now. Benefits: I have been bitten by the "I can't find the original" of this image several times, and I only have about 40,000 images. In some cases I had to use a similar image. In a few I've had to reshoot an image. I have only once had a loss of images cause by a software malfunction -- and that was Nikon's own software. Regards Sherwood On Wed, 29 Jan 2020 at 13:00, wrote: > What would you do when camera makers decided to store a cryptographic > hash or even a signature created over the image and other metadata? > You'd invalidate the whole image. > > I'm not saying they do, but one day they might. Would be a nice > feature to certify to some extent how the image was taken. > > If you want to bet your images on the belief that some spare time > programmers can always keep up with each and every turn > multi-million-dollar companies do on their undocumented, proprietary > formats, be my guest. And bring popcorn. > > I see no benefit that would outweigh the risk. > > > -- > Dr. Stefan Klinger -- Informatiker, Mathematiker o/X > https://stefan-klinger.de /\/ > I prefer receiving plain text messages, not exceeding 32kB. \ > > > darktable user mailing list > to unsubscribe send a mail to > darktable-user+unsubscr...@lists.darktable.org > > darktable user mailing list to unsubscribe send a mail to darktable-user+unsubscr...@lists.darktable.org
Re: [darktable-user] Semi-OT: setting orientation
First of all, I wouldn't use a hex editor, but use an existing tool, such as Phil's exif tool. Orientation is an exif tag 0x0112 *Orientation* int16u IFD0 1 = Horizontal (normal) 2 = Mirror horizontal 3 = Rotate 180 4 = Mirror vertical 5 = Mirror horizontal and rotate 270 CW 6 = Rotate 90 CW 7 = Mirror horizontal and rotate 90 CW 8 = Rotate 270 CW Regards Sherwood On Wed, 29 Jan 2020 at 11:42, Myron Gochnauer wrote: > Hi Sherwood, > > Thanks for the advice and info. Very helpful. > > I was aware of some of the risks of using a hex editor to change program > files (and similar files). For a few years back in the early/mid 80's I had > to tweak my favorite word processor (XyWrite) so that it would work with a > Hyperion computer, whose character table was non-standard. I seem to > recall tweaking a modem driver too. (300 baud. Wow! I could read email > messages as they scrolled in. Pre-spam days… sigh.) > > Re image orientation numerical values: How can I find out what numbers > are "generally regarded as meaningful"? I would expect any competently > written program to handle out-of-range numbers 'gracefully' as long as they > occupied the same number of bits. (Graceful = ignore or substitute a > default, with or without an error message) > > I like your 'extended copyright' idea. Would you be willing to share the > script you use replace the placeholding X's? I assume you automate the > ExifTool??? Looking at ExifTool's instructions and examples is near the > top of my "I've gotta learn that!" list (up there with Brahms Op.79, > controlling DC motor speeds, and figuring out the network wiring in my > house). 🙂 > > (On Mac, I use A Better Finder Rename to rename imported raw files to Exif > date & time, making on-disk file organization by date/time dead simple, > but I don't think it gives access to other exif data.) > > Myron > > > -- > *From:* Sherwood Botsford > *Sent:* January 29, 2020 12:16 PM > *To:* dt-user list > *Subject:* Re: [darktable-user] Semi-OT: setting orientation > > > ✉External message: Use caution. > If you are going to experiment with editing raw files, some risk factors: > > From reading Phil's exif web site, and other stuff I didn't bookmark > corruption risk: > > Safest: > 1. Editing standard metadata (Exif, ITPC) replacing a value with one of > the same size. Image orientation would be one of these, I think, having a > value of 0-3 (but check the range. Suppose it is 0 to3 and you put in 5? > Will other software ignore it, only pay attention to the smallest two bits > of the number, interpreting it as a 1, go wandering off into a cloud of > blue bits and crash?) > 2. Editing standard metadata replacing a value with one that is smaller. > Some fields are text strings, and so can be padded with blanks, changing > the problem to that of problem 1. > 3. Editing makernotes data. This is propriatary to the camara maker, and > it's format can vary with both model and firmware version. > 4. Editing preview images. The big risk here is that the size of the > preview will change, and whatever you edit with won't leave the end of the > data in the same place, potentially clobbering the front end of the raw > data. > 5. Editing uncompressed raw data. > 6. Editing compressed raw data. > Most risky. > > One of my ideas is to deliberately in my camera put an extended copyright > notice such as > > Copyright S. G. Botsford 2020 -- Image ID > Nikon-D7100-S.7614346----- > > Then, with a script read the exif date, and the shutter count, and replace > the string of X's with the date and shutter count. > > This gives a unique ID trackable back to me for the image. I don't know > if all cameras write an ITPC core section in the meta data, or if this > field is in the maker notes. > > In all derived files, copy this information to a few other fields. Not > all photo editors respect metadata. Your image database tracks this. > > * You edited a bunch of images with Photoshop, saved under different > names. -- you still have a way to connect the .PSD, the resulting TIFF > with the original RAW. > * Your assistant used a file browser to "organize" your image archive. -- > you can reconnect xmp files with masters. > * Your database has crashed, taking everything with it. XMP files can be > reconnected with masters even if names are in doubt. > * You exported a bunch of images. Someone wants a modification that will > work better if starting from the original. > > It doesn't work for all cameras. iPhones don't have a shutter count, nor &g
Re: [darktable-user] Semi-OT: setting orientation
If you are going to experiment with editing raw files, some risk factors:
>From reading Phil's exif web site, and other stuff I didn't bookmark
corruption risk:
Safest:
1. Editing standard metadata (Exif, ITPC) replacing a value with one of
the same size. Image orientation would be one of these, I think, having a
value of 0-3 (but check the range. Suppose it is 0 to3 and you put in 5?
Will other software ignore it, only pay attention to the smallest two bits
of the number, interpreting it as a 1, go wandering off into a cloud of
blue bits and crash?)
2. Editing standard metadata replacing a value with one that is smaller.
Some fields are text strings, and so can be padded with blanks, changing
the problem to that of problem 1.
3. Editing makernotes data. This is propriatary to the camara maker, and
it's format can vary with both model and firmware version.
4. Editing preview images. The big risk here is that the size of the
preview will change, and whatever you edit with won't leave the end of the
data in the same place, potentially clobbering the front end of the raw
data.
5. Editing uncompressed raw data.
6. Editing compressed raw data.
Most risky.
One of my ideas is to deliberately in my camera put an extended copyright
notice such as
Copyright S. G. Botsford 2020 -- Image ID
Nikon-D7100-S.7614346-----
Then, with a script read the exif date, and the shutter count, and replace
the string of X's with the date and shutter count.
This gives a unique ID trackable back to me for the image. I don't know if
all cameras write an ITPC core section in the meta data, or if this field
is in the maker notes.
In all derived files, copy this information to a few other fields. Not all
photo editors respect metadata. Your image database tracks this.
* You edited a bunch of images with Photoshop, saved under different
names. -- you still have a way to connect the .PSD, the resulting TIFF
with the original RAW.
* Your assistant used a file browser to "organize" your image archive. --
you can reconnect xmp files with masters.
* Your database has crashed, taking everything with it. XMP files can be
reconnected with masters even if names are in doubt.
* You exported a bunch of images. Someone wants a modification that will
work better if starting from the original.
It doesn't work for all cameras. iPhones don't have a shutter count, nor
AFAIK a raw format available to the public.
Regards
Sherwood
On Wed, 29 Jan 2020 at 05:43, Patrick Shanahan wrote:
> * Myron Gochnauer [01-29-20 03:05]:
> > Several people suggested trying exiftool… carefully!! (I’ve been burned
> often enough to know you only experiment on copies.)
> >
> > Anyway, exiftool -a -u -g1 {fname} showed the following results:
> >
> > “Orientation : Horizontal (normal)” - - - in exif of NEF landscape
> format image file.
> >
> > "Orientation : Rotate 270 CW” - - - in exif of RAF portrait format image
> file.
> >
> > I’ll see if I can figure out how to edit those values. (I’m a total
> newbie with exiftool.)
> >
> > It *does* seem like there is a place for a GUI app, complete with error
> checking, for altering exif information. If nothing else, I’m pretty sure
> that *everyone* sooner or later produces a bunch of images with last year’s
> copyright information, the wrong time, or some other annoying error.
>
> A "mantra" of darktable is to not touch in any way the "raw" image.
> Editing with exiftool or exiv2 both will make changes to the "raw" image
> file with unknown now or future consequences.
>
> You by using darktable have opted to use a photo editor which does not
> change in any way the "raw" image. Why contradict that by makeing changes
> with another apt?
>
> It is no more work to rotate the images using darktable. And darktable
> will store the image orientation that you determine within the xmp file
> and the library. Maintain backups of your library and your raw image
> files and their accompanying xmp files and your orientation will also be
> retained.
>
> Do what you want with exported images, you have an original now so you can
> always make another.
>
> And a bonus, no need to research exiftool/exiv2 and craft commandline
> parameters to rotate your images.
>
> ps: does your chosen email client not make line wrapping available for
> you? Maybe time to change.
>
>
> --
> (paka)Patrick Shanahan Plainfield, Indiana, USA @ptilopteri
> http://en.opensuse.orgopenSUSE Community Memberfacebook/ptilopteri
> Photos: http://wahoo.no-ip.org/piwigo paka @ IRCnet freenode
>
>
> darktable user mailing list
> to unsubscribe send a mail to
> darktable-user+unsubscr...@lists.darktable.org
>
>
darktable user mailing list
to unsubscribe send a mail to darktable-user+unsubscr...@lists.darktable.org
Re: [darktable-user] DT 3.0 Auto-Crop after lens correction
I can't say for sure. I'm a DT newbie. But consider: You correct for something. Any cropping is going to take off some good pixels as well as some blank. The after corrected image isn't rectangular. But the shape you eventually want to have may include some of those trimmed bits. I've never had this with lens corrections. I don't own any of the really weird lenses with serious corrections. But I have encountered this with perspective correction, and with rotate. Regards Sherwood On Sat, 11 Jan 2020 at 10:35, Michael Staats wrote: > On 11/01/2020 18:28, Michael Staats wrote: > [ ... ] > > > I now have some curved black borders around the visible image, obviously > > the image is corrected for distortion, but it's not auto-cropped to > > remove the artefacts from distortion correction. If I switch off > > lens-correction, no black borders any more -- but also no lens > > correction, obviously ;-) > > > > Is this maybe an issue with lensfun (not being updated properly to the > > expected version in my system), or is it a dt issue? I'm pretty sure I > > did not have the issues with dt 2.x on Ubuntu 16.04. > > Ok, here's what I found just now (sorry for reply to self): > > If I click on "auto-scale" (icon beside "scale"), the scale changes from > 1.000 to 1.015, which is exactly what is required. > > Any idea why this is not happening in dt 3.0 automatically? > > Will it help if I diff the xmp files (2.6 one from backup, and current > 3.0 one)? > > Regards, > Michael > > -- > Michael Staats > michael.sta...@gmx.de > > > darktable user mailing list > to unsubscribe send a mail to > darktable-user+unsubscr...@lists.darktable.org > > darktable user mailing list to unsubscribe send a mail to darktable-user+unsubscr...@lists.darktable.org
Re: [darktable-user] Workflow question
Let me try this again:
Master image at 4000 x 6000 pixels.
I want to make the following images:
2800 x 4200
2000 x 3000
1400 x 2100
1000 x 1500
700 x 1000
500 x 700
350 x 500
250 x 350
adjusting the sharpness for each level.
This is a routine operation to create a set of pictures available in
various formats.
At the end of this, I want ALL of these images to still be in DT.
***
Sometimes Photoshop is the best application for a task. Or 'originals' may
come as .psd files
***
How about some generalized hooks within DT?
E.g. Run an arbitrary command in a temporary folder, importing the file(s)
back into DT.
Is all this possible with Lua?
Ok, ok. I'll go play and see what I can figure out.
Regards
Sherwood
On Thu, 9 Jan 2020 at 04:13, Matt Maguire wrote:
> Check out Keifer Hunniford's YouTube channel, he has a couple of videos on
> this topic.
>
> On Thu, 9 Jan 2020 at 16:20, Coding Dave wrote:
>
>> You can also group images. That was you keep your different versions tied
>> to the original one.
>>
>> Kneops schrieb am Do., 9. Jan. 2020, 09:00:
>>
>>> I only edit few images outside of DT because most of the time they don't
>>> need extra editing. But when I do edit images in Gimp, DT saves them as
>>> tiff file next to the original raw file, so they are 'linked'.
>>>
>>> I tag images at the end when all editing is done, so then I won't have
>>> issues with untagged images because I tag the raw and tiff files at the
>>> end.
>>>
>>>
>>>
>>> Op 09-01-2020 om 00:11 schreef August Schwerdfeger:
>>> > * Cropping and sharpening are performed in Darktable as
>>> non-destructive
>>> > edits.
>>> >
>>> > * Resizing/scaling is done as part of the export process rather than
>>> as
>>> > a darkroom edit, although as far as I know there is not a ready-made
>>> > export target ("storage") that allows exporting the same image in
>>> > several resolutions/formats at once.
>>> >
>>> > * The automatic "master"-to-duplicate propagation of tags is also not
>>> > feasible out-of-the-box, as tags added to one version of an image
>>> cannot
>>> > be automatically propagated to the other versions. You could, of
>>> course,
>>> > select all the versions of a specific image when applying the tag, or
>>> if
>>> > this is not possible there are also some Lua add-ons [1]
>>> > <https://github.com/darktable-org/lua-scripts> that would help copy
>>> the
>>> > new tag.
>>> >
>>> > --
>>> > August Schwerdfeger
>>> > aug...@schwerdfeger.name <mailto:aug...@schwerdfeger.name>
>>> >
>>> > On Wed, Jan 8, 2020 at 3:46 PM Patrick Shanahan >> > <mailto:p...@opensuse.org>> wrote:
>>> >
>>> > * Sherwood Botsford >> > <mailto:sgbotsf...@gmail.com>> [01-08-20 16:04]:
>>> > > I've run into this a couple of times related to the catalog
>>> process:
>>> > >
>>> > > Consider:
>>> > >
>>> > > 1 I import a roll of images.
>>> > > 2 I assign keywords to the images.
>>> > > 3 I tweak them, some in various ways.
>>> > > 4 I crop, resize and sharpen some of them.
>>> > > 5 I export them in various resolutions and various file formats.
>>> > >
>>> > > Starting at step 4 I'm performing destructive edits.
>>> >
>>> > so at step 4 you have left dt and are editing in some other
>>> software?
>>> > dt does not due destructive edits.
>>> >
>>> > > Is there a recommended workflow that would enable:
>>> >
>>> > use dt rather than some other software.
>>> >
>>> > --
>>> > (paka)Patrick Shanahan Plainfield, Indiana, USA
>>> > @ptilopteri
>>> > http://en.opensuse.orgopenSUSE Community Member
>>> > facebook/ptilopteri
>>> > Photos: http://wahoo.no-ip.org/piwigo paka @ IRCnet
>>> > freenode
>>> >
>>>
>>>
>>> > darktable user mailing list
>>> > to unsubscribe send a mail to
>>> > darktable-user+unsubscr...@lists.darktable.or
[darktable-user] Workflow question
I've run into this a couple of times related to the catalog process: Consider: 1 I import a roll of images. 2 I assign keywords to the images. 3 I tweak them, some in various ways. 4 I crop, resize and sharpen some of them. 5 I export them in various resolutions and various file formats. Starting at step 4 I'm performing destructive edits. Is there a recommended workflow that would enable: * Keeping the destructively edited versions in DT, and tracking that they were derived from a given master. * Adding keywords to an edited version that didn't necessarily apply to the master image. E.g. One time I did a batch of files with output in 18 different resolutions. Suppose a client sends me back an image, "Can I get the full resolution version of this? Some metadata standard to track derivatives? E.g. I crop a person out of an image. The original image has keyword "Person: John Smith" The new image should be missing that keyword. But the flip side of that: Later I add another keyword to the master: "Weather: Cloudy Bright" Then I want that keyword to propagate to derived images. Regards Sherwood darktable user mailing list to unsubscribe send a mail to darktable-user+unsubscr...@lists.darktable.org
[darktable-user] How does DT store multiple images from the same master.
Consider the following scenario: I take a picture. I import into darktable. I duplicate the image. I edit both of them. =>> At this point there is still only one master file, but two sets of edit files. Does DT in effect store both edits as separate stanzas in the side car file? Regards Sherwood darktable user mailing list to unsubscribe send a mail to darktable-user+unsubscr...@lists.darktable.org
