Re: Retrieving the EXIF date/time from 250k images

I’ll support Jim Crate’s suggestion of EXIFTool. I just came across this the other day while trying to remove the GPS location from a QuickTime video once I had copied it to my Mac. Honestly, the quickest solution for me was to open it on another Mac in QuickTime 7 Pro, press command J, delete

Re: Retrieving the EXIF date/time from 250k images

On Jan 16, 2023, at 00:52, Gabriel Zachmann via Cocoa-dev wrote: > > I had thought of that , too. > The reason is that I have no experience whatsoever with the JPEG file format. > Nor with the EXIF file format. > Also, I need to be able to parse at least JPEG, PNG, TIF, GIF, and I'd like > to

Re: Retrieving the EXIF date/time from 250k images

> Is there any reason why you don't want to do it yourself, simply loading the > first KBs of the file ? I haven't been writing JPEG decoders for a lng > time, but I'm pretty sure the EXIF (APPn) marker must be written before the > image data (the SOS marker(s)), so it is at the beginning

Re: Retrieving the EXIF date/time from 250k images

There is a perl program called exiftool that can load and set exif tool without loading the image data (or at least it doesn’t decode the image data). I don’t know whether it would be faster than loading image data/properties with ImageIO. You could write a perl script that used your bundled

Re: Retrieving the EXIF date/time from 250k images

> On 7 Jan 2023, at 23:31, Gabriel Zachmann via Cocoa-dev > wrote: > >> >> I wonder if there is a method to load the EXIF data out of the files without >> opening them completely. That would seem like the ideal approach. > > That's the approach I am looking for, but I haven't found any

Re: Retrieving the EXIF date/time from 250k images

You don’t seem to understand modern filesystems or modern hardware like some of us. -- Gary L. Wade https://www.garywade.com/ > On Jan 7, 2023, at 3:29 PM, Gabriel Zachmann wrote: > > *Maybe* ... > that would mean that the filesystem performs predictive caching like

Re: Retrieving the EXIF date/time from 250k images

> > I wonder if there is a method to load the EXIF data out of the files without > opening them completely. That would seem like the ideal approach. That's the approach I am looking for, but I haven't found any API to do that. Best, G. smime.p7s Description: S/MIME cryptographic signature

Re: Retrieving the EXIF date/time from 250k images

*Maybe* ... that would mean that the filesystem performs predictive caching like the CPU's cache / memory management does ... Also, I think you might even get worse performance: imagine several threads reading different files *at the same time* - those files could lie on different HD cylinders

Re: Retrieving the EXIF date/time from 250k images

Hi Gabe. I’d add basic logging before you start each image and after you complete each image to see how much each is taking on each of problem tests so you can see the extent of how slow it is on your problem platforms. Then you can add more logging to expose the problems and start to address

Re: Retrieving the EXIF date/time from 250k images

Since file systems and the associated hardware are designed to be efficient by caching data and knowing things like where one file or block is in relation to another, there’s a possibility these mechanisms could work to your advantage, pulling in the data while “in the neighborhood.” There’s no

Re: Retrieving the EXIF date/time from 250k images

Hi Alex, hi everyone, thanks a lot for the many suggestions! And sorry for following up on this so late! I hope you are still willing to engage in this discussion. Yes, Alex, I agree in that the main question is: how can I get the metadata of a large amount of images (say, 100k-300k) *without*

Re: Retrieving the EXIF date/time from 250k images

On Aug 18, 2022, at 7:47 AM, Mike Abdullah wrote: > > It’s not a very good fit, but when you say a “GCD concurrent queue”, you’d > need to be more specific. There are several configs possible. Do you mean a > global queue, or one you made yourself? If you made it yourself, how did you >

Re: Retrieving the EXIF date/time from 250k images

> On Aug 18, 2022, at 6:47 AM, Mike Abdullah via Cocoa-dev > wrote: > > It’s not a very good fit, but when you say a “GCD concurrent queue”, you’d > need to be more specific. There are several configs possible. Do you mean a > global queue, or one you made yourself? If you made it yourself,

Re: Retrieving the EXIF date/time from 250k images

It’s not a very good fit, but when you say a “GCD concurrent queue”, you’d need to be more specific. There are several configs possible. Do you mean a global queue, or one you made yourself? If you made it yourself, how did you configure it? The tricky problem is that GCD aims to optimise CPU

Re: Retrieving the EXIF date/time from 250k images

Hi Jim. You did exactly what I did. You found the level of diminishing returns on threads. Back in 2008, while working on FiOS TV for Verizon I wrote the design to development automation pipeline that automated export of graphic assets from Illustrator, Photoshop and ImageOptim by Kornel

Re: Retrieving the EXIF date/time from 250k images

I have an app that does some image processing, and when I tried to use GCD it created several hundred threads which didn’t work very well. NSOperationQueue allows you to set the max concurrent operations, and the batch exporting process fully utilizes all logical cores on the CPU.

Re: Retrieving the EXIF date/time from 250k images

I love experimenting with processes like this. Create as many operation processes as you have cores on your box and run a few tests to see how quickly the operations complete. Try it with 2x the number of cores, 1x the number of cores, 1/2 x the number of cores and the number of cores + 1

Re: Retrieving the EXIF date/time from 250k images

Instead of using NSOperationQueue, I would use GCD to handle the tasks. Create a new Concurrent queue (dispatch_queue_create(DISPATCH_QUEUE_CONCURRENT)), then enqueue the individual items to the queue for processing (dispatch_async(), using the queue created above). Everything can be handled in

Re: Retrieving the EXIF date/time from 250k images

You mentioned creating and managing threads on your own, but that’s what NSOperationQueue —and the lower-level DispatchQueue— does. It also will be more efficient with thread management since it has an intimate understanding of the capabilities of the processor, etc., and will work to do the

Re: Retrieving the EXIF date/time from 250k images

That is a good idea. Thanks a lot! Maybe, I can turn this into more fine-grained, dynamic load balancing (or latency hiding), as follows: create a number of threads (workers); as soon as a worker is finished with their "current" image, it gets the next one (a piece of work) out of the list,

Re: Retrieving the EXIF date/time from 250k images

One way to speed it up is to do as much work as possible in parallel. One way —and this is just off the top of my head— is: 1. Create a NSOperationQueue, and add a single operation on that queue to manage the entire process. (This is because some parts of the process are synchronous and might

Re: Retrieving the EXIF date/time from 250k images

You should do any whitespace trimming first and be sure your date formatter is set correctly as any deviation will almost always fail. -- Gary > On Aug 15, 2022, at 2:51 AM, Gabriel Zachmann wrote: > > A detail I left out in-between: whitespace trimming. > > Or, does anyone know if

Re: Retrieving the EXIF date/time from 250k images

> I noticed you release the fileProps but didn’t release the image, but I don’t > know if that’s one of those details you left out for clarity. Good point, and sorry for the confusion. Yes, I do release the image. > Also, depending on some factors like mutability, while the initWithString >

Re: Retrieving the EXIF date/time from 250k images

> I think an important question here is, what exactly are you trying to do? Do > you Good point, I should have mentioned it: I would like to sort the list of images by their EXIF date/time. > really expect to process these images repeatedly? 3.5m for a quarter million > pics isn't terrible

Re: Retrieving the EXIF date/time from 250k images

Sorry if there's dupe emails. I think an important question here is, what exactly are you trying to do? Do you really expect to process these images repeatedly? 3.5m for a quarter million pics isn't terrible when one already possesses a working solution. If you have a specific task in mind, it

Re: Retrieving the EXIF date/time from 250k images

I noticed you release the fileProps but didn’t release the image, but I don’t know if that’s one of those details you left out for clarity. Also, depending on some factors like mutability, while the initWithString call with a CFStringRef might essentially be a no-op, you can just do the

Retrieving the EXIF date/time from 250k images

I would like to collect the date/time stored in an EXIF tag in a bunch of images. I thought I could do so with the following procedure (some details and error checking omitted for sake of clarity): NSMutableArray * dates_and_times = [NSMutableArray arrayWithCapacity: [imagefiles count]];