On Dec 3, 2009, at 8:10 AM, gMail.com wrote: > Thanks. I supposed that I was loading from the cache. It's a pity. > It was too nice to load 10,000 files x 4KB each, in only 1.2 secs. > Maybe one day, when I will be not longer on this planet :-) > Just to mention I run MacOSX 10.6.2 and I build against 10.5 (32 bit).
Well.... 10,000 x 4KB files sounds like an excellent design for the benefit of spotlight [which needs individual files for each data item it indexes] but is a very poor design for reading all that data at once. Prior to the disk cache being warmed up, reading those 10,000 files requires a boatload of I/O of the worst kind in that the data is unlikely to be contiguous. If you want to speed up the initial read, cache the contents of the 10,000 files into a single file. Even those 10,000 files laid out contiguously and memory mapped is going to be faster, but you can do much better by effectively 'compiling' the data into some form that is much more convenient to read. This is *exactly* what Address Book, Mail and other applications do. In the case of AB and Mail, they are using CoreData and SQLite directly respectively to store the data into a single file. Perhaps CoreData would fit your needs as well (you haven't said what the 10,000 files contain). > Anyway I would like to say a thing that I wanted to say for years. > Despite to the faster and faster processors and machines, the better and > better OSs... still today with a Core 2 Duo Intel 2.4GHz and MacOS X 10.6.2 > at 64 bits (which is considered mainly as an improvement of the stability > and speed), when I open my /Applications folder, I have to wait for 3, 4 or > 5 seconds to see the contents of the folder. I recall that my Mac II with > the System Mac 1.0 (in 1988) was faster. Think over ;-) Yes, but the one thing that *hasn't* changed in all those years is that hard drives have *not* gotten ~150x faster [CPU speed] and zillions of times faster on the memory speed front. Hard drives are slow, slow, slow. Yet, the metadata being read to -- say -- display that folder full o' applications is considerably larger. Heck; The recommended icon size on Mac OS X Snow Leopard -- 512x512x4 -- is *larger than the total resolution of the original Macintosh's screen*. Even more amusing; a modern icon fully decompressed will take up 1MB of RAM -- it wasn't until the Mac II that a Macintosh had enough memory to even load such a beast! And that is just the icons. You also have the full blown localization support and permissions metadata, too. The first read is several orders of magnitude more data, all spread around the disk, than that Mac II and from a device-- the hard drive-- that is *not* several orders of magnitude. Now, if you want an eye opener, test your 10,000 x 4KB file read w/cold disk cache on a decent SSD drive... b.bum _______________________________________________ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
