Thanks for the suggestion. I'll take some time to look at what could have caused a regression / change
On 2 January 2016 at 22:28, Larry Gritz <[email protected]> wrote: > That's a fascinating clue! > > I'm not using "the latest", but I'm certainly using 2.x (2.1 I think?) > because of our need for deep files. It never occurred me to test with 1.x, > but it will be easy to go back to up the OIIO-based synthetic benchmark I > used for my original findings and rerun it after building against OpenEXR > 1.7. > > This is very hopeful! If it used to be fast to read OpenEXR and now is > slow, that means that the speed problem may not be inherent in the file > layout itself or the basic architecture of the library, but could just be a > performance regression in the library that went undetected and could be > fixed. That would really be great for everybody. > > I'll look into this as soon as I'm back at work on Monday and will report > back. > > Thanks, Peter. > > -- lg > > > On Jan 2, 2016, at 10:06 PM, Peter Pearson <[email protected]> > wrote: > > Hi, > > This is really a continuation of Larry's deviation (but a very useful and > timely one!) in the '.exr vs .tx' thread - I've only just subscribed to > this list, so can't respond directly to that. > > I've seen pretty much the same issue regarding read speeds, with TIFF > being significantly faster (around 1.5-1.7x) in my tests within a renderer > - incoherent access for varying mipmap levels (or subimages in TIFF's > case), on a local disk on Linux. > > There's also this bug: > https://github.com/openexr/openexr/pull/170 > > that in my case severely affects concurrent access when using worker > threads to do the reading work - without Larry's patch, 3/4 threads is the > best I can hope for, after that there's so much contention that CPU usage > just dwindles over time - with the patch, I can push 24 threads easily, and > wall-clock time goes down by orders of magnitude. > > Ignoring the above patch for a moment though, I've previously (2/3 years > ago) found that at least within PRMan, OpenEXR reading was noticeably > faster than TIFF reading, even when reading half for EXR and 8-bit for > TIFF, so TIFF technically has an unfair advantage. This was over NFS on a > saturated network - on a local disk, TIFF could sometimes win. At the time, > profiling (at the NFS level) seemed to indicate that the stat() call done > within TIFFSetDirectory() to change mipmap levels caused disproportionate > slowdowns for TIFF, as the raw reading and decompression was generally > faster than EXR (although EXR compresses more in general, and things like > tile size and planar config will obviously affect the comparison one way or > another). However, as the studio I was with at the time used PRMan, we were > using Pixar's format which was faster and more compact than EXR, so just > used Pixar's format instead of investigating further. > > I've always since assumed based on this that OpenEXR was the better format > than TIFF for rendering, but I've recently been doing some work on texture > reading for a renderer, and as well as noticing the scalability issue Larry > found, have also found that TIFF is faster than OpenEXR is using zip > compression - which as far as I'm aware (ignoring DWA type for the moment) > is the fastest for tiled? > > I've done a quick test today with IlmBase 1.0.2 and OpenEXR 1.7.0 just as > a "I'm sure it didn't used to be *this* bad" sanity-check, and indeed - as > well as the IlmBase ThreadPool bug above not seeming to exist - reading of > the same EXRs with 1.7.0 vs 2.0.1 is almost twice as fast, for the > wall-clock time of reading them in a renderer. > > Given that I've tested this in a renderer, it's a bit difficult to give > exact timings, but based on wall-clock texture read times, I'm roughly > seeing OpenEXR 1.7 read times be 50-70% of the OpenEXR 2.0.1 ones. > > So it seems (possibly pending more investigation and timings) that there > has been a regression in read speeds since OpenEXR 2.0 (and currently, as > I'm assuming Larry was testing with the latest OpenEXR version?) > > Cheers, > Peter > > > > _______________________________________________ > Oiio-dev mailing list > [email protected] > http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org > > > -- > Larry Gritz > [email protected] > > > > _______________________________________________ > Oiio-dev mailing list > [email protected] > http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org > >
_______________________________________________ Oiio-dev mailing list [email protected] http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org
