That's a fascinating clue!
I'm not using "the latest", but I'm certainly using 2.x (2.1 I think?) because
of our need for deep files. It never occurred me to test with 1.x, but it will
be easy to go back to up the OIIO-based synthetic benchmark I used for my
original findings and rerun it after building against OpenEXR 1.7.
This is very hopeful! If it used to be fast to read OpenEXR and now is slow,
that means that the speed problem may not be inherent in the file layout itself
or the basic architecture of the library, but could just be a performance
regression in the library that went undetected and could be fixed. That would
really be great for everybody.
I'll look into this as soon as I'm back at work on Monday and will report back.
Thanks, Peter.
-- lg
> On Jan 2, 2016, at 10:06 PM, Peter Pearson <[email protected]> wrote:
>
> Hi,
>
> This is really a continuation of Larry's deviation (but a very useful and
> timely one!) in the '.exr vs .tx' thread - I've only just subscribed to this
> list, so can't respond directly to that.
>
> I've seen pretty much the same issue regarding read speeds, with TIFF being
> significantly faster (around 1.5-1.7x) in my tests within a renderer -
> incoherent access for varying mipmap levels (or subimages in TIFF's case), on
> a local disk on Linux.
>
> There's also this bug:
> https://github.com/openexr/openexr/pull/170
> <https://github.com/openexr/openexr/pull/170>
>
> that in my case severely affects concurrent access when using worker threads
> to do the reading work - without Larry's patch, 3/4 threads is the best I can
> hope for, after that there's so much contention that CPU usage just dwindles
> over time - with the patch, I can push 24 threads easily, and wall-clock time
> goes down by orders of magnitude.
>
> Ignoring the above patch for a moment though, I've previously (2/3 years ago)
> found that at least within PRMan, OpenEXR reading was noticeably faster than
> TIFF reading, even when reading half for EXR and 8-bit for TIFF, so TIFF
> technically has an unfair advantage. This was over NFS on a saturated network
> - on a local disk, TIFF could sometimes win. At the time, profiling (at the
> NFS level) seemed to indicate that the stat() call done within
> TIFFSetDirectory() to change mipmap levels caused disproportionate slowdowns
> for TIFF, as the raw reading and decompression was generally faster than EXR
> (although EXR compresses more in general, and things like tile size and
> planar config will obviously affect the comparison one way or another).
> However, as the studio I was with at the time used PRMan, we were using
> Pixar's format which was faster and more compact than EXR, so just used
> Pixar's format instead of investigating further.
>
> I've always since assumed based on this that OpenEXR was the better format
> than TIFF for rendering, but I've recently been doing some work on texture
> reading for a renderer, and as well as noticing the scalability issue Larry
> found, have also found that TIFF is faster than OpenEXR is using zip
> compression - which as far as I'm aware (ignoring DWA type for the moment) is
> the fastest for tiled?
>
> I've done a quick test today with IlmBase 1.0.2 and OpenEXR 1.7.0 just as a
> "I'm sure it didn't used to be *this* bad" sanity-check, and indeed - as well
> as the IlmBase ThreadPool bug above not seeming to exist - reading of the
> same EXRs with 1.7.0 vs 2.0.1 is almost twice as fast, for the wall-clock
> time of reading them in a renderer.
>
> Given that I've tested this in a renderer, it's a bit difficult to give exact
> timings, but based on wall-clock texture read times, I'm roughly seeing
> OpenEXR 1.7 read times be 50-70% of the OpenEXR 2.0.1 ones.
>
> So it seems (possibly pending more investigation and timings) that there has
> been a regression in read speeds since OpenEXR 2.0 (and currently, as I'm
> assuming Larry was testing with the latest OpenEXR version?)
>
> Cheers,
> Peter
>
>
>
> _______________________________________________
> Oiio-dev mailing list
> [email protected]
> http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org
--
Larry Gritz
[email protected]
_______________________________________________
Oiio-dev mailing list
[email protected]
http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org
