John, or anybody else interested in this issue, you should follow the
discussion happening in the [issue I
created](https://github.com/OpenImageIO/oiio/issues/2744).
Jan correctly points out that there is a version of
read_scanlines/read_tiles/read_image that takes chbegin and chend arguments,
which can be used to read just one channel (chbegin=c, chend=c+1), and so
calling it once per channel, directing that channel to the appropriate
single-channel buffer, is surprisingly efficient.
This may defy expectation, but there are several factors to keep in mind:
* Deep underneath, the format library is probably reading and decompressing the
scanline or tile just once and then just copying the decompressed results if
you are making subsequent calls requesting the same scanline or tile. So the
total amount of disk read and decompress is no worse than a single interleaved
read.
* Any requested data type conversion is happening for each channel
individually, so the total conversion work is the same as if you'd asked for
all channels together and interleaved (except maybe for a tiny bit of extra
loop overhead).
* Only one channel per call has to be copied to the user buffer, so the total
amount of data copied to the user area is the same as if you'd asked for all
channels together (just split into nchannels chunks).
* The extraction of one channel into the user buffer that happens in our
implementation of read_scanilnes/read_tiles is multithreaded! For this reason,
you SHOULD NOT read into a temp buffer and do the unscrambling on your own, as
I previously advised, unless you are very confident that your implementation of
the unscrambling is more efficient than my multithreaded version.
We also observe that in our OpenEXR reader, the implementation of
read_scanlines/read_tiles where you ask for a single channel is ALREADY as
efficient as it can get, since it requests only that one channel from the exr
library, and as far as we know, incurs no unnecessary copies in the process.
So at most, we might want to beef up our internal TIFF reader implementation to
fully exploit single-channel reads in the case where the tiff file itself is
stored as separate channel planes.
John, I would be curious to hear if Gaffer is doing this the way that we now
suspect is most efficient -- multiple calls requesting a single channel, rather
than asking for all channels in a temp buffer and then descrambling yourself.
We may already be very close to optimal efficiency, at least for some formats,
and can do very focused changes to make other formats more efficient if needed.
-- lg
> On Oct 16, 2020, at 1:27 AM, John Haddon <[email protected]> wrote:
>
> On Fri, 16 Oct 2020 at 08:27, Larry Gritz <[email protected]
> <mailto:[email protected]>> wrote:
> Like I said, the lack of requests for this over the years is strong
> circumstantial evidence that it's not an important enough case to enough
> people to justify the work or the extra API complexity, though if lots of
> people chime in that they have wanted this, I could be convinced.
>
> This is definitely something we would use in Gaffer, as we're currently doing
> the rescrambling for all IO. But right now I don't have any solid numbers for
> how much overhead that represents, so would need to do some profiling before
> I could say it justified the effort. My suspicion is we'd need to optimise
> other aspects of the processing pipeline before it became particularly
> beneficial.
>
> On a more general note, not aimed at OIIO in particular, my experience is
> that "nobody asked" isn't necessarily a good indication that all is well. I'm
> regularly astounded at the ingenious contortions folks will go through as a
> workaround before asking for something.
>
> Cheers...
> John
>
> _______________________________________________
> Oiio-dev mailing list
> [email protected]
> http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org
--
Larry Gritz
[email protected]
_______________________________________________
Oiio-dev mailing list
[email protected]
http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org
