Ramon, I took a stab at the API part of this. I haven't implemented the OpenEXR specifics yet, first wanted you and others to weigh in on the basic design:
https://github.com/OpenImageIO/oiio/pull/1392 <https://github.com/OpenImageIO/oiio/pull/1392> > On Mar 7, 2016, at 1:09 PM, Ramon Montoya Vozmediano <[email protected]> > wrote: > > > Do you think we need write_scanlines similarly modified? write_image? Or is > > the practical use case for this always going to be tiles? > > The format does support those cases, I think at least write_scanlines should > also be modified. The only reason to modify write_image would be for > consistency. For our needs adding the new write_tiles / write_scanlines would > be sufficient in the way you propose. > > > What about input? Do you think it's important to allow reading of a tile > > from one part in a random access way, but without needing a seek_subimage > > to set the current subimage? > > Sounds like with seek_subimage you can already read in a random way. The IO > actions from the OpenEXR library would be the same in both cases (find the > chunk in the table of offsets and get it). So I think it's really a case of > evaluating if the new interface would save OIIO some overhead or if it would > simplify subimage access for OIIO users. > > > I want to add that while I'm happy to add this flexibility to the OIIO API, > > I make NO GUARANTEES about libIlmImf (or any other format libraries) > > performance with random part access. I don't know how it buffers internally > > or whether keeping 200 parts alive in an open file will result in massive > > memory allocation internal to the library that I can't control. > > True. The physical layout of the multipart EXR seems to be specifically > designed to allow that kind of setup, so I imagine it's not bad. > > r > > On Mon, Mar 7, 2016 at 9:45 PM, Piotr Stanczyk <[email protected] > <mailto:[email protected]>> wrote: > I'd be interested in any performance bottlenecks that people run into with > the multi-part path. One of the bigger wins we saw was with stereo image > pipelines where only one view was being asked for from the file, a simple > seek to the requested part vs a not an insignificant amount of IO. > > -Piotr > > On 7 March 2016 at 12:25, Larry Gritz <[email protected] > <mailto:[email protected]>> wrote: > I want to add that while I'm happy to add this flexibility to the OIIO API, I > make NO GUARANTEES about libIlmImf (or any other format libraries) > performance with random part access. I don't know how it buffers internally > or whether keeping 200 parts alive in an open file will result in massive > memory allocation internal to the library that I can't control. > > -- lg > > > > On Mar 7, 2016, at 12:18 PM, Larry Gritz <[email protected] > > <mailto:[email protected]>> wrote: > > > > Thanks for the additional input, Pete. > > > > So is the current proposal is: > > > > * ImageOutput::supports("random_subimage_access") will return true for > > formats that support writing to subimages other than the currently > > designated one (set by the last open() call). > > > > * Add a variant of ImageOutput::write_tiles that accepts an "int subimage" > > parameter to specify which subimage the tile belongs to. > > > > Is that sufficient to meet your needs? > > > > > > > >> On Mar 7, 2016, at 12:10 PM, Pete Black <[email protected] > >> <mailto:[email protected]>> wrote: > >> > >> Hi Larry, > >> > >> I personally can see a need to write entire image parts in a more flexible > >> fashion, rather than tiles - I cant talk about the details on-list, but > >> recently I had to deal with stereo pairs of EXRS each with 400+ channels, > >> and selectively output multipart stereo EXRs with 200+ parts per frame - I > >> could do the job with OIIO, but having more flexible APIs for multipart > >> would be nice. > >> > >> Thanks > >> > >> -Pete > >> > >> > >>> On 8/03/2016, at 8:48 AM, Larry Gritz <[email protected] > >>> <mailto:[email protected]>> wrote: > >>> > >>> Wow, I guess I never fully realized that OpenEXR's API allowed for this! > >>> OpenEXR's addition of "multi-part" was a latecomer, compared to TIFF > >>> which has allowed multiple images in a file, but not random read or write > >>> access to them, and I guess I never reexamined those assumptions when > >>> OpenEXR added the ability. > >>> > >>> Yes, in light of that, I think it does make sense to extend the API to > >>> have a variant of write_tiles that lets you specify the subimage. Of > >>> course, since most image formats (such as TIFF) do not allow this > >>> flexibility, apps will be cautioned to only use it for output formats for > >>> which supports("random_subimage_access") is true, otherwise it would be > >>> an error to write tiles to anything but the currently-opened subimage. > >>> > >>> Do you think we need write_scanlines similarly modified? write_image? Or > >>> is the practical use case for this always going to be tiles? > >>> > >>> What about input? Do you think it's important to allow reading of a tile > >>> from one part in a random access way, but without needing a seek_subimage > >>> to set the current subimage? > >>> > >>> Let me think a bit about how to structure this. It seems to me that this > >>> is going to have to be a master-only feature, since adding it is going to > >>> break link compatibility with the existing ImageOutput class API and > >>> vtable. > >>> > >>> -- lg > >>> > >>> > >>>> On Mar 7, 2016, at 3:38 AM, Ramon Montoya Vozmediano > >>>> <[email protected] <mailto:[email protected]>> wrote: > >>>> > >>>> Hi Larry, > >>>> > >>>> OIIO supports output to multipart EXR images, and they have to be > >>>> written out like this: > >>>> > >>>> open(multipart file with n parts) > >>>> write_tiles(...) > >>>> > >>>> open(...AppendSubimage) > >>>> write_tiles(...) > >>>> > >>>> open(...AppendSubimage) > >>>> write_tiles(...) > >>>> > >>>> close() > >>>> > >>>> So you write out the first subimage, then the next, etc... > >>>> > >>>> The motivation for this email is that with this setup, when you are > >>>> rendering, you need to keep all the tiles in memory before they can be > >>>> written out. > >>>> > >>>> OpenEXR itself lets you write out tiles for any part in any order. > >>>> > >>>> So, when using the EXR API directly, a renderer could write out tiles as > >>>> soon as they are done, and the physical layout of the multipart image > >>>> would end up looking like this: > >>>> > >>>> [chunk offset index] > >>>> [part 0, data for tile (0,0)] > >>>> [part 1, data for tile (0,0)] > >>>> [part 2, data for tile (0,0)] > >>>> ... > >>>> [part 0, data for tile (0,1)] > >>>> [part 1, data for tile (0,1)] > >>>> [part 2, data for tile (0,1)] > >>>> ... > >>>> > >>>> It looks like supporting this means extending OIIO's API, so that you > >>>> can set the part you want to write to, without having to close/open a > >>>> "subimage". > >>>> > >>>> Another way to look at this conceptually, is to consider an EXR part as > >>>> a channel grouping rather than a tiff style subimage. > >>>> > >>>> Have you considered extending OIIO's API in this direction? > >>>> > >>>> Best, > >>>> > >>>> r > >>>> > >>> > >>> -- > >>> Larry Gritz > >>> [email protected] <mailto:[email protected]> > >>> > >>> > >>> _______________________________________________ > >>> Oiio-dev mailing list > >>> [email protected] <mailto:[email protected]> > >>> http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org > >>> <http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org> > >> > >> _______________________________________________ > >> Oiio-dev mailing list > >> [email protected] <mailto:[email protected]> > >> http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org > >> <http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org> > > > > -- > > Larry Gritz > > [email protected] <mailto:[email protected]> > > > > > > _______________________________________________ > > Oiio-dev mailing list > > [email protected] <mailto:[email protected]> > > http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org > > <http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org> > > -- > Larry Gritz > [email protected] <mailto:[email protected]> > > > _______________________________________________ > Oiio-dev mailing list > [email protected] <mailto:[email protected]> > http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org > <http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org> > > > _______________________________________________ > Oiio-dev mailing list > [email protected] <mailto:[email protected]> > http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org > <http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org> > > > _______________________________________________ > Oiio-dev mailing list > [email protected] > http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org -- Larry Gritz [email protected]
_______________________________________________ Oiio-dev mailing list [email protected] http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org
