Proposed fix for this: https://github.com/OpenImageIO/oiio/pull/1481 <https://github.com/OpenImageIO/oiio/pull/1481>
> On Sep 7, 2016, at 12:08 PM, Dan Kripac <[email protected]> wrote: > > Oh sorry, I see what you are saying. > > I understand that it would be tricky to report the accumulated compressed > mipped tiled IO. > > I really just meant it would be good to know the total potential size of all > of the textures files on disk VS. the total potential raw pixels in memory > which is what I'm assuming the "Total size of all images referenced" stat is > referring to? > > Mainly, we were just confused as we were looking at the "Total size of all > images referenced" stat thinking that it was talking about the total size on > disk and that wasn't marrying up to the measured disk file sizes. > > > On 6 September 2016 at 20:35, Larry Gritz <[email protected] > <mailto:[email protected]>> wrote: > I'm not sure I can give you that. > > We can easily report the amount of (uncompressed) texture data read, as we do. > > When we open each file, we can note the total disk size of it (compressed), > and then report that in the per-file stats and the total of all files we ever > touched. > > But I don't think I can give any kind of accurate count of true I/O because > I'm letting the underlying format libraries (such as libtiff or libIlmImf) > handle all the decompression. Neither the TIFF nor OpenEXR libraries have a > way for an app to find out the compressed size of a particular tile that is > read, let alone also account for the fact that to read the tile, some amount > (how much, exactly?) of other I/O is necessary to read headers, seek around, > etc. > > But if you have a good idea for how I can somehow account for this, I am all > ears. > > >> On Sep 6, 2016, at 12:15 PM, Dan Kripac <[email protected] >> <mailto:[email protected]>> wrote: >> >> Thanks Larry, >> >> Yes it would be really good to differentiate between what we are pulling >> over the network compared to what the native raw pixels consume in memory. >> >> Sent from my iThingy >> >> On 6 Sep 2016, at 18:09, Larry Gritz <[email protected] >> <mailto:[email protected]>> wrote: >> >>> It seems reasonable to also collect data on the compressed file size on >>> disk, I'll add this stat. >>> >>> -- lg >>> >>> >>>> On Sep 6, 2016, at 9:43 AM, Larry Gritz <[email protected] >>>> <mailto:[email protected]>> wrote: >>>> >>>> The OIIO stats are the *uncompressed* equivalent -- i.e., the amount of >>>> texture data, not the file size. >>>> >>>> >>>>> On Sep 6, 2016, at 5:12 AM, Søren Ragsdale <[email protected] >>>>> <mailto:[email protected]>> wrote: >>>>> >>>>> I'm looking at the OIIO logs for an OSL render: >>>>> >>>>> OSL ShadingSystem statistics (0x5ab6360) ver 1.7.3 >>>>> [...] >>>>> OpenImageIO ImageCache statistics (shared) ver 1.7.3dev >>>>> [...] >>>>> Images : 1909 unique >>>>> ImageInputs : 17441 created, 100 current, 470 peak >>>>> Total size of all images referenced : 161.7 GB >>>>> >>>>> The problem is that I've isolated the images I'm accessing into a single >>>>> directory, and I'm getting very different numbers: >>>>> >>>>> > du -c -s -h TEXBL_character_arm* >>>>> 65M TEXBL_character_arm-cable_01_texture_default_v002 >>>>> 151M TEXBL_character_arm-cable_01_texture_default_v005 >>>>> 1.2G TEXBL_character_arm-muscles_texture_default_v020 >>>>> 46G TEXBL_character_arm_texture_default_v111 >>>>> 48G total >>>>> >>>>> OIIO seems to be overstating 48 GB of textures as 161 GB. >>>>> >>>>> I've double checked that there aren't any textures being accessed outside >>>>> this directory. If anything, some of the textures in that directory >>>>> should never be accessed so the 'du' total should be even larger. >>>>> _______________________________________________ >>>>> Oiio-dev mailing list >>>>> [email protected] <mailto:[email protected]> >>>>> http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org >>>>> <http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org> >>>> >>>> -- >>>> Larry Gritz >>>> [email protected] <mailto:[email protected]> >>>> >>>> >>>> _______________________________________________ >>>> Oiio-dev mailing list >>>> [email protected] <mailto:[email protected]> >>>> http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org >>>> <http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org> >>> >>> -- >>> Larry Gritz >>> [email protected] <mailto:[email protected]> >>> >>> >>> _______________________________________________ >>> Oiio-dev mailing list >>> [email protected] <mailto:[email protected]> >>> http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org >>> <http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org> >> _______________________________________________ >> Oiio-dev mailing list >> [email protected] <mailto:[email protected]> >> http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org >> <http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org> > > -- > Larry Gritz > [email protected] <mailto:[email protected]> > > > > _______________________________________________ > Oiio-dev mailing list > [email protected] <mailto:[email protected]> > http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org > <http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org> > > > _______________________________________________ > Oiio-dev mailing list > [email protected] > http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org -- Larry Gritz [email protected]
_______________________________________________ Oiio-dev mailing list [email protected] http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org
