OK, let's get this straight once and for all: there is no such thing as a ".tx" file. I'm pretty sure that NONE of the couple dozen image file formats that OIIO supports actually consider the file name extension semantically meaningful. The files are always identified by their internal layout, and at most the file extension is just used to help guess which ImageInput type to try first, but if that fails it will simply try every one it knows about until one successfully reads the file (or none of them do).
By convention -- but in no way required or aided by OIIO -- it is common to name tiled & MIP-mapped TIFF files ".tx", I suppose just to distinguish them from the ordinary TIFF files floating around (but which, usually scanline and almost never MIPmapped, are unsuitable as efficient texture files). I guess this dates from PRMan's "txmake" program, though IIRC it was at least as common in the olden days to name its proprietary files ".tex", but since I use LaTeX a lot myself, I never liked that so tended to use ".tx." Anyway. OpenEXR versus TIFF for textures. It's funny you should ask, because I recently did some extensive benchmarks to shed light on this very issue. The first thing you need to know is that OpenEXR only supports 'half' (16 bit float), float (32 bit), and unsigned 32 bit int (the latter only used for object IDs as far as I know, never for color values). Really, half is what you want, it seems stupidly wasteful to use 32 bit float for most possible uses of texture. But TIFF *doesn't* support half. So it's hard to do an "apples-to-apples" comparison. So the decision has traditionally come down to: if you want uint8 or uint16 textures, use TIFF; if you need extended range (such as for IBL environment maps), use half in OpenEXR. OK, but it turns out I sort of lied. There is this http://chriscox.org/TIFFTN3d1.pdf <http://chriscox.org/TIFFTN3d1.pdf> which is an Adobe tech report describing 16 bit float extension to TIFF. OIIO has for some time been rigged to *read* this properly, but since 'half TIFF' does not read properly by either PhotoShop or Nuke (and worse, Nuke doesn't even notice it's broken, it just badly misinterprets it as uint16), it seemed too dangerous to enable it for writing, for fear of infesting the world with files that will not be read correctly. (Aside: PhotoShop and Nuke, if you used OIIO, it would read just fine. So there.) But I had to know. So I finally enabled this in OIIO just a couple weeks ago (https://github.com/OpenImageIO/oiio/pull/1283 <https://github.com/OpenImageIO/oiio/pull/1283>). For safety, it ONLY will write 'half' TIFF files if you set the global OIIO::attribute("tiff:half", 1). You have to really want it, and you have to be ok knowing that only OIIO-based apps will read it correctly. But once you have it, you can finally compare TIFF vs OpenEXR directly, with the same data format. I took a whole bunch (many GBs) of uint16 textures randomly selected from a certain large VFX show we're working on at the moment, and converted them to half TIFF, and also to half OpenEXR, and then ran a bunch of synthetic benchmarks on them. (Certain modes of OIIO's 'testtex' unit test are really helpful for timing tests.) I'm on vacation for the next couple weeks, so if you really care I can dig up the numbers when I get back, but the bottom line is that file access is significantly faster for TIFF! Like, it's 1.5x faster or more to read the same tile pattern from a half TIFF file as from the equivalent half OpenEXR file. I'm still in the middle of trying to understand why. But it seems pretty incontrovertible. I can give a detailed recipe if you want for how to reproduce these findings (with your own textures, of course). Ironically, the whole purpose of my going down that road was that I was interested in trying the DWA lossy compression for texture files -- for the sake of reducing disk storage and network bandwidth for texture reads. It's great for that, these textures look visually identical while achieving a 2-3x reduction of space and bandwidth, but to make a complete evaluation I did time trials as well, and then it got really confusing because the format with the better compression has such massively worse read performance. The perf difference is not because of compression/decompression. I replicated the experiment for zip compression, and for no compression. The ratio by which TIFF seems faster than OpenEXR seems fairly stable no matter what else I vary. When I use the DWA lossy compression for EXR, it speeds up a bit (fewer bytes read/transmitted!) but still never catches up to TIFF, not even close. Honestly, I'm shocked. I truly expected TIFF (which has no shortage of design issues) to be slower, and thus clear the path to using lossily compressed textures (for some purposes, and subject to aesthetic approval, of course). But no, it's determined not to be a clean win, but a very complex tradeoff. So, since I have you on the line, Bruce, is there anything you can tell us about DWA compression as used by DWA? How do you use it? For what purposes/uses? What compression levels? Have you ever benchmarked it versus other formats? Has anybody at any time tried to validate OpenEXR read/write performance versus any other file format? If so, did they also find that it's a little lacking? -- lg > On Dec 22, 2015, at 4:48 PM, Thiago Ize <[email protected]> wrote: > > .tx files are just .tiff or .exr files that have been renamed .tx. I think > as long as you generated the .tif/.exr file with maketx, it'll be the exact > same thing. You can confirm by looking at your files with the "iinfo -v " > oiio tool. Calling it .tx just helps in letting people know that the texture > has already been optimized without forcing them to parse the file and look at > the file attributes. > > Just to be clear, a .exr/.tif that was not made with maketx will not perform > as well since it doesn't have the AverageColor or SAH-1 oiio attributes. > > Finally, don't take a .tx file, rename it to .exr, edit it in photoshop, and > rename to .tx. Do that and now it's no longer an optimized file. In fact, > you might even get rendering artifacts since the AverageColor and SHA-1 > attributes could be wrong. > > On Tue, Dec 22, 2015 at 5:36 PM, Bruce Tartaglia > <[email protected] <mailto:[email protected]>> > wrote: > Hi all, > > I am using OIIO to manage mipmapped texture lookups, > and I have a question about the exr vs the tx format. > > In brief, is there a preferred / optimized format for the OIIO library? > Both seem to work quite well, both support mip mapping, half > data formats, maketx --oiio flag, and tiling. So is one superior to > the other? > > I suspect that tx is likely closer to an internal, native OIIO format, > so perhaps exr incurs a data conversion cost, but I have not proof of that. > I am starting to read the OIIO code to best understand this, but I thought > I could simply ask here as well. > > Any information is appreciated, and thank you for such > useful software. > > Best, > > Bruce > > _______________________________________________ > Oiio-dev mailing list > [email protected] <mailto:[email protected]> > http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org > <http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org> > > > _______________________________________________ > Oiio-dev mailing list > [email protected] > http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org -- Larry Gritz [email protected]
_______________________________________________ Oiio-dev mailing list [email protected] http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org
