Hi Ben,
here are some thoughts about OpenEXR as a capture format. I must admit up front that I don't know much about the internal processing capabilities of digital cameras, so the following may or may not be particularly relevant. Of course, if any camera manufacturers are interested, I'd be more than happy to discuss what it would take to make a camera output OpenEXR directly. Is it useful? The way I see it, OpenEXR's main advantage would be the format's wide exposure latitude and fine color resolution. A camera's capabilities are not likely to exceed what the file format can store. This would allow the camera to save scene referred (or focal-plane referred) images, where the values stored in the pixels are proportional to the amount of light emitted by the objects in the original scene (as opposed to storing the color you want to see when the image is displayed). Color rendering decisions can be postponed until later. If the camera has knobs to tweak the look of the images, the settings of those knobs could be saved as metadata along with the unmodified scene referred images. This capability to store scene referred images with a wide dynamic range and to postpone color rendering decisions is similar to the various camera raw formats that already exist. One advantage of OpenEXR is that the image representation would not be camera-specific, and that the first stages of processing the raw sensor data have already been done. This includes Bayer pattern demosaicing and estimating actual scene colors from the raw sensor values, operations that are best left to the camera manyfacturer. Unlike raw files, OpenEXR files are ready to use in standard image processing and viewing applications. OpenEXR's multiresolution and tiling capabilities may come in handy for high-resolution images; I guess that those capabilities are more important for stills than for moving images. Is it practical? An important question for a camera manufacturer would be how difficult it is to write and read OpenEXR files. Movie cameras must write the files in real time. In OpenEXR up to version 1.4 only uncompressed files could possibly be written in real time at the resolutions required for digital cinema. The bandwidth required to record high-resolution images with 48 bits or 24 bits per pixel (depending on whether RGB or luminance/chroma encoding is used) is substantial. OpenEXR's lossless compresssion schemes, PIZ, ZIP, etc. don't help here. The encoders are too slow to run in real time on affordable hardware. In addition, since the compression is lossless, the required recording bandwidth changes based on image content. In the worst case, compression does not shrink the image data at all, and the required bandwidth is the same as for uncompressed images. If a recording system can already handle this worst case, it might as well skip the compression step alltogether. In OpenEXR 1.5 we added B44 compression specifically to adress real-time playback of moving image sequences. B44 achieves a fixed compression rate of 2.28:1 or 4.57:1, depending on whether the images are stored in RGB or luminance/chroma format. Compared to uncompressed images, this reliably reduces the required bandwidth by a factor of 2.28 or 4.57, independent of image content. B44 compression is simple enough to allow decoding in real time. Using the open-source playexr program on an HP wx9400 workstation, we are able to play a 2k version of the STEM movie at 35 frames per second, directly from disk. The B44 encoder is more complex than the decoder, but not much. Building a system that records B44-compressed OpenEXR files in real time should be possible. From a camera manufacturer's point of view, the OpenEXR libraries may be dauntingly complex. However, a camera would most likely output images only in one fixed format, for example, B44-compressed luminance and chroma. All the code that deals with arbitrary sets of channels and arbitrary compression schemes could be thrown out. In addition, knowing the camera's internal workings, most of the frame buffer code that supports a multitude of in-memory image layouts could be discarded. Specializing the OpenEXR libraries for a specific camera and output format would allow very significant optimizations, affecting both code size and speed. Some of the code could probably be moved into specialized hardware, for example, an FPGA. The on-disk OpenEXR file format may be the source of another objection: In order to support random access to scan lines and tiles, a scan line or tile position table is inserted near the beginning of the file. With lossless compression the position of the scan lines or tiles within the file depends on image content, and the table entries are not known until the corresponding scan lines or tiles have been stored in the file. Once all the pixels have been saved in the file, the OpenEXR library must perform a seek operation back to the beginning of the table and write the table entries. If the files are recorded on a streaming medium, for example, a tape, a backwards seek operation is incompatible with real-time performance. This means that the entire file would have to be constructed in an in-memory buffer before it is appended to the output stream. However, for B44- compressed and uncompressed files, the scan line or tile positions are known in advance because they do not depend on image content. Seek operations are not necessary, an the file could be streamed out to the recording medium while it is being constructed. Overall I think it should be possible to use OpenEXR as a capture format, but as I mentioned above, there may be problems that I don't know about. Florian _______________________________________________ Openexr-user mailing list [email protected] http://lists.nongnu.org/mailman/listinfo/openexr-user
