Returning a series of unsigned 16 bit ints for a call with the type half feels like a nice middle ground. The consumer will have to know that halfs aren't natively supported in Python, and how to convert from unsigned short to half, but that doesn't feel like a large burden.
I can't speak to the expected behavior of the UNKNOWN in Python. I haven't used that path in Python or C++. HP On Thu, Feb 18, 2016 at 1:23 PM, Larry Gritz <[email protected]> wrote: > I don't have especially strong feelings about this one way or the other. > > Just returning a raw data byte array matches the C++ behavior more > closely, no argument there. > > On the "con" side, perhaps I was thinking of compatibility? We're really > talking about changing the meaning of oiio.UNKNOWN from "use spec.format" > to "return raw data", which differ in the case of mixed channel types. > > Are there Python programs out there that pass UNKNOWN (or pass nothing, > defaulting to UNKNOWN) and rely on getting the right kind of array back > that matches spec.format? > > > > > On Feb 18, 2016, at 12:58 PM, Andrew Gartner <[email protected]> > wrote: > > "Second, I could collapse 2a and 2b, and just say that if you ask for > UNKNOWN, you get an array of uint8 back with the native raw data" > > Just out of curiosity, what are the drawbacks to doing this? I admit I > like having some way of getting at the raw data at any time (hence my > original method of exposing the native calls). That allowed me to check my > imagespec and regardless of whether I had a mixed format image or all half > data I could get everything in one read call. Granted I'm used to keeping > track of and manipulating the strides of those arrays in bytes just out of > old habit (and C++ usage) so maybe I'm the minority opinion. > > Even so, your current thinking still works if that's where the consensus > is I'm happy to use it as such. > > Thanks again > > ~Andrew > > On Thu, Feb 18, 2016 at 12:21 PM, Larry Gritz <[email protected]> wrote: > >> I think that the only format that we can encounter as pixel data, which >> does not exist in Python arrays, is 'half'. >> >> So let me rephrase my current thinking: >> >> 1. If you ask for a specific type (except HALF), you'll get a Python >> array of that type holding the converted values. >> >> 2. Otherwise (i.e., you ask for UNKNOWN or HALF), you will get the native >> (raw) data. >> (a) If all channels are the same data type and it's anything but half, >> you'll get the data as a Python array of that type. >> (b) Otherwise (half, or mixed channel types), you'll get the data as a >> Python array of unsigned bytes. >> >> Note that (1) is the easy case to deal with: ask for the type you want, >> let it do the conversion. If you go for option (2) by asking for native >> data, you get a blob and it's up to you to figure out what to do with it. >> >> There are two other choices we could make. I'm not inclined to at the >> moment, but would be happy to do so if people think it's helpful. First, if >> you ask for HALF, I could have it return float. Second, I could collapse 2a >> and 2b, and just say that if you ask for UNKNOWN, you get an array of uint8 >> back with the native raw data, even if it happened to be all channels of >> the same type, a type that you could have made into a Python array of the >> right type. >> >> >> On Feb 17, 2016, at 11:16 PM, Haarm-Pieter Duiker < >> [email protected]> wrote: >> >> Picking this up a little later in the day. Sorry about that. Adding >> quotes from earlier in the thread just so it's clear what I'm responding to. >> >> The current status: >> " >> If you read_image(oiio.FLOAT) of a half image (on disk), you get floats >> back? >> " >> Yes. >> >> " >> But if you read_image(oiio.HALF) of a half image, you get what appears to >> be an array of floats, but they are actually packed half values? >> " >> Yes. >> >> The proposal: >> " >> 1. If you ask for a (non-UNKNOWN) format that exists in Python, it >> converts to and returns an array of that format. >> " >> This is the current behavior, no? >> >> " >> 2. If you ask for UNKNOWN, or a format that doesn't exist, it returns the >> raw data in an unsigned char array. >> " >> It feels like this is two proposals (Trying not to clash with your >> earlier 2a and 2b): >> 2c. If you ask for UNKNOWN, return raw data in an unsigned char array >> 2d. If you ask for a format that doesn't exist, return raw data in an >> unsigned char array >> >> 2c. feels right. It should work for the case of typical RGB or RGBA >> images but also for multi-layer EXRs. The consumer can convert the channels >> to their intended types using methods from the ImageSpec. I'd suggest that >> asking for UNKNOWN lead unequivocally to a raw unsigned char array. >> Supporting the special cases described in the 2a and 2b listed earlier >> would require additional logic on the consuming code side to account for >> those cases. Feels like a recipe for lots of brittle special case logic. >> >> 2d. is less clear. How is the change in behavior from returning real >> values for known types to returning raw char array data for unknown types >> signaled to the consumer? Is this still something that programmers have to >> just know a priori? How is this different from the current behavior? >> >> I suppose the list of types known to OIIO but not Python is finite and >> likely to shrink over time. Having special cases like we have in that >> example code, isn't such a big deal in the mean time, but then that's just >> saying the the current behavior is fine. >> >> Hope that's helpful in some way. Aside from agreeing that adding an >> UNKNOWN option is a good idea, we're still left without a good way to >> consume half data without accounting for it explicitly. >> >> HP >> >> >> >> >> >> >> >> >> >> >> >> >> >> On Wed, Feb 17, 2016 at 4:35 PM, Andrew Gartner <[email protected]> >> wrote: >> >>> That would certainly take care of things for me. Hopefully not too much >>> of an impact on others as well. >>> >>> ~Andrew >>> >>> >>> >>> On Wed, Feb 17, 2016 at 4:20 PM, Larry Gritz <[email protected]> wrote: >>> >>>> So I'm proposing: >>>> >>>> 1. If you ask for a (non-UNKNOWN) format that exists in Python, it >>>> converts to and returns an array of that format. >>>> >>>> 2. If you ask for UNKNOWN, or a format that doesn't exist, it returns >>>> the raw data in an unsigned char array. >>>> >>>> >>>> There is a variation: >>>> >>>> 2a. If you ask for UNKNOWN, and all channels are the same format and >>>> it's a type that exists in Python, return that type. >>>> 2b. If you ask for UNKNOWN and it's a "mixed type" file, or a single >>>> type but one that doesn't exist in Python, or the type you ask for doesn't >>>> exist in Python, return raw data packed into an unsigned char array. >>>> >>>> >>>> >>>> On Feb 17, 2016, at 4:10 PM, Andrew Gartner <[email protected]> >>>> wrote: >>>> >>>> Yea the C++ implementation works well with oiio.UNKNOWN, I kinda miss >>>> that in the python side to be honest. Right now it looks like things revert >>>> back to spec.format if oiio.UNKNOWN is supplied to read_scanlines, that can >>>> be problematic if you have multiple formats in a single image so I've >>>> avoided it. >>>> >>>> @Larry, to you question about returning an unsigned char array, I like >>>> the idea on principle in that it preserves the decoupling as you said. I'm >>>> wondering if there would be any weirdness if you had to grab multiple >>>> channels of an image that had different data types one of which isn't >>>> representable in python? Would it default to just unsigned char yet again >>>> in that case? >>>> >>>> @Haarm: interesting, I didn't realize they were concatenated/packed >>>> like that! I just saw the 'f' in the python array and assumed I was seeing >>>> promoted values :) I'm still scratching my head over the multiple format >>>> reads though, same as for Larry's idea. >>>> >>>> Thanks for the replies, Cheers, >>>> >>>> ~Andrew >>>> >>>> >>>> >>>> On Wed, Feb 17, 2016 at 3:49 PM, Haarm-Pieter Duiker < >>>> [email protected]> wrote: >>>> >>>>> If you're up for using numpy, this will get you the half float values >>>>> without too much extra work: >>>>> oiioFloats = inputImage.read_image(oiio.HALF) >>>>> oiioHalfs = np.frombuffer(np.getbuffer(np.float32(oiioFloats)), >>>>> dtype=np.float16) >>>>> >>>>> One note, the current OIIO Python implementation doesn't promote the >>>>> halfs to float on read. The 'float' values in the returned buffer are >>>>> actually each two concatenated half values, and the float buffer will have >>>>> half as many entries as you would expect. >>>>> >>>>> Example usage for reading here: >>>>> >>>>> https://github.com/hpd/CLF/blob/master/python/aces/filterImageWithCLF.py#L126 >>>>> and the reverse for writing: >>>>> >>>>> https://github.com/hpd/CLF/blob/master/python/aces/filterImageWithCLF.py#L193 >>>>> >>>>> HP >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> On Wed, Feb 17, 2016 at 3:25 PM, Larry Gritz <[email protected]> >>>>> wrote: >>>>> >>>>>> In C++, you can just call read_scanlines and pass format=UNKNOWN to >>>>>> get back the raw data in its original format. >>>>>> >>>>>> The problem is that in Python, there is no 'half' so it's not quite >>>>>> sure what to return. >>>>>> >>>>>> I kinda like the decoupling of the raw reads (read_native_*) which >>>>>> are the part overloaded by the individual format readers, from the >>>>>> app-callable read_*. So perhaps rather than exposing read_native_*, we >>>>>> should just modify the Python bindings for read_* to notice that if the >>>>>> native raw data is not a type representable in Python, to return it as an >>>>>> unsigned character array? >>>>>> >>>>>> >>>>>> > On Feb 17, 2016, at 2:55 PM, Andrew Gartner < >>>>>> [email protected]> wrote: >>>>>> > >>>>>> > Hey all, >>>>>> > >>>>>> > Apologies if this has come up before, but I'm curious if anyone had >>>>>> considered exposing ImageInput.read_native_scanlines() on the python side >>>>>> before. The reason I ask is mainly because the half datatype doesn't >>>>>> exist >>>>>> in the native python array class which OIIO uses for python reads. >>>>>> Currently the python array will punt and for anything to float (which I'd >>>>>> rather avoid). >>>>>> > >>>>>> > I had put together an implementation in OIIO 1.5 that simply took >>>>>> the pixel size as a parameter and exposed read_native_scanlines that way >>>>>> and that allowed me to get the right data properly into either numpy or a >>>>>> raw char python array. However, I'd rather not be forked off like that as >>>>>> it's a headache trying to remain current with the mainline, plus others >>>>>> may >>>>>> find it useful. >>>>>> > >>>>>> > Does anyone think exposing the function in general makes sense? I'm >>>>>> happy to send the implementation if anyone cares to see it as well. >>>>>> > >>>>>> > Cheers, >>>>>> > >>>>>> > ~Andrew >>>>>> > >>>>>> >>>>>> -- >>>>>> Larry Gritz >>>>>> [email protected] >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> Oiio-dev mailing list >>>>>> [email protected] >>>>>> http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org >>>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> Oiio-dev mailing list >>>>> [email protected] >>>>> http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org >>>>> >>>>> >>>> _______________________________________________ >>>> Oiio-dev mailing list >>>> [email protected] >>>> http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org >>>> >>>> >>>> -- >>>> Larry Gritz >>>> [email protected] >>>> >>>> >>>> >>>> _______________________________________________ >>>> Oiio-dev mailing list >>>> [email protected] >>>> http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org >>>> >>>> >>> >>> _______________________________________________ >>> Oiio-dev mailing list >>> [email protected] >>> http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org >>> >>> >> _______________________________________________ >> Oiio-dev mailing list >> [email protected] >> http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org >> >> >> -- >> Larry Gritz >> [email protected] >> >> >> >> _______________________________________________ >> Oiio-dev mailing list >> [email protected] >> http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org >> >> > _______________________________________________ > Oiio-dev mailing list > [email protected] > http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org > > > -- > Larry Gritz > [email protected] > > > > _______________________________________________ > Oiio-dev mailing list > [email protected] > http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org > >
_______________________________________________ Oiio-dev mailing list [email protected] http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org
