On Wed, Jun 19, 2013 at 4:47 PM, Gregg Tavares <g...@google.com> wrote:
> > > > On Wed, Jun 19, 2013 at 4:03 PM, Rik Cabanier <caban...@gmail.com> wrote: > >> >> >> On Wed, Jun 19, 2013 at 3:24 PM, Gregg Tavares <g...@google.com> wrote: >> >>> >>> >>> >>> On Wed, Jun 19, 2013 at 3:13 PM, Rik Cabanier <caban...@gmail.com>wrote: >>> >>>> >>>> On Wed, Jun 19, 2013 at 2:47 PM, Gregg Tavares <g...@google.com> wrote: >>>> >>>>> In order for ImageBitmap to be useful for WebGL we need more options >>>>> >>>>> Specifically >>>>> >>>>> premultipliedAlpha: true/false (default true) >>>>> Nearly all GL games use non-premultipiled alpha textures. So all those >>>>> games people want to port to WebGL will require non-premultipied >>>>> textures. >>>>> Often in games the alpha might not even be used for alpha but rather >>>>> for >>>>> glow maps or specular maps or the other kinds of data. >>>>> >>>> >>>> When would premultipliedAlpha ever be true? >>>> 2D Canvas always works with non-premultiplied alpha in its APIs. >>>> >>> >>> AFAIK the canvas API expects all images to be premultiplied. Certainly >>> in WebKit and Blink images used in the canvas and displayed in the img tag >>> are loaded premultiplied which is why we had to add the option on WebGL >>> since we needed those images before they had lost data. >>> >>> >>>> >>>> >>>>> >>>>> flipY: true/false (default false) >>>>> Nearly all 3D modeling apps expect the bottom left pixel to be the >>>>> first >>>>> pixel in a texture so many 3D engines flip the textures on load. WebGL >>>>> provides this option but it takes time and memory to flip a large image >>>>> therefore it would be nice if that flip happened before the callback >>>>> from >>>>> ImageBitmap >>>>> >>>> >>>> Couldn't you just draw upside down? >>>> >>> >>> No, games often animate texture coordinates and other things making it >>> far more complicated. There are ways to work around this issue in code yes, >>> they often require a ton of work. >>> >>> Most professional game engines pre-process the textures and flip them >>> offline but that doesn't help when you're downloading models off say >>> http://sketchup.google.com/3dwarehouse/ >>> >>> >>>> >>>> >>>>> >>>>> colorspaceConversion: true/false (default true) >>>>> Some browsers apply color space conversion to match monitor settings. >>>>> That's fine for images with color but WebGL apps often load heightmaps, >>>>> normalmaps, lightmaps, global illumination maps and many other kinds of >>>>> data through images. If the browser applies a colorspace conversion the >>>>> data is not longer suitable for it's intended purpose therefore many >>>>> WebGL >>>>> apps turn off color conversions. As it is now, when an image is >>>>> uploaded to >>>>> WebGL, if colorspace conversion is >>>>> off< >>>>> http://www.khronos.org/registry/webgl/specs/latest/#PIXEL_STORAGE_PARAMETERS >>>>> >, >>>> >>>> >> OK, I see what you're trying to accomplish. You want to pass >> non-premultiplied data and color converted (from sRGB to monitor) pixels to >> WebGL >> I think your API looks fine, except that the defaults should all be >> false... >> > > Yes, that's what I meant. I think I choose bad labels. the intent of the > colorspaceConversion flag is > > colorspaceConversion: true = browser does whatever it thinks is best for > color images. > colorspaceConversion: false = give me the bits in the image file. Don't > manipulate them with either embedded color data or local machine gamma > corrections or anything else. > > So maybe a better name? > yeah. I'm still fuzzy about what exactly you're trying to do. Is this specified anywhere? > > for premultipiedAlpha again, maybe there are 3 options needed > > 1) do whatever is best for drawing with drawImage for perf > 2) give me the data with premutlipied alpha > 3) give me the data with non-premultied alpha. > I think that sounds good. That way implementations aren't forced to premultiply the alpha. Maybe not passing premultipiedAlpha in the dictionary (= leave it undefined) will signal this. > > It's possible that #1 is not needed as maybe GPU code can use different > blend modes for drawImage with non-premultpiled alpha. It's just my > understanding that at least in Chrome all images are loaded premultiplied. > In fact I don't think you can get non-premultipied data from canvas. At > least this does not make it appear that way > > c = document.createElement("canvas"); > ctx = c.getContext("2d"); > i = ctx.getImageData(0, 0, 1, 1); > i.data[0] = 255; > ctx.putImageData(i, 0, 0); > i2 = ctx.getImageData(0, 0, 1, 1); > console.log(i2.data[0]) // prints 0 on both FF and Chrome > > I mean I know you get unpremultiplied data from getIamgeData but the data > in the canvas is premultipied which means if alpha is zero you lose the RGB > data. In other words a round trip of putImageData, getImageData is lossy > for any non 255 alpha. Change the above to > > c = document.createElement("canvas"); > ctx = c.getContext("2d"); > i = ctx.getImageData(0, 0, 1, 1); > i.data[0] = 128; > i.data[3] = 1; > ctx.putImageData(i, 0, 0); > i2 = ctx.getImageData(0, 0, 1, 1); > console.log(i2.data[0]) // prints 255. > > This is because the data in the canvas is premutliplied and it's being > un-premultiplied when calling getImageData > Yes, the canvas spec calls this out: Due to the lossy nature of converting to and from premultiplied alpha color values, pixels that have just been set using putImageDataHD()<http://www.whatwg.org/specs/web-apps/current-work/multipage/the-canvas-element.html#dom-context-2d-putimagedatahd> might be returned to an equivalent getImageDataHD()<http://www.whatwg.org/specs/web-apps/current-work/multipage/the-canvas-element.html#dom-context-2d-getimagedatahd> as different values. I think this is really unfortunate... > > >> >>> >>>>> WebGL has to synchronously re-decode the image. It would be nice if >>>>> ImageBitmap could handle this case so it can decode the image without >>>>> applying any colorspace manipulations. >>>>> >>>> >>>> Shouldn't the color space conversion happen when the final canvas bit >>>> are blitted to the screen? >>>> It seems like you should never do it during compositing since you could >>>> get double conversions. >>>> >>> >>> Maybe but that's not relevant to ImageBitmap is it? The point here is we >>> want the ImageBitmap to give us the data in the format we need. It's >>> designed to be async so it can do this but it we can't prevent it from >>> applying colorspace conversions. >>> Some browsers did that for regular img tags which pointed out the >>> original problem. The browser can't guess how the image is going to be used >>> and since it's a lot of work to decode an image you'd like to be able to >>> tell it what you really need before it guesses wrong. >>> >>> >>>> >>>> >>>>> >>>>> If it was up to me I'd make createImageBitmap take on object with >>>>> properties so that new options can be added later as in >>>>> >>>>> createImageBitmap(src, callback, { >>>>> premultipliedAlpha: false, >>>>> colorspaceConversion: false, >>>>> x: 123, >>>>> }); >>>>> >>>>> But I'm not familiar if there is a common way to make APIs take a >>>>> options >>>>> like this except for the XHR way which is to create a request, set >>>>> properties on the request, and finally execute the request. >>>> >>>> >>>> Like Tab said, it's fine to implement it that way. >>>> Be aware that you might have to do some work in your idl compiler since >>>> I *think* there are no other APIs (in Blink) that take a dictionary. >>>> >>>> >>> >> >