Re: [whatwg] Canvas image to blob/dataurl within Worker
On Tue, Apr 14, 2015 at 10:53 PM, Robert O'Callahan wrote: > I guess there are really two different sets of use-cases: > 1) Use-cases where the ImageBitmap is sized to fill a particular area of > the screen. > 2) Use-cases where the ImageBitmap is sized subject to some other > constraints, e.g. you're processing an existing image. > > For #2, you want the ImageBitmap's size to be the intrinsic size of the > element. > > For #1, you don't care about having an intrinsic size. Instead the > element's size is set by CSS and you want the ImageBitmap to fill the > element, and you want to be able to use renderedPixelWidth/Height to size > the ImageBitmap. However, for these use-cases I think the behavior > currently specced in the OffscreenBitmap proposal doesn't make sense: > // The ImageBitmap, when displayed, is clipped to the rectangle > // defined by the canvas's instrinsic width and height. Pixels that > // would be covered by the canvas's bitmap which are not covered by > // the supplied ImageBitmap are rendered transparent black. > I'm not sure what this means now. Suppose the canvas element has CSS > "width:100px; height:100px" and the canvas rendered size is 200x200. If the > application creates a 200x200 ImageBitmap and installs it in the canvas, as > specced isn't this just going to display some 100x100 subrectangle of the > ImageBitmap, cropping out the rest? I believe this is *intending* to say that ImageBitmap is just spammed into a directly, matching pixels 1-to-1. If the ImageBitmap overflows the in some dimension, the extra is discarded; if it underflows, the underflow is painted transparent black. CSS has no effect on this. This needs to be stated more explicitly, of course. The idea is that this isn't drawing an image into the canvas, it's like loading an ImageData. > What if instead of a new context type, we had a method on the canvas > element itself to install a new ImageBitmap, which replaces the backbuffer > and sets the intrinsic size of the canvas element to the ImageBitmap's > size, so that 'width' and 'height' are thereafter ignored? Then if the > canvas has non-auto CSS 'width' and 'height', the image buffer will be > scaled to fit the CSS size and the example above will work as expected. CSS > object-fit will then also work, so if the ImageBitmap producer lags behind > canvas resizing for some reason, authors can use object-fit:contain (or > cover) to display the mis-sized images in a reasonable way. (object-fit is > also important for the case where a Worker is presenting images through an > OffscreenCanvas and its image production may lag behind resizing of the > canvas element on another thread; we can pass the object-fit value through > to the compositor to ensure the temporary results look OK.) This works for me; it avoids the author having to coordinate between scripts and predict the size of the image beforehand. ~TJ
Re: [whatwg] Canvas image to blob/dataurl within Worker
I guess there are really two different sets of use-cases: 1) Use-cases where the ImageBitmap is sized to fill a particular area of the screen. 2) Use-cases where the ImageBitmap is sized subject to some other constraints, e.g. you're processing an existing image. For #2, you want the ImageBitmap's size to be the intrinsic size of the element. For #1, you don't care about having an intrinsic size. Instead the element's size is set by CSS and you want the ImageBitmap to fill the element, and you want to be able to use renderedPixelWidth/Height to size the ImageBitmap. However, for these use-cases I think the behavior currently specced in the OffscreenBitmap proposal doesn't make sense: // The ImageBitmap, when displayed, is clipped to the rectangle // defined by the canvas's instrinsic width and height. Pixels that // would be covered by the canvas's bitmap which are not covered by // the supplied ImageBitmap are rendered transparent black. I'm not sure what this means now. Suppose the canvas element has CSS "width:100px; height:100px" and the canvas rendered size is 200x200. If the application creates a 200x200 ImageBitmap and installs it in the canvas, as specced isn't this just going to display some 100x100 subrectangle of the ImageBitmap, cropping out the rest? What if instead of a new context type, we had a method on the canvas element itself to install a new ImageBitmap, which replaces the backbuffer and sets the intrinsic size of the canvas element to the ImageBitmap's size, so that 'width' and 'height' are thereafter ignored? Then if the canvas has non-auto CSS 'width' and 'height', the image buffer will be scaled to fit the CSS size and the example above will work as expected. CSS object-fit will then also work, so if the ImageBitmap producer lags behind canvas resizing for some reason, authors can use object-fit:contain (or cover) to display the mis-sized images in a reasonable way. (object-fit is also important for the case where a Worker is presenting images through an OffscreenCanvas and its image production may lag behind resizing of the canvas element on another thread; we can pass the object-fit value through to the compositor to ensure the temporary results look OK.) Rob -- oIo otoeololo oyooouo otohoaoto oaonoyooonoeo owohooo oioso oaonogoroyo owoiotoho oao oboroootohoeoro oooro osoiosotoeoro owoiololo oboeo osouobojoeocoto otooo ojouodogomoeonoto.o oAogoaoiono,o oaonoyooonoeo owohooo osoaoyoso otooo oao oboroootohoeoro oooro osoiosotoeoro,o o‘oRoaocoao,o’o oioso oaonosowoeoroaoboloeo otooo otohoeo ocooouoroto.o oAonodo oaonoyooonoeo owohooo osoaoyoso,o o‘oYooouo ofolo!o’o owoiololo oboeo oiono odoaonogoeoro ooofo otohoeo ofoioroeo ooofo ohoeololo.
Re: [whatwg] Canvas image to blob/dataurl within Worker
On Wed, Apr 15, 2015 at 1:15 PM, Kenneth Russell wrote: > If repeatedly transferring ImageBitmaps of the same size into the same > won't cause repeated re-layouts, that alleviates my concern > about efficiency of using images as the output path. I expect that a > single OffscreenCanvas will be used to produce output to multiple > areas on the page (whether into images or canvases) but that it will > be sized large enough so that all of the ImageBitmaps it produces will > cover the largest of those areas, avoiding repeatedly resizing the > OffscreenCanvas. > So you want to produce multiple ImageBitmaps of the same (largest) size and then render them to multiple elements in the page, each element cropping its ImageBitmap to a (potentially) different size? Isn't that going to be wasteful of memory? There is one other problem which has come up repeatedly in canvas > applications: needing to be able to accurately measure the number of > device pixels covered by a canvas, so pixel-accurate rendering can be > done. https://wiki.whatwg.org/wiki/CanvasRenderedPixelSize addresses > this and it's currently being implemented in Chromium. There is no > such mechanism for images. It would be necessary to understand exactly > how many device pixels the output image is covering in the document so > that the OffscreenCanvas can be sized appropriately. Using a canvas > element for the display of these ImageBitmaps avoids this problem. It > also makes the API more symmetric; a canvas-like object produces the > ImageBitmap, and a canvas element is used to view it. Do you have any > suggestion for how this issue would be solved with ? > We could support renderedPixelWidth/Height on as well as . But this is a valid argument in favor of rendering those ImageBitmaps through . Rob -- oIo otoeololo oyooouo otohoaoto oaonoyooonoeo owohooo oioso oaonogoroyo owoiotoho oao oboroootohoeoro oooro osoiosotoeoro owoiololo oboeo osouobojoeocoto otooo ojouodogomoeonoto.o oAogoaoiono,o oaonoyooonoeo owohooo osoaoyoso otooo oao oboroootohoeoro oooro osoiosotoeoro,o o‘oRoaocoao,o’o oioso oaonosowoeoroaoboloeo otooo otohoeo ocooouoroto.o oAonodo oaonoyooonoeo owohooo osoaoyoso,o o‘oYooouo ofolo!o’o owoiololo oboeo oiono odoaonogoeoro ooofo otohoeo ofoioroeo ooofo ohoeololo.
Re: [whatwg] Canvas image to blob/dataurl within Worker
On Sun, Apr 12, 2015 at 2:41 PM, Robert O'Callahan wrote: > On Sat, Apr 11, 2015 at 1:49 PM, Kenneth Russell wrote: >> >> Suppose src="myimage.png" is set on an image element and then, while >> it is downloading, an ImageBitmap is transferred into it. > > > This should behave as if "src" was removed just before the ImageBitmap is > transferred into it. > > In spec terms, I think we'd define a new state flag "has an ImageBitmap". > Transferring an ImageBitmap into an element would set that flag and > trigger "update the image data". After step 2 of "update the image data", > when the flag is set, we'd synchronously do what "update the image data" > would do with no source: clear "last selected source" (step 4), and do steps > 9.1 and 9.2 (but never fire the "error" event). Then, transferring the > ImageBitmap would do the "load complete" work of step 14: set the > "completely available" state, set the image data, and add the image to the > list of available images. > > Note that this is also exactly what we'd need to do to support srcObject on > , which would be nice to have so you can render a Blob with an > HTMLImageElement without dealing with the annoying createObjectURL lifetime > issues. > > Thus: > >> 1. What is displayed in the webpage as myimage.png is downloading? > > > As normal before an ImageBitmap is transferred, the ImageBitmap afterward. > >> 2. Do the downloaded bits overwrite what you transferred or do we stop >> the download? > > > Stop the download. > >> >> 3. Are onload and onerror events fired? This question applies both to >> the in-progress download and to the transferred-in ImageBitmap. > > > No. > >> >> 4. What should the 'complete' property and 'currentSrc' attribute reflect? > > > True and the empty string respectively. > >> >> 5. Can the developer go back to having the img element contain what >> was set on 'src' and not what was transferred in? > > > As written, yes, by performing another relevant mutation to trigger "update > the image data" again. > >> srcset and the picture element make the situation even more complex. > > > I think the above covers it. Thanks for these answers. They sound reasonable to me from a complexity standpoint, though the needed updates to the tag's spec sound a little involved. I have another concern which is mentioned below. >> In comparison, displaying ImageBitmaps with a custom canvas context >> offers simple semantics and avoids other problems raised on this >> thread like requiring a layout, or firing events, upon recipt of a new >> ImageBitmap. > > > AFAIK there aren't any issues with events. The issue with layout is simply > whether or not we want the element receiving an ImageBitmap to have the > expected intrinsic size. AFAICT for unnecessary layouts to occur would > require repeated ImageBitmap transfers into the same HTMLImageElement, where > the ImageBitmaps have varying sizes and the element has auto 'width' or > 'height', but the author actually intends that the images all be scaled or > cropped to the same size, and doesn't notice this is not happening. Is there > a more realistic scenario? If repeatedly transferring ImageBitmaps of the same size into the same won't cause repeated re-layouts, that alleviates my concern about efficiency of using images as the output path. I expect that a single OffscreenCanvas will be used to produce output to multiple areas on the page (whether into images or canvases) but that it will be sized large enough so that all of the ImageBitmaps it produces will cover the largest of those areas, avoiding repeatedly resizing the OffscreenCanvas. There is one other problem which has come up repeatedly in canvas applications: needing to be able to accurately measure the number of device pixels covered by a canvas, so pixel-accurate rendering can be done. https://wiki.whatwg.org/wiki/CanvasRenderedPixelSize addresses this and it's currently being implemented in Chromium. There is no such mechanism for images. It would be necessary to understand exactly how many device pixels the output image is covering in the document so that the OffscreenCanvas can be sized appropriately. Using a canvas element for the display of these ImageBitmaps avoids this problem. It also makes the API more symmetric; a canvas-like object produces the ImageBitmap, and a canvas element is used to view it. Do you have any suggestion for how this issue would be solved with ? -Ken
Re: [whatwg] Canvas image to blob/dataurl within Worker
On Tue, Apr 14, 2015 at 9:03 AM, Justin Novosad wrote: > On Sun, Apr 12, 2015 at 5:41 PM, Robert O'Callahan > wrote: > > > On Sat, Apr 11, 2015 at 1:49 PM, Kenneth Russell wrote: > > > >> > >> 3. Are onload and onerror events fired? This question applies both to > >> the in-progress download and to the transferred-in ImageBitmap. > >> > > > > No. > > > > From the web developer's perspective wouldn't that lead to > non-deterministic behavior? What I mean is that if the onload listener was > not yet called at the time an ImageBitmap is transferred in, the onload > listener may still get called depending on whether the onload event was > already dispatched when the ImageBitmap transfer occurred. That's a race. > This makes me wonder... is there a precedent for recalling dispatched > events? > Yes. Media elements, for example, dispatch tasks to an element-specific task source, which is emptied when a new load starts: https://html.spec.whatwg.org/multipage/embedded-content.html#dom-media-load Having said that, the sort of races that you've identified are endemic to the Web platform. For example the spec (and implementations) allow an "load" event to fire after a different image has already started loading in the . So I don't think there's a new problem here. Having said that^2, I should probably amend my suggestion so that we dispatch the "error" event per section 9.2, with the condition amended to "if the element has a src attribute or a srcset attribute or a parent that is a picture element *and the has-an-ImageBitmap flag was not already set*", so dispatch of events for non-ImageBitmap-loads is not affected. Rob -- oIo otoeololo oyooouo otohoaoto oaonoyooonoeo owohooo oioso oaonogoroyo owoiotoho oao oboroootohoeoro oooro osoiosotoeoro owoiololo oboeo osouobojoeocoto otooo ojouodogomoeonoto.o oAogoaoiono,o oaonoyooonoeo owohooo osoaoyoso otooo oao oboroootohoeoro oooro osoiosotoeoro,o o‘oRoaocoao,o’o oioso oaonosowoeoroaoboloeo otooo otohoeo ocooouoroto.o oAonodo oaonoyooonoeo owohooo osoaoyoso,o o‘oYooouo ofolo!o’o owoiololo oboeo oiono odoaonogoeoro ooofo otohoeo ofoioroeo ooofo ohoeololo.
Re: [whatwg] Canvas image to blob/dataurl within Worker
On Sun, Apr 12, 2015 at 5:41 PM, Robert O'Callahan wrote: > On Sat, Apr 11, 2015 at 1:49 PM, Kenneth Russell wrote: > >> >> 3. Are onload and onerror events fired? This question applies both to >> the in-progress download and to the transferred-in ImageBitmap. >> > > No. > >From the web developer's perspective wouldn't that lead to non-deterministic behavior? What I mean is that if the onload listener was not yet called at the time an ImageBitmap is transferred in, the onload listener may still get called depending on whether the onload event was already dispatched when the ImageBitmap transfer occurred. That's a race. This makes me wonder... is there a precedent for recalling dispatched events?
Re: [whatwg] Canvas image to blob/dataurl within Worker
On Mon, Apr 13, 2015 at 9:41 AM, Robert O'Callahan wrote: > On Sat, Apr 11, 2015 at 1:49 PM, Kenneth Russell wrote: > >> 5. Can the developer go back to having the img element contain what >> was set on 'src' and not what was transferred in? >> > > As written, yes, by performing another relevant mutation to trigger > "update the image data" again. > Sorry, this is incorrect. As written, no --- it's not possible to stop displaying an ImageBitmap and return to displaying 'src'. We could provide API for that if needed, though I don't think it is. Rob -- oIo otoeololo oyooouo otohoaoto oaonoyooonoeo owohooo oioso oaonogoroyo owoiotoho oao oboroootohoeoro oooro osoiosotoeoro owoiololo oboeo osouobojoeocoto otooo ojouodogomoeonoto.o oAogoaoiono,o oaonoyooonoeo owohooo osoaoyoso otooo oao oboroootohoeoro oooro osoiosotoeoro,o o‘oRoaocoao,o’o oioso oaonosowoeoroaoboloeo otooo otohoeo ocooouoroto.o oAonodo oaonoyooonoeo owohooo osoaoyoso,o o‘oYooouo ofolo!o’o owoiololo oboeo oiono odoaonogoeoro ooofo otohoeo ofoioroeo ooofo ohoeololo.
Re: [whatwg] Canvas image to blob/dataurl within Worker
On Sat, Apr 11, 2015 at 1:49 PM, Kenneth Russell wrote: > Suppose src="myimage.png" is set on an image element and then, while > it is downloading, an ImageBitmap is transferred into it. > This should behave as if "src" was removed just before the ImageBitmap is transferred into it. In spec terms, I think we'd define a new state flag "has an ImageBitmap". Transferring an ImageBitmap into an element would set that flag and trigger "update the image data". After step 2 of "update the image data", when the flag is set, we'd synchronously do what "update the image data" would do with no source: clear "last selected source" (step 4), and do steps 9.1 and 9.2 (but never fire the "error" event). Then, transferring the ImageBitmap would do the "load complete" work of step 14: set the "completely available" state, set the image data, and add the image to the list of available images. Note that this is also exactly what we'd need to do to support srcObject on , which would be nice to have so you can render a Blob with an HTMLImageElement without dealing with the annoying createObjectURL lifetime issues. Thus: 1. What is displayed in the webpage as myimage.png is downloading? > As normal before an ImageBitmap is transferred, the ImageBitmap afterward. 2. Do the downloaded bits overwrite what you transferred or do we stop > the download? > Stop the download. > 3. Are onload and onerror events fired? This question applies both to > the in-progress download and to the transferred-in ImageBitmap. > No. > 4. What should the 'complete' property and 'currentSrc' attribute reflect? > True and the empty string respectively. > 5. Can the developer go back to having the img element contain what > was set on 'src' and not what was transferred in? > As written, yes, by performing another relevant mutation to trigger "update the image data" again. srcset and the picture element make the situation even more complex. > I think the above covers it. > In comparison, displaying ImageBitmaps with a custom canvas context > offers simple semantics and avoids other problems raised on this > thread like requiring a layout, or firing events, upon recipt of a new > ImageBitmap. > AFAIK there aren't any issues with events. The issue with layout is simply whether or not we want the element receiving an ImageBitmap to have the expected intrinsic size. AFAICT for unnecessary layouts to occur would require repeated ImageBitmap transfers into the same HTMLImageElement, where the ImageBitmaps have varying sizes and the element has auto 'width' or 'height', but the author actually intends that the images all be scaled or cropped to the same size, and doesn't notice this is not happening. Is there a more realistic scenario? Rob -- oIo otoeololo oyooouo otohoaoto oaonoyooonoeo owohooo oioso oaonogoroyo owoiotoho oao oboroootohoeoro oooro osoiosotoeoro owoiololo oboeo osouobojoeocoto otooo ojouodogomoeonoto.o oAogoaoiono,o oaonoyooonoeo owohooo osoaoyoso otooo oao oboroootohoeoro oooro osoiosotoeoro,o o‘oRoaocoao,o’o oioso oaonosowoeoroaoboloeo otooo otohoeo ocooouoroto.o oAonodo oaonoyooonoeo owohooo osoaoyoso,o o‘oYooouo ofolo!o’o owoiololo oboeo oiono odoaonogoeoro ooofo otohoeo ofoioroeo ooofo ohoeololo.
Re: [whatwg] Canvas image to blob/dataurl within Worker
On Fri, Apr 10, 2015 at 2:33 PM, Robert O'Callahan wrote: > On Sat, Apr 11, 2015 at 2:18 AM, Justin Novosad wrote: >> >> Riddle me this: What would be the value an HTMLImageElement's src >> attribute >> after an ImageBitmap is transferred in? A data URL? What if the >> ImageBitmap >> was sourced from an actual resource with a URL, would we pipe that >> through? In cases where there is no tractable URL (ex: ImageBitmap was >> grabbed fram a tainted canvas), then what? > > > We have the same issue for HTMLMediaElement.srcObject: > http://dev.w3.org/2011/webrtc/editor/archives/20140619/getusermedia.html#widl-HTMLMediaElement-srcObject > The current approach is to leave 'src' unchanged, and the spec says that if > there is a current srcObject (or ImageBitmap in our case), render that and > ignore 'src'. Here are some more questions, some of which are from a colleague. I'm not trying to claim credit for thinking of them, but want to make it clear that there are several more ambiguities with attempting to use an image element to display ImageBitmaps. Suppose src="myimage.png" is set on an image element and then, while it is downloading, an ImageBitmap is transferred into it. 1. What is displayed in the webpage as myimage.png is downloading? 2. Do the downloaded bits overwrite what you transferred or do we stop the download? 3. Are onload and onerror events fired? This question applies both to the in-progress download and to the transferred-in ImageBitmap. 4. What should the 'complete' property and 'currentSrc' attribute reflect? 5. Can the developer go back to having the img element contain what was set on 'src' and not what was transferred in? srcset and the picture element make the situation even more complex. In comparison, displaying ImageBitmaps with a custom canvas context offers simple semantics and avoids other problems raised on this thread like requiring a layout, or firing events, upon recipt of a new ImageBitmap. -Ken
Re: [whatwg] Canvas image to blob/dataurl within Worker
On Thu, Apr 9, 2015 at 6:25 PM, Tab Atkins Jr. wrote: > On Thu, Apr 9, 2015 at 5:23 PM, Kenneth Russell wrote: >> 1. An image element that didn't have a width or height explicitly >> specified in the markup has an ImageBitmap transferred in. Will its >> width and height attributes change? Will layout have to occur on the >> page? > > Its width and height *attributes* won't change, but its *intrinsic* > width and height will, to match the bitmap and density. (Btw, the > ImageBitmap feature needs to have a notion of density.) I think the idea of introducing "density" to ImageBitmap needs a completely separate thread. I don't understand all the ramifications, but it looks to me like the fact that ImageBitmap's width and height are specified in CSS pixels makes it impossible for users' code to compute the exact number of pixels in the ImageBitmap. This is still a problem for developers using and https://wiki.whatwg.org/wiki/CanvasRenderedPixelSize is intended to solve it. > Yes, layout occurs, assuming the size changed. > >> 2. The same image element has another ImageBitmap transferred in which >> has a different width and height. Do the image's width and height >> change? Will layout have to occur on the page? > > Yes and yes, assuming the size changed. This is problematic. It's essential that displaying a newly produced ImageBitmap be a cheap operation. Layout is expensive. >> When an image's src is set to a URL and its width and height >> attributes aren't set, the page is laid out again when the image is >> loaded. For acceptable performance it's important that this not happen >> when displaying a new ImageBitmap. Using a new canvas context type >> solves this problem. The OffscreenCanvas proposal defines the >> ImageBitmap as being rendered at the upper left corner of the canvas's >> box with no scaling applied, and clipped to the box. It's not as >> flexible as having the object-fit and object-position CSS properties >> available, but will give developers explicit control over when they >> want layout to happen, and still let them display their content as >> they wish. > > Why not just use object-fit/position? I think we support those. No > need to make new weird behavior when you can just opt into the desired > behavior with existing standardized stuff. Can the right settings of object-fit and object-position avoid the layouts in the abovementioned scenarios? -Ken
Re: [whatwg] Canvas image to blob/dataurl within Worker
On Sat, Apr 11, 2015 at 2:18 AM, Justin Novosad wrote: > Riddle me this: What would be the value an HTMLImageElement's src attribute > after an ImageBitmap is transferred in? A data URL? What if the ImageBitmap > was sourced from an actual resource with a URL, would we pipe that > through? In cases where there is no tractable URL (ex: ImageBitmap was > grabbed fram a tainted canvas), then what? > We have the same issue for HTMLMediaElement.srcObject: http://dev.w3.org/2011/webrtc/editor/archives/20140619/getusermedia.html#widl-HTMLMediaElement-srcObject The current approach is to leave 'src' unchanged, and the spec says that if there is a current srcObject (or ImageBitmap in our case), render that and ignore 'src'. Rob -- oIo otoeololo oyooouo otohoaoto oaonoyooonoeo owohooo oioso oaonogoroyo owoiotoho oao oboroootohoeoro oooro osoiosotoeoro owoiololo oboeo osouobojoeocoto otooo ojouodogomoeonoto.o oAogoaoiono,o oaonoyooonoeo owohooo osoaoyoso otooo oao oboroootohoeoro oooro osoiosotoeoro,o o‘oRoaocoao,o’o oioso oaonosowoeoroaoboloeo otooo otohoeo ocooouoroto.o oAonodo oaonoyooonoeo owohooo osoaoyoso,o o‘oYooouo ofolo!o’o owoiololo oboeo oiono odoaonogoeoro ooofo otohoeo ofoioroeo ooofo ohoeololo.
Re: [whatwg] Canvas image to blob/dataurl within Worker
On Thu, Apr 9, 2015 at 9:44 AM, Kenneth Russell wrote: > On Tue, Apr 7, 2015 at 6:42 PM, Robert O'Callahan > wrote: > > As far as I can tell, the only significant difference between > > this and WorkerCanvas is using an HTMLCanvasElement "bitmaprenderer" as a > > zero-copy way to render an ImageBitmap, instead of HTMLImageElement. > > That's the main difference. Another is that OffscreenCanvas can be > directly instantiated from both the main thread and workers. > Oh, it was always intended that WorkerCanvas be instantiable from a Worker, it just wasn't written down (and "Exposed" hadn't been invented yet :-).) > >Can you explain what the problem was with using HTMLImageElement? > > One browser implementer pointed out that HTMLImageElement has several > complex internal states related to loading, and they do not want to > conflate that with the display of ImageBitmaps. I've documented this > in the OffscreenCanvas proposal. (I originally thought Canvas had more > complex internal state, but now think that it can be easier to define > new behavior against a new canvas context than .) > I like your original thought better :-). I don't think circumventing the HTMLImageElement states is an issue, in Gecko at least. To me it seems desirable for an element displaying an ImageBitmap to get its intrinsic size from the ImageBitmap, which fits better with HTMLImageElement than HTMLCanvasElement. For example this would make object-fit and object-position useful on such elements. Rob -- oIo otoeololo oyooouo otohoaoto oaonoyooonoeo owohooo oioso oaonogoroyo owoiotoho oao oboroootohoeoro oooro osoiosotoeoro owoiololo oboeo osouobojoeocoto otooo ojouodogomoeonoto.o oAogoaoiono,o oaonoyooonoeo owohooo osoaoyoso otooo oao oboroootohoeoro oooro osoiosotoeoro,o o‘oRoaocoao,o’o oioso oaonosowoeoroaoboloeo otooo otohoeo ocooouoroto.o oAonodo oaonoyooonoeo owohooo osoaoyoso,o o‘oYooouo ofolo!o’o owoiololo oboeo oiono odoaonogoeoro ooofo otohoeo ofoioroeo ooofo ohoeololo.
Re: [whatwg] Canvas image to blob/dataurl within Worker
On Wed, Apr 8, 2015 at 8:46 AM, Ashley Gullen wrote: > (sorry for double email) Just one question about how requestAnimationFrame > is expected to work. Could we get rAF added to workers? Would it be able to > simply fire whenever it does on the main thread, as if it postMessage'd to > the worker every time it was called but without the extra postMessage > latency? To do that I guess workers need to be associated with a window. For > normal workers this seems fine but if OffscreenCanvas is expected to work in > shared or service workers this is a little tricker to define. Perhaps > requestAnimationFrame could be a method on OffscreenCanvas and it fires > based on the window the HTMLCanvasElement it came from is currently in? Then > you have a convenient way to synchronise rendering with the window it will > ultimately be displayed in, without having to know in advance which window > that is. A usable requestAnimationFrame will be needed in workers in order to reliably do animation. I'm hoping that that can be a somewhat orthogonal proposal to this one. There are two ways to use an OffscreenCanvas and only one of them causes rendering results to be pushed implicitly to an existing canvas on the main thread. (The other way involves directly instantiating an OffscreenCanvas and using postMessage to do some work on the main thread to display its results.) requestAnimationFrame is typically triggered on the main thread as early as possible after the previous frame's vsync interval, and I'd expect that other workers' requestAnimationFrame callbacks should be triggered in the same way. Dedicated workers are always associated with a particular page which is displayed in a window, so the connection to an on-screen window should always be there. -Ken > Ashley > > > > On 8 April 2015 at 16:36, Ashley Gullen wrote: >> >> This looks like it will cover running a HTML5 game engine from a worker >> very nicely and with little performance overhead. Good stuff! >> >> Ashley >> >> >> On 8 April 2015 at 00:41, Kenneth Russell wrote: >>> >>> On Wed, Mar 25, 2015 at 6:41 PM, Kenneth Russell wrote: >>> > On Wed, Mar 25, 2015 at 1:22 PM, Robert O'Callahan >>> > wrote: >>> >> On Wed, Mar 25, 2015 at 11:41 PM, Anne van Kesteren >>> >> wrote: >>> >>> >>> >>> On Fri, Mar 20, 2015 at 11:15 PM, Robert O'Callahan >>> >>> wrote: >>> >>> > My understanding is that the current consensus proposal for canvas >>> >>> > in >>> >>> > Workers is not what's in the spec, but this: >>> >>> > https://wiki.whatwg.org/wiki/WorkerCanvas >>> >>> > See "Canvas in Workers" threads from October 2013 for the >>> >>> > discussion. >>> >>> > svn >>> >>> > is failing me but the CanvasProxy proposal in the spec definitely >>> >>> > predates >>> >>> > those threads. >>> >>> > >>> >>> > Ian, unless I'm wrong, it would be helpful to remove the >>> >>> > CanvasProxy >>> >>> > stuff >>> >>> > from the spec to avoid confusion. >>> >>> > >>> >>> > That proposal contains WorkerCanvas.toBlob, which needs to be >>> >>> > updated to >>> >>> > use promises. >>> >>> >>> >>> There's also https://wiki.whatwg.org/wiki/WorkerCanvas2 it seems. It >>> >>> would be interesting to know what the latest on this is. >>> >> >>> >> >>> >> Right now that appears to be just a copy of WorkerCanvas with some >>> >> boilerplate text added. I assume someone's working on it and will >>> >> mention it >>> >> on this list if/when they're ready to discuss it :-). >>> > >>> > Yes, apologies for letting it sit there in an incomplete state. >>> > Recently feedback from more browser implementers was gathered about >>> > the WorkerCanvas proposal. I will assemble it into WorkerCanvas2 and >>> > follow up on this thread in a day or two. >>> >>> I apologize for taking so long to update this proposal, but it's now >>> in a reasonable state: >>> https://wiki.whatwg.org/wiki/OffscreenCanvas >>> >>> (Renamed per feedback from Anne -- thanks.) >>> >>> Please offer your feedback. Multiple browser implementers agreed on >>> this form of the API at a recent face-to-face meeting of the WebGL >>> working group, and the proposal should be able to address multiple use >>> cases. I'm very interested in prototyping it to see if it can offer >>> better parallelism. >>> >>> -Ken >>> >>> >>> >>> > -Ken >>> > >>> > >>> >> Rob >>> >> -- >>> >> oIo otoeololo oyooouo otohoaoto oaonoyooonoeo owohooo oioso >>> >> oaonogoroyo >>> >> owoiotoho oao oboroootohoeoro oooro osoiosotoeoro owoiololo oboeo >>> >> osouobojoeocoto otooo ojouodogomoeonoto.o oAogoaoiono,o oaonoyooonoeo >>> >> owohooo >>> >> osoaoyoso otooo oao oboroootohoeoro oooro osoiosotoeoro,o >>> >> o‘oRoaocoao,o’o >>> >> oioso >>> >> oaonosowoeoroaoboloeo otooo otohoeo ocooouoroto.o oAonodo >>> >> oaonoyooonoeo >>> >> owohooo >>> >> osoaoyoso,o o‘oYooouo ofolo!o’o owoiololo oboeo oiono >>> >> odoaonogoeoro >>> >> ooofo >>> >> otohoeo ofoioroeo ooofo ohoeololo. >> >> >
Re: [whatwg] Canvas image to blob/dataurl within Worker
On Tue, Apr 7, 2015 at 6:42 PM, Robert O'Callahan wrote: > On Wed, Apr 8, 2015 at 11:41 AM, Kenneth Russell wrote: >> >> I apologize for taking so long to update this proposal, but it's now >> in a reasonable state: >> https://wiki.whatwg.org/wiki/OffscreenCanvas >> >> (Renamed per feedback from Anne -- thanks.) >> >> Please offer your feedback. Multiple browser implementers agreed on >> this form of the API at a recent face-to-face meeting of the WebGL >> working group, and the proposal should be able to address multiple use >> cases. I'm very interested in prototyping it to see if it can offer >> better parallelism. > > > It seems fine. Great. > As far as I can tell, the only significant difference between > this and WorkerCanvas is using an HTMLCanvasElement "bitmaprenderer" as a > zero-copy way to render an ImageBitmap, instead of HTMLImageElement. That's the main difference. Another is that OffscreenCanvas can be directly instantiated from both the main thread and workers. >Can you explain what the problem was with using HTMLImageElement? One browser implementer pointed out that HTMLImageElement has several complex internal states related to loading, and they do not want to conflate that with the display of ImageBitmaps. I've documented this in the OffscreenCanvas proposal. (I originally thought Canvas had more complex internal state, but now think that it can be easier to define new behavior against a new canvas context than .) -Ken > Rob > -- > oIo otoeololo oyooouo otohoaoto oaonoyooonoeo owohooo oioso oaonogoroyo > owoiotoho oao oboroootohoeoro oooro osoiosotoeoro owoiololo oboeo > osouobojoeocoto otooo ojouodogomoeonoto.o oAogoaoiono,o oaonoyooonoeo > owohooo > osoaoyoso otooo oao oboroootohoeoro oooro osoiosotoeoro,o o‘oRoaocoao,o’o > oioso > oaonosowoeoroaoboloeo otooo otohoeo ocooouoroto.o oAonodo oaonoyooonoeo > owohooo > osoaoyoso,o o‘oYooouo ofolo!o’o owoiololo oboeo oiono odoaonogoeoro > ooofo > otohoeo ofoioroeo ooofo ohoeololo.
Re: [whatwg] Canvas image to blob/dataurl within Worker
(sorry for double email) Just one question about how requestAnimationFrame is expected to work. Could we get rAF added to workers? Would it be able to simply fire whenever it does on the main thread, as if it postMessage'd to the worker every time it was called but without the extra postMessage latency? To do that I guess workers need to be associated with a window. For normal workers this seems fine but if OffscreenCanvas is expected to work in shared or service workers this is a little tricker to define. Perhaps requestAnimationFrame could be a method on OffscreenCanvas and it fires based on the window the HTMLCanvasElement it came from is currently in? Then you have a convenient way to synchronise rendering with the window it will ultimately be displayed in, without having to know in advance which window that is. Ashley On 8 April 2015 at 16:36, Ashley Gullen wrote: > This looks like it will cover running a HTML5 game engine from a worker > very nicely and with little performance overhead. Good stuff! > > Ashley > > > On 8 April 2015 at 00:41, Kenneth Russell wrote: > >> On Wed, Mar 25, 2015 at 6:41 PM, Kenneth Russell wrote: >> > On Wed, Mar 25, 2015 at 1:22 PM, Robert O'Callahan < >> rob...@ocallahan.org> wrote: >> >> On Wed, Mar 25, 2015 at 11:41 PM, Anne van Kesteren >> >> wrote: >> >>> >> >>> On Fri, Mar 20, 2015 at 11:15 PM, Robert O'Callahan >> >>> wrote: >> >>> > My understanding is that the current consensus proposal for canvas >> in >> >>> > Workers is not what's in the spec, but this: >> >>> > https://wiki.whatwg.org/wiki/WorkerCanvas >> >>> > See "Canvas in Workers" threads from October 2013 for the >> discussion. >> >>> > svn >> >>> > is failing me but the CanvasProxy proposal in the spec definitely >> >>> > predates >> >>> > those threads. >> >>> > >> >>> > Ian, unless I'm wrong, it would be helpful to remove the CanvasProxy >> >>> > stuff >> >>> > from the spec to avoid confusion. >> >>> > >> >>> > That proposal contains WorkerCanvas.toBlob, which needs to be >> updated to >> >>> > use promises. >> >>> >> >>> There's also https://wiki.whatwg.org/wiki/WorkerCanvas2 it seems. It >> >>> would be interesting to know what the latest on this is. >> >> >> >> >> >> Right now that appears to be just a copy of WorkerCanvas with some >> >> boilerplate text added. I assume someone's working on it and will >> mention it >> >> on this list if/when they're ready to discuss it :-). >> > >> > Yes, apologies for letting it sit there in an incomplete state. >> > Recently feedback from more browser implementers was gathered about >> > the WorkerCanvas proposal. I will assemble it into WorkerCanvas2 and >> > follow up on this thread in a day or two. >> >> I apologize for taking so long to update this proposal, but it's now >> in a reasonable state: >> https://wiki.whatwg.org/wiki/OffscreenCanvas >> >> (Renamed per feedback from Anne -- thanks.) >> >> Please offer your feedback. Multiple browser implementers agreed on >> this form of the API at a recent face-to-face meeting of the WebGL >> working group, and the proposal should be able to address multiple use >> cases. I'm very interested in prototyping it to see if it can offer >> better parallelism. >> >> -Ken >> >> >> >> > -Ken >> > >> > >> >> Rob >> >> -- >> >> oIo otoeololo oyooouo otohoaoto oaonoyooonoeo owohooo oioso oaonogoroyo >> >> owoiotoho oao oboroootohoeoro oooro osoiosotoeoro owoiololo oboeo >> >> osouobojoeocoto otooo ojouodogomoeonoto.o oAogoaoiono,o oaonoyooonoeo >> >> owohooo >> >> osoaoyoso otooo oao oboroootohoeoro oooro osoiosotoeoro,o >> o‘oRoaocoao,o’o >> >> oioso >> >> oaonosowoeoroaoboloeo otooo otohoeo ocooouoroto.o oAonodo oaonoyooonoeo >> >> owohooo >> >> osoaoyoso,o o‘oYooouo ofolo!o’o owoiololo oboeo oiono odoaonogoeoro >> >> ooofo >> >> otohoeo ofoioroeo ooofo ohoeololo. >> > >
Re: [whatwg] Canvas image to blob/dataurl within Worker
This looks like it will cover running a HTML5 game engine from a worker very nicely and with little performance overhead. Good stuff! Ashley On 8 April 2015 at 00:41, Kenneth Russell wrote: > On Wed, Mar 25, 2015 at 6:41 PM, Kenneth Russell wrote: > > On Wed, Mar 25, 2015 at 1:22 PM, Robert O'Callahan > wrote: > >> On Wed, Mar 25, 2015 at 11:41 PM, Anne van Kesteren > >> wrote: > >>> > >>> On Fri, Mar 20, 2015 at 11:15 PM, Robert O'Callahan > >>> wrote: > >>> > My understanding is that the current consensus proposal for canvas in > >>> > Workers is not what's in the spec, but this: > >>> > https://wiki.whatwg.org/wiki/WorkerCanvas > >>> > See "Canvas in Workers" threads from October 2013 for the discussion. > >>> > svn > >>> > is failing me but the CanvasProxy proposal in the spec definitely > >>> > predates > >>> > those threads. > >>> > > >>> > Ian, unless I'm wrong, it would be helpful to remove the CanvasProxy > >>> > stuff > >>> > from the spec to avoid confusion. > >>> > > >>> > That proposal contains WorkerCanvas.toBlob, which needs to be > updated to > >>> > use promises. > >>> > >>> There's also https://wiki.whatwg.org/wiki/WorkerCanvas2 it seems. It > >>> would be interesting to know what the latest on this is. > >> > >> > >> Right now that appears to be just a copy of WorkerCanvas with some > >> boilerplate text added. I assume someone's working on it and will > mention it > >> on this list if/when they're ready to discuss it :-). > > > > Yes, apologies for letting it sit there in an incomplete state. > > Recently feedback from more browser implementers was gathered about > > the WorkerCanvas proposal. I will assemble it into WorkerCanvas2 and > > follow up on this thread in a day or two. > > I apologize for taking so long to update this proposal, but it's now > in a reasonable state: > https://wiki.whatwg.org/wiki/OffscreenCanvas > > (Renamed per feedback from Anne -- thanks.) > > Please offer your feedback. Multiple browser implementers agreed on > this form of the API at a recent face-to-face meeting of the WebGL > working group, and the proposal should be able to address multiple use > cases. I'm very interested in prototyping it to see if it can offer > better parallelism. > > -Ken > > > > > -Ken > > > > > >> Rob > >> -- > >> oIo otoeololo oyooouo otohoaoto oaonoyooonoeo owohooo oioso oaonogoroyo > >> owoiotoho oao oboroootohoeoro oooro osoiosotoeoro owoiololo oboeo > >> osouobojoeocoto otooo ojouodogomoeonoto.o oAogoaoiono,o oaonoyooonoeo > >> owohooo > >> osoaoyoso otooo oao oboroootohoeoro oooro osoiosotoeoro,o > o‘oRoaocoao,o’o > >> oioso > >> oaonosowoeoroaoboloeo otooo otohoeo ocooouoroto.o oAonodo oaonoyooonoeo > >> owohooo > >> osoaoyoso,o o‘oYooouo ofolo!o’o owoiololo oboeo oiono odoaonogoeoro > >> ooofo > >> otohoeo ofoioroeo ooofo ohoeololo. >
Re: [whatwg] Canvas image to blob/dataurl within Worker
Noteworthy: The OffscreenCanvas proposal removes a lot of the friction that we had with the toBlob implementation. On Tue, Apr 7, 2015 at 9:42 PM, Robert O'Callahan wrote: > On Wed, Apr 8, 2015 at 11:41 AM, Kenneth Russell wrote: > > > I apologize for taking so long to update this proposal, but it's now > > in a reasonable state: > > https://wiki.whatwg.org/wiki/OffscreenCanvas > > > > (Renamed per feedback from Anne -- thanks.) > > > > Please offer your feedback. Multiple browser implementers agreed on > > this form of the API at a recent face-to-face meeting of the WebGL > > working group, and the proposal should be able to address multiple use > > cases. I'm very interested in prototyping it to see if it can offer > > better parallelism. > > > > It seems fine. As far as I can tell, the only significant difference > between this and WorkerCanvas is using an HTMLCanvasElement > "bitmaprenderer" as a zero-copy way to render an ImageBitmap, instead of > HTMLImageElement. Can you explain what the problem was with using > HTMLImageElement? > > Rob > -- > oIo otoeololo oyooouo otohoaoto oaonoyooonoeo owohooo oioso oaonogoroyo > owoiotoho oao oboroootohoeoro oooro osoiosotoeoro owoiololo oboeo > osouobojoeocoto otooo ojouodogomoeonoto.o oAogoaoiono,o oaonoyooonoeo > owohooo > osoaoyoso otooo oao oboroootohoeoro oooro osoiosotoeoro,o o‘oRoaocoao,o’o > oioso > oaonosowoeoroaoboloeo otooo otohoeo ocooouoroto.o oAonodo oaonoyooonoeo > owohooo > osoaoyoso,o o‘oYooouo ofolo!o’o owoiololo oboeo oiono odoaonogoeoro > ooofo > otohoeo ofoioroeo ooofo ohoeololo. >
Re: [whatwg] Canvas image to blob/dataurl within Worker
On Wed, Apr 8, 2015 at 11:41 AM, Kenneth Russell wrote: > I apologize for taking so long to update this proposal, but it's now > in a reasonable state: > https://wiki.whatwg.org/wiki/OffscreenCanvas > > (Renamed per feedback from Anne -- thanks.) > > Please offer your feedback. Multiple browser implementers agreed on > this form of the API at a recent face-to-face meeting of the WebGL > working group, and the proposal should be able to address multiple use > cases. I'm very interested in prototyping it to see if it can offer > better parallelism. > It seems fine. As far as I can tell, the only significant difference between this and WorkerCanvas is using an HTMLCanvasElement "bitmaprenderer" as a zero-copy way to render an ImageBitmap, instead of HTMLImageElement. Can you explain what the problem was with using HTMLImageElement? Rob -- oIo otoeololo oyooouo otohoaoto oaonoyooonoeo owohooo oioso oaonogoroyo owoiotoho oao oboroootohoeoro oooro osoiosotoeoro owoiololo oboeo osouobojoeocoto otooo ojouodogomoeonoto.o oAogoaoiono,o oaonoyooonoeo owohooo osoaoyoso otooo oao oboroootohoeoro oooro osoiosotoeoro,o o‘oRoaocoao,o’o oioso oaonosowoeoroaoboloeo otooo otohoeo ocooouoroto.o oAonodo oaonoyooonoeo owohooo osoaoyoso,o o‘oYooouo ofolo!o’o owoiololo oboeo oiono odoaonogoeoro ooofo otohoeo ofoioroeo ooofo ohoeololo.
Re: [whatwg] Canvas image to blob/dataurl within Worker
On Wed, Mar 25, 2015 at 6:41 PM, Kenneth Russell wrote: > On Wed, Mar 25, 2015 at 1:22 PM, Robert O'Callahan > wrote: >> On Wed, Mar 25, 2015 at 11:41 PM, Anne van Kesteren >> wrote: >>> >>> On Fri, Mar 20, 2015 at 11:15 PM, Robert O'Callahan >>> wrote: >>> > My understanding is that the current consensus proposal for canvas in >>> > Workers is not what's in the spec, but this: >>> > https://wiki.whatwg.org/wiki/WorkerCanvas >>> > See "Canvas in Workers" threads from October 2013 for the discussion. >>> > svn >>> > is failing me but the CanvasProxy proposal in the spec definitely >>> > predates >>> > those threads. >>> > >>> > Ian, unless I'm wrong, it would be helpful to remove the CanvasProxy >>> > stuff >>> > from the spec to avoid confusion. >>> > >>> > That proposal contains WorkerCanvas.toBlob, which needs to be updated to >>> > use promises. >>> >>> There's also https://wiki.whatwg.org/wiki/WorkerCanvas2 it seems. It >>> would be interesting to know what the latest on this is. >> >> >> Right now that appears to be just a copy of WorkerCanvas with some >> boilerplate text added. I assume someone's working on it and will mention it >> on this list if/when they're ready to discuss it :-). > > Yes, apologies for letting it sit there in an incomplete state. > Recently feedback from more browser implementers was gathered about > the WorkerCanvas proposal. I will assemble it into WorkerCanvas2 and > follow up on this thread in a day or two. I apologize for taking so long to update this proposal, but it's now in a reasonable state: https://wiki.whatwg.org/wiki/OffscreenCanvas (Renamed per feedback from Anne -- thanks.) Please offer your feedback. Multiple browser implementers agreed on this form of the API at a recent face-to-face meeting of the WebGL working group, and the proposal should be able to address multiple use cases. I'm very interested in prototyping it to see if it can offer better parallelism. -Ken > -Ken > > >> Rob >> -- >> oIo otoeololo oyooouo otohoaoto oaonoyooonoeo owohooo oioso oaonogoroyo >> owoiotoho oao oboroootohoeoro oooro osoiosotoeoro owoiololo oboeo >> osouobojoeocoto otooo ojouodogomoeonoto.o oAogoaoiono,o oaonoyooonoeo >> owohooo >> osoaoyoso otooo oao oboroootohoeoro oooro osoiosotoeoro,o o‘oRoaocoao,o’o >> oioso >> oaonosowoeoroaoboloeo otooo otohoeo ocooouoroto.o oAonodo oaonoyooonoeo >> owohooo >> osoaoyoso,o o‘oYooouo ofolo!o’o owoiololo oboeo oiono odoaonogoeoro >> ooofo >> otohoeo ofoioroeo ooofo ohoeololo.
Re: [whatwg] Canvas image to blob/dataurl within Worker
On Wed, Mar 25, 2015 at 1:22 PM, Robert O'Callahan wrote: > On Wed, Mar 25, 2015 at 11:41 PM, Anne van Kesteren > wrote: >> >> On Fri, Mar 20, 2015 at 11:15 PM, Robert O'Callahan >> wrote: >> > My understanding is that the current consensus proposal for canvas in >> > Workers is not what's in the spec, but this: >> > https://wiki.whatwg.org/wiki/WorkerCanvas >> > See "Canvas in Workers" threads from October 2013 for the discussion. >> > svn >> > is failing me but the CanvasProxy proposal in the spec definitely >> > predates >> > those threads. >> > >> > Ian, unless I'm wrong, it would be helpful to remove the CanvasProxy >> > stuff >> > from the spec to avoid confusion. >> > >> > That proposal contains WorkerCanvas.toBlob, which needs to be updated to >> > use promises. >> >> There's also https://wiki.whatwg.org/wiki/WorkerCanvas2 it seems. It >> would be interesting to know what the latest on this is. > > > Right now that appears to be just a copy of WorkerCanvas with some > boilerplate text added. I assume someone's working on it and will mention it > on this list if/when they're ready to discuss it :-). Yes, apologies for letting it sit there in an incomplete state. Recently feedback from more browser implementers was gathered about the WorkerCanvas proposal. I will assemble it into WorkerCanvas2 and follow up on this thread in a day or two. -Ken > Rob > -- > oIo otoeololo oyooouo otohoaoto oaonoyooonoeo owohooo oioso oaonogoroyo > owoiotoho oao oboroootohoeoro oooro osoiosotoeoro owoiololo oboeo > osouobojoeocoto otooo ojouodogomoeonoto.o oAogoaoiono,o oaonoyooonoeo > owohooo > osoaoyoso otooo oao oboroootohoeoro oooro osoiosotoeoro,o o‘oRoaocoao,o’o > oioso > oaonosowoeoroaoboloeo otooo otohoeo ocooouoroto.o oAonodo oaonoyooonoeo > owohooo > osoaoyoso,o o‘oYooouo ofolo!o’o owoiololo oboeo oiono odoaonogoeoro > ooofo > otohoeo ofoioroeo ooofo ohoeololo.
Re: [whatwg] Canvas image to blob/dataurl within Worker
On Thu, Mar 26, 2015 at 9:43 AM, Justin Novosad wrote:
> createImageBitmap(myImageSource).then(function (image) {
> image.toBlob().then(admireYourShinyNewBlob); });
>
Oh, right. Sorry! Of course :-).
Rob
--
oIo otoeololo oyooouo otohoaoto oaonoyooonoeo owohooo oioso oaonogoroyo
owoiotoho oao oboroootohoeoro oooro osoiosotoeoro owoiololo oboeo
osouobojoeocoto otooo ojouodogomoeonoto.o oAogoaoiono,o oaonoyooonoeo
owohooo
osoaoyoso otooo oao oboroootohoeoro oooro osoiosotoeoro,o o‘oRoaocoao,o’o
oioso
oaonosowoeoroaoboloeo otooo otohoeo ocooouoroto.o oAonodo oaonoyooonoeo
owohooo
osoaoyoso,o o‘oYooouo ofolo!o’o owoiololo oboeo oiono odoaonogoeoro
ooofo
otohoeo ofoioroeo ooofo ohoeololo.
Re: [whatwg] Canvas image to blob/dataurl within Worker
On Wed, Mar 25, 2015 at 4:28 PM, Robert O'Callahan
wrote:
> On Thu, Mar 26, 2015 at 8:40 AM, Justin Novosad wrote:
>
>>
>> Eliminating the transit through canvas also
>> allows for fast paths for capturing blobs from , , svg, URLs,
>> to name a few.
>>
>
> With new APIs on those elements, you mean?
>
Not necessarily. I meant like this:
createImageBitmap(myImageSource).then(function (image) {
image.toBlob().then(admireYourShinyNewBlob); });
Re: [whatwg] Canvas image to blob/dataurl within Worker
On Thu, Mar 26, 2015 at 8:40 AM, Justin Novosad wrote: > One idea we've been brewing that has not surfaced on this thread so far is > that of having an ImageBitmap.toBlob(), which would return a Promise. Also, > ImageBitmap should be transferable. This idea is just partially baked right > now, but I wanted to throw it out into the open to get people thinking > about it. The rationale behind this is that ImageBitmap is basically a > lightweight read-only proxy for pretty much anything that can be used as an > image source. So this single API entry-point can perform as well as a > direct toBlob() method hanging off of any other type of object that an > ImageBitmap can be created from. Because it is an opaque and immutable > object, it gives browser vendors a lot of latitude for implementing high > performance code paths. It allows pixel data from the DOM to cross Worker > boundaries without making copies (at least in cases where the source is a > static image). It allows interactions with blob storage to be driven from a > worker (vs. canvas.toBlob), hence avoiding jank on the main thread. Raw > image data generated in JS would not have to transit through a canvas (just > create an ImageBitmap from an ImageData object), which eliminates a lot of > unnecessary pixel pushing. That makes sense to me. If we do that, and also add HTMLCanvasElement.transferToImageBitmap, then one-shot "draw into a canvas and create a blob" applications could have really minimal memory usage. That method would detach the backbuffer and clear the canvas (which I think all browsers are optimizing to "no backbuffer"). Eliminating the transit through canvas also > allows for fast paths for capturing blobs from , , svg, URLs, > to name a few. > With new APIs on those elements, you mean? Rob -- oIo otoeololo oyooouo otohoaoto oaonoyooonoeo owohooo oioso oaonogoroyo owoiotoho oao oboroootohoeoro oooro osoiosotoeoro owoiololo oboeo osouobojoeocoto otooo ojouodogomoeonoto.o oAogoaoiono,o oaonoyooonoeo owohooo osoaoyoso otooo oao oboroootohoeoro oooro osoiosotoeoro,o o‘oRoaocoao,o’o oioso oaonosowoeoroaoboloeo otooo otohoeo ocooouoroto.o oAonodo oaonoyooonoeo owohooo osoaoyoso,o o‘oYooouo ofolo!o’o owoiololo oboeo oiono odoaonogoeoro ooofo otohoeo ofoioroeo ooofo ohoeololo.
Re: [whatwg] Canvas image to blob/dataurl within Worker
On Wed, Mar 25, 2015 at 11:41 PM, Anne van Kesteren wrote: > On Fri, Mar 20, 2015 at 11:15 PM, Robert O'Callahan > wrote: > > My understanding is that the current consensus proposal for canvas in > > Workers is not what's in the spec, but this: > > https://wiki.whatwg.org/wiki/WorkerCanvas > > See "Canvas in Workers" threads from October 2013 for the discussion. svn > > is failing me but the CanvasProxy proposal in the spec definitely > predates > > those threads. > > > > Ian, unless I'm wrong, it would be helpful to remove the CanvasProxy > stuff > > from the spec to avoid confusion. > > > > That proposal contains WorkerCanvas.toBlob, which needs to be updated to > > use promises. > > There's also https://wiki.whatwg.org/wiki/WorkerCanvas2 it seems. It > would be interesting to know what the latest on this is. > Right now that appears to be just a copy of WorkerCanvas with some boilerplate text added. I assume someone's working on it and will mention it on this list if/when they're ready to discuss it :-). Rob -- oIo otoeololo oyooouo otohoaoto oaonoyooonoeo owohooo oioso oaonogoroyo owoiotoho oao oboroootohoeoro oooro osoiosotoeoro owoiololo oboeo osouobojoeocoto otooo ojouodogomoeonoto.o oAogoaoiono,o oaonoyooonoeo owohooo osoaoyoso otooo oao oboroootohoeoro oooro osoiosotoeoro,o o‘oRoaocoao,o’o oioso oaonosowoeoroaoboloeo otooo otohoeo ocooouoroto.o oAonodo oaonoyooonoeo owohooo osoaoyoso,o o‘oYooouo ofolo!o’o owoiololo oboeo oiono odoaonogoeoro ooofo otohoeo ofoioroeo ooofo ohoeololo.
Re: [whatwg] Canvas image to blob/dataurl within Worker
Sorry for arriving so late to the party... I would like to update this thread on what has been happening around canvas.toBlob in Blink. First of all, there was an attempt at implementing canvas.toBlob a couple years ago, back when Chromium was using WebKit. The experience revealed that the API was not the silver bullet we hoped and was not satisfactorily addressing issues regarding latency, memory usage and jank reduction. There are many technical reasons for this, some of which are related to architectural details that are specific to Chromium (native thread model, out of process blob storage, etc.) Long story short, the feature was put on ice because it failed to provide a significant advantage over solutions that can be implemented using existing APIs. Recently, there has been a flare-up of discussion around the fact that canvas.toBlob is still not shipping in Chrome. So we came clean about our intention to not ship it, and we've asked about people's use cases in order to look for a better solution. It turns out the use cases are quite varied. For the past week Noel Gordon (the Chromium engineer who was originally working on toBlob) and I have been brainstorming on some ideas. We noticed this one on the Mozilla issue tracker ( https://bugzilla.mozilla.org/show_bug.cgi?id=870024) about adding an ImageData.toBlob, and we also got some ideas from Roc's WorkerCanvas proposal. We are still collecting use cases and ideas, on top of the ones that were already posted on our issue tracker and the blink-dev mailing list. If anyone reading this wants to add theirs, please use this form: https://docs.google.com/forms/d/1YHIGf2-0wqbeFLA8Y6S-YaFzCWxlw9xPrOn9ctP5JwA/viewform We will hash it all and report back to this thread in a few days with some data and some ideas. One idea we've been brewing that has not surfaced on this thread so far is that of having an ImageBitmap.toBlob(), which would return a Promise. Also, ImageBitmap should be transferable. This idea is just partially baked right now, but I wanted to throw it out into the open to get people thinking about it. The rationale behind this is that ImageBitmap is basically a lightweight read-only proxy for pretty much anything that can be used as an image source. So this single API entry-point can perform as well as a direct toBlob() method hanging off of any other type of object that an ImageBitmap can be created from. Because it is an opaque and immutable object, it gives browser vendors a lot of latitude for implementing high performance code paths. It allows pixel data from the DOM to cross Worker boundaries without making copies (at least in cases where the source is a static image). It allows interactions with blob storage to be driven from a worker (vs. canvas.toBlob), hence avoiding jank on the main thread. Raw image data generated in JS would not have to transit through a canvas (just create an ImageBitmap from an ImageData object), which eliminates a lot of unnecessary pixel pushing. Eliminating the transit through canvas also allows for fast paths for capturing blobs from , , svg, URLs, to name a few. -Justin On Wed, Mar 25, 2015 at 6:41 AM, Anne van Kesteren wrote: > On Fri, Mar 20, 2015 at 11:15 PM, Robert O'Callahan > wrote: > > My understanding is that the current consensus proposal for canvas in > > Workers is not what's in the spec, but this: > > https://wiki.whatwg.org/wiki/WorkerCanvas > > See "Canvas in Workers" threads from October 2013 for the discussion. svn > > is failing me but the CanvasProxy proposal in the spec definitely > predates > > those threads. > > > > Ian, unless I'm wrong, it would be helpful to remove the CanvasProxy > stuff > > from the spec to avoid confusion. > > > > That proposal contains WorkerCanvas.toBlob, which needs to be updated to > > use promises. > > There's also https://wiki.whatwg.org/wiki/WorkerCanvas2 it seems. It > would be interesting to know what the latest on this is. Canvas in > workers has been discussed for a long time now and we're still not at > the point of implementing. > > > -- > https://annevankesteren.nl/ >
Re: [whatwg] Canvas image to blob/dataurl within Worker
On Fri, Mar 20, 2015 at 11:15 PM, Robert O'Callahan wrote: > My understanding is that the current consensus proposal for canvas in > Workers is not what's in the spec, but this: > https://wiki.whatwg.org/wiki/WorkerCanvas > See "Canvas in Workers" threads from October 2013 for the discussion. svn > is failing me but the CanvasProxy proposal in the spec definitely predates > those threads. > > Ian, unless I'm wrong, it would be helpful to remove the CanvasProxy stuff > from the spec to avoid confusion. > > That proposal contains WorkerCanvas.toBlob, which needs to be updated to > use promises. There's also https://wiki.whatwg.org/wiki/WorkerCanvas2 it seems. It would be interesting to know what the latest on this is. Canvas in workers has been discussed for a long time now and we're still not at the point of implementing. -- https://annevankesteren.nl/
Re: [whatwg] Canvas image to blob/dataurl within Worker
Which is why more browsers shouldn't adopt it. If we have an opportunity to switch it for a promise-based API, we should take it. On Mon, 23 Mar 2015 at 17:21 Garrett Smith wrote: > On 3/23/15, Jake Archibald wrote: > > I'd rather Firefox did a small hack under hood for compatibility than we > > hack the spec with a separate method. > > > > We shouldn't be adding async APIs to browsers that use callbacks. > > Not sure what you mean. Method toBlob is async and uses a callback. > -- > Garrett > @xkit > ChordCycles.com > garretts.github.io > personx.tumblr.com >
Re: [whatwg] Canvas image to blob/dataurl within Worker
On 3/23/15, Jake Archibald wrote: > I'd rather Firefox did a small hack under hood for compatibility than we > hack the spec with a separate method. > > We shouldn't be adding async APIs to browsers that use callbacks. Not sure what you mean. Method toBlob is async and uses a callback. -- Garrett @xkit ChordCycles.com garretts.github.io personx.tumblr.com
Re: [whatwg] Canvas image to blob/dataurl within Worker
I'd rather Firefox did a small hack under hood for compatibility than we hack the spec with a separate method. We shouldn't be adding async APIs to browsers that use callbacks. Only one browser implements the callback version here, and they can maintain compat easily & remove the callback when usage stats drop off. On Mon, 23 Mar 2015 07:43 Garrett Smith wrote: > On 3/21/15, Jake Archibald wrote: > > I'd rather we did that by introducing promises to HTMLCanvasElement. > > Returning a promise from toBlob is easy, making the callback arg optional > > by checking the type of the first arg is hacky but possible (and is done > in > > js libs). > > > Overloading with optional middle arguments is hacky, > > A different-named method is a better idea than in-place > transmogrification. Don't change toBlob to be overloaded, create a new > method, toBlobPromise. IMO. > > In JS libs and even in canvas methods, overloading - especially with > optional middle arguments, causes problems. I went over some of the > branching complexities of that. Not all, but some (the email was > already too long and I have other things to do). > > https://lists.w3.org/Archives/Public/public-whatwg-archive/ > 2015Mar/0059.html > -- > Garrett > @xkit > ChordCycles.com > garretts.github.io > personx.tumblr.com >
Re: [whatwg] Canvas image to blob/dataurl within Worker
On 3/23/15, Garrett Smith wrote: > On 3/21/15, Jake Archibald wrote: [...] > https://lists.w3.org/Archives/Public/public-whatwg-archive/2015Mar/0059.html Sorry, this was the actual post: https://lists.w3.org/Archives/Public/public-whatwg-archive/2015Mar/0064.html same thread though. -- Garrett @xkit ChordCycles.com garretts.github.io personx.tumblr.com
Re: [whatwg] Canvas image to blob/dataurl within Worker
On 3/21/15, Jake Archibald wrote: > I'd rather we did that by introducing promises to HTMLCanvasElement. > Returning a promise from toBlob is easy, making the callback arg optional > by checking the type of the first arg is hacky but possible (and is done in > js libs). > Overloading with optional middle arguments is hacky, A different-named method is a better idea than in-place transmogrification. Don't change toBlob to be overloaded, create a new method, toBlobPromise. IMO. In JS libs and even in canvas methods, overloading - especially with optional middle arguments, causes problems. I went over some of the branching complexities of that. Not all, but some (the email was already too long and I have other things to do). https://lists.w3.org/Archives/Public/public-whatwg-archive/2015Mar/0059.html -- Garrett @xkit ChordCycles.com garretts.github.io personx.tumblr.com
Re: [whatwg] Canvas image to blob/dataurl within Worker
I'm getting confused as to what the problem is. To restate the advantages of async toBlob: suppose I have code that does drawStuffTo(context); context.canvas.toBlob().then(callback); // no further drawing to the canvas This has a few advantages over using getImageData and then encoding to JPEG via JS in a Worker: 1) getImageData will stall the main thread until all canvas rasterization is finished and the data has been read back from GPU memory. toBlob lets the main thread run while those things happen. This is the biggest issue. 2) The first stage of JPEG compression is to convert the data to YUV 4:2:0, which can be done on the GPU and produces a buffer which is only half the size, so even after you've copied it by reading it back this uses no more CPU+GPU memory, less GPU-to-CPU bandwidth, and will probably be faster. You may be able to do even more JPEG compression on the GPU, depending on the underlying architecture. 3) Using the browser's native JPEG encoder is probably a bit faster and lighter-weight than JS in a Worker, especially due to SIMD. Browsers must already have a native JPEG encoder to support canvas.toDataURL(). In the case where someone does drawStuffTo(context); context.canvas.toBlob().then(callback); drawStuffTo(context); the browser has a few options. After rasterizing the first set of commands you can make a copy in GPU memory. Or, you can apply #2 above and make a YUV 4:2:0 copy using half the memory. Or, you can start the readback, and delay rasterizing the second set of commands until the readback has finished. You can vary the strategy depending on the amount of memory available. If the browser is ever really low on memory, it can always stall everything until JPEG encoding is done and intermediate buffers released. So there can never be a memory-use advantage to making the promise fail. Rob -- oIo otoeololo oyooouo otohoaoto oaonoyooonoeo owohooo oioso oaonogoroyo owoiotoho oao oboroootohoeoro oooro osoiosotoeoro owoiololo oboeo osouobojoeocoto otooo ojouodogomoeonoto.o oAogoaoiono,o oaonoyooonoeo owohooo osoaoyoso otooo oao oboroootohoeoro oooro osoiosotoeoro,o o‘oRoaocoao,o’o oioso oaonosowoeoroaoboloeo otooo otohoeo ocooouoroto.o oAonodo oaonoyooonoeo owohooo osoaoyoso,o o‘oYooouo ofolo!o’o owoiololo oboeo oiono odoaonogoeoro ooofo otohoeo ofoioroeo ooofo ohoeololo.
Re: [whatwg] Canvas image to blob/dataurl within Worker
On Sat, 21 Mar 2015 at 18:16 Rik Cabanier wrote: > Justin is worried that in order to make this asynchronous, Chrome has to > create a snapshot of the canvas bits which is slow if it resides on the GPU. > Of course, his workaround to use getImageData is just as slow since it has > to do the same operation. > > To alleviate this, I have 2 proposals: > - After calling toBlob, the canvas is read-only until the promise is > fulfilled > - If the user changes the canvas after calling toBlob, the promise is > cancelled. > > Maybe we should only allow 1 outstanding toBlob per canvas element too. > I don't really understand why we'd need these restrictions, can't this just operate like a queue? Copying from https://code.google.com/p/chromium/issues/detail?id=67587#c87: --- context.[drawing command]; // 1 context.[drawing command]; // 2 context.canvas.toBlob().then(callback); // 3 context.[drawing command]; // 4 var data = context.getImageData(…); // 5 context.[drawing command]; // 6 [browser frame] Lines 1 & 2 would queue up operations to manipulate canvas pixels, but they would not wait on those pixels being painted. The toBlob call (3) queues up an operation to create a blob of a PNG of the canvas state. This means the blob would be created once 1 & 2 have applied. 4 queues up another canvas pixel manipulation, explicitly after the creation of the blob. 5 synchronously waits on the operation queue to complete, which includes 1, 2, the creation of the blob in 3, and the operation in 4. Blocking here kinda sucks & feels much like a synchronous layout operation, but hey that's the API we have. Note: callback hasn't been called yet, as promise resolving is a microtask. 6 queues up an operation to manipulate the canvas. Then we reach the end of the microtask, callback is called with the blob, then we reach the end of the task. The next browser frame is blocked on the completion of 6. Without the synchronous step 5, the JS wouldn't block on paints (aside from callback). --- Sure, we now have 3 copies of the canvas data. One for the on-screen canvas, one for the blob (compressed as PNG), and another for the getImageData, but that's what I, the developer have asked for. If that's taking too much memory, or is slow, I should find another way to do what I'm trying to do. I've created intermediate frames through toBlob and getImageData, rather like how calling offsetWidth between two style modifications creates an intermediate style calc & layout. If that's not what I want, I shouldn't interleave my reads & writes.
Re: [whatwg] Canvas image to blob/dataurl within Worker
On Sat, Mar 21, 2015 at 11:02 PM, Robert O'Callahan wrote: > On Sun, Mar 22, 2015 at 6:45 PM, Rik Cabanier wrote: > >> On Sat, Mar 21, 2015 at 1:44 PM, Robert O'Callahan >> wrote: >> >> > On Sun, Mar 22, 2015 at 7:16 AM, Rik Cabanier >> wrote: >> > >> >> Justin is worried that in order to make this asynchronous, Chrome has >> to >> >> create a snapshot of the canvas bits which is slow if it resides on the >> >> GPU. >> >> Of course, his workaround to use getImageData is just as slow since it >> has >> >> to do the same operation. >> >> >> > >> > One of the advantages of having a native async toBlob API is that the >> > browser can asynchronously read back from GPU memory (when the graphics >> API >> > permits this --- D3D11 does, at least). Gecko currently doesn't take >> > advantage of this, however. >> > >> >> You would need a copy in GPU memory first to do the async readback on. >> > > Not necessarily. > > >> There are many scenarios (ie fullscreen hidi canvas) where this might fill >> the GPU's memory. >> > > Unlikely in practice. > Hopefully Justing can chime in on this. Google maps in particular taxes the GPU a lot. If Chrome has to reserve enough space for another backbuffer, this will certainly make rendering of complex scenes slower. > > To alleviate this, I have 2 proposals: >> >> - After calling toBlob, the canvas is read-only until the promise is >> >> fulfilled >> >> - If the user changes the canvas after calling toBlob, the promise is >> >> cancelled. >> >> >> >> Maybe we should only allow 1 outstanding toBlob per canvas element too. >> >> >> > >> > I don't think we should impose any of these restrictions. They're not >> > necessary. >> > >> >> How else would you avoid making a copy? >> > > It depends on lots of variables, but there are certainly scenarios when > you can do async readback without making a copy. > > Even if you have to make a copy in GPU memory, that's not a big problem > most of the time, not compared to doing a synchronous readback. > > Rob > -- > oIo otoeololo oyooouo otohoaoto oaonoyooonoeo owohooo oioso oaonogoroyo > owoiotoho oao oboroootohoeoro oooro osoiosotoeoro owoiololo oboeo > osouobojoeocoto otooo ojouodogomoeonoto.o oAogoaoiono,o oaonoyooonoeo > owohooo > osoaoyoso otooo oao oboroootohoeoro oooro osoiosotoeoro,o o‘oRoaocoao,o’o > oioso > oaonosowoeoroaoboloeo otooo otohoeo ocooouoroto.o oAonodo oaonoyooonoeo > owohooo > osoaoyoso,o o‘oYooouo ofolo!o’o owoiololo oboeo oiono odoaonogoeoro > ooofo > otohoeo ofoioroeo ooofo ohoeololo. >
Re: [whatwg] Canvas image to blob/dataurl within Worker
On Sun, Mar 22, 2015 at 6:45 PM, Rik Cabanier wrote: > On Sat, Mar 21, 2015 at 1:44 PM, Robert O'Callahan > wrote: > > > On Sun, Mar 22, 2015 at 7:16 AM, Rik Cabanier > wrote: > > > >> Justin is worried that in order to make this asynchronous, Chrome has to > >> create a snapshot of the canvas bits which is slow if it resides on the > >> GPU. > >> Of course, his workaround to use getImageData is just as slow since it > has > >> to do the same operation. > >> > > > > One of the advantages of having a native async toBlob API is that the > > browser can asynchronously read back from GPU memory (when the graphics > API > > permits this --- D3D11 does, at least). Gecko currently doesn't take > > advantage of this, however. > > > > You would need a copy in GPU memory first to do the async readback on. > Not necessarily. > There are many scenarios (ie fullscreen hidi canvas) where this might fill > the GPU's memory. > Unlikely in practice. > To alleviate this, I have 2 proposals: > >> - After calling toBlob, the canvas is read-only until the promise is > >> fulfilled > >> - If the user changes the canvas after calling toBlob, the promise is > >> cancelled. > >> > >> Maybe we should only allow 1 outstanding toBlob per canvas element too. > >> > > > > I don't think we should impose any of these restrictions. They're not > > necessary. > > > > How else would you avoid making a copy? > It depends on lots of variables, but there are certainly scenarios when you can do async readback without making a copy. Even if you have to make a copy in GPU memory, that's not a big problem most of the time, not compared to doing a synchronous readback. Rob -- oIo otoeololo oyooouo otohoaoto oaonoyooonoeo owohooo oioso oaonogoroyo owoiotoho oao oboroootohoeoro oooro osoiosotoeoro owoiololo oboeo osouobojoeocoto otooo ojouodogomoeonoto.o oAogoaoiono,o oaonoyooonoeo owohooo osoaoyoso otooo oao oboroootohoeoro oooro osoiosotoeoro,o o‘oRoaocoao,o’o oioso oaonosowoeoroaoboloeo otooo otohoeo ocooouoroto.o oAonodo oaonoyooonoeo owohooo osoaoyoso,o o‘oYooouo ofolo!o’o owoiololo oboeo oiono odoaonogoeoro ooofo otohoeo ofoioroeo ooofo ohoeololo.
Re: [whatwg] Canvas image to blob/dataurl within Worker
On Sat, Mar 21, 2015 at 1:44 PM, Robert O'Callahan wrote: > On Sun, Mar 22, 2015 at 7:16 AM, Rik Cabanier wrote: > >> Justin is worried that in order to make this asynchronous, Chrome has to >> create a snapshot of the canvas bits which is slow if it resides on the >> GPU. >> Of course, his workaround to use getImageData is just as slow since it has >> to do the same operation. >> > > One of the advantages of having a native async toBlob API is that the > browser can asynchronously read back from GPU memory (when the graphics API > permits this --- D3D11 does, at least). Gecko currently doesn't take > advantage of this, however. > You would need a copy in GPU memory first to do the async readback on. There are many scenarios (ie fullscreen hidi canvas) where this might fill the GPU's memory. > To alleviate this, I have 2 proposals: >> - After calling toBlob, the canvas is read-only until the promise is >> fulfilled >> - If the user changes the canvas after calling toBlob, the promise is >> cancelled. >> >> Maybe we should only allow 1 outstanding toBlob per canvas element too. >> > > I don't think we should impose any of these restrictions. They're not > necessary. > How else would you avoid making a copy?
Re: [whatwg] Canvas image to blob/dataurl within Worker
On Sun, Mar 22, 2015 at 7:16 AM, Rik Cabanier wrote: > Justin is worried that in order to make this asynchronous, Chrome has to > create a snapshot of the canvas bits which is slow if it resides on the > GPU. > Of course, his workaround to use getImageData is just as slow since it has > to do the same operation. > One of the advantages of having a native async toBlob API is that the browser can asynchronously read back from GPU memory (when the graphics API permits this --- D3D11 does, at least). Gecko currently doesn't take advantage of this, however. > To alleviate this, I have 2 proposals: > - After calling toBlob, the canvas is read-only until the promise is > fulfilled > - If the user changes the canvas after calling toBlob, the promise is > cancelled. > > Maybe we should only allow 1 outstanding toBlob per canvas element too. > I don't think we should impose any of these restrictions. They're not necessary. Rob -- oIo otoeololo oyooouo otohoaoto oaonoyooonoeo owohooo oioso oaonogoroyo owoiotoho oao oboroootohoeoro oooro osoiosotoeoro owoiololo oboeo osouobojoeocoto otooo ojouodogomoeonoto.o oAogoaoiono,o oaonoyooonoeo owohooo osoaoyoso otooo oao oboroootohoeoro oooro osoiosotoeoro,o o‘oRoaocoao,o’o oioso oaonosowoeoroaoboloeo otooo otohoeo ocooouoroto.o oAonodo oaonoyooonoeo owohooo osoaoyoso,o o‘oYooouo ofolo!o’o owoiololo oboeo oiono odoaonogoeoro ooofo otohoeo ofoioroeo ooofo ohoeololo.
Re: [whatwg] Canvas image to blob/dataurl within Worker
On Sat, Mar 21, 2015 at 6:21 AM, Ashley Gullen wrote: > Is everyone here aware that currently Google have stated they do not plan > to implement toBlob?: > https://code.google.com/p/chromium/issues/detail?id=67587 > > IMO this is the wrong call. We should be favouring blobs over data URLs > since they are more efficient (no size bloat, can be requested async like > other network resources, no need to copy round very large strings). > Justin is worried that in order to make this asynchronous, Chrome has to create a snapshot of the canvas bits which is slow if it resides on the GPU. Of course, his workaround to use getImageData is just as slow since it has to do the same operation. To alleviate this, I have 2 proposals: - After calling toBlob, the canvas is read-only until the promise is fulfilled - If the user changes the canvas after calling toBlob, the promise is cancelled. Maybe we should only allow 1 outstanding toBlob per canvas element too. I made a small code example of toBlob here: http://codepen.io/adobe/full/raoZdQ/ It works smoothly on my mac and pc laptop, but really janky on my PC desktop. > > On 21 March 2015 at 11:19, Jake Archibald wrote: > >> I'd rather we did that by introducing promises to HTMLCanvasElement. >> Returning a promise from toBlob is easy, making the callback arg optional >> by checking the type of the first arg is hacky but possible (and is done >> in >> js libs). >> >> On Sat, 21 Mar 2015 10:56 Robert O'Callahan wrote: >> >> > On Sat, Mar 21, 2015 at 5:45 PM, Rik Cabanier >> wrote: >> > >> >> Ah, OK. I thought we were changing it for both cases. This will cause a >> >> lot >> >> of confusion... >> >> >> > >> > If we want to keep HTMLCanvasElement and WorkerCanvas in sync, we can. >> > >> > Rob >> > -- >> > oIo otoeololo oyooouo otohoaoto oaonoyooonoeo owohooo oioso oaonogoroyo >> > owoiotoho oao oboroootohoeoro oooro osoiosotoeoro owoiololo oboeo >> > osouobojoeocoto otooo ojouodogomoeonoto.o oAogoaoiono,o oaonoyooonoeo >> > owohooo >> > osoaoyoso otooo oao oboroootohoeoro oooro osoiosotoeoro,o >> o‘oRoaocoao,o’o >> > oioso >> > oaonosowoeoroaoboloeo otooo otohoeo ocooouoroto.o oAonodo oaonoyooonoeo >> > owohooo >> > osoaoyoso,o o‘oYooouo ofolo!o’o owoiololo oboeo oiono odoaonogoeoro >> > ooofo >> > otohoeo ofoioroeo ooofo ohoeololo. >> > >> > >
Re: [whatwg] Canvas image to blob/dataurl within Worker
On Sat, Mar 21, 2015 at 4:19 AM, Jake Archibald wrote: > I'd rather we did that by introducing promises to HTMLCanvasElement. > Returning a promise from toBlob is easy, making the callback arg optional > by checking the type of the first arg is hacky but possible (and is done in > js libs). > The spec (if there is one?) should be updated to return a promise and leave out the callback: promise canvas.toBlob(optional type, optional encoderOptions); Mozilla would keep their existing implementation around and the IDL logic would automatically pick the right call. > On Sat, 21 Mar 2015 10:56 Robert O'Callahan wrote: > >> On Sat, Mar 21, 2015 at 5:45 PM, Rik Cabanier wrote: >> >>> Ah, OK. I thought we were changing it for both cases. This will cause a >>> lot >>> of confusion... >>> >> >> If we want to keep HTMLCanvasElement and WorkerCanvas in sync, we can. >> >
Re: [whatwg] Canvas image to blob/dataurl within Worker
Is everyone here aware that currently Google have stated they do not plan to implement toBlob?: https://code.google.com/p/chromium/issues/detail?id=67587 IMO this is the wrong call. We should be favouring blobs over data URLs since they are more efficient (no size bloat, can be requested async like other network resources, no need to copy round very large strings). On 21 March 2015 at 11:19, Jake Archibald wrote: > I'd rather we did that by introducing promises to HTMLCanvasElement. > Returning a promise from toBlob is easy, making the callback arg optional > by checking the type of the first arg is hacky but possible (and is done in > js libs). > > On Sat, 21 Mar 2015 10:56 Robert O'Callahan wrote: > > > On Sat, Mar 21, 2015 at 5:45 PM, Rik Cabanier > wrote: > > > >> Ah, OK. I thought we were changing it for both cases. This will cause a > >> lot > >> of confusion... > >> > > > > If we want to keep HTMLCanvasElement and WorkerCanvas in sync, we can. > > > > Rob > > -- > > oIo otoeololo oyooouo otohoaoto oaonoyooonoeo owohooo oioso oaonogoroyo > > owoiotoho oao oboroootohoeoro oooro osoiosotoeoro owoiololo oboeo > > osouobojoeocoto otooo ojouodogomoeonoto.o oAogoaoiono,o oaonoyooonoeo > > owohooo > > osoaoyoso otooo oao oboroootohoeoro oooro osoiosotoeoro,o o‘oRoaocoao,o’o > > oioso > > oaonosowoeoroaoboloeo otooo otohoeo ocooouoroto.o oAonodo oaonoyooonoeo > > owohooo > > osoaoyoso,o o‘oYooouo ofolo!o’o owoiololo oboeo oiono odoaonogoeoro > > ooofo > > otohoeo ofoioroeo ooofo ohoeololo. > > >
Re: [whatwg] Canvas image to blob/dataurl within Worker
I'd rather we did that by introducing promises to HTMLCanvasElement. Returning a promise from toBlob is easy, making the callback arg optional by checking the type of the first arg is hacky but possible (and is done in js libs). On Sat, 21 Mar 2015 10:56 Robert O'Callahan wrote: > On Sat, Mar 21, 2015 at 5:45 PM, Rik Cabanier wrote: > >> Ah, OK. I thought we were changing it for both cases. This will cause a >> lot >> of confusion... >> > > If we want to keep HTMLCanvasElement and WorkerCanvas in sync, we can. > > Rob > -- > oIo otoeololo oyooouo otohoaoto oaonoyooonoeo owohooo oioso oaonogoroyo > owoiotoho oao oboroootohoeoro oooro osoiosotoeoro owoiololo oboeo > osouobojoeocoto otooo ojouodogomoeonoto.o oAogoaoiono,o oaonoyooonoeo > owohooo > osoaoyoso otooo oao oboroootohoeoro oooro osoiosotoeoro,o o‘oRoaocoao,o’o > oioso > oaonosowoeoroaoboloeo otooo otohoeo ocooouoroto.o oAonodo oaonoyooonoeo > owohooo > osoaoyoso,o o‘oYooouo ofolo!o’o owoiololo oboeo oiono odoaonogoeoro > ooofo > otohoeo ofoioroeo ooofo ohoeololo. >
Re: [whatwg] Canvas image to blob/dataurl within Worker
On Sat, Mar 21, 2015 at 5:45 PM, Rik Cabanier wrote: > Ah, OK. I thought we were changing it for both cases. This will cause a lot > of confusion... > If we want to keep HTMLCanvasElement and WorkerCanvas in sync, we can. Rob -- oIo otoeololo oyooouo otohoaoto oaonoyooonoeo owohooo oioso oaonogoroyo owoiotoho oao oboroootohoeoro oooro osoiosotoeoro owoiololo oboeo osouobojoeocoto otooo ojouodogomoeonoto.o oAogoaoiono,o oaonoyooonoeo owohooo osoaoyoso otooo oao oboroootohoeoro oooro osoiosotoeoro,o o‘oRoaocoao,o’o oioso oaonosowoeoroaoboloeo otooo otohoeo ocooouoroto.o oAonodo oaonoyooonoeo owohooo osoaoyoso,o o‘oYooouo ofolo!o’o owoiololo oboeo oiono odoaonogoeoro ooofo otohoeo ofoioroeo ooofo ohoeololo.
Re: [whatwg] Canvas image to blob/dataurl within Worker
On Fri, Mar 20, 2015 at 9:42 PM, Robert O'Callahan wrote: > On Sat, Mar 21, 2015 at 3:38 PM, Rik Cabanier wrote: > >> Do you know how many site use toBlob in Firefox? >> A quick search on github shows a very high number of pages [1] so it >> might be too late to change. >> >> Maybe you can keep the callback and return a promise? >> > > None of them use WorkerCanvas.toBlob since we don't implement that yet. > Ah, OK. I thought we were changing it for both cases. This will cause a lot of confusion...
Re: [whatwg] Canvas image to blob/dataurl within Worker
On Sat, Mar 21, 2015 at 3:38 PM, Rik Cabanier wrote: > Do you know how many site use toBlob in Firefox? > A quick search on github shows a very high number of pages [1] so it might > be too late to change. > > Maybe you can keep the callback and return a promise? > None of them use WorkerCanvas.toBlob since we don't implement that yet. Rob -- oIo otoeololo oyooouo otohoaoto oaonoyooonoeo owohooo oioso oaonogoroyo owoiotoho oao oboroootohoeoro oooro osoiosotoeoro owoiololo oboeo osouobojoeocoto otooo ojouodogomoeonoto.o oAogoaoiono,o oaonoyooonoeo owohooo osoaoyoso otooo oao oboroootohoeoro oooro osoiosotoeoro,o o‘oRoaocoao,o’o oioso oaonosowoeoroaoboloeo otooo otohoeo ocooouoroto.o oAonodo oaonoyooonoeo owohooo osoaoyoso,o o‘oYooouo ofolo!o’o owoiololo oboeo oiono odoaonogoeoro ooofo otohoeo ofoioroeo ooofo ohoeololo.
Re: [whatwg] Canvas image to blob/dataurl within Worker
On Fri, Mar 20, 2015 at 3:15 PM, Robert O'Callahan
wrote:
> On Sat, Mar 21, 2015 at 1:13 AM, Jake Archibald
> wrote:
>
> > Receiving a push message results in a 'push' event within the
> > ServiceWorker. The likely action at this point is to show a notification.
> > It should be possible to generate an image to use as an icon for this
> > notification (
> > https://notifications.spec.whatwg.org/#dom-notificationoptions-icon).
> >
> > This could be a badge showing some kind of unread count, some combination
> > of app icon & avatar, or even a default avatar (Google Inbox generates an
> > avatar from the senders names first letter).
> >
> > This is also useful for generating images to go into the cache API, or
> > transforming images as they pass through the ServiceWorker.
> >
> > API:
> >
> > Almost all the pieces already exist, except a way to get the image data
> of
> > a CanvasRenderingContext2D into a format that can be read from a
> > url. ImageBitmap seems like a good fit for such an API:
> >
> > var context = new CanvasRenderingContext2D(192, 192);
> >
> > Promise.all(
> > caches.match('/avatars/ben.png')
> > .then(r => r.blob())
> > .then(b => createImageBitmap(b)),
> > caches.match('/avatars/julie.png')
> > .then(r => r.blob())
> > .then(b => createImageBitmap(b)),
> > ).then(([ben, julie]) => {
> > context.drawImage(ben, 0, 0);
> > context.drawImage(julie, 96, 96);
> > return createImageBitmap(context);
> > }).then(
> > // and here's the bit we're missing…
> > image => image.toDataURL()
> > ).then(icon => {
> > self.registration.showNotificaiton("Hello!", {icon});
> > });
> >
>
> My understanding is that the current consensus proposal for canvas in
> Workers is not what's in the spec, but this:
> https://wiki.whatwg.org/wiki/WorkerCanvas
> See "Canvas in Workers" threads from October 2013 for the discussion. svn
> is failing me but the CanvasProxy proposal in the spec definitely predates
> those threads.
>
> Ian, unless I'm wrong, it would be helpful to remove the CanvasProxy stuff
> from the spec to avoid confusion.
>
> That proposal contains WorkerCanvas.toBlob, which needs to be updated to
> use promises
>
Do you know how many site use toBlob in Firefox?
A quick search on github shows a very high number of pages [1] so it might
be too late to change.
Maybe you can keep the callback and return a promise?
1: https://github.com/search?l=javascript&q=toblob&type=Code&utf8=%E2%9C%93
Re: [whatwg] Canvas image to blob/dataurl within Worker
On Sat, Mar 21, 2015 at 1:13 AM, Jake Archibald
wrote:
> Receiving a push message results in a 'push' event within the
> ServiceWorker. The likely action at this point is to show a notification.
> It should be possible to generate an image to use as an icon for this
> notification (
> https://notifications.spec.whatwg.org/#dom-notificationoptions-icon).
>
> This could be a badge showing some kind of unread count, some combination
> of app icon & avatar, or even a default avatar (Google Inbox generates an
> avatar from the senders names first letter).
>
> This is also useful for generating images to go into the cache API, or
> transforming images as they pass through the ServiceWorker.
>
> API:
>
> Almost all the pieces already exist, except a way to get the image data of
> a CanvasRenderingContext2D into a format that can be read from a
> url. ImageBitmap seems like a good fit for such an API:
>
> var context = new CanvasRenderingContext2D(192, 192);
>
> Promise.all(
> caches.match('/avatars/ben.png')
> .then(r => r.blob())
> .then(b => createImageBitmap(b)),
> caches.match('/avatars/julie.png')
> .then(r => r.blob())
> .then(b => createImageBitmap(b)),
> ).then(([ben, julie]) => {
> context.drawImage(ben, 0, 0);
> context.drawImage(julie, 96, 96);
> return createImageBitmap(context);
> }).then(
> // and here's the bit we're missing…
> image => image.toDataURL()
> ).then(icon => {
> self.registration.showNotificaiton("Hello!", {icon});
> });
>
My understanding is that the current consensus proposal for canvas in
Workers is not what's in the spec, but this:
https://wiki.whatwg.org/wiki/WorkerCanvas
See "Canvas in Workers" threads from October 2013 for the discussion. svn
is failing me but the CanvasProxy proposal in the spec definitely predates
those threads.
Ian, unless I'm wrong, it would be helpful to remove the CanvasProxy stuff
from the spec to avoid confusion.
That proposal contains WorkerCanvas.toBlob, which needs to be updated to
use promises.
Rob
--
oIo otoeololo oyooouo otohoaoto oaonoyooonoeo owohooo oioso oaonogoroyo
owoiotoho oao oboroootohoeoro oooro osoiosotoeoro owoiololo oboeo
osouobojoeocoto otooo ojouodogomoeonoto.o oAogoaoiono,o oaonoyooonoeo
owohooo
osoaoyoso otooo oao oboroootohoeoro oooro osoiosotoeoro,o o‘oRoaocoao,o’o
oioso
oaonosowoeoroaoboloeo otooo otohoeo ocooouoroto.o oAonodo oaonoyooonoeo
owohooo
osoaoyoso,o o‘oYooouo ofolo!o’o owoiololo oboeo oiono odoaonogoeoro
ooofo
otohoeo ofoioroeo ooofo ohoeololo.
Re: [whatwg] Canvas image to blob/dataurl within Worker
+1 I've tried to workaround the limitation with some pureJS libraries (such as http://todataurl-png-js.googlecode.com/svn/trunk/todataurl.js) but having a native implementation would be so much better.
Re: [whatwg] Canvas image to blob/dataurl within Worker
On 3/20/15 8:53 AM, Kenji Baheux wrote: I believe that toDataURL is on Canvas not CanvasRenderingContext2D. https://html.spec.whatwg.org/multipage/scripting.html#canvasrenderingcontext2d The CanvasRenderingContext2D.canvas being null, there is no way to use it. Oh. Yeah, that's silly; we should add something to do this. -Boris
Re: [whatwg] Canvas image to blob/dataurl within Worker
I believe that toDataURL is on Canvas not CanvasRenderingContext2D. https://html.spec.whatwg.org/multipage/scripting.html#canvasrenderingcontext2d The CanvasRenderingContext2D.canvas being null, there is no way to use it.
Re: [whatwg] Canvas image to blob/dataurl within Worker
On 3/20/15 8:13 AM, Jake Archibald wrote: Almost all the pieces already exist, except a way to get the image data of a CanvasRenderingContext2D into a format that can be read from a url. There's CanvasRenderingContext2D.toDataURL; I assume the problem is that it's not async, yes? But if you're inside a worker anyway, do you care? -Boris
[whatwg] Canvas image to blob/dataurl within Worker
Usecase:
Receiving a push message results in a 'push' event within the
ServiceWorker. The likely action at this point is to show a notification.
It should be possible to generate an image to use as an icon for this
notification (
https://notifications.spec.whatwg.org/#dom-notificationoptions-icon).
This could be a badge showing some kind of unread count, some combination
of app icon & avatar, or even a default avatar (Google Inbox generates an
avatar from the senders names first letter).
This is also useful for generating images to go into the cache API, or
transforming images as they pass through the ServiceWorker.
API:
Almost all the pieces already exist, except a way to get the image data of
a CanvasRenderingContext2D into a format that can be read from a
url. ImageBitmap seems like a good fit for such an API:
var context = new CanvasRenderingContext2D(192, 192);
Promise.all(
caches.match('/avatars/ben.png')
.then(r => r.blob())
.then(b => createImageBitmap(b)),
caches.match('/avatars/julie.png')
.then(r => r.blob())
.then(b => createImageBitmap(b)),
).then(([ben, julie]) => {
context.drawImage(ben, 0, 0);
context.drawImage(julie, 96, 96);
return createImageBitmap(context);
}).then(
// and here's the bit we're missing…
image => image.toDataURL()
).then(icon => {
self.registration.showNotificaiton("Hello!", {icon});
});
