[webkit-dev] New web-facing canvas feature: "opaque" attribute
Hi WebKittens, I'm planning to implement the canvas "opaque" attribute, as proposed here: http://lists.w3.org/Archives/Public/public-whatwg-archive/2013Mar/0109.html. This is an attribute that causes the allocation of an opaque backing store for , allowing optimizations at the time the canvas is composited into the page, such as disabling blending and culling obscured content. It is based on the moz-opaque attribute currently shipping in Firefox. I'll be placing the feature behind the build-time flag ENABLE(OPAQUE_CANVAS). Let me know if you have any comments or concerns. Stephen ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] New web-facing canvas feature: "opaque" attribute
Hi Dirk, There have been at least five options considered, with contributions from Chromium, Adobe and Mozilla so far. The moz-opaque idea was first floated by Robert O'Callahan from Mozilla, and Ian Hickson offered to spec it if another browser vendor wanted to implement it. I took him up on that offer, and have made my humble effort to massage it into a concrete proposal in the linked message above. After proposing it here, the alternative suggestion is to sync it up with the WebGL syntax, and use a context creation object at getContext() time rather than an attribute on the element. I have no strong feelings about this either way, and I'm working on a patch to try out the WebGL approach (I already have a WebKit patch which implements the platform-independent parts of the opaque attribute approach). However, if we do go that way, I'd prefer not to make this proposal conditional on changes to the WebGL spec, concerns which I've outlined over on what-wg. Stephen On Wed, Mar 13, 2013 at 12:30 PM, Dirk Schulze wrote: > This is a very long thread and I did not see any conclusions or agreement > on this thread. Can you summarize the topic and the status on the > acceptance level please? > > Greetings, > Dirk > > On Mar 13, 2013, at 9:15 AM, Stephen White > wrote: > > > Hi WebKittens, > > > > I'm planning to implement the canvas "opaque" attribute, as proposed > here: > http://lists.w3.org/Archives/Public/public-whatwg-archive/2013Mar/0109.html > . > > > > This is an attribute that causes the allocation of an opaque backing > store for , allowing optimizations at the time the canvas is > composited into the page, such as disabling blending and culling obscured > content. It is based on the moz-opaque attribute currently shipping in > Firefox. > > > > I'll be placing the feature behind the build-time flag > ENABLE(OPAQUE_CANVAS). > > > > Let me know if you have any comments or concerns. > > > > Stephen > > ___ > > webkit-dev mailing list > > webkit-dev@lists.webkit.org > > https://lists.webkit.org/mailman/listinfo/webkit-dev > > ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] New web-facing canvas feature: "opaque" attribute
On Thu, Mar 14, 2013 at 4:38 PM, Ryosuke Niwa wrote: > On Thu, Mar 14, 2013 at 12:55 PM, Dean Jackson wrote: > >> >> On 15/03/2013, at 6:50 AM, Dana Jansens wrote: >> >> On Thu, Mar 14, 2013 at 3:46 PM, Dean Jackson wrote: >> >>> I'm not sure I like this proposal. Why is canvas special? Why doesn't >>> get an opaque attribute (or flag)? Why not every element? >>> >> >> There is ongoing work to infer opaqueness in every other kind of element >> when possible. See for example >> https://bugs.webkit.org/show_bug.cgi?id=70634 >> >> >> Yes, I'd prefer to infer it rather than specify it. For example, I could >> infer that a canvas is opaque if it has a non-transparent CSS >> background-color. >> > > I like this approach. It means that developers don't have to explicitly > use this feature to get the performance benefits. > > In fact, this is the preferred performance optimization approach on the > Web. We don't provide explicit APIs to optimize performance. We make > sensible APIs which allows us to implement more optimizations on common > cases behind the scene. > The problem is, automatically determining opacity automatically may itself incur a performance hit. For example, it would be impossible to determine if putImageData() was going to result in an opaque canvas without inspecting each pixel. By simply describing the use case and constraints up front, you allow the user agent to optimize what it could not otherwise do. The problem is compounded for subpixel antialiased text, where determining automatically when it is safe to use can require double the memory or VRAM, and potentially double the rendering time, due to canvas being an immediate mode API, rather than retained mode like most of the rest of the DOM. See http://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2013-February/038958.html. For that reason, it would still require an opt-in in order not to impact the performance of existing pages. At any rate, I appreciate your feedback, and welcome you to contribute to the whatwg thread. Stephen > > - R. Niwa > > > ___ > webkit-dev mailing list > webkit-dev@lists.webkit.org > https://lists.webkit.org/mailman/listinfo/webkit-dev > > ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] New web-facing canvas feature: "opaque" attribute
On Thu, Mar 14, 2013 at 5:23 PM, Dean Jackson wrote: > > On 15/03/2013, at 8:06 AM, Gregg Tavares wrote: > > > Because it's not the same as fillRect(0, 0, width, height) on an empty > canvas. The canvas itself has alpha (unless we add the option to not have > it as has been proposed). The contents of the canvas has to stay as the > user created it. If I draw with rgba(255,255,0, 0.5) I expect if I read > data out of the canvas or draw that canvas into another canvas I'll get > that color, not the color blended with the css background. > > Yes, this is what I said in another email. Maybe I'm misunderstanding > this, but if the main concern is to guarantee nice subpixel-antialiased > text in canvas (but not anywhere else, such as the 99.99% of places where > people draw text) then well, I'm still not convinced opaque is a great > idea :) Especially not as an HTML attribute. > > There are obviously ways to get around the problems you mentioned above > (e.g. two buffers + two draws, or keeping a display list until someone > wants to read out, etc) and, even more obviously, they have significant > problems. It just seems to me that we're trying to address the issue of > wanting nice looking text with a very specific solution on one element. > Maybe we should consider what we could do across the platform? > Canvas is somewhat different from the rest of the web platform. Since most DOM rendering is essentially retained mode, you always have the option to decide at render time whether or not to use subpixel antialiasing, or to re-render a previously-rendered buffer when necessary (and Chrome and Safari already do this). Since Canvas is immediate-mode, you don't have that option. If a canvas is drawn once, then later rotated via CSS, the "is LCD text legal" probe is now stale. That said, using the opaque attribute (or alpha:none) doesn't completely solve this either. It only guarantees that at least subpixel AA text within the canvas itself will not fringe. There's no guarantee that it won't when transformed via CSS (or WebGL) later. As you point out, the fully-automatic solutions to those problems (which have been discussed over on whatwg) themselves have unpleasant performance implications, or complexity of implementation, and don't entirely solve the problem either (e.g., canvas->WebGL). My preference is actually for a canvas context attribute which enables subpixel antialiased text unconditionally. It makes no promises at all about immunity from color fringing, but gives the power to the developer (a fully-loaded footgun). Native immediate-mode graphics APIs have this capability, so it seems natural to provide the same power to the web platform, IMHO. But let's continue that discussion on whatwg, since it's not specific to WebKit. Stephen > > Dean > > > > So, the canvas has to be blended if there's alpha. There's no magic > getting around that. The only way around it is to give the user a way to > say "no alpha". > > > > ___ > webkit-dev mailing list > webkit-dev@lists.webkit.org > https://lists.webkit.org/mailman/listinfo/webkit-dev > ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] New web-facing canvas feature: "opaque" attribute
For reference, here is a prototype patch which implements :
https://bugs.webkit.org/show_bug.cgi?id=112624
And one which implements getContext('2d', { alpha: none }):
https://bugs.webkit.org/show_bug.cgi?id=112628
I believe this represents the bulk of the cross-port work that would be
required; there is further port-specific work required, but this should be
mostly in platform/graphics/*.
Stephen
On Thu, Mar 14, 2013 at 5:23 PM, Dean Jackson wrote:
>
> On 15/03/2013, at 8:06 AM, Gregg Tavares wrote:
>
> > Because it's not the same as fillRect(0, 0, width, height) on an empty
> canvas. The canvas itself has alpha (unless we add the option to not have
> it as has been proposed). The contents of the canvas has to stay as the
> user created it. If I draw with rgba(255,255,0, 0.5) I expect if I read
> data out of the canvas or draw that canvas into another canvas I'll get
> that color, not the color blended with the css background.
>
> Yes, this is what I said in another email. Maybe I'm misunderstanding
> this, but if the main concern is to guarantee nice subpixel-antialiased
> text in canvas (but not anywhere else, such as the 99.99% of places where
> people draw text) then well, I'm still not convinced opaque is a great
> idea :) Especially not as an HTML attribute.
>
> There are obviously ways to get around the problems you mentioned above
> (e.g. two buffers + two draws, or keeping a display list until someone
> wants to read out, etc) and, even more obviously, they have significant
> problems. It just seems to me that we're trying to address the issue of
> wanting nice looking text with a very specific solution on one element.
> Maybe we should consider what we could do across the platform?
>
> Dean
>
>
> > So, the canvas has to be blended if there's alpha. There's no magic
> getting around that. The only way around it is to give the user a way to
> say "no alpha".
>
>
>
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev
>
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Canvas performance and memory usage
On Mon, Aug 9, 2010 at 6:17 PM, David Hyatt wrote: > There are other cases as well where you want a copy. Patterns are another > example. For example you can create a pattern from another canvas, and I > don't think it's supposed to be live if that other canvas later changes. > There are examples in SVG as well where patterns are used and I am pretty > sure a copy is the desired behavior. > > I don't think it's as simple as saying "image()" should never copy. > > I spent some time thinking about this from a CG perspective, and I couldn't > really come up with a good solution. We use a bitmap context for efficient > pixel access for get/putImageData. Any platform that wants these functions > to be efficient has to keep the pixels easily obtainable. This means, for > example, that we can't produce a live image using CGLayers. Looking at Qt, > the only current platform to return a "live" image to canvas for rendering, > it looks like they made this tradeoff, i.e., it looks to me like > get/putImageData are slower than other platforms. (The get is having to > call toImage() on a pixmap and then has to convert to obtain a good > representation of the data for a QImage). The put has to render the changes > into the pixmap. > > I tried renaming the two methods (image() and imageForRendering()) to > copiedImage and liveImage, and ended up with about an 80/20 split at the > call sites between the two. It looked really ugly to me, since in nearly > every case the "liveImage" was just being used to paint the current contents > of the ImageBuffer, i.e., it was just being passed to drawImage. The > copiedImage was typically passed to somebody who needed to retain it for > later rendering, e.g., a pattern. > > So to sum up, the call sites want two things: > > (1) To render the current contents of the image buffer efficiently. > Would it make sense to add a GraphicsContext::drawImageBuffer(ImageBuffer*, ...) call for this use case? Then the details of how the image is actually extracted (or not) could be kept down in the platform-specific layers. I don't know if this entirely helps the problem under discussion, but I ran into it in another context and it seemed like it would be useful. Stephen (2) To obtain a copied image that is independent of the image buffer that is > typically retained by someone else. > > And the current implementations of image()/imageForRendering() do three > things: > > (1) Make a copy of the image buffer into an Image (e.g., Skia). > (2) Make an Image that is a live representation of the Image buffer > contents (e.g., Qt using imageForRendering). > (3) Make a copy-on-write image wrapper (e.g., CG). > > Assuming the copy-on-write stuff in CG is actually working (and I suspect > that it is), then it's actually a pretty good solution. You don't copy to > do throwaway drawing. Pixel access is efficient using get/PutImageData. > There's no need to distinguish between copied images vs. live, with the > caveat that the cached image has to be cleared (only an issue for canvas, > since nobody else does continual drawing to ImageBuffers). > > Anyway I'm open to suggestions here. :) > > As for removing the ImageBuffer completely, I don't think you can get away > with that. The canvas buffer's size does not necessarily match the size > into which it is drawn. You need to be able to get to the image pixels of a > possibly larger (or smaller) buffer. You also have to be able to draw the > contents of one canvas into another canvas, so you have to be able to treat > the canvas as an image. Again that means being able to get to the contents > of the canvas itself as a separate buffer/image. > > dave > (hy...@apple.com) > > ___ > webkit-dev mailing list > webkit-dev@lists.webkit.org > http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev > ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] pixel tests and --tolerance (was Re: Pixel test experiment)
I'm not sure if this could be make to work with SVG (might require some additions to LayoutTestController), but Philip Taylor's test suite (in LayoutTests/canvas/philip) compares pixels programmatically in JavaScript. This has the major advantage that it doesn't require pixel results, and allows for a per-test level of fuzziness/tolerance (if required). Obviously we would still want to have some tests remain pixel tests, as these tests only cover a subset of pixels, but it might be a good alternative to consider when writing new tests (especially for regressions, where a single pixel correctly chosen can often correctly isolate the problem). Stephen On Thu, Oct 14, 2010 at 12:06 PM, Ojan Vafai wrote: > Simon, are you suggesting that we should only use pixel results for ref > tests? If not, then we still need to come to a conclusion on this tolerance > issue. > > Dirk, implementing --tolerance in NRWT isn't that hard is it? Getting rid > of --tolerance will be a lot of work of making sure all the pixel results > that currently pass also pass with --tolerance=0. While I would support > someone doing that work, I don't think we should block moving to NRWT on it. > > Ojan > > On Fri, Oct 8, 2010 at 1:03 PM, Simon Fraser wrote: > >> I think the best solution to this pixel matching problem is ref tests. >> >> How practical would it be to use ref tests for SVG? >> >> Simon >> >> On Oct 8, 2010, at 12:43 PM, Dirk Pranke wrote: >> >> > Jeremy is correct; the Chromium port has seen real regressions that >> > virtually no concept of a fuzzy match that I can imagine would've >> > caught. >> > new-run-webkit-tests doesn't currently support the tolerance concept >> > at al, and I am inclined to argue that it shouldn't. >> > >> > However, I frequently am wrong about things, so it's quite possible >> > that there are good arguments for supporting it that I'm not aware of. >> > I'm not particularly interested in working on a tool that doesn't do >> > what the group wants it to do, and I would like all of the other >> > WebKit ports to be running pixel tests by default (and >> > new-run-webkit-tests ;) ) since I think it catches bugs. >> > >> > As far as I know, the general sentiment on the list has been that we >> > should be running pixel tests by default, and the reason that we >> > aren't is largely due to the work involved in getting them back up to >> > date and keeping them up to date. I'm sure that fuzzy matching reduces >> > the work load, especially for the sort of mismatches caused by >> > differences in the text antialiasing. >> > >> > In addition, I have heard concerns that we'd like to keep fuzzy >> > matching because people might potentially get different results on >> > machines with different hardware configurations, but I don't know that >> > we have any confirmed cases of that (except for arguably the case of >> > different code paths for gpu-accelerated rendering vs. unaccelerated >> > rendering). >> > >> > If we made it easier to maintain the baselines (improved tooling like >> > the chromium's rebaselining tool, add reftest support, etc.) are there >> > still compelling reasons for supporting --tolerance -based testing as >> > opposed to exact matching? >> > >> > -- Dirk >> > >> > On Fri, Oct 8, 2010 at 11:14 AM, Jeremy Orlow >> wrote: >> >> I'm not an expert on Pixel tests, but my understanding is that in >> Chromium >> >> (where we've always run with tolerance 0) we've seen real regressions >> that >> >> would have slipped by with something like tolerance 0.1. When you have >> >> 0 tolerance, it is more maintenance work, but if we can avoid >> regressions, >> >> it seems worth it. >> >> J >> >> >> >> On Fri, Oct 8, 2010 at 10:58 AM, Nikolas Zimmermann >> >> wrote: >> >>> >> >>> Am 08.10.2010 um 19:53 schrieb Maciej Stachowiak: >> >>> >> >> On Oct 8, 2010, at 12:46 AM, Nikolas Zimmermann wrote: >> >> > >> > Am 08.10.2010 um 00:44 schrieb Maciej Stachowiak: >> > >> >> >> >> On Oct 7, 2010, at 6:34 AM, Nikolas Zimmermann wrote: >> >> >> >>> Good evening webkit folks, >> >>> >> >>> I've finished landing svg/ pixel test baselines, which pass with >> >>> --tolerance 0 on my 10.5 & 10.6 machines. >> >>> As the pixel testing is very important for the SVG tests, I'd like >> to >> >>> run them on the bots, experimentally, so we can catch regressions >> easily. >> >>> >> >>> Maybe someone with direct access to the leopard & snow leopard >> bots, >> >>> could just run "run-webkit-tests --tolerance 0 -p svg" and mail me >> the >> >>> results? >> >>> If it passes, we could maybe run the pixel tests for the svg/ >> >>> subdirectory on these bots? >> >> >> >> Running pixel tests would be great, but can we really expect the >> >> results to be stable cross-platform with tolerance 0? Perhaps we >> should >> >> start with a higher tolerance level. >> > >> > Sure, we could do that. But I'd really like
Re: [webkit-dev] layout tests: how are some compared against image, and others only text?
On Wed, Jan 4, 2012 at 6:39 PM, Elliot Poger wrote: > What is it that causes some tests to require baseline images (and not text > files) for comparison, while others require text and not image baselines? > > (I know that I can specifically SKIP comparison against IMAGE and/or TEXT > using test_expectations.txt... but even without the use of > test_expectations, I believe that some tests are compared against only text > or only image.) > > As an example, I see that this test has only image baselines and no text > baselines: > > $ ls LayoutTests/platform/*/fast/canvas/canvas-text-baseline*.png | wc -l > 10 > $ ls LayoutTests/platform/*/fast/canvas/canvas-text-baseline*.txt | wc -l > ls: cannot access > LayoutTests/platform/*/fast/canvas/canvas-text-baseline*.txt: No such file > or directory > Tests with platform-independent results will often put the .txt files in the same directory as the .html. (the final fallback path for layout test results is the source directory.) Try: $ ls LayoutTests/fast/canvas/canvas-text-baseline*.txt LayoutTests/fast/canvas/canvas-text-baseline-expected.txt > while this test has only text baselines and no image baselines: > > $ ls LayoutTests/platform/*/fast/canvas/canvas-lineWidth*.png | wc -l > ls: cannot access > LayoutTests/platform/*/fast/canvas/canvas-lineWidth*.png: No such file or > directory > 0 > $ ls LayoutTests/platform/*/fast/canvas/canvas-lineWidth*.txt | wc -l > 5 > > Is there something inherent in each test that indicates whether its > results will be compared against image and/or text baselines? Or is it > simply a matter of what baseline files are found to compare against? > As rniwa@ mentioned, this is controlled by the test's JS code itself with layoutTestController.dumpAsText(). Stephen > > > ___ > webkit-dev mailing list > webkit-dev@lists.webkit.org > http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev > > ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] test_expectations.txt for non-chromium ports
On Mon, Feb 13, 2012 at 9:16 PM, Maciej Stachowiak wrote: > > On Feb 13, 2012, at 3:22 PM, Ojan Vafai wrote: > > On Mon, Feb 13, 2012 at 3:09 PM, Maciej Stachowiak wrote: > >> >> >> >>> I do agree that distinguishing "test not applicable to this port" from >>> "this test is temporarily failing for unknown reasons" is a good thing to >>> do. It is unfortunate that we don't make the distinction very well right >>> now. >>> >> >> test_expectations.txt has a WONTFIX modifier for this purpose. Chromium >> used to have two files test_expectations.txt and >> test_expectations_wontfix.txt instead of having the modifier. I would kind >> of like us to move back to that model because then test_expectations.txt is >> a file that you hope to keep completely empty >> and test_expectations_wontfix.txt is a file that your rarely touch. >> >> >> It's good that there is a state for this, but WONTFIX doesn't seem like a >> great name to me, at least for tests that are inapplicable due to missing >> features. It implies both that the missing feature is by definition a bug, >> and also that the decision will not be reconsidered. Granted, this may be >> bikeshedding, but if I were, say, disabling tests for Apple's port that >> relate to the new WebSockets protocol because we don't have it on yet, I >> would be very reluctant to mark them WONTFIX. >> >> For tests that are inapplicable for reasons other than missing features, >> it may be simply that there is more than one acceptable behavior, in which >> case WONTFIX seems wrong as well. >> > > The intention of WONTFIX is exactly that the decision likely won't need to > be reconsidered (e.g. because the test is platform specific). For the other > purposes (e.g. websockets), we use SKIP. I actually believe we should > rename WONTFIX to NEVERFIX to make it even more clear. > > > I don't know about other organizations, but from Apple's point of view, > it's rare that we'd want to publicly promise that we'll never implement > something. We'd just want to document that we haven't implemented the > feature yet, and thus some tests are inapplicable. So NEVERFIX would be > something we'd be even more reluctant to apply. We would not even want to > mark the difference between "we haven't enabled this feature yet, but > probably will very soon" and "we have no plans to ever implement the > feature unless something changes", as that would be communicating > information about future releases. > > I don't know of the intent of SKIP, but maybe it is ok for this purpose. I > would expect it to be used for tests that are temporarily skipped due to > bugs, based on the name, which seems different to me from "this > functionality is not implemented in this port, rendering the test > inapplicable". > How about NOTIMPL? Stephen > > Regards, > Maciej > > > ___ > webkit-dev mailing list > webkit-dev@lists.webkit.org > http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev > > ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
