Re: [dev-servo] Compositor Tree Redesign
The copy-on-write lock is just an atomic reference count that we put in shared memory when doing multi-process. On Wed, Jul 2, 2014 at 5:49 PM, Patrick Walton pcwal...@mozilla.com wrote: On 7/2/14 5:51 AM, Nicolas Silva wrote: In the internal buffer scenario, we don't need to create a new shmem or swap back and forth between the front and back textures if the compositor is done uploading the texture. So we have a lock that the content thread takes when painting and that is released by the compositor after the texture is uploaded. Next time a tile needs to be painted we only create a back buffer if the front buffer's lock is still taken, in which case we copy the front tile's content into the back and repaint the damaged region (this is the copy-on-write I was referring to). How do you do this locking and reference counting in a multiprocess scenario? Patrick ___ dev-servo mailing list dev-servo@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-servo ___ dev-servo mailing list dev-servo@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-servo
Re: [dev-servo] Compositor Tree Redesign
Hi there, A few words about TextureClient/Host (since it's been mentioned in this thread): we have a bit of design doc about it at: * http://dxr.mozilla.org/mozilla-central/source/gfx/doc/MozSurface.md * http://dxr.mozilla.org/mozilla-central/source/gfx/doc/SharedMozSurface.md Basically TextureClient is the content how the content process views a shared texture and TextureHost is how the compositor process views it. long term, we'd like to share more code between TextureClient/Host. We call MozSurface the hypothetical abstraction over TextureClient and TextureHost that we are in the process of adding). One point that I should insist on is whether the shared texture data should be accessible on both sides at the same time: At first we thought that we could just send ownership of the texture along with it when sending a texture to the compositor. This feels like the right thing to do but has bitten us very hard because at any time in gecko the main thread can synchronously ask read access to any layer's content, for instance to take a screen shot, or do some of the optimization we do (copy-on-write tiles, synchronizing the front and back buffers' valid regions to repaint as few texels as possible, etc.). It turns out there are very few places (canvas/video) in the gecko's compositor where we can just get away with a classic swap-chain like most other compositors do. It may turn out that some of the reasons pure message passing failed for gecko will not apply to servo, but I think that, if only for the sake of having copy-on-write textures, shared ownership is a better solution in the end (at the expense of some complicated logic around deallocating the textures). We also need to support having a TextureClient/Host pair used by several layers (for instance when the source of a video element is the output of another one). So our current architecture lets us attach a texture to several layers which adds some constraints. It also took some effort to cleanly support the child process potentially crashing at any time, without taking down the main process with it. What was very hard to get right is ownership of the shared data. There are tons of constraints from how gecko works and from the underlying rendering backends we support. Also there has been a very unhealthy habit of rushing things for one platform without caring about how other platforms differ, and have to do some major refactorings afterward to get it to work everywhere. A thing that is working very well for us is the separation between the structure of the layer tree (Layer[Client/Host] objects), what manipulates textures (and handle things like double-buffering, tiling, swap-chains, etc) (Compositable[Client/Host] objects), and the textures themselves which abstract out the underlying backends (Texture[Client/Host] objects). It's now rather easy for us to have layers be manipulated within transactions on the main thread while a swap chain (Compositable object) sends textures from, say, a webgl worker or a video decoder to the compositor thread without touching the main thread. Two years ago, all of the logic was contained into Layer classes so we had an awful lot of code duplication and it was hard to decouple the position of elements in the page from the data they contained (which is desirable for efficient video compositing and canvas workers). I also think that it is good to have a solid and universal texture abstraction that content can draw into and share with the compositor without copies early on, since it is rather hard to retro-fit the constraints of shareable textures into the rendering code after the fact, and that as long as there are textures being copied, they will show up in profiles. Sorry for the long email, don't hesitate to ask me questions about anything layers-related in gecko. Cheers, Nical On Tue, Jul 1, 2014 at 7:14 AM, Cameron Zwarich zwar...@mozilla.com wrote: We discussed this a bit on #servo, but it isn’t obvious from your diagram if every Layer will have child layers, or if that will be reserved for specific container layers. CoreAnimation takes the former approach, but I think the latter might be more applicable for Servo, since it would give a better separation of concerns. We would then have the following layer types: 1. ContainerLayer, which can only parent other layers. 2. TileLayer, which would manage tiling, BufferRequests, and all double buffering of individual tile backing stores. 3. TextureLayer (maybe ImageLayer would be a better name?), which is backed by a simple image. Would we even want it to be double buffered? How are iframe layer trees handled? I would prefer a model where each Rust task owns a single ‘rendering context’, or whatever we want to call it, and another layer type (HostLayer, ProxyLayer, ???) that hosts one rendering context in another. Maybe this is more general than what we actually need, but synchronization problems will be easier to solve if each task has
Re: [dev-servo] Compositor Tree Redesign
Thanks for the detailed reply. I never expected that Servo would be able to be able to avoid the possibility of concurrent access to surfaces, for a lot of those same reasons that you mention. I have a few questions about the docs. 1. What copy-on-write optimizations is this referring to? * We do some copy-on-write optimizations on surfaces that are shared with the compositor in order to keep invalid regions as small as possible. Out tiling implementation is an example of that.” 2. How do you do front/back buffer management differently for tiled and untiled layers? Cameron On Jul 1, 2014, at 4:58 AM, Nicolas Silva nical.si...@gmail.com wrote: Hi there, A few words about TextureClient/Host (since it's been mentioned in this thread): we have a bit of design doc about it at: * http://dxr.mozilla.org/mozilla-central/source/gfx/doc/MozSurface.md * http://dxr.mozilla.org/mozilla-central/source/gfx/doc/SharedMozSurface.md Basically TextureClient is the content how the content process views a shared texture and TextureHost is how the compositor process views it. long term, we'd like to share more code between TextureClient/Host. We call MozSurface the hypothetical abstraction over TextureClient and TextureHost that we are in the process of adding). One point that I should insist on is whether the shared texture data should be accessible on both sides at the same time: At first we thought that we could just send ownership of the texture along with it when sending a texture to the compositor. This feels like the right thing to do but has bitten us very hard because at any time in gecko the main thread can synchronously ask read access to any layer's content, for instance to take a screen shot, or do some of the optimization we do (copy-on-write tiles, synchronizing the front and back buffers' valid regions to repaint as few texels as possible, etc.). It turns out there are very few places (canvas/video) in the gecko's compositor where we can just get away with a classic swap-chain like most other compositors do. It may turn out that some of the reasons pure message passing failed for gecko will not apply to servo, but I think that, if only for the sake of having copy-on-write textures, shared ownership is a better solution in the end (at the expense of some complicated logic around deallocating the textures). We also need to support having a TextureClient/Host pair used by several layers (for instance when the source of a video element is the output of another one). So our current architecture lets us attach a texture to several layers which adds some constraints. It also took some effort to cleanly support the child process potentially crashing at any time, without taking down the main process with it. What was very hard to get right is ownership of the shared data. There are tons of constraints from how gecko works and from the underlying rendering backends we support. Also there has been a very unhealthy habit of rushing things for one platform without caring about how other platforms differ, and have to do some major refactorings afterward to get it to work everywhere. A thing that is working very well for us is the separation between the structure of the layer tree (Layer[Client/Host] objects), what manipulates textures (and handle things like double-buffering, tiling, swap-chains, etc) (Compositable[Client/Host] objects), and the textures themselves which abstract out the underlying backends (Texture[Client/Host] objects). It's now rather easy for us to have layers be manipulated within transactions on the main thread while a swap chain (Compositable object) sends textures from, say, a webgl worker or a video decoder to the compositor thread without touching the main thread. Two years ago, all of the logic was contained into Layer classes so we had an awful lot of code duplication and it was hard to decouple the position of elements in the page from the data they contained (which is desirable for efficient video compositing and canvas workers). I also think that it is good to have a solid and universal texture abstraction that content can draw into and share with the compositor without copies early on, since it is rather hard to retro-fit the constraints of shareable textures into the rendering code after the fact, and that as long as there are textures being copied, they will show up in profiles. Sorry for the long email, don't hesitate to ask me questions about anything layers-related in gecko. Cheers, Nical On Tue, Jul 1, 2014 at 7:14 AM, Cameron Zwarich zwar...@mozilla.com wrote: We discussed this a bit on #servo, but it isn’t obvious from your diagram if every Layer will have child layers, or if that will be reserved for specific container layers. CoreAnimation takes the former approach, but I think the latter might be more applicable for Servo, since it would
Re: [dev-servo] Compositor Tree Redesign
On 06/30/2014 10:14 PM, Cameron Zwarich wrote: We discussed this a bit on #servo, but it isn’t obvious from your diagram if every Layer will have child layers, or if that will be reserved for specific container layers. The way I've designed it now, TileLayers are not siblings of ContainerLayers, but are instead data members on ContainerLayers. All ContainerLayers can potentially have children, but none of those children will be TileLayers. CoreAnimation takes the former approach, but I think the latter might be more applicable for Servo, since it would give a better separation of concerns. We would then have the following layer types: 1. ContainerLayer, which can only parent other layers. 2. TileLayer, which would manage tiling, BufferRequests, and all double buffering of individual tile backing stores. 3. TextureLayer (maybe ImageLayer would be a better name?), which is backed by a simple image. Would we even want it to be double buffered? I still need to look a lot more closely at how tiling works in the next phase, but this design works for me. I think most of the pieces are already in place actually. --Martin ___ dev-servo mailing list dev-servo@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-servo
Re: [dev-servo] Compositor Tree Redesign
This looks like a huge improvement. Can you give us an idea of what the CompositorData type looks like? jack. On Mon, Jun 30, 2014 at 9:33 AM, Martin Robinson mrobin...@igalia.com wrote: I've been working on a redesign of the compositor, in order to make rust-layers more functional as a standalone library and also to simplify ownership of the layer tree. In the current design, there are two parallel trees, one of rust-layer ContainerLayers/TextureLayers and one of servo CompositorLayers. The main aspects of the rework: * ContainerLayers (and thus rust-layers) will know about tiling and BufferRequests. * ComposiorLayer will disappear, instead tacked on via generics to ContainerLayer with its methods replaced by ContainerLayer helper methods in servo. * The layer tree will be a simple tree owned by Scene, hopefully eliminating mutable roots. Old design: https://docs.google.com/drawings/d/1xcsfjxAsfhpAFIv0C9bMnC1VgiEPsmY5cawdBdiKCT4/edit?usp=sharing New design: https://docs.google.com/drawings/d/1oohFBhHM-AlhKbehTt8gbPPYWtrX_eZE7W7LXPSSxj0/edit I have a patch for the first two parts, which I hope to post soon. It's big and complicated, because of the inherent messiness of reworking the tree structure. My hope is that it can land soon and I can begin to move onto the last phase of the redesign. After the first patch lands and before the work is done, performance may regress, because I've had to create more mutable roots (std::cell::RefCell) to access the data that used to be in ContainerLayer. These should disappear in the future. My hope is that functionality does not regress, though the change is large and invasive, so there may be new unexpected failures. I will try to stay on top of everything. --Martin ___ dev-servo mailing list dev-servo@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-servo ___ dev-servo mailing list dev-servo@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-servo
Re: [dev-servo] Compositor Tree Redesign
On Tue, Jul 1, 2014 at 11:46 AM, Patrick Walton pcwal...@mozilla.com wrote: On 6/30/14 4:44 PM, Robert O'Callahan wrote: BTW have you or anyone else working on Servo looked at TextureClient/TextureHost in Gecko? Getting buffer management interfaces correct across many platforms with multi-process and robustness in the face of crashes and platform quirks has proven to be very hard. I think we're in a good place now but we went down many blind alleys so there's some valuable lessons to be learned. By TextureClient/TextureHost I assume you mean cross-process texture sharing? If so, yes, we have been doing this from day one. (We have to, because we do painting on a separate thread.) Yes. But if you haven't got robust support for D3D10 and Android gralloc and Windows Media Foundation accelerated video decoding and all that sort of thing, you probably don't have the design right unless you're cleverer and luckier than the rest of us :-). Rob -- Jtehsauts tshaei dS,o n Wohfy Mdaon yhoaus eanuttehrotraiitny eovni le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o Whhei csha iids teoa stiheer :p atroa lsyazye,d 'mYaonu,r sGients uapr,e tfaokreg iyvoeunr, 'm aotr atnod sgaoy ,h o'mGee.t uTph eann dt hwea lmka'n? gBoutt uIp waanndt wyeonut thoo mken.o w ___ dev-servo mailing list dev-servo@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-servo
Re: [dev-servo] Compositor Tree Redesign
On 6/30/14 6:20 PM, Robert O'Callahan wrote: Yes. But if you haven't got robust support for D3D10 and Android gralloc and Windows Media Foundation accelerated video decoding and all that sort of thing, you probably don't have the design right unless you're cleverer and luckier than the rest of us :-). It'd be good to discuss what the right design is then. I'm not wedded to our current design--it's pretty bare-bones at the moment anyhow (as things like synchronization are basically unhandled). Patrick ___ dev-servo mailing list dev-servo@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-servo
Re: [dev-servo] Compositor Tree Redesign
On Tue, Jul 1, 2014 at 1:21 PM, Patrick Walton pcwal...@mozilla.com wrote: On 6/30/14 6:20 PM, Robert O'Callahan wrote: Yes. But if you haven't got robust support for D3D10 and Android gralloc and Windows Media Foundation accelerated video decoding and all that sort of thing, you probably don't have the design right unless you're cleverer and luckier than the rest of us :-). It'd be good to discuss what the right design is then. I'm not wedded to our current design--it's pretty bare-bones at the moment anyhow (as things like synchronization are basically unhandled). I don't know the details myself. Nicolas Silva would be a good person to talk to. Maybe you should go through a design review with him. Rob -- Jtehsauts tshaei dS,o n Wohfy Mdaon yhoaus eanuttehrotraiitny eovni le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o Whhei csha iids teoa stiheer :p atroa lsyazye,d 'mYaonu,r sGients uapr,e tfaokreg iyvoeunr, 'm aotr atnod sgaoy ,h o'mGee.t uTph eann dt hwea lmka'n? gBoutt uIp waanndt wyeonut thoo mken.o w ___ dev-servo mailing list dev-servo@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-servo
Re: [dev-servo] Compositor Tree Redesign
On Jun 30, 2014, at 6:20 PM, Robert O'Callahan rob...@ocallahan.org wrote: On Tue, Jul 1, 2014 at 11:46 AM, Patrick Walton pcwal...@mozilla.com wrote: On 6/30/14 4:44 PM, Robert O'Callahan wrote: BTW have you or anyone else working on Servo looked at TextureClient/TextureHost in Gecko? Getting buffer management interfaces correct across many platforms with multi-process and robustness in the face of crashes and platform quirks has proven to be very hard. I think we're in a good place now but we went down many blind alleys so there's some valuable lessons to be learned. By TextureClient/TextureHost I assume you mean cross-process texture sharing? If so, yes, we have been doing this from day one. (We have to, because we do painting on a separate thread.) Yes. But if you haven't got robust support for D3D10 and Android gralloc and Windows Media Foundation accelerated video decoding and all that sort of thing, you probably don't have the design right unless you're cleverer and luckier than the rest of us :-). How much of that changed the basic nature of the abstraction and how much of it was just difficult platform-specific implementation work? Most of my experience is with IOSurface on Darwin, which is comparatively sane. Cameron ___ dev-servo mailing list dev-servo@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-servo
Re: [dev-servo] Compositor Tree Redesign
On Tue, Jul 1, 2014 at 3:27 PM, Cameron Zwarich zwar...@mozilla.com wrote: How much of that changed the basic nature of the abstraction and how much of it was just difficult platform-specific implementation work? Depends on the level of abstraction you're talking about, but we had to restructure our IPC protocols a couple of times to account for specific platform issues. The API between layers and layout did not change, but a lot of the layers implementation did. Rob -- Jtehsauts tshaei dS,o n Wohfy Mdaon yhoaus eanuttehrotraiitny eovni le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o Whhei csha iids teoa stiheer :p atroa lsyazye,d 'mYaonu,r sGients uapr,e tfaokreg iyvoeunr, 'm aotr atnod sgaoy ,h o'mGee.t uTph eann dt hwea lmka'n? gBoutt uIp waanndt wyeonut thoo mken.o w ___ dev-servo mailing list dev-servo@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-servo
Re: [dev-servo] Compositor Tree Redesign
We discussed this a bit on #servo, but it isn’t obvious from your diagram if every Layer will have child layers, or if that will be reserved for specific container layers. CoreAnimation takes the former approach, but I think the latter might be more applicable for Servo, since it would give a better separation of concerns. We would then have the following layer types: 1. ContainerLayer, which can only parent other layers. 2. TileLayer, which would manage tiling, BufferRequests, and all double buffering of individual tile backing stores. 3. TextureLayer (maybe ImageLayer would be a better name?), which is backed by a simple image. Would we even want it to be double buffered? How are iframe layer trees handled? I would prefer a model where each Rust task owns a single ‘rendering context’, or whatever we want to call it, and another layer type (HostLayer, ProxyLayer, ???) that hosts one rendering context in another. Maybe this is more general than what we actually need, but synchronization problems will be easier to solve if each task has its own namespace(s) with the compositor and any relationship between tasks is tracked explicitly. Cameron On Jun 30, 2014, at 8:33 AM, Martin Robinson mrobin...@igalia.com wrote: I've been working on a redesign of the compositor, in order to make rust-layers more functional as a standalone library and also to simplify ownership of the layer tree. In the current design, there are two parallel trees, one of rust-layer ContainerLayers/TextureLayers and one of servo CompositorLayers. The main aspects of the rework: * ContainerLayers (and thus rust-layers) will know about tiling and BufferRequests. * ComposiorLayer will disappear, instead tacked on via generics to ContainerLayer with its methods replaced by ContainerLayer helper methods in servo. * The layer tree will be a simple tree owned by Scene, hopefully eliminating mutable roots. Old design: https://docs.google.com/drawings/d/1xcsfjxAsfhpAFIv0C9bMnC1VgiEPsmY5cawdBdiKCT4/edit?usp=sharing New design: https://docs.google.com/drawings/d/1oohFBhHM-AlhKbehTt8gbPPYWtrX_eZE7W7LXPSSxj0/edit I have a patch for the first two parts, which I hope to post soon. It's big and complicated, because of the inherent messiness of reworking the tree structure. My hope is that it can land soon and I can begin to move onto the last phase of the redesign. After the first patch lands and before the work is done, performance may regress, because I've had to create more mutable roots (std::cell::RefCell) to access the data that used to be in ContainerLayer. These should disappear in the future. My hope is that functionality does not regress, though the change is large and invasive, so there may be new unexpected failures. I will try to stay on top of everything. --Martin ___ dev-servo mailing list dev-servo@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-servo ___ dev-servo mailing list dev-servo@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-servo
