Re: Inter-client surface embedding
Mark Thomas wrote: > > As part of my current attempts to get MATE fully working on Wayland, I was > planning to take a look at porting mate-panel this weekend. However, I > pretty quickly hit the first snag, which is that Gtk on Wayland doesn't > support the GtkSocket/GtkPlug interface, which mate-panel relies heavily > upon. The question I find myself asking is "Do you really need a full-on surface to which you render arbitrarily?" As I understand it, this is intended for the system-tray type use case, correct? Which generally has much simpler requirements in the common case, and the exceptional cases really do make more sense as shell plugins. Isn't that exactly what the proposed StatusNotifier[1] spec (which is used by KDE, I think Gnome as well, likely others) was intended to solve? In particular, it was explicitly intended to get well clear of XEmbed precisely because of how horrid it was to deal with. Because of the way it was designed, it should work with little-to-no modification, regardless of X11, Wayland, or anything else, so long as DBus is around. [1] http://www.notmart.org/misc/statusnotifieritem/index.html ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: Inter-client surface embedding
On 02/19/2014 02:42 AM, Pekka Paalanen wrote: Besides, it's not that much code, really. clients/nested.c is 1140 lines, and that includes support for EGL-passthrough so that nested-client.c can efficiently use GLES rendering. Sample code like this does help a lot. I would still be worried that a client is going to screw up support for something that does not work on that developer's machine, for instance EGL does not work on mine, or some complex synchronization protocol. It is not clear if there is some kind of generic wayland message so the subcompositor client can just pass messages both ways without really understanding them. For my purposes I don't plan to resolve this. I'm going to make a mate-panel-specific pair of interfaces that are used by mate-panel and libmatepanelapplet respectively, and not worry about the interactive resize issue because you can't interactively resize panels. A wise decision. :-) I am still worried that he prefers writing a shell plugin over writing a client program because of this. It seems like we may end up with a lot of shell plugins for reasons users do not understand. If subcompositors are the way to go then this should be addressed so that people are encouraged to use them. ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: Inter-client surface embedding
On Tue, 18 Feb 2014 19:09:45 + (GMT) Mark Thomas wrote: > On Mon, 17 Feb 2014, Bill Spitzak wrote: > > > I do believe users are looking for something more like this than for > > implementing a subcompositor. Subcompositor really worries me as it relies > > on > > the buffers being passed through the intermediate client as fast as > > possible > > (ie without copying), and information that clients use to figure out how to > > allocate their buffers being passed the opposite direction. I can't believe > > every client is going to get this right, and would seem to make it > > impossible > > for a sub-client to take advantage of any new wayland api until the parent > > client is updated. Correct, but the replacement shell interface offered to the mini-compositor's clients is nothing like a generic desktop shell interface, but tailored precisely for the embedding use case at hand. As for other interfaces, that is true, the mini-compositor needs to implement everything its clients may want to use, but that should be quite limited. If some generic Wayland core interface is too complicated, you can simply not implement it, and have the needed subset of the functionality in your own interfaces. > Subcompositor worries me too, particularly for something like a panel, > which does very little, really. We have one compositor already so it > seems strange for it to abdicate responsibility for compositing just > because the parts of the image come from two different clients - surely > the whole point of the compositor is to composite bits from different > clients. "The whole point of the compositor is to composite bits from different client." Deja vu. Indeed, is panel not exactly like a tiny compositor, who takes the bits from panel applets and composites them to form the panel as a whole? Besides, it's not that much code, really. clients/nested.c is 1140 lines, and that includes support for EGL-passthrough so that nested-client.c can efficiently use GLES rendering. > I understand there is the issue synchronization when interactively > resizing, but even that isn't completely solved with a subcompositor, as > what should it do if the nested client isn't responsive? If the mini-compositor's client is unresponsive, the mini-compositor just does what any compositor would: keep showing the old content until the client responds. The mini-compositor being an app itself can delay reacting to the resize action it received. Avoiding drawing an inconsistent window is trivial here. > For my purposes I don't plan to resolve this. I'm going to make a > mate-panel-specific pair of interfaces that are used by mate-panel and > libmatepanelapplet respectively, and not worry about the interactive > resize issue because you can't interactively resize panels. A wise decision. :-) Thanks, pq ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: Inter-client surface embedding
Mark Thomas wrote: I suggest you read up on the subsurface protocol. A "subsurface" object takes two "surface" arguments, one is the parent to attach to, and the other is the child surface that becomes the subsurface. You are right, the actual object I wanted to send from A to B is a surface, not a wl subsurface: A creates a surface A does calls so that surface is a subsurface A gets the id number for that surface and sends it to B using IPC B sends that number to wayland and gets the surface to attach buffers to. I think things like firefox are potentially better served by passing through the buffers from the plugin, rather than the plugin connecting to wayland itself -- a parent-child relationship rather than a sibling one. That way it gets to do things to the buffer synchronously as it passes through (like crop or scale it)*. I think this is what is meant by the "subcompositor" model. This does sound like what is planned for wayland with the subcompositor idea. Firefox would act as a compositor, it's clients talk to it, and it then copies the buffers (hopefully in some zero-copy way) from the phony surfaces the clients are using to the actual surfaces that it sends to wayland. It still bothers me that the subcompositor is not actually doing any compositing, and wayland is therefore aware of the full complexity of the scene, yet all this information must pass through the intermediate compositor. ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: Inter-client surface embedding
On Tue, Feb 18, 2014 at 2:22 PM, Bill Spitzak wrote: > On 02/18/2014 11:09 AM, Mark Thomas wrote: > > I think the above description can be greatly simplified by removing >>> the "hole" and "plug" objects and just using a subsurface: >>> >>> A creates a main surface >>> A creates a subsurface for the "hole" >>> A gets the uid of the subsurface from the compositor >>> A passes that uid to B via IPC >>> B uses this uid to get access to the subsurface >>> B can now attach buffers and do other actions to the subsurface >>> >> >> The "hole" and "plug" are meaningful objects and are needed, at least >> server-side, for some associated state. They're also helpful for >> limiting the amount of uid-dipping a client can do, as only holes and >> plugs have uids. >> > > I proposed that the "uid" be an encrypted number so clients can't guess > them (or "dipping" as you call it). Basically if the server is asked for > the object for a given uid it fails if it is not a number recently > delivered by it to another client. Otherwise you are going to have to > create a "hole" and "plug" object for every single Wayland object. > I have no idea what an "encrypted number" is. > Also you can't create a subsurface without both >> surfaces available, so there is a chicken-and-egg issue, too. >> > > I don't understand this. Obviously when the subsurface is created there is > only one surface available (the parent) because the subsurface is not > created yet. That is why I have A create the subsurface. B is still in > charge of the buffers so there seems to be no problem here. > > > Having thought about it, I think I'm in agreement with Pekka and the >> others that a generic interface is probably not appropriate, as the >> requirements change depending on the use case. What's probably more >> useful is making it easy for shell plugins to do it the way they want. >> So that's my plan. >> > > I don't like the idea that we are saying that things like Firefox have to > be shell plugins. > > > > ___ > wayland-devel mailing list > wayland-devel@lists.freedesktop.org > http://lists.freedesktop.org/mailman/listinfo/wayland-devel > -- Jasper ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: Inter-client surface embedding
On Tue, 18 Feb 2014, Bill Spitzak wrote: On 02/18/2014 11:09 AM, Mark Thomas wrote: The "hole" and "plug" are meaningful objects and are needed, at least server-side, for some associated state. They're also helpful for limiting the amount of uid-dipping a client can do, as only holes and plugs have uids. I proposed that the "uid" be an encrypted number so clients can't guess them (or "dipping" as you call it). Basically if the server is asked for the object for a given uid it fails if it is not a number recently delivered by it to another client. I've essentially done that, although the uid is more of a nonce. Even a random number can be guessed or snooped. Otherwise you are going to have to create a "hole" and "plug" object for every single Wayland object. Only the ones you want to embed. It's a rare occurrence. Plus creating objects is easy because the wayland protocol is very lightweight. Also you can't create a subsurface without both surfaces available, so there is a chicken-and-egg issue, too. I don't understand this. Obviously when the subsurface is created there is only one surface available (the parent) because the subsurface is not created yet. That is why I have A create the subsurface. B is still in charge of the buffers so there seems to be no problem here. I suggest you read up on the subsurface protocol. A "subsurface" object takes two "surface" arguments, one is the parent to attach to, and the other is the child surface that becomes the subsurface. Having thought about it, I think I'm in agreement with Pekka and the others that a generic interface is probably not appropriate, as the requirements change depending on the use case. What's probably more useful is making it easy for shell plugins to do it the way they want. So that's my plan. I don't like the idea that we are saying that things like Firefox have to be shell plugins. I think things like firefox are potentially better served by passing through the buffers from the plugin, rather than the plugin connecting to wayland itself -- a parent-child relationship rather than a sibling one. That way it gets to do things to the buffer synchronously as it passes through (like crop or scale it)*. I think this is what is meant by the "subcompositor" model. It sounds like it could work, though I expect there would be lots of edge cases to work through. Until someone actually tries it, though, this is all just hot air. * - I had an idea for smooth, synchronized resizing and scrolling, where the outer client requests the inner client provide it a buffer with an extra amount of rendered data around the outside of the subwindow that it is providing. The outer window crops this buffer to the correct dimensions (by asking the compositor to do so) and also draws the border of the subwindow and its scroll bars. Then the outer client can scroll and resize synchronously by repainting the existing buffer it has with a differenct crop region, whilst asynchronously requesting a new buffer from the inner client with the new geometry. I think this would work pretty well for the "document viewer" class of use case. Just an idea, though. I don't plan to do anything with it. -- Mark. ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: Inter-client surface embedding
On 02/18/2014 11:09 AM, Mark Thomas wrote: I think the above description can be greatly simplified by removing the "hole" and "plug" objects and just using a subsurface: A creates a main surface A creates a subsurface for the "hole" A gets the uid of the subsurface from the compositor A passes that uid to B via IPC B uses this uid to get access to the subsurface B can now attach buffers and do other actions to the subsurface The "hole" and "plug" are meaningful objects and are needed, at least server-side, for some associated state. They're also helpful for limiting the amount of uid-dipping a client can do, as only holes and plugs have uids. I proposed that the "uid" be an encrypted number so clients can't guess them (or "dipping" as you call it). Basically if the server is asked for the object for a given uid it fails if it is not a number recently delivered by it to another client. Otherwise you are going to have to create a "hole" and "plug" object for every single Wayland object. Also you can't create a subsurface without both surfaces available, so there is a chicken-and-egg issue, too. I don't understand this. Obviously when the subsurface is created there is only one surface available (the parent) because the subsurface is not created yet. That is why I have A create the subsurface. B is still in charge of the buffers so there seems to be no problem here. Having thought about it, I think I'm in agreement with Pekka and the others that a generic interface is probably not appropriate, as the requirements change depending on the use case. What's probably more useful is making it easy for shell plugins to do it the way they want. So that's my plan. I don't like the idea that we are saying that things like Firefox have to be shell plugins. ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: Inter-client surface embedding
On Mon, 17 Feb 2014, Bill Spitzak wrote: Mark Thomas wrote: I've pushed some doc updates to the protocol.xml file my git repo. But in terms of Jonas Ådahl's proposal, my protocol works the other way round: A creates a main surface A creates a "hole" on that surface and sets its position and size A gets the uid (handle) from the server A passes that uid to B via IPC B creates a surface B creates a "plug" on that surface with the uid it got from A B receives a configure event from the server with the size of the hole B creates a buffer of the correct size and renders its image to the surface I do believe users are looking for something more like this than for implementing a subcompositor. Subcompositor really worries me as it relies on the buffers being passed through the intermediate client as fast as possible (ie without copying), and information that clients use to figure out how to allocate their buffers being passed the opposite direction. I can't believe every client is going to get this right, and would seem to make it impossible for a sub-client to take advantage of any new wayland api until the parent client is updated. Subcompositor worries me too, particularly for something like a panel, which does very little, really. We have one compositor already so it seems strange for it to abdicate responsibility for compositing just because the parts of the image come from two different clients - surely the whole point of the compositor is to composite bits from different clients. I understand there is the issue synchronization when interactively resizing, but even that isn't completely solved with a subcompositor, as what should it do if the nested client isn't responsive? For my purposes I don't plan to resolve this. I'm going to make a mate-panel-specific pair of interfaces that are used by mate-panel and libmatepanelapplet respectively, and not worry about the interactive resize issue because you can't interactively resize panels. I think the above description can be greatly simplified by removing the "hole" and "plug" objects and just using a subsurface: A creates a main surface A creates a subsurface for the "hole" A gets the uid of the subsurface from the compositor A passes that uid to B via IPC B uses this uid to get access to the subsurface B can now attach buffers and do other actions to the subsurface The "hole" and "plug" are meaningful objects and are needed, at least server-side, for some associated state. They're also helpful for limiting the amount of uid-dipping a client can do, as only holes and plugs have uids. Also you can't create a subsurface without both surfaces available, so there is a chicken-and-egg issue, too. I proposed this before but I think I am failing to communicate what I wanted. The term "uid" seems pretty good and maybe makes it clearer. It's purpose is so B can't just guess at objects and get access to them, and to be a simple piece of data (probably a big number) that can be passed through IPC, in particular as part of the argv sent to exec the child. Having thought about it, I think I'm in agreement with Pekka and the others that a generic interface is probably not appropriate, as the requirements change depending on the use case. What's probably more useful is making it easy for shell plugins to do it the way they want. So that's my plan. As far as I can tell Wayland will have no difficulties with two clients both owning a subsurface and trying to update it, as it will serialize all the requests and perform them in whatever order they appear. Either client can destroy the surface and the other will cleanly hear about it. I don't think that's true, at least for weston. That may just be an implementation detail, though. -- Mark.___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: Inter-client surface embedding
On Mon, Feb 17, 2014 at 6:41 PM, Jasper St. Pierre wrote: > GtkPlug and GtkSocket are really implemented in terms of XEmbed. As we've > found, XEmbed has a surprising number of problems in real-world use cases, > so it's considered deprecated. > > Building something special-case for panels seems much better than trying > to implement something generic like WaylandEmbed. > > I agree with Jasper - GtkPlug/GtkSocket should be considered X11-specific, and we probably don't want to bring back a generic embedding api. Write something specific that solves your applet embedding needs for the panel. ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: Inter-client surface embedding
On Mon, Feb 17, 2014 at 10:59:11PM +, Mark Thomas wrote: > On Mon, 17 Feb 2014, Pekka Paalanen wrote: > > >On Mon, 17 Feb 2014 00:04:19 + (GMT) > >Mark Thomas wrote: > > > >> - The subsurface has separate focus from the main window surface. For > >>the usual use cases of embedding like this, you'd prefer the parent > >>surface to remain focused (or at least, appear focused) while the embedded > >>surface is being interacted with. Not sure if this is a general feature > >>of subsurfaces, nor what could be done about it. > > > >set an empty input region, and the input events will fall through the > >surface. So you intend that the embedded client never gets input for > >the embedded surface directly? > > I think that the embedded client should still get input as normal, > however the surface that it's embedded within should still appear to > be focussed. I guess this is a shell interface question, and will > probably need a shell interface extension. I'll think about this > more later on when I come round to writing the shell plugin that > I'll undoubtedly need. > > >Did you know about the earlier attempts on this? I think you should be > >able to find some discussion or a proposal by searching the > >wayland-devel@ archives for "foreign surface" protocol, IIRC. At that > >time, the conclusion was to use a nested mini-compositor approach > >instead, which is demoed in weston clients as weston-nested. > > I did not. That's quite frustrating, I could have saved myself some > time. I went back and looked and none of the posts mentioned "embed" > or "plug/socket" so that's why I didn't find them. :( > > Do you know if any code came about from the foreign surface proposals? Hi, I did the previous proposal first called "logical surfaces" and then "foreign surfaces". I had a proof-of-concept implementation, but did not finalize nor publish it as the discussion concluded that it was not the way forward, instead favoring the nested compositor approach. Jonas > > The nested mini-compositor example doesn't build for me as I don't > have working EGL, so I never even noticed it! Reading about it the > approach seems to be more suited to nested application situations, > e.g. a web browser embedding a document viewer. > > For the panel use case it seems like the wrong approach, as the > embedded panel objects are merely fastened to the panel like badges, > rather than part of the panel itself. It seems a shame to > reimplement a compositor in the panel when we've already got a > perfectly good compositor to use. > > >I see your protocol definition lacks all documentation on how it is > >supposed to be used and implemented. A verbal description would be nice, > >giving an overview. > > I did try to give a quick overview in the email, but it was late > last night and I may not have been clear. > > I've pushed some doc updates to the protocol.xml file my git repo. But > in terms of Jonas Ådahl's proposal, my protocol works the other way round: > > A creates a main surface > A creates a "hole" on that surface and sets its position and size > A gets the uid (handle) from the server > A passes that uid to B via IPC > B creates a surface > B creates a "plug" on that surface with the uid it got from A > B receives a configure event from the server with the size of the hole > B creates a buffer of the correct size and renders its image to the > surface > > >How do you handle glitch-free resizing? Sub-surfaces handle glitch-free > >resizing by temporarily changing the sub-surface into synchronized > >mode, assuring the sub-surface has new content in the correct new size, > >and then atomically commits the whole tree of sub-surfaces with a > >commit to the root wl_surface. Do you have any synchronization > >guarantees like that? With separate processes cooperating to create a > >single window it will be even more important than with the > >existing sub-surfaces case, and you will need more IPC between the two > >clients. Using client1<->client2 IPC would be more efficient than > >client1<->server<->client2. > > I don't. Sorting out glitch-free interactive resizing is delegated > to the clients, although you can get pretty good glitchy resizing by > B repainting whenever it receives the configure event. > > My anticipated use case is applets inside panels, which aren't > typically resized, so this implementation should be sufficient. > > >Have you considered if your use case could be better served by > >moving some functionality into a special DE-specific client (e.g. > >weston-desktop-shell) and having separate protocol (an alternative > >"shell" interface) for panel clients to tell their wl_surface needs to > >be embeded into the panel, rather than implementing a generic > >mechanism where you need to solve all corner-cases in a generic way? > >If the protocol extension was designed particularly for panels, you > >might have an easy way out by defining special resize behavio
Re: Inter-client surface embedding
On Mon, 17 Feb 2014 22:59:11 + (GMT) Mark Thomas wrote: > On Mon, 17 Feb 2014, Pekka Paalanen wrote: > > > On Mon, 17 Feb 2014 00:04:19 + (GMT) > > Mark Thomas wrote: > > > >>- The subsurface has separate focus from the main window surface. For > >> the usual use cases of embedding like this, you'd prefer the parent > >> surface to remain focused (or at least, appear focused) while the embedded > >> surface is being interacted with. Not sure if this is a general feature > >> of subsurfaces, nor what could be done about it. > > > > set an empty input region, and the input events will fall through the > > surface. So you intend that the embedded client never gets input for > > the embedded surface directly? > > I think that the embedded client should still get input as normal, however > the surface that it's embedded within should still appear to be focussed. > I guess this is a shell interface question, and will probably need a shell > interface extension. I'll think about this more later on when I come > round to writing the shell plugin that I'll undoubtedly need. > > > Did you know about the earlier attempts on this? I think you should be > > able to find some discussion or a proposal by searching the > > wayland-devel@ archives for "foreign surface" protocol, IIRC. At that > > time, the conclusion was to use a nested mini-compositor approach > > instead, which is demoed in weston clients as weston-nested. > > I did not. That's quite frustrating, I could have saved myself some time. > I went back and looked and none of the posts mentioned "embed" or > "plug/socket" so that's why I didn't find them. :( > > Do you know if any code came about from the foreign surface proposals? I don't recall if it did. I assume you found some email discussions, you could ask directly from the people proposing them. > The nested mini-compositor example doesn't build for me as I don't have > working EGL, so I never even noticed it! Reading about it the approach > seems to be more suited to nested application situations, e.g. a web > browser embedding a document viewer. > > For the panel use case it seems like the wrong approach, as the embedded > panel objects are merely fastened to the panel like badges, rather than > part of the panel itself. It seems a shame to reimplement a compositor in > the panel when we've already got a perfectly good compositor to use. > > > I see your protocol definition lacks all documentation on how it is > > supposed to be used and implemented. A verbal description would be nice, > > giving an overview. > > I did try to give a quick overview in the email, but it was late last > night and I may not have been clear. > > I've pushed some doc updates to the protocol.xml file my git repo. But > in terms of Jonas Ådahl's proposal, my protocol works the other way round: > >A creates a main surface >A creates a "hole" on that surface and sets its position and size >A gets the uid (handle) from the server >A passes that uid to B via IPC >B creates a surface >B creates a "plug" on that surface with the uid it got from A >B receives a configure event from the server with the size of the hole >B creates a buffer of the correct size and renders its image to the > surface > > > How do you handle glitch-free resizing? Sub-surfaces handle glitch-free > > resizing by temporarily changing the sub-surface into synchronized > > mode, assuring the sub-surface has new content in the correct new size, > > and then atomically commits the whole tree of sub-surfaces with a > > commit to the root wl_surface. Do you have any synchronization > > guarantees like that? With separate processes cooperating to create a > > single window it will be even more important than with the > > existing sub-surfaces case, and you will need more IPC between the two > > clients. Using client1<->client2 IPC would be more efficient than > > client1<->server<->client2. > > I don't. Sorting out glitch-free interactive resizing is delegated to the > clients, although you can get pretty good glitchy resizing by B repainting > whenever it receives the configure event. Except if you do not define a way to push *all* new state accross the multiple clients to the server in an atomic way, nothing the clients can do will guarantee glitch-free operation. You cannot rely on e.g. two wl_surface.commits happening back-to-back, there is always a possibility that the server will repaint between them, which means the server uses only half of the new state, leading to a visual glitch. It is very easy to design a protocol that works glitch-free most of the time, and it may be hard to even purposefully trigger the potential glitches, but the race can still be there. > My anticipated use case is applets inside panels, which aren't typically > resized, so this implementation should be sufficient. Right, but if you build a generic protocol for embedding, you really should
Re: Inter-client surface embedding
Mark Thomas wrote: I've pushed some doc updates to the protocol.xml file my git repo. But in terms of Jonas Ådahl's proposal, my protocol works the other way round: A creates a main surface A creates a "hole" on that surface and sets its position and size A gets the uid (handle) from the server A passes that uid to B via IPC B creates a surface B creates a "plug" on that surface with the uid it got from A B receives a configure event from the server with the size of the hole B creates a buffer of the correct size and renders its image to the surface I do believe users are looking for something more like this than for implementing a subcompositor. Subcompositor really worries me as it relies on the buffers being passed through the intermediate client as fast as possible (ie without copying), and information that clients use to figure out how to allocate their buffers being passed the opposite direction. I can't believe every client is going to get this right, and would seem to make it impossible for a sub-client to take advantage of any new wayland api until the parent client is updated. I think the above description can be greatly simplified by removing the "hole" and "plug" objects and just using a subsurface: A creates a main surface A creates a subsurface for the "hole" A gets the uid of the subsurface from the compositor A passes that uid to B via IPC B uses this uid to get access to the subsurface B can now attach buffers and do other actions to the subsurface I proposed this before but I think I am failing to communicate what I wanted. The term "uid" seems pretty good and maybe makes it clearer. It's purpose is so B can't just guess at objects and get access to them, and to be a simple piece of data (probably a big number) that can be passed through IPC, in particular as part of the argv sent to exec the child. As far as I can tell Wayland will have no difficulties with two clients both owning a subsurface and trying to update it, as it will serialize all the requests and perform them in whatever order they appear. Either client can destroy the surface and the other will cleanly hear about it. ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: Inter-client surface embedding
GtkPlug and GtkSocket are really implemented in terms of XEmbed. As we've found, XEmbed has a surprising number of problems in real-world use cases, so it's considered deprecated. Building something special-case for panels seems much better than trying to implement something generic like WaylandEmbed. On Mon, Feb 17, 2014 at 5:59 PM, Mark Thomas wrote: > On Mon, 17 Feb 2014, Pekka Paalanen wrote: > > On Mon, 17 Feb 2014 00:04:19 + (GMT) >> Mark Thomas wrote: >> >> - The subsurface has separate focus from the main window surface. For >>> the usual use cases of embedding like this, you'd prefer the parent >>> surface to remain focused (or at least, appear focused) while the >>> embedded >>> surface is being interacted with. Not sure if this is a general feature >>> of subsurfaces, nor what could be done about it. >>> >> >> set an empty input region, and the input events will fall through the >> surface. So you intend that the embedded client never gets input for >> the embedded surface directly? >> > > I think that the embedded client should still get input as normal, however > the surface that it's embedded within should still appear to be focussed. I > guess this is a shell interface question, and will probably need a shell > interface extension. I'll think about this more later on when I come round > to writing the shell plugin that I'll undoubtedly need. > > > Did you know about the earlier attempts on this? I think you should be >> able to find some discussion or a proposal by searching the >> wayland-devel@ archives for "foreign surface" protocol, IIRC. At that >> time, the conclusion was to use a nested mini-compositor approach >> instead, which is demoed in weston clients as weston-nested. >> > > I did not. That's quite frustrating, I could have saved myself some time. > I went back and looked and none of the posts mentioned "embed" or > "plug/socket" so that's why I didn't find them. :( > > Do you know if any code came about from the foreign surface proposals? > > The nested mini-compositor example doesn't build for me as I don't have > working EGL, so I never even noticed it! Reading about it the approach > seems to be more suited to nested application situations, e.g. a web > browser embedding a document viewer. > > For the panel use case it seems like the wrong approach, as the embedded > panel objects are merely fastened to the panel like badges, rather than > part of the panel itself. It seems a shame to reimplement a compositor in > the panel when we've already got a perfectly good compositor to use. > > > I see your protocol definition lacks all documentation on how it is >> supposed to be used and implemented. A verbal description would be nice, >> giving an overview. >> > > I did try to give a quick overview in the email, but it was late last > night and I may not have been clear. > > I've pushed some doc updates to the protocol.xml file my git repo. But > in terms of Jonas Ådahl's proposal, my protocol works the other way round: > > A creates a main surface > A creates a "hole" on that surface and sets its position and size > A gets the uid (handle) from the server > A passes that uid to B via IPC > B creates a surface > B creates a "plug" on that surface with the uid it got from A > B receives a configure event from the server with the size of the hole > B creates a buffer of the correct size and renders its image to the > surface > > > How do you handle glitch-free resizing? Sub-surfaces handle glitch-free >> resizing by temporarily changing the sub-surface into synchronized >> mode, assuring the sub-surface has new content in the correct new size, >> and then atomically commits the whole tree of sub-surfaces with a >> commit to the root wl_surface. Do you have any synchronization >> guarantees like that? With separate processes cooperating to create a >> single window it will be even more important than with the >> existing sub-surfaces case, and you will need more IPC between the two >> clients. Using client1<->client2 IPC would be more efficient than >> client1<->server<->client2. >> > > I don't. Sorting out glitch-free interactive resizing is delegated to the > clients, although you can get pretty good glitchy resizing by B repainting > whenever it receives the configure event. > > My anticipated use case is applets inside panels, which aren't typically > resized, so this implementation should be sufficient. > > > Have you considered if your use case could be better served by >> moving some functionality into a special DE-specific client (e.g. >> weston-desktop-shell) and having separate protocol (an alternative >> "shell" interface) for panel clients to tell their wl_surface needs to >> be embeded into the panel, rather than implementing a generic >> mechanism where you need to solve all corner-cases in a generic way? >> If the protocol extension was designed particularly for panels, you >> might have an easy way out by defining special
Re: Inter-client surface embedding
On Mon, 17 Feb 2014, Pekka Paalanen wrote: On Mon, 17 Feb 2014 00:04:19 + (GMT) Mark Thomas wrote: - The subsurface has separate focus from the main window surface. For the usual use cases of embedding like this, you'd prefer the parent surface to remain focused (or at least, appear focused) while the embedded surface is being interacted with. Not sure if this is a general feature of subsurfaces, nor what could be done about it. set an empty input region, and the input events will fall through the surface. So you intend that the embedded client never gets input for the embedded surface directly? I think that the embedded client should still get input as normal, however the surface that it's embedded within should still appear to be focussed. I guess this is a shell interface question, and will probably need a shell interface extension. I'll think about this more later on when I come round to writing the shell plugin that I'll undoubtedly need. Did you know about the earlier attempts on this? I think you should be able to find some discussion or a proposal by searching the wayland-devel@ archives for "foreign surface" protocol, IIRC. At that time, the conclusion was to use a nested mini-compositor approach instead, which is demoed in weston clients as weston-nested. I did not. That's quite frustrating, I could have saved myself some time. I went back and looked and none of the posts mentioned "embed" or "plug/socket" so that's why I didn't find them. :( Do you know if any code came about from the foreign surface proposals? The nested mini-compositor example doesn't build for me as I don't have working EGL, so I never even noticed it! Reading about it the approach seems to be more suited to nested application situations, e.g. a web browser embedding a document viewer. For the panel use case it seems like the wrong approach, as the embedded panel objects are merely fastened to the panel like badges, rather than part of the panel itself. It seems a shame to reimplement a compositor in the panel when we've already got a perfectly good compositor to use. I see your protocol definition lacks all documentation on how it is supposed to be used and implemented. A verbal description would be nice, giving an overview. I did try to give a quick overview in the email, but it was late last night and I may not have been clear. I've pushed some doc updates to the protocol.xml file my git repo. But in terms of Jonas Ådahl's proposal, my protocol works the other way round: A creates a main surface A creates a "hole" on that surface and sets its position and size A gets the uid (handle) from the server A passes that uid to B via IPC B creates a surface B creates a "plug" on that surface with the uid it got from A B receives a configure event from the server with the size of the hole B creates a buffer of the correct size and renders its image to the surface How do you handle glitch-free resizing? Sub-surfaces handle glitch-free resizing by temporarily changing the sub-surface into synchronized mode, assuring the sub-surface has new content in the correct new size, and then atomically commits the whole tree of sub-surfaces with a commit to the root wl_surface. Do you have any synchronization guarantees like that? With separate processes cooperating to create a single window it will be even more important than with the existing sub-surfaces case, and you will need more IPC between the two clients. Using client1<->client2 IPC would be more efficient than client1<->server<->client2. I don't. Sorting out glitch-free interactive resizing is delegated to the clients, although you can get pretty good glitchy resizing by B repainting whenever it receives the configure event. My anticipated use case is applets inside panels, which aren't typically resized, so this implementation should be sufficient. Have you considered if your use case could be better served by moving some functionality into a special DE-specific client (e.g. weston-desktop-shell) and having separate protocol (an alternative "shell" interface) for panel clients to tell their wl_surface needs to be embeded into the panel, rather than implementing a generic mechanism where you need to solve all corner-cases in a generic way? If the protocol extension was designed particularly for panels, you might have an easy way out by defining special resize behaviour which would avoid most client<->client negotiation. My plan was to patch Gtk3 to implement GtkPlug and GtkSocket in terms of this new interface and see how far that got me, so I hadn't considered that as an alternative. I could see that working. I'm going to see exactly how much mate-panel and its applets are wedded to the idea of using GtkPlug and GtkSocket. If I can do it with a mate-desktop-shell protocol extension, with minimal additional support from weston core, that would be ideal. Thanks for your feedback. -- M
Re: Inter-client surface embedding
On Mon, 17 Feb 2014 00:04:19 + (GMT) Mark Thomas wrote: > > As part of my current attempts to get MATE fully working on Wayland, I was > planning to take a look at porting mate-panel this weekend. However, I > pretty quickly hit the first snag, which is that Gtk on Wayland doesn't > support the GtkSocket/GtkPlug interface, which mate-panel relies heavily > upon. > > This is because Wayland doesn't understand the idea of linking different > clients' surfaces together, so I've had a first pass at implementing an > interface that lets you do that. > > My changes are available in github at > > https://github.com/markbt/weston/tree/embed > > I'd be interested to hear what people think. > > I've got it working with a couple of toy clients, so I think the theory is > sound, but there are a few niggles to work through. > > What works: > >- Clients can create one or more "hole" objects on a surface. This > gives them a "uid" (32-bit number) which they can pass to other clients of > the same wayland server (e.g. via a socket, D-Bus, etc.). For the toy > apps you do this manually. > >- Other clients with the uid in hand can create a "plug" that attaches > to that hole. This effectively creates a subsurface of the other client's > surface. (In fact, internally a plug is a weston_subsurface with a couple > of extra fields in use). > >- The hole can have a position and size set. The size is notified > to the "plug" client who is expected to conform to it. Currently there's > no policing of this, but I expect a more robust implementation would want > to. > >- Either side can tear down the connection between the two. If the > hole goes away, or the hole's client calls "remove" on the hole, then the > plug client receives a "removed" event. > > > What needs some work: > >- The subsurface has to be unsynchronized, otherwise the plug client's > updates don't render unless the hole client is also updating. Not sure > how this will work with nested subsurfaces if synchronization is > inherited. > >- The subsurface has separate focus from the main window surface. For > the usual use cases of embedding like this, you'd prefer the parent > surface to remain focused (or at least, appear focused) while the embedded > surface is being interacted with. Not sure if this is a general feature > of subsurfaces, nor what could be done about it. Hi, set an empty input region, and the input events will fall through the surface. So you intend that the embedded client never gets input for the embedded surface directly? >- The toy examples are quite rapidly thrown together and so aren't > particularly good. Improvements gladly welcomed. > >- The interfaces are incomplete (and undocumented, sorry). > >- There's probably a bunch of other things I've not thought about. > Did you know about the earlier attempts on this? I think you should be able to find some discussion or a proposal by searching the wayland-devel@ archives for "foreign surface" protocol, IIRC. At that time, the conclusion was to use a nested mini-compositor approach instead, which is demoed in weston clients as weston-nested. I see your protocol definition lacks all documentation on how it is supposed to be used and implemented. A verbal description would be nice, giving an overview. How do you handle glitch-free resizing? Sub-surfaces handle glitch-free resizing by temporarily changing the sub-surface into synchronized mode, assuring the sub-surface has new content in the correct new size, and then atomically commits the whole tree of sub-surfaces with a commit to the root wl_surface. Do you have any synchronization guarantees like that? With separate processes cooperating to create a single window it will be even more important than with the existing sub-surfaces case, and you will need more IPC between the two clients. Using client1<->client2 IPC would be more efficient than client1<->server<->client2. Have you considered if your use case could be better served by moving some functionality into a special DE-specific client (e.g. weston-desktop-shell) and having separate protocol (an alternative "shell" interface) for panel clients to tell their wl_surface needs to be embeded into the panel, rather than implementing a generic mechanism where you need to solve all corner-cases in a generic way? If the protocol extension was designed particularly for panels, you might have an easy way out by defining special resize behaviour which would avoid most client<->client negotiation. You need something special to place the panel itself on the desktop anyway, so I think it's ok to have the compositor specifically help the special DE-specific client to do the embedding properly. Thanks, pq ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Inter-client surface embedding
As part of my current attempts to get MATE fully working on Wayland, I was planning to take a look at porting mate-panel this weekend. However, I pretty quickly hit the first snag, which is that Gtk on Wayland doesn't support the GtkSocket/GtkPlug interface, which mate-panel relies heavily upon. This is because Wayland doesn't understand the idea of linking different clients' surfaces together, so I've had a first pass at implementing an interface that lets you do that. My changes are available in github at https://github.com/markbt/weston/tree/embed I'd be interested to hear what people think. I've got it working with a couple of toy clients, so I think the theory is sound, but there are a few niggles to work through. What works: - Clients can create one or more "hole" objects on a surface. This gives them a "uid" (32-bit number) which they can pass to other clients of the same wayland server (e.g. via a socket, D-Bus, etc.). For the toy apps you do this manually. - Other clients with the uid in hand can create a "plug" that attaches to that hole. This effectively creates a subsurface of the other client's surface. (In fact, internally a plug is a weston_subsurface with a couple of extra fields in use). - The hole can have a position and size set. The size is notified to the "plug" client who is expected to conform to it. Currently there's no policing of this, but I expect a more robust implementation would want to. - Either side can tear down the connection between the two. If the hole goes away, or the hole's client calls "remove" on the hole, then the plug client receives a "removed" event. What needs some work: - The subsurface has to be unsynchronized, otherwise the plug client's updates don't render unless the hole client is also updating. Not sure how this will work with nested subsurfaces if synchronization is inherited. - The subsurface has separate focus from the main window surface. For the usual use cases of embedding like this, you'd prefer the parent surface to remain focused (or at least, appear focused) while the embedded surface is being interacted with. Not sure if this is a general feature of subsurfaces, nor what could be done about it. - The toy examples are quite rapidly thrown together and so aren't particularly good. Improvements gladly welcomed. - The interfaces are incomplete (and undocumented, sorry). - There's probably a bunch of other things I've not thought about. -- Mark. ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
