Re: Compositor grabs (was: Re: [PATCH] protocol: Add DnD actions)
sorry, quite late to the discussion here On Fri, Apr 17, 2015 at 04:27:19PM +0200, Michal Suchanek wrote: > On 17 April 2015 at 16:15, Carlos Garnacho wrote: > > Hey Michal, > > > > On Fri, Apr 17, 2015 at 2:47 PM, Michal Suchanek wrote: > > > > > >>> In the X11 world, this would roughly be a Virtual Core > >>> Pointer+Keyboard grab (not that touch and active grabs are trouble > >>> free in X11, but...), GTK+ for example does grab both devices on every > >>> of those grabbing places wayland/xdg protocols are trying to cater for > >>> (I've even pondered about adding a gdk_device_grab_pair() for years). > >>> > >>> I think some consistent model should be devised here, and embedded > >>> into the protocol (docs). > >>> > >> > >> The serious problem with X11 grabs is that they are completely > >> independent of the event that triggered them and can only be released > >> by the application that started them. > >> > >> So it happens that an application that gets stuck due to code error or > >> is running in a debugger at the time a grab is active never releases > >> the grab which > >> > >> 1) prevents other applications from receiving input > >> 2) prevents further grabs > >> > >> It might be worth considering if there is generic enough drag semantic > >> that the grab could be handled in compositor outside of application > >> code, even with click-lock and whatnot. > > > > Nothing prevents wayland compositors today from undoing grabs when > > clients get destroyed, or surfaces don't respond to pings. Affecting > > this proposal, the pointer should re-enter the surface underneath and > > keyboard focus restablished after the grab is "broken". > > Nothing prevents the X server either. It hopefully breaks the grabs > initiated by clients when they are destroyed, too. However, there is > no mechanism for breaking a grab of a client other than destroying it > *and* destroying the client cannot be done from within the X session > because it is grabbed. that's not quite true. the protocol specifies the behaviour of a grab in detail and there is no server-initiated escape from an activated grab, active or passive. you could cancel the grab de-facto (i.e. make the device interact with other client again) but you'd then have to maintain a fake grab state for the grabbing client until it releases. and since you can't know what the trigger for the release is (movement? button?), you're pretty much stuck. forcibly killing the grab is possible, but then you have to assume the client will crash in one way or another. Cheers, Peter ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: Compositor grabs (was: Re: [PATCH] protocol: Add DnD actions)
On Tue, Jun 09, 2015 at 07:26:15PM +0200, Carlos Garnacho wrote: > Hey Jonas :), > > On jue, 2015-06-04 at 12:34 +0800, Jonas Ådahl wrote: > > On Fri, Apr 17, 2015 at 02:16:41PM +0200, Carlos Garnacho wrote: > > > > > > Let's expand on that example, maybe far-streched, but certainly > > > possible: > > > - I'm manipulating a client window with 2 fingers on the > > > touchscreen > > > (say zooming an image) > > > - Any other interaction on the client makes it pop up an xdg_popup > > > (say a third touch, a key combo, or the pointer) > > > - Q: what happens with the two first touches? > > > > > > Ideally, these touches should be "cancelled" on the first surface > > > (wl_touch.cancelled seems to be per client, not per-surface > > > though), > > > and stay non-reactive within the grab, so they wouldn't trigger > > > anything unintended (they're implicitly grabbed on another surface > > > after all) > > > > It seems rather unspecified in the protocol, but AFAIK at least in > > weston only one surface may have any touch points active at any given > > time. Mutter seems to do the opposite, i.e. any number of surfaces > > can > > have "touch focus". > > TBH I did this on purpose on mutter, again the protocol is quite open > in this regard, so I made each touch go where its implicit grab > defines. This doesn't go against anything in the protocol (eg. touch > "focus" doesn't imply keyboard focus), and it seemed to me like > artificially capping what we're able to. Considering that with the current version it is impossible to cancel touches on an individual surface, i.e. we can't implement the semantics discussed, should we maybe only support single surface touch points and then add support for it when we can cancel touches on a per surface basis? > > > I'm don't know what is intentional here, but > > assuming single touch focus is, wl_touch.cancelled being per client > > is > > not a problem when an explicit grab is initiated on another surface, > > since we'd have to cancel the touches of the previously focused > > surface > > (which is all on that client), and then start them on the new one > > > You've got a point here, although the questions still hold for > pointer/keyboard. > > > (hmm, > > should the popup get new 'down' events? I assume it must, since one > > should not have to raise the finger and then touch again to interact > > with the popup). > > Hmm, I'm not sure we should, it IMO introduces inconsistency for this > one usecase. I would much prefer to encourage wl_subsurface if such > thing is a must, this is grabless on the compositor, so the client can > define its own semantics. Yea, mabe it makes more sense to not support it and not provide xdg_popup like semantics for that use case. > > I eg. wonder what would happen with coordinates laid outside the > surface, do we get "down" events anyway? Or, say a popup containing a > scrolled view is shown so its triggering touch lies in the middle, do > we want it to scroll kinetic scrolling, select an item or whatnot? > > Pointers aren't that much of an issue because press/released is > separated semantically from enter/leave, on wl_touch these are put > together into down/up. IMO, fixing this requires either separating > these likewise here, or refining wl_touch.cancelled somehow. The difference is that, for a popup that was clicked but not released, the new surface will still receive motion events, and can by itself keep track of whether the button was clicked or not. For touches, if we don't receive a 'down', and the previous touches are cancelled, there won't be any new motion events. The biggest difference would be that for a xdg_popup like menu, with a pointer we could click-hold-drag-select-release, but for touch, for the same menu, we would require touch-release-touch-release for the same action. > > > > > Or, the other way, where there is no such thing as single touch > > focus. > > I.e. that multiple surfaces can have touch points at the same time, > > then > > we'd need changing the protocol to support what you describe (as as > > you > > say, wl_touch.cancelled is per client). > > > > Anyway, either we need to properly clarify that in wl_touch I think, > > and > > it'd be helpful to know if this was considered when designing the > > protocol. > > Would be helpful indeed :). Given how wl_touch.cancelled works, I would at least assume its single-surface, but that, as I said, just my assumption. > > > > > > > > > Currently, on the weston code, focusing a bit on the all-touch > > > case, > > > it actually happens the worst that could happen, the xdg_popup > > > touch > > > grab redirects already started touch sequences to the grabbing > > > surface > > > right away, and the original surface will be deaf to them of a > > > sudden, > > > leading to inconsistent state on both the original and the grabbing > > > surface wrt those touch sequences. The DnD touch grab doesn't fare > > > much better, it will
Re: Compositor grabs (was: Re: [PATCH] protocol: Add DnD actions)
Hey Jonas :), On jue, 2015-06-04 at 12:34 +0800, Jonas Ådahl wrote: > On Fri, Apr 17, 2015 at 02:16:41PM +0200, Carlos Garnacho wrote: > > Hey Jonas, > > > > This is drifting a bit off the topic of the original thread, better > > to > > spin this off. I'll reply to the DnD bits in another email. > > Hey, sorry for the delay. Some comments/questions below. > > > > > On Fri, Apr 17, 2015 at 9:50 AM, Jonas Ådahl > > wrote: > > > On Thu, Apr 16, 2015 at 12:55:31PM +0200, Carlos Garnacho wrote: > > > > Hey Jonas, > > > > > > > > On Thu, Apr 16, 2015 at 10:15 AM, Jonas Ådahl > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > I'd have to agree on that it doesn't seem like the best thing > > > > > to let the > > > > > compositor choose the preferred action. Having it apply > > > > > compositor > > > > > specific policy given what the keyboard state or similar will > > > > > probably > > > > > never work out very well, given that for example what > > > > > modifier state > > > > > means what type of action is very application dependent. > > > > > > > > > > On the other hand, I'm not sure we can currently rely on > > > > > either side > > > > > having keyboard focus during the drag. In weston the source > > > > > will have the > > > > > focus because starting the drag was done with a click which > > > > > gave the > > > > > surface keyboard focus implicitly, but what'd happen if the > > > > > compositor > > > > > has keyboard-focus-follows-mouse? We could probably say that > > > > > drag implies > > > > > an implicit grab on another device on the same seat to > > > > > enforce no > > > > > changing of keyboard focus, but not sure that is better. > > > > > > > > In gtk+/gnome we currently have the following keybindings > > > > active during DnD: > > > > > > > > - Cursor keys move the drag point, modifiers affect speed > > > > - Esc key cancels drag > > > > - Modifiers alone pick an action from the offered list > > > > > > > > So ok, the latter is dubious to punt to compositors, but > > > > there's > > > > basically no other choice with the 2 first ones. > > > > > > > > More generally, I have the opinion that compositors grabs > > > > should > > > > behave all consistently, as in: > > > > > > > > - Ensuring clients reset all input state (we eg. don't cancel > > > > ongoing > > > > touches when xdg_popup/dnd/... grabs kick in) > > > > > > What does "client reset all input state" mean? What state can a > > > client > > > reset? > > > > Let's expand on that example, maybe far-streched, but certainly > > possible: > > - I'm manipulating a client window with 2 fingers on the > > touchscreen > > (say zooming an image) > > - Any other interaction on the client makes it pop up an xdg_popup > > (say a third touch, a key combo, or the pointer) > > - Q: what happens with the two first touches? > > > > Ideally, these touches should be "cancelled" on the first surface > > (wl_touch.cancelled seems to be per client, not per-surface > > though), > > and stay non-reactive within the grab, so they wouldn't trigger > > anything unintended (they're implicitly grabbed on another surface > > after all) > > It seems rather unspecified in the protocol, but AFAIK at least in > weston only one surface may have any touch points active at any given > time. Mutter seems to do the opposite, i.e. any number of surfaces > can > have "touch focus". TBH I did this on purpose on mutter, again the protocol is quite open in this regard, so I made each touch go where its implicit grab defines. This doesn't go against anything in the protocol (eg. touch "focus" doesn't imply keyboard focus), and it seemed to me like artificially capping what we're able to. > I'm don't know what is intentional here, but > assuming single touch focus is, wl_touch.cancelled being per client > is > not a problem when an explicit grab is initiated on another surface, > since we'd have to cancel the touches of the previously focused > surface > (which is all on that client), and then start them on the new one You've got a point here, although the questions still hold for pointer/keyboard. > (hmm, > should the popup get new 'down' events? I assume it must, since one > should not have to raise the finger and then touch again to interact > with the popup). Hmm, I'm not sure we should, it IMO introduces inconsistency for this one usecase. I would much prefer to encourage wl_subsurface if such thing is a must, this is grabless on the compositor, so the client can define its own semantics. I eg. wonder what would happen with coordinates laid outside the surface, do we get "down" events anyway? Or, say a popup containing a scrolled view is shown so its triggering touch lies in the middle, do we want it to scroll kinetic scrolling, select an item or whatnot? Pointers aren't that much of an issue because press/released is separated semantically from enter/leave, on wl_touch these are put together into down/up. IMO, fixing t
Re: Compositor grabs (was: Re: [PATCH] protocol: Add DnD actions)
On Fri, Apr 17, 2015 at 02:16:41PM +0200, Carlos Garnacho wrote: > Hey Jonas, > > This is drifting a bit off the topic of the original thread, better to > spin this off. I'll reply to the DnD bits in another email. Hey, sorry for the delay. Some comments/questions below. > > On Fri, Apr 17, 2015 at 9:50 AM, Jonas Ådahl wrote: > > On Thu, Apr 16, 2015 at 12:55:31PM +0200, Carlos Garnacho wrote: > >> Hey Jonas, > >> > >> On Thu, Apr 16, 2015 at 10:15 AM, Jonas Ådahl wrote: > >> > >> > >> > >> > > >> > I'd have to agree on that it doesn't seem like the best thing to let the > >> > compositor choose the preferred action. Having it apply compositor > >> > specific policy given what the keyboard state or similar will probably > >> > never work out very well, given that for example what modifier state > >> > means what type of action is very application dependent. > >> > > >> > On the other hand, I'm not sure we can currently rely on either side > >> > having keyboard focus during the drag. In weston the source will have the > >> > focus because starting the drag was done with a click which gave the > >> > surface keyboard focus implicitly, but what'd happen if the compositor > >> > has keyboard-focus-follows-mouse? We could probably say that drag implies > >> > an implicit grab on another device on the same seat to enforce no > >> > changing of keyboard focus, but not sure that is better. > >> > >> In gtk+/gnome we currently have the following keybindings active during > >> DnD: > >> > >> - Cursor keys move the drag point, modifiers affect speed > >> - Esc key cancels drag > >> - Modifiers alone pick an action from the offered list > >> > >> So ok, the latter is dubious to punt to compositors, but there's > >> basically no other choice with the 2 first ones. > >> > >> More generally, I have the opinion that compositors grabs should > >> behave all consistently, as in: > >> > >> - Ensuring clients reset all input state (we eg. don't cancel ongoing > >> touches when xdg_popup/dnd/... grabs kick in) > > > > What does "client reset all input state" mean? What state can a client > > reset? > > Let's expand on that example, maybe far-streched, but certainly possible: > - I'm manipulating a client window with 2 fingers on the touchscreen > (say zooming an image) > - Any other interaction on the client makes it pop up an xdg_popup > (say a third touch, a key combo, or the pointer) > - Q: what happens with the two first touches? > > Ideally, these touches should be "cancelled" on the first surface > (wl_touch.cancelled seems to be per client, not per-surface though), > and stay non-reactive within the grab, so they wouldn't trigger > anything unintended (they're implicitly grabbed on another surface > after all) It seems rather unspecified in the protocol, but AFAIK at least in weston only one surface may have any touch points active at any given time. Mutter seems to do the opposite, i.e. any number of surfaces can have "touch focus". I'm don't know what is intentional here, but assuming single touch focus is, wl_touch.cancelled being per client is not a problem when an explicit grab is initiated on another surface, since we'd have to cancel the touches of the previously focused surface (which is all on that client), and then start them on the new one (hmm, should the popup get new 'down' events? I assume it must, since one should not have to raise the finger and then touch again to interact with the popup). Or, the other way, where there is no such thing as single touch focus. I.e. that multiple surfaces can have touch points at the same time, then we'd need changing the protocol to support what you describe (as as you say, wl_touch.cancelled is per client). Anyway, either we need to properly clarify that in wl_touch I think, and it'd be helpful to know if this was considered when designing the protocol. > > Currently, on the weston code, focusing a bit on the all-touch case, > it actually happens the worst that could happen, the xdg_popup touch > grab redirects already started touch sequences to the grabbing surface > right away, and the original surface will be deaf to them of a sudden, > leading to inconsistent state on both the original and the grabbing > surface wrt those touch sequences. The DnD touch grab doesn't fare > much better, it will ignore other touches than the one starting the > drag, so the pre-grab touches would effectively go nowhere, and AFAICS > similar issues arise with pointer grabs vs touch. Yes this seems wrong. > > With keyboards, it happens likewise, if compositors are to possibly > consume events there, focus should move out of the previous surface. > IMO, any grabbing model that does not redirect all input, nor ensures > a coherent state in clients is calling for trouble... > > In the X11 world, this would roughly be a Virtual Core > Pointer+Keyboard grab (not that touch and active grabs are trouble > free in X11, but...), GTK+ for example does grab both devices on every
Re: [PATCH] protocol: Add DnD actions
Hey Jonas, Late reply, let's try to pick this up again... On mar, 2015-04-21 at 14:29 +0800, Jonas Ådahl wrote: > On Sat, Apr 18, 2015 at 04:53:46PM +0200, Carlos Garnacho wrote: > > Hey Jonas, > > Hi, > > Thanks for the explanations. I'll reply inline. > > > > > On vie, 2015-04-17 at 15:50 +0800, Jonas Ådahl wrote: > > > > > > > > > > > For the touch case, depending on how the grab is implemented, > > > > with > > > > the > > > > current guidelines the only 2 choices are "leave the client in > > > > inconsistent state" or "make the client still receives ongoing > > > > touches > > > > despite the pointer grab" (same applies if the grab is touch > > > > triggered, only with the other touches that didn't trigger the > > > > grab). > > > > > > > > More on topic, keyboards are also funky if we keep focus on > > > > clients, > > > > you can conceivably Esc/Ctrl-Q/... to close the app you're > > > > dragging > > > > from. IMO the way forward is precisely this, the compositor > > > > becomes in > > > > control of the keyboard, and we offer the missing semantics to > > > > cover > > > > for this. > > > > > > Meaning its the compositor that decides whether a drag is a copy > > > or a > > > move? I.e. either we hard code "Ctrl" to be copy in the > > > protocol, or > > > DND > > > will behave different on each compositor. Not sure I like any of > > > those > > > options. > > > > Yes, this would be implementation-dependent in the compositor as > > my > > proposal goes. We have 3 players here, whoever gets to handle the > > modifier->action translation, there's room for confusion in cross- > > DE/toolkit cases. > > The problem as I see it is that it'd be even confusing for DND within > the same application as it would be depending on the DE how it'd > work. > > > > Focusing on actions, I see the following possible data flows here > > (depicting the same situation on all: initial negotiation, changes > > on > > the dest on say pointer motion, and a modifier change): > > > > 1. If handled purely by the source: > > > > wl_data_source compositorwl_data_offer > > == === > > -> notify_actions <- > > dest_actions <- > > -> preferred_action > > action <--> action > > > > ... > > (pointer moves across > > widgets) > > notify_actions <- > > dest_actions <- > > -> preferred_action > > action <--> action > > > > ... > > (modifiers change) > > modifiers<- > > -> preferred_action > > action <--> action > > > > > > 2. If handled purely by the dest: > > > > wl_data_source compositorwl_data_offer > > == === > > -> notify_actions > >-> source_actions > > notify_actions <- > > preferred_action <- > > action <--> action > > > > ... > > (pointer moves across > > widgets) > > notify_actions <- > > preferred_action <- > > action <--> action > > > > ... > > (modifiers change) > >-> modifiers > > preferred_action <- > > action <--> action > > > > > > 3. If handled purely by the compositor: > > > > wl_data_source compositorwl_data_offer > > == === > > -> notify_actions <- > > action <--> action > > > > ... > > (pointer moves across > > widgets) > > notify_actions <- > > action <--> action > > > > ... > > (modifiers change) > > action <--> action > > > > > > Options #1 and #2 involve roundtrips, option #3 doesn't. Options > > #1 > > and #2 would still need some validation on the compositor to avoid > > picking options unknown to either side. > > I think its wrong to refer to these as roundtrips. A roundtrip is > typically a client that need to wait for a reply from a server, but > here > in any of the three options no one is waiting for anything, thus we > have > no round trips at all. Here I was referring to the places where the compositor sends an event to either side, and awaits an immediate reply for it. It is true that those are non block
Re: [PATCH] protocol: Add DnD actions
On Sat, Apr 18, 2015 at 04:53:46PM +0200, Carlos Garnacho wrote: > Hey Jonas, Hi, Thanks for the explanations. I'll reply inline. > > On vie, 2015-04-17 at 15:50 +0800, Jonas Ådahl wrote: > > > > > > > For the touch case, depending on how the grab is implemented, with > > > the > > > current guidelines the only 2 choices are "leave the client in > > > inconsistent state" or "make the client still receives ongoing > > > touches > > > despite the pointer grab" (same applies if the grab is touch > > > triggered, only with the other touches that didn't trigger the > > > grab). > > > > > > More on topic, keyboards are also funky if we keep focus on > > > clients, > > > you can conceivably Esc/Ctrl-Q/... to close the app you're dragging > > > from. IMO the way forward is precisely this, the compositor > > > becomes in > > > control of the keyboard, and we offer the missing semantics to > > > cover > > > for this. > > > > Meaning its the compositor that decides whether a drag is a copy or a > > move? I.e. either we hard code "Ctrl" to be copy in the protocol, or > > DND > > will behave different on each compositor. Not sure I like any of > > those > > options. > > Yes, this would be implementation-dependent in the compositor as my > proposal goes. We have 3 players here, whoever gets to handle the > modifier->action translation, there's room for confusion in cross- > DE/toolkit cases. The problem as I see it is that it'd be even confusing for DND within the same application as it would be depending on the DE how it'd work. > > Focusing on actions, I see the following possible data flows here > (depicting the same situation on all: initial negotiation, changes on > the dest on say pointer motion, and a modifier change): > > 1. If handled purely by the source: > > wl_data_source compositorwl_data_offer > == === > -> notify_actions <- > dest_actions <- > -> preferred_action > action <--> action > > ... > (pointer moves across widgets) > notify_actions <- > dest_actions <- > -> preferred_action > action <--> action > > ... > (modifiers change) > modifiers<- > -> preferred_action > action <--> action > > > 2. If handled purely by the dest: > > wl_data_source compositorwl_data_offer > == === > -> notify_actions >-> source_actions > notify_actions <- > preferred_action <- > action <--> action > > ... > (pointer moves across widgets) > notify_actions <- > preferred_action <- > action <--> action > > ... > (modifiers change) >-> modifiers > preferred_action <- > action <--> action > > > 3. If handled purely by the compositor: > > wl_data_source compositorwl_data_offer > == === > -> notify_actions <- > action <--> action > > ... > (pointer moves across widgets) > notify_actions <- > action <--> action > > ... > (modifiers change) > action <--> action > > > Options #1 and #2 involve roundtrips, option #3 doesn't. Options #1 > and #2 would still need some validation on the compositor to avoid > picking options unknown to either side. I think its wrong to refer to these as roundtrips. A roundtrip is typically a client that need to wait for a reply from a server, but here in any of the three options no one is waiting for anything, thus we have no round trips at all. The main differences as I see it are: In option 1 and 2 we pass an additional modifier state, and make either side be responsible for choosing. In option 3 we move this and make it compositor choose (either with hard coded policy in the protocol or some arbitrary policy given some private state inside the compositor). In option 1 and 2, we have a slightly longer delay in visual feedback regarding the action (caused by the outsourcing of the decision making). I'd say these delays are in most cases insignificantly small. In any solution we end up with latency as we are dealing w
Re: Compositor grabs (was: Re: [PATCH] protocol: Add DnD actions)
On 04/20/2015 04:05 AM, Carlos Garnacho wrote: - The last touch drags the window, the previous ones remain static and are eventually lifted. Why do they apply where your finger no longer is? Why do they apply at all? can one of these touches create an xdg_popup (and another grab on another touch) while window dragging is going on? It should send touch-moved events to the first client. So if they hold down their fingers on the painting program and then use the mouse to move the window, they will cause paintstrokes to be made. Note as Wayland is currently designed the client is in control of whether any event moves it's window, so it will know if this is happening, and is free to either cancel the touch handling or not drag the window, if it does not like this. - You're about to start some critical/expensive operation and the screensaver kicks in before you lift the finger, because of low battery. Should you be allowed to blindly get the operation started while the system is preparing to halt/suspend? Your actual question applies just as well to an expensive operation that was launched before the screensaver started, but is still running. Maybe there is a "battery is low and we are stopping" event that the client can use to stop or prevent the expensive operation. I think this event could be sent whether or not a grab is happening. The screensaver should be blocked by server-started grabs (ie due to buttons being held down). Client-started grabs probably will survive and continue after the screensaver is unlocked. I see this model falling apart really soon. I suggest we first identify if there's usecases we're actually fixing here before we go original about grabs/capturing as the user interaction paradigm. Grabs are basically a contract, in which you get a guarantee that: a) You get all input from that device b) You are the only receiver of such input Touch events are highly transient (eg. there's not even guarantee that the touch starting the grab is the one that finishes it), as such it doesn't make sense to grab on sequences individually, but on the whole capability. This "we grab all future touches, we don't grab the past ones though, except the triggering one" policy seems highly inconsistent to me. I certainly intended that all touches on the same device go to the same grab. Since the triggering one will be sent to the same client as the past ones which already caused a grab, your concern does not happen. The problem is if there are multiple seats. It is also possible that there are multiple grabs per seat, perhaps divided up by device, or perhaps sets of devices or devices split between grabs. I am unsure if this is necessary, I feel like there could be exactly one grab per seat and everything, including keystrokes, goes to the grab client. ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: [PATCH] protocol: Add DnD actions
On 04/18/2015 07:53 AM, Carlos Garnacho wrote: Let me try to correct this by removing some unnecessary data. In particular no list of actions is sent by the destination. Also there is no need for the compositor to echo the selected action back to the destination. More importantly I have added "update the user interface" requests from both source an destination, to make round-trips more clear (this includes changing the cursor and changing graphics in the surfaces): 2. If handled purely by the dest: wl_data_source compositorwl_data_offer == === -> notify_actions -> source_actions action <- update_ui <- action <- -> update_ui ... (pointer moves across widgets) action <- update_ui <- action <- -> update_ui ... (modifiers change) -> modifiers action <- update_ui <- action <- -> update_ui For reference here is your proposal. You seem to have added sending the source actions to the destination and allowing that to edit the dest action list, over what you originally proposed. I am not sure if this is a copy+paste error or you actually mean it to be there, I marked it with "new??" wl_data_source compositorwl_data_offer == === -> notify_actions -> source_actions ### new?? notify_actions <- action <- -> action update_ui <- -> update_ui ... (pointer moves across widgets) notify_actions <- action <- -> action update_ui <- -> update_ui ... (modifiers change) action <--> action update_ui <- -> update_ui Options #1 and #2 involve roundtrips, option #3 doesn't. Sorry I don't see this. I was going to say that they are exactly the same, but I now see that #2 has one less round trip on the destination side for the pointer-moved case (it can update the display simultaneously with sending the selected action, while yours requires it to wait for a response after sending the notify actions). So it wins there. Though your version has the same number of round trips for modifier-changed, they can be done simultaneously, while #2 requires them to be done serially. So your version wins there. I believe the mouse is moved more often than the modifiers are changed, thus in total proposal #2 has fewer round trips. Options #1 and #2 would still need some validation on the compositor to avoid picking options unknown to either side. Option 3 has to "validate" in that it has to do something if there is no intersection of the lists. And I think you could consider the whole "intersect the lists and choose based on modifier keys" a very large "validation" function. Not only that, I am pretty certain it would be harmless to send whatever action the dest chose to the source without any checking at all, thus the "validation" concern is moot. The source can check it pretty easily. You are not saving anything as sources will have many other failure modes (such as async deletion of the source data while the drag is happening) and this is a trivial addition. ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: Compositor grabs (was: Re: [PATCH] protocol: Add DnD actions)
On vie, 2015-04-17 at 14:40 -0700, Bill Spitzak wrote: > On 04/17/2015 05:16 AM, Carlos Garnacho wrote: > > > Let's expand on that example, maybe far-streched, but certainly > > possible: > > - I'm manipulating a client window with 2 fingers on the > > touchscreen > > (say zooming an image) > > - Any other interaction on the client makes it pop up an xdg_popup > > (say a third touch, a key combo, or the pointer) > > - Q: what happens with the two first touches? > > The touches held down should continue to go to the surface they were > going to before, while the events related to the event that > triggered > the grab will go to the grab client. I very much disagree. Let's see the following scenarios: - The last touch drags the window, the previous ones remain static and are eventually lifted. Why do they apply where your finger no longer is? Why do they apply at all? can one of these touches create an xdg_popup (and another grab on another touch) while window dragging is going on? - You're about to start some critical/expensive operation and the screensaver kicks in before you lift the finger, because of low battery. Should you be allowed to blindly get the operation started while the system is preparing to halt/suspend? I see this model falling apart really soon. I suggest we first identify if there's usecases we're actually fixing here before we go original about grabs/capturing as the user interaction paradigm. Grabs are basically a contract, in which you get a guarantee that: a) You get all input from that device b) You are the only receiver of such input Touch events are highly transient (eg. there's not even guarantee that the touch starting the grab is the one that finishes it), as such it doesn't make sense to grab on sequences individually, but on the whole capability. This "we grab all future touches, we don't grab the past ones though, except the triggering one" policy seems highly inconsistent to me. > > When the two touches are fully released then the next press of them > will > go to the grab client. > > Wayland could guarantee that the release and drag events go to the > same > client that got the press event. Grabs just dictate where new press > events go. > > I don't think this situation will happen much, due to server-induced > grabs which are fully synchronous, so you cannot press two buttons > in > two different widgets and get two grabs. Instead one of them will > get > the grab and that one will see the other button. There's legit places where you want to overwrite/stack a grab within the compositor, say when the screensaver kicks in while a xdg_popup is present. The same concerns with your model arise in this case of grabbed devices being transferred from one grab to another, is the previous grab held partly in control? Cheers, Carlos ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: [PATCH] protocol: Add DnD actions
Hey Jonas, On vie, 2015-04-17 at 15:50 +0800, Jonas Ådahl wrote: > > > For the touch case, depending on how the grab is implemented, with > > the > > current guidelines the only 2 choices are "leave the client in > > inconsistent state" or "make the client still receives ongoing > > touches > > despite the pointer grab" (same applies if the grab is touch > > triggered, only with the other touches that didn't trigger the > > grab). > > > > More on topic, keyboards are also funky if we keep focus on > > clients, > > you can conceivably Esc/Ctrl-Q/... to close the app you're dragging > > from. IMO the way forward is precisely this, the compositor > > becomes in > > control of the keyboard, and we offer the missing semantics to > > cover > > for this. > > Meaning its the compositor that decides whether a drag is a copy or a > move? I.e. either we hard code "Ctrl" to be copy in the protocol, or > DND > will behave different on each compositor. Not sure I like any of > those > options. Yes, this would be implementation-dependent in the compositor as my proposal goes. We have 3 players here, whoever gets to handle the modifier->action translation, there's room for confusion in cross- DE/toolkit cases. Focusing on actions, I see the following possible data flows here (depicting the same situation on all: initial negotiation, changes on the dest on say pointer motion, and a modifier change): 1. If handled purely by the source: wl_data_source compositorwl_data_offer == === -> notify_actions <- dest_actions <- -> preferred_action action <--> action ... (pointer moves across widgets) notify_actions <- dest_actions <- -> preferred_action action <--> action ... (modifiers change) modifiers<- -> preferred_action action <--> action 2. If handled purely by the dest: wl_data_source compositorwl_data_offer == === -> notify_actions -> source_actions notify_actions <- preferred_action <- action <--> action ... (pointer moves across widgets) notify_actions <- preferred_action <- action <--> action ... (modifiers change) -> modifiers preferred_action <- action <--> action 3. If handled purely by the compositor: wl_data_source compositorwl_data_offer == === -> notify_actions <- action <--> action ... (pointer moves across widgets) notify_actions <- action <--> action ... (modifiers change) action <--> action Options #1 and #2 involve roundtrips, option #3 doesn't. Options #1 and #2 would still need some validation on the compositor to avoid picking options unknown to either side. So option #3 seems neater, although in practice drag destinations are non-uniform, some action negotiation similar to #2 is needed, so my proposal goes: wl_data_source compositorwl_data_offer == === -> notify_actions -> source_actions notify_actions <- action <--> action ... (pointer moves across widgets) notify_actions <- action <--> action ... (modifiers change) action <--> action > > > > > > > > > I also don't see the variable state to be a good thing > > > considering we'd > > > have three independent states, meaning it'd get a very racy and > > > non-deterministic protocol. > > > > Ideally all of this would have been right from the start as > > parameters > > to wl_data_device.start_drag, wl_data_device.enter and > > wl_data_offer.accept. Despite the extra combinations in data flow, > > I > > fail to see how this gets racy or non-deterministic, you surely > > will > > get a supported action a
Re: Compositor grabs (was: Re: [PATCH] protocol: Add DnD actions)
On 17 April 2015 at 23:40, Bill Spitzak wrote: > On 04/17/2015 05:16 AM, Carlos Garnacho wrote: > >> Let's expand on that example, maybe far-streched, but certainly possible: >> - I'm manipulating a client window with 2 fingers on the touchscreen >> (say zooming an image) >> - Any other interaction on the client makes it pop up an xdg_popup >> (say a third touch, a key combo, or the pointer) >> - Q: what happens with the two first touches? > > > The touches held down should continue to go to the surface they were going > to before, while the events related to the event that triggered the grab > will go to the grab client. > > When the two touches are fully released then the next press of them will go > to the grab client. Yes, I think from user point of view this is the only option that makes sense. Making gesture result dependent on exact touch and release event order will lead to very inconsistent behaviour depending on minor timing changes in performing the touches. > > Wayland could guarantee that the release and drag events go to the same > client that got the press event. Grabs just dictate where new press events > go. > > I don't think this situation will happen much, due to server-induced grabs > which are fully synchronous, so you cannot press two buttons in two > different widgets and get two grabs. Instead one of them will get the grab > and that one will see the other button. Actually I think this should be possible. Consider a hypothetical photo organizing application which has photo preview in one widget and thumbnail organizer in another widget. As a user of such app you should be able to zoom and rotate the preview and drag around the thumbnails around and to different windows (eg file manager) at the same time. If starting a drag in the thumbnail management widget (accidentally or intentionally) cancels the zoom gesture in the preview you are going to have a hell of a time trying to explain to a user WTF is going on. Thanks Michal ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: Compositor grabs (was: Re: [PATCH] protocol: Add DnD actions)
On 04/17/2015 05:16 AM, Carlos Garnacho wrote: Let's expand on that example, maybe far-streched, but certainly possible: - I'm manipulating a client window with 2 fingers on the touchscreen (say zooming an image) - Any other interaction on the client makes it pop up an xdg_popup (say a third touch, a key combo, or the pointer) - Q: what happens with the two first touches? The touches held down should continue to go to the surface they were going to before, while the events related to the event that triggered the grab will go to the grab client. When the two touches are fully released then the next press of them will go to the grab client. Wayland could guarantee that the release and drag events go to the same client that got the press event. Grabs just dictate where new press events go. I don't think this situation will happen much, due to server-induced grabs which are fully synchronous, so you cannot press two buttons in two different widgets and get two grabs. Instead one of them will get the grab and that one will see the other button. ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: Compositor grabs (was: Re: [PATCH] protocol: Add DnD actions)
On 17 April 2015 at 14:16, Carlos Garnacho wrote: > Hey Jonas, > > This is drifting a bit off the topic of the original thread, better to > spin this off. I'll reply to the DnD bits in another email. > > On Fri, Apr 17, 2015 at 9:50 AM, Jonas Ådahl wrote: >> On Thu, Apr 16, 2015 at 12:55:31PM +0200, Carlos Garnacho wrote: >>> Hey Jonas, >>> >>> On Thu, Apr 16, 2015 at 10:15 AM, Jonas Ådahl wrote: >>> >>> >>> >>> > >>> > I'd have to agree on that it doesn't seem like the best thing to let the >>> > compositor choose the preferred action. Having it apply compositor >>> > specific policy given what the keyboard state or similar will probably >>> > never work out very well, given that for example what modifier state >>> > means what type of action is very application dependent. >>> > >>> > On the other hand, I'm not sure we can currently rely on either side >>> > having keyboard focus during the drag. In weston the source will have the >>> > focus because starting the drag was done with a click which gave the >>> > surface keyboard focus implicitly, but what'd happen if the compositor >>> > has keyboard-focus-follows-mouse? We could probably say that drag implies >>> > an implicit grab on another device on the same seat to enforce no >>> > changing of keyboard focus, but not sure that is better. >>> >>> In gtk+/gnome we currently have the following keybindings active during DnD: >>> >>> - Cursor keys move the drag point, modifiers affect speed >>> - Esc key cancels drag >>> - Modifiers alone pick an action from the offered list >>> >>> So ok, the latter is dubious to punt to compositors, but there's >>> basically no other choice with the 2 first ones. >>> >>> More generally, I have the opinion that compositors grabs should >>> behave all consistently, as in: >>> >>> - Ensuring clients reset all input state (we eg. don't cancel ongoing >>> touches when xdg_popup/dnd/... grabs kick in) >> >> What does "client reset all input state" mean? What state can a client >> reset? > > Let's expand on that example, maybe far-streched, but certainly possible: > - I'm manipulating a client window with 2 fingers on the touchscreen > (say zooming an image) > - Any other interaction on the client makes it pop up an xdg_popup > (say a third touch, a key combo, or the pointer) > - Q: what happens with the two first touches? > > Ideally, these touches should be "cancelled" on the first surface > (wl_touch.cancelled seems to be per client, not per-surface though), > and stay non-reactive within the grab, so they wouldn't trigger > anything unintended (they're implicitly grabbed on another surface > after all) > The other option is to keep the implicit grab for those touches. You could just keep those two fingers grabbed to the old surface until released but if the old surface wants some other input there is no way to tell. The user can, however, potentially get rid of the popup/dnd and continue the operation that was started in the previous grab, presumably. Which may not make that much sense with a modal popup but might actually work with dnd. Thanks Michal ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: Compositor grabs (was: Re: [PATCH] protocol: Add DnD actions)
Hey Michal, On Fri, Apr 17, 2015 at 4:27 PM, Michal Suchanek wrote: >> Nothing prevents wayland compositors today from undoing grabs when >> clients get destroyed, or surfaces don't respond to pings. Affecting >> this proposal, the pointer should re-enter the surface underneath and >> keyboard focus restablished after the grab is "broken". > > Nothing prevents the X server either. It hopefully breaks the grabs > initiated by clients when they are destroyed, too. However, there is > no mechanism for breaking a grab of a client other than destroying it > *and* destroying the client cannot be done from within the X session > because it is grabbed. Sure, Xorg's server will also clean up the grab after the client is destroyed. There's a XF86Ungrab keysym that also does that, but it's usually disabled by default, because screensavers and security. Wayland actually got this right, grabs happen within the compositor, on well defined entry points, so the compositor can impose its lifetime. This is a merit of wayland I'm not discussing at all, I'm more concerned on what happens with the devices that didn't *seemingly* need grabbing. Cheers, Carlos ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: Compositor grabs (was: Re: [PATCH] protocol: Add DnD actions)
On 17 April 2015 at 16:15, Carlos Garnacho wrote: > Hey Michal, > > On Fri, Apr 17, 2015 at 2:47 PM, Michal Suchanek wrote: > > >>> In the X11 world, this would roughly be a Virtual Core >>> Pointer+Keyboard grab (not that touch and active grabs are trouble >>> free in X11, but...), GTK+ for example does grab both devices on every >>> of those grabbing places wayland/xdg protocols are trying to cater for >>> (I've even pondered about adding a gdk_device_grab_pair() for years). >>> >>> I think some consistent model should be devised here, and embedded >>> into the protocol (docs). >>> >> >> The serious problem with X11 grabs is that they are completely >> independent of the event that triggered them and can only be released >> by the application that started them. >> >> So it happens that an application that gets stuck due to code error or >> is running in a debugger at the time a grab is active never releases >> the grab which >> >> 1) prevents other applications from receiving input >> 2) prevents further grabs >> >> It might be worth considering if there is generic enough drag semantic >> that the grab could be handled in compositor outside of application >> code, even with click-lock and whatnot. > > Nothing prevents wayland compositors today from undoing grabs when > clients get destroyed, or surfaces don't respond to pings. Affecting > this proposal, the pointer should re-enter the surface underneath and > keyboard focus restablished after the grab is "broken". Nothing prevents the X server either. It hopefully breaks the grabs initiated by clients when they are destroyed, too. However, there is no mechanism for breaking a grab of a client other than destroying it *and* destroying the client cannot be done from within the X session because it is grabbed. Thanks Michal ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: Compositor grabs (was: Re: [PATCH] protocol: Add DnD actions)
Hey Michal, On Fri, Apr 17, 2015 at 2:47 PM, Michal Suchanek wrote: >> In the X11 world, this would roughly be a Virtual Core >> Pointer+Keyboard grab (not that touch and active grabs are trouble >> free in X11, but...), GTK+ for example does grab both devices on every >> of those grabbing places wayland/xdg protocols are trying to cater for >> (I've even pondered about adding a gdk_device_grab_pair() for years). >> >> I think some consistent model should be devised here, and embedded >> into the protocol (docs). >> > > The serious problem with X11 grabs is that they are completely > independent of the event that triggered them and can only be released > by the application that started them. > > So it happens that an application that gets stuck due to code error or > is running in a debugger at the time a grab is active never releases > the grab which > > 1) prevents other applications from receiving input > 2) prevents further grabs > > It might be worth considering if there is generic enough drag semantic > that the grab could be handled in compositor outside of application > code, even with click-lock and whatnot. Nothing prevents wayland compositors today from undoing grabs when clients get destroyed, or surfaces don't respond to pings. Affecting this proposal, the pointer should re-enter the surface underneath and keyboard focus restablished after the grab is "broken". > > During a grab you might want to > > 1) process seemingly unrelated events in the grabbing application (eg. > Esc for canceling the action in the application) This IMO belongs in the context of the grab, xdg_popup grabs are just "grab on this surface", this comes off naturally if "focus it in" is implied. As for DnD grabs, the compositor essentially replaces wl_pointer/keyboard/touch semantics with wl_data_device ones, I believe whatever happens must happen on the current semantics, not as result of a mixture. > 2) process 'global' window management keybindings (eg. close any > random application, including the one that is grabbing input) > 3) perform window management action (eg. raise the intended drag > target with mouse movement if you have bindings for raising widows > with mouse movement) > - note that you might want to be able to raise windows with DnD > active but not with a slider active > > As to what happens with unrelated input that is not consumed by > compositor is questionable. Canceling all but the input that started > the grab might seem like a good idea but what if I am manipulating a > window with two fingers and then I get the idea to slide a slider with > third finger? That'd be an "implicit" grab, I was mainly talking about "active" compositor grabs (using X11isms here). Implicit grabs shouldn't ever affect each other. > > IMHO the two finger gesture should continue at the very least after I > release the slider but preferably even as I manipulate the slider. What I'd expect to happen (and mutter/gtk+ certainly do/allow that) is: - both things may happen simultaneously - if those two operations are mutually exclusive, it is the client responsibility to make them so. Note I'm talking exclusively about implicit grabs here. The toolkit should automatically cater for the cases where input is being taken away, be it compositor or toolkit-internal grabs. Cheers, Carlos ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: Compositor grabs (was: Re: [PATCH] protocol: Add DnD actions)
On 17 April 2015 at 14:16, Carlos Garnacho wrote: > Hey Jonas, > > This is drifting a bit off the topic of the original thread, better to > spin this off. I'll reply to the DnD bits in another email. > > On Fri, Apr 17, 2015 at 9:50 AM, Jonas Ådahl wrote: >> On Thu, Apr 16, 2015 at 12:55:31PM +0200, Carlos Garnacho wrote: >>> Hey Jonas, >>> >>> On Thu, Apr 16, 2015 at 10:15 AM, Jonas Ådahl wrote: >>> More generally, I have the opinion that compositors grabs should >>> behave all consistently, as in: >>> >>> - Ensuring clients reset all input state (we eg. don't cancel ongoing >>> touches when xdg_popup/dnd/... grabs kick in) >> >> What does "client reset all input state" mean? What state can a client >> reset? > > Let's expand on that example, maybe far-streched, but certainly possible: > - I'm manipulating a client window with 2 fingers on the touchscreen > (say zooming an image) > - Any other interaction on the client makes it pop up an xdg_popup > (say a third touch, a key combo, or the pointer) > - Q: what happens with the two first touches? > > Ideally, these touches should be "cancelled" on the first surface > (wl_touch.cancelled seems to be per client, not per-surface though), > and stay non-reactive within the grab, so they wouldn't trigger > anything unintended (they're implicitly grabbed on another surface > after all) > > Currently, on the weston code, focusing a bit on the all-touch case, > it actually happens the worst that could happen, the xdg_popup touch > grab redirects already started touch sequences to the grabbing surface > right away, and the original surface will be deaf to them of a sudden, > leading to inconsistent state on both the original and the grabbing > surface wrt those touch sequences. The DnD touch grab doesn't fare > much better, it will ignore other touches than the one starting the > drag, so the pre-grab touches would effectively go nowhere, and AFAICS > similar issues arise with pointer grabs vs touch. > > With keyboards, it happens likewise, if compositors are to possibly > consume events there, focus should move out of the previous surface. > IMO, any grabbing model that does not redirect all input, nor ensures > a coherent state in clients is calling for trouble... > > In the X11 world, this would roughly be a Virtual Core > Pointer+Keyboard grab (not that touch and active grabs are trouble > free in X11, but...), GTK+ for example does grab both devices on every > of those grabbing places wayland/xdg protocols are trying to cater for > (I've even pondered about adding a gdk_device_grab_pair() for years). > > I think some consistent model should be devised here, and embedded > into the protocol (docs). > The serious problem with X11 grabs is that they are completely independent of the event that triggered them and can only be released by the application that started them. So it happens that an application that gets stuck due to code error or is running in a debugger at the time a grab is active never releases the grab which 1) prevents other applications from receiving input 2) prevents further grabs It might be worth considering if there is generic enough drag semantic that the grab could be handled in compositor outside of application code, even with click-lock and whatnot. During a grab you might want to 1) process seemingly unrelated events in the grabbing application (eg. Esc for canceling the action in the application) 2) process 'global' window management keybindings (eg. close any random application, including the one that is grabbing input) 3) perform window management action (eg. raise the intended drag target with mouse movement if you have bindings for raising widows with mouse movement) - note that you might want to be able to raise windows with DnD active but not with a slider active As to what happens with unrelated input that is not consumed by compositor is questionable. Canceling all but the input that started the grab might seem like a good idea but what if I am manipulating a window with two fingers and then I get the idea to slide a slider with third finger? IMHO the two finger gesture should continue at the very least after I release the slider but preferably even as I manipulate the slider. Thanks Michal ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Compositor grabs (was: Re: [PATCH] protocol: Add DnD actions)
Hey Jonas, This is drifting a bit off the topic of the original thread, better to spin this off. I'll reply to the DnD bits in another email. On Fri, Apr 17, 2015 at 9:50 AM, Jonas Ådahl wrote: > On Thu, Apr 16, 2015 at 12:55:31PM +0200, Carlos Garnacho wrote: >> Hey Jonas, >> >> On Thu, Apr 16, 2015 at 10:15 AM, Jonas Ådahl wrote: >> >> >> >> > >> > I'd have to agree on that it doesn't seem like the best thing to let the >> > compositor choose the preferred action. Having it apply compositor >> > specific policy given what the keyboard state or similar will probably >> > never work out very well, given that for example what modifier state >> > means what type of action is very application dependent. >> > >> > On the other hand, I'm not sure we can currently rely on either side >> > having keyboard focus during the drag. In weston the source will have the >> > focus because starting the drag was done with a click which gave the >> > surface keyboard focus implicitly, but what'd happen if the compositor >> > has keyboard-focus-follows-mouse? We could probably say that drag implies >> > an implicit grab on another device on the same seat to enforce no >> > changing of keyboard focus, but not sure that is better. >> >> In gtk+/gnome we currently have the following keybindings active during DnD: >> >> - Cursor keys move the drag point, modifiers affect speed >> - Esc key cancels drag >> - Modifiers alone pick an action from the offered list >> >> So ok, the latter is dubious to punt to compositors, but there's >> basically no other choice with the 2 first ones. >> >> More generally, I have the opinion that compositors grabs should >> behave all consistently, as in: >> >> - Ensuring clients reset all input state (we eg. don't cancel ongoing >> touches when xdg_popup/dnd/... grabs kick in) > > What does "client reset all input state" mean? What state can a client > reset? Let's expand on that example, maybe far-streched, but certainly possible: - I'm manipulating a client window with 2 fingers on the touchscreen (say zooming an image) - Any other interaction on the client makes it pop up an xdg_popup (say a third touch, a key combo, or the pointer) - Q: what happens with the two first touches? Ideally, these touches should be "cancelled" on the first surface (wl_touch.cancelled seems to be per client, not per-surface though), and stay non-reactive within the grab, so they wouldn't trigger anything unintended (they're implicitly grabbed on another surface after all) Currently, on the weston code, focusing a bit on the all-touch case, it actually happens the worst that could happen, the xdg_popup touch grab redirects already started touch sequences to the grabbing surface right away, and the original surface will be deaf to them of a sudden, leading to inconsistent state on both the original and the grabbing surface wrt those touch sequences. The DnD touch grab doesn't fare much better, it will ignore other touches than the one starting the drag, so the pre-grab touches would effectively go nowhere, and AFAICS similar issues arise with pointer grabs vs touch. With keyboards, it happens likewise, if compositors are to possibly consume events there, focus should move out of the previous surface. IMO, any grabbing model that does not redirect all input, nor ensures a coherent state in clients is calling for trouble... In the X11 world, this would roughly be a Virtual Core Pointer+Keyboard grab (not that touch and active grabs are trouble free in X11, but...), GTK+ for example does grab both devices on every of those grabbing places wayland/xdg protocols are trying to cater for (I've even pondered about adding a gdk_device_grab_pair() for years). I think some consistent model should be devised here, and embedded into the protocol (docs). > >> - Ensuring the grab affects the routing of all devices/events, and >> that no client gets partial streams > > You mean that pointer grab should implicitly grab the touch and keyboard > device as well? What do you mean with partial stream? See the example above with surfaces receiving not properly started/finalized series of event for touch sequences, or the suggestions in the DnD thread about having drag source/dest handle keyboard events, while the compositor is certainly interested in handling/consuming some keyboard events too. Cheers, Carlos ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: [PATCH] protocol: Add DnD actions
On Thu, Apr 16, 2015 at 12:55:31PM +0200, Carlos Garnacho wrote:
> Hey Jonas,
>
> On Thu, Apr 16, 2015 at 10:15 AM, Jonas Ådahl wrote:
>
>
>
> >
> > I'd have to agree on that it doesn't seem like the best thing to let the
> > compositor choose the preferred action. Having it apply compositor
> > specific policy given what the keyboard state or similar will probably
> > never work out very well, given that for example what modifier state
> > means what type of action is very application dependent.
> >
> > On the other hand, I'm not sure we can currently rely on either side
> > having keyboard focus during the drag. In weston the source will have the
> > focus because starting the drag was done with a click which gave the
> > surface keyboard focus implicitly, but what'd happen if the compositor
> > has keyboard-focus-follows-mouse? We could probably say that drag implies
> > an implicit grab on another device on the same seat to enforce no
> > changing of keyboard focus, but not sure that is better.
>
> In gtk+/gnome we currently have the following keybindings active during DnD:
>
> - Cursor keys move the drag point, modifiers affect speed
> - Esc key cancels drag
> - Modifiers alone pick an action from the offered list
>
> So ok, the latter is dubious to punt to compositors, but there's
> basically no other choice with the 2 first ones.
>
> More generally, I have the opinion that compositors grabs should
> behave all consistently, as in:
>
> - Ensuring clients reset all input state (we eg. don't cancel ongoing
> touches when xdg_popup/dnd/... grabs kick in)
What does "client reset all input state" mean? What state can a client
reset?
> - Ensuring the grab affects the routing of all devices/events, and
> that no client gets partial streams
You mean that pointer grab should implicitly grab the touch and keyboard
device as well? What do you mean with partial stream?
Do I understand you correctly in that you mean when a DND is triggered,
the surface looses its grab? That would be consistent with other server
driven user interactions such as move and resize.
>
> For the touch case, depending on how the grab is implemented, with the
> current guidelines the only 2 choices are "leave the client in
> inconsistent state" or "make the client still receives ongoing touches
> despite the pointer grab" (same applies if the grab is touch
> triggered, only with the other touches that didn't trigger the grab).
>
> More on topic, keyboards are also funky if we keep focus on clients,
> you can conceivably Esc/Ctrl-Q/... to close the app you're dragging
> from. IMO the way forward is precisely this, the compositor becomes in
> control of the keyboard, and we offer the missing semantics to cover
> for this.
Meaning its the compositor that decides whether a drag is a copy or a
move? I.e. either we hard code "Ctrl" to be copy in the protocol, or DND
will behave different on each compositor. Not sure I like any of those
options.
>
> >
> > I also don't see the variable state to be a good thing considering we'd
> > have three independent states, meaning it'd get a very racy and
> > non-deterministic protocol.
>
> Ideally all of this would have been right from the start as parameters
> to wl_data_device.start_drag, wl_data_device.enter and
> wl_data_offer.accept. Despite the extra combinations in data flow, I
> fail to see how this gets racy or non-deterministic, you surely will
> get a supported action and mimetype on the drag dest, or the drag will
> be cancelled. If it is more conforting, we can make it more explicit
> that wl_data_offer.notify_actions is the central point where DnD
> success/action is decided, and that wl_data_offer.accept/receive are a
> second step after it.
We can extend a request by "prefixing" it with another request that
depends on the final request (start_drag for example) to take affect.
We can do the same for events, i.e. we first send an event that only
takes effect when another event is sent. This is for example how we are
extending wl_pointer.axis with axis source information.
This way we can effectively add new parameters to start_drag or accept,
just that in the protocol we make them separate requests/events.
>
> >
> > If we'd want to have the destination choose the action, the source
> > should advertise its possible actions, forwarded by the compositor to
> > the destination ("atomically", without intermediate committed state).
>
> wl_data_offer.source_actions?
>
> >
> > If we want to enable one of the clients to rely on keyboard modifier
> > state, I think this should be communicated to the deciding end point;
>
> Which used to be the drag source in XDND, as the holder of
> pointer/keyboard grabs. It would update the "preferred action" that
> was communicated then to the drag dest. If we do this 1:1 we probably
> don't get rid of any of the "racyness" you see, and I suspect punting
> it to the drag dest will involve a few changes in toolkits, plus
> separate event ha
Re: [PATCH] protocol: Add DnD actions
On 04/16/2015 03:55 AM, Carlos Garnacho wrote: In gtk+/gnome we currently have the following keybindings active during DnD: - Cursor keys move the drag point, modifiers affect speed - Esc key cancels drag - Modifiers alone pick an action from the offered list The user has to hold the mouse button down during these? That does not sound very useful. Or is it possible to start DnD with a keystroke? What happens then if the user moves the mouse or pushes the mouse button? I really need a clearer explanation of these keystrokes. I suspect some requirements of implementing things on X are leaking into your description, and you are talking about a global handicapped-assistance api (which I agree would be done by the compositor, but applies everywhere, not just DnD). So ok, the latter is dubious to punt to compositors, but there's basically no other choice with the 2 first ones. It can be handled by either the source or dest getting a grab and responding to the key events with some new api that manually moves the drop target. In fact there is a lot of proposed things for the grab to be able to completely control the cursor position, this would be another one. I do agree that making Esc cause the compositor to cancel a grab unconditionally is an acceptable idea. If you don't have to hold down the mouse to do DnD, or if there is no position on the screen that you can drop without anything happening, then this is necessary. It does seem like the source getting the keyboard drag is consistent, and match how dragging any other part of the source's ui works. However I think it will work better for the destination to get the grab. This is because the destination has a more complex job, including actions that the source does not need to distinguish (insert vs replace, link vs copy), and if there is a popup menu to let the user choose the action they will think it is part of the destination and it would look far better if it was drawn with the destination's style, and user preferences for it are set on the destination. So during a DnD the keyboard focus should be (in effect) point-to-type. I don't think it should get mouse enter/exit/move events, instead the drag enter/exit/move would replace them. There will unfortunately have to be a keyboard focus-in event as that is the only way to communicate the current keymap to the client (it may be useful to add an event for that as there already is the problem where the compositor eats keystrokes, but can't tell the client about the new keymap without a fake exit/enter). I think both the source and destination should be able to modify the cursor. If the destination says it will accept the drop then it is it's job to update the cursor, as it may modify it to show clearly what will happen (for instance it may update it's own display to show the result of the drop, and change the cursor to just the pointer). If it says it won't, then the source sets the cursor to it's default version. Race conditions with this idea need to be solved... ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: [PATCH] protocol: Add DnD actions
Hey Jonas,
On Thu, Apr 16, 2015 at 10:15 AM, Jonas Ådahl wrote:
>
> I'd have to agree on that it doesn't seem like the best thing to let the
> compositor choose the preferred action. Having it apply compositor
> specific policy given what the keyboard state or similar will probably
> never work out very well, given that for example what modifier state
> means what type of action is very application dependent.
>
> On the other hand, I'm not sure we can currently rely on either side
> having keyboard focus during the drag. In weston the source will have the
> focus because starting the drag was done with a click which gave the
> surface keyboard focus implicitly, but what'd happen if the compositor
> has keyboard-focus-follows-mouse? We could probably say that drag implies
> an implicit grab on another device on the same seat to enforce no
> changing of keyboard focus, but not sure that is better.
In gtk+/gnome we currently have the following keybindings active during DnD:
- Cursor keys move the drag point, modifiers affect speed
- Esc key cancels drag
- Modifiers alone pick an action from the offered list
So ok, the latter is dubious to punt to compositors, but there's
basically no other choice with the 2 first ones.
More generally, I have the opinion that compositors grabs should
behave all consistently, as in:
- Ensuring clients reset all input state (we eg. don't cancel ongoing
touches when xdg_popup/dnd/... grabs kick in)
- Ensuring the grab affects the routing of all devices/events, and
that no client gets partial streams
For the touch case, depending on how the grab is implemented, with the
current guidelines the only 2 choices are "leave the client in
inconsistent state" or "make the client still receives ongoing touches
despite the pointer grab" (same applies if the grab is touch
triggered, only with the other touches that didn't trigger the grab).
More on topic, keyboards are also funky if we keep focus on clients,
you can conceivably Esc/Ctrl-Q/... to close the app you're dragging
from. IMO the way forward is precisely this, the compositor becomes in
control of the keyboard, and we offer the missing semantics to cover
for this.
>
> I also don't see the variable state to be a good thing considering we'd
> have three independent states, meaning it'd get a very racy and
> non-deterministic protocol.
Ideally all of this would have been right from the start as parameters
to wl_data_device.start_drag, wl_data_device.enter and
wl_data_offer.accept. Despite the extra combinations in data flow, I
fail to see how this gets racy or non-deterministic, you surely will
get a supported action and mimetype on the drag dest, or the drag will
be cancelled. If it is more conforting, we can make it more explicit
that wl_data_offer.notify_actions is the central point where DnD
success/action is decided, and that wl_data_offer.accept/receive are a
second step after it.
>
> If we'd want to have the destination choose the action, the source
> should advertise its possible actions, forwarded by the compositor to
> the destination ("atomically", without intermediate committed state).
wl_data_offer.source_actions?
>
> If we want to enable one of the clients to rely on keyboard modifier
> state, I think this should be communicated to the deciding end point;
Which used to be the drag source in XDND, as the holder of
pointer/keyboard grabs. It would update the "preferred action" that
was communicated then to the drag dest. If we do this 1:1 we probably
don't get rid of any of the "racyness" you see, and I suspect punting
it to the drag dest will involve a few changes in toolkits, plus
separate event handling paths from X11's.
> which I suspect is what Bill is talking about regarding the 'state'
> that is sent from the compositor.
>
>> >
>> > I am VERY much in favor of moving as much logic as possible from the
>> > compositor to the clients. And f(A,B,state) is a very complicated
>> > function. B may not be a list, it could be, in effect, infinite in
>> > size
>>
>> Are you maybe folding mimetypes and actions as A/B/C above? The only
>> thing that can grow "unbounded" is the mimetype list, the possible
>> actions are always a fixed set, and resolved after the mimetype is
>> negotiated. AFAICS "B" corresponds to the dest side, which confuses
>> me, because both the picked mimetype and action will always be a
>> subset of A's.
>>
>> > (a client conceivably could ask the user to type a filename that the
>> > drop should go to), can vary quickly (as the user moves across
>> > widget
>> > boundaries), and can contain items the compositor has no business
>> > knowing about (a paint program may ask how to tile a dropped
>> > pattern).
>>
>> Ah, I see, perhaps it's rather "varying over time" than "infinite"?
>> TBH I don't see how this is different to how mimetypes are dealt with,
>> you definitely don't have to calculate all possible states at once,
>> just for the position you're in.
>
> For clarification: I think
Re: [PATCH] protocol: Add DnD actions
Hi,
Disclaimer: I don't know much how XDND works, but have some comments
anyway. See below:
On Sat, Apr 11, 2015 at 12:59:48PM +0200, Carlos Garnacho wrote:
> On vie, 2015-04-10 at 12:44 -0700, Bill Spitzak wrote:
> > On 04/10/2015 02:29 AM, Carlos Garnacho wrote:
> >
> > > > However the
> > > > compositor could send state along with A to the dest.
> > >
> > > This could also work,
> >
> > Thanks I think I finally made it clear what I am asking for.
>
> In fairness, your proposal started as something quite different, we
> happened to find the middle ground.
>
> >
> > So "intersection" is now f(A,B,state), and what I propose is:
> >
> > source sends A to compositor
> > compositor sends A to dest
> > compositor sends state to dest
> > dest does C = f(A,B,state)
> > dest sends C to compositor
> > compositor sends C to source
> >
> > "compositor sends state to dest" could very well mean that the dest
> > gets
> > events for the seat, as though it was temporarily focused. This
> > would
> > also allow the dest to make a popup menu to let the user choose an
> > action.
> >
> > > I however think it makes sense to abstract this a
> > > bit wrt the devices involved. Generally, wl_data_device nicely
> > > abstracts from the pointer/touch/device being used throughout the
> > > operation. It makes sense to me to do the same here, if what we are
> > > pursuing is a DnD action. Also, this IMO gives some degree of
> > > liberty
> > > wrt form factors, testing...
> >
> > Any such concerns like this apply to *all* events, there is nothing
> > special about the events during a DnD, and the clients must already
> > have
> > code to "abstract wrt the devices involved". The DnD should indicate
> > which "seat" is doing the drag however.
>
> DnD can be driven by mouse, touch, and stili when wl_tablet arrives.
> There's however no exposure of this on the events sent by data_device,
> everything happens in its own semantics rather than wl_pointer's or
> wl_touch's. I'm not sure why the actions derived from keyboards
> deserve a different treatment.
>
> >
> > > It's true that the modifier mapping becomes compositor-dependent
> > > (It's
> > > been traditionally on toolkit domain), but then so is every
> > > keycombo
> > > the compositor wants to implement.
> >
> > That seems completely different, as compositor bindings do actions
> > without communicating with any client (ie a key that launches a
> > particular client will work when there are no clients).
> >
> > > But I won't be as hard positioned on this one, if the consensus is
> > > that we better send pressed buttons/modifiers to the dest, it can
> > > be
> > > made to work.
> >
> > Anybody else want to say anything?
I'd have to agree on that it doesn't seem like the best thing to let the
compositor choose the preferred action. Having it apply compositor
specific policy given what the keyboard state or similar will probably
never work out very well, given that for example what modifier state
means what type of action is very application dependent.
On the other hand, I'm not sure we can currently rely on either side
having keyboard focus during the drag. In weston the source will have the
focus because starting the drag was done with a click which gave the
surface keyboard focus implicitly, but what'd happen if the compositor
has keyboard-focus-follows-mouse? We could probably say that drag implies
an implicit grab on another device on the same seat to enforce no
changing of keyboard focus, but not sure that is better.
I also don't see the variable state to be a good thing considering we'd
have three independent states, meaning it'd get a very racy and
non-deterministic protocol.
If we'd want to have the destination choose the action, the source
should advertise its possible actions, forwarded by the compositor to
the destination ("atomically", without intermediate committed state).
If we want to enable one of the clients to rely on keyboard modifier
state, I think this should be communicated to the deciding end point;
which I suspect is what Bill is talking about regarding the 'state'
that is sent from the compositor.
> >
> > I am VERY much in favor of moving as much logic as possible from the
> > compositor to the clients. And f(A,B,state) is a very complicated
> > function. B may not be a list, it could be, in effect, infinite in
> > size
>
> Are you maybe folding mimetypes and actions as A/B/C above? The only
> thing that can grow "unbounded" is the mimetype list, the possible
> actions are always a fixed set, and resolved after the mimetype is
> negotiated. AFAICS "B" corresponds to the dest side, which confuses
> me, because both the picked mimetype and action will always be a
> subset of A's.
>
> > (a client conceivably could ask the user to type a filename that the
> > drop should go to), can vary quickly (as the user moves across
> > widget
> > boundaries), and can contain items
Re: [PATCH] protocol: Add DnD actions
On vie, 2015-04-10 at 12:44 -0700, Bill Spitzak wrote: > On 04/10/2015 02:29 AM, Carlos Garnacho wrote: > > > > However the > > > compositor could send state along with A to the dest. > > > > This could also work, > > Thanks I think I finally made it clear what I am asking for. In fairness, your proposal started as something quite different, we happened to find the middle ground. > > So "intersection" is now f(A,B,state), and what I propose is: > > source sends A to compositor > compositor sends A to dest > compositor sends state to dest > dest does C = f(A,B,state) > dest sends C to compositor > compositor sends C to source > > "compositor sends state to dest" could very well mean that the dest > gets > events for the seat, as though it was temporarily focused. This > would > also allow the dest to make a popup menu to let the user choose an > action. > > > I however think it makes sense to abstract this a > > bit wrt the devices involved. Generally, wl_data_device nicely > > abstracts from the pointer/touch/device being used throughout the > > operation. It makes sense to me to do the same here, if what we are > > pursuing is a DnD action. Also, this IMO gives some degree of > > liberty > > wrt form factors, testing... > > Any such concerns like this apply to *all* events, there is nothing > special about the events during a DnD, and the clients must already > have > code to "abstract wrt the devices involved". The DnD should indicate > which "seat" is doing the drag however. DnD can be driven by mouse, touch, and stili when wl_tablet arrives. There's however no exposure of this on the events sent by data_device, everything happens in its own semantics rather than wl_pointer's or wl_touch's. I'm not sure why the actions derived from keyboards deserve a different treatment. > > > It's true that the modifier mapping becomes compositor-dependent > > (It's > > been traditionally on toolkit domain), but then so is every > > keycombo > > the compositor wants to implement. > > That seems completely different, as compositor bindings do actions > without communicating with any client (ie a key that launches a > particular client will work when there are no clients). > > > But I won't be as hard positioned on this one, if the consensus is > > that we better send pressed buttons/modifiers to the dest, it can > > be > > made to work. > > Anybody else want to say anything? > > I am VERY much in favor of moving as much logic as possible from the > compositor to the clients. And f(A,B,state) is a very complicated > function. B may not be a list, it could be, in effect, infinite in > size Are you maybe folding mimetypes and actions as A/B/C above? The only thing that can grow "unbounded" is the mimetype list, the possible actions are always a fixed set, and resolved after the mimetype is negotiated. AFAICS "B" corresponds to the dest side, which confuses me, because both the picked mimetype and action will always be a subset of A's. > (a client conceivably could ask the user to type a filename that the > drop should go to), can vary quickly (as the user moves across > widget > boundaries), and can contain items the compositor has no business > knowing about (a paint program may ask how to tile a dropped > pattern). Ah, I see, perhaps it's rather "varying over time" than "infinite"? TBH I don't see how this is different to how mimetypes are dealt with, you definitely don't have to calculate all possible states at once, just for the position you're in. > > The list A does not vary much and is much shorter IMHO. I think it > is > really move vs copy in almost all cases. Even file linking could be > considered a "copy" as far as the source is concerned (it does not > have > to know that a link was made rather than the data copied). Cheers, Carlos ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: [PATCH] protocol: Add DnD actions
On 04/10/2015 02:29 AM, Carlos Garnacho wrote: However the compositor could send state along with A to the dest. This could also work, Thanks I think I finally made it clear what I am asking for. So "intersection" is now f(A,B,state), and what I propose is: source sends A to compositor compositor sends A to dest compositor sends state to dest dest does C = f(A,B,state) dest sends C to compositor compositor sends C to source "compositor sends state to dest" could very well mean that the dest gets events for the seat, as though it was temporarily focused. This would also allow the dest to make a popup menu to let the user choose an action. I however think it makes sense to abstract this a bit wrt the devices involved. Generally, wl_data_device nicely abstracts from the pointer/touch/device being used throughout the operation. It makes sense to me to do the same here, if what we are pursuing is a DnD action. Also, this IMO gives some degree of liberty wrt form factors, testing... Any such concerns like this apply to *all* events, there is nothing special about the events during a DnD, and the clients must already have code to "abstract wrt the devices involved". The DnD should indicate which "seat" is doing the drag however. It's true that the modifier mapping becomes compositor-dependent (It's been traditionally on toolkit domain), but then so is every keycombo the compositor wants to implement. That seems completely different, as compositor bindings do actions without communicating with any client (ie a key that launches a particular client will work when there are no clients). But I won't be as hard positioned on this one, if the consensus is that we better send pressed buttons/modifiers to the dest, it can be made to work. Anybody else want to say anything? I am VERY much in favor of moving as much logic as possible from the compositor to the clients. And f(A,B,state) is a very complicated function. B may not be a list, it could be, in effect, infinite in size (a client conceivably could ask the user to type a filename that the drop should go to), can vary quickly (as the user moves across widget boundaries), and can contain items the compositor has no business knowing about (a paint program may ask how to tile a dropped pattern). The list A does not vary much and is much shorter IMHO. I think it is really move vs copy in almost all cases. Even file linking could be considered a "copy" as far as the source is concerned (it does not have to know that a link was made rather than the data copied). ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: [PATCH] protocol: Add DnD actions
Hey Bill, On mié, 2015-04-08 at 14:18 -0700, Bill Spitzak wrote: > > >* Additionally to modifier state, there's other > > keyboard/accessibility features as DnD is done in > > GNOME/GTK+ > > (eg. DnD driven by cursor keys), these must be > > implemented on > > the compositor, this sounds like conflicting with the > > expectation in your proposal to have the drag source > > receive > > key events [1]. > > I don't understand at all. Your proposal also requires the drag > source > to receive key events, since it can use those to change the set of > actions (for instance to add the "ask" action like you said in the > previous question). No, that's precisely why my proposal involves the compositor at all, the source offers all actions it can handle, after data_device.data_offer/enter, the offer does likewise. The compositor has then a map of the actions that both parts agree on, and can toggle actions when the user presses modifier keys, but these modifier keys should only be honored if both parts recognize such option, it should always fall back into an action that both parts agree on, or the drag will be cancelled. Note that this way the compositor is the sole handler of pointer and keyboard events during DnD. It perhaps confuses you that "notify_actions" can be called multiple times on both sides, that would be barely useful on the drag source tbh, too late to add arguments to start_drag() though. It's more of a necessity on the dest side, where toolkits have to update the mask as the pointer drifts across widgets. Changes in either the action mask from any side or the action picked by the compositor must always result in "action" events being sent to both sides. One thing the drag source stays responsible of is the pointer cursor, so it can change on reaction to wl_data_source.action. Cheers, Carlos > ___ > wayland-devel mailing list > wayland-devel@lists.freedesktop.org > http://lists.freedesktop.org/mailman/listinfo/wayland-devel ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: [PATCH] protocol: Add DnD actions
On 04/08/2015 07:46 AM, Carlos Garnacho wrote: I may be missing something. As far as I can tell your proposal is that the source and destination send sets of actions to the compositor, the compositor then intersects the sets, and sends the result of the intersection to the source and destination. My proposal is that only the source sends a list of actions, and the compositor sends that list to the destination. The destination intersects with it's list internally and sends the result of the intersection to the compositor, which then sends it back to the source. Otherwise everything is identical to what you are proposing. * How are actions conveyed? do we encode these in the mimetype string? how do we standardize on the actions? how do we make that backwards compatible? The exact same way they are being conveyed by your proposal. You have both the source and destination sending a list of actions to the compositor, use the same api here. * How do drag destinations react to unhandled options? Say a drag source appends "?action=ask" when you press Alt and the drag destination only knows copy/move, how does this fallback to an action that both parts recognize? Since the destination figures out which of the offered actions to use, it would not select this one. It would select another one. If it does not like any of them then we are in the exact same state as your proposal when the intersection is empty. * Additionally to modifier state, there's other keyboard/accessibility features as DnD is done in GNOME/GTK+ (eg. DnD driven by cursor keys), these must be implemented on the compositor, this sounds like conflicting with the expectation in your proposal to have the drag source receive key events [1]. I don't understand at all. Your proposal also requires the drag source to receive key events, since it can use those to change the set of actions (for instance to add the "ask" action like you said in the previous question). ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: [PATCH] protocol: Add DnD actions
Hey Bill, On mié, 2015-03-18 at 19:40 -0700, Bill Spitzak wrote: > This is still bothering me as being much too complicated. The bad news is, DnD is complex, there's plenty of prior usecases here that won't be fair/possible to have "simplified". > > I think a list of actions can be sent from the DnD source to the > target, > and the target selects one. There is no need for the compositor to > do > any intersection of the sets and there is no need to communicate the > set > of target actions to the compositor. > > Your concerns about shift state being handled by the source are > misplaced. This would not change at all under what I am proposing, > the > source could still use the shift state to change the list of actions. I see several issues with this: * How are actions conveyed? do we encode these in the mimetype string? how do we standardize on the actions? how do we make that backwards compatible? * How do drag destinations react to unhandled options? Say a drag source appends "?action=ask" when you press Alt and the drag destination only knows copy/move, how does this fallback to an action that both parts recognize? * Additionally to modifier state, there's other keyboard/accessibility features as DnD is done in GNOME/GTK+ (eg. DnD driven by cursor keys), these must be implemented on the compositor, this sounds like conflicting with the expectation in your proposal to have the drag source receive key events [1]. > > My proposal is basically to take yours, and remove the ability for > the > target to send a set of actions that the compositor then interesect > with > the source list. Instead this "intersection" is done by the target, > and > the target sends *one* action (or "none") indicating the result of > the > intersection. > > Unless you want the compositor to draw user interface to allow the > user > to choose the action, which seems very much a bad idea, I cannot see > what your proposal will allow to happen that this simplified version > would not. IMO, it kind of misses the point that DnD is a negotiation. I was suggesting that the "ask" action were implemented completely by the drag destination BTW, that wouldn't change much compared to XDnD. > > I do believe any kind of popup (like a menu for choosing "move or > copy") > would have to be done by the target. This is because the target may > have > extra actions that the source does not care about or does not know > about, such as "insert" verses "replace". The popup would grab the > keyboard focus but when dismissed it may go back to a different > client > than the target. > Agreed about the grabbing behavior, I'm unclear though on how would the actions in such popup work in your proposal: Say you start a drag with the special "ask" modifier, and the drag source changes its action list to convey "ask" (how exactly? is this the only option exposed?). When the destination shows the popup, how does it tell the source of the chosen action, so that eg. the selection is deleted after "move"? Cheers, Carlos [1] I'm actually meaning to propose some doc updates with more consistent guidelines for grabbing behavior, IMO how do keyboard/touch devices behave during the various pointer "grabs" is somewhat underdocumented... ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: [PATCH] protocol: Add DnD actions
Hey Bryce, Thanks for the review! I'm attaching new patches fixing the issues you/Marek have seen, the doc blurbs have been reworded in a few places. This v2 also adds a wl_data_offer.source_actions event. This allows drag destinations to adapt their action mask accordingly, instead just being practically able to set a static one, things get a bit more akin to XDnD in this regard. With v2, I've been able to make DnD on par with X in GTK+ (feature- wise, that is): https://git.gnome.org/browse/gtk+/log/?h=wip/wayland-dnd-actions https://git.gnome.org/browse/mutter/log/?h=wip/dnd-actions Cheers, Carlos On mié, 2015-03-18 at 16:56 -0700, Bryce Harrington wrote: > On Mon, Feb 16, 2015 at 04:37:34PM +0100, Carlos Garnacho wrote: > > These 2 requests have been added: > > > > - wl_data_source.notify_actions request: Notifies the compositor > > of the > > available actions on the data source. > > - wl_data_offer.notify_actions request: Notifies the compositor of > > the > > available actions on the destination side, plus the preferred > > action. > > > > Out of the data from these requests, the compositor can determine > > the action > > both parts agree on (and optionally let the user play a role > > through eg. > > keyboard modifiers). The chosen option will be notified to both > > parties > > through the following two requests: > > > > - wl_data_source.action > > - wl_data_offer.action > > > > Compared to the XDND protocol, there is one notable change: XDND > > lets the > > source suggest an action, whereas wl_data_device lets the > > destination > > prefer a given action. The difference is subtle here, it comes off > > as > > convenience because it is the drag destination which receives the > > motion > > events (unlike in X) and can perform action updates. > > > > The drag destination seems also in a better position to update the > > preferred action based on things like the data being transferred, > > the > > place being dropped, and whether the drag is client-local. > > > > Additionally, the wl_data_source.drop_performed and finished > > events will > > notify the source of the different termination phases of the DnD > > operation. > > > > Roughly based on previous work by Giulio Camuffo < > > giuliocamu...@gmail.com> > > --- > > protocol/wayland.xml | 97 > > +--- > > 1 file changed, 92 insertions(+), 5 deletions(-) > > > > diff --git a/protocol/wayland.xml b/protocol/wayland.xml > > index 2a49805..110804e 100644 > > --- a/protocol/wayland.xml > > +++ b/protocol/wayland.xml > > @@ -408,7 +408,7 @@ > > > > > > > > - > > + > > > >A wl_data_offer represents a piece of data offered for > > transfer > >by another client (the source client). It is used by the > > @@ -461,9 +461,37 @@ > > > > > > > > + > > + > > Watch your spaces and tabs, looks like there's a mix... > > > + > > + > > + This event notifies the offer of the action selected by the > > compositor. > > The phrasing of this first sentence seems cumbersome. Maybe: > > This event indicates the action selected by the compositor from > the > offered set of actions. > > > +The action will be determined after matching the options > > offered by the > > +source side (through data_source.set_actions) and the > > destination side > > +(through data_offer.notify_actions). > > + > > +This event can be emitted multiple times during the > > lifetime of a > > +data_offer, the most recent action received is always the > > valid one. > > + > > + > > + > > + > > + > > + > > + Sets the actions that the client supports for this operation. > > This > > +request may trigger a data_offer.action event if the > > compositor needs > > +changing the selected option after the destination-side > > change. > > + > > + Compositors wishing to stay compatible with earlier data_device > > versions > > + should set the "copy" action by default. > > Perhaps do you mean? > > "copy" action *as the* default > > > + > > + > > + > > + > > > > > > - > > + > > > >The wl_data_source object is the source side of a > > wl_data_offer. > >It is created by the source client in a data transfer and > > @@ -510,14 +538,61 @@ > > > > > > > > - This data source has been replaced by another data source. > > + This data source has been replaced by another data source, or > > + the drop operation finished, but resulted on no mimetype > > requested > > + through data_source.target or no action notified through > > data_source.action. > > The wording here is very confusing. Also I think you meant "in no" > rather than "on no". > > Perhaps try phrasing it, "This data source is no longer valid. There > are several reasons why this could happen: ..." > > >
Re: [PATCH] protocol: Add DnD actions
This is still bothering me as being much too complicated. I think a list of actions can be sent from the DnD source to the target, and the target selects one. There is no need for the compositor to do any intersection of the sets and there is no need to communicate the set of target actions to the compositor. Your concerns about shift state being handled by the source are misplaced. This would not change at all under what I am proposing, the source could still use the shift state to change the list of actions. My proposal is basically to take yours, and remove the ability for the target to send a set of actions that the compositor then interesect with the source list. Instead this "intersection" is done by the target, and the target sends *one* action (or "none") indicating the result of the intersection. Unless you want the compositor to draw user interface to allow the user to choose the action, which seems very much a bad idea, I cannot see what your proposal will allow to happen that this simplified version would not. I do believe any kind of popup (like a menu for choosing "move or copy") would have to be done by the target. This is because the target may have extra actions that the source does not care about or does not know about, such as "insert" verses "replace". The popup would grab the keyboard focus but when dismissed it may go back to a different client than the target. ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: [PATCH] protocol: Add DnD actions
On Mon, Feb 16, 2015 at 04:37:34PM +0100, Carlos Garnacho wrote: > These 2 requests have been added: > > - wl_data_source.notify_actions request: Notifies the compositor of the > available actions on the data source. > - wl_data_offer.notify_actions request: Notifies the compositor of the > available actions on the destination side, plus the preferred action. > > Out of the data from these requests, the compositor can determine the action > both parts agree on (and optionally let the user play a role through eg. > keyboard modifiers). The chosen option will be notified to both parties > through the following two requests: > > - wl_data_source.action > - wl_data_offer.action > > Compared to the XDND protocol, there is one notable change: XDND lets the > source suggest an action, whereas wl_data_device lets the destination > prefer a given action. The difference is subtle here, it comes off as > convenience because it is the drag destination which receives the motion > events (unlike in X) and can perform action updates. > > The drag destination seems also in a better position to update the > preferred action based on things like the data being transferred, the > place being dropped, and whether the drag is client-local. > > Additionally, the wl_data_source.drop_performed and finished events will > notify the source of the different termination phases of the DnD operation. > > Roughly based on previous work by Giulio Camuffo > --- > protocol/wayland.xml | 97 > +--- > 1 file changed, 92 insertions(+), 5 deletions(-) > > diff --git a/protocol/wayland.xml b/protocol/wayland.xml > index 2a49805..110804e 100644 > --- a/protocol/wayland.xml > +++ b/protocol/wayland.xml > @@ -408,7 +408,7 @@ > > > > - > + > >A wl_data_offer represents a piece of data offered for transfer >by another client (the source client). It is used by the > @@ -461,9 +461,37 @@ > > > > + > + Watch your spaces and tabs, looks like there's a mix... > + > + > + This event notifies the offer of the action selected by the compositor. The phrasing of this first sentence seems cumbersome. Maybe: This event indicates the action selected by the compositor from the offered set of actions. > +The action will be determined after matching the options offered by > the > +source side (through data_source.set_actions) and the destination > side > +(through data_offer.notify_actions). > + > +This event can be emitted multiple times during the lifetime of a > +data_offer, the most recent action received is always the valid one. > + > + > + > + > + > + > + Sets the actions that the client supports for this operation. This > +request may trigger a data_offer.action event if the compositor needs > +changing the selected option after the destination-side change. > + > + Compositors wishing to stay compatible with earlier data_device versions > + should set the "copy" action by default. Perhaps do you mean? "copy" action *as the* default > + > + > + > + > > > - > + > >The wl_data_source object is the source side of a wl_data_offer. >It is created by the source client in a data transfer and > @@ -510,14 +538,61 @@ > > > > - This data source has been replaced by another data source. > + This data source has been replaced by another data source, or > + the drop operation finished, but resulted on no mimetype requested > + through data_source.target or no action notified through > data_source.action. The wording here is very confusing. Also I think you meant "in no" rather than "on no". Perhaps try phrasing it, "This data source is no longer valid. There are several reasons why this could happen: ..." > The client should clean up and destroy this data source. > > > > + > + > + > + > + This event notifies the data_source of the action selected by the > +compositor. The action will be determined after matching the options as above > +offered by the source side (through data_source.set_actions) and the > +destination side (through data_offer.notify_actions). > + > +This event can be emitted multiple times during the lifetime of a > +data_offer, the most recent action received is always the valid one. > + > + > + > + > + > + > + Sets the actions the source client supports for this operation. This > +request may trigger a data_source.action event if the compositor > needs > +changing the selected option after the source-side change. needs *to change* the > + > + Clients are recommended to hardcode the "ask" action, so they can honor > + this action as the preferred by the destination side. Maybe:
Re: [PATCH] protocol: Add DnD actions
Hi Bill, On Mon, Feb 16, 2015 at 10:17 PM, Bill Spitzak wrote: > I would think the drop target client can figure out what action to perform, > there is no need to communicate it's set of actions to the compositor or for > the compositor to have any of this logic to "determine the action both parts > agree on". I unfortunately think such simple approach won't cover all usecases I'm intending to, most notably the user role in action selection. On XDND, it is the DnD source app which keeps keyboard/pointer grabs, and orchestrates all DnD message interaction with the DnD destination, all it must do there is updating the "suggested action" based on the currently pressed modifiers. On wayland, it is true that the semantics of compositor-side device grabs are mostly up to the compositor, but delegating this on the destination client means we need to start sending wl_keyboard.modifiers to random unfocused clients. Compositors may also want, for accessibility reasons, to let DnD to be driven by keyboard (eg. cursor keys), so you end up performing selective treatment of key events, I hope you see why I think this soon becomes irksome. With my proposed action system, in exchange for having the compositor play a minimal role in this (basically ANDing the options and picking one), we get the compositor doing all of this, and clients sticking to wl_data_[source|offer] events/requests, which I find more consistent. > > The drop target will have to say what action it did. It also sounds like you > are concerned that the source may want to know the action on drag events so > the source can update the cursor or redraw the source location to show the > result of the action. But this is only one action, not a list. Yes, the "action" events must have just one bit set, after the compositor picked the current action. The cursor icon would be something that changes during the DnD operation, say attempting a drop on local vs remote drives on the file manager (with different move vs copy preferred actions), or user modifications. > > So basically it seems like any lists from the drop target can be avoided. > The drop target would only send a single action. It could, if it were to know the available actions from the drag source. Given my concerns with the move/copy/ask semantics, I preferred to keep both isolated in that regard. Matching the actions in the compositor also gives a solid place where you can just emit wl_data_source.cancelled on failed drags, which is not as clear if the compositor is merely the carrier here. > > The compositor can throw a protocol error if the client sends an action that > is not in the list. Or it could just convey it right through and let the > drag source client ignore it. On this proposal, the drag source would recevice wl_data_source.action with 0, it may also want to set the pointer surface to a "forbidden" sign or somesuch. Carlos ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: [PATCH] protocol: Add DnD actions
I would think the drop target client can figure out what action to perform, there is no need to communicate it's set of actions to the compositor or for the compositor to have any of this logic to "determine the action both parts agree on". The drop target will have to say what action it did. It also sounds like you are concerned that the source may want to know the action on drag events so the source can update the cursor or redraw the source location to show the result of the action. But this is only one action, not a list. So basically it seems like any lists from the drop target can be avoided. The drop target would only send a single action. The compositor can throw a protocol error if the client sends an action that is not in the list. Or it could just convey it right through and let the drag source client ignore it. On 02/16/2015 07:37 AM, Carlos Garnacho wrote: These 2 requests have been added: - wl_data_source.notify_actions request: Notifies the compositor of the available actions on the data source. - wl_data_offer.notify_actions request: Notifies the compositor of the available actions on the destination side, plus the preferred action. Out of the data from these requests, the compositor can determine the action both parts agree on (and optionally let the user play a role through eg. keyboard modifiers). The chosen option will be notified to both parties through the following two requests: - wl_data_source.action - wl_data_offer.action ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
