On Mon, Mar 01, 2010 at 02:41:51PM +0200, Daniel Stone wrote: > On Mon, Mar 01, 2010 at 12:42:40PM +0100, Bradley T. Hughes wrote: > > On 03/01/2010 12:22 PM, ext Daniel Stone wrote: > >> (What happens in the hybrid system when I get an event from finger 1, > >> decide I like it, take out a grab, and then finger 2 presses on another > >> window. Do I respect the event and give the app the finger 2 press it > >> likely doesn't want, or break the grab and deliver it to another client? > >> Neither answer is pleasant.) > > > > But what happens in the sub-device system? Finger 1's event goes via it's > > MD to the client under the finger, but what about the second finger if > > it's over another client's window? Does it go to the client that got > > finger1 since it's on the same MD (which precludes the multi-user use > > case)? > > You just extend the MD/SD semantics down, so if you have: > MD -> Touchscreen -> Fingers [123...], then: > -> Mouse > then: > * grab on finger 1 results in only finger 1 being grabbed, fingers > 2345 and the mouse are delivered as normal to any client > * grab on the ts results in all fingers being grabbed, but the mouse > being delivered as normal > * grab on the MD results in everything being grabbed > * grab on finger 1 by client A followed by a grab on the MD by client > B results in finger 1 being delivered to client A, and all other > fingers and the mouse being delivered to client B > > So, you just take the existing MD/SD semantics and extend them to n > levels instead of 2. Just walk up the list of parents looking for > grabs/interest, stopping at the first one you find. > > Peter, correct me if I'm missing something blindingly obvious, or wholly > non-obvious. :)
Sure. Photo of whiteboard brainstorm is available on the link below, it's the closest thing I have to a diagram. http://people.freedesktop.org/~whot/patches/multitouch-brainstorm-03.03.2010.jpg My notes to this approach below, sorry about the unedited format, they're just what I jotted down right now but I figured the full train of thought is still better than just short answers. direct answers to your approach are at the end of the email. Spoiler alert: approach breaks down on keyboards during core grabs. Note that I'm _not_ trying to solve the synaptics multi-finger swipe case, this one is easy and could be solved by packing the touchpoints into the event as additional valuators. I'm trying to solve the case where one or more users interact with one or more apps on the same screen (core, XI, XI2, multi-touch, or a combination thereof). Basic premise: Data from devices comes in terms of Press/Motion/Release from fingers. These can be grouped, some way or another. If it can't be ID'd, we've already lost anyway. We have one physical device (the touchsceen) and multiple input points on that physical devices (call them FDs for FingerDevices). All FDs come through the same SD and MD. Assuming we can make the cost of creating an FD zero, we can do so on the fly as touchpoints show up. They are not exposed to core or XI clients. So in the default event delivery, you'd have FD - SD - MD, which is great for synaptics gestures since you get all the data you want. But for multi-touch core, you need multiple MDs. So what about FDs sending core, XI and XI2 events? This is justified since a FD is a direct interaction device and the distinction between physical device and virtual device doesn't exist. (This distinction led to the creation of the MD/SD hierarchy in the first place) Hierarchy would then be: core event MD -/|| | || SD --/| | | FD ---/ (this diagram best viewed in monospace) Got to be careful though that the same core event isn't delivered twice from different devices - may cause doubleclick problem. XI is severely limited if FDs are shortlived, since the device is likely to disappear before the register event comes in - if IDs are reused, the mask may refer to a different device that has since disappeared and reappeared → make devices long-lived and re-usable for fingers. Only propagate core and XI event if the event was not delivered. XI2 events still need to propagate (what to do with the source_id??) So test case is: one touchdevice, press one finger. Init state: MD → SD touch state MD → SD → FD FD sticks around until device is removed (??). After touch, hierarchy event about FD (how will that look like??). Core event propagated through FD or MD (SDs don't do core), XI event are propagated through FD or SD (MDs don't do XI). XI2 event through all three (XI2 slave grab - double event??). Assuming core event delivered, grab comes in from client. What device to grab? • MD - looses all multitouch if all fingers are grabbed • FD - seems like sanest solution, but might be tricky as fuck with thaw&replay (MPX semantics take care of this??) • Grab FD if ClientPointer is on MD, let other fingers go through MD but only for non-interfering grabs. i.e. a core grab grabs through the MD. Could be awkward if the FD disappears and the next FD with the same id (still grabbed) then untriggers the grab. Still the most sensible solution. this is not an issue for passive grabs, they switch ownership dynamically What about the keyboard? If one finger is grabbed as sync - freeze the MD's keyboard? Where do keyboard events go if two fingers are grabbed? Having two pointer grabs is easy, but if both grabs wait for a keyboard event, then you're screwed. You'd have to flip the keyboard over to the second grab when the first exists but. DEADLOCK, a client might wait for the key event to happen. UI ISSUE - user doesn't know where keyboard is Also, we don't know if a grab comes in response to a FD device or an SD-generated core event on the MD. so we don't know whether to grab the FD or the MD from a core client. > * grab on finger 1 results in only finger 1 being grabbed, fingers > 2345 and the mouse are delivered as normal to any client "grab on finger" isn't defined in your example. Does this mean a specific XI2 call? We still want core events (well, we do otherwise we might as well ignore all issues and create our own custom API and ignore the rest) With core, you cannot know when "Finger 1" is supposed to be grabbed. example: touch device and mouse on the same MD, you use your finger, then the mouse button, grab comes in. Do you grab the finger? or the mouse (and therefore the MD?) if you grab the MD for all fingers, then you lose multitouch whenever a grab is active. even if you would know when to grab the finger - once you have finger 1, finger 2 and the MD grabbed by three clients, where do you send key events to? If all three are a sync grab (i.e. waiting for some key combo) you have a nice confusing half-deadlock. > * grab on the ts results in all fingers being grabbed, but the mouse > being delivered as normal > * grab on the MD results in everything being grabbed you've just lost the ability to click on anything because some other client has a menu open. So you're down to single-client multitouch. > * grab on finger 1 by client A followed by a grab on the MD by client > B results in finger 1 being delivered to client A, and all other > fingers and the mouse being delivered to client B that's what we already do in MPX with the temporary detachment of slave devices on a direct grab. We can't do so for XI1 devices though, so that'll likely get tricky on the details again. Cheers, Peter _______________________________________________ xorg-devel mailing list xorg-devel@lists.x.org http://lists.x.org/mailman/listinfo/xorg-devel