Re: touchscreens in multiscreen setups

[Keith Packard]

On Thu, 08 Apr 2010 17:22:56 +0200, Peter Korsgaard  wrote:

> - Add an XInput property to evdev to assign a screen number / randr
>   output name to a device

Use a window ID to identify the screen; the protocol never uses screen
'numbers'.

> - Add an API to forward this info to the server (xf86XInputSetOutput?)

I don't know how this should work; feel free to try whatever strikes
you, including adding fields to the input device structure
itself. That might be easiest. Just a rectangle that identifies the
region of the screen that the touchscreen is aligned to may suffice,
along with a matrix to do the transforms.

> - Translate the raw coordinates in GetPointerEvents() if that flag is
>   set (POINTER_OUTPUT?), and optionally ignore event if output is
>   disabled.

Sounds reasonable.

-- 
keith.pack...@intel.com


pgp2dFEgva4OJ.pgp
Description: PGP signature
___
xorg@lists.freedesktop.org: X.Org support
Archives: http://lists.freedesktop.org/archives/xorg
Info: http://lists.freedesktop.org/mailman/listinfo/xorg

Re: touchscreens in multiscreen setups

[Peter Korsgaard]

> "Keith" == Keith Packard  writes:

Old thread, but I unfortunately couldn't work on it before now - The
issue is about binding touchscreen events to a randr output

Original thread is here:

http://thread.gmane.org/gmane.comp.freedesktop.xorg/40071

 Keith> It would need to be tied to an output, not a crtc. And, what
 Keith> we'd need to do is let the input device know which output it is
 Keith> associated with, and then provide a helper function to translate
 Keith> coordinates relative to that output. That could deal with output
 Keith> transforms as well. I don't see how xf86XInputSetScreen has much
 Keith> to do with this though, other than having that be used as a hint
 Keith> to go lookup the output property in the config file?

So just to make sure I get it all (this is the first time I hack on X) -
I'll need to:

- Add an XInput property to evdev to assign a screen number / randr
  output name to a device
- Add an API to forward this info to the server (xf86XInputSetOutput?)
- Translate the raw coordinates in GetPointerEvents() if that flag is
  set (POINTER_OUTPUT?), and optionally ignore event if output is disabled.

Correct?

 >> - randr notification. with screens being added and removed, there's no
 >> driver interface that I know of that input drivers can use to get notified
 >> about this stuff. Obviously the simple thing to do would be to handle this
 >> stuff in the server - if a screen goes away the device defaults back to
 >> whatever screen is still there. this would be without the driver knowing
 >> though - not ideal.

 Keith> It seems like xf86PostMotionEventP could deal with this internally,
 Keith> by having the device know which output it was associated with and having
 Keith> that code deal with finding the transform and doing the right
 Keith> thing. Input devices not associated with an output would not be
 Keith> transformed, and input devices associated with disabled outputs could
 Keith> either be ignored (?) or have their output untransformed.

-- 
Bye, Peter Korsgaard
___
xorg@lists.freedesktop.org: X.Org support
Archives: http://lists.freedesktop.org/archives/xorg
Info: http://lists.freedesktop.org/mailman/listinfo/xorg

Re: touchscreens in multiscreen setups

[David De La Harpe Golden]

2009/8/6 David De La Harpe Golden :
> 2009/8/3 Peter Korsgaard :
>> Now, with xrandr instead of Xinerama there is only one screen and the
>> above doesn't work (the coordinates gets scaled to the entire screen
>> instead of the individual outputs).
>>
>> What is the suggested solution for this? Add randr output tracking to
>> evtouch or should evdev handle this instead?
>>
>
> FWIW - I have a wacom tablet, I presently (ab)use the four wacom
> "calibration" properties to map input device' core pointer updating to
> particular screen subregions.   I keep meaning to wrap a friendly
> systray applet around it to do it on the fly.


FWIW, I've now released a small command line tool to automate this abuse
for wacom users:
http://harpegolden.net/misc/wacform/wacform-0.1.tgz

allowing one to write:

wacform stylus DFP2

to map wacom input device "stylus" to (you guessed it) xrandr output DFP2
(actually the relevant crtc's panning area, but anyway).

This is just intended as a temporary stopgap measure for wacom tablet
xrandr multihead users until randr and input play nicer together, independent of
any proper general solution X.org, linuxwacom etc. might be cooking up and
not endorsed by them  (that said, imo  there's something to be said for
the simplicity of the approach).  Take it or leave it...
___
xorg mailing list
xorg@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/xorg

Re: touchscreens in multiscreen setups

[Peter Korsgaard]

> "Peter" == Peter Hutterer  writes:

Hi,

 Peter> I don't think either is a particularly large undertaking, so
 Peter> it's worth tackling if you have the hardware and the test setups
 Peter> to actually do it.

Thanks - I do. I'm going on holidays from Tuesday on, but I'm back at
work where I have access to multiple NW (and other) touchscreens the
first week of January.

-- 
Bye, Peter Korsgaard
___
xorg mailing list
xorg@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/xorg

Re: touchscreens in multiscreen setups

[Alan Coopersmith]

Peter Hutterer wrote:
> - randr notification. with screens being added and removed, there's no
>   driver interface that I know of that input drivers can use to get notified
>   about this stuff.

A hack I used years ago (well before RandR 1.2 and XI2 - I haven't checked
in a long time if it still works) for handling scaling of mouse input events
from the absolute coordinate space reported by some of our remote KVM cards,
was to wrap pScrn->AdjustFrame - you can still see some of that in the
xf86-input-mouse driver in vuidMouseAdjustFrame() in src/sun_mouse.c.

Of course, that works because the remote KVM's have exactly one and only one
screen that the input device can be used with due to their nature, but that
would be the same as a touchscreen.

-- 
-Alan Coopersmith-   alan.coopersm...@sun.com
 Sun Microsystems, Inc. - X Window System Engineering

___
xorg mailing list
xorg@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/xorg

Re: touchscreens in multiscreen setups

[Daniel Stone]

Hi,

On Thu, Dec 17, 2009 at 10:35:16PM -0800, Keith Packard wrote:
> I'd say tablets and touch screens should report raw device coordinates
> and let the X server transform them as appropriate. It'd be really cool
> if we could make all absolute devices report position in floating point
> using a [0..1] range on each axis. I think. The X server would then
> transform those as appropriate. I think all that would take is a new
> PostMotionEvent function that took doubles instead of ints; fixed
> drivers would use that, broken drivers would continue to report
> coordinates as they do today.

Modulo floating point, this is how it's happened since approximately
forever ...

Cheers,
Daniel


pgpquH8RLKkPP.pgp
Description: PGP signature
___
xorg mailing list
xorg@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/xorg

Re: touchscreens in multiscreen setups

[Peter Korsgaard]

> "Keith" == Keith Packard  writes:

Hi,

 Keith> I'd say tablets and touch screens should report raw device coordinates
 Keith> and let the X server transform them as appropriate. It'd be really cool
 Keith> if we could make all absolute devices report position in floating point
 Keith> using a [0..1] range on each axis. I think. The X server would then
 Keith> transform those as appropriate. I think all that would take is a new
 Keith> PostMotionEvent function that took doubles instead of ints; fixed
 Keith> drivers would use that, broken drivers would continue to report
 Keith> coordinates as they do today.

Does this also include calibration? Some touchscreens apply calibration
parameters in firmware (like nextwindow), so the coordinates you receive
are already calibrated, whereas others provide raw coordinates and we
need to do calibration in software.

It would be nice to have generic calibration support in evdev, instead
of doing this different for each touchscreen. Related to this is
how/where to store calibration parameters. Notice that it sometimes is
needed to recalibrate at runtime (E.G. on resolution change if you don't
have your screen set for scale to fullscreen / don't keep aspect).

-- 
Bye, Peter Korsgaard
___
xorg mailing list
xorg@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/xorg

Re: touchscreens in multiscreen setups

[Matthieu Herrb]

On Thu, Dec 17, 2009 at 09:27:07PM -0800, Keith Packard wrote:
> On Fri, 18 Dec 2009 15:04:48 +1000, Peter Hutterer  
> wrote:
> 
> > there's two parts to it:
> > - xf86InputSetScreen seems broken, this should be fixed in the server.
> >   though it's probably optimised for the Xinerama scenario, I think it
> >   should be possible to set it up for randr output or CRTC tracking in a
> >   similar manner.
> 
> It would need to be tied to an output, not a crtc. And, what we'd need
> to do is let the input device know which output it is associated with,
> and then provide a helper function to translate coordinates relative to
> that output. That could deal with output transforms as well. I don't see
> how xf86XInputSetScreen has much to do with this though, other than
> having that be used as a hint to go lookup the output property in the
> config file?

For touchscreens you want to tie it to the particular output it's 
physically bound to in (almost) all cases. And in multi-head setups
you need another pointer for heads without associated touchscreens.

Tablets may be a bit different: you may want to use it as the only
input device, covering all the screen. (Although the geometry will
probably be awfully distorded). Other people, like graphic designers
will prefer to have the tablet restricted to the output bound to the
head where their main canvas is displayed and use a second input
driver to drive the 2nd head where the menus with all tools are
displayed. 

>From the configuration point of view, in most cases the underlaying
driver knows if it's a touchscreen or a tablet and can choose a
reasonable default, but the configuration should be modifiable on the
fly. 
-- 
Matthieu Herrb
___
xorg mailing list
xorg@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/xorg

Re: touchscreens in multiscreen setups

[Matthieu Herrb]

Hi,

On Thu, Dec 17, 2009 at 10:35:16PM -0800, Keith Packard wrote:
> From what I can see, there's no way of fixing the coordinate space issue
> with any driver that uses xf86XInputSetScreen today as they all convert
> From device to screen coordinates internally, before posting those
> events to the X server. And, they use cached versions of the screen size
> to make those transforms.

That's not completely true. XInput can take care of this (although it
probably suffers from the problems you mention too). I've worked on
the BSD specific wscons input driver recently to update it to use
modern XInput features  and it doesn't need the screen dimensions
anymore (except in one specific case where the calibration is done 
in the kernel, but this mode is currently not so great)



> 
> I'd say tablets and touch screens should report raw device coordinates
> and let the X server transform them as appropriate. It'd be really cool
> if we could make all absolute devices report position in floating point
> using a [0..1] range on each axis. I think. The X server would then
> transform those as appropriate. I think all that would take is a new
> PostMotionEvent function that took doubles instead of ints; fixed
> drivers would use that, broken drivers would continue to report
> coordinates as they do today.

Not really necessary: X Input has the min/max values for each axes and
can do the computation for the driver. But why not ?

> 
> > xf86InputSetScreen comes into play mostly when it comes to figuring out if
> > this feature is possible to bring back without breaking the ABI. If we can
> > re-use it in a sensible manner, great.
> 
> Yeah, I don't see it as useful; the drivers are using the target 
> screen size to scale their data. Not helpful.
> 
> > if you modify the coordinates, you break raw events though and lose
> > precision on the screen you're restricted to. I think this needs to be tied
> > to miPointerSetScreen and possibly the pointer accel code, but that includes
> > a fair deal of speculation as well.
> 
> I don't see how miPointerSetScreen relates; most X servers have only one
> screen today.

If we break the API, it doesn't matter. It just currently the only
callback to upper layer beside the post event one to tell the X server
to update any data related to the current state of the input driver. 




> ___
> xorg mailing list
> xorg@lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/xorg


-- 
Matthieu Herrb
___
xorg mailing list
xorg@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/xorg

Re: touchscreens in multiscreen setups

[Keith Packard]

On Fri, 18 Dec 2009 15:41:00 +1000, Peter Hutterer  
wrote:

> by default, devices cross screen boundaries and tablets/touchscreens map to
> the range of the screen. With xf86InputSetScreen a driver could be mapped
> to a single screen only, thus a right-hand corner would always be the
> right-hand corner on that screen, not on the right-hand corner of the
> right-most screen.

From what I can see, there's no way of fixing the coordinate space issue
with any driver that uses xf86XInputSetScreen today as they all convert
From device to screen coordinates internally, before posting those
events to the X server. And, they use cached versions of the screen size
to make those transforms.

I'd say tablets and touch screens should report raw device coordinates
and let the X server transform them as appropriate. It'd be really cool
if we could make all absolute devices report position in floating point
using a [0..1] range on each axis. I think. The X server would then
transform those as appropriate. I think all that would take is a new
PostMotionEvent function that took doubles instead of ints; fixed
drivers would use that, broken drivers would continue to report
coordinates as they do today.

> xf86InputSetScreen comes into play mostly when it comes to figuring out if
> this feature is possible to bring back without breaking the ABI. If we can
> re-use it in a sensible manner, great.

Yeah, I don't see it as useful; the drivers are using the target 
screen size to scale their data. Not helpful.

> if you modify the coordinates, you break raw events though and lose
> precision on the screen you're restricted to. I think this needs to be tied
> to miPointerSetScreen and possibly the pointer accel code, but that includes
> a fair deal of speculation as well.

I don't see how miPointerSetScreen relates; most X servers have only one
screen today.

-- 
keith.pack...@intel.com


pgp6LbzvIivhF.pgp
Description: PGP signature
___
xorg mailing list
xorg@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/xorg

Re: touchscreens in multiscreen setups

[Peter Hutterer]

On Thu, Dec 17, 2009 at 09:27:07PM -0800, Keith Packard wrote:
> On Fri, 18 Dec 2009 15:04:48 +1000, Peter Hutterer  
> wrote:
> 
> > there's two parts to it:
> > - xf86InputSetScreen seems broken, this should be fixed in the server.
> >   though it's probably optimised for the Xinerama scenario, I think it
> >   should be possible to set it up for randr output or CRTC tracking in a
> >   similar manner.
> 
> It would need to be tied to an output, not a crtc. And, what we'd need
> to do is let the input device know which output it is associated with,
> and then provide a helper function to translate coordinates relative to
> that output. That could deal with output transforms as well. I don't see
> how xf86XInputSetScreen has much to do with this though, other than
> having that be used as a hint to go lookup the output property in the
> config file?
> 

by default, devices cross screen boundaries and tablets/touchscreens map to
the range of the screen. With xf86InputSetScreen a driver could be mapped
to a single screen only, thus a right-hand corner would always be the
right-hand corner on that screen, not on the right-hand corner of the
right-most screen.

xf86InputSetScreen comes into play mostly when it comes to figuring out if
this feature is possible to bring back without breaking the ABI. If we can
re-use it in a sensible manner, great.

> > - randr notification. with screens being added and removed, there's no
> >   driver interface that I know of that input drivers can use to get notified
> >   about this stuff. Obviously the simple thing to do would be to handle this
> >   stuff in the server - if a screen goes away the device defaults back to
> >   whatever screen is still there. this would be without the driver knowing
> >   though - not ideal.
> 
> It seems like xf86PostMotionEventP could deal with this internally,
> by having the device know which output it was associated with and having
> that code deal with finding the transform and doing the right
> thing. Input devices not associated with an output would not be
> transformed, and input devices associated with disabled outputs could
> either be ignored (?) or have their output untransformed.

if you modify the coordinates, you break raw events though and lose
precision on the screen you're restricted to. I think this needs to be tied
to miPointerSetScreen and possibly the pointer accel code, but that includes
a fair deal of speculation as well.

Cheers,
  Peter
___
xorg mailing list
xorg@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/xorg

Re: touchscreens in multiscreen setups

[Keith Packard]

On Fri, 18 Dec 2009 15:04:48 +1000, Peter Hutterer  
wrote:

> there's two parts to it:
> - xf86InputSetScreen seems broken, this should be fixed in the server.
>   though it's probably optimised for the Xinerama scenario, I think it
>   should be possible to set it up for randr output or CRTC tracking in a
>   similar manner.

It would need to be tied to an output, not a crtc. And, what we'd need
to do is let the input device know which output it is associated with,
and then provide a helper function to translate coordinates relative to
that output. That could deal with output transforms as well. I don't see
how xf86XInputSetScreen has much to do with this though, other than
having that be used as a hint to go lookup the output property in the
config file?

> - randr notification. with screens being added and removed, there's no
>   driver interface that I know of that input drivers can use to get notified
>   about this stuff. Obviously the simple thing to do would be to handle this
>   stuff in the server - if a screen goes away the device defaults back to
>   whatever screen is still there. this would be without the driver knowing
>   though - not ideal.

It seems like xf86PostMotionEventP could deal with this internally,
by having the device know which output it was associated with and having
that code deal with finding the transform and doing the right
thing. Input devices not associated with an output would not be
transformed, and input devices associated with disabled outputs could
either be ignored (?) or have their output untransformed.

> I don't think either is a particularly large undertaking, so it's worth
> tackling if you have the hardware and the test setups to actually do
> it.

It seems like this could be isolated entirely within the xf86 code, not
touching either DIX or the device drivers.

All of this is fairly ill-informed speculation at this point; I haven't
dealt with Xinput at this level much.

-- 
keith.pack...@intel.com


pgpAqOSqxgOwH.pgp
Description: PGP signature
___
xorg mailing list
xorg@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/xorg

Re: touchscreens in multiscreen setups

[Peter Hutterer]

late reply, but better than never I suppose...

On Mon, Aug 03, 2009 at 03:23:39PM +0200, Peter Korsgaard wrote:
> How are touchscreens in multiscreen setups supposed to work nowadays?
> The issue with touchscreens is that their input events have to be
> transformed according to configuration of the screen they are
> physically connected to.
> 
> For a concrete example, I have a dual screen setup with nextwindow USB
> (HID) touchscreens. This used to work nicely with the evtouch driver,
> where you could configure each touchscreen to bind to a specific
> screen number in xorg.conf, which evtouch would then use to deliver
> the input events transformed to that screen.
> 
> Now, with xrandr instead of Xinerama there is only one screen and the
> above doesn't work (the coordinates gets scaled to the entire screen
> instead of the individual outputs).
> 
> What is the suggested solution for this? Add randr output tracking to
> evtouch or should evdev handle this instead?

there's two parts to it:
- xf86InputSetScreen seems broken, this should be fixed in the server.
  though it's probably optimised for the Xinerama scenario, I think it
  should be possible to set it up for randr output or CRTC tracking in a
  similar manner.
- randr notification. with screens being added and removed, there's no
  driver interface that I know of that input drivers can use to get notified
  about this stuff. Obviously the simple thing to do would be to handle this
  stuff in the server - if a screen goes away the device defaults back to
  whatever screen is still there. this would be without the driver knowing
  though - not ideal.

I don't think either is a particularly large undertaking, so it's worth
tackling if you have the hardware and the test setups to actually do it.

Keith, any comments to either one?

Cheers,
  Peter
___
xorg mailing list
xorg@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/xorg

Re: touchscreens in multiscreen setups

[David De La Harpe Golden]

2009/8/3 Peter Korsgaard :
> Now, with xrandr instead of Xinerama there is only one screen and the
> above doesn't work (the coordinates gets scaled to the entire screen
> instead of the individual outputs).
>
> What is the suggested solution for this? Add randr output tracking to
> evtouch or should evdev handle this instead?
>

FWIW - I have a wacom tablet, I presently (ab)use the four wacom
"calibration" properties to map input device' core pointer updating to
particular screen subregions.   I keep meaning to wrap a friendly
systray applet around it to do it on the fly.  I've never used
evtouch, a similar trick might work there though:

Basically, if a driver has a wacom-like facility, where rectilinear
calibration uses TopX/TopY/BottomX/BottomY  coords in device space and
 allows large positive and negative values for them, you can (ab)use
them to map the tablet's zero-to-m/zero-to-n xy area in device space
to any rectangular subarea of your screen in pixel space if you want.
Make them correspond to subarea that aligns with the output you want
it to align with, and you're done.   The basic trick is to set the
TopX/TopY to the top-left of the X11 screen ...in tablet-relative
device coordinates! (picturing the tablet area laid atop the screen
area), and similar for the BottomX/BottomY and the bottom-right of the
X11 screen.  (If you actually need some actual calibration too, well
the maths gets trickier)

But it could even be that input devices offering those four parameters
is all that's necessary for an adequate solution for absolute input
devices and xrandr multihead, so long as you don't mind roundtripping
on randr reconfigurations (the systray thingy would need to watch for
reconfigurations, introspect the new configuration, and recalculate
the appropriate device calibration parameters).

e.g. Here's what I use to statically align wacom input devices of a
1280x1024 screen /  7220x5780 device cintiq below a 1600x1200 screen:

Option  "TopX"  "0"
Option  "TopY"  "-6773"
Option  "BottomX"   "9025"
Option  "BottomY"   "5780"

Note the huge negative value for TopY - that's so that the driver is
"fooled" into thinking the tablet covers the whole X11 screen with a Y
axis from -6773 to 5780, and since the X11 screen is a rectangle and I
have a dead area to the right of the 1280x1024 screen, I also need to
account for that with the BottomX value.
5780 * 1200 / 1024  = 6773 (ish)
7220 * 1600 / 1280  = 9025

This is actually a similar trick  to the "lockstep multihead
proportional panning" trick outlined in the proportional panning patch
I sent a bit back (that I do mean to do a revised version of... just
working at glacial pace...)
___
xorg mailing list
xorg@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/xorg