Hi Sergey,
The absolute deltas are /optional /and provided only when they are
available, so there's no need to generate these values on all platforms,
for all events. For example, mouse wheel doesn't generate absolute
deltas, because a wheel step naturally corresponds to a line (or several
discrete lines). We can use 0.0 value to indicate the absence of
absolute delta (so, if some platform doesn't provide absolute deltas,
0.0 is a valid value, and the platform is still "supported").
The Apple's scrollingDeltaY description tells
<https://developer.apple.com/reference/appkit/nsevent/1535387-scrollingdeltay>:
"When hasPreciseScrollingDeltas is false, /multiply /the value returned
by this method by the line or row height. Otherwise /scroll by /the
returned amount". That particular API uses a /single /variable to hold
both relative- and absolute deltas, together with the additional flag
that tells how to interpret the value (i.e. when
hasPreciseScrollingDeltas is false, scrollingDeltaY contains deltaY).
It's just a slightly different way to represent the two kinds of data,
using a separate boolean flag, instead of the 0.0 marker (i.e. it puts
the two disparate values into a single variable and then provides a way
to distinguish them).
Apple doesn't explicitly specify units for the absolute deltas (it only
tells to "scroll /by/ the returned amount"). The Windows' Direct
Manipulation API states that the absolute transformations are given in
device-independent pixels. It seems that we can also specify
"device-independent pixels" explicitly.
There's no reliable way to convert absolute deltas to relative deltas,
because the relative deltas are /relative /to arbitrary scrolling unit
size, which is not a part of input, but a part of the content rendering
(which might not even exist). OS generates the two kinds of data
separately, and both spatial- and temporal resolutions differ
considerably (so, an attempt to somehow channel the absolute delta into
the already existing, relative one, can potentially raise compatibility
/ performance problems).
We have /already /implemented "smooth scrolling" in IDEA itself. One
doesn't even need high-precision input to render the transitions
smoothly, yet high-resolution input is useful to increase the
positioning precision, to transfer the dynamics better and to reduce the
input latency.
Besides, there's no much sense in trying to "squeeze" the absolute delta
into the relative one as "a temporary solution" – just as well, we can
temporary add the absolute deltas to JetBrains Runtime directly, which
seems to be a more reliable and a better-performing solution that really
delivers Mac-like scrolling feel.
The central purpose of the proposal is not to create some temporary
workaround for a particular task, but rather to enhance and improve the
Java itself, so it can be up-to-date with the state of modern hardware /
OSes. If JDK 9 is already frozen, I hope that the idea can still be
useful for the next versions.
On 11.01.2017 16:18, Sergey Bylokhov wrote:
H, Pavel.
Thanks for this proposal.
First of all if you have a plan to implement smooth scrolling in the
jdk9, then it will be good to use only existing API. (We already had a
the feature freeze).
So for now a solution will be better to implement the fix via
getPreciseWheelRotation
<https://docs.oracle.com/javase/8/docs/api/java/awt/event/MouseWheelEvent.html#getPreciseWheelRotation-->().
At least in this case you will be able to implement it only on
Mac/win, but if you provide the new API then it should be supported on
all platforms.
Is it possible to convert absolute data to the deltas? What is the
difference between absolute data and deltas? The main problem which I
see here is that we have no strict definition of units which we use
for scrolling(ticks, pixels, lines or 2-pixels on the retina etc).
Hi All,
Next follows a proposal on enhancingMouseWheelEventto include
absolute scrolling delta.
My name is Pavel Fatin, I work at JetBrains (IntelliJ IDEA). Although
this is my first message here, I have been working with Java / AWT /
Swing since Java 1.2 and have some in-depth experience (for example,
my research onlow-latency painting in AWT and Swing
<https://pavelfatin.com/low-latency-painting-in-awt-and-swing/>).
Because we at JetBrains receive manyrequests
<https://youtrack.jetbrains.com/issue/IDEA-76396>to implement
smooth-scrolling in IDEA, as an experiment, weextended
<https://github.com/JetBrains/intellij-community/commit/34b9dfd0585937c3731e06a89554d1dc86f7f235>our
custom scroll pane class to
handleMouseWheelEvent'sgetPreciseWheelRotation()
<https://docs.oracle.com/javase/8/docs/api/java/awt/event/MouseWheelEvent.html#getPreciseWheelRotation-->data
(that method was introduced in Java 7, but it’s not yet used by AWT /
Swing directly). As a result, the scrolling "smoothness" and
precision improved substantially, however on OS X it was still a far
cry from the native scrolling experience, particularly:
*
Scrolling speed was unit-dependent (while in OS X it's not). The
concept of "wheel rotation" is inherently tied to the idea of
"scrolling unit" (usually 1 or 3 lines). This makes sense for a
real mouse scrolling wheel, but doesn't make much sense for a
touchpad – one expects a uniform mapping between physical touch
gesture and a screen update, consistent across different
applications. The same goes for touchscreen interfaces.
Introduction of the fractional wheel rotations increases
precision but still retains the "rotation-unit" bound.
*
Spatial precision was lacking. Typical scrolling deltas were in
range 0.5..>10, and this was only a slight improvement over
1..>10 deltas.
*
Temporal resolution was lacking, so the scrolling was rather
coarse-grained and there was a substantial lag between starting a
gesture and a reaction on the screen. This mostly stems from the
insufficient spatial resolution. Added interpolation helped to
artificially increase spatial- and temporal resolutions, but
couldn't solve the input lag and the lack of positioning precision.
Wondering how the native OS X applications can perform much better we
started to investigate the Cocoa API. It turns out, that since Mac OS
X 10.7 (Lion)NSEvent
<https://developer.apple.com/reference/appkit/nsevent>, in addition
to thedeltaX
<https://developer.apple.com/reference/appkit/nsevent/1534871-deltax>anddeltaY
<https://developer.apple.com/reference/appkit/nsevent/1534158-deltay>introducedscrollingDeltaX
<https://developer.apple.com/reference/appkit/nsevent/1524505-scrollingdeltax>andscrollingDeltaY
<https://developer.apple.com/reference/appkit/nsevent/1535387-scrollingdeltay>properties
that supply high-resolution scrolling information
(withhasPrecisionScrollingDeltas
<https://developer.apple.com/reference/appkit/nsevent/1525758-hasprecisescrollingdeltas>property
that specifies how to interpret the data). So far so good, but those
properties are quite a different beast – they
contain/absolute/scrolling deltas rather than relative deltas,
supplied by the previous values.
Can you please clarify this. Its is unclear from the specification
what is the difference between deltaX,
scrollingDeltaX(hasPrecisionScrollingDeltas=true),
scrollingDeltaX(hasPrecisionScrollingDeltas=false).
How can we integrate such values in the existing Java API?
One way is to try to somehow translate the absolute deltas to
relative "wheel rotations", but the wheel event per se represents
input data and doesn't deal with possible scrolling units as such.
Another problem with this approach is that OS might provide so-called
"pixel-perfect" scrolling, where smallest delta is guaranteed to be 1
pixel (yet still preserving large-distance scrolling via OS-level
acceleration), as OS knows real physical resolution of particular
input / output devices and the acceleration curve, and such a
translation cannot reliably handle this. And one more problem with
the masking is compatibility – because the new data have much more
temporal resolution, existing applications that process precise wheel
events synchronously might be overwhelmed, hog CPU and lag
unnecessarily. All in all, "partial wheel rotation" is an imperfect
and "leaky" abstraction for the absolute scrolling delta.
A possible "hack" is to put the absolute values directly into the
relative delta property and then to hardcode unit size to be 1 on Mac
OS (that is what GTK+ 3does
<https://github.com/GNOME/gtk/blob/master/gtk/gtkscrolledwindow.c#L1245>),
but that is what it is – a hack, not compatible with the existing
codebase.
It seems, that the most reasonable solution is to introduce a new,
"absolute" kind of scrolling deltas, in addition to the relative one.
That is, for example, whatQWheelEvent <https://goo.gl/kZqxVc>does
with itsangleDelta <https://goo.gl/zwxBt9>andpixelDelta
<https://goo.gl/F5A37V>properties. We tried that approach in our
JetBrains Runtime byextending
<https://github.com/JetBrains/jdk8u_jdk/commit/568f2dae82b0fe27b79ce6943071d89463758610>MouseWheelEventwithgetScrollingDelta()method
andimplementing
<https://github.com/JetBrains/jdk8u_jdk/commit/a3cb8807b148879e9c70a74a8a16c30a28991581>it
in Mac OS – the result is a genuine, OSX-like scrolling experience in
Java.
We chose the method name “getScrollingDelta” (a la Apple) instead of
“getPixelDelta” (a la Qt) because the input event by itself doesn't
necessarily results in display scrolling, and because the concept of
"pixel" is not that precise. Likewise, we chosedoubletype for further
extensibility, considering the case withgetWheelDelta()(NSEventalso
uses a floating point value).
It should be noted, that the absolute delta/complements/rather than
replaces the relative value. Those two values are not interchangeable
and OS generates the values separately (for legacy devices only the
relative delta is generated and, in such cases, we can use 0 as an
absolute one). Addition of the absolute scrolling delta fully
preserves the functioning of Java's existing event processing mechanisms.
Although currently only Mac OS supplies that kind of data, Windows
Precision Touchpad and XInput2 / libinput might follow, as smooth
scrolling becomes a thing nowadays. Additionally, touchscreen-based
scrolling needs precisely this kind of scrolling deltas.
While we may implement this functionality privately in our JetBrains
Runtime, we believe that more people can benefit from it. Moreover,
the enhancement ofMouseWheelEventto support pixel-perfect scrolling
can pave a road for smooth scrolling in AWT and Swing (which is a
good thing to improve usability, both GTK+ 3 and Qt 5 already support
that).
--
Sincerely, Pavel
--
Sincerely, Pavel
