Re: Running mousemove events from the refresh driver

[Neil]

Robert O'Callahan wrote:


WM_MOUSEMOVE events, like other native events, would still take priority over 
all Gecko events and potentially starve the refresh driver.
 

Sorry, I misread the code, I didn't realise that it only prioritised 
keyboard and mouse events over other native events.


--
Warning: May contain traces of nuts.
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Running mousemove events from the refresh driver

[Robert O'Callahan]

On Sun, Feb 24, 2013 at 1:41 PM, Neil  wrote:

> Robert O'Callahan wrote:
>
>  I suppose we could try ignoring WM_MOUSE_MOVEs when there's a Gecko event
>> pending, but that sounds kinda scary. I think deferring DOM mousemove
>> events to the next refresh driver tick would be safer than that.
>>
>>
> Currently we peek for WM_MOUSEFIRST to WM_MOUSELAST, where conveniently
> WM_MOUSEFIRST == WM_MOUSEMOVE, so couldn't we exclude WM_MOUSEMOVE from the
> native events that we check for? Then they would get picked up when the
> system wasn't so busy.


WM_MOUSEMOVE events, like other native events, would still take priority
over all Gecko events and potentially starve the refresh driver.

Rob
-- 
Wrfhf pnyyrq gurz gbtrgure naq fnvq, “Lbh xabj gung gur ehyref bs gur
Tragvyrf ybeq vg bire gurz, naq gurve uvtu bssvpvnyf rkrepvfr nhgubevgl
bire gurz. Abg fb jvgu lbh. Vafgrnq, jubrire jnagf gb orpbzr terng nzbat
lbh zhfg or lbhe freinag, naq jubrire jnagf gb or svefg zhfg or lbhe fynir
— whfg nf gur Fba bs Zna qvq abg pbzr gb or freirq, ohg gb freir, naq gb
tvir uvf yvsr nf n enafbz sbe znal.” [Znggurj 20:25-28]
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Running mousemove events from the refresh driver

[Neil]

Robert O'Callahan wrote:


I suppose we could try ignoring WM_MOUSE_MOVEs when there's a Gecko event 
pending, but that sounds kinda scary. I think deferring DOM mousemove events to 
the next refresh driver tick would be safer than that.
 

Currently we peek for WM_MOUSEFIRST to WM_MOUSELAST, where conveniently 
WM_MOUSEFIRST == WM_MOUSEMOVE, so couldn't we exclude WM_MOUSEMOVE from 
the native events that we check for? Then they would get picked up when 
the system wasn't so busy.


--
Warning: May contain traces of nuts.
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Running mousemove events from the refresh driver

[Robert O'Callahan]

On Sat, Feb 23, 2013 at 5:37 PM, Jonas Sicking  wrote:

> Well, my proposal was to default all pages to type 1, and only send them
> mouse events at 60 frames per second.
>
> And then let them opt in to being type 2.
>
> This should be fine since I strongly suspect that type 1 is the by far
> more common case and is also what chrome does.
>

I don't think we can count on all sites (especially existing sites) that
could benefit from a higher rate of mouse events opting into type 2.
Choosing automatic anti-flooding will let us be better than Chrome on those
sites. That's good.

Rob
-- 
Wrfhf pnyyrq gurz gbtrgure naq fnvq, “Lbh xabj gung gur ehyref bs gur
Tragvyrf ybeq vg bire gurz, naq gurve uvtu bssvpvnyf rkrepvfr nhgubevgl
bire gurz. Abg fb jvgu lbh. Vafgrnq, jubrire jnagf gb orpbzr terng nzbat
lbh zhfg or lbhe freinag, naq jubrire jnagf gb or svefg zhfg or lbhe fynir
— whfg nf gur Fba bs Zna qvq abg pbzr gb or freirq, ohg gb freir, naq gb
tvir uvf yvsr nf n enafbz sbe znal.” [Znggurj 20:25-28]
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Running mousemove events from the refresh driver

[Jonas Sicking]

On Feb 21, 2013 3:12 PM, "Robert O'Callahan" 
wrote:
>
> On Wed, Feb 20, 2013 at 9:41 AM, Anthony Jones  wrote:
>
> > We really have to choices:
> > A. Provide an API that allows applications to specify whether they are
> > type 1 or type 2. It could be implicitly done by including a mouse event
> > history array.
> > B. Automatically prevent flooding (as per roc's suggestion)
> > C. Both of the above.
> >
> > Option A fixes most cases but adds more complexity to the API (although
> > not much). Poorly written applications or slow systems may still end up
> > flooding.
> >
> > Option B will mostly fix the problem but will waste CPU cycles if your
> > application is type 1.
> >
>
> We pretty much have to do something automatic, so I think we do B, and
then
> if and when it's worthwhile, A.

Well, my proposal was to default all pages to type 1, and only send them
mouse events at 60 frames per second.

And then let them opt in to being type 2.

This should be fine since I strongly suspect that type 1 is the by far more
common case and is also what chrome does.

In this case just doing A would be enough, no?

Though I guess even then we might need to do the anti-flooding thing for
websites that opt in to being type 2?

/ Jonas
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Running mousemove events from the refresh driver

[Robert O'Callahan]

On Wed, Feb 20, 2013 at 9:41 AM, Anthony Jones  wrote:

> We really have to choices:
> A. Provide an API that allows applications to specify whether they are
> type 1 or type 2. It could be implicitly done by including a mouse event
> history array.
> B. Automatically prevent flooding (as per roc's suggestion)
> C. Both of the above.
>
> Option A fixes most cases but adds more complexity to the API (although
> not much). Poorly written applications or slow systems may still end up
> flooding.
>
> Option B will mostly fix the problem but will waste CPU cycles if your
> application is type 1.
>

We pretty much have to do something automatic, so I think we do B, and then
if and when it's worthwhile, A.

Rob
-- 
Wrfhf pnyyrq gurz gbtrgure naq fnvq, “Lbh xabj gung gur ehyref bs gur
Tragvyrf ybeq vg bire gurz, naq gurve uvtu bssvpvnyf rkrepvfr nhgubevgl
bire gurz. Abg fb jvgu lbh. Vafgrnq, jubrire jnagf gb orpbzr terng nzbat
lbh zhfg or lbhe freinag, naq jubrire jnagf gb or svefg zhfg or lbhe fynir
— whfg nf gur Fba bs Zna qvq abg pbzr gb or freirq, ohg gb freir, naq gb
tvir uvf yvsr nf n enafbz sbe znal.” [Znggurj 20:25-28]
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Running mousemove events from the refresh driver

[Anthony Jones]

On 19/02/13 23:52, Robert O'Callahan wrote:
> I think my idea for an anti-flood state makes this unnecessary.

There are two distinct use cases:
1. You only care where the mouse currently is.
2. You need to know both where the mouse is and how it got there.

We really have to choices:
A. Provide an API that allows applications to specify whether they are
type 1 or type 2. It could be implicitly done by including a mouse event
history array.
B. Automatically prevent flooding (as per roc's suggestion)
C. Both of the above.

Option A fixes most cases but adds more complexity to the API (although
not much). Poorly written applications or slow systems may still end up
flooding.

Option B will mostly fix the problem but will waste CPU cycles if your
application is type 1.
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Running mousemove events from the refresh driver

[Robert O'Callahan]

On Tue, Feb 19, 2013 at 7:24 PM, Jonas Sicking  wrote:

> But I would expect that in the majority of other cases, each mousemove
> event will not leave persisted data and dispatching more mouse moves
> will simply mean that the page will redo the same calculations over
> and over only. The result is that at best 60 results per second being
> used to affect what's drawn on screen.
>
> One possible solution is to allow pages to opt in to high-precision
> mousemove events. Then a drawing program could do that on the
> mousedown event end opt out again on mouseup. Or if it's really lazy
> opt in for the lifetime of the page. Something like this would have to
> be coordinated with other vendors of course. Given that it looks like
> Chrome is throtteling to 60 mousemove events per second, I would
> expect them to be interested in working on such an API.
>

I think my idea for an anti-flood state makes this unnecessary.

Rob
-- 
Wrfhf pnyyrq gurz gbtrgure naq fnvq, “Lbh xabj gung gur ehyref bs gur
Tragvyrf ybeq vg bire gurz, naq gurve uvtu bssvpvnyf rkrepvfr nhgubevgl
bire gurz. Abg fb jvgu lbh. Vafgrnq, jubrire jnagf gb orpbzr terng nzbat
lbh zhfg or lbhe freinag, naq jubrire jnagf gb or svefg zhfg or lbhe fynir
— whfg nf gur Fba bs Zna qvq abg pbzr gb or freirq, ohg gb freir, naq gb
tvir uvf yvsr nf n enafbz sbe znal.” [Znggurj 20:25-28]
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Running mousemove events from the refresh driver

[Robert O'Callahan]

On Tue, Feb 19, 2013 at 8:42 PM, Justin Dolske  wrote:

> On 2/18/13 10:24 PM, Jonas Sicking wrote:
>
>  One possible solution is to allow pages to opt in to high-precision
>> mousemove events. Then a drawing program could do that on the
>> mousedown event end opt out again on mouseup.
>>
>
> Hmm, we could even do a bit of this today without an opt-in -- throttle
> events unless you're in a mousedown state. (Or throttle less when in a
> mousedown state.) That would, roughly, make existing drawing apps work well
> while also making the common case (not moving while holding a button)
> faster.
>

Unfortunately bug 837985, the motivator here, happens when the button is
down.

Rob
-- 
Wrfhf pnyyrq gurz gbtrgure naq fnvq, “Lbh xabj gung gur ehyref bs gur
Tragvyrf ybeq vg bire gurz, naq gurve uvtu bssvpvnyf rkrepvfr nhgubevgl
bire gurz. Abg fb jvgu lbh. Vafgrnq, jubrire jnagf gb orpbzr terng nzbat
lbh zhfg or lbhe freinag, naq jubrire jnagf gb or svefg zhfg or lbhe fynir
— whfg nf gur Fba bs Zna qvq abg pbzr gb or freirq, ohg gb freir, naq gb
tvir uvf yvsr nf n enafbz sbe znal.” [Znggurj 20:25-28]
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Running mousemove events from the refresh driver

[Justin Dolske]

On 2/18/13 10:24 PM, Jonas Sicking wrote:


One possible solution is to allow pages to opt in to high-precision
mousemove events. Then a drawing program could do that on the
mousedown event end opt out again on mouseup.


Hmm, we could even do a bit of this today without an opt-in -- throttle 
events unless you're in a mousedown state. (Or throttle less when in a 
mousedown state.) That would, roughly, make existing drawing apps work 
well while also making the common case (not moving while holding a 
button) faster.


Just a thought.

Justin
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Running mousemove events from the refresh driver

[Jonas Sicking]

On Mon, Feb 18, 2013 at 4:51 PM, Robert O'Callahan  wrote:
> On Tue, Feb 19, 2013 at 1:40 PM, Brian Birtles  wrote:
>
>> I'm not sure if this is a relevant data point but we had an issue[1] with
>> touch event coalescing on fennec that produced poor results for the
>> following drawing application on some devices such as the Dell Streak:
>>
>>   
>> http://parapara-editor.**mozlabs.jp/test
>>
>> This application uses splines to interpolate between the points but still
>> the lack of events produced poor results. It was fixed by turning off touch
>> event coalescing. See bug 757680.
>>
>
> Thanks, that's helpful.

For drawing programs it does indeed seem true that the more mousemove
events that we dispatch, the better the user experience will be.

But I would expect that in the majority of other cases, each mousemove
event will not leave persisted data and dispatching more mouse moves
will simply mean that the page will redo the same calculations over
and over only. The result is that at best 60 results per second being
used to affect what's drawn on screen.

One possible solution is to allow pages to opt in to high-precision
mousemove events. Then a drawing program could do that on the
mousedown event end opt out again on mouseup. Or if it's really lazy
opt in for the lifetime of the page. Something like this would have to
be coordinated with other vendors of course. Given that it looks like
Chrome is throtteling to 60 mousemove events per second, I would
expect them to be interested in working on such an API.

/ Jonas
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Running mousemove events from the refresh driver

[Robert O'Callahan]

On Tue, Feb 19, 2013 at 1:40 PM, Brian Birtles  wrote:

> I'm not sure if this is a relevant data point but we had an issue[1] with
> touch event coalescing on fennec that produced poor results for the
> following drawing application on some devices such as the Dell Streak:
>
>   
> http://parapara-editor.**mozlabs.jp/test
>
> This application uses splines to interpolate between the points but still
> the lack of events produced poor results. It was fixed by turning off touch
> event coalescing. See bug 757680.
>

Thanks, that's helpful.

Rob
-- 
Wrfhf pnyyrq gurz gbtrgure naq fnvq, “Lbh xabj gung gur ehyref bs gur
Tragvyrf ybeq vg bire gurz, naq gurve uvtu bssvpvnyf rkrepvfr nhgubevgl
bire gurz. Abg fb jvgu lbh. Vafgrnq, jubrire jnagf gb orpbzr terng nzbat
lbh zhfg or lbhe freinag, naq jubrire jnagf gb or svefg zhfg or lbhe fynir
— whfg nf gur Fba bs Zna qvq abg pbzr gb or freirq, ohg gb freir, naq gb
tvir uvf yvsr nf n enafbz sbe znal.” [Znggurj 20:25-28]
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Running mousemove events from the refresh driver

[Brian Birtles]

(2013/02/15 11:36), Robert O'Callahan wrote:

I created http://people.mozilla.com/~roc/mousemove-freq.html. We get up to
120-ish mousemoves per second on my machine in Firefox, and a bit more in
IE9, but it caps out at 60fps in Chrome which suggests to me they're doing
something like what I suggested already.

In that testcase you can artificially limit mousemove frequency to 60 per
second. It doesn't make much difference to my drawing, athough I'm poorly
coordinately. But wouldn't it make sense for a drawing app to intelligently
interpolate between mousemove points anyway, using splines or something? I
sort of doubt the difference between 60 points per second and 120 points
per second is critical. And I *really* doubt we should design for that
constraint.


I'm not sure if this is a relevant data point but we had an issue[1] 
with touch event coalescing on fennec that produced poor results for the 
following drawing application on some devices such as the Dell Streak:


  http://parapara-editor.mozlabs.jp/test

This application uses splines to interpolate between the points but 
still the lack of events produced poor results. It was fixed by turning 
off touch event coalescing. See bug 757680.


Best regards,

Brian
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Running mousemove events from the refresh driver

[Robert O'Callahan]

On Tue, Feb 19, 2013 at 12:44 PM, Karl Tomlinson  wrote:

> I don't know exactly what happens with WM_MOUSEMOVE, it would seem
> unfortunate if a WM_MOUSEMOVE with an updated mouse position is
> not received before key events.  Changing the order of events
> changes the meaning considerably.
>

On Windows key events are already prioritized over mousemoves, by Windows
itself.

Rob
-- 
Wrfhf pnyyrq gurz gbtrgure naq fnvq, “Lbh xabj gung gur ehyref bs gur
Tragvyrf ybeq vg bire gurz, naq gurve uvtu bssvpvnyf rkrepvfr nhgubevgl
bire gurz. Abg fb jvgu lbh. Vafgrnq, jubrire jnagf gb orpbzr terng nzbat
lbh zhfg or lbhe freinag, naq jubrire jnagf gb or svefg zhfg or lbhe fynir
— whfg nf gur Fba bs Zna qvq abg pbzr gb or freirq, ohg gb freir, naq gb
tvir uvf yvsr nf n enafbz sbe znal.” [Znggurj 20:25-28]
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Running mousemove events from the refresh driver

[Karl Tomlinson]

Robert O'Callahan writes:

> On Tue, Feb 19, 2013 at 11:47 AM, Karl Tomlinson  wrote:
>
>> Robert O'Callahan writes:
>>
>> > How about this idea: after processing a WM_MOUSEMOVE event, go into an
>> > "anti-flood" state where WM_MOUSEMOVE is ignored. After we service the
>> > Gecko event queue, exit the anti-flood state.

>> I expect "ignored" will have to be "ignored at least until there
>> is another user input event" so that the document will know where
>> the mouse is when a key is pressed.
>>
>
> Why? We currently don't do anything to ensure mousemoves are flushed before
> key events.

When the GTK-port widget code coalesces mousemoves, it uses the
coordinates of the last mousemove on the queue.  There is no other
suppression of mousemoves that I am aware of.

I don't know exactly what happens with WM_MOUSEMOVE, it would seem
unfortunate if a WM_MOUSEMOVE with an updated mouse position is
not received before key events.  Changing the order of events
changes the meaning considerably.
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Running mousemove events from the refresh driver

[Robert O'Callahan]

On Tue, Feb 19, 2013 at 11:47 AM, Karl Tomlinson  wrote:

> Robert O'Callahan writes:
>
> > How about this idea: after processing a WM_MOUSEMOVE event, go into an
> > "anti-flood" state where WM_MOUSEMOVE is ignored. After we service the
> > Gecko event queue, exit the anti-flood state.
>
> The general approach sounds good.
>
> I expect "ignored" will have to be "ignored at least until there
> is another user input event" so that the document will know where
> the mouse is when a key is pressed.
>

Why? We currently don't do anything to ensure mousemoves are flushed before
key events.

Rob
-- 
Wrfhf pnyyrq gurz gbtrgure naq fnvq, “Lbh xabj gung gur ehyref bs gur
Tragvyrf ybeq vg bire gurz, naq gurve uvtu bssvpvnyf rkrepvfr nhgubevgl
bire gurz. Abg fb jvgu lbh. Vafgrnq, jubrire jnagf gb orpbzr terng nzbat
lbh zhfg or lbhe freinag, naq jubrire jnagf gb or svefg zhfg or lbhe fynir
— whfg nf gur Fba bs Zna qvq abg pbzr gb or freirq, ohg gb freir, naq gb
tvir uvf yvsr nf n enafbz sbe znal.” [Znggurj 20:25-28]
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Running mousemove events from the refresh driver

[Karl Tomlinson]

Robert O'Callahan writes:

> How about this idea: after processing a WM_MOUSEMOVE event, go into an
> "anti-flood" state where WM_MOUSEMOVE is ignored. After we service the
> Gecko event queue, exit the anti-flood state.

The general approach sounds good.

I expect "ignored" will have to be "ignored at least until there
is another user input event" so that the document will know where
the mouse is when a key is pressed.
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Running mousemove events from the refresh driver

[Robert O'Callahan]

How about this idea: after processing a WM_MOUSEMOVE event, go into an
"anti-flood" state where WM_MOUSEMOVE is ignored. After we service the
Gecko event queue, exit the anti-flood state.

This is very simple and I think it would work well for all cases. When DOM
mousemove handlers are cheap and there isn't much else going on, we'll be
able to fire mousemove at very high rates. But when mousemove handlers are
expensive (bug 837985) we'd never delay processing the Gecko event queue
(and running the refresh driver) by more than the duration of one mousemove
handler.

Rob
-- 
Wrfhf pnyyrq gurz gbtrgure naq fnvq, “Lbh xabj gung gur ehyref bs gur
Tragvyrf ybeq vg bire gurz, naq gurve uvtu bssvpvnyf rkrepvfr nhgubevgl
bire gurz. Abg fb jvgu lbh. Vafgrnq, jubrire jnagf gb orpbzr terng nzbat
lbh zhfg or lbhe freinag, naq jubrire jnagf gb or svefg zhfg or lbhe fynir
— whfg nf gur Fba bs Zna qvq abg pbzr gb or freirq, ohg gb freir, naq gb
tvir uvf yvsr nf n enafbz sbe znal.” [Znggurj 20:25-28]
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Running mousemove events from the refresh driver

[Jean-Marc Desperrier]

Benjamin Smedberg a écrit :

it's important for certain kinds of drawing apps especially to have as
much mouse input as possible,


Yes


and I doubt that only receiving mouse input at 60fps would be adequate in 
general


I don't know if what I'm saying is stupid, but I think it would be seen 
as decent to have an API that request a high refresh frequency when a 
specific area has focus, in order to be able to get it but not use too 
much CPU when not really useful.


Of course, this should get normalized.
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Running mousemove events from the refresh driver

[Robert O'Callahan]

On Sat, Feb 16, 2013 at 6:16 AM, Steve Fink  wrote:

> It suggests a solution where a quick handler sees all mouse move events
> and batches them up, delivering the batches at a lower rate (60fps isn't
> completely unreasonable). Which is of course completely not
> spec-compliant.


I think it is spec-compliant, actually, if we deliver the batched events as
a series of individually dispatched mousemove DOM events. It might even be
a reasonable solution. In bug 837985 it wouldn't be quite as good as
coalescing --- we'd deliver multiple mouse-move events on each refresh
driver tick, and all but the last one would do significant unnecessary work
--- but it would stop the refresh driver from being starved so it would
make that bug a lot better. The drawing application case would work
perfectly.

It could mess with the timing of sites that use Date.now() in a mousemove
event handler to estimate mouse velocity. On the other hand that is already
probably super-inaccurate over very short intervals. We could make
event.timeStamp return the right thing anyway, if it's OK for DOM event
dispatch order to not match the timeStamp order.

I'm still not sure that this approach would be worth implementing. The
extra precision for drawing applications doesn't seem like a big win given
the mouse isn't a very good drawing tool anyway.

Rob
-- 
Wrfhf pnyyrq gurz gbtrgure naq fnvq, “Lbh xabj gung gur ehyref bs gur
Tragvyrf ybeq vg bire gurz, naq gurve uvtu bssvpvnyf rkrepvfr nhgubevgl
bire gurz. Abg fb jvgu lbh. Vafgrnq, jubrire jnagf gb orpbzr terng nzbat
lbh zhfg or lbhe freinag, naq jubrire jnagf gb or svefg zhfg or lbhe fynir
— whfg nf gur Fba bs Zna qvq abg pbzr gb or freirq, ohg gb freir, naq gb
tvir uvf yvsr nf n enafbz sbe znal.” [Znggurj 20:25-28]
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Running mousemove events from the refresh driver

[Chris Peterson]

On 2/14/13 6:36 PM, Robert O'Callahan wrote:

I created http://people.mozilla.com/~roc/mousemove-freq.html. We get up to
120-ish mousemoves per second on my machine in Firefox, and a bit more in
IE9, but it caps out at 60fps in Chrome which suggests to me they're doing
something like what I suggested already.


Using my MacBook Pro's touchpad, Firefox maxes out at 60 fps, Chrome at 
60 fps, and Safari at 120 fps.



chris

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Running mousemove events from the refresh driver

[Steve Fink]

On 02/14/2013 07:48 PM, Robert O'Callahan wrote:
> On Fri, Feb 15, 2013 at 4:44 PM, John Volikas  wrote:
>
>> I tried the test on Nightly runnig Windows 7 64bit.
>>
>> I get up to 1000(!) mousemoves per second but I have a Logitech G400
>> "gaming" mouse that defaults to a 1000Hz polling rate without Logitech's
>> software. I guess it depends on the peripheral and the OS. The lines are
>> very smooth even with rapid mouse movements. [1]
>>
>> Selecting the checkbox to cap the rate and rapidly moving the mouse around
>> results in a very ugly result though. [2]
>>
>> [1] http://ompldr.org/vaGd2bg/moverate.png
>> [2] http://ompldr.org/vaGd2bw/moveratecapped.png
>>
> Thanks. That is useful information. Unfortunately it doesn't help me figure
> out what to do :-).
>
> Rob
It suggests a solution where a quick handler sees all mouse move events
and batches them up, delivering the batches at a lower rate (60fps isn't
completely unreasonable). Which is of course completely not
spec-compliant. So as long as I'm ignoring the spec: have a quick
handler that observes every move event. If nobody cares, drop them on
the floor. If any standard handlers are defined, downsample them to
60fps and deliver one sampled move event per tick. If anybody registered
for batched events, batch them up and deliver batches at 60fps. And/or
if anybody registered for spline events, approximate them with a spline
and deliver that at 60fps. (And set a max rate ceiling so you don't blow
memory when some goofy test harness starts triggering mouse moves at 50MHz.)

This completely and totally fails to solve the problem of regressing
1000Hz mice. Uh... reorder events? Send one mouse event per 60Hz tick,
then deprioritize further events behind everything else, so that if the
queue is empty then the remaining events are allowed through?

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Running mousemove events from the refresh driver

[smaug]

On 02/14/2013 05:48 AM, Robert O'Callahan wrote:

On Thu, Feb 14, 2013 at 3:21 AM, Benjamin Smedberg wrote:


On what OSes? Windows by default coalesces mouse move events. They are
like WM_PAINT events in that they are only delivered when the event queue
is empty. See
http://blogs.msdn.com/b/oldnewthing/archive/2011/12/19/10249000.aspx

This should basically mean that we process mousemove events on windows up
to 100% CPU, but we should never be "flooded" by them. Although I do wonder
if WM_MOUSEMOVE has priority over WM_PAINT so that if the mouse is moving a
lot, that could affect the latency of WM_PAINT.



We are definitely getting flooded. Here's what I think is happening on the
page I'm looking at, it's pretty simple:
1) nsAppShell::ProcessNextNativeEvent checks for input events, finds a
WM_MOUSE_MOVE, and dispatches it, which takes a little while because this
page's mousemove handler modifies and flushes layout on every mouse move.
2) While that's happening, the mouse keeps moving.
3) After processing that WM_MOUSE_MOVE, ProcessNextNativeEvent calls
PeekMessage again and finds another WM_MOUSE_MOVE is ready. Go to step 1.

Hmm, why do we call PeekMessage at that point and not go to gecko event loop.
IIRC we still have the problem at least on OSX that we check native events too 
often.




4) Meanwhile the refresh driver timer has fired and queued an event, but we
don't get around to running it until NATIVE_EVENT_STARVATION_LIMIT has
expired (one second).

I suppose we could try ignoring WM_MOUSE_MOVEs when there's a Gecko event
pending, but that sounds kinda scary. I think deferring DOM mousemove
events to the next refresh driver tick would be safer than that.

Rob



___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Running mousemove events from the refresh driver

[Robert O'Callahan]

On Fri, Feb 15, 2013 at 4:44 PM, John Volikas  wrote:

> I tried the test on Nightly runnig Windows 7 64bit.
>
> I get up to 1000(!) mousemoves per second but I have a Logitech G400
> "gaming" mouse that defaults to a 1000Hz polling rate without Logitech's
> software. I guess it depends on the peripheral and the OS. The lines are
> very smooth even with rapid mouse movements. [1]
>
> Selecting the checkbox to cap the rate and rapidly moving the mouse around
> results in a very ugly result though. [2]
>
> [1] http://ompldr.org/vaGd2bg/moverate.png
> [2] http://ompldr.org/vaGd2bw/moveratecapped.png
>

Thanks. That is useful information. Unfortunately it doesn't help me figure
out what to do :-).

Rob
-- 
Wrfhf pnyyrq gurz gbtrgure naq fnvq, “Lbh xabj gung gur ehyref bs gur
Tragvyrf ybeq vg bire gurz, naq gurve uvtu bssvpvnyf rkrepvfr nhgubevgl
bire gurz. Abg fb jvgu lbh. Vafgrnq, jubrire jnagf gb orpbzr terng nzbat
lbh zhfg or lbhe freinag, naq jubrire jnagf gb or svefg zhfg or lbhe fynir
— whfg nf gur Fba bs Zna qvq abg pbzr gb or freirq, ohg gb freir, naq gb
tvir uvf yvsr nf n enafbz sbe znal.” [Znggurj 20:25-28]
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Running mousemove events from the refresh driver

[John Volikas]

On Friday, February 15, 2013 4:36:21 AM UTC+2, Robert O'Callahan wrote:
> On Fri, Feb 15, 2013 at 10:14 AM, Benjamin Smedberg
> 
> wrote:
> 
> 
> 
> > I think we should try to process mousemoves as quickly as we can: it's
> 
> > important for certain kinds of drawing apps especially to have as much
> 
> > mouse input as possible, and I doubt that only receiving mouse input at
> 
> > 60fps would be adequate in general. Can we build a very basic layer on top
> 
> > of the OS mousemove events that coalesces them and dispatches them as soon
> 
> > as there are no pending events (native or gecko)? This should solve the
> 
> > problem of starving gecko events (especially refresh driver events).
> 
> >
> 
> 
> 
> That could starve mousemove events, though.
> 
> 
> 
> I created http://people.mozilla.com/~roc/mousemove-freq.html. We get up to
> 
> 120-ish mousemoves per second on my machine in Firefox, and a bit more in
> 
> IE9, but it caps out at 60fps in Chrome which suggests to me they're doing
> 
> something like what I suggested already.
> 
> 
> 
> In that testcase you can artificially limit mousemove frequency to 60 per
> 
> second. It doesn't make much difference to my drawing, athough I'm poorly
> 
> coordinately. But wouldn't it make sense for a drawing app to intelligently
> 
> interpolate between mousemove points anyway, using splines or something? I
> 
> sort of doubt the difference between 60 points per second and 120 points
> 
> per second is critical. And I *really* doubt we should design for that
> 
> constraint.
> 
> 
> 
> Rob
> 
> -- 
> 
> Wrfhf pnyyrq gurz gbtrgure naq fnvq, “Lbh xabj gung gur ehyref bs gur
> 
> Tragvyrf ybeq vg bire gurz, naq gurve uvtu bssvpvnyf rkrepvfr nhgubevgl
> 
> bire gurz. Abg fb jvgu lbh. Vafgrnq, jubrire jnagf gb orpbzr terng nzbat
> 
> lbh zhfg or lbhe freinag, naq jubrire jnagf gb or svefg zhfg or lbhe fynir
> 
> — whfg nf gur Fba bs Zna qvq abg pbzr gb or freirq, ohg gb freir, naq gb
> 
> tvir uvf yvsr nf n enafbz sbe znal.” [Znggurj 20:25-28]

I tried the test on Nightly runnig Windows 7 64bit.

I get up to 1000(!) mousemoves per second but I have a Logitech G400 "gaming" 
mouse that defaults to a 1000Hz polling rate without Logitech's software. I 
guess it depends on the peripheral and the OS. The lines are very smooth even 
with rapid mouse movements. [1]

Selecting the checkbox to cap the rate and rapidly moving the mouse around 
results in a very ugly result though. [2]

[1] http://ompldr.org/vaGd2bg/moverate.png
[2] http://ompldr.org/vaGd2bw/moveratecapped.png
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Running mousemove events from the refresh driver

[Robert O'Callahan]

On Fri, Feb 15, 2013 at 10:14 AM, Benjamin Smedberg
wrote:

> I think we should try to process mousemoves as quickly as we can: it's
> important for certain kinds of drawing apps especially to have as much
> mouse input as possible, and I doubt that only receiving mouse input at
> 60fps would be adequate in general. Can we build a very basic layer on top
> of the OS mousemove events that coalesces them and dispatches them as soon
> as there are no pending events (native or gecko)? This should solve the
> problem of starving gecko events (especially refresh driver events).
>

That could starve mousemove events, though.

I created http://people.mozilla.com/~roc/mousemove-freq.html. We get up to
120-ish mousemoves per second on my machine in Firefox, and a bit more in
IE9, but it caps out at 60fps in Chrome which suggests to me they're doing
something like what I suggested already.

In that testcase you can artificially limit mousemove frequency to 60 per
second. It doesn't make much difference to my drawing, athough I'm poorly
coordinately. But wouldn't it make sense for a drawing app to intelligently
interpolate between mousemove points anyway, using splines or something? I
sort of doubt the difference between 60 points per second and 120 points
per second is critical. And I *really* doubt we should design for that
constraint.

Rob
-- 
Wrfhf pnyyrq gurz gbtrgure naq fnvq, “Lbh xabj gung gur ehyref bs gur
Tragvyrf ybeq vg bire gurz, naq gurve uvtu bssvpvnyf rkrepvfr nhgubevgl
bire gurz. Abg fb jvgu lbh. Vafgrnq, jubrire jnagf gb orpbzr terng nzbat
lbh zhfg or lbhe freinag, naq jubrire jnagf gb or svefg zhfg or lbhe fynir
— whfg nf gur Fba bs Zna qvq abg pbzr gb or freirq, ohg gb freir, naq gb
tvir uvf yvsr nf n enafbz sbe znal.” [Znggurj 20:25-28]
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Running mousemove events from the refresh driver

[Benjamin Smedberg]

On 2/13/2013 10:48 PM, Robert O'Callahan wrote:

On Thu, Feb 14, 2013 at 3:21 AM, Benjamin Smedberg wrote:


On what OSes? Windows by default coalesces mouse move events. They are
like WM_PAINT events in that they are only delivered when the event queue
is empty. See
http://blogs.msdn.com/b/oldnewthing/archive/2011/12/19/10249000.aspx

This should basically mean that we process mousemove events on windows up
to 100% CPU, but we should never be "flooded" by them. Although I do wonder
if WM_MOUSEMOVE has priority over WM_PAINT so that if the mouse is moving a
lot, that could affect the latency of WM_PAINT.


We are definitely getting flooded. Here's what I think is happening on the
page I'm looking at, it's pretty simple:
1) nsAppShell::ProcessNextNativeEvent checks for input events, finds a
WM_MOUSE_MOVE, and dispatches it, which takes a little while because this
page's mousemove handler modifies and flushes layout on every mouse move.
2) While that's happening, the mouse keeps moving.
3) After processing that WM_MOUSE_MOVE, ProcessNextNativeEvent calls
PeekMessage again and finds another WM_MOUSE_MOVE is ready. Go to step 1.
4) Meanwhile the refresh driver timer has fired and queued an event, but we
don't get around to running it until NATIVE_EVENT_STARVATION_LIMIT has
expired (one second).

I suppose we could try ignoring WM_MOUSE_MOVEs when there's a Gecko event
pending, but that sounds kinda scary. I think deferring DOM mousemove
events to the next refresh driver tick would be safer than that.


I think we should try to process mousemoves as quickly as we can: it's 
important for certain kinds of drawing apps especially to have as much 
mouse input as possible, and I doubt that only receiving mouse input at 
60fps would be adequate in general. Can we build a very basic layer on 
top of the OS mousemove events that coalesces them and dispatches them 
as soon as there are no pending events (native or gecko)? This should 
solve the problem of starving gecko events (especially refresh driver 
events).


Other possible solutions are to special case refresh driver events in 
the event loop and dispatch/coalesce them at specific times (ignoring 
the native event starvation rules, for example), similar to what Windows 
already does with WM_PAINT.


--BDS

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Running mousemove events from the refresh driver

[Robert O'Callahan]

On Thu, Feb 14, 2013 at 3:21 AM, Benjamin Smedberg wrote:

> On what OSes? Windows by default coalesces mouse move events. They are
> like WM_PAINT events in that they are only delivered when the event queue
> is empty. See
> http://blogs.msdn.com/b/oldnewthing/archive/2011/12/19/10249000.aspx
>
> This should basically mean that we process mousemove events on windows up
> to 100% CPU, but we should never be "flooded" by them. Although I do wonder
> if WM_MOUSEMOVE has priority over WM_PAINT so that if the mouse is moving a
> lot, that could affect the latency of WM_PAINT.
>

We are definitely getting flooded. Here's what I think is happening on the
page I'm looking at, it's pretty simple:
1) nsAppShell::ProcessNextNativeEvent checks for input events, finds a
WM_MOUSE_MOVE, and dispatches it, which takes a little while because this
page's mousemove handler modifies and flushes layout on every mouse move.
2) While that's happening, the mouse keeps moving.
3) After processing that WM_MOUSE_MOVE, ProcessNextNativeEvent calls
PeekMessage again and finds another WM_MOUSE_MOVE is ready. Go to step 1.
4) Meanwhile the refresh driver timer has fired and queued an event, but we
don't get around to running it until NATIVE_EVENT_STARVATION_LIMIT has
expired (one second).

I suppose we could try ignoring WM_MOUSE_MOVEs when there's a Gecko event
pending, but that sounds kinda scary. I think deferring DOM mousemove
events to the next refresh driver tick would be safer than that.

Rob
-- 
Wrfhf pnyyrq gurz gbtrgure naq fnvq, “Lbh xabj gung gur ehyref bs gur
Tragvyrf ybeq vg bire gurz, naq gurve uvtu bssvpvnyf rkrepvfr nhgubevgl
bire gurz. Abg fb jvgu lbh. Vafgrnq, jubrire jnagf gb orpbzr terng nzbat
lbh zhfg or lbhe freinag, naq jubrire jnagf gb or svefg zhfg or lbhe fynir
— whfg nf gur Fba bs Zna qvq abg pbzr gb or freirq, ohg gb freir, naq gb
tvir uvf yvsr nf n enafbz sbe znal.” [Znggurj 20:25-28]
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Running mousemove events from the refresh driver

[Neil]

Benjamin Smedberg wrote:

I do wonder if WM_MOUSEMOVE has priority over WM_PAINT so that if the 
mouse is moving a lot, that could affect the latency of WM_PAINT.


I think we used to artificially bump keyboard and mouse messages over 
posted messages to stop posted messages from affecting the latency of 
mouse messages. Either way, WM_PAINT comes second last just before 
WM_TIMER. See 
http://msdn.microsoft.com/en-gb/library/windows/desktop/ms644936.aspx


--
Warning: May contain traces of nuts.
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Running mousemove events from the refresh driver

[Benjamin Smedberg]

On 2/12/2013 10:18 PM, Robert O'Callahan wrote:

Context: bug 837985.

At times we can be flooded by OS-level mousemove events.
On what OSes? Windows by default coalesces mouse move events. They are 
like WM_PAINT events in that they are only delivered when the event 
queue is empty. See 
http://blogs.msdn.com/b/oldnewthing/archive/2011/12/19/10249000.aspx


This should basically mean that we process mousemove events on windows 
up to 100% CPU, but we should never be "flooded" by them. Although I do 
wonder if WM_MOUSEMOVE has priority over WM_PAINT so that if the mouse 
is moving a lot, that could affect the latency of WM_PAINT.


If other OSes don't do this, I definitely think we should consider 
emulating this behavior somehow; I don't know if it's possible to know 
on other widget layers whether the event queue is empty.


--BDS

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Running mousemove events from the refresh driver

[Robert O'Callahan]

On Wed, Feb 13, 2013 at 5:14 PM, Rob Arnold  wrote:

> I agree; it should be no worse than today. I do have some concerns with
> dispatching a mouse move event that contains coordinates the mouse may not
> have been at but the visual results for scrolling ought to be nice. Only
> other concern is if there are applications that make use of the higher
> frequency mouse events to do extra processing (ex: a finger painting app
> would want more samples for curve reconstruction). Otherwise I think it
> makes sense.
>

I did some tests with various browsers on Windows and we normally only get
60-ish mouse-moves per second max. So if we're keep our frame rate up, we
should be OK.

Rob
-- 
Wrfhf pnyyrq gurz gbtrgure naq fnvq, “Lbh xabj gung gur ehyref bs gur
Tragvyrf ybeq vg bire gurz, naq gurve uvtu bssvpvnyf rkrepvfr nhgubevgl
bire gurz. Abg fb jvgu lbh. Vafgrnq, jubrire jnagf gb orpbzr terng nzbat
lbh zhfg or lbhe freinag, naq jubrire jnagf gb or svefg zhfg or lbhe fynir
— whfg nf gur Fba bs Zna qvq abg pbzr gb or freirq, ohg gb freir, naq gb
tvir uvf yvsr nf n enafbz sbe znal.” [Znggurj 20:25-28]
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Running mousemove events from the refresh driver

[Rob Arnold]

On Tue, Feb 12, 2013 at 7:57 PM, Robert O'Callahan wrote:

> On Wed, Feb 13, 2013 at 4:45 PM, Rob Arnold  wrote:
>
>> Would you want to predict the mouse location based on past events when
>> you dispatch the synthetic event? I guess it depends on how frequently you
>> get the events but this is done for touches on mobile where the input
>> frequency is close (with 2x) the display frequency. If not, what would the
>> timestamp on the event be?
>>
>
> I hadn't thought about that. Is it really necessary? I think without
> prediction, we'd get the same visual results as we do today. With
> prediction we might make drags seem to track the mouse a bit better,
> although it sounds a bit scary to have the user mouse over something they
> actually didn't.
>

I agree; it should be no worse than today. I do have some concerns with
dispatching a mouse move event that contains coordinates the mouse may not
have been at but the visual results for scrolling ought to be nice. Only
other concern is if there are applications that make use of the higher
frequency mouse events to do extra processing (ex: a finger painting app
would want more samples for curve reconstruction). Otherwise I think it
makes sense.

-Rob
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Running mousemove events from the refresh driver

[Robert O'Callahan]

On Wed, Feb 13, 2013 at 4:45 PM, Rob Arnold  wrote:

> Would you want to predict the mouse location based on past events when you
> dispatch the synthetic event? I guess it depends on how frequently you get
> the events but this is done for touches on mobile where the input frequency
> is close (with 2x) the display frequency. If not, what would the timestamp
> on the event be?
>

I hadn't thought about that. Is it really necessary? I think without
prediction, we'd get the same visual results as we do today. With
prediction we might make drags seem to track the mouse a bit better,
although it sounds a bit scary to have the user mouse over something they
actually didn't.

Rob
-- 
Wrfhf pnyyrq gurz gbtrgure naq fnvq, “Lbh xabj gung gur ehyref bs gur
Tragvyrf ybeq vg bire gurz, naq gurve uvtu bssvpvnyf rkrepvfr nhgubevgl
bire gurz. Abg fb jvgu lbh. Vafgrnq, jubrire jnagf gb orpbzr terng nzbat
lbh zhfg or lbhe freinag, naq jubrire jnagf gb or svefg zhfg or lbhe fynir
— whfg nf gur Fba bs Zna qvq abg pbzr gb or freirq, ohg gb freir, naq gb
tvir uvf yvsr nf n enafbz sbe znal.” [Znggurj 20:25-28]
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Running mousemove events from the refresh driver

[Robert O'Callahan]

Context: bug 837985.

At times we can be flooded by OS-level mousemove events. I think it would
make sense to process mousemoves at most once per refresh driver tick. This
matters for a couple of reasons: mousemove processing can cause arbitrary
JS handlers to run which can do slow things, and also for performance
reasons we currently don't flush layout before all mousemoves, but we
really should (and we've been burned by bugs where we didn't). If we
dispatch mousemoves in the refresh driver that should reduce the impact of
having to flush, since the refresh driver flushes layout anyway.

I think I'd do this by usurping the synthetic-mouse-move code to dispatch
"real" mousemoves as well, so platform-level mousemove events do nothing
but notify the presshell that a mousemove is needed and schedule a refresh
driver tick.

Can anyone see any problems with this?

Rob
-- 
Wrfhf pnyyrq gurz gbtrgure naq fnvq, “Lbh xabj gung gur ehyref bs gur
Tragvyrf ybeq vg bire gurz, naq gurve uvtu bssvpvnyf rkrepvfr nhgubevgl
bire gurz. Abg fb jvgu lbh. Vafgrnq, jubrire jnagf gb orpbzr terng nzbat
lbh zhfg or lbhe freinag, naq jubrire jnagf gb or svefg zhfg or lbhe fynir
— whfg nf gur Fba bs Zna qvq abg pbzr gb or freirq, ohg gb freir, naq gb
tvir uvf yvsr nf n enafbz sbe znal.” [Znggurj 20:25-28]
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform