Frame synchronization open questions
Some open questions in my head about the frame synchronization work: * Is GdkPaintClock the right name? It might imply that it only has to do about painting and not about other things like layout. GdkFrameClock would be an alternative. GdkClock is possible but likely too generic. * For pausing the main event delivery, what we currently do is that we queue events but don't dispatch them. This could conceivably cause ordering problems for apps that use filters, or for work in GDK that is done at the translate stage - since we are not pausing translation, just pausing delivery. Alternatives: - Remove the file descriptor and don't unqueue events from the OS queue until event delivery is unpaused. Since we can wait and sleep currently while event delivery is paused, we have to be careful that we don't spin in this case. - Unpause event delivery earlier - before we freeze waiting for _NET_WM_FRAME_DRAWN. Then we don't need to worry about spinning when there are OS events pending, since we'll never sleep with event delivery paused. * Do we need something like GtkTimeline but "rawer" - where you can just get updates and a raw elapsed time? Should we make GtkTimeline with a negative duration do this with the progress being the elapsed time? * Is it OK for the paint-clock to be an immutable property set at GdkWindow construction time? Right now, it's mutable, but not notified, and not handled within gtk. * Right now GdkPaintClockTarget only has a set_clock() method. Would it make sense to also have it have a update() method, and have the behavior that adding a paint clock target to a widget or directly to a GdkPaintClock implicitly requests the ::update phase until the target is removed? This would simplify the code in the places where I'm using GdkPaintClockTarget currently a bit, but I don't see implementing GdkPaintClockTarget directly to be a common thing. ___ gtk-devel-list mailing list gtk-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: Frame synchronization open questions
> Some open questions in my head about the frame synchronization work: > > * Is GdkPaintClock the right name? It might imply that it only has to > do about painting and not about other things like layout. > GdkFrameClock would be an alternative. GdkClock is possible but > likely too generic. I agree that Paint is confusing, GdkFrameClock seems to be the right name to me. Frame is what this is always called in gameloop style code. > * For pausing the main event delivery, what we currently do is that > we queue events but don't dispatch them. This could conceivably > cause ordering problems for apps that use filters, or for work > in GDK that is done at the translate stage - since we are not > pausing translation, just pausing delivery. Alternatives: Are you sure that is really a problem? The x11 gdk_event_source_check() code will never even look for a message if there is any GdkEvent queued. And if there is nothing queued _gdk_x11_display_queue_events() will stop as soon any X event was converted to a queued GdkEvent. And, since this queued event will not be read due to the pause we will never process more X events until resume events. So, we may process multiple events that are not queued into gdk events and at most one that is queued, but no more until the queued event is handled. This does not look like it could ever cause a reordering problem to me. At most it means we may handle some filtered X events during the frame cycle, I'm not sure that is as much of a problem. What I do however worry about is the combination of multiple GdkPaintClocks and gdk_display_set_events_paused(). It seems to me that each paint clock assumes it has full ownership of the display, calling set_events_paused(FALSE) whenever it has finished its paint cycle. However, could there not be other paintclocks (say for another toplevel) being active at that time? Another thing I worry about is offscreen window animation. If you have an window with an embedded offscreen inside, then queueing a redraw on a widget inside the offscreen will cause a repaint cycle. When drawing to the offscreen this will generate damage events that will cause the embedding widget to repaint itself. However the damage events will be emitted during the paint phase, and the parent widget will not get these until the next frame. This will cause a delay of on frame which may look weird. ___ gtk-devel-list mailing list gtk-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: Frame synchronization open questions
On Thu, 2012-10-04 at 08:09 -0400, Alexander Larsson wrote: > > Some open questions in my head about the frame synchronization work: [...] > > * For pausing the main event delivery, what we currently do is that > > we queue events but don't dispatch them. This could conceivably > > cause ordering problems for apps that use filters, or for work > > in GDK that is done at the translate stage - since we are not > > pausing translation, just pausing delivery. Alternatives: > > Are you sure that is really a problem? The x11 gdk_event_source_check() > code will never even look for a message if there is any GdkEvent queued. > And if there is nothing queued _gdk_x11_display_queue_events() will stop > as soon any X event was converted to a queued GdkEvent. And, since this > queued event will not be read due to the pause we will never process more > X events until resume events. That gets into the details of exactly how I implemented event pausing, but logically speaking, if event delivery is paused, and a new event happens, it needs to be queued up in GDK, Xlib, or in the X server. If we don't translate events, then either: * Event is queued in Xlib, XPending() starts returning TRUE * Event is queued in XServer, the file descriptor Either situation is going to make us start spinning currently. But leaving that aside, we *need* to translate events or we'll never get _NET_WM_FRAME_DONE and unfreeze event delivery. My current preference is to just unfreeze all event delivery at the end of drawing the frame, and not wait for _NET_WM_FRAME_DONE. Then we can make pausing *really* pausing - not unqueue, not translate, etc. I don't see any real disadvantage of doing things that way. [...] > What I do however worry about is the combination of multiple GdkPaintClocks > and gdk_display_set_events_paused(). It seems to me that each paint clock > assumes it has full ownership of the display, calling set_events_paused(FALSE) > whenever it has finished its paint cycle. However, could there not be other > paintclocks (say for another toplevel) being active at that time? Yes, this is an issue. However, if you switch to the above approach, then all you have to do is to properly reference count the event-pausing - once we reach the GDK_PRIORITY_EVENTS + 1 idle, we want to pause *all* event delivery until we are done with new frame handling. > Another thing I worry about is offscreen window animation. If you have an > window > with an embedded offscreen inside, then queueing a redraw on a widget inside > the > offscreen will cause a repaint cycle. When drawing to the offscreen this will > generate damage events that will cause the embedding widget to repaint itself. > However the damage events will be emitted during the paint phase, and the > parent > widget will not get these until the next frame. This will cause a delay of on > frame > which may look weird. Hmm. So your concern is that the connection between the embedded offscreen and the embedder is purely at the GTK+ level - if GDK_DAMAGE events are not delivered to GTK+, then the the embedder won't know to repaint. I think the normal way it would end up working is: offscreen paints a frame, damage events are generated damage events are delivered, new frame is queued for the embedder embedder paints a frame, waits for _NET_WM_FRAME_DONE offscreen paints a frame, damage events are generated damage events are delivered, new frame is queued for the embedder _NET_WM_FRAME_DONE arrives embedder paints a frame, waits for _NET_WM_FRAME_DONE offscreen paints a frame, damage events are generated damage events are delivered, new frame is queued for the embedder _NET_WM_FRAME_DONE arrives So the offscreen will generally paint during the wait for _NET_WM_FRAME_DONE, and immediately get incorporated into the next output frame. Though once things get loaded down and slow, there is no guarantee of this. To do better than this, I think you need to slave the clock for the offscreen into the clock for the window that is embedding it. If the size of the embedded widget is free-floating and not constrained by the allocation of the embedder, then the slave clock would simply do the complete cycle inside an ::update handler. If you want integration of size-negotiation, then you'd have to do different phases at different points in the master clock cycle. And yes, the delivery of damage as events is problematical for that kind of slaving - maybe we would need to have a signal on GdkOffscreenWindow to allow getting as-it-happens notification of damage. - Owen ___ gtk-devel-list mailing list gtk-devel-list@gnome.org https://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: Frame synchronization open questions
On tor, 2012-10-04 at 15:46 -0400, Owen Taylor wrote: > On Thu, 2012-10-04 at 08:09 -0400, Alexander Larsson wrote: > > Are you sure that is really a problem? The x11 gdk_event_source_check() > > code will never even look for a message if there is any GdkEvent queued. > > And if there is nothing queued _gdk_x11_display_queue_events() will stop > > as soon any X event was converted to a queued GdkEvent. And, since this > > queued event will not be read due to the pause we will never process more > > X events until resume events. > > That gets into the details of exactly how I implemented event pausing, > but logically speaking, if event delivery is paused, and a new event > happens, it needs to be queued up in GDK, Xlib, or in the X server. > If we don't translate events, then either: If we start to translate and queue multiple X events into the gdk queue there is a lot more risk of the translation getting things "out of order", so i don't think that is quite safe. So, yeah, queuing in xlib seems better. > * Event is queued in Xlib, XPending() starts returning TRUE > * Event is queued in XServer, the file descriptor > > Either situation is going to make us start spinning currently. But leaving > that aside, we *need* to translate events or we'll never get > _NET_WM_FRAME_DONE > and unfreeze event delivery. We could find it via XCheckIfEvent, although that risks not being able to find it if the server side fd is full when its written and we never read from the X fd. Not sure if that can happen. > My current preference is to just unfreeze all event delivery at the end of > drawing the frame, and not wait for _NET_WM_FRAME_DONE. Then we can make > pausing *really* pausing - not unqueue, not translate, etc. I don't see > any real disadvantage of doing things that way. I agree. This seems clearly safest. > > Another thing I worry about is offscreen window animation. If you have an > > window > > with an embedded offscreen inside, then queueing a redraw on a widget > > inside the > > offscreen will cause a repaint cycle. When drawing to the offscreen this > > will > > generate damage events that will cause the embedding widget to repaint > > itself. > > However the damage events will be emitted during the paint phase, and the > > parent > > widget will not get these until the next frame. This will cause a delay of > > on frame > > which may look weird. > > Hmm. So your concern is that the connection between the embedded > offscreen and the embedder is purely at the GTK+ level - if GDK_DAMAGE > events are not delivered to GTK+, then the the embedder won't know to > repaint. Actually, it depends a bit on how the event freezing works. Damage events for offscreens are "emulated" (to make it work even without damage X support and on other platforms) it the sense that gdk_window_process_updates_recurse does: if (gdk_window_is_offscreen (window->impl_window) && gdk_window_has_impl (window)) _gdk_window_add_damage ((GdkWindow *) window->impl_window, expose_region); Which will queue a gdk side event (i.e. X and X event translation is not involved). So, if we freeze at the gdk level we will not get these during the paint cycle, but if we do it at the X level we might. > So the offscreen will generally paint during the wait for > _NET_WM_FRAME_DONE, and immediately get incorporated into the next > output frame. Though once things get loaded down and slow, there is no > guarantee of this. Yes, there will always be a delay of one frame for the embedded side vs the embedder side. > To do better than this, I think you need to slave the clock for the > offscreen into the clock for the window that is embedding it. If the > size of the embedded widget is free-floating and not constrained by the > allocation of the embedder, then the slave clock would simply do the > complete cycle inside an ::update handler. If you want integration > of size-negotiation, then you'd have to do different phases at different > points in the master clock cycle. Maybe the widget could manually ensure the offscreen widget gets the same clock at construction time. But thats not really enough is it? You have to guarantee that the embedded widget draws before the embedder. > And yes, the delivery of damage as events is problematical for that kind > of slaving - maybe we would need to have a signal on GdkOffscreenWindow > to allow getting as-it-happens notification of damage. Yeah. Also, for something completely different. With the frame based motion compression we should imho completely remove the motion hints support. There is a bunch of code in the csw stuff to emulate it which becomes pretty much useless with this. And we never pass on motion hint masks to the xserver anyway since we only get events on the toplevel and motion hints are generally enabled on some subwindow. ___ gtk-devel-list mailing list gtk-devel-list@gnome.org https://mail.
