[PATCH 0/4] Cursor's update inside kernel only
Hi guys, Under KMS, we can build a feature to update the cursor directly to screen without the continuous intervention of the userspace application (X server, wayland, etc). It's a fastpath for DRM based cursors obtained by short-circuit the kernel input layer and DRM module. This would solve all cursor's latency issues that we have in our current model [0]. This series of patches implement such feature using Xorg as the application. Through an ioctl, Xorg tells which devices are responsible for cursors' updates and the kernel evdev driver will spit the events to DRM. DRM will take care about the event informations and also screen limits, and then will draw the cursor on screen. Very intuitive. Right now a thing that is annoying me is how others cursors, sw rendered, could be implemented. I want to avoid two differents sets of the same code in different contexts. IMHO conceptually all these cursor update things must be in-kernel. Painting a cursor image seems to be quite hard as we have to save/restore areas under the cursor. I remember that anholt had an idea concerning this, but I do not remember details. Well, the patches are far from ready to go upstream, but it deploys a system working on this way. So, for now, this mail has two goals: - to people comment on and see in what kind of world we can move. - get a feedback how we can proceed in the case of sw cursors. Please, comment on. Thank you, [0] http://vignatti.wordpress.com/2008/07/29/improving-input-latency/ -- Tiago Vignatti C3SL - Centro de Computação Científica e Software Livre www.c3sl.ufpr.br ___ xorg mailing list xorg@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/xorg
Re: [PATCH 0/4] Cursor's update inside kernel only
On Monday, January 5, 2009 12:55 pm Tiago Vignatti wrote: > Right now a thing that is annoying me is how others cursors, sw rendered, > could be implemented. I want to avoid two differents sets of the same code > in different contexts. IMHO conceptually all these cursor update things > must be in-kernel. Painting a cursor image seems to be quite hard as we > have to save/restore areas under the cursor. I remember that anholt had an > idea concerning this, but I do not remember details. I really like the idea of having this in the kernel for latency reasons, but yeah we do need to solve the sw case as well as implementing some acceleration code. OTOH it might be reasonable to push the problem of multiple, large, and or funky format cursors out to userspace, since those are relatively uncommon (64x64 32 bit ARGB ought to be enough for everybody right? :). -- Jesse Barnes, Intel Open Source Technology Center ___ xorg mailing list xorg@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/xorg
Re: [PATCH 0/4] Cursor's update inside kernel only
Le 16/01/2009 21:21, Jesse Barnes a écrit : > On Monday, January 5, 2009 12:55 pm Tiago Vignatti wrote: >> Right now a thing that is annoying me is how others cursors, sw rendered, >> could be implemented. I want to avoid two differents sets of the same code >> in different contexts. IMHO conceptually all these cursor update things >> must be in-kernel. Painting a cursor image seems to be quite hard as we >> have to save/restore areas under the cursor. I remember that anholt had an >> idea concerning this, but I do not remember details. > > I really like the idea of having this in the kernel for latency reasons, but > yeah we do need to solve the sw case as well as implementing some > acceleration code. OTOH it might be reasonable to push the problem of > multiple, large, and or funky format cursors out to userspace, since those > are relatively uncommon (64x64 32 bit ARGB ought to be enough for everybody > right? :). Not for firefox. Any drag&drop in gecko will result in huge ARGB cursors. Even just dragging and dropping a tab results in a cursor that's at least 150x20px. Cheers Rémi ___ xorg mailing list xorg@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/xorg
Re: [PATCH 0/4] Cursor's update inside kernel only
On Friday, January 16, 2009 1:55 pm Rémi Cardona wrote: > Le 16/01/2009 21:21, Jesse Barnes a écrit : > > On Monday, January 5, 2009 12:55 pm Tiago Vignatti wrote: > >> Right now a thing that is annoying me is how others cursors, sw > >> rendered, could be implemented. I want to avoid two differents sets of > >> the same code in different contexts. IMHO conceptually all these cursor > >> update things must be in-kernel. Painting a cursor image seems to be > >> quite hard as we have to save/restore areas under the cursor. I remember > >> that anholt had an idea concerning this, but I do not remember details. > > > > I really like the idea of having this in the kernel for latency reasons, > > but yeah we do need to solve the sw case as well as implementing some > > acceleration code. OTOH it might be reasonable to push the problem of > > multiple, large, and or funky format cursors out to userspace, since > > those are relatively uncommon (64x64 32 bit ARGB ought to be enough for > > everybody right? :). > > Not for firefox. Any drag&drop in gecko will result in huge ARGB > cursors. Even just dragging and dropping a tab results in a cursor > that's at least 150x20px. Gah, yeah forgot about drag & drop of big icons... Maybe Kristian was right that all cursors should be done in software; hardware just doesn't provide the flexibility desktops want these days. -- Jesse Barnes, Intel Open Source Technology Center ___ xorg mailing list xorg@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/xorg
RE: [PATCH 0/4] Cursor's update inside kernel only
> -Original Message- > From: xorg-boun...@lists.freedesktop.org > [mailto:xorg-boun...@lists.freedesktop.org] On Behalf Of Jesse Barnes > Sent: Monday, January 19, 2009 11:03 AM > To: dri-de...@lists.sourceforge.net > Cc: xorg@lists.freedesktop.org > Subject: Re: [PATCH 0/4] Cursor's update inside kernel only > > On Friday, January 16, 2009 1:55 pm Rémi Cardona wrote: > > Not for firefox. Any drag&drop in gecko will result in huge ARGB > > cursors. Even just dragging and dropping a tab results in a cursor > > that's at least 150x20px. > > Gah, yeah forgot about drag & drop of big icons... Maybe > Kristian was right > that all cursors should be done in software; hardware just > doesn't provide > the flexibility desktops want these days. Are they (Firefox/Gecko's d&d) really X cursors or are they pixmaps tracking the cursor? If I'm moving a window, is it OK to define the cursor to be a fullsize image of that window? I'd so no, cursors are small, limited icons. In "X Window System" by Scheiler & Gettys, it says "The components of the cursor can be transformed arbitrari;y to meet display limitations" (pg 178) and "Applications should be prepared to use smaller cursors on displays that cannot support large ones" (pg 180). Mike McDonald GDC4S/FCS/PVM/SDC ___ xorg mailing list xorg@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/xorg
RE: [PATCH 0/4] Cursor's update inside kernel only
> -Original Message- > From: Jim Gettys [mailto:j...@freedesktop.org] > Sent: Monday, January 19, 2009 1:24 PM > To: McDonald, Michael-p7438c > Cc: xorg@lists.freedesktop.org > Subject: RE: [PATCH 0/4] Cursor's update inside kernel only > > At this date, it seems to me that arbitrary size cursors should be > supported > And XQueryBestCursor already provides a way for clients to > find out the > sizes that the hardware can support; toolkits/applications can become > aware of cursor size limitations if they So then XQueryBestCursor should be read as XQueryBestHWCursor? What about arbitrary depth cursor images? Is a monochrome screen suppose to handle a ARGB cursor image? Arbitrary size and depth "cursors" can already be implemented with pixmaps and pointer grabs, can't they? Mike McDonald GDC4S/FCS/PVM/SDC ___ xorg mailing list xorg@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/xorg
Re: [PATCH 0/4] Cursor's update inside kernel only
Le 19/01/2009 19:03, Jesse Barnes a écrit : > Gah, yeah forgot about drag& drop of big icons... Maybe Kristian was right > that all cursors should be done in software; hardware just doesn't provide > the flexibility desktops want these days. Maybe there could be a way to prioritize input events over rendering requests within the server, but going full software for the cursor sounds like the best solution. It's only a pixmap after all... Cheers -- Rémi Cardona LRI, INRIA remi.card...@lri.fr r...@gentoo.org ___ xorg mailing list xorg@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/xorg
Re: [PATCH 0/4] Cursor's update inside kernel only
On Tue, Jan 20, 2009 at 12:47:44AM +0100, Rémi Cardona wrote: > Le 19/01/2009 19:03, Jesse Barnes a écrit : > > Gah, yeah forgot about drag& drop of big icons... Maybe Kristian was right > > that all cursors should be done in software; hardware just doesn't provide > > the flexibility desktops want these days. > > Maybe there could be a way to prioritize input events over rendering > requests within the server, but going full software for the cursor > sounds like the best solution. It's only a pixmap after all... Input events are already hugely prioritised, and indeed done in a SIGIO handler. The main place you tend to fall apart is where the X server is paged out, and you have to wait while other parts of it get paged in. Cheers, Daniel signature.asc Description: Digital signature ___ xorg mailing list xorg@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/xorg
Re: [PATCH 0/4] Cursor's update inside kernel only
On Mon, 2009-01-19 at 10:03 -0800, Jesse Barnes wrote: > On Friday, January 16, 2009 1:55 pm Rémi Cardona wrote: > > Le 16/01/2009 21:21, Jesse Barnes a écrit : > > > On Monday, January 5, 2009 12:55 pm Tiago Vignatti wrote: > > >> Right now a thing that is annoying me is how others cursors, sw > > >> rendered, could be implemented. I want to avoid two differents sets of > > >> the same code in different contexts. IMHO conceptually all these cursor > > >> update things must be in-kernel. Painting a cursor image seems to be > > >> quite hard as we have to save/restore areas under the cursor. I remember > > >> that anholt had an idea concerning this, but I do not remember details. > > > > > > I really like the idea of having this in the kernel for latency reasons, > > > but yeah we do need to solve the sw case as well as implementing some > > > acceleration code. OTOH it might be reasonable to push the problem of > > > multiple, large, and or funky format cursors out to userspace, since > > > those are relatively uncommon (64x64 32 bit ARGB ought to be enough for > > > everybody right? :). > > > > Not for firefox. Any drag&drop in gecko will result in huge ARGB > > cursors. Even just dragging and dropping a tab results in a cursor > > that's at least 150x20px. > > Gah, yeah forgot about drag & drop of big icons... Maybe Kristian was right > that all cursors should be done in software; hardware just doesn't provide > the flexibility desktops want these days. Yeah, I suspect allowing the compositing manager to render cursors might be more useful in the long term. Then we 'just' need to make sure the composited cursors don't lag more than one frame behind the input devices. -- Earthling Michel Dänzer |http://www.vmware.com Libre software enthusiast | Debian, X and DRI developer ___ xorg mailing list xorg@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/xorg
Re: [PATCH 0/4] Cursor's update inside kernel only
Hi, On Mon, Jan 05, 2009 at 06:55:50PM -0200, Tiago Vignatti wrote: > Under KMS, we can build a feature to update the cursor directly to > screen without the continuous intervention of the userspace > application (X server, wayland, etc). It's a fastpath for DRM based > cursors obtained by short-circuit the kernel input layer and DRM > module. This would solve all cursor's latency issues that we have in > our current model [0]. IHMO punting and pushing this into the kernel in totally the wrong approach. Following this method, you will end up running *everything* in kernel space. (Hi, DOS!) Rather, you should try to figure out *why* the update in user space has latencies not present when doing it in kernel space. Is it because all kernel memory is pinned? (Not true on all systems, BTW.) Is it because event delivery between kernel space and user space triggers bad CPU scheduling? Is it because the handling in user space is not properly prioritized? Once you know the real caus(es), you can go about fixing them -- adapting the design of the user space part, and possibly creating now kernel interfaces is necessary. Ultimately, the kernel/user space distinction is just an address space boundary. There is no fundamental reason why that should cause any latencies. -antrik- ___ xorg mailing list xorg@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/xorg
Re: [PATCH 0/4] Cursor's update inside kernel only
Hi, On Sun, Jan 25, 2009 at 12:28:37PM +0200, Pekka Paalanen wrote: > Consider a case where user space is stalled due to excessive load, and > let's think about usability. Much of usability comes from feedback > given to a user. > > If cursor updates are done completely inside the kernel, the mouse > will continue to work without any hiccups under severe load (this is > what you are aiming for, right?). The user clicks a button, and > nothing happens in the GUI, since user space is stalled. The user > clicks again. And again. Then clicks another button. It takes several > seconds for the user to realize, that the clicks are not getting > processed. What's worse, all the clicks are probably queued now and > will be processed later, possibly leading to unexpected results. > > If cursor updates had to visit user space, the mouse cursor would > stall and jump. This is bad behaviour in itself, but it is also an > immediate feedback to the user, that the system is not responsive. The > user cannot even reach a button to click it, before he sees that > something is going on. In a bad situation, I think this is the less > evil choice. I totally agree that when the GUI is totally stalled, the curser better be stalled as well... But this is actually a rather untypical case. Much more often the cursor just becomes somewhat jerky under load. The GUI becomes sluggish, but still quite usable. The jerky cursor however makes it much harder to work with it -- a smooth cursor indeed improves usability in this case. On top of that, there is an important psychological effect: With a jerky cursor, the system *feels* much slower than it actually is. Users think, "This X stuff is reall crap: OS X/Windows/whatever is so much smoother..." Sad but true. So ideally, the cursor should remain smooth under some load, but stall when the system is really busy... Any suggestions how to achieve that? :-) -antrik- ___ xorg mailing list xorg@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/xorg
Re: [PATCH 0/4] Cursor's update inside kernel only
On Sunday, January 25, 2009 2:28 am Pekka Paalanen wrote: > On Mon, 5 Jan 2009 18:55:50 -0200 > > Tiago Vignatti wrote: > > Hi guys, > > > > Under KMS, we can build a feature to update the cursor directly to screen > > without the continuous intervention of the userspace application (X > > server, wayland, etc). It's a fastpath for DRM based cursors obtained by > > short-circuit the kernel input layer and DRM module. This would solve all > > cursor's latency issues that we have in our current model [0]. > > Reducing latency is a good idea, but I don't think circumventing user space > altogether is so good. Consider a case where user space is stalled due to > excessive load, and let's think about usability. Much of usability comes > from feedback given to a user. > > If cursor updates are done completely inside the kernel, the mouse will > continue to work without any hiccups under severe load (this is what you > are aiming for, right?). The user clicks a button, and nothing happens in > the GUI, since user space is stalled. The user clicks again. And again. > Then clicks another button. It takes several seconds for the user to > realize, that the clicks are not getting processed. What's worse, all the > clicks are probably queued now and will be processed later, possibly > leading to unexpected results. > > If cursor updates had to visit user space, the mouse cursor would stall > and jump. This is bad behaviour in itself, but it is also an immediate > feedback to the user, that the system is not responsive. The user cannot > even reach a button to click it, before he sees that something is going on. > In a bad situation, I think this is the less evil choice. SGI did some use case studies on this. The conclusion they drew was that the cursor should *always* be responsive, and so that's what they did back in the old days on IRIX. Unfortunately we don't have access to that data so we don't know the circumstances under which they tested users, or what the tradeoffs were. But personally I think it sucks when the pointer jumps around, no matter what the situation is with the machine. It makes the whole OS seem cheap and crappy. And for a more "typical" user, I still think smooth movement makes sense. We already have other ways of communicating "system/program busy" than cursor movement (greyed out windows for instance), so I don't buy that argument either. -- Jesse Barnes, Intel Open Source Technology Center ___ xorg mailing list xorg@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/xorg
