Re: BSOD
On Tue, 2007-03-20 at 20:19 +1100, Paul Mackerras wrote: > Anything is better than nothing. At the moment we get nothing if you > are in X when the panic occurs, even for the nicest, most well-behaved > panics. :) If we can change that to getting something sometimes, > that's a win. Treating the display as a frame buffer if at all possible is more likely to work than anything else. All the kernel needs to know is beginning address, stride (# bytes/line; not necessarily the same as the number of pixels on a row), number of rows, and depth of the display. Thereafter, it's just a SMOP as they say (and the sometimes maligned fbdev driver of course already has all this code, and usually oops messages do ok on fbdev). The other interesting headache that comes up sometimes is when the oops message is more than a screen-full long. One may also need to reset lookup tables to some sane value, and halt the GPU, and maybe worry about video overlays. - Jim -- Jim Gettys One Laptop Per Child - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: BSOD
On Tue, 2007-03-20 at 20:19 +1100, Paul Mackerras wrote: Anything is better than nothing. At the moment we get nothing if you are in X when the panic occurs, even for the nicest, most well-behaved panics. :) If we can change that to getting something sometimes, that's a win. Treating the display as a frame buffer if at all possible is more likely to work than anything else. All the kernel needs to know is beginning address, stride (# bytes/line; not necessarily the same as the number of pixels on a row), number of rows, and depth of the display. Thereafter, it's just a SMOP as they say (and the sometimes maligned fbdev driver of course already has all this code, and usually oops messages do ok on fbdev). The other interesting headache that comes up sometimes is when the oops message is more than a screen-full long. One may also need to reset lookup tables to some sane value, and halt the GPU, and maybe worry about video overlays. - Jim -- Jim Gettys One Laptop Per Child - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [linux-pm] [2/6] 2.6.21-rc2: known regressions
On Mon, 2007-03-19 at 16:33 -0400, Bill Davidsen wrote:
>
> What you say sounds good, assuming that the cost of a sleep is less than
> the cost of the busy wait. But this may be hardware, the waits may be
> very small and frequent, and if it's hitting a small hardware window
> like retrace, delays in response will cause the time period to be missed
> completely. This probably less critical with very smart cards, many of
> us don't run them.
> >
Actually, various strategies involving short busy waiting, or looking at
DMA address registers before sleeping were commonplace. But a
syscall/sleep/wakeup is/was pretty fast. If you have an operation
blitting the screen (e.g. scrolling), it takes a bit of time for the GPU
to execute the command. I see this right now on OLPC, where a wonderful
music application needs to scroll (most of) the screen left),
periodically, and we're losing samples sometimes at those operation.
Remember also, that being nice to everyone else by sleeping, there are
more cycles to go around, and the scheduler can nicely boost the X
server's priority as it will for "interactive" processes that are being
cooperative.
- Jim
--
Jim Gettys
One Laptop Per Child
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: BSOD
I agree with David and Jessie. All the kernel needs to know for oopses is frame buffer depth, start, stride, and how to get the GPU to "stop" (or it may very well overwrite the screen). Trying to get back to a default mode setting first in the middle of oopsing is very unlikely to succeed. The time you want mode switching (or lack othereof) is at suspend/resume time, not at oopses. This presumes the frame buffer is accessible in the address space of the kernel. There have been display devices where this is not true; but not many. - Jim On Mon, 2007-03-19 at 13:05 -0700, David Miller wrote: > From: Jesse Barnes <[EMAIL PROTECTED]> > Date: Mon, 19 Mar 2007 12:54:36 -0700 > > > Kernel based modesetting should get us a lot of things: > > But for panics you're ignoring what Peter and I are saying. > > Mode setting is complex and it is not going to work exactly when you > need the kernel crash message the most. > > After debugging the kernel for 10+ years I can tell you one thing, for > a bad crash what's going to happen is you'll get the printk but that's > about all that will work at that point, and the kernel is going to > hang next. Sometimes you won't get the whole panic message, just > the beginning, even with the most simplistic printk implementation. > > You will not, I repeat, will not be able to mode switch or anything > non-trivial like that when the kernel is in this state. > > Mode switching on panic, just say no. :-) > -- Jim Gettys One Laptop Per Child - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: BSOD
I agree with David and Jessie. All the kernel needs to know for oopses is frame buffer depth, start, stride, and how to get the GPU to stop (or it may very well overwrite the screen). Trying to get back to a default mode setting first in the middle of oopsing is very unlikely to succeed. The time you want mode switching (or lack othereof) is at suspend/resume time, not at oopses. This presumes the frame buffer is accessible in the address space of the kernel. There have been display devices where this is not true; but not many. - Jim On Mon, 2007-03-19 at 13:05 -0700, David Miller wrote: From: Jesse Barnes [EMAIL PROTECTED] Date: Mon, 19 Mar 2007 12:54:36 -0700 Kernel based modesetting should get us a lot of things: But for panics you're ignoring what Peter and I are saying. Mode setting is complex and it is not going to work exactly when you need the kernel crash message the most. After debugging the kernel for 10+ years I can tell you one thing, for a bad crash what's going to happen is you'll get the printk but that's about all that will work at that point, and the kernel is going to hang next. Sometimes you won't get the whole panic message, just the beginning, even with the most simplistic printk implementation. You will not, I repeat, will not be able to mode switch or anything non-trivial like that when the kernel is in this state. Mode switching on panic, just say no. :-) -- Jim Gettys One Laptop Per Child - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [linux-pm] [2/6] 2.6.21-rc2: known regressions
On Mon, 2007-03-19 at 16:33 -0400, Bill Davidsen wrote: What you say sounds good, assuming that the cost of a sleep is less than the cost of the busy wait. But this may be hardware, the waits may be very small and frequent, and if it's hitting a small hardware window like retrace, delays in response will cause the time period to be missed completely. This probably less critical with very smart cards, many of us don't run them. Actually, various strategies involving short busy waiting, or looking at DMA address registers before sleeping were commonplace. But a syscall/sleep/wakeup is/was pretty fast. If you have an operation blitting the screen (e.g. scrolling), it takes a bit of time for the GPU to execute the command. I see this right now on OLPC, where a wonderful music application needs to scroll (most of) the screen left), periodically, and we're losing samples sometimes at those operation. Remember also, that being nice to everyone else by sleeping, there are more cycles to go around, and the scheduler can nicely boost the X server's priority as it will for interactive processes that are being cooperative. - Jim -- Jim Gettys One Laptop Per Child - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [linux-pm] [2/6] 2.6.21-rc2: known regressions
On Sun, 2007-03-18 at 17:07 +0100, Ingo Molnar wrote: > * Pavel Machek <[EMAIL PROTECTED]> wrote: > > > > Some day we may have modesetting support in the kernel for some > > > graphics hw, right now it's pretty damn spotty. > > > > Yep, that's the way to go. > > hey, i wildly supported this approach ever since 1996, when GGI came up > :-/ > So wildly you wrote tons of code ;-). More seriously, at the time, XFree86 would have spat in your face for any such thing. Thankfully, times are changing. Also more seriously, a somewhat hybrid approach is in order for mode setting: simple mode setting isn't much code and is required for sane behavior on crash (it is nice to get oopses onto a screen); but the full blown mode setting/configuration problem is so large that on some hardware, it is likely left best left to a helper process (not the X server). Also key to get sane behavior out of the scheduler is to get the X server to yield (sleep in the kernel) rather than busy waiting when the GPU is busy; a standardized interface for this for both fbdev and dri is in order. Right now, X is a misbehaving compute bound process rather than the properly interactive process it can/should/will be, releasing the CPU whenever the hardware is busy. Needless to say, this wastes cycles and hurts interactivity with just about any scheduler you can devise. It isn't as if this is hard; on UNIX systems we did it in 1984 or thereabouts. Of course, in 1996, XFree86 would have ignored any such interfaces, in its insane quest for operating system independent user space drivers requiring no standard kernel interfaces (it is the second part of this where the true insanity lay). - Jim -- Jim Gettys One Laptop Per Child - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [linux-pm] [2/6] 2.6.21-rc2: known regressions
On Sun, 2007-03-18 at 17:07 +0100, Ingo Molnar wrote: * Pavel Machek [EMAIL PROTECTED] wrote: Some day we may have modesetting support in the kernel for some graphics hw, right now it's pretty damn spotty. Yep, that's the way to go. hey, i wildly supported this approach ever since 1996, when GGI came up :-/ So wildly you wrote tons of code ;-). More seriously, at the time, XFree86 would have spat in your face for any such thing. Thankfully, times are changing. Also more seriously, a somewhat hybrid approach is in order for mode setting: simple mode setting isn't much code and is required for sane behavior on crash (it is nice to get oopses onto a screen); but the full blown mode setting/configuration problem is so large that on some hardware, it is likely left best left to a helper process (not the X server). Also key to get sane behavior out of the scheduler is to get the X server to yield (sleep in the kernel) rather than busy waiting when the GPU is busy; a standardized interface for this for both fbdev and dri is in order. Right now, X is a misbehaving compute bound process rather than the properly interactive process it can/should/will be, releasing the CPU whenever the hardware is busy. Needless to say, this wastes cycles and hurts interactivity with just about any scheduler you can devise. It isn't as if this is hard; on UNIX systems we did it in 1984 or thereabouts. Of course, in 1996, XFree86 would have ignored any such interfaces, in its insane quest for operating system independent user space drivers requiring no standard kernel interfaces (it is the second part of this where the true insanity lay). - Jim -- Jim Gettys One Laptop Per Child - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: State of Linux graphics
Not at all. We're pursuing two courses of action right now, that are not mutually exclusive. Jon Smirl's argument is that we can satisfy both needs simultaneously with a GL only strategy, and that doing two is counter productive, primarily on available resource grounds. My point is that I don't think the case has (yet) been made to put all eggs into that one basket, and that some of the arguments presented for that course of action don't hold together. - Jim On Thu, 2005-09-01 at 16:39 +, Andreas Hauser wrote: > jg wrote @ Thu, 01 Sep 2005 11:59:33 -0400: > > > Legacy hardware and that being proposed/built for the developing world > > is tougher; we have code in hand for existing chips, and the price point > > is even well below cell phones on those devices. They don't have > > anything beyond basic blit and, miracles of miracles, alpha blending. > > These are built on one or two generation back fabs, again for cost. > > And as there are no carriers subsidizing the hardware cost, the real > > hardware cost has to be met, at very low price points. They don't come > > with the features Allen admires in the latest cell phone chips. > > So you suggest, that we, that have capable cards, which can be had for > < 50 Euro here, see that we find something better than X.org to run > on them because X.org is concentrating on < 10 Euro chips? > Somehow i always thought that older xfree86 trees were just fine for them. > > Andy - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: State of Linux graphics
On Thu, 2005-09-01 at 09:24 -0600, Brian Paul wrote: > > If the blending is for screen-aligned rects, glDrawPixels would be a > far easier path to optimize than texturing. The number of state > combinations related to texturing is pretty overwhelming. > > > Anyway, I think we're all in agreement about the desirability of > having a single, unified driver in the future. > Certainly for most hardware in the developed world I think we all agree with this. The argument is about when we get to one driver model, not if we get there, and not what the end state is. In my view, the battle is on legacy systems and the very low end, not in hardware we hackers use that might run Windows Vista or Mac OS X I've had the (friendly) argument with Allen Akin for 15 years that due to reduction of hardware costs we can't presume OpenGL. Someday, he'll be right, and I'll be wrong. I'm betting I'll be right for a few more years, and I nothing would tickle me pink more to lose the argument soon... Legacy hardware and that being proposed/built for the developing world is tougher; we have code in hand for existing chips, and the price point is even well below cell phones on those devices. They don't have anything beyond basic blit and, miracles of miracles, alpha blending. These are built on one or two generation back fabs, again for cost. And as there are no carriers subsidizing the hardware cost, the real hardware cost has to be met, at very low price points. They don't come with the features Allen admires in the latest cell phone chips. I think the onus of proof that we can immediately completely ditch a second driver framework in favor of everything being OpenGL, even a software tuned one, is in my view on those who claim that is viable. Waving one's hands and claiming there are 100 kbyte closed source OpenGL/ES implementations doesn't cut it in my view, given where we are today with the code we already have in hand. So far, the case hasn't been made. Existence proof that we're wrong and can move *entirely* to OpenGL sooner rather than later would be gratefully accepted.. Regards, Jim - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: State of Linux graphics
On Thu, 2005-09-01 at 09:24 -0600, Brian Paul wrote: If the blending is for screen-aligned rects, glDrawPixels would be a far easier path to optimize than texturing. The number of state combinations related to texturing is pretty overwhelming. Anyway, I think we're all in agreement about the desirability of having a single, unified driver in the future. Certainly for most hardware in the developed world I think we all agree with this. The argument is about when we get to one driver model, not if we get there, and not what the end state is. In my view, the battle is on legacy systems and the very low end, not in hardware we hackers use that might run Windows Vista or Mac OS X I've had the (friendly) argument with Allen Akin for 15 years that due to reduction of hardware costs we can't presume OpenGL. Someday, he'll be right, and I'll be wrong. I'm betting I'll be right for a few more years, and I nothing would tickle me pink more to lose the argument soon... Legacy hardware and that being proposed/built for the developing world is tougher; we have code in hand for existing chips, and the price point is even well below cell phones on those devices. They don't have anything beyond basic blit and, miracles of miracles, alpha blending. These are built on one or two generation back fabs, again for cost. And as there are no carriers subsidizing the hardware cost, the real hardware cost has to be met, at very low price points. They don't come with the features Allen admires in the latest cell phone chips. I think the onus of proof that we can immediately completely ditch a second driver framework in favor of everything being OpenGL, even a software tuned one, is in my view on those who claim that is viable. Waving one's hands and claiming there are 100 kbyte closed source OpenGL/ES implementations doesn't cut it in my view, given where we are today with the code we already have in hand. So far, the case hasn't been made. Existence proof that we're wrong and can move *entirely* to OpenGL sooner rather than later would be gratefully accepted.. Regards, Jim - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: State of Linux graphics
Not at all. We're pursuing two courses of action right now, that are not mutually exclusive. Jon Smirl's argument is that we can satisfy both needs simultaneously with a GL only strategy, and that doing two is counter productive, primarily on available resource grounds. My point is that I don't think the case has (yet) been made to put all eggs into that one basket, and that some of the arguments presented for that course of action don't hold together. - Jim On Thu, 2005-09-01 at 16:39 +, Andreas Hauser wrote: jg wrote @ Thu, 01 Sep 2005 11:59:33 -0400: Legacy hardware and that being proposed/built for the developing world is tougher; we have code in hand for existing chips, and the price point is even well below cell phones on those devices. They don't have anything beyond basic blit and, miracles of miracles, alpha blending. These are built on one or two generation back fabs, again for cost. And as there are no carriers subsidizing the hardware cost, the real hardware cost has to be met, at very low price points. They don't come with the features Allen admires in the latest cell phone chips. So you suggest, that we, that have capable cards, which can be had for 50 Euro here, see that we find something better than X.org to run on them because X.org is concentrating on 10 Euro chips? Somehow i always thought that older xfree86 trees were just fine for them. Andy - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: State of Linux graphics
Well, I'm sure you'll keep us honest... ;-). - Jim On Wed, 2005-08-31 at 12:06 -0700, Allen Akin wrote: > On Wed, Aug 31, 2005 at 01:48:11PM -0400, Jim Gettys wrote: > | Certainly replicating OpenGL 2.0's programmability through Render makes > | no sense at all to me (or most others, I believe/hope). If you want to > | use full use of the GPU, I'm happy to say you should be using OpenGL. > > When expressed that way, as a question of whether you're using the GPU > all-out, I think it's easy for everyone to agree. But we also need to > beware of the slippery slope where functionality gets duplicated a piece > at a time. > > This has already happened in several areas, so it remains a concern. > For me at least. :-) > > Allen - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: State of Linux graphics
Certainly replicating OpenGL 2.0's programmability through Render makes no sense at all to me (or most others, I believe/hope). If you want to use full use of the GPU, I'm happy to say you should be using OpenGL. - Jim On Tue, 2005-08-30 at 23:33 -0700, Allen Akin wrote: > On Tue, Aug 30, 2005 at 01:26:53PM -0400, David Reveman wrote: > | On Tue, 2005-08-30 at 12:03 -0400, Jon Smirl wrote: > | > In general, the whole concept of programmable graphics hardware is > | > not addressed in APIs like xlib and Cairo. This is a very important > | > point. A major new GPU feature, programmability is simply not > | > accessible from the current X APIs. OpenGL exposes this > | > programmability via its shader language. > | > | ... I don't > | see why this can't be exposed through the Render extension. ... > > What has always concerned me about this approach is that when you add > enough functionality to Render or some new X extensions to fully exploit > previous (much less current and in-development!) generations of GPUs, > you've essentially duplicated OpenGL 2.0. You need to identify the > resources to be managed (framebuffer objects, vertex objects, textures, > programs of several kinds, etc.); explain how they're specified and how > they interact and how they're owned/shared; define a vocabulary of > commands that operate upon them; think about how those commands are > translated and executed on various pieces of hardware; examine the > impact of things like graphics context switching on the system > architecture; and deal with a dozen other matters that have already been > addressed fully or partly in the OpenGL world. > > I think it makes a lot of sense to leverage the work that's already been > done: Take OpenGL as a given, and add extensions for what's missing. > Don't create a parallel API that in the long run must develop into > something at least as rich as OpenGL was to start with. That costs time > and effort, and likely won't be supported by the hardware vendors to the > same extent that OpenGL is (thanks to the commercial forces already at > work). Let OpenGL do 80% of the job, then work to provide the last 20%, > rather than trying to do 100% from scratch. > > Allen > ___ > xorg mailing list > [EMAIL PROTECTED] > http://lists.freedesktop.org/mailman/listinfo/xorg - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: State of Linux graphics
Certainly replicating OpenGL 2.0's programmability through Render makes no sense at all to me (or most others, I believe/hope). If you want to use full use of the GPU, I'm happy to say you should be using OpenGL. - Jim On Tue, 2005-08-30 at 23:33 -0700, Allen Akin wrote: On Tue, Aug 30, 2005 at 01:26:53PM -0400, David Reveman wrote: | On Tue, 2005-08-30 at 12:03 -0400, Jon Smirl wrote: | In general, the whole concept of programmable graphics hardware is | not addressed in APIs like xlib and Cairo. This is a very important | point. A major new GPU feature, programmability is simply not | accessible from the current X APIs. OpenGL exposes this | programmability via its shader language. | | ... I don't | see why this can't be exposed through the Render extension. ... What has always concerned me about this approach is that when you add enough functionality to Render or some new X extensions to fully exploit previous (much less current and in-development!) generations of GPUs, you've essentially duplicated OpenGL 2.0. You need to identify the resources to be managed (framebuffer objects, vertex objects, textures, programs of several kinds, etc.); explain how they're specified and how they interact and how they're owned/shared; define a vocabulary of commands that operate upon them; think about how those commands are translated and executed on various pieces of hardware; examine the impact of things like graphics context switching on the system architecture; and deal with a dozen other matters that have already been addressed fully or partly in the OpenGL world. I think it makes a lot of sense to leverage the work that's already been done: Take OpenGL as a given, and add extensions for what's missing. Don't create a parallel API that in the long run must develop into something at least as rich as OpenGL was to start with. That costs time and effort, and likely won't be supported by the hardware vendors to the same extent that OpenGL is (thanks to the commercial forces already at work). Let OpenGL do 80% of the job, then work to provide the last 20%, rather than trying to do 100% from scratch. Allen ___ xorg mailing list [EMAIL PROTECTED] http://lists.freedesktop.org/mailman/listinfo/xorg - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: State of Linux graphics
Well, I'm sure you'll keep us honest... ;-). - Jim On Wed, 2005-08-31 at 12:06 -0700, Allen Akin wrote: On Wed, Aug 31, 2005 at 01:48:11PM -0400, Jim Gettys wrote: | Certainly replicating OpenGL 2.0's programmability through Render makes | no sense at all to me (or most others, I believe/hope). If you want to | use full use of the GPU, I'm happy to say you should be using OpenGL. When expressed that way, as a question of whether you're using the GPU all-out, I think it's easy for everyone to agree. But we also need to beware of the slippery slope where functionality gets duplicated a piece at a time. This has already happened in several areas, so it remains a concern. For me at least. :-) Allen - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: X15 alpha release: as fast as TUX but in user space (fwd)
> > Short summary: depending on how much you were talking general idea versus > specifics, you can go arbitrarily far back (I wouldn't be surprised if > shared memory techniques were used regularly before memory protection.) > > Fair? Very fair. > > Not to pick on you or anyone else, but it is well-known to everyone > except the U.S. patent office that "there are no new ideas in computer > science." :) > Exactly why I noted in my mail that I didn't consider it novel even back then; just a good engineering idea that we went ahead and used a long time ago... - Jim -- Jim Gettys Technology and Corporate Development Compaq Computer Corporation [EMAIL PROTECTED] - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: #define HZ 1024 -- negative effects?
The biggest single issue in GUI responsiveness on Linux has been caused
by XFree86's implementation of mouse tracking in user space.
On typical UNIX systems, the mouse was often controlled in the kernel
driver. Until recently (XFree86 4.0 days), the XFree86 server's reads
of mouse/keyboard events were not signal driven, so that if the X server
was loaded, the cursor stopped moving.
On most (but not all) current XFree86 implementations, this is now
signal drive, and further the internal X schedular has been reworked to
make it difficult for a single client to monopolize the X server.
So the first thing you should try is to make sure you are using an X server
with this "silken mouse" enabled; anotherwords, run XFree86 4.0x and make
sure the implementation has it enabled
There may be more to do in Linux thereafter, but until you've done this, you
don't get to discuss the matter further
- Jim Gettys
--
Jim Gettys
Technology and Corporate Development
Compaq Computer Corporation
[EMAIL PROTECTED]
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: X15 alpha release: as fast as TUX but in user space (fwd)
The "put the time into a magic location in shared memory" goes back, as
far as I know, to Bob Scheifler or myself for the X Window System, sometime
around 1984 or 1985: we put it into a page of shared memory where we used
a circular buffer scheme to put input events (keyboard/mice), so that
we could avoid the read system call overhead to get these events (and
more importantly, check between each request if there was input to
process). I don't think we ever claimed it was novel, just that we did
it that way (I'd have to ask Bob if he had heard of that before we did
it). We put it into the same piece of memory we put the circular event
buffer, avoiding both the get-time-of day calls, but also the much more
expensive reads that would have been required (we put the events into a
circular buffer, with the kernel only updating one value, and user space
updating the other value defining the circular buffer).
In X, it is important for interactivity to get input events and send them
to clients ASAP: just note the effect of Keith Packard's recent implementation
of "silken mouse", where signals are used to deliver events to the X server.
This finally has made mouse tracking (done in user space on Linux; generally
done by kernel drivers on most UNIX boxes) what we were getting on 1 mip machines
under load (Keith has also done more than this with his new internal X
scheduler, which prevents clients from monopolizing the X server anywhere
like the old implementation).
This shared memory technique is very powerful to allow a client application to know if
it needs to do a system call, and is very useful for high performance servers
(like X), where a system call is way too expensive.
I've certainly mentioned this technique in the past in the Web community
(but HTTP servers are processing requests about 1/100-1/1000 the rate of
an X server, which gets into the millions of requests/second on current machines.
So if you want to get user space to really go fast, sometimes you resort
to such trickery I think the technique has real value: the interesting
question is should there be general kernel facilities to make this easy
(we did it via ugly hacks on VAX and MIPS boxes) for kernel facilities
to provide.
"X is an exercise in avoiding system calls". I think I said this around
1984-1985.
- Jim
--
Jim Gettys
Technology and Corporate Development
Compaq Computer Corporation
[EMAIL PROTECTED]
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: X15 alpha release: as fast as TUX but in user space (fwd)
The put the time into a magic location in shared memory goes back, as far as I know, to Bob Scheifler or myself for the X Window System, sometime around 1984 or 1985: we put it into a page of shared memory where we used a circular buffer scheme to put input events (keyboard/mice), so that we could avoid the read system call overhead to get these events (and more importantly, check between each request if there was input to process). I don't think we ever claimed it was novel, just that we did it that way (I'd have to ask Bob if he had heard of that before we did it). We put it into the same piece of memory we put the circular event buffer, avoiding both the get-time-of day calls, but also the much more expensive reads that would have been required (we put the events into a circular buffer, with the kernel only updating one value, and user space updating the other value defining the circular buffer). In X, it is important for interactivity to get input events and send them to clients ASAP: just note the effect of Keith Packard's recent implementation of silken mouse, where signals are used to deliver events to the X server. This finally has made mouse tracking (done in user space on Linux; generally done by kernel drivers on most UNIX boxes) what we were getting on 1 mip machines under load (Keith has also done more than this with his new internal X scheduler, which prevents clients from monopolizing the X server anywhere like the old implementation). This shared memory technique is very powerful to allow a client application to know if it needs to do a system call, and is very useful for high performance servers (like X), where a system call is way too expensive. I've certainly mentioned this technique in the past in the Web community (but HTTP servers are processing requests about 1/100-1/1000 the rate of an X server, which gets into the millions of requests/second on current machines. So if you want to get user space to really go fast, sometimes you resort to such trickery I think the technique has real value: the interesting question is should there be general kernel facilities to make this easy (we did it via ugly hacks on VAX and MIPS boxes) for kernel facilities to provide. X is an exercise in avoiding system calls. I think I said this around 1984-1985. - Jim -- Jim Gettys Technology and Corporate Development Compaq Computer Corporation [EMAIL PROTECTED] - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: #define HZ 1024 -- negative effects?
The biggest single issue in GUI responsiveness on Linux has been caused by XFree86's implementation of mouse tracking in user space. On typical UNIX systems, the mouse was often controlled in the kernel driver. Until recently (XFree86 4.0 days), the XFree86 server's reads of mouse/keyboard events were not signal driven, so that if the X server was loaded, the cursor stopped moving. On most (but not all) current XFree86 implementations, this is now signal drive, and further the internal X schedular has been reworked to make it difficult for a single client to monopolize the X server. So the first thing you should try is to make sure you are using an X server with this silken mouse enabled; anotherwords, run XFree86 4.0x and make sure the implementation has it enabled There may be more to do in Linux thereafter, but until you've done this, you don't get to discuss the matter further - Jim Gettys -- Jim Gettys Technology and Corporate Development Compaq Computer Corporation [EMAIL PROTECTED] - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: X15 alpha release: as fast as TUX but in user space (fwd)
Short summary: depending on how much you were talking general idea versus specifics, you can go arbitrarily far back (I wouldn't be surprised if shared memory techniques were used regularly before memory protection.) Fair? Very fair. Not to pick on you or anyone else, but it is well-known to everyone except the U.S. patent office that there are no new ideas in computer science. :) Exactly why I noted in my mail that I didn't consider it novel even back then; just a good engineering idea that we went ahead and used a long time ago... - Jim -- Jim Gettys Technology and Corporate Development Compaq Computer Corporation [EMAIL PROTECTED] - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] Single user linux
Not to mention fold up keyboard, IBM microdrive, etc. So you
can run the ARM Debian distro either via NFS (with the problems that
entails), or even locally on a microdrive (or I suppose you could
also play with an IDE or SCSI controller if you were really insane).
On the kernel software side, we also have IPV6/mobile IP running. We're
using Dave Woodhouse's JFFS2 with compression for our file system (Compressed
journalling flash file system) on flash.
In terms of apps, various PIM stuff, though needs lots of work,
other goodies like GPS applications, etc. Mozilla in previous versions
has been known to work. Tons of games, doom, etc.
MP3 players (at least 3). Gnome core libraries.
Python, Java 2 standard edition, swing, all running etc.
Lots of work/fun left to do, of course, in all areas.
Shall we just say we're having lots and lots and lots of fun :-).
These are real computers.
Lots of dust in the air: lots should have settled by June. In particular,
look at the Familiar work.
See www.handhelds.org. I apologize about the state of our web site:
I've done much of the maintenance in the past, but I've been out for some
surgery and life has been insane ever since. Most of the interesting
stuff is in the Wiki. And iPAQ's are not as unobtanium as they once were:
we're in really high volume production (>100K/month) but demand still
outstrips supply (sigh...).
Come join the party...
- Jim Gettys
> Sender: [EMAIL PROTECTED]
> From: Disconnect <[EMAIL PROTECTED]>
> Date: Wed, 25 Apr 2001 10:17:55 -0400
> To: Ronald Bultje <[EMAIL PROTECTED]>
> Cc: [EMAIL PROTECTED]
> Subject: Re: [PATCH] Single user linux
> -
> On Wed, 25 Apr 2001, Ronald Bultje did have cause to say:
>
> > Who says it needs to compile? Who says it needs software installed? Who
> > says it needs to run the software itself?
>
> My current project (and I'm just waiting for nfs and wvlan_cs to stabalize
> on ARM before putting the final touches on it) is an ipaq nfsrooted to a
> Debian image, over the wireless lan. Works like a champ, and it -does-
> compile stuff reasonably fast (well, reasonably fast considering the data
> is all on the far side of 11M/sec wireless.) My kit is mostly portable as
> well, since the nfs server is on the libretto and runs just fine in my
> backpack ;)
>
> The next step is bludgeoning debian-arm into not running 50-100 little
> servers I don't need on my PIM. But that may be the function of a
> task-nfs-ipaq package or some such.
>
> So far -multiuser- linux on PIMs ("true" linux, with X, etc, as distinct
> from pocketlinux/qpe/etc, which are a different animal in this case) is
> almost there. Web browsers are coming along nicely (and remote-X netscape
> is usable, although barely) and there are several nice imap clients. (and
> input methods ranging from a handwriting system to a little onscreen
> keyboard, if you are in a situation where an external keyboard is not
> feasable.)
>
> ---
--
Jim Gettys
Technology and Corporate Development
Compaq Computer Corporation
[EMAIL PROTECTED]
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] Single user linux
Not to mention fold up keyboard, IBM microdrive, etc. So you can run the ARM Debian distro either via NFS (with the problems that entails), or even locally on a microdrive (or I suppose you could also play with an IDE or SCSI controller if you were really insane). On the kernel software side, we also have IPV6/mobile IP running. We're using Dave Woodhouse's JFFS2 with compression for our file system (Compressed journalling flash file system) on flash. In terms of apps, various PIM stuff, though needs lots of work, other goodies like GPS applications, etc. Mozilla in previous versions has been known to work. Tons of games, doom, etc. MP3 players (at least 3). Gnome core libraries. Python, Java 2 standard edition, swing, all running etc. Lots of work/fun left to do, of course, in all areas. Shall we just say we're having lots and lots and lots of fun :-). These are real computers. Lots of dust in the air: lots should have settled by June. In particular, look at the Familiar work. See www.handhelds.org. I apologize about the state of our web site: I've done much of the maintenance in the past, but I've been out for some surgery and life has been insane ever since. Most of the interesting stuff is in the Wiki. And iPAQ's are not as unobtanium as they once were: we're in really high volume production (100K/month) but demand still outstrips supply (sigh...). Come join the party... - Jim Gettys Sender: [EMAIL PROTECTED] From: Disconnect [EMAIL PROTECTED] Date: Wed, 25 Apr 2001 10:17:55 -0400 To: Ronald Bultje [EMAIL PROTECTED] Cc: [EMAIL PROTECTED] Subject: Re: [PATCH] Single user linux - On Wed, 25 Apr 2001, Ronald Bultje did have cause to say: Who says it needs to compile? Who says it needs software installed? Who says it needs to run the software itself? My current project (and I'm just waiting for nfs and wvlan_cs to stabalize on ARM before putting the final touches on it) is an ipaq nfsrooted to a Debian image, over the wireless lan. Works like a champ, and it -does- compile stuff reasonably fast (well, reasonably fast considering the data is all on the far side of 11M/sec wireless.) My kit is mostly portable as well, since the nfs server is on the libretto and runs just fine in my backpack ;) The next step is bludgeoning debian-arm into not running 50-100 little servers I don't need on my PIM. But that may be the function of a task-nfs-ipaq package or some such. So far -multiuser- linux on PIMs (true linux, with X, etc, as distinct from pocketlinux/qpe/etc, which are a different animal in this case) is almost there. Web browsers are coming along nicely (and remote-X netscape is usable, although barely) and there are several nice imap clients. (and input methods ranging from a handwriting system to a little onscreen keyboard, if you are in a situation where an external keyboard is not feasable.) --- -- Jim Gettys Technology and Corporate Development Compaq Computer Corporation [EMAIL PROTECTED] - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Subtle MM bug
> Sender: [EMAIL PROTECTED] > From: Linus Torvalds <[EMAIL PROTECTED]> > Date: Wed, 10 Jan 2001 09:03:03 -0800 (PST) > To: David Woodhouse <[EMAIL PROTECTED]> > Cc: Zlatko Calusic <[EMAIL PROTECTED]>, > "Eric W. Biederman" <[EMAIL PROTECTED]>, > Rik van Riel <[EMAIL PROTECTED]>, [EMAIL PROTECTED] > Subject: Re: Subtle MM bug > - > On Wed, 10 Jan 2001, David Woodhouse wrote: > > > > > [EMAIL PROTECTED] said: > > > The no-swap behaviour shoul dactually be pretty much identical, > > > simply because both 2.2 and 2.4 will do the same thing: just skip > > > dirty pages in the page tables because they cannot do anything about > > > them. > > > > So the VM code spends a fair amount of time scanning lists of pages which > > it really can't do anything about? > > It can do _tons_ of stuff. > > Remember, on platforms like this, one of the reasons for being low on > memory is things like running X and netscape: maybe you have 64MB of RAM > and you don't think you need a swap device, and you want to have a web > browser. > > The fact that we cannot touch _dirty_ pages doesn't mean that there's > nothing to do: instead of running out of memory we can at least make the > machine usable by dropping the text pages and the page cache.. > And pushing out old text pages is a very good idea on most embedded systems. Getting the pages back is a (relatively) cheap operation: no disk seeks, some joules spent on decompression (if on CRAMFS or other compressed file system). There is an interesting question on such devices as to whether you are better off dropping text pages or pages out of the page cache first, or to what degree... - Jim -- Jim Gettys Technology and Corporate Development Compaq Computer Corporation [EMAIL PROTECTED] - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Subtle MM bug
Sender: [EMAIL PROTECTED] From: Linus Torvalds [EMAIL PROTECTED] Date: Wed, 10 Jan 2001 09:03:03 -0800 (PST) To: David Woodhouse [EMAIL PROTECTED] Cc: Zlatko Calusic [EMAIL PROTECTED], "Eric W. Biederman" [EMAIL PROTECTED], Rik van Riel [EMAIL PROTECTED], [EMAIL PROTECTED] Subject: Re: Subtle MM bug - On Wed, 10 Jan 2001, David Woodhouse wrote: [EMAIL PROTECTED] said: The no-swap behaviour shoul dactually be pretty much identical, simply because both 2.2 and 2.4 will do the same thing: just skip dirty pages in the page tables because they cannot do anything about them. So the VM code spends a fair amount of time scanning lists of pages which it really can't do anything about? It can do _tons_ of stuff. Remember, on platforms like this, one of the reasons for being low on memory is things like running X and netscape: maybe you have 64MB of RAM and you don't think you need a swap device, and you want to have a web browser. The fact that we cannot touch _dirty_ pages doesn't mean that there's nothing to do: instead of running out of memory we can at least make the machine usable by dropping the text pages and the page cache.. And pushing out old text pages is a very good idea on most embedded systems. Getting the pages back is a (relatively) cheap operation: no disk seeks, some joules spent on decompression (if on CRAMFS or other compressed file system). There is an interesting question on such devices as to whether you are better off dropping text pages or pages out of the page cache first, or to what degree... - Jim -- Jim Gettys Technology and Corporate Development Compaq Computer Corporation [EMAIL PROTECTED] - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Framebuffer orientation
The right place to ask is "[EMAIL PROTECTED]". The answer is that if performance is not a major issue, implementing this for a given graphics card is now pretty easy, using the shadow frame buffer code. For straight frame buffers, the TinyX server will now do this "out of the box". - Jim > Sender: [EMAIL PROTECTED] > From: Dennis Noordsij <[EMAIL PROTECTED]> > Date: Tue, 21 Nov 2000 16:07:26 +0200 > To: [EMAIL PROTECTED] > Subject: Framebuffer orientation > - > Hi, > > Is there any way to use the framebuffer on the i386 architecture in a > 'portrait' way? I am using a QBE web tablet and it favours the screen in that > position. (768x1024). > > I know X can do it for a few videocards, but not this one :-) > > > Regards, > Dennis > - > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > the body of a message to [EMAIL PROTECTED] > Please read the FAQ at http://www.tux.org/lkml/ -- Jim Gettys Technology and Corporate Development Compaq Computer Corporation [EMAIL PROTECTED] - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Framebuffer orientation
The right place to ask is "[EMAIL PROTECTED]". The answer is that if performance is not a major issue, implementing this for a given graphics card is now pretty easy, using the shadow frame buffer code. For straight frame buffers, the TinyX server will now do this "out of the box". - Jim Sender: [EMAIL PROTECTED] From: Dennis Noordsij [EMAIL PROTECTED] Date: Tue, 21 Nov 2000 16:07:26 +0200 To: [EMAIL PROTECTED] Subject: Framebuffer orientation - Hi, Is there any way to use the framebuffer on the i386 architecture in a 'portrait' way? I am using a QBE web tablet and it favours the screen in that position. (768x1024). I know X can do it for a few videocards, but not this one :-) Regards, Dennis - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/ -- Jim Gettys Technology and Corporate Development Compaq Computer Corporation [EMAIL PROTECTED] - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Linux's implementation of poll() not scalable?
Note that there is another aspect to the efficiency / performance of the
select/poll style of interfaces not immediately obvious, but which occurs
as a result of how some (streaming/batching) protocols work.
An X server does not call select all that often (probably one of the two items most
frequently used that care;
though I believe getting the Apache case right is more important).
X is such a streaming protocol: it is a feature that I don't have to do
extra reads or system calls to deal with more data arriving from a client.
An X server doesn't want one event generated for each incoming TCP segment:
it merely needs to know there is data available on a file descriptor as
a binary condition. I really don't want to have to do one operation per
segment; this is less efficient than the current situation.
Similarly, it is a feature that with one call I can find out that there
is work to do on multiple file descriptors.
In short, the X server does a select, and then loops across all the file
descriptors with work to do before doing another select: the system call
overhead gets amortized across multiple clients and buffers received from
the client. As the server gets busier, it is more and more likely
that there is more than one client with work to do, and/or multiple
TCP segments have arrived to process (in the common single client
is busy case). So we make the system call less and less often
as a fraction of work done.
This has the happy consequence that the select caused overhead DROPS as
a fraction of total work as the X server gets busier, and X is most efficient
at the point in time you care the most: when you have the most work to
do. The system call is returning more information each time it is called,
and some of that information is aggregated as well (additional data arriving).
It doesn't practically matter how efficient the X server is when
you aren't busy, after all.
This aggregation property is therefore important, and there needs to be
some way to achieve this, IMHO.
Web servers often have similar behavior, though since most current
HTTP clients don't implement streaming behavior, the benefit is currently
much lower (would that HTTP clients start driving HTTP servers the
way the HTTP/1.1 protocol allows... Sigh...). Right now, scaling
to large numbers of descriptors is most urgent for big web servers.
So I want an interface in which I can get as many events as possible
at once, and one in which the events themselves can have appropriate
aggregation behavior. It isn't quite clear to me if the proposed interface
would have this property.
As I said in early talks about X: "X is an exercise in avoiding system
calls"
- Jim Gettys
--
Jim Gettys
Technology and Corporate Development
Compaq Computer Corporation
[EMAIL PROTECTED]
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/
Re: Linux's implementation of poll() not scalable?
Note that there is another aspect to the efficiency / performance of the
select/poll style of interfaces not immediately obvious, but which occurs
as a result of how some (streaming/batching) protocols work.
An X server does not call select all that often (probably one of the two items most
frequently used that care;
though I believe getting the Apache case right is more important).
X is such a streaming protocol: it is a feature that I don't have to do
extra reads or system calls to deal with more data arriving from a client.
An X server doesn't want one event generated for each incoming TCP segment:
it merely needs to know there is data available on a file descriptor as
a binary condition. I really don't want to have to do one operation per
segment; this is less efficient than the current situation.
Similarly, it is a feature that with one call I can find out that there
is work to do on multiple file descriptors.
In short, the X server does a select, and then loops across all the file
descriptors with work to do before doing another select: the system call
overhead gets amortized across multiple clients and buffers received from
the client. As the server gets busier, it is more and more likely
that there is more than one client with work to do, and/or multiple
TCP segments have arrived to process (in the common single client
is busy case). So we make the system call less and less often
as a fraction of work done.
This has the happy consequence that the select caused overhead DROPS as
a fraction of total work as the X server gets busier, and X is most efficient
at the point in time you care the most: when you have the most work to
do. The system call is returning more information each time it is called,
and some of that information is aggregated as well (additional data arriving).
It doesn't practically matter how efficient the X server is when
you aren't busy, after all.
This aggregation property is therefore important, and there needs to be
some way to achieve this, IMHO.
Web servers often have similar behavior, though since most current
HTTP clients don't implement streaming behavior, the benefit is currently
much lower (would that HTTP clients start driving HTTP servers the
way the HTTP/1.1 protocol allows... Sigh...). Right now, scaling
to large numbers of descriptors is most urgent for big web servers.
So I want an interface in which I can get as many events as possible
at once, and one in which the events themselves can have appropriate
aggregation behavior. It isn't quite clear to me if the proposed interface
would have this property.
As I said in early talks about X: "X is an exercise in avoiding system
calls"
- Jim Gettys
--
Jim Gettys
Technology and Corporate Development
Compaq Computer Corporation
[EMAIL PROTECTED]
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] VM fix for 2.4.0-test9 & OOM handler
"Albert D. Cahalan" <[EMAIL PROTECTED]> writes: > Date: Mon, 9 Oct 2000 19:13:25 -0400 (EDT) > > >> From: Linus Torvalds <[EMAIL PROTECTED]> > > >> One of the biggest bitmaps is the background bitmap. So you have a > >> client that uploads it to X and then goes away. There's nobody to > >> un-count to by the time X decides to switch to another background. > > > > Actually, the big offenders are things other than the background > > bitmap: things like E do absolutely insane things, you would not > > believe (or maybe you would). The background pixmap is generally > > in the worst case typically no worse than 4 megabytes (for those > > people who are crazy enough to put images up as their root window > > on 32 bit deep displays, at 1kX1k resolution). > > Still, it would be nice to recover that 4 MB when the system > doesn't have any memory left. > Yup. The X server could give back the memory for some cases like the background without too much hackery. > X, and any other big friendly processes, could participate in > memory balancing operations. X could be made to clean out a > font cache when the kernel signals that memory is low. When > the situation becomes serious, X could just mmap /dev/zero over > top of the background image. I agree in principle, though the problem is difficult, as the memory pool may get fragmented... Most memory usage is less monolithic than the background pixmap. And maintaining separate memory pools often wastes more memory than it saves. > > Netscape could even be hacked to dump old junk... or if it is > just too leaky, it could exec itself to fix the problem. Netscape 4.x is hopeless; it is leakier than the Titanic. There is hope for Mozilla. - Jim -- Jim Gettys Technology and Corporate Development Compaq Computer Corporation [EMAIL PROTECTED] - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] VM fix for 2.4.0-test9 & OOM handler
> From: Linus Torvalds <[EMAIL PROTECTED]> > Date: Mon, 9 Oct 2000 14:50:51 -0700 (PDT) > To: Jim Gettys <[EMAIL PROTECTED]> > Cc: Alan Cox <[EMAIL PROTECTED]>, Andi Kleen <[EMAIL PROTECTED]>, > Ingo Molnar <[EMAIL PROTECTED]>, Andrea Arcangeli <[EMAIL PROTECTED]>, > Rik van Riel <[EMAIL PROTECTED]>, > Byron Stanoszek <[EMAIL PROTECTED]>, > MM mailing list <[EMAIL PROTECTED]>, [EMAIL PROTECTED] > Subject: Re: [PATCH] VM fix for 2.4.0-test9 & OOM handler > - > On Mon, 9 Oct 2000, Jim Gettys wrote: > > > > > > On Date: Mon, 9 Oct 2000 14:38:10 -0700 (PDT), Linus Torvalds > <[EMAIL PROTECTED]> > > said: > > > > > > > > The problem is that there is no way to keep track of them afterwards. > > > > > > So the process that gave X the bitmap dies. What now? Are we going to > > > depend on X un-counting the resources? > > > > > > > X has to uncount the resources already, to free the memory in the X server > > allocated on behalf of that client. X has to get this right, to be a long > > lived server (properly debugged X servers last many months without problems: > > unfortunately, a fair number of DDX's are buggy). > > No, but my point is that it doesn't really work. > > One of the biggest bitmaps is the background bitmap. So you have a client > that uploads it to X and then goes away. There's nobody to un-count to by > the time X decides to switch to another background. Actually, the big offenders are things other than the background bitmap: things like E do absolutely insane things, you would not believe (or maybe you would). The background pixmap is generally in the worst case typically no worse than 4 megabytes (for those people who are crazy enough to put images up as their root window on 32 bit deep displays, at 1kX1k resolution). > > Does that memory just disappear as far as the resource handling is > concerned when the client that originated it dies? No, X recovers the memory when a connection dies, unless the client has gone out of its way to arrange to preserve things across connection termination. Few, if any clients do this: it is primarily possible mostly for debugging purposes, that (fortunately, or unfortunately, depending on your opinion) what happens not just vanish before you can see what happened. So the X server does extensive bookkeeping of its memory usage, and retrieves all memory used by clients when they terminate (with the above rare exception). > > What happens with TCP connections? They might be local. Or they might not. > In either case X doesn't know whom to blame. At least on BSD kernels, it was reasonably straightforward to determine if a TCP connection was local: in that case, the code actually did an upcall and delivered data directly to the appropriate socket. Dunno about the insides of Linux. I suspect it should not be hard to find the right process for local connections. Distant connections are, indeed, a challenge. > > Basically, the only thing _I_ think X can do is to really say "oh, please > don't count my memory, because everything I do I do for my clients, not > for myself". > > THAT is my argument. Basically there is nothing we can reliably account. Your argument has alot of validity, though the X server does a better job of accounting than you might think. BUT, I'm actually more interested in dealing with scheduling preferences, to get really first rate interactive feel. > > So we might as well fall back on just saying "X is more important than > some random client", and have a mm niceness level. Which right now is > obviously approximated by the IO capabilities tests etc. > As I say above, the principle here may be more useful than for the memory example, but for controlling scheduling so we can get great interactive feel. THAT is what is really worth discussing. - Jim -- Jim Gettys Technology and Corporate Development Compaq Computer Corporation [EMAIL PROTECTED] - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] VM fix for 2.4.0-test9 & OOM handler
On Date: Mon, 9 Oct 2000 14:38:10 -0700 (PDT), Linus Torvalds <[EMAIL PROTECTED]>
said:
>
> The problem is that there is no way to keep track of them afterwards.
>
> So the process that gave X the bitmap dies. What now? Are we going to
> depend on X un-counting the resources?
>
X has to uncount the resources already, to free the memory in the X server
allocated on behalf of that client. X has to get this right, to be a long
lived server (properly debugged X servers last many months without problems:
unfortunately, a fair number of DDX's are buggy).
- Jim
--
Jim Gettys
Technology and Corporate Development
Compaq Computer Corporation
[EMAIL PROTECTED]
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] VM fix for 2.4.0-test9 & OOM handler
> > Sounds like one needs in addition some mechanism for servers to "charge" > clients for > > consumption. X certainly knows on behalf of which connection resources > > are created; the OS could then transfer this back to the appropriate client > > (at least when on machine). > > Definitely - and this is present in some non Unix OS's. We do pass credentials > across AF_UNIX sockets so the mechanism is notionally there to provide the > credentials to X, just not to use them Stephen Tweedie, Dave Rosenthal, Keith Packard and myself had an extensive discussion on similar ideas around process quantum scheduling (the X server would like to be able to forward quantum to clients) as well at Usenix. This is closely related, and needed to finally fully control interactive feel in the face of "greedy" clients. My memory is that it sounded like things could become very interesting with such a facility, and might be ripe for 2.5. Keith, Stephen, Dave, do you remember the details of our discussion? - Jim -- Jim Gettys Technology and Corporate Development Compaq Computer Corporation [EMAIL PROTECTED] - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] VM fix for 2.4.0-test9 & OOM handler
> Sender: [EMAIL PROTECTED] > From: "Andi Kleen" <[EMAIL PROTECTED]> > Date: Mon, 9 Oct 2000 22:58:22 +0200 > To: Linus Torvalds <[EMAIL PROTECTED]> > Cc: Andi Kleen <[EMAIL PROTECTED]>, Ingo Molnar <[EMAIL PROTECTED]>, > Andrea Arcangeli <[EMAIL PROTECTED]>, > Rik van Riel <[EMAIL PROTECTED]>, > Byron Stanoszek <[EMAIL PROTECTED]>, > MM mailing list <[EMAIL PROTECTED]>, [EMAIL PROTECTED] > Subject: Re: [PATCH] VM fix for 2.4.0-test9 & OOM handler > - > On Mon, Oct 09, 2000 at 01:52:21PM -0700, Linus Torvalds wrote: > > One thing we _can_ (and probably should do) is to do a per-user memory > > pressure thing - we have easy access to the "struct user_struct" (every > > process has a direct pointer to it), and it should not be too bad to > > maintain a per-user "VM pressure" counter. > > > > Then, instead of trying to use heuristics like "does this process have > > children" etc, you'd have things like "is this user a nasty user", which > > is a much more valid thing to do and can be used to find people who fork > > tons of processes that are mid-sized but use a lot of memory due to just > > being many.. > > Would not help much when "they" eat your memory by loading big bitmaps > into the X server which runs as root (it seems there are many programs > which are very good at this particular DOS ;) > This is generic to any server program, not unique to X. Sounds like one needs in addition some mechanism for servers to "charge" clients for consumption. X certainly knows on behalf of which connection resources are created; the OS could then transfer this back to the appropriate client (at least when on machine). - Jim -- Jim Gettys Technology and Corporate Development Compaq Computer Corporation [EMAIL PROTECTED] - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] VM fix for 2.4.0-test9 OOM handler
Sender: [EMAIL PROTECTED] From: "Andi Kleen" [EMAIL PROTECTED] Date: Mon, 9 Oct 2000 22:58:22 +0200 To: Linus Torvalds [EMAIL PROTECTED] Cc: Andi Kleen [EMAIL PROTECTED], Ingo Molnar [EMAIL PROTECTED], Andrea Arcangeli [EMAIL PROTECTED], Rik van Riel [EMAIL PROTECTED], Byron Stanoszek [EMAIL PROTECTED], MM mailing list [EMAIL PROTECTED], [EMAIL PROTECTED] Subject: Re: [PATCH] VM fix for 2.4.0-test9 OOM handler - On Mon, Oct 09, 2000 at 01:52:21PM -0700, Linus Torvalds wrote: One thing we _can_ (and probably should do) is to do a per-user memory pressure thing - we have easy access to the "struct user_struct" (every process has a direct pointer to it), and it should not be too bad to maintain a per-user "VM pressure" counter. Then, instead of trying to use heuristics like "does this process have children" etc, you'd have things like "is this user a nasty user", which is a much more valid thing to do and can be used to find people who fork tons of processes that are mid-sized but use a lot of memory due to just being many.. Would not help much when "they" eat your memory by loading big bitmaps into the X server which runs as root (it seems there are many programs which are very good at this particular DOS ;) This is generic to any server program, not unique to X. Sounds like one needs in addition some mechanism for servers to "charge" clients for consumption. X certainly knows on behalf of which connection resources are created; the OS could then transfer this back to the appropriate client (at least when on machine). - Jim -- Jim Gettys Technology and Corporate Development Compaq Computer Corporation [EMAIL PROTECTED] - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] VM fix for 2.4.0-test9 OOM handler
Sounds like one needs in addition some mechanism for servers to "charge"
clients for
consumption. X certainly knows on behalf of which connection resources
are created; the OS could then transfer this back to the appropriate client
(at least when on machine).
Definitely - and this is present in some non Unix OS's. We do pass credentials
across AF_UNIX sockets so the mechanism is notionally there to provide the
credentials to X, just not to use them
Stephen Tweedie, Dave Rosenthal, Keith Packard and myself had an extensive
discussion on similar ideas around process quantum scheduling (the X server
would like to be able to forward quantum to clients) as well at Usenix.
This is closely related, and needed to finally fully control interactive
feel in the face of "greedy" clients.
My memory is that it sounded like things could become very interesting
with such a facility, and might be ripe for 2.5.
Keith, Stephen, Dave, do you remember the details of our discussion?
- Jim
--
Jim Gettys
Technology and Corporate Development
Compaq Computer Corporation
[EMAIL PROTECTED]
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] VM fix for 2.4.0-test9 OOM handler
On Date: Mon, 9 Oct 2000 14:38:10 -0700 (PDT), Linus Torvalds [EMAIL PROTECTED] said: The problem is that there is no way to keep track of them afterwards. So the process that gave X the bitmap dies. What now? Are we going to depend on X un-counting the resources? X has to uncount the resources already, to free the memory in the X server allocated on behalf of that client. X has to get this right, to be a long lived server (properly debugged X servers last many months without problems: unfortunately, a fair number of DDX's are buggy). - Jim -- Jim Gettys Technology and Corporate Development Compaq Computer Corporation [EMAIL PROTECTED] - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] VM fix for 2.4.0-test9 OOM handler
From: Linus Torvalds [EMAIL PROTECTED]
Date: Mon, 9 Oct 2000 14:50:51 -0700 (PDT)
To: Jim Gettys [EMAIL PROTECTED]
Cc: Alan Cox [EMAIL PROTECTED], Andi Kleen [EMAIL PROTECTED],
Ingo Molnar [EMAIL PROTECTED], Andrea Arcangeli [EMAIL PROTECTED],
Rik van Riel [EMAIL PROTECTED],
Byron Stanoszek [EMAIL PROTECTED],
MM mailing list [EMAIL PROTECTED], [EMAIL PROTECTED]
Subject: Re: [PATCH] VM fix for 2.4.0-test9 OOM handler
-
On Mon, 9 Oct 2000, Jim Gettys wrote:
On Date: Mon, 9 Oct 2000 14:38:10 -0700 (PDT), Linus Torvalds
[EMAIL PROTECTED]
said:
The problem is that there is no way to keep track of them afterwards.
So the process that gave X the bitmap dies. What now? Are we going to
depend on X un-counting the resources?
X has to uncount the resources already, to free the memory in the X server
allocated on behalf of that client. X has to get this right, to be a long
lived server (properly debugged X servers last many months without problems:
unfortunately, a fair number of DDX's are buggy).
No, but my point is that it doesn't really work.
One of the biggest bitmaps is the background bitmap. So you have a client
that uploads it to X and then goes away. There's nobody to un-count to by
the time X decides to switch to another background.
Actually, the big offenders are things other than the background bitmap:
things like E do absolutely insane things, you would not believe (or maybe
you would). The background pixmap is generally in the worst case typically
no worse than 4 megabytes (for those people who are crazy enough to put
images up as their root window on 32 bit deep displays, at 1kX1k resolution).
Does that memory just disappear as far as the resource handling is
concerned when the client that originated it dies?
No, X recovers the memory when a connection dies, unless the client has
gone out of its way to arrange to preserve things across connection
termination. Few, if any clients do this: it is primarily possible mostly
for debugging purposes, that (fortunately, or unfortunately, depending
on your opinion) what happens not just vanish before you can see what
happened.
So the X server does extensive bookkeeping of its memory usage, and retrieves
all memory used by clients when they terminate (with the above rare
exception).
What happens with TCP connections? They might be local. Or they might not.
In either case X doesn't know whom to blame.
At least on BSD kernels, it was reasonably straightforward to determine
if a TCP connection was local: in that case, the code actually did an upcall
and delivered data directly to the appropriate socket. Dunno about the
insides of Linux.
I suspect it should not be hard to find the right process for local
connections. Distant connections are, indeed, a challenge.
Basically, the only thing _I_ think X can do is to really say "oh, please
don't count my memory, because everything I do I do for my clients, not
for myself".
THAT is my argument. Basically there is nothing we can reliably account.
Your argument has alot of validity, though the X server does a better job
of accounting than you might think.
BUT, I'm actually more interested in dealing with scheduling preferences, to
get really first rate interactive feel.
So we might as well fall back on just saying "X is more important than
some random client", and have a mm niceness level. Which right now is
obviously approximated by the IO capabilities tests etc.
As I say above, the principle here may be more useful than for the memory
example, but for controlling scheduling so we can get great interactive
feel. THAT is what is really worth discussing.
- Jim
--
Jim Gettys
Technology and Corporate Development
Compaq Computer Corporation
[EMAIL PROTECTED]
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] VM fix for 2.4.0-test9 OOM handler
"Albert D. Cahalan" [EMAIL PROTECTED] writes: Date: Mon, 9 Oct 2000 19:13:25 -0400 (EDT) From: Linus Torvalds [EMAIL PROTECTED] One of the biggest bitmaps is the background bitmap. So you have a client that uploads it to X and then goes away. There's nobody to un-count to by the time X decides to switch to another background. Actually, the big offenders are things other than the background bitmap: things like E do absolutely insane things, you would not believe (or maybe you would). The background pixmap is generally in the worst case typically no worse than 4 megabytes (for those people who are crazy enough to put images up as their root window on 32 bit deep displays, at 1kX1k resolution). Still, it would be nice to recover that 4 MB when the system doesn't have any memory left. Yup. The X server could give back the memory for some cases like the background without too much hackery. X, and any other big friendly processes, could participate in memory balancing operations. X could be made to clean out a font cache when the kernel signals that memory is low. When the situation becomes serious, X could just mmap /dev/zero over top of the background image. I agree in principle, though the problem is difficult, as the memory pool may get fragmented... Most memory usage is less monolithic than the background pixmap. And maintaining separate memory pools often wastes more memory than it saves. Netscape could even be hacked to dump old junk... or if it is just too leaky, it could exec itself to fix the problem. Netscape 4.x is hopeless; it is leakier than the Titanic. There is hope for Mozilla. - Jim -- Jim Gettys Technology and Corporate Development Compaq Computer Corporation [EMAIL PROTECTED] - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
