Re: Rfork'd threads, signals, and LDTs
On Sat, 5 May 2001, Daniel Eischen wrote: On Sat, 5 May 2001, Andrew Gallatin wrote: Daniel Eischen writes: OK, thanks. Here's my guess at what should be changed for the Linux emulator. If this looks correct, I'll commit it. Hmm, I wonder how linuxthreads works under FreeBSD without this change... This breaks at least one version of the IBM JDK that I have laying around.. If anything, we may have two errors (at least partially) canceling each other out. I think it needs more work/thought prior to committing. We're still OK with the change to FreeBSDs native signal trampoline though, right? I'll hold off on the Linux emulator changes until we can figure out what the problem is. I was confused about what Linux does. Now I think it only copies %fs to the signal context struct. It loads %fs with USER_DS for the signal handler. It passes the previous (process) value of %gs to the signal handler. (Its early mistake of of switching %fs on every entry to the kernel was moved to FreeBSD.) Bruce To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: ssh reports no RSA support in libssl and libcrypto.
On Mon, May 07, 2001 at 12:11:16AM -0600, Warner Losh wrote: In message [EMAIL PROTECTED] Kris Kennaway writes: : Check the archives, this is a FAQ with probably the usual answer. Hint: the answer is in UPDATING. Perhaps we could add Make sure you have loaded random.ko to the output ? /Jesper -- Jesper Skriver, jesper(at)skriver(dot)dk - CCIE #5456 Work:Network manager @ AS3292 (Tele Danmark DataNetworks) Private: FreeBSD committer @ AS2109 (A much smaller network ;-) One Unix to rule them all, One Resolver to find them, One IP to bring them all and in the zone to bind them. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: Rfork'd threads, signals, and LDTs
On Mon, 7 May 2001, Bruce Evans wrote: On Sat, 5 May 2001, Daniel Eischen wrote: On Sat, 5 May 2001, Andrew Gallatin wrote: Daniel Eischen writes: OK, thanks. Here's my guess at what should be changed for the Linux emulator. If this looks correct, I'll commit it. Hmm, I wonder how linuxthreads works under FreeBSD without this change... This breaks at least one version of the IBM JDK that I have laying around.. If anything, we may have two errors (at least partially) canceling each other out. I think it needs more work/thought prior to committing. We're still OK with the change to FreeBSDs native signal trampoline though, right? I'll hold off on the Linux emulator changes until we can figure out what the problem is. I was confused about what Linux does. Now I think it only copies %fs to the signal context struct. It loads %fs with USER_DS for the signal handler. It passes the previous (process) value of %gs to the signal handler. (Its early mistake of of switching %fs on every entry to the kernel was moved to FreeBSD.) I think the only reason we used %fs instead of %gs was WINE. I think Linux uses %gs for TSD, so if WINE were to ever depend on linuxthreads, one of them would have to change. We can easily switch to using %gs instead of %fs if necessary. I'll make the appropriate change to the Linux emulator. Thanks -- Dan Eischen To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: WinModem Support/Learning the kernel Internals
On Tue, May 01, 2001 at 02:13:05AM +0300, Tomi Vainio - Sun Finland - wrote: Benjamin Close writes: Is anyone looking into converting the Linux winmodem driver ( Lucent Technologies binary object file compiled together with the linux kernel serial driver) into a freebsd device? Please check http://www.geocities.com/wtnbkysh/ . It should work under 4.2R. There is also some 5.0C patches but still some problems probably with interrupts. Any idea who is the author of this? I also tried it on -current and it needs some refinement probably. I am willing to do that but it would be nice if the author could give me some hints. Mark -- Mark Santcroos RIPE Network Coordination Centre http://www.ripe.net/home/mark/ New Projects Group/TTM To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: pgm to kill 4.3 via vm
On Sun, 6 May 2001, Kris Kennaway wrote: On Sat, May 05, 2001 at 02:37:07PM -0700, Dennis Glatting wrote: I wrote a trivial program to fill vm and found I can reliably freeze my system. It may not work on the first attempt, but certainly within three. My command line is: a.out;a.out;a.out;a.out;a.out The goal of my program is simply to see how the system behaves under memory exhaustion which, as it turns out on two similar systems, the systems freeze. Specifically, I can switch between consoles but the login prompts do not respond and the system does not respond on the network. I am running 4.3 on a dual processor system. Below are some things. First, the program. Second, dmesg. Finally, my /etc/rc.config. What resource limits have you set? I haven't set any, so it's the default. I am intentionally testing at the limits to see what happens, usually interesting things. :) In this case, the application is well behaved (in the error proccesing sense): it'll exit, thus releasing its memory resources, when the kernel reports a memory allocation failure. (Within the last six months I commited less than 10 systems to production roles. Consequently, I'm interested in how FreeBSD behaves should any of the applications behave badly.) I've run this code on four systems, all 4.3. Two of the systems are similar SMP boxes; one a P3 233; and the last a P3 500 lap top. The code eventually crashed all systems (less than three runs) and sometimes printed an error message about a swap allocation failure. In the worst case I expect an error message, like gone fishing, bye., but that isn't the case; so my suspecion is a bug that those who know the kernel better than I might pursue. TIA To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: pgm to kill 4.3 via vm
Dennis Glatting [EMAIL PROTECTED] writes: I am intentionally testing at the limits to see what happens, usually interesting things. :) In this case, the application is well behaved (in the error proccesing sense): it'll exit, thus releasing its memory resources, when the kernel reports a memory allocation failure. No. malloc() will return NULL only if you hit a resource limit or exhaust address space. There may or may not be memory (real or virtual) available at that time. Plus, your program doesn't even do what you think it does (because a) it has at least one significant bug and b) malloc() doesn't behave the way you think it does). And even if it did, the /dev/random stuff is pointless, you can achieve the same effect by setting every byte you allocate (possibly even just the first byte of every chunk) to 0. To really test what you think your program tests, you should mmap() an amount of memory larger than RAM + swap and touch every page. Even then, the result will be a SIGSEGV, not a graceful termination. DES -- Dag-Erling Smorgrav - [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: pgm to kill 4.3 via vm
On Monday 07 May 2001 08:10 am, Dag-Erling Smorgrav wrote: Dennis Glatting [EMAIL PROTECTED] writes: I am intentionally testing at the limits to see what happens, usually interesting things. :) In this case, the application is well behaved (in the error proccesing sense): it'll exit, thus releasing its memory resources, when the kernel reports a memory allocation failure. No. malloc() will return NULL only if you hit a resource limit or exhaust address space. There may or may not be memory (real or virtual) available at that time. Isn't memory exhaustion a resource limit? Plus, your program doesn't even do what you think it does (because a) it has at least one significant bug and b) malloc() doesn't behave the way you think it does). And even if it did, the /dev/random stuff is pointless, you can achieve the same effect by setting every byte you allocate (possibly even just the first byte of every chunk) to 0. /dev/random is left over from a different test and isn't relevant. Explain the bug and malloc() behaviour. According to the malloc() man page: RETURN VALUES The malloc() and calloc() functions return a pointer to the allocated memory if successful; otherwise a NULL pointer is returned and errno is set to ENOMEM. I assert memory exhaustion is would return unsuccessful on the malloc() call, no? To really test what you think your program tests, you should mmap() an amount of memory larger than RAM + swap and touch every page. Even then, the result will be a SIGSEGV, not a graceful termination. DES To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: pgm to kill 4.3 via vm
Dennis Glatting [EMAIL PROTECTED] writes: On Monday 07 May 2001 08:10 am, Dag-Erling Smorgrav wrote: malloc() will return NULL only if you hit a resource limit or exhaust address space. There may or may not be memory (real or virtual) available at that time. Isn't memory exhaustion a resource limit? What is memory exhaustion? Namespace exhaustion will cause malloc() to fail. But FreeBSD overcommits memory, so exhaustion of physical and virtual memory will manifest itself as an unsatisfiable page fault, which will cause the kernel to start killing more-or-less-random processes until some memory is available again. Explain the bug and malloc() behaviour. According to the malloc() man page: The bug: I believe you meant i -= j when you wrote i =- j. The result is that i becomes negative and in all likelihood you smash the malloc() arena. Regarding malloc(), you seem to believe that malloc() allocates *exactly* the amount you ask, no more, no less, and actually allocates physical or virtual memory. It doesn't. First of all, it allocates memory of its own for bookkeeping purposes, which your program doesn't account for. Second, it usually allocates a little bit more than you asked for, and keeps the leftovers for later. Third, it only allocates namespace. Actual memory (physical or virtual) is allocated by the kernel the first time you touch a page. What's more, your program does extra stuff (such as needlessly fork() / exec()ing sync(1) instead of just calling sync(2), which doesn't serve any purpose anyway, since virtual memory isn't managed by the file system code, so sync(2) doesn't affect it) which consumes memory, which further invalidates your results (for instance, memory exhaustion while running sync(1) could cause the kernel to randomly kill your program) I assert memory exhaustion is would return unsuccessful on the malloc() call, no? No, just namespace exhaustion, which your program won't even come close to. DES -- Dag-Erling Smorgrav - [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
setcred and sshd trouble?
Hi, Can't seem to find the no doubtedly obvious answer to the problem with sshd after remaking world (today's -current with new openssh 2.9) Trying to log in remotely results in: May 7 17:37:45 tiggr /boot/kernel/kernel: May 7 17:37:45 tiggr sshd[718]: fatal: PAM setcred failed[6]: Permission denied which I can't seem to find a way to fix :-( Is this a side effect of pam_ssh leaving or what? Further more: /etc/rc* do not create ssh_host_rsa_key so far. Mark -- Nice testing in little China... To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: ssh reports no RSA support in libssl and libcrypto.
Peter Jeremy wrote: Having built and installed -current from last Friday, I find that ssh* is now reporting no RSA support in libssl and libcrypto. See ssl(8) and will only talk SSH2. I couldn't find anything relevant in ssl(8). I haven't specified WITH_RSA=NO anywhere and checking through the build logs, there's no sign of -DNO_RSA. libcrypto.a contains an RSA_generate_key() which doesn't look like a stub. Does anyone have any ideas? I seem to remember that I once got a misleading message just like that. It turned out that /dev/random was not available or not working properly. Cheers, Jeroen -- Jeroen C. van Gelderen - [EMAIL PROTECTED] A government that robs Peter to pay Paul can always depend upon the support of Paul. -- George Bernard Shaw To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: Rfork'd threads, signals, and LDTs
In article [EMAIL PROTECTED], Daniel Eischen [EMAIL PROTECTED] wrote: I think the only reason we used %fs instead of %gs was WINE. I think Linux uses %gs for TSD, so if WINE were to ever depend on linuxthreads, one of them would have to change. At least on Red Hat 7.0 (glibc-2.1.92-14), Linux does not use a segment register to find TSD. It aligns all stacks at a multpile of 2MB and then does bit ops on the current stack pointer to find a thread control block at the base (highest address) of the stack. There is an alternate implementation in that version of glibc which uses %gs to find TSD. However, it is not used in this version of Linux. I don't know whether it's used in other versions or not. John -- John Polstra [EMAIL PROTECTED] John D. Polstra Co., Inc.Seattle, Washington USA Disappointment is a good sign of basic intelligence. -- Chögyam Trungpa To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: pgm to kill 4.3 via vm
: The malloc() and calloc() functions return a pointer to : the allocated memory if successful; otherwise a NULL : pointer is returned and errno is set to ENOMEM. : :I assert memory exhaustion is would return unsuccessful on the :malloc() call, no? malloc() returns NULL if the process datasize resource limit is exceeded, as set by the setrlimit() system call, 'limit' csh/tcsh command, or 'ulimit' sh/bash command. You are required to set this (and other) limits to reasonable values for your machine... the machine has no clue as to when you want things to start to fail because often systems become unusable long before they run out of swap. By default the machine tries to accomodate everything. malloc() does not return NULL if the machine itself runs out of VM. mmap() currently bypasses the datasize resource limit... it does not reserve VM space from same area as malloc(). We've known this for a while and know we need to add another setrlimit type to handle it. If you use mmap() to allocate huge amounts of anonymous memory and you don't have the swap to back it up, you can crash the machine. Theoretically the system is supposed to start killing large processes when memory + swap gets full, but that code does not appear to be working as well as it should at the moment. This argument rears its head about once a year and usually turns into a huge flame war. It's one of those things that you can make quite excellent theoretical arguments for, which look great on paper, but the reality is so incompatible with the theory (read: the theoretical solutions break more people then it fixes) that the smarter people huck the theory into the dumpster and stick with the reality. -Matt To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: ssh reports no RSA support in libssl and libcrypto.
In message [EMAIL PROTECTED] Jesper Skriver writes: : On Mon, May 07, 2001 at 12:11:16AM -0600, Warner Losh wrote: : In message [EMAIL PROTECTED] Kris Kennaway writes: : : Check the archives, this is a FAQ with probably the usual answer. : : Hint: the answer is in UPDATING. : : Perhaps we could add Make sure you have loaded random.ko to the output ? There's two problems that cause this. First is not having random in the kernel. Second is not having done MAKEDEV in /dev for the random device. Warner To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: pgm to kill 4.3 via vm
On Mon, 07 May 2001 09:55:44 MST, Matt Dillon wrote: Theoretically the system is supposed to start killing large processes when memory + swap gets full, but that code does not appear to be working as well as it should at the moment. I think that's all that Dennis was questioning. All he needed was your answer. :-) This argument rears its head about once a year and usually turns into a huge flame war. I think DES was responding to that flame war, rather than to Dennis' actual question. Ciao, Sheldon. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: pgm to kill 4.3 via vm
Matt Dillon [EMAIL PROTECTED] writes: This argument rears its head about once a year and usually turns into a huge flame war. Yes, I was going to mention that - search the archives for memory overcommit and you'll find most of what I've already said in this thread, and plenty I haven't. DES -- Dag-Erling Smorgrav - [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: pgm to kill 4.3 via vm
On Mon, May 07, 2001 at 07:02:32PM +0200, Dag-Erling Smorgrav wrote: Matt Dillon [EMAIL PROTECTED] writes: This argument rears its head about once a year and usually turns into a huge flame war. Yes, I was going to mention that - search the archives for memory overcommit and you'll find most of what I've already said in this thread, and plenty I haven't. A Top UNIX Myths page anyone? Along the lines of: I have heard UNIX is the most secure OS! - No, unless you make it so and bring up the necessary expertise and time investment... I have heard you cannot crash a UNIX machine! - No, ... It would serve those people well who were only educated about UNIX-type systems during MS vs anything else flamewars... -- Regards: Szilveszter ADAM Szeged University Szeged Hungary To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: pgm to kill 4.3 via vm
:While that's a reasonable question when you're in a support role, I'd :certainly like to hear whether FreeBSD freezes on memory exhaustion is :something people should just live with or whether it's symptomatic of :a bug that someone might one day want to fix but which folks should, for :now, protect themselves against using resource limits. : :Ciao, :Sheldon. It's something I'd like to fix... but the proper solution is the simplest one, not something complex. The simplest one is to start killing bloated processes. We do this now, just not very well. The reality is that modern day disks have so much disk space compared to physical ram that the vast majority of sysads simply setup a reasonably-sized swap partition, reasonable resource limits, and wash their hands of the whole affair. A production machine with 128M of ram and 1G of swap is going to go down the tubes performance-wise long before it runs out of swap. Performance degredation under heavy memory loads is a much more interesting and important problem then swap exhaustion. Swap exhaustion is an interesting theoretical problem, and we certainly want to do something (i.e. kill processes). But that's it. -Matt To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: pgm to kill 4.3 via vm
Sheldon Hearn [EMAIL PROTECTED] writes: I think DES was responding to that flame war, rather than to Dennis' actual question. Actually, I was responding to Dennis' incorrect assumptions about FreeBSD's VM system, as exhibited by his code (which reflects the way he *thinks* the VM system works), and to the bugginess of his code. BTW, I was wrong - his code doesn't smash the malloc() arena, because the termination condition in the if loop guarantees it never makes more than one iteration. DES -- Dag-Erling Smorgrav - [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: Rfork'd threads, signals, and LDTs
On Mon, 7 May 2001, John Polstra wrote: In article [EMAIL PROTECTED], Daniel Eischen [EMAIL PROTECTED] wrote: I think the only reason we used %fs instead of %gs was WINE. I think Linux uses %gs for TSD, so if WINE were to ever depend on linuxthreads, one of them would have to change. At least on Red Hat 7.0 (glibc-2.1.92-14), Linux does not use a segment register to find TSD. It aligns all stacks at a multpile of 2MB and then does bit ops on the current stack pointer to find a thread control block at the base (highest address) of the stack. There is an alternate implementation in that version of glibc which uses %gs to find TSD. However, it is not used in this version of Linux. I don't know whether it's used in other versions or not. I was looking at our linuxthreads port and noticed some %gs fiddling. If linuxthreads wants to allow POSIX semantics for specifying thread stack allocation, they'll have to stop relying on stack alignments for TSD. -- Dan Eischen To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: lockup after resume
One surprising observation: If I disable APM in /boot/device.hints, my machine suspends and resumes JUST FINE. The BIOS alone seems to be able to suspend and awake the hardware behind FreeBSD's back. The system only hangs if FreeBSD is involved in the process. Hmm, I might try that. BTW, last time I asked Warner about this his reply was (I paraphrase) it's not supposed to work, and if it ever worked for you it was out of sheer luck, which I find surprising. Me too, since it used to work on that same hardware. (A ThinkPad 600 was what I did the original suspend/resume work on during the FreeBSD 2.2 days.) Nate To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: cp -u patch
On Sun, May 06, 2001 at 06:11:56AM +0200, Cyrille Lefevre wrote: Question is, do we want to add this to our cp? please, what is the difference between this : cp -Ruv mozilla mozilla-test and that : cd mozzila; find . | cpio -pdm ../mozzila-test Lets try another realistic example: cp -uvp ab* cde*.f* g? h/*.i? j/kl /m What's the find | cpio invocation for that? When you come up with it, it will be very messy. BTW, I don't consider my example contrived, I've copied files like that before for backup purposes. -- -- David ([EMAIL PROTECTED]) To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: pgm to kill 4.3 via vm
On Mon, 7 May 2001, Matt Dillon wrote: their hands of the whole affair. A production machine with 128M of ram and 1G of swap is going to go down the tubes performance-wise long before it runs out of swap. Performance degredation under heavy memory loads is a much more interesting and important problem then swap exhaustion. Indeed, this is an interesting area. In the process of researching how to best implement this for Linux I have found various reasons why both FreeBSD's and NetBSD's load control systems cannot work in various realistic scenarios. The next step is designing a load control system that does work (not too hard) and having a reliable way of detecting when exactly the system is thrashing (next to impossible?). I'll make a detailed writeup of exactly why FreeBSD's load control system cannot work and will post it to [EMAIL PROTECTED] and [EMAIL PROTECTED] soon... ;) regards, Rik -- Linux MM bugzilla: http://linux-mm.org/bugzilla.shtml Virtual memory is like a game you can't win; However, without VM there's truly nothing to lose... http://www.surriel.com/ http://www.conectiva.com/ http://distro.conectiva.com/ To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: pgm to kill 4.3 via vm
:Indeed, this is an interesting area. In the process of :researching how to best implement this for Linux I have :found various reasons why both FreeBSD's and NetBSD's :load control systems cannot work in various realistic :scenarios. It's not a load control system. It's an emergency measure, period. A load control system is something like... oh, the 20 second enforced swap out that can be triggered when the VM system is under extreme memory pressure. Load control is a completely different issue from swap exhaustion. The two are entirely unrelated to each other. -Matt To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: pgm to kill 4.3 via vm
On Mon, 7 May 2001, Matt Dillon wrote: :Indeed, this is an interesting area. In the process of :researching how to best implement this for Linux I have :found various reasons why both FreeBSD's and NetBSD's :load control systems cannot work in various realistic :scenarios. A load control system is something like... oh, the 20 second enforced swap out that can be triggered when the VM system is under extreme memory pressure. Yup. Too bad the 20 second enforced swap out isn't enforced... (at least, not by the code in vm_glue.c) I'll write up an email about this later this week... Load control is a completely different issue from swap exhaustion. The two are entirely unrelated to each other. I know, that's why I only quoted the part of your message that was about load control and snipped the swap exhaustion part ;) regards, Rik -- Linux MM bugzilla: http://linux-mm.org/bugzilla.shtml Virtual memory is like a game you can't win; However, without VM there's truly nothing to lose... http://www.surriel.com/ http://www.conectiva.com/ http://distro.conectiva.com/ To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: pgm to kill 4.3 via vm
: : A : load control system is something like... oh, the 20 second enforced swap : out that can be triggered when the VM system is under extreme memory : pressure. : :Yup. Too bad the 20 second enforced swap out isn't enforced... :(at least, not by the code in vm_glue.c) : :I'll write up an email about this later this week... Sure, that's an interesting topic. -Matt To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: Rfork'd threads, signals, and LDTs
In article [EMAIL PROTECTED], Daniel Eischen [EMAIL PROTECTED] wrote: I was looking at our linuxthreads port and noticed some %gs fiddling. If linuxthreads wants to allow POSIX semantics for specifying thread stack allocation, they'll have to stop relying on stack alignments for TSD. Agreed. It appears that they use %gs if it is determined (at glibc build time) that the target Linux kernel is new enough (2.3.99) to support it. The Red Hat 7.0 kernel is 2.2.16. John -- John Polstra [EMAIL PROTECTED] John D. Polstra Co., Inc.Seattle, Washington USA Disappointment is a good sign of basic intelligence. -- Chögyam Trungpa To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: pgm to kill 4.3 via vm
On Mon, 7 May 2001, Alfred Perlstein wrote: * Rik van Riel [EMAIL PROTECTED] [010507 10:59] wrote: The next step is designing a load control system that does work (not too hard) and having a reliable way of detecting when exactly the system is thrashing (next to impossible?). You might as well not bother unless you're actually going to provide code that fixes the issue. For some reason banning you from the irc channel hasn't convinced you that complaining without providing patches isn't the way we do things around here. How about first analysing the problem in detail and trying to fix it after we understand the problem ? The current stage is that I've pretty much figured out the problem and know why the code in FreeBSD and NetBSD doesn't currently work (while it would have worked in the original Mach VM). A next stage is getting some smart people together and coming up with a thorough solution. Patch first, think later is definately not the attitude I'm used to seeing in the FreeBSD world, no matter how often you and phk have shouted this at me yesterday ;) Anyhow, since you do seem to code a bit for Linux you might want to take a shot at fixing the Linux code. Last I checked, given enough time (30 minutes or so) FreeBSD recovers from such situations while Linux wets the bed and requires a power cycle. Indeed, at the moment Linux doesn't have any load control code at all while FreeBSD should be able to recover in some situations. Ideally both systems should be able to recover from just about any situation. regards, Rik -- Linux MM bugzilla: http://linux-mm.org/bugzilla.shtml Virtual memory is like a game you can't win; However, without VM there's truly nothing to lose... http://www.surriel.com/ http://www.conectiva.com/ http://distro.conectiva.com/ To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: pgm to kill 4.3 via vm
On 7 May 2001, Dag-Erling Smorgrav wrote: Dennis Glatting [EMAIL PROTECTED] writes: On Monday 07 May 2001 08:10 am, Dag-Erling Smorgrav wrote: malloc() will return NULL only if you hit a resource limit or exhaust address space. There may or may not be memory (real or virtual) available at that time. Isn't memory exhaustion a resource limit? What is memory exhaustion? Uh, when I perform a malloc() and get a NULL back? I dunno, what do you call that? Namespace exhaustion will cause malloc() to fail. But FreeBSD overcommits memory, so exhaustion of physical and virtual memory will manifest itself as an unsatisfiable page fault, which will cause the kernel to start killing more-or-less-random processes until some memory is available again. Why would it kill random processes as opposed to the offending process? Explain the bug and malloc() behaviour. According to the malloc() man page: The bug: I believe you meant i -= j when you wrote i =- j. The result is that i becomes negative and in all likelihood you smash the malloc() arena. Yup. Regarding malloc(), you seem to believe that malloc() allocates *exactly* the amount you ask, no more, no less, and actually allocates physical or virtual memory. It doesn't. First of all, it allocates memory of its own for bookkeeping purposes, which your program doesn't account for. Second, it usually allocates a little bit more than you asked for, and keeps the leftovers for later. Third, it only allocates namespace. Actual memory (physical or virtual) is allocated by the kernel the first time you touch a page. I am not under the impression malloc allocs an exact space. What's more, your program does extra stuff (such as needlessly fork() / exec()ing sync(1) instead of just calling sync(2), which doesn't serve any purpose anyway, since virtual memory isn't managed by the file system code, so sync(2) doesn't affect it) which consumes memory, which further invalidates your results (for instance, memory exhaustion while running sync(1) could cause the kernel to randomly kill your program) Regardless, the code shouldn't be crashing the kernel. I assert memory exhaustion is would return unsuccessful on the malloc() call, no? No, just namespace exhaustion, which your program won't even come close to. I don't really understand your point. Who really cares whether it is a name space exhaustion or an exhaustion of virual memory or a fly sat on the keyboard: malloc() was called and the kernel crashed. The kernel should have failed the offending system call or killed the offending application. In the real world there are plenty of reasons why an operating system may run at or near its limits (whatever a limit may be). Example anomolies include: * a web server hit by a DoS, * a database server hit with a flurry of processing tasks * someone sending spam through your mail server Is it reasonable to kill the operating system when these things happen? No. The kernel should always be better behaved than the application. DES -- Dag-Erling Smorgrav - [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: pgm to kill 4.3 via vm
Okay, enough said. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: pgm to kill 4.3 via vm
On Mon, 7 May 2001, Alfred Perlstein wrote: In FreeBSD we submit a patch perhaps after having an N-way conversation (*) about the problem being addressed. I'm are awaiting your patch, I'll let this contradiction speak for itself. You'll see a detailed analysis soon, patches will come only after we've agreed on a way to fix the problem. regards, Rik -- Linux MM bugzilla: http://linux-mm.org/bugzilla.shtml Virtual memory is like a game you can't win; However, without VM there's truly nothing to lose... http://www.surriel.com/ http://www.conectiva.com/ http://distro.conectiva.com/ To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
gdb -k not able to access BSS?
Hi, I was trying to debug a problem where sound card is not working because it ends up with a irq that is shared with the VGA card. So I figured I'd use gdb to look at the interrupt handler vector to see what was there and to find out what is going on. However gdb is unable to dump the data : munin# gdb -k -wcore -s ./kernel.debug GNU gdb 4.18 Copyright 1998 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type show copying to see the conditions. There is absolutely no warranty for GDB. Type show warranty for details. This GDB was configured as i386-unknown-freebsd... (kgdb) print intr_handler Cannot access memory at address 0xc0395b80. (kgdb) x/20x 0xc0395b80 0xc0395b80 intr_handler: Cannot access memory at address 0xc0395b80. Anyone got an idea? Nm on the running kernel agrees with the above address. munin# nm /boot/kernel/kernel | fgrep intr_handler c0395b80 B intr_handler Lars To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: pgm to kill 4.3 via vm
: For some reason banning you from the irc channel hasn't convinced : you that complaining without providing patches isn't the way we do : things around here. : :How about first analysing the problem in detail and :trying to fix it after we understand the problem ? : :The current stage is that I've pretty much figured out :the problem and know why the code in FreeBSD and NetBSD :doesn't currently work (while it would have worked in :the original Mach VM). : :A next stage is getting some smart people together :and coming up with a thorough solution. : :Patch first, think later is definately not the :attitude I'm used to seeing in the FreeBSD world, :no matter how often you and phk have shouted this at :me yesterday ;) Well, we'll see what you have to say in your paper. VM/Swap is a tough nut to crack... remember when I thought I had optimized the pageout daemon to not launder dirty pages unnecessarily? And then it turned out that, in fact, the pageouts were necessary to maintain stability in certain environments (like USENET news servers). That was somewhat embarassing because I waxed poetic to you and Linus and then had to correct myself and revert part of the optimization after real-life problems started cropping up. About 8 months ago I introduce code that would prevent sequential reads of large files from blowing away the rest of the system's VM page cache, to reduce the impact sequential reads would have in a heavily loaded system. It's a wonderful idea and I had thought up a great algorithm... too bad it didn't work in real life. It took a month to find all the edge cases and change the algorithm to be more of a heuristic to handle them all. I'm still not completely happy with it so at the moment it's only used by madvise(). At some point soon I am going to make the buffer cache use it, though. In anycase... insofar as VM goes, A theory doesn't usually last beyond the first implementation. So we'll see. -Matt To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: pgm to kill 4.3 via vm
You'll see a detailed analysis soon, patches will come only after we've agreed on a way to fix the problem. You've already had some folks respond to this, though I think the argument has been mischaracterized as a BSD vs Linux thing. It's not. What people are (IMHO) really trying to argue here is the pragmatic approach. You can't really agree on a way to fix the problem when your operating criteria are as vague as the existing system doesn't work, we need to fix it. That's like saying that putting someone into orbit is a simple matter of determining what escape velocity is necessary from an object with earth's mass and deciding how many tons of payload you want to insert at what altitude. The devil is, as they say, all in the details and all people here want is the necessary level of detail. Software is also largely an operational art where it's easier to describe something through a body of code with accompanying comments than it is to try and describe it on a purely theoretical basis. You don't need to necessarily adopt that code, it simply provides you with a more solid framework in which to discuss ways to fix the problem and that is what I believe Alfred and others are basically asking for. It's a reasonable request. - Jordan To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: select(2) converted to use a condition variable, and optimis
On 06-May-01 Alfred Perlstein wrote: * Seigo Tanimura [EMAIL PROTECTED] [010506 04:40] wrote: http://people.FreeBSD.org/~tanimura/patches/selectopt.diff Please do not remove the spl calls, they serve as a useful guide for making finer grained locks as well as error checking the new locks. Actually, in this case, the proc lock provides all the locking needed, so the spl's can go I think. -- John Baldwin [EMAIL PROTECTED] -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.baldwin.cx/~john/pgpkey.asc Power Users Use the Power to Serve! - http://www.FreeBSD.org/ To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
RE: select(2) converted to use a condition variable, and optimis
On 06-May-01 Seigo Tanimura wrote:
As conversion of select(2) from msleep(9) to a condition variable is
in the SMPng TODO list, I have done that task.
Also, we do not have to lock a process in order to evaluate the result
of {sel,poll}scan() and the remaining time of {select,poll}(2). It
should be enough to do those pieces of work first, followed by locking
a process and wait for a condition variable or selwakeup().
You need the lock when clearing the bit in p_flag. That is why the proc locks
are there, so all those proc locks need to stay. When you clear a bit, you are
writing all the bits, so you need to ensure that you can atomically
read/modify/write all the bits in p_flag, hence the need for the proc lock.
Please back out the changes to not lock the process around the p_flag change.
The rest of the patch looks ok, though I'd like to review the updated version
before it is committed. Thanks.
Those changes are in the patch at:
http://people.FreeBSD.org/~tanimura/patches/selectopt.diff
--
John Baldwin [EMAIL PROTECTED] -- http://www.FreeBSD.org/~jhb/
PGP Key: http://www.baldwin.cx/~john/pgpkey.asc
Power Users Use the Power to Serve! - http://www.FreeBSD.org/
To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-current in the body of the message
subscribe
Get your FREE download of MSN Explorer at http://explorer.msn.com To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: pgm to kill 4.3 via vm
Dennis Glatting [EMAIL PROTECTED] writes:
On 7 May 2001, Dag-Erling Smorgrav wrote:
Dennis Glatting [EMAIL PROTECTED] writes:
On Monday 07 May 2001 08:10 am, Dag-Erling Smorgrav wrote:
malloc() will return NULL only if you hit a resource limit or exhaust
address space. There may or may not be memory (real or virtual)
available at that time.
Isn't memory exhaustion a resource limit?
What is memory exhaustion?
Uh, when I perform a malloc() and get a NULL back? I dunno, what do you
call that?
Address space exhaustion (which I incorrectly called namespace
exhaustion in a previous mail - braino!), or a bug in malloc().
Consider the following program:
#include err.h
#include stdlib.h
int
main(void)
{
int i;
_malloc_options = j;
for (i = 0; i 4096; ++i)
if (malloc(1048576) == NULL)
err(1, malloc() (i == %d), i);
exit(0);
}
The result may surprise you.
Why would it kill random processes as opposed to the offending process?
Theoretically, it should kill the largest process around. I'm not
sure it works like that in practice.
Regardless, the code shouldn't be crashing the kernel.
Are you sure it does? Did you try running it on the console of a
DDB-equipped system, press Ctrl-Alt-Esc, and find out what the kernel
is doing when it appears frozen?
I don't really understand your point. Who really cares whether it is a
name space exhaustion or an exhaustion of virual memory or a fly sat on
the keyboard: malloc() was called and the kernel crashed. The kernel
should have failed the offending system call or killed the offending
application.
Malloc() doesn't translate into a system call. It sometimes performs
one or more system calls (mostly one of brk(), sbrk() and mmap()), and
sometimes doesn't, depending largely on what kind of malloc() / free()
activity there's been so far.
Even when malloc() really does request memory from the kernel, what it
really allocates is address space. The actual memory pages to fill
that address space are allocated by the kernel as they are touched.
Is it reasonable to kill the operating system when these things happen?
No. The kernel should always be better behaved than the application.
I didn't claim the contrary. I'm trying to show you that things don't
work the way you think they do. Now be a good boy and go read the
archives, it's all been discussed before.
DES
--
Dag-Erling Smorgrav - [EMAIL PROTECTED]
To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-current in the body of the message
Re: pgm to kill 4.3 via vm
Dag-Erling Smorgrav wrote: Dennis Glatting [EMAIL PROTECTED] writes: I am intentionally testing at the limits to see what happens, usually interesting things. :) In this case, the application is well behaved (in the error proccesing sense): it'll exit, thus releasing its memory resources, when the kernel reports a memory allocation failure. No. malloc() will return NULL only if you hit a resource limit or exhaust address space. There may or may not be memory (real or virtual) available at that time. Plus, your program doesn't even do what you think it does (because a) it has at least one significant bug and b) malloc() doesn't behave the way you think it does). And even if it did, the /dev/random stuff is pointless, you can achieve the same effect by setting every byte you allocate (possibly even just the first byte of every chunk) to 0. To really test what you think your program tests, you should mmap() an amount of memory larger than RAM + swap and touch every page. Even then, the result will be a SIGSEGV, not a graceful termination. Regardless, the machine should recover once all (trouble) programs have been killed. -- Daniel C. Sobral(8-DCS) [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED] Caffeine is proof that God hates mornings too To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
