Re: malloc does not return null when out of memory
On Thu, 24 Jul 2003 19:18:06 -0400 Chuck Swiger <[EMAIL PROTECTED]> wrote: > Muttley wrote: > > Yes, I thought briefly about something like this. > > > > Then I thought 'there's a race condition'. > > Where? The FreeBSD implementation is wrapped in a THREAD_LOCK()...? Good point, well made > > Then I realised that other processes might not link against this malloc. > > Perhaps. Statically linked binaries, for example. Maybe Linux ones too? > > Then I realised the race condition doesn't even matter; processes will > > still be killed, as the kernel doesn't care that you're still in > > malloc() when the overcommitted memory is touched, it just knows you've > > touched it and there's no actual memory there. This will result in far > > more processes being killed. I believe that's a bad thing. > > Someone stated that it was a problem that malloc() returned pointers to > virtual address space that had been mapped but not allocated. This patch does > not guarantee that malloc() will return, but, if malloc() does returns a > pointer, using the memory being pointed to will refer to memory that is > allocated. Their main problem was that when memory ran out, processes got killed. The fact the process gets killed earlier doesn't alter the fact that it was killed. > As Barny Wolff said: > > Won't this merely die in malloc, not return 0? > > True. This isn't a perfect solution, but given the choice between: > > 1) malloc(LOTS) returning a pointer, and then sometime later the program dies > with a bus error when using that memory because no more VM is available, or > > 2) malloc(LOTS) causing an immediate failure in malloc(), > > ...choice #2 appears to be significantly better. > > Figuring out what went wrong from a coredump or backtrace for #2 when the > signal happens in malloc() should be obvious; determining why the program > crashed in the middle of referencing memory in some large buffer is > potentially misleading. If you re-read the original post in this thread, you will see that this appeared in the poster's syslog: Jul 23 01:37:57 m0n0wall /kernel: pid 80 (racoon), uid 0, was killed: out of swap space Finding out why the process died was never the big worry. Besides, see below... > Programs which take care to preallocate regions of memory they need before > they start doing a transaction or some other operation that needs to be atomic > would also prefer #2; the patch I proposed could have a beneficial impact on > data integrity for such programs. > Except that the process which cops the bullet in the head is the largest runnable non-system process. Check /usr/src/sys/vm/vm_pageout.c near the end of vm_pageout_scan(). Neither data integrity nor debugging from cores is helped by the patch. > -- > > People who encounter programs crashing in malloc() are likely going to > continue to complain about malloc() not returning NULL when the system is out > of memory. > > If malloc() is referencing memory before returning the pointer, means that the > system is going to reserve VM resources with temporal locality towards memory > _allocation_ rather than memory _reference_. Having the program crash at > memory allocation time rather than usage helps identify when and where this > problem actually happens more clearly, if only by a little bit. See above, and above that. > I'm not sure whether allocating memory sooner that way will make it more > likely that brk()/sbrk() or mmap() will return ENOMEM to the libc malloc() > implementation, but if it does not help, perhaps that means something and > we've identified the location of problem more precisely. Other posts suggest these calls won't ever return ENOMEM based on total system usage, as the kernel doesn't even track it. Even if they went for something in the style of Brent's description of vswap, surely this patch would effectively prevent ENOMEM due forcing the overcommit to zero by killing something as soon as a process requests memory beyond the total physical+swap. I don't see how you can change the undesired behaviour in userland. Cheers, Matt -- Matthew Whelan <[EMAIL PROTECTED]> ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: FreeBSD Stability
On Fri, 3 Jan 2003 10:05:05 -0800 (PST) Philip Hallstrom <[EMAIL PROTECTED]> wrote: > 26 www.cravath.com 102 ok892 939 940 Solaris 8 > Apache/1.3.27 (Unix) PHP/4.2.3 > > That's certainly more than 492 days... so even if they do reboot, > netcraft is ignoring it or accomodating it seems like. As it's running Solaris 8, it could be a 64-bit machine. -- Matthew Whelan <[EMAIL PROTECTED]> To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-stable" in the body of the message
Re: OT: Belkin KVM (Re: moused, psm0, and Logitech)
10/06/2002 23:28:07, [EMAIL PROTECTED] (D J Hawkey Jr) wrote: >In article <[EMAIL PROTECTED]>, > [EMAIL PROTECTED] writes: >> >> At 2002-06-10T05:30:39Z, "Yeasah Pell" <[EMAIL PROTECTED]> writes: >> >>> I make similar recommendations with respect to Belkin and KVMs, but I feel >>> I should comment on the quality of their USB KVM switches that they now >>> make. Having purchased one of the first models they made that supported >>> USB, and more recently a just-released USB model, I have to say both are >>> excellent in all regards save one -- the USB keyboard implementation is >>> terrible. >> >> Thanks for the heads-up. I also have a Belkin USB KVM (F1DS104T) and would >> never consider anything else, but I'm using it with a PS/2 Happy Hacking >> keyboard. > >FWIT, I've had zero problems and full functionality with my Belkin OmniView >PS/2 4-port (model #F1D066). Similarly, I am very happy with the video quality on my D-Link 2-port KVM, running at 1280x1024@85Hz, my only gripe being despite following standard advice on kernel options for the mouse, it still desynchs sometimes (whereas the win2k box that I'm using to write this mail on is fine with it). Caveat: I don't have any need for X at the moment, so I haven't spent any real time looking at the mouse problem. Matthew To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-stable" in the body of the message
Re: Error in make installworld
05/02/2002 15:12:50, z thompson <[EMAIL PROTECTED]> wrote: > >Hi, > >Trying to run make installworld yesterday, with source cvsupped 2/5/2002 >at around 18:00 MST, I kept getting the following error: > >ln -fs ../la_LN.ISO8859-1/LC_COLLATE >/usr/share/locale/pt_PT.ISO8859-1/LC_COLLATE >ln -fs ../la_LN.ISO8859-15/LC_COLLATE >/usr/share/locale/da_DK.ISO8859-15/LC_COLLATE >ln: /usr/share/locale/da_DK.ISO8859-15/LC_COLLATE: Operation not >permitted >*** Error code 1 That's normally the sort of thing you see when a file is immutable or append only... use ls -lo to show the flags, and chflags to remove any you don't want (like, all of them, if you're replacing it anyway ;p) Matthew To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-stable" in the body of the message
Re: Handholding Editor for rc.conf
05/02/2002 05:06:39, Chris BeHanna <[EMAIL PROTECTED]> wrote:
>I should think that it should be a SMOP to add some macros to vim
>to do this, given that vim is already sh-syntax aware, and rc.conf
>and rc.firewall are merely shell scripts.
My experience is that vim's shell syntax checking is a first order
approximation. I have certainly written scripts which confused it, although
admittedly they're not the sort of thing you should ever put in rc.conf ;p
(rc.firewall is of course a different matter)
Heh, just goes to show what a vim newbie I am though, I wouldn't have known
where to start on this but on closer inspection it looks like a very small
fraction of 2html.vim already covers the important part of what would be
needed to be shoved into a {Buf,File}WritePre autocommand.
Matthew
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-stable" in the body of the message
Re: Can't build unoptimized/debugging kernel
01/02/2002 21:32:51, Rahul Siddharthan <[EMAIL PROTECTED]> wrote: >Also, I was getting the above error with or without the -g flag, if I >didn't specify optimisation. Is the kernel meant to be built always with >-O or above? Yes... someone mentioned on this list a while back that it's very difficult to write assembler constraints that work both with and without optimisation. Matthew To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-stable" in the body of the message
Re: Problem with /usr/ports/graphics/glide3
27/12/2001 22:48:36, Josh Tolbert <[EMAIL PROTECTED]> wrote: >I have a working X 4.1.0/DRI setup for my voodoo3/3000 at home >(documentation of how I managed to get it working is at >http://halcyon.scoundrelz.net/~hemi/dri.txt). I found a PC at work and >decided I'd try to replicate my X 4.1.0/DRI setup at work. > >X installed fine from ports and /usr/ports/graphics/glide3 >downloaded fine. I applied anholt's patch from ports/31767. On build (make >-DWITH_VOODOO3=1 install), I get errors such as the following, with stop >errors afterwards: > >automake: configure.in: required file `./depcomp' not found >automake: cvg/glide3/src/makefile.autoconf.am: Assembler source seen but >`ASFLAGS' not defined in `configure.in' >automake: h3/glide3/src/makefile.autoconf.am: object `cpudtect.lo' created >by `cpudtect.c' and `cpudtect.S' >automake: h3/glide3/src/makefile.autoconf.am: object `cpudtect.lo' created >by `cpudtect.c' and `cpudtect.S' >automake: h3/glide3/src/makefile.autoconf.am: object `cpudtect.lo' created >by `cpudtect.c' and `cpudtect.S' >automake: h3/glide3/src/makefile.autoconf.am: object `cpudtect.lo' created >by `cpudtect.c' and `cpudtect.S' >automake: h3/glide3/src/makefile.autoconf.am: Assembler source seen but >`ASFLAGS' not defined in `configure.in' Could this be related to the issue described in this message from Boxing day? http://docs.freebsd.org/cgi/getmsg.cgi?fetch=634450+0+current/freebsd-ports Matthew To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-stable" in the body of the message
Re: dirpref benefit on virtual disks
12/11/2001 18:23:52, Bill Moran <[EMAIL PROTECTED]> wrote: >> Now I have some huge filesystems on RAID partitions. To recreate >> all their directories involves some hassle, but I would think >> about doing it. But since these are no real but virtual disks, >> spread over a set of disks in a hardware raidbox, I'm not sure, >> if I would even benefit from the better algorithm. It sounded >> a bit like designed for filesystems on a (single?) disk? > >My thought would be that, yes, it will improve performance. If >you've got a typical 3 disk, RAID-5, a read of the directory still >causes a seek on all three disks, and if there's continual seeking >across the three disks, you'll see performance degredation. If >dirpref can layout the directory info so that it's close together, >seek time is reduces, whether on 3 disks or one. There were some benchmarks posted to this list, which confirmed this. A speedup, and a very healthy one, but maybe not -quite- as much as on a single disk, ISTR maybe a 6x improvement on rm -rf /usr/ports where you'd get 10x on a single disk. Check the archives if you want better figures than my dim memory can provide. >> Also I would like to know, if there is a certain limit of free >> space, on the disk, so that the algorithm can actually use >> the better layout? The disks have some space left, in an >> absolute way, but not that much from a relative point of view >> (like 12GB left which is just 6% minfree not taken into account). > >Have a read of the original paper on FFS (which is in /usr/share/doc >if you installed docs with FreeBSD) there is an explanation of why >8% is reserved on the drive - man tunefs comments about this as well. >I would assume that the new dirpref code needs plenty of free space >to use the optimal layout policy, but I don't know if the minimal >free space is still 8% or not, and I don't know if anyone has even >tested the new dirpref code to see if that number has changed. I'd imagine a lot would depend on how fragmented your free space is, in particular, avg. fragment size vs. avg. directory size (as configured in sysctl). People have tried tar -c/rm -rf/tar -x cycles, with wildly varying improvements - from nearly no difference to nearly as good as newfs. Matthew To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-stable" in the body of the message
