> -----Original Message----- > From: Matthew Dillon [mailto:dil...@apollo.backplane.com] > Sent: Tuesday, July 20, 1999 1:53 PM > To: Kelly Yancey > Cc: crand...@matchlogic.com; freebsd-hackers@FreeBSD.ORG > Subject: Re: RE: Overcommit and calloc() > > > I think this would be a waste of time. As I have said, very few > large allocations use calloc(). Nearly all small allocations come > off the heap. The cost of adding the complexity to calloc to avoid > zeroing the data is not going to be worth the minor (and unnoticeable) > improvement in performance that you reap from it. Right now > malloc/calloc > add new blocks to the heap in chunks and while they may know > that a page > is zero on allocation, once the first small bit has been removed from > the heap they have no idea that the remainder is still zero. > Making them > knowledgeable of that will simply make the critical path more > costly and > probably end up in reducing your overall performance. > > -Matt > Matthew Dillon > <dil...@backplane.com> >
I don't dare claim to know as much as about the system as you do, how about this: I am sure that I can at least just find a spare bit in struct pgfree (/usr/src/lib/libc/stdlib/malloc.c) in which I can store a boolean flag indicating whether or not the run is already zeroed. Then all I add in complexity is setting the flag when a run is added to the list via map_pages/extend_pgdir, and a fairly small calloc() implementation which looks much like the existing calloc() but checking specifically for a run with the bit set. malloc()'s implementation doesn't have to change a bit since it doesn't care whether it's memory is zeroed or not, so the critical path isn't affected. calloc() will then use already zero'ed pages if they are available, otherwise just take what it can get and bzero() them itself. Since there will only ever be at most a single run of pre-zero'ed pages, I'll probably keep a separate pointer directly to that struct pgfree to optimize calloc() further (which would add to the critical path, so I may reconsider). I can't actually make patches, though, until I get home. As far as your other point, you imply that because pages are be pre-zeroed during the idle loop, we are currently no worse off then if they were never zeroed at all. Which is true. But is that not to say that we wouldn't be better off by taking advantage of the work we have already done? Besides, how many of us are running things like rc5 or s...@home which prevent the idle loop from doing any work? Os it not possible that we run out of zeroed pages and then when a process calls sbrk(), the VM system has to zero the memory to give to us (which may be zeroed again by calloc())...in which case a call to calloc() results in zeroing the memory twice *while we wait*. Admittingly, calloc() isn't called nearly as often as malloc(). But malloc() is already pretty darned optimized thanks to phk, I'm just trying to extend his optimizations to include cases where calloc() could further benefit. Don't worry I'll post benchmarks before and after, with and without rc5 running in the background, along with the patches. I really appreciate all the feedback though, Kelly ~kby...@posi.net~ FreeBSD - The Power To Serve - http://www.freebsd.org/ Join Team FreeBSD - http://www.posi.net/freebsd/Team-FreeBSD To Unsubscribe: send mail to majord...@freebsd.org with "unsubscribe freebsd-hackers" in the body of the message
