Thanks Justin.
The problem was that the patch was developed across 3 different OS's and
about 6 revisions, so whilst trying to keep about 4 tree's in sync I guess
I missed some stuff. Apologies for that :(
I'm sitting in a Cyber Cafe waiting for a train, but am trying to catch up
having had
now... you guys may have palloc() returning NULL because you've put a
layer of indirection into pools so that you can have pools on top of
exotic storages. if that's the case, then i really hope that one of
those exotic pools is never passed to an httpd routine expecting a
plain old memory
Justin Erenkrantz wrote:
Yeah, David had some local changes that he didn't commit that were in
his sms_pools_patch.
Since we weren't building, I just committed the relevant chunks into
APR. Try again. Works here without his patch. This may make his
posted patch go freaky. Oh, well.
I may
On Sat, Jul 07, 2001 at 04:01:35PM -0700, dean gaudet wrote:
what's it really matter? trying to clean up from OOM is near impossible.
Which brings me back to my original question: Why do we have an abort
function? As you said (I was originally thinking this, but admittedly,
I didn't bring
On Sat, 7 Jul 2001, Justin Erenkrantz wrote:
On Sat, Jul 07, 2001 at 04:01:35PM -0700, dean gaudet wrote:
what's it really matter? trying to clean up from OOM is near impossible.
It doesn't matter if it's impossible or not. The problem is that we are
in a library, so we can't just write to
I used gprof to do some performance profiling of an httpd built
using SMS-based pools.
Below is the list of the top 100 functions, ranked by CPU usage.
(The full output of gprof, showing what functions contribute the
most invocations of what other functions, is ~300KB, so I haven't
included it
On Sat, 7 Jul 2001, Brian Pane wrote:
I used gprof to do some performance profiling of an httpd built
using SMS-based pools.
Below is the list of the top 100 functions, ranked by CPU usage.
(The full output of gprof, showing what functions contribute the
most invocations of what other
The following belongs in apr/test.
It will replay files generated by the following patch:
There are 2 sample replay files which should also get commited at
webperf.org/a2/pool (they are too large to post)
..Ian
Index: apr_pools.c
===
On Sun, 8 Jul 2001, Cliff Woolley wrote:
You know, it'd be really cool if there were a version of apr_pstrcat()
that returned the length of the concatenated output. There are LOTS of
places where we call apr_pstrcat() and then immediately have to call
strlen() on its output, when
On Sat, 7 Jul 2001 [EMAIL PROTECTED] wrote:
+1, personally I would prefer to have the length returned in the parameter
list.
I was leaning that way as well for consistency with apr_pstrcat().
Or, we could do something I have been considering since I started
APR. We could create an
On Sat, 7 Jul 2001 [EMAIL PROTECTED] wrote:
+1, personally I would prefer to have the length returned in the parameter
list.
I was leaning that way as well for consistency with apr_pstrcat().
Or, we could do something I have been considering since I started
APR. We could create an
On Sat, 7 Jul 2001 [EMAIL PROTECTED] wrote:
I think you misunderstand my meaning.
Would you expect anything else? That's my specialty. ;-)
We should do both.
Cool. +1.
--Cliff
--
Cliff Woolley
[EMAIL PROTECTED]
On Sun, 8 Jul 2001, Cliff Woolley wrote:
On Sat, 7 Jul 2001 [EMAIL PROTECTED] wrote:
I think you misunderstand my meaning.
Would you expect anything else? That's my specialty. ;-)
I tend to jump ahead of myself and not explain what I mean. I just expect
everybody to read my mind. :-)
I added some instrumentation to apr_sms_trivial_malloc
to study in more detail where its bottlenecks were.
As a result, I found a few interesting phenomena:
1. The good news is that allocations of small amounts
of memory are very efficient. They almost always
take the fastest path through the
On Sun, Jul 08, 2001 at 12:47:32AM -0700, Brian Pane wrote:
I added some instrumentation to apr_sms_trivial_malloc
to study in more detail where its bottlenecks were.
As a result, I found a few interesting phenomena:
1. The good news is that allocations of small amounts
of memory are
On Sat, Jul 07, 2001 at 10:07:14PM -0700, Ian Holsman wrote:
The following belongs in apr/test.
It will replay files generated by the following patch:
There are 2 sample replay files which should also get commited at
webperf.org/a2/pool (they are too large to post)
You know, this would be
Justin Erenkrantz wrote:
Aaron pointed out that CLR (Cormen/Leiserson/Rivest for those that never
took algorithms classes) has many memory allocation algorithms. I'd
bet we could find one that would work better. -- justin
At USENIX there was a talk about extending the slab allocator to
1. The good news is that allocations of small amounts
of memory are very efficient. They almost always
take the fastest path through the code, in which
some available space is reserved from the
sms-used_sentinel.prev block with a handful of
pointer arithmetic
1. The good news is that allocations of small amounts
of memory are very efficient. They almost always
take the fastest path through the code, in which
some available space is reserved from the
sms-used_sentinel.prev block with a handful of
pointer arithmetic operations.
I've been working on a power of 2 allocator. I still haven't got that up
to speed yet, but it is worth a look. I'll play with it this week and post
it to the list
I hacked a simple power of two allocator together as a proof of concept that
replacing
malloc/free with apr_malloc/apr_free
Changes:
* use memory/unix/apr_pools.c instead of lib/apr_pools.c
* include sms_blocks sms_trivial in build.
--Ian
Index: apr.dsp
===
RCS file: /home/cvspublic/apr/apr.dsp,v
retrieving revision 1.73
diff -u -r1.73
On Sun, 8 Jul 2001, Ian Holsman wrote:
Bill Stoddard wrote:
I've been working on a power of 2 allocator. I still haven't got that up
to speed yet, but it is worth a look. I'll play with it this week and post
it to the list
I hacked a simple power of two allocator together as a proof
On Sun, Jul 08, 2001 at 01:21:36AM -0700, Justin Erenkrantz wrote:
On Sat, Jul 07, 2001 at 10:07:14PM -0700, Ian Holsman wrote:
The following belongs in apr/test.
It will replay files generated by the following patch:
There are 2 sample replay files which should also get commited at
On Sun, 8 Jul 2001, Ian Holsman wrote:
I was wondering why the brigade code doesn't make use of pools/sms to
handle it's memory.
It has a pool passed to it from what I can see...
The brigade is allocated out of a pool, and that pool is used to register
a cleanup on the brigade to make sure
On Sun, 8 Jul 2001, Justin Erenkrantz wrote:
Yup. I've brought this up to Sander and David before, but this is how
pools
woah. no way really?
that's not at all how it was in 1.3 or in early 2.0 ...
in 2.0 as of uh a year ago say, there was one free list per process,
and locks were used
On Sun, Jul 08, 2001 at 10:14:16AM -0700, dean gaudet wrote:
an ideal situation for free-lists (blocks of freed, but not free()d
memory) is one per cpu.
a less ideal situation is one per thread.
an even less ideal situation is one per process (which requires locking).
it's insane to
On Sun, Jul 08, 2001 at 10:18:12AM -0700, dean gaudet wrote:
On Sun, 8 Jul 2001, Justin Erenkrantz wrote:
Yup. I've brought this up to Sander and David before, but this is how
pools
woah. no way really?
that's not at all how it was in 1.3 or in early 2.0 ...
in 2.0 as of uh a
On Sun, Jul 08, 2001 at 10:34:22AM -0700, Jon Travis wrote:
There is still all this tomfoolery with locking, though, which I think
would be nice to fix with different sized buckets in the freelist.
Stuff that the malloc in glibc does. I cringe at the thought of how
much overhead due to
Stuff that the malloc in glibc does. I cringe at the thought of how
much overhead due to abstraction this whole project is causing.
The other thing this abstraction buys us is the ability to pass in
different memory to standard APR operations. Hopefully we can get
shared memory to work in this
On Sun, Jul 08, 2001 at 10:41:12AM -0700, dean gaudet wrote:
also -- it would be worthwhile for someone to try a version of pools which
keeps no freelist, and uses malloc()/free() on each block. i'm guessing
this is faster in a multithreaded server than us doing a per-process
global list
On Sun, Jul 08, 2001 at 11:04:09AM -0700, Jon Travis wrote:
As for the ability to use shared memory for this ... yeah that is
an interesting idea. What are your actual use cases for that?
Ian posted a patch to do something like this - I think he wanted a hash
table that was shared across all
On Sun, 8 Jul 2001, Jon Travis wrote:
I talked to rbb about this not too long ago, and he told me that you
did a lot of performance work, and that is why pools use their own
allocation scheme (8k blocks, freelists, etc.) instead of using a
total malloc() oriented scheme.
yes and no. i can't
On Sun, 8 Jul 2001, Justin Erenkrantz wrote:
Also, I did try having the pools use malloc/free directly
(ALLOC_USE_MALLOC) and the performance was dreadful. At least on
Solaris. -- justin
yes, ALLOC_USE_MALLOC is dreadful -- that's not what i meant.
what i mean is something like the below
dean gaudet wrote:
On Sun, 8 Jul 2001, Justin Erenkrantz wrote:
Also, I did try having the pools use malloc/free directly
(ALLOC_USE_MALLOC) and the performance was dreadful. At least on
Solaris. -- justin
yes, ALLOC_USE_MALLOC is dreadful -- that's not what i meant.
what i mean is something
so while hacking up that patch to eliminate the block_freelist i thought i
could also get rid of the alloc_mutex entirely. but it appears that
currently the pool code sort of supports multithreaded access to the same
pool.
i say sort of because cleanups don't support multithreaded access -- no
On Sun, 8 Jul 2001, Ian Holsman wrote:
dean gaudet wrote:
it removes block_freelist and uses malloc/free for blocks. this is much
less expensive than malloc/free on each allocation (amortization).
If you can wait till monday, and you are interested, I'll put it through
the ringer on a
On Sun, 8 Jul 2001, Brian Pane wrote:
dean gaudet wrote:
On Sun, 8 Jul 2001, Ian Holsman wrote:
dean gaudet wrote:
it removes block_freelist and uses malloc/free for blocks. this is much
less expensive than malloc/free on each allocation (amortization).
If you can wait till monday,
Jon Travis wrote:
On Sun, Jul 08, 2001 at 11:27:08AM -0700, Ian Holsman wrote:
Justin Erenkrantz wrote:
On Sun, Jul 08, 2001 at 11:04:09AM -0700, Jon Travis wrote:
[snip]
I'm still curious about the use case. I'm sure there is one, if you
could just enlighten me. I already know it's 'cool'.
for
On Sat, Jul 07, 2001 at 05:14:47PM -0700, Justin Erenkrantz wrote:
[...]
Error-checking
is something that we pointedly avoid doing in APR.
[...]
Please tell me this is not a general policy, and that APR has a small
amount of error checking. Pretty please? I don't advocate pedantic
error
dean gaudet wrote:
On Sun, 8 Jul 2001, Ben Laurie wrote:
Justin Erenkrantz wrote:
Aaron pointed out that CLR (Cormen/Leiserson/Rivest for those that never
took algorithms classes) has many memory allocation algorithms. I'd
bet we could find one that would work better. -- justin
On Sun, 8 Jul 2001, Aaron Bannert wrote:
On Sat, Jul 07, 2001 at 05:14:47PM -0700, Justin Erenkrantz wrote:
[...]
Error-checking
is something that we pointedly avoid doing in APR.
[...]
Please tell me this is not a general policy, and that APR has a small
amount of error checking.
Since my last patch re upgrading autoconf and libtool was not received
with open arms, I decided to take the decidedly less fun route and
get things working with both autoconf 2.13 and 2.50 as well as
libtool 1.3 and 1.4. Here is a patch that implements that change.
The first step is to remove
42 matches
Mail list logo
