Am 12.12.2013 11:27, schrieb Nick Kew:
On 12 Dec 2013, at 02:41, Ingo Walz wrote:
So socache with shared memory (shmcb) is the way you would suggest?
How important is the sharing in your case? If it's just about performance,
you may be better-off accepting that each process will repeat the work
of populating the hash, if that's a smaller overhead than IPC.
If sharing is necessary, then socache will indeed serve, though you can
leave the choice of socache engine to your config.
Well, depends. Of course, I want to get the best out of my
implementation, so best-case scenario might be:
- no global (no idea if I ever can avoid that though, I guess not if shm
will be used) or thread mutex
- no external dependencies (such as memcache e.g.)
On the other side, it should have a clear "scaling pattern". As of now,
it only depends on the number of possible HTML rewrites (due to the
number of lookups in apr_hash_t) and the number of assets delivered by
factor one. Changing that on my current environment might blow up factor
1 by factor 3 from a memory perspective. And thus changes on different
servers with a different number of workers - which is a kind of
unacceptable for me.
Before I do that, I'd introduce a smaller overhead on the way how it's
stored but still have a clear scaling pattern.
But as I said, I'm still trying to evaluate what exactly is the best
case scenario in details and which options do I have.
Memcache would be a great opportunity either, because it will give me a
"Cluster capable" implementation for free, if I share the cache across
multiple servers - but that's a use-case for the future and not of
interest right now.
If I use server->process->pool, does every worker internally has it's
own process->pool scope? Or is it 1 pool used as a parent process?
Checking my process list gave me 3 workers running as my apache user and
1 as root - so I guess there is a parent process, used to bind the
socket and "dispatch" the work to the workers - but I've no idea if
something is shared between those processes at all (beside the
dispatched request itself).
I was also a little uneasy about your original description: the use of a
forever-pool becomes a memory leak if it is populated with per-request data.
Modules that do such things usually create their own pool (as a child of
the process pool) which they can then maintain privately.
Really a memory leak and not a potential double free or even a segfault?
Because the request pool is cleaned up before the server pool and the
reference, pointing to the memory segment remains in the process pool -
but it might be freed (or even re-used by another process). I made an
apr_pstrndup from the request data, assigned to server->process->pool to
avoid this issue. I've also tried an own pool, but server->process->pool
does 100% fit my use-case (I thought).
I'll try to work with socache inside the implementation to see if it
covers all my needs in detail.
But I still try to understand why it happens. Will see if I can figure
out what's going on currently (gdb is not easy to use in this particular
use-case ... ), try socache and get back to you, once I have the first
results.
Thanks!
Ingo
