Re: [naviserver-devel] Quest for malloc
On 16.12.2006, at 19:31, Vlad Seryakov wrote: But if speed is not important to you, you can supply Tcl without zippy, then no bloat, system is returned with reasonable speed, at least on Linux, ptmalloc is not that bad OK. I think I've reached the peace of mind with all this alternate malloc implementations... This is what I found: On all plaforms (except the Mac OSX), it really does not pay to use anything else beside system native malloc. I mean, you can gain some percent of speed with hoard/tcmalloc/nedmalloc/zippy and friends, but you pay this with bloating memory. If you can afford it, then go ahead. I believe, at least from what I've seen from my tests, that zippy is quite fast and you gain very little, if at all (speedwise) by replacing it. You can gain some less memory fragmentation by using something else, but this is not a thing that would make me say: Wow! Exception to that is really Mac OSX. The native Mac OSX malloc sucks tremendously. The speed increase by zippy and nedmalloc are so high that you can really see (without any fancy measurements), how your application flies! The nedmalloc also bloats less than zippy (normally, as it clears per-thread cache on thread exit). So for the Mac (at least for us) I will stick to nedmalloc. It is lightingly fast and reasonably conservative with memory fragmentation. Conclusion: Linux/solaris = use system malloc Mac OSX = use nedmalloc Ah, yes... windows... this I haven't tested but nedmalloc author shows some very interesting numbers on his site. I somehow tend to believe them as some I have seen by myself when experimenting on unix platforms. So, most probably the outcome will be: Windows = use nedmalloc What this means to all of us:? I would say: very little. We know that zippy is bloating and now we know that is reasonably fast and on-pair with most of the other solutions out there. For people concerned with speed, I believe this is the right solution. For people concerned with speed AND memory fragmentation (in that order) the best is to use some alternative malloc routines. For people concerned with fragmentation the best is to stay with system malloc; exception: Mac OSX. There you just need to use something else and nedmalloc is the only thing that compiles (and works) there, to my knowledge. I hope I could help somebody with this report. Cheers Zoran
Re: [naviserver-devel] Quest for malloc
I tried to run this program, it crahses with all allocators on free when
it was allocated in other thread. zippy does it as well, i amnot sure
how Naviserver works then.
#include tcl.h
#define MemAlloc ckalloc
#define MemFree ckfree
int nbuffer = 16384;
int nloops = 5;
int nthreads = 4;
int gAllocs = 0;
void *gPtr = NULL;
Tcl_Mutex gLock;
void MemThread(void *arg)
{
int i,n;
void *ptr = NULL;
for (i = 0; i nloops; ++i) {
n = 1 + (int) (nbuffer * (rand() / (RAND_MAX + 1.0)));
if (ptr != NULL) {
MemFree(ptr);
}
ptr = MemAlloc(n);
// Testing inter-thread alloc/free
if (n % 5 == 0) {
Tcl_MutexLock(gLock);
if (gPtr != NULL) {
MemFree(gPtr);
}
gPtr = MemAlloc(n);
gAllocs++;
Tcl_MutexUnlock(gLock);
}
}
if (ptr != NULL) {
MemFree(ptr);
}
if (gPtr != NULL) {
MemFree(gPtr);
}
}
void MemTime()
{
int i;
Tcl_ThreadId *tids;
tids = (Tcl_ThreadId *)malloc(sizeof(Tcl_ThreadId) * nthreads);
for (i = 0; i nthreads; ++i) {
Tcl_CreateThread( tids[i], MemThread, NULL,
TCL_THREAD_STACK_DEFAULT, TCL_THREAD_JOINABLE);
}
for (i = 0; i nthreads; ++i) {
Tcl_JoinThread(tids[i], NULL);
}
}
int main (int argc, char **argv)
{
MemTime();
}
Doesn't zippy also clear it's per-thread cache on exit?
It puts blocks into shared queue which other threads can re-use.
But shared cache never gets returned so conn threads exit will not help
with memory bloat.
--
Vlad Seryakov
571 262-8608 office
[EMAIL PROTECTED]
http://www.crystalballinc.com/vlad/
Re: [naviserver-devel] Quest for malloc
Still, even without the last free and with mutex around it, it core
dumps in free(gPtr) during the loop.
Stephen Deasey wrote:
On 12/18/06, Vlad Seryakov [EMAIL PROTECTED] wrote:
I tried to run this program, it crahses with all allocators on free when
it was allocated in other thread. zippy does it as well, i amnot sure
how Naviserver works then.
I don't think allocate in one thread, free in another is an unusual
strategy. Googling around I see a lot of people doing it. There must
be some bugs in your program. Here's one:
At the end of MemThread() gPtr is checked and freed, but the gMutex is
not held. This thread may have finished it's tight loop, but the other
3 threads could still be running. Also, the gPtr is not set to NULL
after the free(), leading to a double free when the next thread checks
it.
#include tcl.h
#define MemAlloc ckalloc
#define MemFree ckfree
int nbuffer = 16384;
int nloops = 5;
int nthreads = 4;
int gAllocs = 0;
void *gPtr = NULL;
Tcl_Mutex gLock;
void MemThread(void *arg)
{
int i,n;
void *ptr = NULL;
for (i = 0; i nloops; ++i) {
n = 1 + (int) (nbuffer * (rand() / (RAND_MAX + 1.0)));
if (ptr != NULL) {
MemFree(ptr);
}
ptr = MemAlloc(n);
// Testing inter-thread alloc/free
if (n % 5 == 0) {
Tcl_MutexLock(gLock);
if (gPtr != NULL) {
MemFree(gPtr);
}
gPtr = MemAlloc(n);
gAllocs++;
Tcl_MutexUnlock(gLock);
}
}
if (ptr != NULL) {
MemFree(ptr);
}
if (gPtr != NULL) {
MemFree(gPtr);
}
}
void MemTime()
{
int i;
Tcl_ThreadId *tids;
tids = (Tcl_ThreadId *)malloc(sizeof(Tcl_ThreadId) * nthreads);
for (i = 0; i nthreads; ++i) {
Tcl_CreateThread( tids[i], MemThread, NULL,
TCL_THREAD_STACK_DEFAULT, TCL_THREAD_JOINABLE);
}
for (i = 0; i nthreads; ++i) {
Tcl_JoinThread(tids[i], NULL);
}
}
int main (int argc, char **argv)
{
MemTime();
}
-
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.phpp=sourceforgeCID=DEVDEV
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel
--
Vlad Seryakov
571 262-8608 office
[EMAIL PROTECTED]
http://www.crystalballinc.com/vlad/
Re: [naviserver-devel] Quest for malloc
On 12/18/06, Vlad Seryakov [EMAIL PROTECTED] wrote:
Still, even without the last free and with mutex around it, it core
dumps in free(gPtr) during the loop.
OK. Still doesn't mean your program is bug free :-)
There's a lot of extra stuff going on in your example program that
makes it hard to see what's going on. I simplified it to this:
#include tcl.h
#include stdlib.h
#include assert.h
#define MemAlloc ckalloc
#define MemFree ckfree
void *gPtr = NULL; /* Global pointer to memory. */
void
Thread(void *arg)
{
assert(gPtr != NULL);
MemFree(gPtr);
gPtr = NULL;
}
int
main (int argc, char **argv)
{
Tcl_ThreadId tid;
int i;
for (i = 0; i 10; ++i) {
gPtr = MemAlloc(1024);
assert(gPtr != NULL);
Tcl_CreateThread(tid, Thread, NULL,
TCL_THREAD_STACK_DEFAULT, TCL_THREAD_JOINABLE);
Tcl_JoinThread(tid, NULL);
assert(gPtr == NULL);
}
}
Works for me.
I say you can allocate memory in one thread and free it in another.
Let me know what the bug turns out to be..!
Stephen Deasey wrote:
On 12/18/06, Vlad Seryakov [EMAIL PROTECTED] wrote:
I tried to run this program, it crahses with all allocators on free when
it was allocated in other thread. zippy does it as well, i amnot sure
how Naviserver works then.
I don't think allocate in one thread, free in another is an unusual
strategy. Googling around I see a lot of people doing it. There must
be some bugs in your program. Here's one:
At the end of MemThread() gPtr is checked and freed, but the gMutex is
not held. This thread may have finished it's tight loop, but the other
3 threads could still be running. Also, the gPtr is not set to NULL
after the free(), leading to a double free when the next thread checks
it.
#include tcl.h
#define MemAlloc ckalloc
#define MemFree ckfree
int nbuffer = 16384;
int nloops = 5;
int nthreads = 4;
int gAllocs = 0;
void *gPtr = NULL;
Tcl_Mutex gLock;
void MemThread(void *arg)
{
int i,n;
void *ptr = NULL;
for (i = 0; i nloops; ++i) {
n = 1 + (int) (nbuffer * (rand() / (RAND_MAX + 1.0)));
if (ptr != NULL) {
MemFree(ptr);
}
ptr = MemAlloc(n);
// Testing inter-thread alloc/free
if (n % 5 == 0) {
Tcl_MutexLock(gLock);
if (gPtr != NULL) {
MemFree(gPtr);
}
gPtr = MemAlloc(n);
gAllocs++;
Tcl_MutexUnlock(gLock);
}
}
if (ptr != NULL) {
MemFree(ptr);
}
if (gPtr != NULL) {
MemFree(gPtr);
}
}
void MemTime()
{
int i;
Tcl_ThreadId *tids;
tids = (Tcl_ThreadId *)malloc(sizeof(Tcl_ThreadId) * nthreads);
for (i = 0; i nthreads; ++i) {
Tcl_CreateThread( tids[i], MemThread, NULL,
TCL_THREAD_STACK_DEFAULT, TCL_THREAD_JOINABLE);
}
for (i = 0; i nthreads; ++i) {
Tcl_JoinThread(tids[i], NULL);
}
}
int main (int argc, char **argv)
{
MemTime();
}
-
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.phpp=sourceforgeCID=DEVDEV
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel
--
Vlad Seryakov
571 262-8608 office
[EMAIL PROTECTED]
http://www.crystalballinc.com/vlad/
-
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.phpp=sourceforgeCID=DEVDEV
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] Quest for malloc
On 18.12.2006, at 22:08, Stephen Deasey wrote: Works for me. I say you can allocate memory in one thread and free it in another. Nice. Well I can say that nedmalloc works, that is, that small program runs to end w/o coring when compiled with nedmalloc. Does this prove anything?
Re: [naviserver-devel] Quest for malloc
On 18.12.2006, at 19:57, Stephen Deasey wrote: Are you saying you tested your app on Linux with native malloc and experienced no fragmentation/bloating? No. I have seen bloating but less then on zippy. I saw some bloating and fragmentation on all optimizing allocators I have tested. I think some people are experiencing fragmentation problems with ptmalloc -- the Squid and OpenLDAP guys, for example. There's also the malloc-in-one-thread, free-in-another problem, which if your threads don't exit is basically a leak. Really a leak? Why? Wouln't that depend on the implementation? Doesn't zippy also clear it's per-thread cache on exit? No. It showels all the rest to shared pool. The shared pool is never freed. Hence lots of bloating. Actually, did you experiment with exiting the conn threads after X requests? Seems to be one of the things AOL is recommending. Most of our threads are Tcl threads, not conn threads. We create them to do lots of different tasks. They are all rather short-lived. Still, the mem footprint grows and grows... One thing I wonder about this is, how do requests average out across all threads? If you set the conn threads to exit after 10,000 requests, will they all quit at roughly the same time causing an extreme load on the server? Also, this is only an option for conn threads. With scheduled proc threads, job threads etc. you get nothing. Well, if they all start to exit at the same time, they will serialize at the point where per-thread cache is pushed to the shared pool. -- --- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT business topics through brief surveys - and earn cash http://www.techsay.com/default.php? page=join.phpp=sourceforgeCID=DEVDEV ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] Quest for malloc
I suspect something i am doing wrong, but still it crashes and i do not
see it why
#include tcl.h
#include stdlib.h
#include memory.h
#include unistd.h
#include signal.h
#include pthread.h
#define MemAlloc malloc
#define MemFree free
static int nbuffer = 16384;
static int nloops = 5;
static int nthreads = 4;
static void *gPtr = NULL;
static Tcl_Mutex gLock;
void MemThread(void *arg)
{
int i,n;
void *ptr = NULL;
for (i = 0; i nloops; ++i) {
n = 1 + (int) (nbuffer * (rand() / (RAND_MAX + 1.0)));
if (ptr != NULL) {
MemFree(ptr);
}
ptr = MemAlloc(n);
if (n % 50 == 0) {
Tcl_MutexLock(gLock);
if (gPtr != NULL) {
MemFree(gPtr);
gPtr = NULL;
} else {
gPtr = MemAlloc(n);
}
Tcl_MutexUnlock(gLock);
}
}
}
int main (int argc, char **argv)
{
int i;
Tcl_ThreadId *tids;
tids = (Tcl_ThreadId *)malloc(sizeof(Tcl_ThreadId) * nthreads);
for (i = 0; i nthreads; ++i) {
Tcl_CreateThread( tids[i], MemThread, NULL,
TCL_THREAD_STACK_DEFAULT, TCL_THREAD_JOINABLE);
}
for (i = 0; i nthreads; ++i) {
Tcl_JoinThread(tids[i], NULL);
}
}
Stephen Deasey wrote:
On 12/18/06, Vlad Seryakov [EMAIL PROTECTED] wrote:
Still, even without the last free and with mutex around it, it core
dumps in free(gPtr) during the loop.
OK. Still doesn't mean your program is bug free :-)
There's a lot of extra stuff going on in your example program that
makes it hard to see what's going on. I simplified it to this:
#include tcl.h
#include stdlib.h
#include assert.h
#define MemAlloc ckalloc
#define MemFree ckfree
void *gPtr = NULL; /* Global pointer to memory. */
void
Thread(void *arg)
{
assert(gPtr != NULL);
MemFree(gPtr);
gPtr = NULL;
}
int
main (int argc, char **argv)
{
Tcl_ThreadId tid;
int i;
for (i = 0; i 10; ++i) {
gPtr = MemAlloc(1024);
assert(gPtr != NULL);
Tcl_CreateThread(tid, Thread, NULL,
TCL_THREAD_STACK_DEFAULT, TCL_THREAD_JOINABLE);
Tcl_JoinThread(tid, NULL);
assert(gPtr == NULL);
}
}
Works for me.
I say you can allocate memory in one thread and free it in another.
Let me know what the bug turns out to be..!
Stephen Deasey wrote:
On 12/18/06, Vlad Seryakov [EMAIL PROTECTED] wrote:
I tried to run this program, it crahses with all allocators on free when
it was allocated in other thread. zippy does it as well, i amnot sure
how Naviserver works then.
I don't think allocate in one thread, free in another is an unusual
strategy. Googling around I see a lot of people doing it. There must
be some bugs in your program. Here's one:
At the end of MemThread() gPtr is checked and freed, but the gMutex is
not held. This thread may have finished it's tight loop, but the other
3 threads could still be running. Also, the gPtr is not set to NULL
after the free(), leading to a double free when the next thread checks
it.
#include tcl.h
#define MemAlloc ckalloc
#define MemFree ckfree
int nbuffer = 16384;
int nloops = 5;
int nthreads = 4;
int gAllocs = 0;
void *gPtr = NULL;
Tcl_Mutex gLock;
void MemThread(void *arg)
{
int i,n;
void *ptr = NULL;
for (i = 0; i nloops; ++i) {
n = 1 + (int) (nbuffer * (rand() / (RAND_MAX + 1.0)));
if (ptr != NULL) {
MemFree(ptr);
}
ptr = MemAlloc(n);
// Testing inter-thread alloc/free
if (n % 5 == 0) {
Tcl_MutexLock(gLock);
if (gPtr != NULL) {
MemFree(gPtr);
}
gPtr = MemAlloc(n);
gAllocs++;
Tcl_MutexUnlock(gLock);
}
}
if (ptr != NULL) {
MemFree(ptr);
}
if (gPtr != NULL) {
MemFree(gPtr);
}
}
void MemTime()
{
int i;
Tcl_ThreadId *tids;
tids = (Tcl_ThreadId *)malloc(sizeof(Tcl_ThreadId) * nthreads);
for (i = 0; i nthreads; ++i) {
Tcl_CreateThread( tids[i], MemThread, NULL,
TCL_THREAD_STACK_DEFAULT, TCL_THREAD_JOINABLE);
}
for (i = 0; i nthreads; ++i) {
Tcl_JoinThread(tids[i], NULL);
}
}
int main (int argc, char **argv)
{
MemTime();
}
-
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.phpp=sourceforgeCID=DEVDEV
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel
--
Vlad Seryakov
571 262-8608 office
[EMAIL PROTECTED]
http://www.crystalballinc.com/vlad/
Re: [naviserver-devel] Quest for malloc
On 12/18/06, Zoran Vasiljevic [EMAIL PROTECTED] wrote:
On 18.12.2006, at 19:57, Stephen Deasey wrote:
One thing I wonder about this is, how do requests average out across
all threads? If you set the conn threads to exit after 10,000
requests, will they all quit at roughly the same time causing an
extreme load on the server? Also, this is only an option for conn
threads. With scheduled proc threads, job threads etc. you get
nothing.
Well, if they all start to exit at the same time, they will
serialize at the point where per-thread cache is pushed to
the shared pool.
I was worried more about things like all the Tcl procs needing to be
recompiled in the new interp for the thread, and all the other stuff
which is cached. If threads exit regularly, say after 10,000
requests, and the requests average out over all threads, then your
site will regularly go down, effectively. It would be nice if we could
make sure the thread exits were spread out.
Anyway...
I think some people are experiencing fragmentation problems with
ptmalloc -- the Squid and OpenLDAP guys, for example. There's also
the malloc-in-one-thread, free-in-another problem, which if your
threads don't exit is basically a leak.
Really a leak? Why? Wouln't that depend on the implementation?
Yes, and I thought that was the case with Linux ptmalloc, but maybe I
got it wrong or this is old news...
This program allocates memory in a worker thread and frees it in the
main thread. If all free()'s put memory into a thread-local cache then
you would expect this program to bloat, but it doesn't, so I guess
it's not a problem (at least not on Fedora Core 5).
#include tcl.h
#include stdlib.h
#include stdio.h
#include assert.h
#define MemAlloc malloc
#define MemFree free
void *gPtr = NULL;
static void Thread(void *arg);
static void PrintMemUsage(const char *msg);
int
main (int argc, char **argv)
{
Tcl_ThreadId tid;
int i;
PrintMemUsage(start);
for (i = 0; i 10; ++i) {
Tcl_CreateThread(tid, Thread, NULL,
TCL_THREAD_STACK_DEFAULT, TCL_THREAD_JOINABLE);
Tcl_JoinThread(tid, NULL);
MemFree(gPtr);
gPtr = NULL;
}
PrintMemUsage(stop);
}
static void
Thread(void *arg)
{
assert(gPtr == NULL);
gPtr = MemAlloc(1024);
assert(gPtr != NULL);
}
static void
PrintMemUsage(const char *msg)
{
FILE *f;
int m;
f = fopen(/proc/self/statm, r);
if (f == NULL) {
perror(fopen failed: );
exit(-1);
}
if (fscanf(f, %d, m) != 1) {
perror(fscanf failed: );
exit(-1);
}
fclose(f);
printf(%s: %d\n, msg, m);
}
